How Do You Measure Up?

There are two possible outcomes: if the result confirms the hypothesis, then you've made a measurement. If the result is contrary to the hypothesis, then you've made a discovery.
  -    Enrico Fermi

I'm big on measurement. I don't consider a learning development project complete unless I have a metric for success defined. Two years ago I was challenged to create a program that would shorten the time our engineers would get to a productive level. Next week we will debut a 5 week bootcamp program I designed to meet that challenge. It was a long and sometimes difficult road, but in the end its something we're all proud of.  But there was still something missing.

One of the things I pushed back to management was the ability to measure success. Survey sheets are good for gauging the mood of the learner at the end of the class (and not a lot more), and I could give a test (which I do) but that just confirms a short term memory.  I forced management to dig into their data and determine exactly how long it was that new engineers received assignments they could complete on their own, and we would compare that to measurements in the out years. That was difficult because it isn't just a field in a database, but a determination that has to be gleaned from many sources.

I feel that most of the time we don't measure success because, well, it's hard!  We like to think that if we identify a need and carefully design a learning program to fill that need, then we must have had some success.  Right?

But the bottom line is, you simply don't know.

So my question is, how do you measure success of your programs, and do you find it difficult to get management buy in?