David Wentworth is an experienced research analyst with Brandon Hall Group in Florida, and a guest speaker at this year’s Halogen Software customer conference, TalentSpace Live 2017. Get a preview of what he’ll be talking about at the conference, and register for the conference here.
As we talk more and more about new learning experiences and incorporating more informal and experiential learning into our efforts, we inevitably end up on the subject of measurement. Everyone gets excited talking about collaboration, games, simulations, virtual reality, etc.; but the excitement comes to a screeching halt once someone asks, “How are we going to measure that?”
It’s not surprising we feel like that. Most organizations really aren’t that good at measuring the traditional learning they’ve been delivering for years. In Brandon Hall Group’s 2016 Learning Measurement Study, just 54 percent of companies said they were effective or very effective at measuring formal learning. It’s far, far worse when it comes to informal (13 percent) and experiential (22 percent) learning.
Wasted learning measurement
When we look at what is being measured, it’s easy to see why we aren't effective. The most commonly used metric for learning is completion of courses. I have absolutely zero idea how the fact that a certain percentage of employees completed a course translates into any sort of result. It is simply a measure of the efficiency of learning, not the efficacy. Numbers two and three aren’t much better: post-course questionnaires and smile sheets, respectively. Learner engagement is important, but the fact that someone liked something doesn’t tell you much about its effectiveness, either. Number four on the list is learner assessments. Now we’re into something with a little bit of teeth, but it still is not a reflection of whether or not the learning is having an impact.
It isn’t until number five on the list that companies start getting concerned with actual results – meeting corporate objectives, to be precise. What this tells me is that, in general, the learning function is far more concerned with how well it is delivering learning than it is with what the learning means to the business. The industry of corporate learning is engaged in some serious navel-gazing, where the outcome learning is learning when it should be behavior and performance.
This is where we see the transition from learning consultants to performance consultants. We need learning professionals that are focused on improving performance metrics, not learning metrics. Instead of fixating on completion rates, we need to be focused on outcomes that have an actual impact on the business.
Getting learning right
Don’t get me wrong. There are many companies who actually do get this and are doing it right. In fact, when we look at our data and separate out companies that we consider to be high performers, something interesting happens. High performers are those companies whose KPIs (revenue, market share, customer satisfaction, etc.) improve significantly year over year. These companies do a far better job of measuring learning against actual outcomes than their lower-performing counterparts. Let’s take a look at a list of outcomes, and see the difference in how many high performers use them as measurements regularly or consistently versus everyone else:
Source: Brandon Hall Group, 2016 Learning Measurement Survey
In every single category, high performers are far more likely to use these outcomes, sometimes by factors of two, three, and even five! Moral of the story: High organizational performance is correlated with solid learning measurement.
I plan to dig into this and more during session at Halogen’s TalentSpace Live 2017, I hope to see you there!