Why completion rates are no longer the metric of success
Shaping the future of learning
When you’re looking to develop a holy grail of just in time information and resources for your learners, don’t be surprised when they dip in and out, abandon mid-module, or skip to page three of a PDF. Unless we’re talking compliance or a course awarding credentials, a drop in course completions might actually be a good thing. The whole point of pulling your learners in and delivering at the point of need is that you’re providing what your people need to do their jobs. What’s more important to you: a completed course or a job well done?
A mental shift in how L&D evaluates success
There’s this inherent value we in L&D put on course completion rates. It’s engrained from the days when all learning was formal education - when you’d complete a course and get a grade to signify success and that you’re done. But in corporate training the biggest signal of success is really about getting the task at hand done.
How many times have you turned to YouTube to figure out how to reset your phone or a hack for tying a pretty bow for a present you’re wrapping and stopped watching halfway because you had what you needed? People turn to Google and YouTube to get jobs done – why should it be any different with their company’s learning solution? Shouldn’t a corporate learning platform act as a space in which learners can cycle through courses and resources – coming back again and again, if and when they need it? If that’s how you want your employees to view and use your corporate learning, then we’ve got to accept that they’re never done and that there isn’t really such a thing as a completed course.
So how do you measure the success of your learning initiatives?
The straight answer? That’s mostly one for you to figure out. It’s based on what you want the purpose of your learning and development to be. You’ve got to think about what the behaviour or action you want your learning initiatives to accomplish. Use the New World Kirkpatrick Model and dig even deeper using technology that’s only a click away. There are so many tools you can use to start forming what you consider factors of success. Here’s six to consider:
1. Feedback surveys
Ever have those pop-ups appear when you’re about to close a browser begging you to sign up to their newsletter or stay on the website? While annoying when poorly executed, you could steal this tool from marketing’s playbook to understand why your learners chose a course or resource and find out how well it helped them accomplish that task. Just remember to keep it short and sweet.
2. Peer review systems
Enable your learners to leave a review of your courses, modules, and even resources. This system is a double win because you can use the feedback from these responses to understand how well the content went down, and possibly of even greater benefit, your learners will have a signal of what learning is most beneficial for them to do their job. Check out this blog post for some more ideas on the power of peer reviews.
3. Search queries tracking
You’ll know when your learning solution is successful when it’s turning up answers to your people’s problems. A few years ago there was a complex set of symbols needed to refine what search results Google gave you. But with Siri and Google getting ever more sophisticated, people are asking questions. These questions can give you insights into exactly what your learners are trying to accomplish. By tracking what people are searching in your learning platform, you can better tag solutions to their problems and fill the gaps.
Kineo is a Learning Locker partner with xAPI innovators, HT2 Labs, and it’s amazing the whole wealth of information Experience API opens up. xAPI enables you to track just about anything in a learning experience, and heck, can even help you deliver learning and resources at the point of need. Just make sure you don’t go overboard with data overload.
5. Google Analytics
Tracking visits, time spent and interactions and video plays can give you some interesting insights. But tracking this information can often raise as many questions as it does answers because these insights could instead flag that a course or resource isn’t meeting needs. For example, why are people going back to a PDF? Is it because they can’t find what they need? Are people quitting a video early not because they got what they wanted but because it was boring? If no one is watching past the three-minute mark, should the video be cut or maybe re-ordered so they don’t miss that important bit at the end? In short, make sure if you use these analytics as a measure of success, you investigate alternative explanations. Check out this blog post on using Google Analytics in your learning solutions.
6. Heatmapping and session recordings
Going a bit beyond Google Analytics, you can easily find out where people are spending the most time within a resource or course with heatmapping or session recordings. A heatmap can display the most viewed sections of a page, where the most clicks are or to track movement in a colourful visual. It will show what content or section is the most interesting to viewers (and thus most useful) and even reveal what never sees the light of day. To really get inside the head of your learners, you can also do session recordings, which do just as the name implies by recording everything your viewer does. Find out more and get a list of suggested heatmapping and recording tools with this blog post.
With the rise of just in time learning, the days of declaring a course a failure and binning it completely are pretty much gone. Optimisation now rules the day. You have the resources at your fingertips to make even the smallest of improvements. Thankfully running in parallel to the rise of microlearning and resources at the point of need, is this wealth of technology to continually tweak, test, and perfect. While it might be a bit challenging or overwhelming to overhaul an hour-long course, an element in a five-minute module or PDF will keep you at the cutting edge of meeting learner demands.
Perhaps the greatest measure of success is really about how well and how quickly your team can respond to learner demands. Get our guide, The micro manual: getting microlearning, resources and performance support right.