In 1959, learning expert Donald Kirkpatrick designed a Four-Level Training Evaluation Model to evaluate the impact of training on performance. The Kirkpatrick Ladder has since become one of the most influential training evaluation frameworks in the world.
According to Kirkpatrick’s model, organizations should measure:
- How did participants feel about the training?
- What were their expectations of the course, and were they adequately met?
- Did they think the course material was valuable?
- Did the instructor deliver it effectively?
- What did the trainees learn from the course?
- How well have they retained their new knowledge?
- Are learners proficient enough to apply their new skills on the job?
- How often do they do so?
- What is the effect on their overall performance?
- How has the training benefited the organization as a whole?
- What critical business metrics have been measurably improved?
- Has the training raised productivity or customer satisfaction?
- Has the training increased sales or reduced costs?
Kirkpatrick noted that while many organizations were able to track levels 1 and 2 reasonably well, most struggle to reach the higher levels. Almost six decades later, this observation still rings true. A recent poll conducted by Lambda Solutions found that only 11% of organizations have successfully implemented a measurement program for levels 3 and 4.
This trend is troubling, because the results of an evaluation program tend to follow an S-Curve, where the greatest benefits are realized at the upper rungs of the Ladder. Being stuck on the lower levels means organizations are not even close to getting full value from their learning data.
The Missing Piece: Learning Analytics
In theory, training evaluation should not be such a big challenge for the modern organization. Technology has progressed greatly since the 1950s, and we now have advanced LMS software that records detailed data on every course, trainee and instructor.
Shouldn’t this be making things easier?
Unfortunately, it turns out that the abundance of data is part of the problem. Without an effective way to make sense of the raw data, and turn it into meaningful information, learning & development (L&D) professionals are no better off than they were in Kirkpatrick’s day.
In our experience, there are three common barriers to climbing the Kirkpatrick Ladder, and an effective LMS reporting tool should be able to address all of them.
Barrier 1: Limited Access to Data
Accessing the wealth of data available in your LMS should be easy, but unfortunately, this is not always the case. LMS platforms can be very limited in their options for data access, making it difficult to pull tables with the exact data you need.
The problem is serious enough that many organizations have built their own in-house software for accessing LMS data more effectively. However, such software is often not properly integrated with the LMS, and we have heard several instances where sites have crashed while attempting to access data.
An effective LMS reporting solution should allow you to easily generate custom data tables based on a wide range of chosen criteria. It should also integrate seamlessly with all the major LMS platforms, such as Moodle, Totara, Canvas and Blackboard.
Barrier 2: Limited Capabilities to Analyze Data
Once organizations have access to their data, they need an effective way to analyze it.
Most LMS platforms have very basic data analysis capabilities. As a result, many organizations resort to pulling the raw data out of the LMS, then manually compiling it in Excel for analysis and visualization. This process is not only time-consuming, but also vulnerable to human error.
An LMS reporting tool should enable you to visualize your data through interactive lists, graphical charts and executive dashboards. It should also include a variety of out-of-the box reports, as well as the ability for advanced users to assemble their own custom metrics.
Barrier 3: Limited Options to Present and Share Data
For data to have actionable value, the findings must be shared with the appropriate decision-makers.
Too often, arranging data insights for presentation becomes a huge time sink. The information must be manually compiled from various sources, arranged into a visually appealing report, and be sent out at the right time to the right people.
Ideally, your LMS reporting software will allow you to automate this process. It should enable you to style your reports with branded templates that reflect your organization’s look and feel, while minimizing repeat design work. It should also have the ability to export your reports in a variety of formats, and send them out to selected recipients at a scheduled date and time.
Scaling the Ladder
With an LMS reporting solution that solves these three challenges, your organization will be well-equipped to reach the top of the Kirkpatrick Ladder. Knowing which courses really deliver, and why, you will be able to improve your training practices and maximize the ROI of your learning budget.
Zoola Analytics is a learning analytics and reporting solution that includes all the above features, and more. We have designed Zoola to make it as easy as possible for organizations to access, analyze and present their LMS data. Contact us and let us show you how Zoola can help you improve your training evaluation process.