Patti P. Phillips, Ph.D., and Jack J. Phillips, Ph.D.

This article was originally published on December 21, 2020, on ChiefLearningOfficer.com.

In today’s uncertain economic climate, the need to ensure that virtual learning provides value and improves organizational outcomes is increasingly important. Because of the COVID-19 pandemic and the need to work-from-home, most learning programs have shifted to virtual formats. This environment has created the need for program owners to design and implement virtual learning with the desired outcomes in mind. Additionally, there is an increasing economic need to demonstrate that the virtual programs are working and delivering the desired results.

Studies show that many virtual learning programs break down when measured at the application level (participants not using what they learn) and impact level (the business impact connected to application). Fortunately, L&D professionals can avoid this breakdown. To secure the support and funding that virtual learning needs, the programs must deliver business results. This article will explore how designers, developers, providers, and program owners can ensure virtual learning will achieve the desired results using design-thinking principles that focus on delivering outcomes at the application and impact levels.

Serious Challenges for L&D

Many learning and development professionals have faced challenges in delivering the results that executives appreciate and understand. Answers to our recent polling questions of L&D teams show the following statements to be true from the perspective of the L&D professionals. However, to deliver the value that executives need from virtual learning, the answers should actually be false.

1. Most learning and development content is not used (wasted). Unfortunately, there continues to be a transfer of learning problem as individuals learn the content but subsequently do not actually use what they have learned. There are many barriers to this transfer, with the most significant barrier being a lack of management support. For in-person learning and development, this barrier to transfer is much less of a problem.
2. Organizations rarely measure the learning outcomes desired by executives. Studies confirm that executives want and need to see the business connections to learning, whether the learning is delivered in-person, virtually, or through a mobile device. If participants don’t use what they learned, the program delivers no business value to the organization. The key to successful results is to ensure that the learning is used and has an impact.
3. The learning providers do not have data that shows they make a difference in the organization. To make a difference, the individuals involved in programs must use what they have learned, and that use must have a corresponding impact. Both are necessary. Without use, there will be no impact. Unfortunately, many learning and development practitioners do not even measure at this level to ensure application on the job and the corresponding business impact.
4. Most executives see learning as a cost and not an investment. When executives see learning as a cost, bad things may happen in uncertain times. The learning budget may be cut, reduced, paused, minimized, or controlled. If executives see learning as an investment, they may protect it, support it, and even enhance it in downturns. The challenge is for L&D professionals to show the executives that learning delivers a positive ROI. The best way to do that is to actually measure the ROI on a major program using an ROI formula from the finance and accounting literature. Doing this will remove the doubt that learning is an investment.

The Chain of Value is Always There

It is important to understand how success is achieved with virtual learning. How do you deliver more productivity or improve the quality of services, or have more innovation, or be more responsive to customers? If a lack of knowledge or skills is a stumbling block to success, then putting those processes in place will enable success. Virtual learning can easily provide this if the learning is connected to the business measure in the beginning, the right solution is implemented, and it’s designed to deliver the impact results. More importantly, the amount of improvement in the impact measures caused by the virtual program can always be pinpointed. The success follows a chain of value, outlined in Figure 1.

Figure 1. The Value Chain for Virtual Programs

Why Virtual Learning Fails: A Quick Review

We have had the opportunity to evaluate virtual learning for many years, and have compared instructor-led learning with eLearning, sometimes in parallel, to see the differences. We have found that virtual learning typically breaks down at Level 3 (Application) and Level 4 (Impact). Why does this happen? There are four key reasons:

1. Multitasking inhibits learning. The research shows that multitasking reduces a participant’s ability to learn. There is a myth that a person can multitask and still absorb in-depth knowledge and information. That doesn’t happen.¹ With instructor-led learning, multitasking is better controlled in many ways. This is accomplished not only by good design, activities, and engagement but also by the facilitator managing the learning environment and removing the opportunity to multitask. As we all have witnessed multiple times with virtual learning in virtual conferences, meetings, and online sessions, multitasking runs rampant. The reality is if learning is diminished, then application will be diminished, and impact will be diminished. This will decrease the ROI significantly.
2. Manager’s support is usually missing. Participants typically leave their work areas to attend instructor-led learning programs. Their managers know they are involved, and the manager is likely involved in the decision for them to participate in the session. The manager creates expectations for the program. The participant will be away from the job, and the organization will miss that person’s work. With this scenario, it is in the managers’ best interest for the learning to translate into something useful and worthwhile for the department. Because of this, the managers are often involved before the program with setting goals and then after the program with following-up. For virtual learning, the manager is usually not involved. For the most part, the manager may not even know that participants are involved. This is because most virtual learning usually occurs during regular work hours in the participant’s everyday environment, and the manager may often not even be aware of it. Without the manager’s presence, the most significant influencer for transferring learning to the job is removed.
3. Success is based on learning, not impact. Instructional systems designers design virtual programs to deliver learning. Designers think their work is complete when the participant has learned knowledge or skills. However, the executives who provide the program’s budget want to see the business connection from participants actually using the learning. It is rare to find a virtual program designed for impact. Very few virtual learning programs have impact objectives. Without impact objectives, some key stakeholders may not fully understand why the program is being implemented. When impact objectives are in place, the team can design the program for application and the desired business impact. Designing for impact is more likely to occur in an instructor-led program.
4. The role of the facilitator is usually missing. There may be a live or recorded facilitator, a program administrator, host, or coordinator, but what is truly missing in virtual learning programs is the power and influence of a great facilitator. Facilitators usually inspire the participants, encourage them, and support them to use what they have learned. Sometimes, a group of participants will bond together to deliver the required results. Virtual learning program designers must design for the delivery of learning and its application, and they must also create a replacement for the influence of the in-person facilitator.

How to Make Virtual Learning Deliver a Positive ROI

Building on the reasons for failure, here are seven steps to design virtual learning to deliver a positive ROI:

1. Start with why defined as a clear business need. Make sure that a business need is clearly identified for the program. Pinpoint the business need by asking questions, analyzing data, and exploring what will occur when this program is implemented.
2. Make it feasible by selecting the right solution. Ensure that learning is the right solution and virtual learning is the right learning solution to improve the business measure.
3. Expect success. Define success for the program at the impact level and set objectives for each lower level (reaction, learning, application, and impact). Provide those objectives to all stakeholders to make sure that they design for application and business impact.
4. Design for the desired outcomes. Make sure that all stakeholders develop tools, templates, and processes, and provide the support and encouragement to make the virtual learning deliver success at Levels 3 and 4. With this step, you are essentially designing for those outcomes.²
5. Measure application and impact. Sample a few virtual programs to ensure that application and impact results are delivered. Use simple, easy to provide, application and impact data to understand how the programs are working.
6. Analyze the data. When the impact data is collected, the first step is to sort out the effects of the virtual learning from other influences. This is critical to achieving credibility of the data. This shows the actual amount of improvement connected to the program. This would complete a Level 4 Impact study. If ROI is desired, the impact data would be converted to money. Those conversions are already accomplished in the organization, it is just a matter of finding the data or the right person who knows the values. Next, the fully loaded cost of the program is tabulated. This is the direct and indirect cost of the virtual learning program. The costs and the benefits are compared by a formula to calculate the ROI

For example, if the ROI is 48%, this means that for every dollar invested into the program, a dollar is returned plus an additional 48 cents.

7. Tell the story. The final step is to present the results to the appropriate audiences using numbers and narrative and think process improvement throughout the process.

There you have it, seven steps to make sure that virtual learning works and delivers the success that your organization needs. For a copy of a virtual learning ROI study, please contact ROI Institute.