By Patti P. Phillips, Ph.D., Jack J. Phillips, Ph.D., and Klaas Toes, MSc
This article was originally published in Strategic HR Review on January 29, 2021.
Executives want to know if major learning programs are working. The executive’s perspective focuses on whether these programs impact the organization and do they deliver a positive return on investment (ROI). An executive’s request for ROI will lead human resources professionals to plan an evaluation, collect data, analyze data, and present the results to the program sponsors and funders. The success of this presentation will influence program support and future allocation of funding to HR programs. To be credible, the presentation must include five moments of truth. These five truths require credible data, collected from credible sources, presented in five categories, and a conservative analysis that executives can believe, including proof the program has made a difference. . The key challenge for the HR team is to involve executives in each of the five truths.
Five Moments of Truth:
Presenting Business Results to Senior Executives
During several recent webinars, we polled the audience asking how many participants had received an actual request to show the ROI of a program. Almost 40 percent answered yes, an indication of the current economic environment. With major programs in place and other programs planned, executives want to know if they are working. From their perspective, whether a program is working, or not, focuses on this issue does it have an impact and deliver a positive ROI? These ROI requests lead to evaluation planning, data collection, data analysis, and results presentations to senior executives. A key issue in the presentation is the credibility of the results. Do the executives believe the results?
We have been involved in this process hundreds of times and have taught thousands of others how to make credible presentations. It is condensed to five moments of truth. We use the word truth because the presentations require credible data from credible sources and conservative analysis that executives can believe, including proof the program has made a difference. Executives must believe the results are true or the presentation becomes a waste of time. In our experience, the most successful programs had high executive involvement, starting with investment alignment. There are five categories of data and each must have a moment of truth.
The Five Moments of Truth
Truth #1: Investment Alignment
A substantial investment in a program has been made, sometimes approaching millions of dollars. Executives would like to see the value of the investment, including the ROI. The moment of truth for investment alignment involves a critical executive question: When the program benefits are converted to monetary values, is the value greater than the cost of the program? If so, the ROI is positive.
Investment alignment ensures that the program is treated as a normal investment and aligned to the executives’ investment expectations. Usually, this is defined as an objective set at the beginning of the program, the minimal acceptable ROI. This may be 10, 20, or 30 percent? A 20 percent ROI objective means, for every dollar invested, a dollar is returned plus another 20 cents. For most executives, this is acceptable. In our benchmarking database, we find that well-designed leadership programs deliver an ROI in the range of 100 -300 percent. This exceeds expectations and changes the paradigm on the value of programs.
Truth #2: Business Alignment
The monetary benefits described above come from impact data already in the system. It includes business measures such as output (sales, production), quality (errors, waste), time (downtime, service time), and costs (operating costs, supply costs). The moment of truth for business alignment involves an important question: Did the program drive the expected improvement in business measures? Normally, this expected improvement comes from impact objectives set either by the individuals involved in the program or by the owners of the program, or both. In the last few years, we have involved management in the business alignment and program design sessions, which has brought about mindset change to impact of the programs. The stakeholders now see our programs in a different way and are highly motivated to give input.
Truth #3: Solution Implementation
The impact has occurred because the proper solution was implemented. It could be a learning solution or a non-learning solution. It could involve reward systems, new technology, job design, new procedures, or new policies. In the learning and development field, these are often learning solutions. The moment of truth for solution implementation is simple: Was the solution implemented successfully? Successful implementation has three indicators: Are participants able to (do they have the knowledge and skills), are the willing to (do they have the motivation) and are they allowed(by managers, colleagues, and organizational culture) to show the desired performance. Did it meet the expectations for use in the workplace or with the individuals involved in the program? Again, this comes from objectives set at the application level.
These objectives involve the extent, frequency, and success of use. This level provides an opportunity to identify the barriers and enablers of the program’s success. If the desired success wasn’t achieved, the barriers will explain why, and can help identify improvements. If success was achieved, the enablers will explain how the success occurred, and identify the processes in place to ensure the solution is used and properly implemented.
Truth #4: Capability Development
A solution is implemented properly when participants involved know how to use it properly. This involves learning knowledge and skills. The moment of truth is basic: Did participants learn what they need to know and know what to do to make the solution successful? Executives need to know that success was achieved because the participants learned something, and they applied it later. Capability development is driven by learning objectives, and meeting these objectives confirms that participants actually learned what was expected.
Truth #5: Motivation Attainment
Participants learn and apply the solution because it is something they have decided they need or want. The moment of truth involves several issues: Is the solution important to them? Is the solution relevant to their work? Is this something they will implement? Is this something they would recommend to others? These perceptions describe the motivation of participants to do what you want them to do.
Motivation is captured as reaction, and reaction expectations are set with the reaction objectives, which define how participants will perceive this particular solution. If they are not motivated to use what they learn, the program fails. The two big reasons why programs fail is rejection by the users or the users not knowing how to use the solution properly. Reaction and learning go together, and the amount of learning affects participant reaction. This moment of truth ensures that participants are motivated to take action. During the program we measure reaction. These indicators are early predictors of success. By identifying these predictive performance analytics, improvements can be made during the program.
This completes the story of how alignment was achieved. The outcomes are delivered and measured in the reverse order, starting with reaction and learning and moving to application, impact, and ROI.
The Actual Presentation
The five moments of truth are presented in the order that the program was conceived, designed, and implemented. The executives want a positive ROI (payoff need) delivered by specific business measures (business need), driven by a solution implemented to deliver the desired business impact (performance need). To have full executive commitment, the HR team must present data at all five moments of truths, using the technique of storytelling with data. To make programs successful, participants must learn what to do to implement the solution (learning need), and they must perceive the solution to be important to them (preference needs).
This is the sequence of program development and it presents the most important data sets for executives first with the least important data presented last. Some presentations flip this around and show executives how the participants reacted (motivation), what they’ve learned (capability development), what they actually did (application), the business impact from implementing the solution (business alignment), and finally, the ROI calculation (investment alignment).
What Happens if this is Negative?
It is not unusual for a program that is already implemented to fail to meet the expectations from each moment of truth. These are process improvement opportunities that can improve the program the next time it is offered or improve the next program. It is important to approach an evaluation from an improvement perspective. In the presentation, you will reveal that the program fell short but explain what should be adjusted in the future to make it more successful. The data will show where the program failed, which is usually something the participants did not do, a lack of support, or something in the system inhibiting the solution.
We have had many opportunities with these kinds of presentations, and we would be happy to share some tips if you would like to know more. The success of this presentation makes a difference in terms of the buy-in for the program, the support that it may need, and the funding that might be allocated to talent development in the future.