E-Learning 1.0

Many Happy Returns: Calculating E-Learning ROI
By John Setaro

E-learning may be the wave that carries the future of corporate training. But if that wave is ever to crest, corporations must be able to determine a financial return on their investment. Only if early adopters can truly demonstrate an ROI in dollar figures will e-learning wash over corporate culture and change its landscape.

It's difficult to assign a dollar figure to e-learning's most important benefits. Sure, you can calculate how many airline tickets your company didn't have to buy because a whole division took Microsoft Office training online. But can you calculate how much better your managers communicate with their direct reports since they took that course on online communication skills? E-learning's less tangible improvements are no less valuable than the savings in travel expenses the method yields, but intangibles are more difficult to document and justify to senior-level decision makers.

Fortunately, many training evaluation experts have developed models for determining the value of training experiences. Companies can use those formulas to calculate the ROI of e-learning initiatives. Two models stand out. Developed by researcher Donald Kirkpatrick and enhanced by his colleague Jack Phillips, the two form a logical framework to examine ROI from both a human and business performance perspective.

The models evaluate training benefits on several levels. Critical to both is the concept of "chain of effect," which links each benefit level to others. Each level of measurement depends on the previous level, as well as the next. Without this link, it's difficult to conclude with any degree of confidence that training is responsible for improvements in performance.

Kirkpatrick's original model considers the value of training on four levels and was laid out in Evaluating Training Programs: The Four Levels (Berrett-Koehler, 1998 2nd edition). Philips expanded on Kirkpatrick's model in Accountability in Human Resource Management (Gulf Professional Publishing Company, 1996), suggesting that another level be added to calculate a company's return on investment. Thus, a corporation cannot ultimately measure ROI at the fifth level of training benefits without taking accurate measurements at the other four levels.

Evaluating training programs begins with Level I, which answers the question, "What are participants' reactions to the training and what do they plan to do with the material?" Trainers measure this with what they call "smile sheets" -- surveys or questionnaires that measure whether the training was meaningful or enjoyable. These surveys should also include sections on how the employee plans to use the lessons learned.

Level II answers, "What skills, knowledge, or attitudes have been changed or acquired [with the training] and to what extent?" Achievement tests measure how well the employee learned the information or skill presented.

Level III answers, "Did participants apply what they learned in training to their jobs?" Observer ratings and observations measure the degree to which the employee applies what he or she has learned. Observers (usually managers and supervisors) must be thoroughly trained in the evaluation system. Managers need to establish a system for leveling out the inconsistencies between observers' judgments.

Level IV answers, "Did this on-the-job application produce measurable results?" These results may include increases in productivity and efficiency, decreases in absenteeism and occupational accidents, decreases in customer complaints, and so forth. Isolating the effects of training from other variables that produce an effect in these areas, either through statistics or by using a control group, is vital to getting a clear picture of ROI.

Level V answers, "Did the monetary value of the produced results exceed the cost of training?" This is the measurement of ROI, which can be calculated in several ways.

Formulating training's value

This fifth level gets down to the brass tacks of quantifying the return on a company's monetary investment in training and requires a mathematical formula to determine an answer. Evaluation experts have developed three common formulas for measuring training ROI, each reflecting a different concept of a company's return on training investment.

TACTP - TACNP = PNS

Subtracting the total administrative costs of the new program (TACNP) from that of the former training program (TACTP) gives the projected net savings (PNS) for training administration. Although cost savings are certainly important, ROI encompasses much more than just that.

TCT # of students

= CPS

Dividing the total cost of training (TCT) by the number of students gives the cost per student (CPS) of the training. This is useful, but again is not a true measure of return on investment. Both formulas, although frequently cited as measures of return on investment, do not measure what monetary value or profit is derived from a training investment.

TB (in $) x 100 TTC

= ROI %

Multiplying the total benefits (TB) of training in dollars by 100 and dividing that by the total training program cost (TTC) gives the true percentage of ROI in a new program. This formula, included in Philips's Accountability in Human Resource Management, is the most accurate of the three.

To calculate the total training cost, make sure you include costs for paper and other office supplies, advertising, rentals or purchases of needed equipment, facility usage, and postage, as well as the cost of your trainees missing work.

To calculate the total benefits, you must evaluate the tangible results of training and assign a monetary value to such factors as:

• increased productivity (units produced, items sold, forms processed, tasks completed)
• improved quality (less scrap, less waste, less rework of product, fewer defects)
• reduced turnover
• reduction in lost-time injuries
• reduction in workers' compensation insurance claims
• increase in customer satisfaction as reflected in an increase in repeat sales.

These benefits are often called "hard benefits" because they can be converted easily to a monetary value. Other training benefits such as improved communication, enhanced corporate image, improved conflict resolution, increased sensitivity to human diversity, improved employee morale, and increased employee loyalty are less tangible, more difficult to convert to dollar figures, and called "soft benefits."

Soft benefits are important, and although they cannot be directly measured, they can be inferred or indirectly measured by associated outcomes. One way to approximate the value of soft benefits is to ask experts within your organization to give a monetary figure for these intangibles. Talk to employees, managers, supervisors, and executives and then take an average of the numbers they give you.

It's often difficult to demonstrate increased value of a company's human capital. But that doesn't mean that your training investment must end up on the expense side of the balance sheet, ripe for budget slicing. By measuring carefully the results of training and tying training to the strategic metrics a company uses to measure its business success, a company can increase--and demonstrate the increase in--its return on training dollars.

Published: June 2001