ROI of E-Learning: Closing In
By Paul Harris

In most departments within a corporation, determining the return on a given investment is a straightforward accounting exercise that produces a factual and typically uncontested result. But when it comes to e-learning, computing ROI suddenly becomes a complicated procedure requiring thoughtful chinstroking, serious seminar time, and earnest input from consultants and vendors.

Why is that?

There are reasons aplenty. Among them are that the acquisition of an e-learning platform, courseware, or technology produces an array of hard and soft cost-savings that often factor into the equation. But not always. Every situation is slightly different.

How much money can a company directly save by replacing its dependence on the classroom with a distance learning operation? It’s not rocket science. The standard ROI equation calculates Return = Benefits ÷ Cost of the System.

Easily determined hard savings include reduction in training budgets and materials, travel, instructors, physical facilities, administrative time, and hours of lost productivity when employees are off-site—among other savings. But what about improved productivity and proficiency, learning curve, and employee retention, satisfaction, and maybe even morale? They’re not always so easily measured. In some cases, they’re not even relevant. Or they may be relevant to one level of the operation but not to another. In addition, the increased productivity of some departments, such as sales, is more easily measured than that of others. And when do you begin measuring those soft ROI factors? How often do you do it? When do you stop, or do you? How do you measure the impact of e-learning when it’s part of a blended learning solution?

How do you get business units to buy into the concept so that ROI calculations can be solidly based on their shortand long-term strategic goals? (Wait. Where was that seminar again?) And how do you ensure that ROI calculations really determine the true value of e-learning to an enterprise?

Fortunately for training professionals, there’s no shortage of opinions or expertise. Indeed, an army of e-learning vendors stands ready to assist any perplexed executive on the nuances of ROI and the benefits to be gained by their particular products. For example, SkillSoft dispatches representatives twice a year on 30-city tours to provide workshops on e-learning ROI. The free seminars focus on best practices for analyzing the ROI of various types of learning programs, business factors involved in ROI analyses, tips on how ROI studies can help solve business problems, and case studies. Attendees learn how to design ROI studies in these typical areas of cost justification: classroom expenses, employee turnover, training duration, employee productivity, and legal costs.

Visit any vendor’s Website and you’ll likely find a white paper or two devoted to ROI issues. One such document from LMS provider Plateau Systems seeks to help the training manager sell the Plateau enterprise system to senior managers in a language they understand. It carefully explains the short- and long-term, as well as strategic and individual benefits, and offers formulas to measure them.

And there is plenty to learn.

“Suddenly, the conversation on ROI is changing,” says Brenda Benedet, director of e-learning strategies for SkillSoft and a principal leader of its ROI workshops. “In the past, organizations merely looked at cost-savings and benefits from an e-learning perspective. Today, as they position e-learning as a strategic and majority element of blended learning, they’re also looking for more tangible returns that relate to the business—performance changes as well as the business impact.” Adds Benedet, “Gone are days when at the end of the year, organizations were content to report x amount of e-learning usage. Today, they must know if it made a difference. Hence, an interest in ROI and multilevel measurement.”

Benedet says e-learning ROI is a vitally important topic within companies and training departments face challenges in making it relevant. “Historically, learning professionals were good at determining cost-effectiveness and delivering that story based on utilization. But as e-learning addresses business problems, they need to look at actual impact. We see content now addressing specific kinds of business solutions, such as call center training, sales training, IT certification, and leadership. Those are strategic issues.”

Companies can measure performance and business impact more aggressively by looking at the learning solution in terms of the potential business impact and potential ROI, says Benedet. She believes it’s important that when conducting ROI of e-learning, corporate trainers ensure they have appropriate stakeholders within the organization on board. “It’s important to remember that learning professionals don’t own the business problem or the data for measurement, so they must be a partner with the unit. They must understand what that unit’s business needs are, as well as the objectives that the unit is being directed to deliver that year.”

Benedet points out that directors in training departments tend to be close to the business but not a true partner. “There’s the rub. Should an organization want to implement a strategy for ROI, it must first revisit how its own learning organization is organized. It should include ‘performance consultant’ as part of a trainer’s job.”

Benedet suggests that organizations considering implementing an ROI strategy start the process by delivering an ROE (return-on-expectation) survey. “It should be highly visible, and should pave the way for future sponsorship and participation in real ROI projects.” She advises companies to look at an opportunity and determine whether measures are in place. “If not,” she advises, “do a return on expectations.”

Continual roadmap

Nucleus Research, a Wellesley, Massachusetts-based research firm, keeps careful track of ROI issues throughout e-commerce. It reported last fall that elearning solutions and e-business integration platforms ranked among the ROI leaders; businesses such as b2b marketing and stand-alone content management ranked as ROI laggards.

Along with providing such analysis, Nucleus advises companies on how to quantify their ROI. Its own reports lean heavily on the hard costs and overhead savings that are easier to obtain than productivity data. Nucleus says that most companies gain significant returns from even modest investments in e-learning technology. “Most e-learning customers have quickly recognized first-tier benefits, including reduced costs for travel, customer support, human resources overhead, and regulatory compliance—and eventually second- tier benefits, such as increased employee performance that directly impact profitability.”

Rebecca Wettemann, founding partner and senior analyst at Nucleus, says that ROI analysis remains a valuable roadmap for e-learning. “Calculating ROI isn’t difficult; it’s merely structured,” she says.

Wettemann reminds that the typical CFO conducts an ROI analysis on every other business component. Similarly, the CIO must articulate the business benefits of e-learning based on clear assumptions, she advises. “The strategy for the CIO is to identify the benefits and work to quantify them based on characteristics of the organization, and to look at expected best-case and worst-case scenarios.” She says companies can maximize the return on their e-learning investment by being as efficient as possible with the technology.

“Companies need to take a critical look at what they’re trying to deliver. If they can use off-the-shelf content, they should do that. If basic tracking is all that’s needed, they shouldn’t buy an expensive LMS. If they need high-fidelity learning, they’ll get the best returns by developing the content. They should do this across their learning goals.”

ROI analysis is a valuable roadmap to continually measure the success of a company’s e-learning solution, notes Wettermann. “I do ROI analysis before I deploy e-learning and once deployed, I use it to see if I get the benefits I expected.” When analyzing benefits, consider only those that have an impact on the business and not those that offer only warm and fuzzy feelings, she tells clients.

Kodak’s big KISS (keep it simple, stupid)

Sometimes an analysis of every conceivable ROI benefit yields too much information, figures Catherine Nowaski, director of e-learning, worldwide learning and development at Eastman Kodak Company. In fact, an ROI analysis should be as simple as possible, she believes.

That’s the conclusion Nowaski and her colleagues drew after pondering spreadsheets full of factors deemed potentially relevant to the adoption of an enterprise- wide LMS at Eastman Kodak two years ago. After examining the smorgasbord of hard and soft measurements, including various models, a taskforce opted for the squeaky-clean approach to a business case for its proposed LMS candidate.

“We made a determination that the best way to look at our investments was to ask what the cost for classroom training is now, including vendor payments, as well as clerical and other overhead, and what it would take for us to deliver the same or more training in a self-study environment,” says Nowaski. All other yardsticks were rejected.

“We didn’t even count things like time off the job, even though that’s important,” she says. She adds, “We looked at all of the other measurements and decided they weren’t right for us.” Why? With 70,000 employees worldwide and countless training needs, the size and scope of the project precluded the use of most measurements, says Nowaski. “Vendor advice became too complicated. We didn’t have the data.”

Soft ROI counts

Sometimes the adoption of e-learning is such an obvious choice and its benefits are so apparent that an ROI analysis seems to write itself.

Such was the case at A.T. Kearney, a Plano, Texas-based global management consulting firm owned by EDS. With an army of consultants scattered across 60 offices in 30 countries, it faces a constant need to train new employees and keep veterans abreast of best practices. It trained exclusively in classroom settings until opting several years ago for an even mix of classroom and online learning. A.T. Kearney turned to Vuepoint, a Long Island-based e-learning vendor that offers a single platform that combines an LMS, an LCMS, and collaboration technology.

From a training standpoint, A.T. Kearney had three distinct problems, explains Al Morrison, vice president of global professional development. For starters, its consultants spend most of their time on location with clients, making classroom attendance difficult. Two, the rapid pace of technological change presents a challenge in keeping employees current on the latest offerings, delivery mechanisms, and other issues. Expensively produced learning materials were typically obsolete before everyone could take the class. Last, the wrong people tended to show up for the training. People who were properly grounded and ready for training were always outnumbered by colleagues not yet grounded and qualified or others who already knew the material.

Online e-learning solved all three problems, says Morrison. It solved the disbursement problem by enabling the company’s consultants to access courses anywhere for use on their laptops. It stayed abreast of technological change with just-in-time courses that are so inexpensively produced they’re considered a throw-away. And the e-learning courseware is self-selecting, weeding out employees for whom a course is inappropriate.

“We made our investment in Vuepoint without considering ROI,” says Morrison. “We went back and said, ‘This is a good idea and will pay for itself, but let’s run some numbers and see what happens.’” A.T. Kearney discovered that it could develop programs for half of what they used to cost, and that development time dropped to 44 percent of the traditional approach. The firm reported savings of more than US$20 million with more than 4000 days worth of Web-based training within a 12- month period.

Ara Ohanian, CEO of Vuepoint, agrees that improving proficiency and the learning curve are key ROI benefits of e-learning that companies shouldn’t discount. Although such benefits may appear soft to a training director, they most definitely are not regarded as such by the executive in charge of any P&L area, he says.

Ohanian cites another example, Vuepoint client, the homebuilder David Weekley Homes. A line manager concluded one day that increased sales would result if the company could improve the satisfaction of its customers in the post-purchase phase of the relationship. After all, that’s where most referrals come from, he reasoned. J.D. Power & Associates was enlisted to measure client satisfaction, and an e-learning program was adopted to address the areas of concern with employee training. David Weekley Homes is currently enjoying its most profitable year, due in part to that effort, the company claims. “It began by measuring the particular thing that the business person knew would have an impact,” says Ohanian.

Another soft but critical ROI component involves speed. Not only is time-to-market crucial for most sales organizations, but in this era of frequent product changes, relaunches, and complex product mixes, the rapid transfer of knowledge to the sales channel is extremely vital.

The Hartford Company asked Vuepoint if it could help shorten the six-month period needed to fully train its sales channel about new products. With adoption of the Vuepoint platform, that time was reduced to two weeks. “Speed-to-knowledge may be a soft measurement, but the impact is greater than any travel reduction would ever be,” says Ohanian.

Improving ROI

If there’s anyone who still classifies performance improvement as a second-tier benefit of e-learning, don’t tell that to Brian Carlin. He’s president and CEO of Maritz Learning, a Fenton, Missouri-based provider of custom training, e-learning, and performance management for sales and service organizations.

Maritz Learning helps companies ensure that e-learning is the means to a profitable end. Maritz Learning, a unit of Martiz, a US$1.3 billion provider of performance improvement, research, and travel services, views performance improvement as a three-legged stool: 1) training and professional development, 2) motivation through rewards and recognition, and 3) performance measurement. The ultimate goal of the integrated approach is to get “energized competence” out of employees as measured by the approach’s impact on a client’s top line, not its expense line.

By marrying performance measurement with professional development, Maritz goes well beyond the average e-learning vendor. A case in point is telecom client Sprint, which sought increased performance from its outbound call center. Maritz Learning began by measuring the performance of Sprint’s sales reps in a dozen core competencies, followed by a 360-degree skills assessment to determine how they ranked. Following a gap analysis, it prescribed training interventions and developmental activities, using its own LMS and courseware.

Part 2 involved customer satisfaction. Calling on Maritiz’s research business unit, Maritz Learning conducted follow-up surveys to measure satisfaction ratings against specific competencies. The resulting interactive report detailed each call center rep’s skills assessment rating and customer satisfaction outlook. Maritz Learning also helped office furniture maker the Hon Company double its sales with an e-learning program for its network of 500 nondedicated dealers and 8000 representatives, developing the Hon Masters Program to reward reps for selling Hon products and successfully completing specially designed Hon product training modules.

Carlin believes that the growing ability of Maritz and other vendors to help their customers meet their strategic goals—and to effectively measure e-learning’s impact—will increase the corporate world’s reliance on e-learning to solve training needs. That message is resonating with customers, he says. And the value of ROI analysis is only beginning to be felt.

Published: February 18, 2003