The LMS Guess
By Tom Barron

A frothy sea of competitors are jostling for dominance. Business and technology models are evolving at Internet speed. New products and capabilities are sprouting up weekly. Pity the training professional who must select a learning management system.

The tension in the air was palpable. An undersized room at a recent training-industry conference was filled to capacity--and then some--with attendees seeking advice on a key decision.

"How many of you are being asked to evaluate and purchase a learning management system for your organization--say, in the next six months?" asked the session speaker, Brandon Hall. Roughly three-quarters of the assembly shot their hands up.

"Well, if you thought choosing an authoring program was scary," said Hall, "you haven't had to choose an LMS before."

Hall wasn't kidding. As more organizations migrate toward e-learning, or upgrade earlier systems, they inevitably face a tough choice--one analogous to choosing an operating system for their computer or an ERP system for their organization. In selecting an LMS (also known as training management systems, training administration systems, integrated learning systems, and other variations), training professionals must choose among a wide array of systems that will form the bedrock of their e-learning infrastructure. It's a major commitment--one they'll be proud of or live down, in the months and years ahead.

"These are large, complex software programs. They're evolving very quickly, they're expensive, and they're very visible to the organization," Hall advised his audience. "If you buy the wrong authoring tool, you can always go get another one. Get the wrong LMS and you're going to want to change your business card."

Trouble is, evaluating LMS products is like trying to hit a moving target. Among the most competitive of e-learning software areas, the LMS field is churning out new vendors (or vendor combinations), products, and capabilities on a weekly basis. Features aside, a litany of variables must be weighed in when considering LMS options, and it's anyone's guess how to prioritize them. They include the following issues:

• Interoperability. The ease with which an LMS works with third-party content and authoring tools, other database-driven software, such as ERP systems, synchronous e-learning systems and other software is an obvious concern. But what's more important--de facto software standards and informal "alliances" among vendors that pledge product interoperability, or compliance with industry-wide draft standards? And will data generated and captured by the system be salvageable when the organization migrates to successor systems?

• Pricing. The headaches involved in comparing prices of different systems have vendors on the defensive. "Most of the vendors have pricing structures that can best be described as vague," says Rick Reichenbach, the design manager of operations technology for McDonald's Corporation who participated in a recent evaluation of LMS systems. With differing combinations of per-seat pricing, per-use pricing, per-employee pricing, and other models, comparing costs of two or more systems can be exquisitely frustrating.

• IT buy-in. In case there's any doubt, tying LMS systems in with a company's existing HR and enterprise management systems calls for considerable technical expertise and the need to coordinate with the IT folks. Evaluating LMS systems without buy-in from IT is asking for trouble. More than a casual amount of custom programming is needed to integrate LMS systems into the "mother ship," as one analyst puts it.

• The ASP model. A new approach to e-learning management presented by the application service provider model forces LMS hunters to conduct a "rent-versus-buy" analysis. Using a vendor or third-party ASP to manage the IT architecture is the more expensive route, analysts say, but it's a wise choice for those in a hurry or where IT savvy isn't a core competency. A side benefit is that it delays a tough purchasing decision while the LMS market matures.

• Market consolidation. Analysts predict that a consolidation already underway will whittle the number of LMS vendors to a small fraction of their current number, which is estimated to be more than 100. On the bright side, most existing systems in place in organizations will be obsolete in the two-year timeframe by which the industry will coalesce around a handful of dominant vendors.

Following is a look at these and other issues facing training professionals as they try to make sense of the confounding LMS arena.

Features and futures

"It's no stretch to compare them with ERP systems, in terms of their capabilities and their complexity," says Reichenbach, who is scouting LMS options on behalf of the McDonald's training division.

Last November, Reichenbach took part in a thorough review of leading LMS systems organized by PC Week magazine in conjunction with the Masie Center and Department of Defense. An evaluation of seven high-end LMS systems by a panel that included industry, government, and academic representatives, the "LMS Shootout" provides a good framework for understanding general LMS capabilities and the merits of several systems. The results of the evaluation can be found at http://www.zdnet.com/pcweek/stories/news/0,4153,2391276,00.html, and a series of Webcasts that capsulize the evaluation and its participants is at http://www.zdnet.com/pcweek/webcast/shootouts/lms99.html.

Reichenbach says the effort, while exhausting, left the panel impressed at the capabilities offered by top systems. The skill assessment and competency management capabilities of the various systems were of particular appeal, he says. But the panel wrestled with how to distinguish the strengths of different systems based on the needs they're designed to address.

"There were several categories of products we were looking at, and in some cases we were trying to compare apples to oranges," he says. "Some products were integrated, full-service products," encompassing classroom and e-learning, "and others were specifically aimed at online learning," he adds. Others that were clearly geared toward academic institutions were evaluated alongside products for corporate users.

Reichenbach and fellow panelists were also stymied when trying to pin vendors down on various capabilities. "The answer we kept getting back was, 'We'll support it in the next release' or 'We can do a custom bridge to that [third-party] application.' It was never a simple yes or no."

Those responses are understandable, he says, given the amount of customization that has to occur in implementing an LMS. "What you get out of the box and what you end up with aren't going to be the same thing," he says. "They're not something you slam in and expect to work."

The relevancy of the PC Week evaluation is evaporating as participating vendors roll out new versions, acquire or are acquired by other e-learning vendors, or form partnerships to extend the capabilities of their systems. But it's a good starting point for understanding what to consider in evaluating LMS systems.

The vendor landscape

On the high end, a subcategory of LMS systems aims to be an all-in-one solution by including content authoring, knowledge management, and synchronous delivery capabilities. Though often lumped in with other LMS providers, these systems--from such companies as Docent, Lotus/IBM, and WBT Systems--are referred to in some quarters as "integrated learning systems." They face competition from LMS makers--such as Pinnacle, Saba and TrainingServer--that offer a core platform and pledge smooth interoperability with third-party content, authoring tools, and synchronous software. Many of these newer LMS systems use a Web browser as the primary interface, whether they're running over an intranet or the Internet.

The challenge facing a training professional who seeks to compare systems is the fact that they each emphasize a different functionality. Indeed, any two LMS products can seem as different as night and day. For example, the competency orientation of KnowledgePlanet.com's KP2000 LMS emphasizes the ability to carefully assess learners, test them, and manage a learner profile that maps their future training needs. Another LMS provider, Saba Software, trumpets its system's ability to serve as an extended enterprise learning platform that will manage training of employees, customers, and suppliers. The two systems actually share many common features--and are working to add strengths touted by the other--though the spin provided in marketing materials makes them appear vastly different.

Various research firms have compiled comparative analyses of different LMS systems that are worth exploring. A Website hosted by the Marshall University Center for Instructional Technology also provides evaluations--though its product list looks conspicuously outdated. The site is at http://multimedia.marshall.edu/cit/webct/compare/comparison.html.

Scalability is an important issue, particularly for large or multifacility organizations. Such vendors as Saba and VCampus tout the ability of their learning platforms to accommodate learner populations into the six figures; both have scored contracts with federal agencies that involve providing e-learning to tens of thousands of federal workers. But they offer very different approaches; VCampus was among the first ASP in the e-learning arena, while Saba is primarily a software vendor (it recently began offering ASP hosting, as well).

Looming Consolidation

Clark Aldrich, who tracks the e-learning market on behalf of market research firm GartnerGroup, notes that today's dominant e-learning vendors are establishing linkages with one another to create best-of-breed product suites. The most obvious example is Saba, which has established a large alliance of other vendors whose products it says function over the company's LMS. Other LMS vendors have picked up on the approach; most recently, TrainingServer announced its own "B2B Learning Network" with vendors, including NETg and others. The network uses the TrainingServer LMS as the basis for a courseware portal.

The alliance phenomenon is a strategy to assure customers that they're not heading up a blind alley in the absence of strong interoperability standards, says Aldrich. It remains to be seen whether industry-wide standards under development by the IMS Global Learning Consortium (see Newsbytes for story) will chip away at the "buddy system" approach, but Aldrich says the standards-setting effort's biggest challenge is keeping up with vendors' technology innovations. In any case, he says, the industry is heading toward fully integrated learning systems as major vendors coalesce in a maturing market. LMS systems serve as the anchor for these turnkey systems, and a key success variable will be their ability to provide knowledge management capabilities.

"The real criterion of success for [LMS systems] must be their ability to enable knowledge management processes by creating a mini-economy, efficiently bringing together people who have content with people who need it, both synchronously and asynchronously," Aldrich wrote in a recent report.

The ASP model is hastening the era of integrated learning systems as vendors compete to offer turnkey hosted services. In February, Click2learn.com announced a hosted LMS service, Click2learn Manager, that integrates with its authoring tool offerings. More recently, MindLever.com unveiled a hosted service using its LMS. Many other LMS makers are also rolling out hosting services, while purely service-based firms such as VCampus are beginning to emerge.

In many ways, consultants say, the speed of technology growth and the changing business models created by the Internet has trainers struggling to keep up with the realm of possibilities offered by vendors.

"The thinking of vendors is way ahead of the thinking of most of us in the field," says Brandon Hall. "It's a phenomenon of technology that we have to get used to."

"Different models of e-learning are emerging as numerous suppliers rush to fill the vastly increased demand for Web-based training in the form of content, tools, and services," says Jim Ayube, a training analyst with market research firm Aberdeen Group.

Some 60 percent of organizations will have persevered through the uncertainty and will be using an LMS of one form or another by 2003, predicts Aldrich, and by then learning platforms will have become more "commoditized." Until then, however, trainers have a tough row to hoe.

"Content can be flopped in and out, synchronous vendors will come and go, but you live and die by your LMS," he says. "The worst thing you can do is hurry your decision."

Published: April 2000