All About ADL
By J.D. Fletcher and Philip Dodds

The federal government's effort to foster interoperability of learning courseware, known as the Advanced Distributed Learning initiative.

The Advanced Distributed Learning initiative is a technology development effort sponsored by the U.S. Department of Defense (DoD) in coordination with the White House Office of Science and Technology Policy (OSTP). It's motivated by a vision of the future--one in which shareable courseware objects are assembled in real time from a global information network to create on-the-fly instructional or performance-aiding interactions with learners.

With the rapid growth of electronic commerce and the World Wide Web, much of this vision will occur without DoD or OSTP involvement. The ADL initiative is intended to take advantage of this global activity, accelerate it, and apply it to the needs of the DoD and other federal agencies. It will help provide the learning resources that the DoD needs to ensure the operational effectiveness of its forces. It will help provide similar resources to all federal agencies, which also depend on human performance and competence. Cooperative development among economic sectors--government, private industry, and academic--is needed and is being used to achieve the ADL initiative goals.

Few enterprises touch the lives of as many people as those concerned with education and training. High-quality education and training benefit individuals whose knowledge and skills are upgraded, businesses seeking a competitive edge, and nations in their overall productivity and increased global competitiveness. The ADL initiative is intended to ensure that all individuals have access to the education and training they need and that the teaching and learning enterprise becomes a high-performance activity.

We envision a future in which everyone will have an electronic personal learning associate. This device will be able to assemble learning or mentor presentations on demand and in real time--any time, anywhere. The presentations will be tailored to the needs, capabilities, intentions, and learning state of each individual or group (for example, crew, team, or staff) of individuals. Communication with the device will be based on natural language dialogue initiated by the device or by its users. The device will be portable, perhaps small enough to be carried in a shirt pocket.

Most of the technology needed to build such a device exists now. Though we cannot yet fit it into a shirt pocket, we expect advances in electronics to take care of that. What's lacking is content--in the form of instructional objects that we call "shareable courseware objects." These objects, shown on the left side of the diagram below, must be readily accessible across the Web, or whatever future form our global information network takes. Once these objects exist, they can be assembled in real time and sent to our personal learning associates.

The ADL initiative is focused primarily on the design and development of shareable courseware objects--and on fostering an instructional-object economy.

Computers are key

• computer-based instruction
• interactive multimedia instruction
• intelligent tutoring systems
• networked tutorial simulation
• Web-based training.

The graph below shows the combined results from three studies that compared one-on-one teaching (one instructor with one learner) with one-on-many teaching (one instructor with a classroom of 25 to 30 students). We might expect such a difference in instructional presentation to matter--and to favor one-on-one teaching. What's surprising is how much it matters. The difference in student achievement from these two approaches amounts to two standard deviations. It's roughly equivalent to raising the achievement of 50th percentile students to the 98th level of achievement.

Then why don't we provide one-on-one instruction to all our learners? The answer is simple: We can't afford it. Although one-on-one teaching may be an instructional imperative, it's also an economic impossibility.

Enter computer technology. Because computers can interact with users and tailor presentations to their needs--and because the necessary computer capabilities are less expensive than people--some of the gaps between one-on-one and one-on-many instruction can be filled affordably by computer technology.

Support for applying the tutorial capabilities that technology-based instruction allows is found also in other research. For instance, how much are learners in a typical classroom likely to differ in the rates that they master instructional material? It turns out that, over a range of material and settings, the rate with which the fastest 10 percent of students can progress compared to the rate of the slowest 10 percent of students varies by a ratio of about five to one. That means some students are able to master in a day what requires others five days to master. Even in highly selected instructional institutions, such as our best research universities, the ratio can be as high as seven to one. There are then significant economies and substantial effectiveness to be gained by adjusting for pace in instruction. This is something that technology-based instruction does well.

Is there research evidence that using technology to instruct reduces learning time? In more than 40 assessments performed since the 1960s, we've found time savings to average about 30 percent. This seems to have occurred without any particular effort to reduce instructional time. Much more might be gained if we tried harder.

The value of technology-based approaches may also be seen in comparisons with classroom interactivity--the frequency with which questions are asked and answered. Research shows that this frequency increases by about 4,000 percent--a ratio of 40 to 1--with technology-based instruction. The increased intensity of technology-based instruction seems hard to deny.

We also find significant improvements in knowledge and skills gained through using technology-based instruction. We are not yet at the two standard deviation level, or "two sigma," but during the course of 233 assessments of computer-based instruction, we found an improvement of 0.39 sigma (roughly raising achievement from the 50th percentile level to the 65th percentile). Another 44 assessments of interactive multimedia instruction produced an improvement of 0.50 sigma (roughly raising achievement from the 50th percentile level to the 69th percentile), and five assessments of recent intelligent tutoring systems that directly mimic one-on-one instruction produced an improvement of 1.05 sigma (roughly raising achievement from the 50th percentile level to the 85th percentile).

Cost reductions are also notable. We've found that reductions in operating and support costs average about 63 percent.

Overall, we have assumed a rule of "thirds" in our assessments of technology-based instruction. That means that use of these technologies--our ADL technologies--reduces the cost of instruction by about one-third and it either reduces time of instruction by about one-third or it increases the amount of skills and knowledge acquired by about one-third.

The real payoff for the U.S. military is that by increasing the effectiveness and accessibility of instruction and by reducing instructional costs, the ADL initiative will significantly enhance the military's operational effectiveness. The operational military payoffs of education, training, and mentoring remain difficult to quantify and assess, but they are the real and ultimate objective of our efforts. Similar increases in operational capability and productivity should result from application of ADL technologies in civilian education.

ADL "ilities"

• It must be possible to find needed and shareable courseware objects. They must be accessible. Basically, we need widely accepted and standard ways to store objects so that widely accepted and standard ways can be used to find and retrieve them.
• Once found, the objects should be usable. This means that they must be interoperable and portable across most--if not all--platforms, operating systems, browsers, and courseware tools.
• Once implemented, the objects should continue to operate reliably. If the underlying platform, operating system, or browser is modified (for instance, when a new version is released and installed), courseware objects should continue to operate as before. They should also be durable.
• Courseware objects should be reusable. Other platforms, operating systems, browsers, and courseware tools should be able to reuse, and perhaps modify as needed, the original courseware objects.

To discuss guidelines and, most important, make them implementable so they can be used in everyday practice to develop instructional objects, we need to refer to a model. This model isn't intended to replace other models for developing courseware, but it is intended only to provide a foundation so that we can accurately express our guidelines and make them usable. For these reasons, we call it a reference model. The product of this effort is a shareable courseware objects reference model (SCORM).

As the graphic below details, SCORM is a software model that defines the interrelationship of course components, data models, and protocols so that courseware objects are shareable across systems that conform with the same model. It remains a reference model. We have shown that other models of courseware development can be mapped into our reference model.

Fortunately, businesses in the courseware tool development industry have as much of a stake in the production of shareable courseware objects as we do. So, much of the work required to create the SCORM is being done by them. The primary function of the ADL initiative in this process is to organize, encourage, orchestrate, and document their development efforts--and to ensure that defense education and training requirements are reflected in their work.

A successful shareable courseware objects reference model must meet three primary criteria:

• It must support full articulation of guidelines that can be understood and implemented in the production of shareable courseware objects.
• It must be adopted, understood, and used by as wide a variety of stakeholders as possible (courseware developers, courseware tool developers, and courseware customers, for example).
• It must permit mapping of any stakeholder's model for instructional systems design and development into itself.

With so many current and potential stakeholders in the ADL initiative, developing a SCORM is as much an organizational challenge as anything else. Any organization concerned with learning through education, training, or performance monitoring is a likely stakeholder. These organizations include U.S. military services and those of other countries; businesses; academic institutions; and various laboratory activities concerned with ADL-type courseware development. Development under the ADL initiative is therefore cooperative and keyed to the promotion of each stakeholder's self-interest.

Many organizations have worked hard with the ADL initiative to build consensus. Draft versions of the ADL SCORM have been circulated for comment in hard copy and on the Website.

We've helped establish co-laboratories that will test ideas for developing shareable courseware objects and assess courseware objects developed elsewhere for their conformance with current versions of the SCORM. The co-laboratories are also assessing the costs and effectiveness of different approaches to technology-based instruction; demonstrating the capabilities and value of ADL technologies applied to education, training, and mentoring; acting as clearinghouses for information generated by them and others; and performing additional ADL support functions. The family of cooperating co-laboratories formed to support the ADL initiative is growing. There's enough work to go around to keep all fully employed. And there is much more to be done.

What can other stakeholders do to participate in the ADL initiative? Here are five possibilities:

• Provide review and feedback to the ADL SCORM process.
• Start designing content in small, logical chunks with sharing and reuse in mind.
• Develop an organizational strategy that links your core competencies to ADL capabilities.
• Share data, processes, findings, lessons learned, and so forth with ADL co-laboratories.
• Participate in ADL efforts to build consensus and cooperation through demonstrations and sharing of findings.

All interested organizations are encouraged to join the ADL initiative. Coordination of ADL activities is necessarily kept loose, with an emphasis on cooperation and helping participants best serve their own self-interests. Insofar as ADL represents the awaiting future and our vision is correct, we all have a stake in the success of this initiative.

Published: May 2000