The Auditory Advantage
By Lenn Millbower
E-learning often lacks instructionally designed audio. And yet history suggests audio is critical to success. Given a choice, people fled video-alone formats for audio-video combinations. Films and video games have created audio parameters applicable to e-learning. E-learning programs that establish emotionally warm environments through integrated audio will gain competitive advantage. Here’s how to enhance e-learning with sound.
We've all heard the complaints about e-learning: It's boring.It's one-dimensional. It's impersonal. Many of these comments are the result of a little considered factor, the lack of a well-developed auditory signal. In other words, there’s the absence of an instructionally designed auditory component.
For most of us, the auditory signal is the rhythmic accompaniment to our lives. People are born to music; they study, drive, eat, sleep, and dream to music. We expect to hear an auditory signal and will usually supply one when none exists. Some people will talk to avoid silence. Others will leave a TV or radio on in the background, or provide their own music by singing, humming, or whistling.
Audio is also a critical success factor in any live training event. Imagine training devoid of audio (no conversation, lecture, or laughter) and you see the point. Yet, the auditory signal in e-learning is often non-existent. Given this information, e-learning designers should ask themselves the following questions:
What do your learners listen to when they take your e-learning class? Short of monitoring every e-learner's behavior, there’s no way to know. By not supplying an auditory component, e-learning designers have ceded the auditory input decision to each individual learner.
Does what the learners listen to aid or hinder learning? Composer George Burt once commented, “When we see pictures and hear music at the same time, we invariably make a connection, if only on an unconscious level.” When music is combined with information, powerful brain connections occur. Anyone who learned their ABCs by singing a song or remembers the song that was playing on the radio during a critical moment in their life, knows first hand the power of auditory signals.
Wouldn’t you like to control the auditory component so that you can increase the retention of learning? Of course you would, and e-learning should harness the auditory advantage.
There’s ample evidence of the effectiveness of music in aiding learning. Accelerated learning, neurolinguistic programming, multiple intelligences, and learning style theory all attest to the usefulness of music. For instance, both accelerated learning and multiple intelligences suggest that music aids provide sensory memory keys, or anchors, for learning points. In addition, a wealth of studies from the education, musicology, and healthcare fields demonstrate the effectiveness of music in increasing learning, healing, and self-esteem. Finally, the Muzak Corporation has proved repeatedly that music makes tedious tasks enjoyable. But it isn't necessary to rely on studies; history also proves the point.
Radio and silent films
Silent films were introduced to the American public at the Koster and Bial's Music Hall in New York City on April 23, 1896. The films consisted of short scenes that lasted only a few minutes. Considered sensational at first, their novelty quickly wore off and audiences stopped patronizing theaters that featured them within five years.
Live music accompaniment continued out of necessity until 1926 when the Bell Telephone Laboratories perfected a sound-on-disc system. In 1927, Warner Brother's studio produced the first talking picture, The Jazz Singer. Within one year, silent films were extinct.
Video games
Home video games had been around for several years when Atari introduced a game called Pong. In many ways, Pong was the home video game equivalent of the silent film. Based on a Ping-Pong game, it was one-dimensional, with an auditory signal that consisted of a single "pong" that sounded as the electronic ball bounced from side to side. The home video game manufacturers had learned little from the experience of silent films, and the bottom fell out of the video game market in the early 1980s.
But in 1985, something revolutionary happened. A company named Nintendo introduced its NES system. Fortunately, a team led by Minoru Arakawa realized that no amount of hype could sell a game. Arakawa and his designers set out to introduce buyers to a new kind of game, the Super Mario Brothers, featuring a new kind of hero, Mario the Plumber. By 1991, Nintendo had supplanted Toyota as Japan’s most successful company, and by 1995, Mario games had sold a total of 120 million copies worldwide. In 2003, video game sales are expected to generate more than US$17 billion.
How did Nintendo do it? What made Super Mario Brothers a success where other games had failed? Designed by Shigeru Miyamoto with music composed by Koji Kondo, the game offered something all previous video games had lacked: striking visuals married to auditory coherence. Miyamoto, in describing the game's design stated, "The [player’s] state of mind when he enters a cave alone must be realized in the game. Going in, he must feel the cold air around him. Not just the experiences, but the feelings connected with those events were essential to make the game meaningful." The game's music was instrumental in connecting those feelings.
E-learning
Through years of trial and experimentation, movies have developed a series of rules for the use of music. To achieve video game success, Miyamoto and Kondo applied those rules to video games. This revolutionary cross-platform usage has established a set of common film-video game auditory parameters. How do those commonalities affect potential e-learning applications?
Music compensates for lifeless dialog. In silent films, music provided audio when spoken dialog was absent. In an e-learning environment, character dialog isn’t always possible. In these situations, background music can compensate. Additionally, just as music provided background while silent film viewers read captions, background music can accompany the reading of on-screen instructions and open up more areas of the brain to the information.
Music provides editing continuity. Films jump from image to image. Video games jump from scene to scene. Background music provides continuity as scenes change. Silent film music also provided distractions while projectionists changed film reels. A similar problem exists in e-learning when a new scene is loading. Music can make the wait between scenes more tolerable.
In addition, video games' music is totally dependent on player choice. As the player directs the characters the game must jump from scene to scene, often at a moments notice. If a new scene is accompanied by music similar to the music that accompanied the prior scenes, the whole of the video game world seems connected. Musical themes that repeat scene after scene provide the illusion of seamlessness where little exists, and can aid e-learning in like fashion.
Music identifies time and place. Movies use ethnic, cultural, and time-specific music to indicate the time and place of a film. In the Super Mario Brothers game, each of Mario's worlds--surface, underground cave, undersea, and villains lair, had its own music theme. E-learning environments can also use music to indicate time and place. For example, each location in an office environment (personnel, scheduling, the shop floor, and the boss's office) could have its own identifying theme.
Music reflects character emotions. One important application of music in films and video games is to clarify the moods of the characters and situations, and help viewers feel those emotions. Most early composers for film had studied Wagnerian opera, which assigns a musical theme called a leitmotif to each of the main characters and specific situations. In films, the major leitmotifs would be introduced in the overture, reintroduced when the assigned character or situation first occurred, restated with the character's subsequent appearances, and concluded in the film's finale. For example, three of the most common leitmotifs are those assigned to the hero, the villain, and the love interest. From Psycho to Jaws to Star Wars to the James Bond series, virtually every successful film uses this leitmotif system.
That same set of procedures applies to video games. In Super Mario Brothers, Mario had a happy, bouncy, positive melody; Princess Daisy had a royal fanfare; and each villain Mario met was accompanied by an ominous theme.
E-learning could profit from similar musical treatments. Different personalities in an e-learning program could be assigned a leitmotif. In a customer service training program, for example, each type of customer could be identified musically. The curious customer might be assigned a playful, inquisitive theme; the angry customer an ominous theme; the demanding customer a rushed, impatient theme; and the satisfied customer a happy theme.
Music enhances action. Action feels more exciting when the auditory signal matches the pace and energy level of the visual image. The shark's theme in Jaws provides a perfect example of auditory enhancement. The rapid, low cello rumblings of the theme created an illusion of quick movement through the water.
In video games, music emphasizes action by aligning auditory cues to critical information. In composing the music for Super Mario Brothers, Koji Kondo assigned an auditory theme to the game's repeating events. From coin catching to jumping to flame throwing, every action had its own assigned sound. One way that the Super Mario Brothers used auditory cues was as an indicator that time was running out. The game allowed a player only five minutes to complete each scene. As the last minute of play approached, a fanfare would sound and the music would double in speed. This change in pace served as a warning that time would soon be up.
E-learning designers can use the same techniques to enhance the action in their program. Important information can be announced with appropriate fanfares, key on-screen activities can be linked to appropriate music themes, and quizzes can be timed with music. Finally, music can be linked to the practicing of repetitive tasks. As the learner becomes adept at the task, accelerating the music can help the learner increase task speed.
Music heightens dramatic impact. Music that suggests joy, offers hope, or warns of danger supports a film's success. Jaws provides an excellent example. During most of the filming, the Hollywood designed shark refused to work. Director Steven Spielberg was placed in the awkward position of filming a shark picture without the shark; therefore, terror came out of the music that accompanied basic water images. As the camera moved through the murky water and the musical theme sounded, the tension grew. The music captured the dramatic mood so effectively that Spielberg used it to tease the audience.
E-learning music should also highlight dramatic moments. Background music could suggest appropriate moods, be it anxious themes for test-taking, scary music for time in the boss's office (or positive music depending on the characteristics of the boss), and uplifting music that celebrates a big sale.
Film composer George Burt once commented, "When video and audio are placed together to achieve a common goal, a great deal more is expressed than would be possible by means of either medium alone." Where language communicates information, music communicates feeling. To communicate the emotional context of a scene, to open a window into characters' feelings, and to build humanity into a disjointed, cold medium is the heart of music's function in multi-media.
In both film and video games, an audio-visual medium faced a crossroads. In both cases, success resulted when the auditory and the visual signals were integrated. I believe that e-learning faces a parallel situation. E-learning providers who, with history as their guide, create emotionally warm electronic environments with an integrated auditory component will gain a huge advantage over the competition. For those products, there will be no complaints about the coldness of e-learning.
Published: January 13, 2003