A Research Effort To Develop Ways In Which Computers Can Be Used To Create Music Is Producing A Broader Understanding Of Computer Science, Cognition And Other Subjects Far From The Musical Sphere.
In Brief:
Researchers in IBM's Computer Music Center have a two-part mission: adapting computers to make music, and using music and audio to explore broader topics in computer science and cognition. The center creates applications such as Kid Riffs® (released in 1995) and Cyberband (now in development), that enable children and adults, with or without musical sophistication, to compose their own music. Another emerging technology, DMIX, helps composers to turn musical ideas into compositions. And in a broader-based project called Sonnet, scientists are studying how sounds can enhance human-computer interaction.
Few people would regard music as a broadly defined research topic. But an unusual group of scientist-musicians in IBM's Computer Music Center at the Thomas J. Watson Research Center takes a different view. Headed by David Jameson, a one-time rock musician in the Irish band Time Machine, the center has a dual mission: developing ways to use computers to make music, and using music and other forms of sound as vehicles to investigate other fields.
"People underestimate the importance and value of music, but we learn so much and pick up so many cues through sound that we view it as a natural way of broadening our interaction with computers," says Jameson. Significantly, adds Jim Wright, a researcher who has collaborated on many of the music center's projects, "software and music are both abstract in very similar ways." Thus, the center's focus on systems for interactive music composition and performance has led to tools and technologies applicable to understanding topics as diverse as real-time systems, visual programming environments, and sonification - the use of audio to help understand data.
Simple scoring
The music center has already created products. The first, called Kid Riffs and aimed at children, places its users in a musical playground. There, they can select an instrument or a group of instruments and compose a variety of scales, rhythms, songs and musical patterns. Kid Riffs won a 1995 Parent's Choice Approval Award and a 1996 Consumer Electronics Show Innovation Award, along with several informal endorsements. Kid Riffs has also been translated into French, German, Italian and Spanish for the European market, and was highlighted in an IBM brand-image commercial featuring the African vocal group Ladysmith Black Mambaza.
Krawczyk, however, felt that the techniques that the IPSEC group were
considering adopting for message authentication were not demonstrably secure. In
effect, the confidence placed in the security features of the techniques was
based on immunity to specific kinds of attacks rather than on detailed
cryptographic analysis. So, along with colleagues Mihir Bellare and Ran Canetti,
he set out to design a new function that would authenticate both the
participants and the data.
Now, the center is working on Cyberband, a program for musical experimentation and composition aimed at adults. "Cyberband is like Kid Riffs on steroids for grownups," says Jameson. Cyberband's users will be able to create a tune, such as a country-and-western song, by arranging small chunks of music and decorating the chunks to change the way they sound, much like a clip-art program. After a tune is constructed, it can be transformed in many ways, such as playing it back using the exotic notes of a Middle Eastern scale.
While Kid Riffs offers some of those functions, Cyberband will contain more sophisticated features and a much stronger focus on composition and musical expression. "We wanted to provide tools to allow regular people to manipulate music in complex ways," says Wright. It will also permit individuals to collaborate online, composing together over the Internet.
Since it works mainly with pattern-based music, the program largely focuses on contemporary music. But it does not demand any ability to read music or play an instrument. "We like to say the program went to music school so you don't have to," says Wright, "but it still works on a high level." Indeed, professional musicians can use Cyberband as an idea generator or an expansion tool for their compositions. As a sketch tool, it becomes a way to form and document high-level manipulations. Cyberband compositions can be exported into traditional Musical Instrument Digital Interface (MIDI) sequencers or scoring programs to generate music transcription.
IBM plans to release Cyberband within a year, initially as a free item on the Internet. The twin objectives: to create a buzz about the technology and to publicize its capabilities, after the pattern of alphaWorks®: (see Research, Number 1, 1997).
Morphing music
Another project under development in the music center is DMIX. The brainchild of Daniel V. Oppenheim, an Israel-born computer musician who began working on it some eight years ago while a graduate student at Stanford University's famed Center for Computer Research in Music and Acoustics, DMIX had the goal of simplifying the composer's task of turning an idea into a composition.
Early computer music programs, first written in the 1950s, ran on large machines with little or no
audio capabilities and the music couldn't be created in real time. This limited the use of computers for creating music to composers with sophisticated programming ability and with access to large comput ers. The barriers began to fall in the 1980s, as MIDI, music synthesizers and graphical interfaces became available. However, says Oppenheim, "none of these approaches supported the way I wanted to compose." Reflecting his belief that "there is no single best way" of composing, DMIX incorporates the ability to blend different styles of composing. Because it allows a composer to begin with a particular set of notes and "slap on" a rhythmic structure or a tonal pattern, Oppenheim calls the underlying methodology "slappability."
DMIX allows the composer to draw a shape that the notes should follow. Through its graphical user interface, the user can morph one musical passage into another. For example, a Bach concerto can be turned slowly into a tango and then, by tiny increments, transformed again by adding a touch of a salsa beat.
Oppenheim sees DMIX as freeing composers to explore. "When you're composing on a computer, you can try out more radical and experimental ideas and see if they work," he says. "You can try a variety of different things and see what the acoustical and psychoacoustical effects will be."
Significant sounds
For all its beauty, music is simply a sequence of sounds. Over the past half-decade, several sorts of sound have gained importance in computer circles as means of communicating nonverbal information. The idea is to use "sonification," or sound, to deliver information via audio channels in the same way that visualization delivers data graphically. Sonnet, a project that started with Jameson's Ph.D. thesis on the use of audio for debugging programs, is exploring ways in which sounds can enhance human-computer interaction.
Sonnet is a visual programming language - a means of writing programs by connecting functional blocks together instead of writing code - for developing real-time applications. Sonnet programs are typically triggered by events in the physical world, which are then processed and converted to useful audio cues. The sounds allow users to monitor and react to what is happening in much the same way that, for example, the driver of a manual-shift car changes gear in response to the sound of the engine. Just as a driver pays attention to the engine's sound only if it changes dramatically, a user of an application written with Sonnet would react only if unexpected sounds occurred.
Jameson sees an obvious use for sonification in desktop computers. "For example," he says, "our eyes are being overloaded with constantly changing information appearing in various windows. You could easily use audio to inform you of diverse events, such as the arrival of important email or a change in a stock price." In the latter, Sonnet can enable investors to literally play the stock market, using a Sonnet block that receives price quotes and transmits them to other blocks to process and convert into sound. Listening to the market in this way, an investor might develop an intuition about what's going on, and "jump in" in response to auditory, rather than analytical, cues.
Sonnet also has important applications in computer science. Because it must react to events in real time with sophisticated scheduling algorithms, Sonnet can provide a test bed to explore better ways to build real-time systems addressing such critical issues as timing, parallelism and distributed applications.
Katherine Silberger is a freelance writer based in New York City.
More Information:
Superior Sound
Watson's Computer Music Center is not alone in IBM in helping computers to make music. At the Haifa Research Laboratory, Gal Ashour and his colleagues in the audio/video group have developed WaveCUBE® for Windows 95, a technique that greatly improves the quality of music played by personal computers that use conventional sound cards.
There are two main methods for producing music from Musical Instrument Digital Interface (MIDI): Frequency Modulation (FM) synthesis and Wavetable synthesis. FM synthesis is the conventional method for synthesizing music through computer sound systems.
To play a specific instrument, the technique reproduces the requested notes of the different instruments by referring to a table that specifies how certain sound features should be produced for each instrument. The effect, while reasonably accurate, sounds noticeably artificial. Today, this method is implemented by most pre-installed soundcards.
WaveCUBE uses the more advanced technique of Wavetable synthesis, which is based on a database consisting of miniature recordings of the various instruments. In most cases, the database contains more than one sound sample of the same instrument for different notes along its spectrum, to ensure accurate reproduction over a wide frequency range. When playing a specific piece of music, WaveCUBE selects the appropriate instruments and frequency ranges to produce the actual sounds. The result: high-quality sound close to CD quality.
WaveCUBE can be installed as a software upgrade. Once installed, it is transparent to the user. In an extension of their work on WaveCUBE, Ronen Ozer and Gal Ashour have developed an interface and a unique graphical user interface (GUI) that permits listeners to interactively adjust their position relative to the musical sounds emanating from a PC. By clicking a mouse on icons, the user can change the position of instruments shown on the GUI and - with the help of the Dolby® Prologic Surround encoder, which is built into the WaveCUBE synthesizer - produce sound that appears to come from all directions. Among other uses, the WaveCUBE-Studio GUI has potential value as a musical educational tool.
Altec Lansing Technologies, Inc., holds worldwide rights to WaveCUBE
