A Pen-Based Computer System Gives Composers A Fast Efficient Means Of
Writing Down Their Scores
In Brief:
Research has devised a pen-based computer system for writing and editing
music. Known as the Pen-based Music Editor, it is simpler and more natural to
use than currently available systems. It does not rely heavily on pull-down
menus nor require its users to learn complicated commands. Musicians all over
the world can use editor, because it has no preset system of music notation.
Having long been regarded as a technique in search of a suitable
application, pen-based technology has increasingly begun to find some useful
niches. One of the most recent is composing musical scores.
Because they allow composers to mix different parts of a composition and
edit a score efficiently, and play back their work instantaneously, computer
music-writing programs have an obvious appeal. Unfortunately, most of the
programs on the market are keyboard- or mouse-based. That makes them slower than
writing music in longhand, so most composers continue to write music the
old-fashioned way.
However, pen-based input provides a natural alternative that offers the best
of both worlds. "People grab for the pen almost as second nature,"
says Krishna Nathan, senior manager of pen-input systems at IBMs Thomas J.
Watson Research Center. "With this new pen-based system we are developing,
people can do the same thing when theyre writing music."
The project, called Pen-based Music Editor, owes much to the fact that some
of the researchers in Nathans group have a strong feel for music. For example,
Homayoon S.M. Beigi, who initiated the project, is a classical musician who also
plays two Persian instruments the kamancheh and the tar.
Composing naturally
That type of background helped the team to understand and compensate for the
weaknesses of existing music-writing programs. Unlike such programs, the new
system allows users to compose, play back, and edit music without spending a lot
of time dragging down menus or entering numerous combinations of keystrokes, as
other programs require. "Its closer to what people do naturally," says
Nathan.
In the course of developing the music editor, Beigi experimented with
several approaches to entering notes. Because handwriting recognition software
is still limited to 85 or 90 percent accuracy, a system based solely on writing
all the notes did not seem feasible, although it could be used as an option.
Initially, Beigi designed an interface in which the notes and other symbols were
arranged along the border of the screen. A composer could then use the pen to
grab a symbol and place it on the score in the middle of the screen. In a later
approach, the user begins simply by touching the location on the screen where he
or she wants to put a note.
Beigi has tried two ways of actually inputting the notes. In the first, the
composer holds down the pen for the interval of time that the note will last.
The longer the touch, the longer the notes duration. Shortening a note requires
the composer simply to lift the pen a short distance away from the screen, while
a simple gesture directs the stem of a note up or down.
More recently, Beigi has introduced a new approach in which touching the
screen with the pen causes a vertical menu of notes to pop up at the desired
location. A note or rest of a given duration can be created by touching the
appropriate box in the menu, and the stem direction can be chosen in a similar
manner.
The system has a connection for a Musical Instrument Digital Interface
(MIDI), a format for transmitting information that can link a music synthesizer
to another synthesizer or a sequencer. That permits the composer to play a
passage on a keyboard and input it directly into the computer for annotation and
editing. The pen also has a button, and to write on the score, the user holds
the button down while writing. That permits the user to write words both lyrics
and musical annotations, such as allegro and andante as well as symbols, such
as that for crescendo.
Subtle details are important in composing music, so the system is
constructed to permit composers to program in the nuances. For instance, the
writer can specify an individual notes pitch as it starts and ends. Similarly,
users can indicate the speed of impact and release of the note equivalent to
the force of a musicians hands on a piano, or a violinists draw on the bow. To
hear how the work will sound, the composer can click on 128 different pictures
of standard general MIDI instruments. These include drums, strings, woodwinds
and even such beyond-the-auditorium sound effects as hovering helicopters.
Worldwide compatibility
The music editor is not restricted to Western music. "With this system
I can write Persian musical notation, which includes the use of quarter tones,
with equal ease," Beigi points out. "Music uses a wide range of forms
of scales and notations, depending on where you go in the world. For example,
musicians in India use different total intervals than we do. Our pen-based
system has no preset method of musical notation; you can train it yourself by
defining new shorthand symbols. So a system like this opens doors and
possibilities for musicians all over the world."
Beigi points out that incorporating handwriting recognition into the product
has made a simple system even more user-friendly. That represents a marked
contrast with present music interface software. "With current music editor
systems you need to go through many menus or learn a complicated system of
commands," says Beigi. "But with the pen-based music editor the
learning curve is very short, and could probably be explained in just a few
paragraphs. You could start using this technology right out of the box."
Beyond composition
The music editors uses go beyond composition. Beigi foresees its use in such
areas as teaching young music students, annotating music, and permitting
composers to edit scores that they have already written in longhand. He adds
that it may eventually lead to electronic scores, into which musicians could
insert a conductors directions.
Research staffers outside Nathans group who are also musicians have high
praise for the editor. "What theyre developing has the best potential for
music notation that Ive ever seen," says David Jameson, a one-time member
of an Irish rock band who set up and manages Watson's Computer Music Center.
Carl Tait, a Watson researcher who plays the piano well enough to have
competed in the 1990 national Chopin Competition, comments: "The main
advantage is that its so much more intuitive than the other systems I have seen.
Its the pen thats the critical element. Instead of the user having to conform to
the system, the system conforms to the musicians fairly natural method of input.
I would buy it myself if it were commercially available."
The team has already taken the first steps toward commercialization. Beigi
is working to attract internal IBM funds to make the Pen-based Music Editor into
a stand-alone product for release in the next year or two. And discussions are
under way with several companies who are interested in including the music
editor in their personal digital assistants.
Katherine Silberger is a freelance writer based in New York City.
