In May 1997, grandmaster Garry Kasparov looked into the Deep and knew fear. Kasparov, the reigning world's chess champion,
had just been defeated in a six-game match by an IBM supercomputer named Deep Blue. "I'm a human being," a stunned
Kasparov told the press. "When I see something that is well beyond my understanding, I'm afraid." What had
spooked Kasparov exhilarated scientists and engineers at IBM. "When we had gotten to the end of Deep Blue and
had beaten Kasparov, we looked at it and asked, where do we go next?" recalls William R. Pulleyblank, director
of the Mathematical Sciences Department at the Thomas J. Watson Research Center. It was clear to Pulleyblank
and others at IBM that the convergence of technologies that had led to Deep Blue's stunning victory in the
64-square world of chess could be used to make important moves in the real world. "This time,"
says Mark Bregman, then general manager of the RS/6000 division, "the winner will
be everyone whose life is touched by information technology."
Scientists at IBM Research recognized that Deep Blue had defined a new genre of computing, one that combines the
powerful parallel computers and advanced algorithms typical of scientific computing with the vast databases
typical of business computing to make human decisions. "It's doing science on nonphysical data," Pulleyblank
explains. The new genre, dubbed Deep Computing, has already been applied to a wide variety of problems,
from more precisely predicting the paths of thunderstorms to finding patterns in grocery purchases.
The possibilities opened up by powerful computers and algorithms are so profound that IBM has recently
formed a Deep Computing Institute under Pulleyblank's directorship to help explore and define this new field.
The most important factor in the emergence of Deep Computing is the availability of relatively inexpensive, extremely
fast and powerful systems such as the RS/6000® SP™ parallel computer, the guts of Deep Blue
. "Each node in an SP is essentially a workstation," explains Marc Snir, senior manager for scalable parallel systems.
"The glue of the SP is a hardware switch that can connect dozens or hundreds of these nodes together." The SP
architecture makes it relatively easy to scale up a system to the size required by the application.
What inspired Deep Computing was the convergence of powerful computers and the massive data sets typical
of business computing. "All of a sudden," says Pulleyblank, "it was possible to take the kind of processing
used for scientific computing and apply it to the commercial sector to get real business impact."
Deep Computing provides the methodology to uncover patterns and trends hidden in terabytes of data,
and to do so rapidly enough to make informed and timely decisions.
For some tasks, such as airline scheduling or customer profiling, systems with fast processors like OS/390®
mainframes or high-end RS/6000 workstations are fast enough. Other tasks demand parallel computers, like the RS/6000 SP,
which yoke together many processors. But according to Moore's Law, which says that computing power doubles every 18
months or so, such hardware distinctions will gradually disappear. "In the long run," says Nick Bowen, director
of servers at IBM Research, "Deep Computing will be something you do on any platform."
Processing speed is not the whole story behind Deep Computing. A clever algorithm can achieve overnight what progress in hardware would require decades to accomplish. Algorithms are the recipes computers use to solve problems -- the sequences of simple steps they use to arrive at complex results.
If Moore's Law is like amassing a fortune through compound interest, an algorithm can be like winning a lottery.
"The algorithmic things are really startling," says Pulleyblank, "because when you get those right you can jump
three orders of magnitude in an afternoon." That's the equivalent of 15 years of Moore's law progress. For
example, the same airline scheduling problems that used to take many hours to solve on a powerful mainframe
can today be solved in minutes on a ThinkPad® laptop computer. While some of this improvement
is due to better computer chips, much of the progress stems from faster algorithms.
The final technology driving Deep Computing is the advent of an inexpensive global communications network
. "This is absolutely crucial," says Pulleyblank. "I may never have a server capable of solving large weather
models on my notebook computer. But because I can link into these servers seamlessly, I'll have the same capability."
In Pulleyblank's vision, Deep Computing is the key to making sense of our explosively complex world.
Yet at the same time, it should be so widely adopted as to be taken for granted. "When you walk into
a room and click the lights on," says Pulleyblank, "you don't even think about what it takes for that
to happen. You just know that the switch turns the lights on. If we're successful, Deep Computing
will have the same level of pervasiveness, the same level of transparency."
Bruce Schechter, who has just completed a Knight Science Journalism Fellowship at MIT, is the author of My Brain Is Open: The Mathematical Journeys of Paul Erdös.
