CHI 97 Electronic Publications: Late-Breaking/Short Talks
Possibilities for the Digital Baton as a General-Purpose Gestural Interface
Teresa Marrin
MIT Media Laboratory
marrin@media.mit.edu
Abstract
This paper describes issues and results from the design and use of the Digital
Baton, a new interface for real-time gestural control. Its construction was
originally motivated by the need for a new instrument on which to perform
computer music, and it was designed to replicate as closely as possible the
feel of a traditional conducting baton. However, it has unexpectedly become
a model for the design of new interfaces and digital objects, and is currently
being used to record data for analysis in gesture-recognition research. Some
preliminary results and future research areas are discussed at the end.
Keywords
gestural input, hand-held device, controller, musical instrument, conducting
© 1997 Copyright on this material is held by the authors.
Introduction
The principle motivation for the design of the Digital Baton was to create a
gestural controller which replicated as closely as possible the feel of a
traditional conducting baton while retaining the maximum number of intuitive
control parameters for the user. Therefore, it contains sensing systems which
capture many of the modalities of hand motion for input and control of both
discrete and continuous functionalities. These systems include an infrared
LED tracking system, accelerometers, and pressure sensors, all of which send
continuous values for the baton's position, orientation, acceleration, and
surface pressure to a computer via an external tracking unit. The approach
taken in building the Digital Baton was to mold, as seamlessly as possible,
these systems into the flexible plastic body of the baton. During the design
and construction phase of the project, it was realized that the enormous
number of separate control parameters available with the Digital Baton and its
easy-to-hold design made it an excellent candidate for a more general-purpose
gestural interface.
![[Teresa with the Baton...]](tm.gif)
Figure 1. Teresa Marrin conducting with the Digital Baton
(photo by Webb Chappell)
Digital Baton Hardware System
The Digital Baton hardware system consists of a baton, an external infrared
sensor, a tracking unit, and a computer. The sensors on the baton include an
infrared LED for positional tracking, five piezo-resistive strips for finger
and palm pressure, and three orthogonal accelerometers for beat-tracking.
Both the infrared sensor and the baton send separate data streams (including
values for absolute 2D position, 3-axis acceleration, 3-axis orientation, and
surface pressure) via cable to the tracking unit, which converts and sends the
signals to the computer. The body of the instrument consists of a clear tube
attached to a urethane base into which the sensors have been molded.
Underneath the pliable surface of the base is a hollow, hard shell which
houses the more delicate electronics. The whole instrument was designed and
molded to be small and easy to hold for any sized hand, with unobtrusive and
optimal placement of sensors.
Why Make New Musical Instruments?
The motivation for even exploring the issues of embedding sensor systems in
musical instruments is that current instruments for the performance of
computer music, such as electronic keyboards, breath controllers, and MIDI
guitars, are too difficult and unwieldy to use for most situations. On the
other hand, standard computer interfaces, such as the mouse and joystick, do
not have enough simultaneous degrees of freedom to phrase a musical line with
satisfying complexity. The reason the baton chosen was because it represents
a kind of meta-instrument; it makes no sound of its own and has no internal
mechanical functionality, and yet, in the conductor's hand, it is used to
direct the flow and form of the total musical result. We wanted to combine
this notion of the possibility inherent in the baton with the functionalities
of a traditional instrument, and ended up with a digital instrument which can
execute both the discrete, exactly-timed actions of individual notes and
higher-level functions such as shaping volumes and coordinating separate
events in time.
Unexpected Results and Conclusions
The Digital Baton project, in addition to revealing a need for new theoretical
models and analytical techniques in gesture recognition, taught us a number of
important lessons about designing digital electronics into already-existing
objects. For one, the set of common suppositions about how an object is to be
used do not necessarily hold, once that object is embedded with sensors.
Thus, the Digital Baton, while it resembles a traditional baton in many
senses, can be used in many more ways than its parent object. Also,
interpreting and responding to gestural events -- particularly expressive
gestures for music, whose meaning is not always clearly defined -- requires a
completely new theoretical framework. The tradition of musical conducting,
an existent gestural language for music, provides a good initial framework,
because it is a system of mappings between specific gestural cues and their
intended musical results. However, the powerful sensing technologies of the
Digital Baton provide the ability to go far beyond the metaphor and
functionality of musical conducting, providing the functionalities of a 3D
mouse, inertial guidance system, and mini-keyboard in one hand-held device.
The Digital Baton therefore has the potential to provide higher-level control
for general-purpose gesture applications in many different fields, and, in
addition to continuing in its role as an instrument for real-time performance
of computer music, it will also be used in the near future as a
data-gathering instrument for gesture-recognition projects in a variety of
fields.
Acknowledgements
I gratefully acknowledge the support of my advisor, Professor Tod Machover of
the MIT Media Laboratory, whose own project, the Brain Opera, gave my work a
context and an opportunity in which to explore new directions. I also have
been blessed with the encouragement of Professor Hiroshi Ishii, whose comments
have been invaluable, and who suggested that I consider the larger
implications of research in gestural music. Finally, I am proud to name my
collaborators in the technical implementation of the Digital Baton, most
notably Joseph Paradiso, Maggie Orth, and Chris Verplaetse.
References
1. Arent, Michael, Henry Lieberman, and Laurie Vertelney. "Two Disciplines
in Search of an Interface: Reflections on a Design Problem" in The Art of
Human-Computer Interface Design, Brenda Laurel, ed., Addison-Wesley, August
1989
2. Darrell, Trevor and Alex. P. Pentland, "Space Time Gestures," Proceedings
of the IEEE Conference, CVPR, 1993.
3. Fitzmaurice, G., Ishii, H., Buxton, W.,
"Bricks: Laying the Foundations for Graspable User Interfaces,"
Proceedings of the Conference on Human Factors
in Computing Systems (CHI '95), ACM, Denver, May 1995, pp. 442-449.
4. Ishii, Hiroshi, Minoru Kobayashi, and Kazuho Arita. "Iterative Design of
Seamless Collaboration Media" Communications of the ACM, vol. 37, no. 8,
August 1994. pp. 83-97.
5. Morita, Hideyuki, Shuji Hashimoto, and Sadamu Ohteru. "A Computer Music
System that Follows a Human Conductor." Computer Magazine, vol. 24, no. 7,
July 1991, pp. 44-53.
6. Paradiso, Joseph. "Gesture Sensing Technologies in the Brain Opera," to
appear in the proceedings of the International Computer Music Conference,
September 1997.
7. Sawada, Hideyuki, ShinÕya Ohkura, and Shuji Hashimoto. "Gesture Analysis
Using 3D Acceleration Sensor for Music Control." Proceedings of the
International Computer Music Conference, 1995. San Francisco: Computer
Music Association, 1995, pp. 257-260.
CHI 97 Electronic Publications: Late-Breaking/Short Talks
