Office and engineering workers' workloads are reduced with a
computer-augmented desk named InteractiveDESK. The desk has a large
desktop display with a pen-input facility and an ordinary upright
display with a keyboard, thus integrating features of conventional
systems and pen-based systems. The desk detects the operations on
real objects on its real desk top, and responds to the operations to
reduce users' workloads. The prototype of the desk assists users in
switching input methods and retrieving electronic files.
User-friendly interfaces are essential to make computers easy to use
for everyone, and many pen-based systems which realize friendly
interfaces are commercially available. However, most pen-based
systems are designed as mobile computing tools, and have some
drawbacks when used as computing tools in an office. For example,
small displays and text-input methods based on handwriting recognition
techniques are not desirable for certain office tasks. Because
portability for mobile computing is not always compatible with
usability in an office, it is necessary to study pen-based systems not
only for mobile computing, but also for office or engineering
environments.
We think that a pen-based system used in an office should be
integrated with conventional computer systems, which have upright
displays and ordinary input devices (keyboards and mice), and it
should provide a large display to replace an entire desk top. Then we
are trying to develop a computer-augmented desk which has an upright
display embedded in a partition facing the user and a large desktop
display with a pen-input facility on its surface. The desk enables
the user to use the keyboard and mouse with the upright display, and
to manipulate and make annotations on "electronic paper" on the large
desktop display with the pen. Switching the input methods and
displays ensures work is carried out effectively. We have named this
desk the InteractiveDESK.
The InteractiveDESK, on the other hand, forces the user to manage
multiple input methods and a large working space. It is accordingly
necessary to provide certain assists to reduce the workloads. Then we
try to utilize operations on real objects. There are many operations
on real objects in the real world, and almost all are ignored by
computer systems, even though the operations are unavoidable for users
in most cases. Our system tries to detect such operations, and
responds to them to reduce the number of the commands that the user is
expected to perform.
Figure 1 shows a schematic of the InteractiveDESK. The desk features
a camera to monitor the real desk top, and it processes the camera
images to detect the user's operations on real objects. Figure 2
shows a photo of an InteractiveDESK prototype. A 26" CRT display is
used in place of the partition display, and a projector display with
an A1-sized transparent tablet is used to provide the desktop display
with a pen-input facility. We are studying some new techniques for
intuitive and efficient interaction on this prototype.
FIGURE 1: Concepts Implemented on the InteractiveDESK
FIGURE 2: Photo of the Prototype
The system provides a keyboard connected to the upright display for
text entry, and a desktop display with a stylus pen for annotating.
However, it is sometimes inconvenient to switch input methods, so the
system should assist the user in switching them.
In order to make this possible, the positions of the keyboard are
utilized to anticipate the current input method. If the keyboard is
placed at the center of the desk, the system displays documents on the
upright display. Otherwise, it displays documents on the desktop
display. The prototype uses the desk monitoring camera to detect the
position of the keyboard.
Operations on the keyboard to change its position are unavoidable for
the user in many cases. But, ordinary systems are not concerned with
these operations. Our system, on the other hand, tries to respond to
them to reduces the users' workloads on switching the displays in
accordance with the input methods.
In this section, we propose another feature which enables users to
retrieve electronic files or directories maintained by computers
easily. The InteractiveDESK provides a method for linking real
objects to electronic files or directories. For example, it is
possible to link a scrapbook holding some transparent sheets to an
electronic file holding the original figures for the sheets. Once the
link has been established, the user can retrieve the file by simply
putting the scrapbook on the desk, without having to consider where
the file is stored. Most users would probably prefer to retrieve
files by just showing real objects rather than digging into the deep
hierarchy of a file system.
In the prototype, when a scrapbook is placed on the desk, the system
displays the list of the files linked to the scrapbook. The user can
retrieve the appropriate files easily by choosing items from the list.
If some documents have been displayed on the desktop when the
scrapbook is placed on the desk, the system assumes the user is trying
to link the scrapbook to those documents, and it shows a button on
each document. The buttons enable the user to link the corresponding
document to the scrapbook. In the current implementation, the
prototype is able to detect only a few real objects with differently
colored tags.
Weiser[1] has described a "tab," a "pad," and a "board" as essential
information instruments, but we would like to add "desk" to the set.
Desks have been the center of the space for intellectual human
activities, and they will retain their importance. We want to bridge
the real and electronic worlds via desks.
Wellner, working alone[2] and with Newman[3] have developed a system
called DigitalDesk. The idea of realizing interactions between
computers and users with real objects is proposed in the system. In
the system, a method for monitoring the real desk top is used to allow
users to input commands not only through electronic devices, such as a
pen or a mouse, but with unencumbered hands and papers. In our
system, however, the method is used to reduce the number of commands
users are expected to input. But, the features of the DigitalDesk can
also be applied to our system.
The idea of using real objects to retrieve electronic files is similar
to the idea proposed by Johnson, et al.[4]. They developed a system
called XAX, in which users can retrieve electronic files with a paper
form on which computer-readable marks are printed. In the system, the
form is sent to the document server by a FAX or a scanner, and the
results are sent back to the user by the FAX or a printer. The XAX
does not require any particular equipment. On the other hand, our
system requires a camera to monitor the desk top, but it allows users
to establish and trace links interactively.
We have described the features of the InteractiveDESK, which responds
to users' operations on real objects on the desk top to reduce the
users' workload. We are now investigating application fields to which
our system is well suited. For example, it will work well in CAD
systems that require a large working space and various input methods.
Linking real objects to electronic files will improve the interfaces
used by office workers.
Abstract
Keywords:
Augmented reality, pen-based system, computer-augmented desk
Introduction
FEATURES OF THE DESK
Switching the Input Methods
Retrieving Files
DISCUSSION AND RELATED WORK
CONCLUSION
