Logo AHome
Logo BIndex
Logo CACM Copy

papersTable of Contents

A Palmtop Display for Dextrous Manipulation with Haptic Sensation

Haruo NOMA , Tsutomu MIYASATO and Fumio KISHINO

ATR Communication Systems Research Laboratories
Hikaridai2-1,Seika-cho,
Kyoto, 619-02, Japan
Tel: +81-774-95-1211
<noma,miyasato,kishino>@atr-sw.atr.co.jp


ABSTRACT

Palmtop displays have been extensively studied, but most of them simply refocus information in the real or virtual world. The palmtop display for dextrous manipulation (PDDM) proposed in this paper allows the users to manipulate a remote object as if they were holding it in their hands. The PDDM system has a small LCD, a 3D mouse and a mechanical linkage (force display). When the user locks onto an object in the center of the palmtop display, s/he can manipulate the object through motion input on the palmtop display with haptic sensation. In this paper, the features of a PDDM with haptic sensation are described, then four operating methods and the haptic representation methods for a trial model are proposed and evaluated.

Keywords

Palmtop display, Haptic sensation, Force display, Virtual reality, Teleconference, User interface

INTRODUCTION

A virtual world generated by VR is expected to accommodate applications in many fields, such as remote control, CAD and art. In general, such applications allow users to arrange both virtual and real objects in the virtual world by a number of methods. We will suppose that the basic operation is a sequence of the following three unit phases. These unit phases make up a basic motion, and a complex operation is considered to be an assemblage of basic motions. In many VR related studies, specialized interface devices are designed and applied based on a certain goal. However, not many VR interfaces are good at balancing both observing (output) and controlling (input) methods. If one side should prevail, the sequence of the unit phase becomes unstable and the user could encounter problems in use.
In this study, a new manipulating device using a palmtop display is proposed as one solution to this balancing problem. It is called the Palmtop Display for Dextrous Manipulation (PDDM) shown in Figure 1 and was designed especially for fine manipulating tasks in our virtual space teleconferencing system (VSTC) [7]. The most important design concept of the PDDM is an interface device which combines an input method and an output method. This device allows the user to be unaware of mode shifts, which interrupt the phase flow in manipulation.
Figure 1 Pamltop display for dextrous manipulation
In the next section, we explain the features of the PDDM and clarify its layout in other I/O devices. A trial PDDM system was designed, and four manipulating methods were tested. Additionally a haptic representation method was tested to evaluate our design. In the last section, the remaining problems and prospects of the PDDM are discussed.

PALMTOP DISPLAY FOR DEXTROUS MANIPULATION

The VSTC has been designed to generate a virtual conference room with a high realistic sensation. The trial VSTC has a large fixed screen, a glove-like device and a magnetic 3D position tracker. Generally, such a system provides a direct manipulating interface for the participants and they can handle virtual objects roughly without learning. It is, however, difficult to manipulate objects exactly with such devices. One of the main reasons for this is the excessive degrees of freedom (DOF) in the manipulation. When we wish to draw a straight line on a peace of paper, we use a straight scale to restrict the DOF of the hand to 1. At this point, two solutions can be proposed: one is to give a virtual constraint to the dynamical motion objects in the virtual space [5], and the other is to give a physical limitation to the user's motion using a mechanical linkage [3][8]. A mechanical linkage that provides a reaction force for the user is called a force display.
In this paper, a kind of force display is introduced. In addition using reaction force to restrict the user's motion, the force display can render the features of objects, such as their weight, hardness, texture and so on. Almost everyone who has tried our trial VSTC has complained about the lack of haptic sensation, so it is assumed that these factors are important for generating a realistic manipulating environment. However, using a force display with a large fixed screen requires us to solve the following problems. Considering these problems, we proposed a palmtop display as a motion input device. The PDDM consists of a palmtop display, a position and orientation sensor and a force display, and can act as a visual display and as a motion input device with haptic sensation in one unit.
The features of the PDDM will be explained using the ideal system in the VSTC shown in figure 2. The right side is a real space where the PDDM user is, and the left side is a virtual world where other participants and objects are displayed on the large screen. When the PDDM user wishes to modify the lidless box out of hand, the user copies it into the special space (local studio) in front of him/she. The camera metaphor could be introduced for this motion using the palmtop display. The local studio is the working area for the PDDM user and only s/he can modify objects in this space. A change in the copy is immediately reflected in the original. Here, s/he can observe the object from any viewpoint with the palmtop display (observing phase). Deciding the best viewpoint for the modification, s/he focuses on the part of the object that s/he intends to modify in any way (handling phase). Here, the motion of the palmtop display is directly related to the motion of the object. It will allow the user to feel as if s/he is grasping the object directly in front of him/herself. Moreover, the force display system applies a reaction force to the user through the palmtop display in the handling phase. After the user finishes modifying the object, s/he releases it and return to the palmtop display mode to confirm the result (confirming phase).
Figure 2 Conception of the PDDM system
As the sequence of these operational phases is shifted with a grasping motion, the user does not need to pay much attention to the shifting operation. In the trial model mentioned later, this is realized by locating the object in the center of the display and pushing and holding a button behind the palmtop display. We can also propose a touch panel on the display as a more immersive method.
The problems with the force display in large-screen VR can thus be solved, as follows: Additionally, the following features are expected: In the following sections, additional features of the PDDM are described with comparison to other interface devices.

A LAYOUT OF THE PDDM IN INTERFACE DEVICES

 As mentioned above, the PDDM is an interface device that has two faces: information output and operation input. We clarify the position of the PDDM in comparison with usual interface devices.

As an output device for visual information

 A typical visual output device for VR systems makes virtual world images using a virtual camera that follows the user's motion. Through such animation, the user gets an intuitive sensation.
The PDDM employs a palmtop display for visual information. Palmtop displays [1,4,6,11] usually have a smaller display than other VR devices, such as fish tanks [2,9], large screens [7] and HMDs. Studies related to the palmtop display reported that the immersive sensation does not always depend on the screen size. This is because the palmtop display gives a higher degree of freedom than other devices, and the viewing point can be easily and intuitively changed to any place and direction the user wishes. The user can observe all of the directions with a palmtop type interface immediately.
As a related work, the Chameleon [4] is well known. The Chameleon allows a user to search information in virtual space interactively. In this case, the motion of the palmtop display is mapped to control the region where the user focuses upon in the database. The biggest difference between the PDDM and similar others is that the motion of the PDDM is applied not only to change the focused point, but also to manipulate remote objects as if they are in front of the user with kinesthetic feedback.

As an input device for motion

 Next, we will propose a taxonomy for input devices. Figure 3 shows the two dimensions of the taxonomy. The horizontal dimension is "the frame of reference of the interface". This means what frame is referred for evaluating the input motion. The options are "user based" and "world based". The other dimension is "the relation between the controller's domain and the effector's domain." This means the spatial relation between the domain in which the user inputs motion and the domain in which the effector is controlled, and there are either tied to "overlap" or "separate". Almost all input interfaces can be classified into two groups: "based on the world and overlap" or "based on the user and separate".
Figure 3 Taxonomy of the input device
Based on the world and overlap
This group contains a glove-like device and a touch panel. The user's motion is always evaluated according to the frame of reference fixed on the virtual world with them. The effector appears in the same domain where motions are input. For example, when the glove-like device user wants to move an object in the north direction, the user has to stretch his/her hand to the object and grasp it. Then, s/he has to move the hand in the north direction. Such a style is called direct manipulation and offers intuitive operation. Even a novice user can manipulate objects well without much practice. Conversely, the user's motion and the effector's motion must be directly engaged, so the operation strongly depends on the user's position in the virtual world. For instance, the user can't manipulate an object that is located out of his/her reach. When s/he wants to do so, s/he has to move beside the object.
Based on the user and separate
This group contains mechanical mouse, joystick and so on. The user's motion is evaluated according to the frame of reference based on the user with them. The domain where the motion is input is separate from the domain where the effector moves. An icon representing the user in the virtual world is an essential part of this category. As the input motion is mapped to the icon's motion with some transferring functions, it is difficult for a novice user to use this well. The transfer function, however, doesn't depend on the spacial relation between the user and the world. Usually, such devices can manipulate almost all objects in the virtual world within only a compact domain. Optical mouse, track ball, tablet and SpaceBallª are very close to this category, but their frames of reference for evaluating input motion is semi-fixed in the world. These devices are located in the middle of the horizontal dimension.

The PDDM in this taxonomy

Since the user can fundamentally focus on and manipulate objects without an icon in the display, the PDDM is located vertically in "overlap". On the other hand, the user can directly handle an object in his/her frame of reference at the handling phase. This means that the PDDM is located in "based on the user". Therefore, the PDDM is located as a device "based on the user and overlap", and it is expected that the user can manipulate a remote object as if s/he is holding it in the frame based on him/her.

THE TRIAL MODEL

Configuration of the trial PDDM

The goal is to obtain an intuitive interface for manipulation in the VSTC. The PDDM is proposed as one solution. Evaluating this concept, we built a trial PDDM with an Ultra Sonic Motor (USM) force display [13]. (Figure 4) The palmtop display of the PDDM is a 4-inch LCD (resolution 220x480 dots, weight 350 gf) and has two push-button on the back. The user holds both sides of the display and presses the buttons with the left fingertips. Here, only one push-button is used to shift between the observing and the handling phase. A magnetic position sensor is installed on the side of the display. This sensor provides the orientation and position of the palmtop display.
The display is connected to a 6 DOF USM force display that we have developed. 3 of these are actuated with USM and generate a force in any direction at the display. The others are an unactuated 3-axis universal joint at the end of the arm. Thus, the system can't restrict rotating torque. Each arm's length is 30 cm, and the workable volume is a half sphere with a radius of 60 cm. In this configuration, the force display can generate a maximum 5 N reacting and sustaining force in the volume.
A USM works with frictional force, which completely differs from an electric magnetic motor (EMM). It can generate a high sustaining and rotating torque in one component and the output can be controlled from a full sustaining torque to a full rotating torque continuously with our proposed method [13]. The rotating torque per unit weight of our USM is almost twofold that of a typical EMM. In addition, a USM can output maximum torque at lower speed (1.3 Nm at 60 rpm in ours), so it does not need reduction gears, which reduce back driveability. Furthermore, since a USM makes no magnetic noise at all, it does not affect the results from the magnetic position sensor introduced in many VR systems.
An electrical touch sensor is installed on the back of the palmtop display. When the user holding the display, all joints can be freely rotated. When the user releases it, the system detects it and immediately locks all joints. It is an important feature that the PDDM keeps the same position as that where the user released it. The system does not support this now, but the weight and the inertia of both the display and the mechanical linkage can be reduced to a level sufficiently small with dynamic impedance control. It is expected that fatigue in the user's arm through extended use can also be reduced.
Figure 4 shows the configuration with the VSTC. Both the PDDM and the large screen display the same virtual world. Figure 5 is a diagram of the system. The SGIª workstations contain a master server process, a drawing process and communication processes. A PC is used to control the force display and communicate with the server process through the ethernet. The drawing process can update an image in real.
Figure 4 The PDDM with the VSTC
Figure 5 Configuration of the trial system

MANIPULATING METHOD

 As the PDDM's operation in the observing phase is very similar to the use of a video camera with an LCD, even a novice user can use it well. On the other hand, the usage of the PDDM in the handling phase is highly unique. As shown in figure 6, the motion input on the display is reflected on the grabbed objects and the virtual camera in many way. The camera takes the image of the virtual space for the palmtop display. Four combinations can be offered for the handling phase: the view on the display is fixed on the background, fixed on the object, or two halfway between them. The features and acronyms of each method are described below and in figure 6-b,c,d,e . Table summarizes them.
Fixed View in Background (FVB)
In the FVB mode, the motion of the PDDM is directly reflected on the grabbed object as if the user is grasping it. The virtual camera is fixed when s/he grasps it. This means that the background image is fixed on the display. When the user releases the object, the direction of the PDDM contradicts the gazing vector of the virtual camera, and the user loses it from his/her view.
Semi Fixed View in an Object (SFVO)
In the SFVO mode, the motion of the PDDM is directly reflected on the grabbed object as with the FVB. When the object is grasped, not the rotated motion, but only the translated motion of the PDDM is reflected on the virtual camera. Then the grabbed object is fixed in the center of the display and its appearance is changed as the PDDM is rotated. When the object is released, the user loses it from his/her view.
Camera Centered Magic Hand (CCMH)
A magic hand is a toy for children and looks like an extended manipulator with only a link. In the CCMH mode, a metaphor of the magic hand is introduced. When a user grasps an object, a geometrical relation between the PDDM and the object is fixed as if it is being held with the magic hand attached behind the PDDM. When the PDDM is rotated, the object is rotated in the center of the PDDM. As the gazing vector of the virtual camera always follows the motion of the PDDM, the object is fixed in the center of the display and never changes its appearance. Additionally, the user never loses the object from his/her view.
Object Centered Magic Hand (OCMH)
The OCMH mode is similar to the CCMH mode, but the virtual camera is rotated in the center of the grasped object as the PDDM is rotated. Here, the motion of the PDDM is reflected on the motion of the object in the same way as the first two. Then the object is fixed in the center of the display and never changes its appearance. When a user releases the object, s/he loses it from view for a while. When the same operation is applied to an object with these methods, the results of its motion are the same in the FVO, SFVO and OCMH methods. In the following section, an experiment to compare these methods is presented.
Figure 6-a Initial situation (a) shows an initial layout of the virtual world and an image on the palmtop display. (b)-(e) show results of the virtual world after same operations are applied and image on the palmtop display. In this case, a cube is manipulated and a palmtop display is shifted to right and rotated clockwise. Each image is taken by the virtual camera that is moved as a motion of the palmtop display.
Figure 6-b Fixed View in Background: FVB
Figure 6-c Semi-Fixed View in the Object: SFVO
Figure 6-d Camera Centered Magic Hand: CCMH
Figure 6-e Object Centered Magic Hand: OCMH
Table 1 Four proposed methods in the handling phase

Experiment

The main purpose of the experiment was to clarify how these methods affect the performance and to find a standard method for future models. The subjects were requested to put a cube in the marker on the left side corner as correctly and fast as possible. The distance between the target and the initial position of the cubes was 50 cm, but they appeared in different orientation.
Every subject tried each method 16 times in a training phase, at first. In the test phase, they tried each method 8 times. In each trial, the position and the orientation of the cube and how many times they grasped and released the cubes were recorded in every 3 sec. and in the end of trial. The completion of each trial was reported by each subject.
The subjects for this experiment were six volunteers from our staff. They are all computer engineers. The number of total trials was 6 (subjects) x 4 (methods) x 8 (trials) = 192 times. They were asked to fill in an interview sheet about the usability after all trials.

Results and Discussion

To limit the range of the force display, a large operation consisted of repetitive smaller operations in the same way as operation with a mouse. In this case, the number of grasps and releases means the number of these unit operations. The task completion time divided by this number means the passing time per unit operation. Using an analysis of the variance of test phase data, it was found that all subjects could do the unit operation within almost the same time for every method. Therefore, it was expected that they became sufficiently skillful in all methods within the test phase.
Figure 7 presents the four manipulating methods versus mean task completion time in the test phase. Some subjects spent too much time to place the cubes accurately. So each trial was terminated when the positional error between the cube and target was reduced to the level the accuracy defined from the results from each subject. The error bar shows the standard error of the results. This shows that all subjects needed much time in the CCMH method. On the other hand, the OCMH and the SFVO require shorter time than the others, and there is no significant difference between them (P(t>1.31)=0.19).
Figure 7 Mean completion time of four methods
The subjective ratings from the interview sheets are shown in Figure 8. In the interview sheet, each subject was asked to decide scores from 0 to 10 for three questions: "ease of observing the result"," ease of use to manipulate" and "intuitiveness of use."
Figure 8 Subjective rating of the each method
The CCMH method introduced the metaphor of the magic hand and is equal to a manipulator with a monitoring camera at the end effector. As it is difficult to realize other methods in the real world, we assumed that the CCMH method could provide the most intuitive interface. However, the CCMH's score in the interview was worst also. It is supposed that this is because input motion to the display is magnified and then reflected on the object. This means that the user can move an object from one point to another easier then with the others, but it is difficult to control the orientation of the object with the CCMH method. In this task, the subjects were asked to arrange both the position and orientation of the cubes, so it is guessed the CCMH method is not suit to such type task.
As for the other methods, the OCMH and SFVO methods got slightly better scores than the other methods. Especially in the OCMH method, some subjects reported that they felt as if they manipulated not the object but the background and they gave the OCMH method a full score for the question of "ease of use." This means that they used the PDDM according to the frame of reference based on the world in the observing phase, but that they could handle not the object but the world in the frame based on the user in the handling phase. This suggests that they were not aware of the shift in the frame of reference during in the repetitive operation. Since the methods are designed to afford the same result when the user applies the same operation, we believed that a difference in the motion of the virtual camera might cause such a result. In future research concerning the PDDM, we should clarify the relation among the frame of the world, the PDDM, the virtual camera, and the user operating with the PDDM.

HAPTIC REPRESENTATION WITH THE PDDM

 In this section, the implementation of haptic feedback with the PDDM and an evaluation are described. As the gratest advantage of the PDDM, it provides reaction force in a VR space. A primitive operation for manipulating objects is to catch and place it next to another one, so we used a virtual flat wall at the beginning. As a most object consists of many pieces of small planes (polygons), this method can also be extended to such more complex objects.

Simulation of dynamics of the virtual wall

Figure 9 shows a diagram of a collision between a wall and a handled object. P(t) represents the position of the nearest vertex of the object to the wall and 't' is the cycle of the simulation. When a part of the object collides with the wall, the system generates a reaction force using a simulation of a virtual spring and a virtual damper. In the trial PDDM, the depth, 'Depth(t)', and the velocity, 'Vwall(t)', of the nearest vertex are used to determine the feedback mode.
Figure 9 Simulation of dynamic of a wall
Figure 10 shows a block diagram of the haptic simulation. The force display can generate passive force and active force for the user, and the passive force is larger than the active force. The system switches these modes according to the situation.
Figure 10 Block diagram of the simulation of a virtual wall
In the case of the first impact, as the normal velocity is sufficiently higher than the threshold (LIMIT_V), the impact of the collision is reduced by passive feedback, with the USMs acting as a brake. In another case, i.e., rubbing the surface of the wall, as the normal velocity is smaller than the threshold, the spring has lager effect than the damper in the dynamic simulation, and the active force mode operates. Controlling the coefficients of the spring and the damper, ks and Kd, the property of the wall can be changed from soft to semi-hard.

Experiment

Evaluating the proposed method for haptic representations with the PDDM, a "catch and placing" task was conducted. The top view of the experimental setup is illustrated in Figure 11. The setup indicates a cube that the subject manipulates, a floor board and a wall. Each subject was asked to catch the cube and place it against the wall. The manipulating method of the PDDM is the OCMH. When the subject places the grabbed cube onto the wall, s/he gets the reaction force in the method stated above.
Figure 11 Experimental setup: Catch and place
The initial position of each cube was constant ( z=10cm,) and the position of the wall was selected randomly from three conditions:( Z=25, 45 and 65 cm.) The subjects sat down 50 cm away from the initial position of the cube. It is difficult for the traditional glove-like device user to manipulate the cube around to the farthest wall without changing the standing point in some way.
Each subject was asked to catch the cube and place it as fast and accurately as s/he could. To evaluate the effect of the haptic feedback, we held the two mode: "with visual feedback" and "with haptic and visual feedback". The subjects were four volunteers from ATR. They had all experienced the PDDM before. The total number of trials was 4 (subjects) x 3 (wall positions) x 2 (modes) x 10 (trials) = 240 times.

Results and Discussion

All subjects could put the cubes next to the wall. Here figure 12 shows the standard error of the cubes' position versus the mean task completion time for each condition (10 trials.)
Figure 12 Results of the task
The markers' character means the three positions of the wall and the feedback modes. Closed curves point up the modes. Generally, it is expected that both the standard error and the mean task completion time will correspond to the distance between the subject and the wall. However, from the curves' shape, it is found that the standard error with haptic and visual feedback is independent of the distance, and even in the farthest wall it is as small as in the nearest . Besides this, the mean task completion times with haptic feedback are shorter than those with visual feedback. It is supposed that the excessive time in the visual feedback mode was spent on final adjustments.
Additionally, it is worth noting that the darkened area in figure 11 shows the movable area of the PDDM, and walls B and C were located outside of this area. When the subject moved the PDDM near the edge of the movable area, they released the cube temporarily, then moved the PDDM to the center of the movable area and repeated the motion. This motion is the same as the repetitive motion mode with a mouse on a pad that is smaller than the display. This indicates that the subject can manipulate an object that is out of the movable area as well as an object that is in his/her hands. This is because the motion of the cube depends on the motion of the palmtop display.
As a suitable application for these PDDM's advantages, a large-scale simulation, such as an urban planning simulator, can be proposed. Using the PDDM, a reaction force against ground would assist an operator to arrange buildings straightly. The operator could manipulate the targets without having to move near them.

FUTURE WORKS

 We employed 6 DOF force display, but 3 of these were unactuated, as stated above. We can construct a fully actuated 6 DOF force display using three of our arms. However, such a mechanism would become complicated and the movable area would be limited. As another solution, we are planning to employ a dynamic constraint method [5] to aid the lack of DOF.
As one future work for the PDDM, we have designed ,but have not yet achieved, a PDDM system in an augmented reality [10]. As shown in figure 13, a small CCD camera is mounted on the PDDM. The image on the PDDM is a superimposed one of a real scene captured with the camera. S/he can observe virtual and real objects on the PDDM. When the user manipulates a virtual object with the PDDM and it collides with another real/virtual object, the user will be able to feel the reaction force with the PDDM.
The system premeasures the shape of real objects. By measuring their position in any ways, i.e., a visual sensor or magnetic position sensor, the simulation process can treat real or virtual objects using the same method as we have now. The z-buffer method can be applied to effectively make such a superimposed image [14]. We expect the such a system could be applied to the engineering CAD.
Figure 13 The PDDM system in an augmented reality

CONCLUSION

 We believe that the concept of the PDDM is new and unique and it offers possible applications in a wide variety of fields. In this paper, we proposed a PDDM for manipulating objects in the VSTC. First, we presented a conceptual design and then clarified its features in comparison with other devices. Offering the advantages of both a palmtop display and a direct manipulating device, the PDDM allows a user to handle a remote object as if it is within his/her own hands. Using a trial model, we conducted two experiments. In the first, we confined ourselves to four operational methods with the PDDM, and found that each of them had advantages and that the relation of the frame of the reference should be further clarified in the PDDM in future. In the second, we evaluated the combination of haptic and visual feedback. The results showed that the PDDM with haptic feedback offer grate positing accuracy in a catch and place task.

Acknowledgments

The authors wish to thank Dr. N. Terashima, President of ATR CSRL, and Dr. K. Habara, Executive Vice President of ATR International (Chairman of the Board of ATR CSRL), for their thoughtful advice and encouragement in this research. Authors also wish to thank Mr. Akaba, who took part in implementing the system.

Reference
  1. Amselem, D., "A Window on Shared Virtual Environments", Presence, Vol. 4, No. 2. : pp.130-145. 1995
  2. Arthur, K. W., Booth, K. S., and Ware, C., "Evaluating 3D Task Performance for Fish Tank Virtual Worlds", ACM Transaction on Information System, Vol. 11, No. 3 : pp.239-265. 1993
  3. Brooks, F. P., Ough-Young, M., and J. Batter, J., "Project GROPE-Haptic Display for Scientific Visualization", ACM Computer Graphics, Vol. 24, No. 4 : pp.177-185. 1990
  4. Fitzmaurice, G. W., Zhai, S., and Chignell, M.H., "Virtual Reality Palmtop Computers", ACM Transactions on Information Systems, Vol. 11, No. 3 : pp.197-218, 1993
  5. Kitamura, Y., Miyasato, T., and Kishino, F. "A Sophisticated Manipulation Aid in a Virtual Environment Based on the Dynamic Constraints among Object Faces", proc. of IEEE International Conference on Systems, Man and Cybernetics. Vancouver, Canada: 1995.
  6. Kuzuoka, H., Kosuge, T., and Tanaka, M. "GestureCam: A Video Communication System for Sympathetic Remote Collaboration" , proc. of CSCW' 94. : pp.35-43. 1994.
  7. Noma, H., et al., "Multi-Point Virtual Space Teleconferencing System", IEICE Trans. on Com., Vol. E78-B, No.7 : pp.970-979. 1995.
  8. Noma, H. and Iwata, H. "Presentation of multiple dimensional data by 6DOF force display", proc. of IEEE/RSJ Intelligent Robots and Systems, Yokohama, Japan: pp.1495-1500. 1993.
  9. McKenna, M. "Interactive Viewpoint Control and Three Dimensional Operation" , proc. of ACM Symposium on Interactive 3D Graphics. pp.53-56. 1992.
  10. Milgram, P. and Kishino, F., "A Taxonomy of Mixed Reality Visual Displays", IEICE Trans. INF. & SYST., Vol. E77-D, No.12 : pp.1321-1329. 1994
  11. Rekimoto, J. "Augmented Interaction: Interacting with the real world through a computer", proc. of HCI'95., Yokohama, Japan : pp.225-264. 1995.
  12. Shimojo, S. and Nakayama, K."Real World Occlusion Constraints and Binocular Rivalry", Vision Res. , Vol. 30, No. 1, pp.69-80. 1990
  13. Takemura, H., and Kishino, F. "Cooperative Work Environment Using Visual Workspace", proc. of CSCW '92 : pp.226-232. 1992.
  14. Wloka, M. M. and Anderson, B.G. "Resolving Occlusion in Augmented Reality", proc. of ACM 1995 Symposium on Interactive 3D Graphics., Monterey CA USA: pp.5-12. 1995.