Designing the PenPal: Blending Hardware and Software in a User-Interface for Children

Philippe P. Piernot⁺ (1) , Ramon M. Felciano (2), Roby Stancel, Jonathan Marsh and Marc Yvon

(1) Knowledge Systems Laboratory
Stanford University
701 Welch Road, Bldg. C
Palo Alto, CA 94306
Tel: 1-415-323-2531
piernot@ksl.stanford.edu

(2) Section on Medical Informatics
Stanford University School of Medicine
MSOB X-215
Stanford, CA 94305-5479
Tel: 1-415-725-3398
felciano@camis.stanford.edu

⁺visiting from LIAP-5, Université René Descartes, Paris, France.

Abstract

As part of the 1994 Apple Interface Design Competition, we designed and prototyped the PenPal, a portable communications device for children aged four to six. The PenPal enables children to learn by creating images and sending them across the Internet to a real audience of friends, classmates, and teachers. A built-in camera and microphone allow children to take pictures and add sounds or voice annotations. The pictures can be modified by plugging in different tools and sent through the Internet using the PenPal Dock. The limited symbolic reasoning and planning abilities, short attention span, and pre- literacy of children in this age range were taken into account in the PenPal design. The central design philosophy and main contribution of the project was to create a single interface based on continuity of action between hardware and software elements. The physical interface flows smoothly into the software interface, with a fuzzy boundary between the two. We discuss the design process and usability tests that went into designing the PenPal, and the insights that we gained from the project.

Keywords

Hardware and software integration, user-centered design for children, Internet and multimedia application, educational application, portable computing.

Introduction

In January 1994, Apple Computer organized the Third Interface Design Competition. The theme was to design a portable device that uses the Internet to foster learning, using technology that would be commonly available within 2-3 years [1]. Competing teams had to submit a documented interface design, including prototypes of the main aspects of the interface and video footage of user studies. Eleven schools from around the world were invited to participate in the competition. At Stanford University, the competition was integrated into an interdisciplinary course on Human-Computer Interaction and Product Design taught by Terry Winograd, David Kelly, and Brad Hartfield. The authors designed the PenPal (Figure 1), a visual communications device to introduce children between the ages of four and six to creating digital media and exchanging it across the Internet. The PenPal project won an award for "Best Hardware/Software Integration and User Involvement" and an award for "Best Presentation".

FIGURE 1. The PenPal.

We decided to design for children of kindergarten and preschool age because their special needs and interests are not addressed in today's typical software and hardware solutions. In particular, we wanted to let children interact with other children at distant locations without requiring adult assistance. The communication capabilities of computers could help children contact and collaborate with other children around the world. To address the special needs of this user group, we developed two central principles that are present throughout the design. These principles were to (a) use physical interactions to take advantage of childrens' affinity for tactile manipulations, and (b) blur the boundaries between hardware and software to provide a single fluid interface.

The PenPal meets the thematic requirements of the competition because:

The PenPal is a communication tool. The Internet is already being used by older children to communicate. For example, KidLink is an Internet service that offers to connect children with e-mail correspondents [8]. The PenPal expands the richness of conventional electronic communications by focusing on the use of pictures and sounds, thus lowering the age and skill level required for participation.
The PenPal is a learning tool. At the young end of our user group, learning comes from the process of creating and sending a message--the process provides the educational value [6]. For older children, plug-in PenPal tools can help them learn to read, to tell time and dates, to plan ahead, to speak a new language, or to budget their allowance.
The PenPal is portable. The PenPal can be carried by hand or on a shoulder strap, so that learning can happen alone, in a small group, or with the whole class--wherever the child happens to be.

We designed all aspects of the PenPal user-interface, but due to temporal and financial constraints (10 weeks from initial design to final presentation, with a limited budget), we were only able to fully prototype the main elements of the interface and we didn't do a formal evaluation. In particular, we prototyped and tested the seamless integration of hardware and software, which we believe is an important contribution of this project. We first present our user group and our design principles, then we describe the design process, usability tests, and details of our solution, and we conclude with a discussion of the insights gained.

USER GROUP

Our goal was to combine three activities children enjoy: creating, communicating and learning. Our approach was framed by some unique design constraints imposed by our chosen user group of four to six year old children:

Children may be unable to read. Literacy is not a requirement for using the PenPal, but text remains present in the user interface to help children as they learn to read. More directed learning could come in the form of PenPal tools that teach children to read.
Children have limited symbolic reasoning, so we decided not rely on the ability to process the graphical abstractions present in many current user interfaces. The PenPal doesn't use a desktop metaphor with icons to represent applications and files. Instead, interactions are tied to physical actions: manually plugging a tool into the PenPal launches the software.
Children have limited planning capabilities and may have trouble carrying out tasks that require several steps. The PenPal uses auditory and visual coaching to help users carry out sequences of operations, and provide positive feedback for their actions.
Children have a shorter attention span than adults. The PenPal provides an immediacy of results (e.g., as soon as one takes a picture, it is placed on the screen and ready to manipulate), as well as an entertaining interface to keep the user's attention.

Clearly, these constraints present a worst case scenario, since some of our users could read or had previous experience using computers. Indeed, the challenge presented by these constraints was compounded by the fact that children of this age have widely differing real world knowledge, experiences, and abilities. We decided to design within these confines so that the interface would not be too complicated for our youngest users.

DESIGN PRINCIPLES

To address these issues, we developed two principles, inspired by the nature of our user group, that guided our design:

The PenPal features a Physically Manipulable Interface. Many toys for children of this age contain physical elements for children to operate (e.g., knobs, buttons, levers). We decided the PenPal should have a similarly tactile interface to take advantage of this natural interest in touching and manipulating such objects. For example, users switch into drawing mode by plugging in the PaintSet tool, not clicking on an icon.
Because functionality is tied to physical actions and objects, the features of the PenPal are always visible and available to the user. The focus on physical interactions allows us to use direct manipulation to the greatest extent possible in order to get what we call WYTIWYG ("What You Touch Is What You Get"). We believe physical modes and actions afford more natural mappings for our users, and therefore lead to a more intuitive user-interface.
We wanted the PenPal to have a single interface that consisted of both the physical interactions and the software ones, with a purposefully fuzzy boundary between the two. For example, when one slides a MessageCard into a slot on the PenPal, a life-sized picture of the card slides onto the screen next to the slot: the card is both a physical and a software interface element. Every physical interaction with the PenPal produces similar feedback, which creates a spatial and temporal continuity of action between the physical interface and the software interface.
Images on the screen are not iconic or abstract in character, but instead are realistic representations of real-world objects. By tightly coupling on-screen images and interactions with physical actions, we hope to reduce the semantic and articulatory distance required for the child to understand the meaning and use of the image [7].

In addition to these basic design principles, we had some overall design goals in mind:

The PenPal should be personal and fun to use. Children should perceive the PenPal as being built exclusively for them, rather than a computer for adults that has been retrofitted for younger users. We provide a number of ways to customize the hardware-software interface: a personal address book with pictures of other users, a custom software startup sequence that runs when the PenPal is turned on, and exchangeable hardware screen frames with different colors and visual themes.
The PenPal should be expandable. Although we wanted to keep the interface simple, we wanted to take advantage of the computer's ability to perform many different tasks. The PenPal must be able to grow with the children, expanding functionality as they learn to do more with the tool.

DESIGN PROCESS

The Stanford course focused on teaching user-centered design. The first phase involved brainstorming and exchanging ideas with the goal of forming teams and elaborating product concepts. The course faculty emphasized the unique advantages of interdisciplinary design [9]; our five-person team has members from France, Germany, and the United States, with backgrounds in Computer Science, Product Design, Art and Engineering.

The rest of the course was broken down into user observation and scenario generation, hardware and software prototyping, and usability testing. Oral design critiques were performed every two weeks. The user-centered design cycle iterated through several prototypes and usability tests (12 boys and girls between the ages of two and a half and six years old) [5]. The process was supported by market observations, expert interviews (parents, teachers, education experts), user interviews, and visits to schools and toy stores. Because we were unable to build a fully integrated prototype, each test involved a hardware and a software phase.

We built two series of prototypes in parallel. The first series of prototypes explored the appearance, ergonomics and function of the physical unit. We generated several life-sized prototypes using materials from clay and cardboard, to laser-cut acrylic (Figure 2), and conducted usability tests to determine which form-factors worked best for our users.

FIGURE 2. Previous hardware prototypes.

The second series of prototypes helped us develop the interactive aspects of the PenPal. In order to understand the important issues in merging physical and screen interfaces, we conducted a Wizard of Oz experiment with software prototypes developed in Macromedia Director. To model the physical interactions, we taped a touch-screen onto a computer monitor and taped a large-scale cardboard mock-up of the PenPal on top of the touch-screen (Figure 3) [3]. The cardboard mock-up included the main hardware features of the PenPal: two tool slots, and a coaching button at the top of the screen. The software was controlled through a combination of behind-the-scenes manipulation by a project member and of direct user interaction with the touch-screen.

FIGURE 3. Setting for the software user tests.

PHYSICAL DESIGN

In recognition that a "user interface is more than software" [10], we discuss the design of the physical product as well as its software elements.

Form Factor

We determined the initial shape of the PenPal by asking one of our users to take modeling clay and "wrap" a small pad of paper. The pad was wrapped irregularly and in several colors of clay, indicating that the PenPal didn't necessarily need a conventional rectangular viewing area. We used this fact to create fun and colorful screen frames that occasionally overlap the screen.

Our final hardware design features a landscape- oriented color liquid-crystal display (LCD) touch-screen, about 3 inches by 5 inches in size. The screen is embedded in a slim frame with two barrel-shaped grips. On either side of the screen are slots for plug-in tools or credit-card-sized MessageCards (Figure 4). A camera is embedded at the top of the left grip, and a microphone and speaker in at the top of the right grip. A single button is positioned at the top of the screen, and brings up the PenPal's prompting and coaching feature. The back of the unit holds the battery storage area, clips for a shoulder strap, and various contacts for connection to the Dock.

FIGURE 4. PenPal, MessageCard and PaintSet tool.

When lying on a flat surface, the PenPal is tapered towards the user so that the screen slopes slightly upward to face the user for more ergonomic usage. We added this taper after users repeatedly placed one of our clay prototypes in a portrait orientation rather than the intended landscape orientation, because the left side of the prototype was higher than the right side. Users placed the prototype sloping toward them rather than toward the right, even though this orientation made controls and tools asymmetrical. This challenged our assumption about the best orientation for the PenPal; we did not have enough time to perform additional tests to determine which orientation our users preferred.

Data Storage

A major design challenge was to find a way for children to store pictures and sounds, and exchange them with other users. We developed several concepts such as "data marbles" that would contain a single image, or "memory coins" that plugged into the grips of the PenPal.

FIGURE 5. Memory coin.

However, feedback from teachers, parents, and other students in the course led us to believe that users would have difficulty thinking about coins or marbles as items that could be sent to someone, so we settled for the concept of a postcard. In the final design, camera images or drawings are stored on MessageCards, credit-card-sized cards that have six LCD fields on one side and an addressing area on the other (Figure 6). The LCD fields are magnetized to indicate how full the card is.

FIGURE 6. MessageCards.

The PenPal Dock

To provide an inexpensive connection to the Internet, we developed a docking station rather than a wireless connection in the PenPal itself (Figure 7). The PenPal Dock has a build-in modem, a MessageCard slot for sending and receiving messages, and a flashing light for indicating when a new message has arrived. The dock sits on the wall or on a table, and also serves as a storage rack and recharging station for the PenPal.

FIGURE 7. The PenPal Dock.

Customization

Both teachers and parents emphasized the importance of including personal elements in the PenPal. This is achieved by customizing their address book with friends' addresses, adding important names and dates, or choosing from a variety of PenPal screen frames that have different visual themes such as the interior of a car or a tropical jungle (Figure 8).

The screen frames occasionally intrude upon the screen area. For example, the Jungle screen frame has a few trees and bushes that overlap from the frame onto the screen area; this again blurs the hardware-software separation. Although we did not prototype this idea, we envision that screen frames could have software elements as well: the jungle scene, for example, could change the animated coaching character to a monkey or parrot.

FIGURE 8. Screen frames.

INTERACTION DESIGN

We now turn to the design of the software elements in the PenPal, concentrating on how they relate and blend with the hardware features.

Data Storage

MessageCards contain six fixed-size images. We chose fixed-size images to avoid the need for scrolling, and a fixed number of images per card to simplify browsing. To verify that users could create on a canvas the size of the PenPal screen, we stuck sheets of paper onto our hardware prototypes and had our users draw on them.

Inserting a MessageCard into a PenPal slot slides an on-screen image of the card into view next to the slot. This virtual MessageCard shows a reduced-size version ("thumbnail") of each image on the card; tapping on one of these images zooms it to full size and plays any sounds associated with the image (Figure 9). Because of the speed limitations of our testing setup (touch-screen and Director prototype), users often tapped repeatedly until the image zoomed to full size. We are convinced that with a better implementation, the system will react quickly so that users will understand that a single tap is sufficient.

FIGURE 9. Browsing the contents of a MessageCard.

Addressing

Once we chose MessageCards as the data storage mechanism, we realized we could use the address side of the card to embody both the hardware and software aspects of addressing the card. The back of a MessageCard looks like the address side of a postcard with an address area and a stamp; all but two users recognized the visual similarity between MessageCards and postcards after we added a stamp. If a MessageCard is inserted into the PenPal with the address side facing up, a personal Address Book with faces of other correspondents appears on the screen (Figure 10). Touching one of the faces moves it to the address area of the on-screen card, and a spoken voice reinforces the action ("This card is addressed to Pierre"). To send an addressed MessageCard across the Internet, the user inserts it into the slot on the PenPal Dock the same way one would drop a real postcard into a mailbox.

FIGURE 10. Addressing a MessageCard.

In our initial design, touching a face moved it directly to the addressing area. Users commented that it behaved differently than the data side of the MessageCard, where touching on images zoomed them to full size. In the final design, touching a face in the Address Book zooms it to full size, then zooms it back down into the addressing area. Once we added this animation, users seemed to feel more comfortable with the interface: they recognized pictures of their friends, and understood that they could now send the MessageCard to these friends.

Tool Design

We tested a number of different methods for plugging in tool modules, including extensions to the grips and various cabled plug-ins. Most of these ideas were discarded because they appeared to make the PenPal awkward to hold and use. We decided to use the same slots for the tools as for the MessageCards: all plug-in / pull-out interactions consistently occur in conjunction with one of these two slots that are clearly visible from the top of the unit.

While children can use drawing programs like KidPix [2], desktop computer operating systems are not designed for children. In the PenPal, the physical aspects of a plug-in tool govern its usage. For example, plugging in the PaintSet tool (Figure 11) switches the software into paint mode: a voice confirms the new functionality (e.g., "This is the PaintSet! Touch the screen to draw on it!")."Dipping a finger" into one of the colored patches on the PaintSet switches colors, while the PenPal pronounces the name of the new color. Users can draw with the PenPal as if finger-painting.

FIGURE 11. PaintSet and MessageCards.

Tool Slots

Early tests showed that users had difficulty finding the slots. Because the slots were located on the side of the prototypes and not easily seen from above, users tried to place the cards directly onto the screen. We made the slots clearly visible from the top, and added distinctive color differences on the handles (Figure 12).

FIGURE 12. PenPal tool slot.

We ran into some difficulties when we tried prototyping different MessageCard sizes and PenPal slot designs at the same time. As a result, some cards simply would not fit into the slots. Users might have noticed the slots but not tried to insert the cards because the card they happened to be holding was clearly too large to fit into that prototype. Once we settled on a fixed size card and slot, users were able to perform the action repeatedly without trouble.

Camera and Microphone

The built-in camera is activated by sliding back a lever thus revealing the snapshot button on the front, and uncovering the camera lens on the back (Figure 13). The gesture is similar to winding up a photo camera, and our young users enjoyed playing with the lever.

FIGURE 13. Camera lever.

As soon as the lever is turned, the display becomes a live viewfinder, revealing what the camera lens is pointing at. Pressing the shutter button takes a picture and freezes it on the screen. This is similar to taking a conventional photograph, but the result is immediately available for the user to work with (e.g. plug in the paint tool and start drawing on the picture).

For self-portraits, the camera lever can be rotated 180deg. so that the lens faces the user; in this case, pressing the snapshot button starts a 10-second, audible countdown instead of taking the picture right away. A similar mechanism controls the microphone on the right side: after the lever is pushed to start recording, it slowly rotates back into place like a kitchen timer to indicate the amount of recording time left.

An earlier design placed the camera on a pivoting arm; lowering the arm would activate the camera. We discarded this idea after users pointed out that it seemed quite fragile. However, their reactions were probably biased by the fragility of our hardware prototype: several users accidentally pulled the arm off the unit while trying to rotate it, after which they handled all interactions with the unit with an exaggerated delicacy. Our final prototype was much sturdier and the camera lever fit more snugly into the unit, making it harder to remove accidentally.

Coaching and Help

Educators we spoke with emphasized the importance of auditory feedback and reinforcement during a child's learning period. We added auditory and animated coaching to the PenPal to reinforce what users were doing and to guide them. Usability tests showed that children occasionally needed prompting to suggest what options were available to them. A "help me" button above the screen activates an animated character that gives users options about what they might like to do next (Figure 14). Whenever possible, the animations attempt to reinforce the fuzzy border between hardware and software by making the character aware of surrounding hardware such as slots and levers. For example, in our usability test, the animated character suggests that the user plug in a PaintSet tool: the character drops down from under the "help me" button that the user just pressed, walks across the screen and points to the physical slot (located next to the right edge of the screen) as she talks.

FIGURE 14. Animated character.

INSIGHTS

Constrained Design Space

We ascribe our accomplishments primarily to the course requirements, and the constrained design space imposed by both our user group and the prerequisites of the competition. The course curriculum focused on a by-the-book, user-centered design methodology, and forced us to explicitly articulate our design for bi-weekly presentations to a group of peers and faculty for criticism. Selecting a well-focused user group permitted an in- depth understanding of users' needs, abilities, and daily lives. As the computing industry moves towards special purpose devices that perform a narrower range of tasks than the typical desktop computer, designing for such narrowly-defined user groups will be an increasingly common goal.

Designing for Children

Usability testing. Performing usability studies with children required tailoring conventional testing methods [4]. Adult users are often asked to evaluate features by rating them according to a scale. Some of our users found this difficult, so we experimented with simple, directed questions or task statements that attempted to extract useful feedback without influencing users' responses. For example, when testing our early designs for the tool slots, we found that questions that required even simple hypothetical thinking on the part of the user were less useful than simple task assignements. Thus, we told users to "Put this card into the PenPal" instead of asking them "Where would you insert this card in the PenPal?" The most useful tests were brief and straightforward, and involved intriguing tasks to keep users interested.

Children had difficulty with explicit "thinking aloud" techniques, and we sometimes struggled to understand the verbal feedback they gave during post-test interviews, due either to their inability to articulate their thoughts, or our inability to understand them. They were also more likely to verbalize positive reactions than negative ones, and the tendency to seek approval from the tester that is often seen in adult usability tests appeared exaggerated in our young users. Therefore, we relied more on user observation than on interviews to find flaws in our design; often the most valuable information came from interpreting users' facial expressions and spontaneous reactions. We found it vital to have at least two cameras running at all times: one videotaping the user's face, the other taping the interaction with the user interface, typically over the user's shoulder.

Customization. The customization and "personalization" of the PenPal turned out to be a very important factor in making it appealing to our user group. Our users loved the animated start-up sequence, the address book filled with familiar faces of friends, and the colorful screen frames. The success of colorful mouse pads and animated screen-saver software for desktop computers would suggest that this kind of customization appeals to adults as well, yet it is rarely integrated into the basic design of the computer.

Animation at the Interface

Developing a design featuring hardware and software working in fluid unison was one of the major challenges in this project. The use of sound and animation proved to be a valuable tool in developing our solution. When a MessageCard is inserted on the right side of the PenPal, the user moves the card from the right to the left as it slides into the slot. The on-screen image of the card mimics this dynamic motion so that it appears at the same location and moves in the same fashion as the physical card. Through the integrity of this illusion, the PenPal serves as a magic magnifier to allow users see what is on their MessageCards.

While other interface systems use graphical feedback about physical actions (e.g. a floppy disk icon appearing on the Macintosh desktop after a disk is inserted into the drive), the PenPal provides a "continous representation of the object of interest", which requires less cognitive interpretation on the part of the user [7]. The spatial and temporal continuity of this parallelism results in physical modes and actions that appeared to be more natural for our users than software-only manipulations.

Acknowledgments

We thank Terry Winograd, David Kelley, Brad Hartfield, Harry Saddler, Deanna Marsh, Jean-Michel Moreau and Angel Puerta as well as our young users for their contributions. Special thanks to Joy Mountford and Greg Thomas for organizing the Apple Design Competition, and to Vicky O'Day for providing valuable assistance in performing our usability tests. Certain computing services were provided by Apple Computer, Inc. and the CAMIS resource at Stanford, NIH grant No. LM-05305.

References

1. Apple Computer, design competition brief, 1994:
<http://www- pcd.stanford.edu/hci/apple.txt>

2. Brøderbund Software, KidPix User Manual, 1991.

3. Ehn, P. and Kyng, M. Cardboard Computers: Mocking-it-up or Hands-on the Future, in Greenbaum, J. and Kyng, M., eds., Design at Work: Cooperative Design of Computer Systems. Lawrence Erlbaum Associates, Hillsdale, New Jersey, 1991, pp. 169-195.

4. Gomoll, K. Some Techniques for Observing Users, in Laurel, B., ed., The Art of Human-Computer Interface Design. Addison-Wesley Publishing Company, Inc., Reading, Massachusetts, 1990, pp. 85-90.

5. Gould, J.D. and Lewis, C. Designing for Usability: Key Principles and What Designers Think. Communications of the ACM. 28, 3 (March 1985), pp. 300-311.

6. Hakansson, J. Lessons Learned from Kids: One Developer's Point of View, in Laurel, B., ed., The Art of Human-Computer Interface Design. Addison-Wesley Publishing Company, Inc., Reading, Massachusetts, 1990, pp. 123-130.

7. Hutchins, E., Hollan, J. and Norman, D. Direct Manipulation Interfaces, in D. Norman and S. Draper, eds., User Centered System Design. Lawrence Erlbaum Associates, Hillsdale, New Jersey, 1986, pp. 88-124.

8. KidLink Home Page:
<http://kidlink.ccit.duq.edu:70/0/kidlink-general.html>

9. Kim, S. Interdisciplinary Cooperation, in Laurel, B., ed., The Art of Human-Computer Interface Design. Addison-Wesley Publishing Company, Inc., Reading, Massachusetts, 1990, pp. 31-44.

10. Vertelney, L. and Booker, S. Designing the Whole-Product User Interface, in Laurel, B., ed., The Art of Human-Computer Interface Design. Addison-Wesley Publishing Company, Inc., Reading, Massachusetts, 1990, pp. 57-63.

⁺visiting from LIAP-5, Université René Descartes, Paris, France.