|
|
The conventional HCI-based, 'user-centred' approach has been typically to position users as a testing or evaluation service for designers to ensure those users' needs are met [cf. 11]. By placing users in this reacting role, designers can obtain a range of feedback as to what is good, bad and ugly about their designs. However, such a set-up means that the kind of feedback obtained from users is exclusively based on reaction rather than initiation [cf. 7]. A further problem with this kind of asymmetrical relationship is that the onus is entirely on the designers to take on board and translate the users reactions. Many obstacles can prevent this from happening in a beneficial way, ranging from the designers' own reluctance to reconsider and possibly throw away their own design ideas to organisational constraints that demand the product be shipped before any redesign can be put into practice. All too often the actual contribution made by users to the redesign of a system/interface is "too little, too late".
The more recently popularised, alternative 'participatory design' (PD) approach is to respect users more as partners in the design process and in doing so explicitly give them a more equal and responsible role. In this way users can jointly work together with the designers to develop a system to fit their needs. This kind of relationship is exemplified by the Scandinavian Approach to system design [cf 2] and more recently, by Participatory Design [13]. Here "the goal is to provide an equal opportunity design environment in which all participants can contribute as peer co-designers" [7].
Such approaches have proven to be effective for adult users, where it is possible for users and designers to view each other as peers. But what happens when the users are children and the domain is something that they know little about but which the software is specifically intended to teach them about? In what respects is it desirable, useful and sensible to set up design teams whereby children are given equal responsibilities to those of adult designers in such contexts? Moreover, is it possible for adults to treat children like themselves whilst at the same time not being patronising? Conversely, can children make contributions about the content and the way they should be taught - something which adults have always been responsible for?
The creative involvement of children early in the process of design has increased considerably in recent years. For example, Cypher and Smith not only used fifth graders in informal tests of their own examples of the KidSim rule set but also of rules written by other children [3]. Druin has long demonstrated the considerable value of inclusion of children on design teams, arguing strongly for their competence in giving opinions and suggesting metaphors for the designer to work with [e.g. 4]. Similarly, Oosterholt, et al, describe the process of co-design of a communication device with children, pointing to the need to "enter their world", as well as to recognise the sophistication of the current generation. In the latter case the children were involved throughout the development process [9]. There is, therefore, a growing body of evidence that shows that children can and should be involved more in the design process. However it is not clear what status they should have in their relationship with designers. Should we view them in a participatory design context, where they are given the role of partners [4] or in a more user-centred context, where their level of involvement is more circumscribed by us?
Clearly, there is much to be gained from adopting a child-centred framework that allows for different kinds of inputs from children. In particular, they can have insight into what learning methods are effective and what motivates them, based on their experiences. Children are very good at letting us know what is it that keeps them engaged, which is often not what adult designers or their proxies (e.g. market research) would have expected. We also need to recognise, however, that children cannot design their own learning goals. Here input from teachers, psychologists and educational technologists can play a valuable role. But, again, we need to appreciate what each has to offer and not be tempted simply to use them as token guiding voices. For example, teachers are very good at informing us about what children find difficult to learn using traditional materials but generally do not have much to offer in terms of how to put interactive multimedia to good use. Similarly, educational technologists can let us know what kinds of learning metaphors might be effectively utilised in the design of interactive learning environments but not know how best to transform them into interactive media.
Another aspect of collaborative design is that it typically emphasises the use of 'low-tech' and 'lightweight' communicative and creative tools. Here, the emphasis is very much on co-making mock-ups and co-constructing rapid prototypes, as opposed to one group making them and another commenting on them. For example, one of the most well known methods, called PICTIVE, provides common office objects (such as Post-It notes, scissors, sticky tape and coloured pens) with which the users and designers can together model and design putative user interface objects for a given application. The success of such approaches has been documented elsewhere [8]. Use of such materials and methods seems particularly well-suited for children, who spend much of their time constructing. For example Druin and Solomon [4] describe a number of different ways of using such prototyping techniques. Typically low-tech prototypes feature early in the design process, before any coding begins. Our approach is to blend different low-tech and hi-tech methods that work in parallel whilst also informing each other. In this respect, the controversy over the merits of lo-fi prototyping versus hi-fi prototyping becomes a non issue [14]. Instead, the important point is to know how to use different methods, how they can inform and build on each other, and by doing so be clear about what it is that we are trying to prototype with different kinds of prototypes [6].
Where children are involved we need to consider how best to use low-tech materials to bring out their contributions. In particular, it is important to be clear what sort of data we might elicit from them. It is frequently noted that (adult) users have difficulty in articulating the ways in which a system might help them. However, children are likely to have difficulties, by definition, with articulating what needs the interactive learning environment should be meeting - since they do not know how to express concepts that they have not yet grasped. We think it useful, therefore, to try and be clear exactly what kinds of benefits accrue from the application of different methods. Likewise, we think it useful to be quite specific about what kinds of suggestions (their level and scope) a designer might expect from working with children, teachers, educational psychologists and others.
The table shows how each informant provides different inputs to the design at various phases of the project using different methods. Each phase is only briefly described in the table so as to give an overview of the value of different methods and contributors. It should also be pointed out that each phase is not necessarily sequential, but may overlap and run in parallel. More detail will be provided in the next section where we explain our particular project, using this framework.
In the first phase of design the emphasis is necessarily on specifying appropriate learning goals and identifying the pros and cons of current teaching practices within the domain (here biology teaching for 7 to 11 year-olds). At this stage children and teachers are involved separately as informants, since they have quite different perspectives on what problems are most salient for teaching and learning. These data enabled the design team to itemise existing problems of teaching and learning a difficult topic. The output from this is a list of problems that are then turned into high level functionality requirements for multimedia implementation in the second phase. This is done through cognitive and interactivity analysis [see 10, 12]. These specifications are then used in phase 3 for designing the low-tech materials which are used by the child informants to come up with designs and suggestions as to what is motivating for them. Input from these sessions is then used for evolving the hi-tech prototypes, (which were begun in Phase 1 - owing to the amount of time required to build them). These are subsequently tested by the child informants and teachers and their design further iterated. Finally we would envisage a phase not shown in the table - the testing of prototypes in a classroom context, as part of a lesson plan.
| Phase of Design | Informant /Design Team Contributor | Input | Methods |
| Phase 1 - Define Domain & Problems | Teachers | Specify learning goals; Identify teaching practices /difficulties; Compare conventional &;multimedia materials | Teacher interviews Curriculum requirements Teacher Panel |
| Children | Explain difficulties with learning particular topics for identified goals | Talk with pairs of children in school context with existing materials | |
| Psychologists | Analyse learning goals | Cognitive-Devel opmental analysis | |
| Design team (all) | Explore and define scope of interactivity | Theoretical analysis of external representations | |
| Software /graphic designer | Begin prototyping | Preliminary sketches and ideas for representing domain | |
| Phase 2 - Translation of specification | HCI analyst and psychologist | Target high-level functionality of multimedia implementation | Cognitive and interactivity analysis |
| Software /graphic designer and HCI analyst | Turn requirements into software specification and determine feasibility | Storyboarding, sketching, scenario creation | |
| Phase 3 - Design low-tech materials & test | Psychologist and Designer | Work to create low-tech materials | Cognitive analysis |
| Software /graphic designer | Flesh out spec. and test design assumptions | Make low-tech materials - paper cut-outs, etc. | |
| Psychologists and designer | Test validity of cognitive assumptions | Facilitate child design and evaluation | |
| Children | Provide insight on building interface and motivational factors | Design through scenarios, games, etc. | |
| Phase 4 - Design and test hi- tech materials | Software /graphic designer | Flesh out and validate design aims based on output from above phases | Prototype hi-tech designs using multimedia programming environment |
| Psychologist and HCI analyst | Test validity of cognitive and pedagogical aims | Cognitive analysis and pre-, during and post-tests | |
| Children | Evaluate interactivity and iterating designs | learning tasks | |
| Teachers | Verify whether prototypes are an improvement over existing methods | Try out the prototype, suggest how could be used in teaching contexts |
At this point we were also concerned to make some estimate of the current and potential value of computer-based teaching materials. One way in which we elicit this kind of information is via a 'teacher panel', conceived of as an analogue to a consumer panel. Here we had eight teachers of the 7-11 year-old group examine existing software for teaching ecology, both at home and in class, present individual reports on their experiences and meet, under the direction of the psychologists, for a focus group discussion on what kinds of topics are problematic to teach, comparing traditional and computer-based media (or more detail, see 1, forthcoming).
All of these discussions and interviews were reviewed by the whole design team. We recognised that the data from the informants described different aspects of the same problem space and we used the data from the interviews in several ways. One was as input for a developmental analysis of the cognitive difficulties likely to be posed by different kinds of representation in learning. For example: what kind of diagram would work with an eight year old? What did they seem to understand? This discussion was linked to our analysis of how any external representation was effective, what the pros and cons of different forms might be, e.g. text versus graphic. Thus, even at a preliminary stage, the designer had a basis to begin sketching out a range of ideas for alternative representations.
The two most salient outcomes of the interviews/panel were the surprisingly wide range of approaches to teaching and the shallowness of learning evident in the students. Why was this so? Part of the problem lies in the complexity of the learning task - many of the concepts are hard, e.g. photosynthesis or energy transfer. But it is also the case that conventional media - paper, pictures, video - are ill-suited to represent these processes in ways that make them easy to visualise. Typically the child at school first encounters the carbon cycle, say, as a diagram consisting of boxes of text-labelled processes connected by arrows of uncertain meaning and where issues such as transformation of state (e.g. CO2 gas 'becomes' C6 sugar) may be unexplained although fundamental to understanding. To grasp such concepts requires a lot of further work - something which teachers or children rarely have the opportunity to do. It is no surprise that teachers do not have a strong common intuition about how to teach from, nor students a good grasp of what is conveyed, by these materials.
In this phase, then, we are principally concerned to take the problems identified in phase 1 and relate them to the possibilities afforded by the interactive software environment. Our point of departure was to try to devise an interactive learning environment to bridge the gap between the abstract representation (food web diagram) and the events that it represented (species interactions in the ecosystem). We believe that this would make the underlying ecological concepts easier to understand and the material more accessible to the child. Interactive multimedia allows us to do this in a novel way by allowing dynamic changes to different displays in response to user interaction. We call this dynamic linking or 'dynalinks'. For example, for our first designs, we built a suite of interactive multimedia prototypes that used a pond as a model ecosystem. The series of prototypes leads the learner from a mode where s/he simply observes the ecosystem to one where s/he can effect changes in it which are simultaneously linked to a diagram representing the food web in the pond. Each module, therefore, introduces more cognitive complexity as well as different kinds of control of the changes that could be made to the pond. An example is illustrated in Image 1 where clicking on a link of the web diagram activates behavioural actions in the animated pond, e.g. fish eats beetle. The important feature of this space is that the effects of manipulating a food web as an abstract representation can be directly viewed in the adjacent concrete animation. We are capitalising on a novel kind of interactivity to 'fix' a profound cognitive difficulty. During this phase much of the effort is from the interaction of HCI analyst and software designer, exploring the possibilities of mapping the interactivity ideal into runnable prototype animations. Another example of an early prototype from PondWorld is shown in our first experimental Shockwave Movie called JamJars Laboratory. This module is based on an analogue experiment where the children have to create a stable system by dropping and dragging organims between a palette and a schematic jamjar in both directions. (N.B. This is an example of the first built protoype by our software designer - prior to input from our informants - and has been included as a simple shockwave movie to illustrate the animations, sounds and interactivity of PondWorld. More functionality has been added since in more recent versions of the prototype. To see the more advanced interactive prototypes please email the authors).
Image 1: PondWorld ecosystem ; two interlinked displays are shown: a canonical food web diagram and a concrete animation of it. The learner can 'turn on' a feeding relationship in the food web and observe the effect (circled) in the adjacent animation.
Shockwave movie: JamJars Laboratory. This module is based on an analogue experiment where the children have to create a stable system by dropping and dragging organims between a palette and a schematic jamjar in both directions.
A Shockwave Movie of this prototype is available.
These materials were taken to schools in order to test a number of assumptions about our project:
(i) our design assumptions: that children had no problems recognising the figures and cross-section as a pond community with a series of feeding relationships
(ii) our cognitive assumptions: that they could manipulate the scenes to construct a food web and that having both the web and the opportunity to manipulate the figures would be helpful in answering questions about the effects of changes to the population
Sixteen children from the ages of 9 to 11 years worked in same-age pairs, selected by the teacher, separate from the class. Teachers typically selected same-sex friends who worked together well. At the beginning of the session we explained to the children that we were developing software, primarily for CD-ROMs (since they were all familiar with this technology), and that it aimed to teach children in a better way than books currently did. What we wanted was for them to help us out by testing some of our ideas and to help us with their design. The pairs of children were seen by an adult 'facilitator', who was a psychologist from the design team. We structured the sessions as a series of three design-validation exercises.
The procedure we adopted was fairly loose-format. We asked the children what they thought would make a good CD-ROM for teaching about food webs. The facilitator encouraged them to use the low-tech materials that they had been working with in the previous exercises. Typically each pair of children would talk for anywhere between ten and twenty minutes about their ideas, which we recorded on audio or video tape. The children would often talk extensively without prompting but if they seemed to be stuck the facilitator would ask about the consequences of the imaginary user behaving in a certain way, e.g. making incorrect links in the food chain. At the end the facilitator asked for suggestions about special effects, such as noises made during eating. Throughout the session the emphasis was on eliciting as many suggestions for animations as possible while minimising input from the facilitator.
(On possibility of gruesome noises for animals dying) ".. depends on what age they are .. if they were quite old you could have some really revolting ones .. boys prefer more gruesome noises"
Responses to the hi-tech prototypes were primarily focused on a wide range of interface issues. Typical concerns here were: the benefit of better narration, feedback and cues for action. Examples include: the need for commentary for indicating the range of possible actions which users would otherwise have to discover - such as 'add food', 'click on a link'; the need to distinguish buttons (support actions) from representations (such as circular icons in the food web). These data came primarily from the child informants. Teachers typically did not notice or mention these aspects, taking their brief to be one of commenting on the classroom value of the prototypes. One persistent concern that they had was the motivating power of the software, something they shared with the children but expressed in a different form.
In addition, we discovered how good children are at testing the non-obvious aspects of our design, stretching the limits of the software functionality. For example in one module where the task was to stock a jam jar from a 'palette' of animals, children would drag and drop animals outside of the jar just to see what happened. Finally we were able to broadly support our cognitive and design assumptions that dynamic linking of abstract representations (e.g. schematic diagram) and the physical world view (e.g. the pond) helped understanding. This remains to be put to rigorous empirical test but questioning of children, together with their spontaneous comments, suggests that many understood the links between the two views after interaction with Pondworld where they had not before.
2. Bjerknes, G., Ehn, P. and Kyng, M. Computers And Democracy: A Scandinavian Challenge. Gower, Brookfield, VT, 1987.
3. Cypher, A.and Smith, D.C. KidSim: End User Programming Of Simulations, in Proceedings of CHI'95, (Denver, CO, May 1995), ACM Press, 27-34.
4. Druin, A. and Solomon, C. Designing Multimedia Environments for Children: Computers, Creativity and Kids. John Wiley and Sons, NY, 1996.
5. Eason, K. D. User-centred design: for users or by users? Ergonomics 38, 8 (1995), 1667-1673.
6. Houde, S. and Hill, C. (In Press). "What do Prototypes Prototype?" In Handbook of Human-Computer Interaction (2nd Edition), M. Helander, T. Landauer, and P. Prabhu (eds.), Elsevier Science B.V. Amsterdam
7. Müller M.J., Wildman, D.M. and White, E.A. 'Equal Opportunity' PD using PICTIVE, Commun. ACM 36, 4, 64-65, 1993.
8. Nielsen, J., Bush, R.M., Dayton, J.T., Mond, N.E., Müller, M.J. and Root, R.W. Teaching experienced developers to design graphical user interfaces, in Proc. of CHI'92 (Monterey, May 1992) ACM Press, 557-564.
9. Oosterholt, R., Kusano, M. and de Vries, G. Interaction design and human factors support, in the development of a personal communicator for children, in Proc. of CHI 96 (Vancouver, BC, April 1996) ACM, 450-457.
10. Rogers, Y. and Scaife, M. How can interactive multimedia facilitate learning? To appear in CD-ROM Proceedings of the First International Workshop on Multimodality in Multimedia Intterfaces.
11. Rubinstein, R. and Hersh, H. The Human Factor: Designing Computer Systems for People. Digital Press, USA, 1984.
12. Scaife, M. and Rogers, Y. External Cognition: How do graphical representations work? International Journal of Human-Computer Studies, 45, 185-213, 1996.
13. Schuler, D. and Mamioka, A. (eds.) Participatory Design: Principles and Practices. Lawrence Earlbaum, Hillsdale, NJ, 1993.
14. Virzi, R.A., Sokolov, J.L. and Karis, D. Usability Problem identification Using Both Low- and High-Fidelity Prototypes, in Proceedings of CHI 96. (Vancouver, BC, April 1996) ACM Press, 236-243.
15. Webb, B.R. The role of users in interactive systems design: when computers are theatre, do we want the audience to write the script? Behaviour and Information Technology, 15, 2 (1996), 76-83.
16. Wong, Y.Y. Rough and Ready Prototypes: Lessons from Graphic Design, in CHI'92 Conference Companion (Monterey, May 1992), ACM Press, 83-84.
|
|