|
|
The paper reports four studies concerning attention to and comprehension of Multimedia presentations. The Multimedia sequence used was taken from a commercially produced CD-ROM, 'The Etiology of Cancer'. First, an eye tracking study of the presentation is reported. A second study was then conducted on the recall of the materials used in the eye tracking study. The results of these studies were used to propose design guidelines for Multimedia presentations. The guidelines were applied to produce a re-authored version of the original presentation. A further two studies were then conducted on the re-authored version to assess the impact of the design guidelines.
One of the problems in the design of multimedia (MM) interfaces is knowing whether the presentation will successfully deliver information to an audience. While studies have been conducted on cognitive issues relating to MM design e.g. [10, 12], there are very few studies of attention and comprehension; hence current usability guidelines are based on minimal evidence. This paper outlines a series of studies to provide improved design advice for MM presentations.
Our first study used an eye tracking system to investigate the sequence of visual information processing by users interacting with a multimedia presentation. We were interested in describing the attention and the viewing process for different presentation components, such as text captions, labels and still and moving objects; and how the serial nature of visual attention would cope with a complex and rapidly changing MM presentation. Few studies have made use of eye tracking equipment to look at animated materials [3,15], and none have attempted to track subjects viewing a complex multimedia presentation. However, attention to appropriate information is vital if a message is to be successfully delivered by an MM presentation.
Our second study examined comprehension of the same presentation by testing free recall, in which subjects wrote down what they could remember of the presentation. The comprehension process is though to act upon the elements processed by attention, and combines them with the user's existing domain knowledge to produce propositions, or units of meaning [16]. For instructional presentations the propositions we used were. objects, action, procedure and causal relationships [2]. We were interested in the information types recalled from the presentation and in relating them to the components in the MM design. We also investigated the effects of the user's existing knowledge upon comprehension by testing two subject groups with, or without, domain knowledge. The scope of the investigation focused on training titles. There is evidence that pedagogical design is a key determinant of the effectiveness of MM tutorial applications; however, few studies have addressed how presentation design effects learning. Accordingly we focus on attention and information assimilation rather than the effect of pedagogy on interactive learning.
We used the results of the first two studies to propose a set of design guidelines, which were applied to re-author the presentation. In the latter two studies, we assessed the impact of the guidelines by testing information recall with the re-authored presentation in a multimedia and a text and speech only version to investigate the effect of redesign in mono-medium or multimedia format.
The first study explored users' attention to a MM presentation using an eye tracking system to match the pattern of subjects' fixations in the visual media. An 18 second animation was taken from a commercial MM title 'Etiology of Cancer' showing DNA repair by photoreactivation. This is illustrated as a series of screen shots in figure 1. The speech track which accompanied the presentation is given under the relevant screen shots. Figure 1a shows the initial state of the presentation, with the DNA and pyrimidine dimer labelled. In figure 1b the photolyase object is shown moving across the screen and attaching to the pyrimidine dimer. In figure 1c the photolyase is struck by light energy from the lower right of the screen to the center of the photolyase. In figure 1d an arrow, a second photolyase and the repaired DNA are revealed; and then the photolyase object is shown moving off the screen to the left.
Six subjects were drawn from research students within City University. A pupilometer system was fixed upon the subjects eye, whilst they viewed the MM presentation. This produced an x,y location of the pupil every 50 msec and filtered out spurious data from eye blinks. Subjects used a bite clamp to avoid head movement and were held approximately 50cm from a 17 inch VGA colour monitor.
Figure 1a : Pyrimidine dimers are reconverted to monomers
Figure 1b : by the action of photo- reactivating enzymes called photolyases which
Figure 1c : form a stable protien-pyrimidine dimer complex and then use light energy absorbed from the visible range
Figure 1d : to convert the dimer into monomers without disrupting the double strands.
The raw data of pupil location was time sliced into four 5 second parts for each subject to ease analysis. A PC based tool was built to plot the data for each subject as a scan path (see figure 2). Each subject's scan path was analysed into a fixation graph, representing fixation clusters or pauses of the subject's eye at a location. The data for the six subjects was synchronised over time to build up a composite trace.
Figure 2 : Single subject scan path on figure 1a
Whilst it is known that pupil location and attention may not always coincide, in general a fixation at a location will indicate attentional processing [8]. The fixation graphs were overlaid on the presentation. Figure 3 a,b,c,d shows the order of fixations for the six subjects, with the sequence of fixation shown as circled numbers.
Figure 3a
Figure 3b
Figure 3c
Figure 3d
(1) Five subjects initially fixated on the pyrimidine dimer label and object; one subject fixated on the deformed, central part of the DNA (see figure 3a).
(2a), (2b) Subjects shifted to the pyrimidine dimer and switched back and forth to its static label. The pyrimidine dimer object and label received a greater number of fixations than the DNA object and label; possibly because the pyrimidine dimer was cued within the speech track.
(3a), (3b)Following fixation upon the pyrimidine dimer and label, three subjects then shifted between the DNA object and it's label (figure 3a). Fewer fixations may have been given to the DNA label than to the pyrimidine dimer label because it was not reinforced by the speech track.
(4) The photolyase object appears and then moves along a path toward the pyrimidine dimer object. For all six subjects this produced a rapid alignment upon the photolyase object when it appeared, with subsequent fixations tracking its path (see figure 3b).
(5a), (5b) Fixations followed the photolyase object's path to its end point (5a), where revealing the photolyase label (5b) caused a shift between it and its label for all subjects (figure 3c). The photolyase label was also cued by the speech track.
(6) Three of the subjects shifted fixation to animation of the light energy object striking the photolyase. The motion of the light energy was more rapid than for (5), causing shifts of fixation rather then tracking. The other three subjects shifted to the photolyase object but did not track the light energy.
(7a), (7b) Subjects shifted back to the photolyase object and label (figure 3c); possibly directed by the end point of the light object's motion in (6). Attention to the pyrimidine dimer may also have been partially cued by the phrase 'protein pyrimidine complex' in the speech track.
(8) All subjects shifted from the photolyase to the revealed arrow symbol (see figure 3d) and followed the arrow downwards. Four subjects then fixated on the second photolyase object in the centre of the repaired DNA object; while two attended to the repaired DNA label.
(9a), (9b) Five subjects initially tracked the second photolyase object as it detached and moved away. Two subjects broke off tracking to re-fixate upon the first photolyase object; possibly to compare the two photolyase objects.
(10) After (9), two subjects shifted to the repaired DNA label (figure 1d). The label may have not been fixated in (8) because of the onset of motion of the photolyase shifting attention. Tracking the photolyase path in (9), may, however have directed fixation to the repaired DNA label which lies at the end of the path.
(11) Five of the subjects shifted to the repaired DNA object at the end of the presentation. Fixations were generally upon the centre of the repaired DNA from where the second photolyase had departed in (9).
The results provide the basis for proposing the following set of tentative guidelines for MM attentional design :
Our second study investigated comprehension of the same MM presentation using two groups of eight subjects. The first group was drawn from research students within City University, who had low domain knowledge; the second from biochemistry post-graduates who had high domain knowledge, but were pretested to ensure that they had no specific knowledge of the subject matter itself. Both groups were shown the presentation once and were then instructed to write down all that they could recall of the presentation.
In order to score the accuracy of subject recall, the MM presentation was analysed into a set of ten reference propositions by an independent expert. The propositions were grouped into either high level organising concepts of sequence, problem or cause-effect, or into lower level actions or objects within each part of the sequence. The subjects written recall was split into propositions, and then matched to the reference propositions.
There was no significant difference in recall between high and low domain knowledge subjects. The results (see table 1) show that some parts of the presentation were well recalled, whilst others were lost. Slightly more was recalled by the high domain knowledge group, in particular proposition (iv), 'Photolyase attaches to the pyrimidine dimer'. This may be because these subjects were better able to interpret the implications of the animated actions. Propositions (iii, vi-ix) were well recalled by both groups, possibly because they were conveyed by animation and reinforced by speech; while the other propositions were not. This demonstrates how poor design of the viewing sequence may lead important parts of the message (e.g the reaction in v, vi) to be missed.
Generally components which were effectively attended to from the eye tracking study experiment were also well recalled. However, other factors in the design may have effected comprehension; leading to some refinement of the initial guidelines :
Table 1 : Low 'v' High domain knowledge recall for original presentation
We used the guidelines to produce a re-authored version of the presentation. We began by adding a caption to the first frame, to support initial state of the DNA and locating damage by the Pyrimidine Dimer (figure 5a). The existing Pyrimidine Dimer label was revealed first to attract attention to Dimer object, then after allowing reading time, the new caption was revealed. A cue was also added to the speech track to reinforce the caption.
Figure 5a : DNA is damaged by a pyrimidine dimer, which distorts the DNA's double strands. A photoreactivating enzyme, photolyase, binds to the pyrimidine dimer
Figure 5b : The pyrimidine dimer and photolyase now form a protien-pyrimidine complex.
Figure 5c : Next, protien pyrimidine complex absorbs light energy from visble range.
Figure 5d This causes the DNA to be repaired by photoreactivation
Figure 5e : The pyrimidine dimer is now reconverted to a monomer, without disrupting the DNA double strands.Finally, the photolyase dissociates from the DNA.
The animation of the photolyase and revealed labels for Pyrimidine Dimer and Photolyase were left unchanged. To re-inforce the animation of Photolyase binding the Pyrimidine dimer, an additional speech cue was added and synchronised with the arrival of the Photolyase : 'photolyase binds to the pyrimidine dimer'.
To improve recall of the Protein-Pyrimidine complex formation, an animation was added to emphasise the process of the Photolyase and Dimer combining. A new Protein-Pyrimidine complex object formed from its two constituents, which then changed colour and shape, to signify a new combined state (figure 5b). An additional Protein-Pyrimidine complex label and caption were also added and synchronised with the speech track .
The animation of light energy striking the Protein-Pyrimidine complex was then amended, to show a reaction by a small explosion style effect added to the complex as the light energy struck (figure 5c). The light energy was paused as it struck, to allow attention to shift to it. An additional caption was then shown re-inforcing the speech .
After the light energy animation was finished, the Protein-Pyrimidine complex was shown to pull apart, separating the photolyase and dimer (figure 5d). An additional caption and speech cue re-inforced 'photoreactivation'. The arrow symbol was then displayed and the repaired DNA revealed. To make the process clear, animation showed the dimer disappearing and the DNA re-joining (figure 5e). Finally, the repaired DNA label was given as a caption and moved to the centre of the DNA, where attention had been directed by the animation (figure 5e) allowing reading time. The animation of the photolyase was shown, with a speech cue 'photolyase detaches' to re-inforce it
To investigate whether our design guidelines would improve recall, we used a further group of eight subjects with low domain knowledge and showed them the re-authored presentation (figure 5). We then scored recall in the same way as before.
The results are shown in table 2 in the 'Multimedia' column. A significant difference was found between recall in the low domain knowledge group in the original design (table 1, col 1) and the improved design (Mann-Whitney U, P<0.05). There was a clear improvement in recall over the original design, and in some cases performance was superior to those of the high domain knowledge group, although this difference was not significant. The improvement in recall gives some validation for our design guidelines :
Table 2 :Recall of the re-authored multimedia design and text-speech only for low domain knowledge subjects
Whilst the results from study three seemed to demonstrate the value of our guidelines, they did not differentiate between the effect of redesign and adding information to the original presentation. In an attempt to be sure of the impact of these changes, we performed a further study using text and speech only. The text script derived from study three was accompanied by a matching speech track. If recall was improved above that found for the original presentation in study two, then this could be attributed to the addition of text captions and information in the speech track. If the recall was poorer than in study three, then the use of the re-authored images and animations had aided performance.
The results, shown in table 2 in the 'text-speech' column, demonstrate the value of effective use of multimedia. Recall performance with text-speech was significantly worse than for the low knowledge group in the improved MM design (Mann Whitney U, P<0.001), but was not significantly different from the low knowledge group for the original presentation. This showed that for re-authoring to provide additional linguistic information was not in itself sufficient to improve comprehension : animation and images were also vital for low knowledge subjects to understand the presentation.
The studies reported have shown some of the dangers of attempting to design MM presentations without concern for attention and comprehension. MM designers should consider how to control shifts of attention according to the content being shown. Attention and the viewing sequence can be influenced by design of highlighting, and reinforcement with speech. Designers also need to model the user's viewing process to provide time for reading captions and to avoid attentional clashes. If the presentation is complex it may also be useful to guide attention to important elements using reveals, labels, symbols or speech.
Attentional design may be more important for low domain knowledge subjects, who need to be guided at exactly what to attend to. This is vital for instructional design, and may useful when information delivery is critical e.g. in command and control applications.
Our second study of subject's recall showed that design problems may lead to difficulties in comprehension. We suggested comprehension may be improved by design of components for attracting the users' attention, thereby ensuring that the presentation could be formed into coherent whole. This may be achieved by scripting the speech track to more tightly reference and synchronise with the visual elements of the presentation; and by using animations and captions to support the content.
The final two studies demonstrated the effectiveness of our design guidelines. In study three, the re-authored version of the presentation showed a marked improvement in recall while the fourth study showed the value of presenting the information in a range of media. However MM presentation design is only one component of delivering effective learning. Pedagogical techniques, e.g. active engagement are key components, as is rhetorical design of the message. We have partially addressed the latter in [14] and intend to explore how different types of message (e.g. causation, procedure) may be delivered by media combinations.
The main implication of our work is to point out how attention and the viewing process can be controlled for effective information delivery. In task centred MM getting the message across is vital, so immediate recall is an appropriate measure. However immediate memory does not always correlated with longer term recall [13], so in tutorial applications pedagogical techniques to encourage depth of memory encoding may be the key, although these techniques will still fail if they do not employ effective means of directing the user's viewing sequence, as demonstrated in our study.
This work provides some tentative steps toward placing MM design and evaluation within a firmer theoretic framework. Several problems remain for future research. We would like to increase the number of subjects in study three and four by performing the studies on high domain knowledge subjects. We also feel it is important to address how design advice can be integrated into the life cycle of MM production, following [14]. Design advice is necessary during the formative processes of authoring, such as when scripting and choosing media.
The Etiology of Cancer CD-ROM is copyright of American Medical Television and Silver Platter.
1. Baggett, P. The Role of Temporal overlap of Visual and Auditory Material in Forming Dual Media Associations. Journal of Educational Psychology 76 (1984) 408 - 417
2. Bieger, G.R. & Glock, M.D. The Information Content of Picture-Text Instructions. Journal of Experimental Education 53 (1984), 68-76.
3. Brandt, H. The Psychology of Seeing. New York : The Philosophical library (1954)
4. Grimes, T. Mild Auditory-Visual dissonance in TV News may exceed viewer attention. Human Communication Research 18, 2 (1991), 268 - 298
5. Hegarty, M. & Just, M.A. Constructing mental models of machines from text and diagrams. Journal of Memory & Language, 32, 717-742, 1993.
6. Hillstrom, A.P. & Yantis, S. Visual motion and attentional capture. Perception & Psychophysics, 55(4), (1994), 399-411.
7. Hochberg, J & Brooks, V. The Perception of Motion Pictures. In Handbook of Perception and Human Performance X, LEA, 1980
8. Johnson, W.A. & Dark, V..J. Selective Attention. Annual review of psychology, 37, (1987), 43-45.
9. Loftus, G.R. Eye Fixations on Text and Scenes. In Eye Movements in Reading, Perception, and Language Processes. Academic Press, 1983, pp.359-376.
10. Large, A, Beheshti & J., Breuleux, A. Multimedia and Comprehension : A cognitive study. Journal of American Soc. for Information Science, 515-528, 1994
11. Park, O. & Hopkins, R. Instructional conditions for using dynamic visual displays : a review. Instructional Science 21 (1993), 427 - 449
12. Rieber, L. P. Animation in Computer Based Instruction. ETR&D 38 (1990), 77 - 86
13. Stasko, J., Badre, A. & Lewis, C. Do Algorithm Animations Assist Learning ? Proceedings CHI'93 ACM, 1993, pp. 61-66
14. Sutcliffe, A.G. & Faraday, P.M. Designing Presentation in Multimedia Interfaces. Proceedings CHI'94 ACM, 1994, pp. 92-98
15. Yarbus A. Eye Movements & Vision. Plenum.1967.
16. Van Dijk, T.A. & Kintsch, W. Strategies of Discourse Comprehension. New York: Academic Press. 1983
|
|