Peter Faraday
Centre for HCI Design
School of Informatics
City University
Northampton Square, London EC1 0HB, UK
Tel : +44 - 71 - 477 - 8411
Email : P.M.Faraday@city.ac.uk
FIGURE1Task information model for laser printer cleaning
(ii) Presentation unit analysis : The presentation provided is
segmented into presentation units, either as base media (such
as image, text, still-motion image or animation) or
presentation techniques which add or emphasise information
in the base media (such as caption, highlight or zoom).
Figure 2 gives an analysis for the 'open cover' example
presentation.
FIGURE 2Presentation unit analysis
(iii) Topic focus evaluation : This is concerned with the
order presentation units will be attended to and processed in,
and how they should be linked together, against the
information order required by the task. It is based on an
analysis of attentional ordering and topic focus control [1].
The evaluator assigns focus control operators to each unit :
initialisation takes place if a presentation unit has no
coherence with any other units (eg when a new base media is
introduced); a shift occurs if the units are not coherent (eg
the units do not refer to the same level of the topic); links are
assigned between two units having a common referent (eg
between two base media units referring to the same object,
or a presentation technique unit and the base media unit it is
upon). Links also have a type : foreground is used if a unit
gives further information, background if it gives context, or
emphasise for importance. Figure 3 gives a topic focus
evaluation for the 'open cover' example in the form of a
graph : focus is initialised on the still-motion image, then
shifted to the title 'Open the Printer Cover', then back to the
still-motion image and is linked to the inset-zoom and arrow
presentation techniques.
FIGURE 3Topic focus analysis
Topic focus ordering is compared against the order of
information types required within the task model. The
evaluator is given a series of questions : will topic focus be
initialised, linked and shifted on the correct units to support
the task ? In general, focus should be initialised at the start
of the task step; presentation units within the goal, pre-
condition, action, or post-condition task parts should be
linked together; and shifting should occur between the task
parts. For the 'open cover' example it can be noted that the
ordering given by the presentation is pre-condition (still-
motion image), goal (text title), action (inset-zoom, arrow),
and that no post condition is provided. This suggests that the
presentation may be incorrectly formed.
(iii) Mental model evaluation : This evaluates the mental
model structure formed from the presentation units against
the information types needed by the task. It has two stages :
content evaluation and structure evaluation. Content
evaluation asks whether the mental models formed meet the
task information needs. The presentation unit's conceptual
content is defined within one or more types of mental model
[2] : (a) relational, giving a frame of object tokens and
properties; (b) spatial, consisting of a relational model in
which object properties are spatial relations or locations; (c)
kinematic, representing the path of motion of an object
undergoing action; (d) temporal, giving an ordered sequence
of spatial/relational or kinematic frames; (e) dynamic :
consisting of spatial/relational, kinematic or temporal frames
with causal relations between. Rules are provided which map
the base media or presentation technique to the type of
mental model formed : figure 4 gives an example set of
presentation units with mental model mappings. The
evaluator then compares the models formed with the
information type requirements of the task. Figure 5 (left)
illustrates the models formed for 'open cover' : the title gives
a dynamic model; the still-motion image, inset and zoom
techniques give relational and spatial models and the arrow
provides a kinematic model. This diagnoses a problem : the
dynamic model formed by the title 'Open the Printer Cover',
which maps to the sub-goal of the task, has no 'effect'; thus
failing to meet the task's post condition requirements.
FIGURE 4 Simple base media and presentation techniques
The second stage of evaluation concerns the structuring of
the mental models formed : asking how the models built
from each presentation unit will be integrated together. Each
presentation unit may either construct a new mental model,
be integrated within an existing model, or combine models
together. If topic focus is initialised then a new set of models
will be built; if it is shifted to a unit then a new model will be
constructed, either afresh or by joining existing models; if
the current focus is linked to another unit, then it will be
integrated within the existing model. The type of link will
indicate how it is combined. Figure 5 (left) shows the model
structure formed for the 'open cover' example. This suggests
a problem : the title 'Open the Printer Cover' and still-motion
image may fail to form an integrated printer-cover concept
within the relational model if the 'printer-cover' object can
not be identified within the image by the user.
A revised presentation is shown in figure 5 (right) : using a
still-motion image of the printer cover in is open state the
required temporal model is now produced to support the
task's post-condition. The 'printer-cover' object is
highlighted in both images to give an emphasise effect for
identification and to produce a topic focus link. This will
then form an integrated printer-cover concept within the
relational model, together with that produced by the '..Printer
Cover' title.
FIGURE 5Mental models for revised presentations
Abstract
The paper reports the basis for a cognitive walkthrough
method to support the formative evaluation of visually based
multimedia expository presentations.
Keywords :
Multimedia, Evaluation, Presentation
Introduction
One of the problems in the design of multimedia (MM)
presentation interfaces is the difficulty in using existing
iterative prototyping techniques : the costs of authoring and
content design may make such testing difficult and
expensive. A solution may lie in formative evaluation;
however present MM evaluation is largely a matter of rules
of thumb, or gross measures of usability based on existing
metrics. These approaches lead to difficulty as they fail to
accurately pin-point and diagnose presentation problems. As
a solution we propose a cognitive walkthrough [3] method
for MM evaluation. The walkthrough is founded upon an
analysis of the cognitive processes and representations
formed by the comprehension of an MM presentation. This
provides a series of questions for evaluation such as to the
use of media, scripting and presentation techniques.
MODELLING COMPREHENSION
The process of MM comprehension is adapted from theories
of cognition of visual scenes, animation and text. The model
does not address the complexities of natural language or
visual image processing; instead its focus is on media
integration during comprehension. This is divided between
cognitive processes such as attention and topic focus control;
and the construction of a set of mental models [2]
representing the presentations conceptual content. The
model proposes a two stage process : firstly of analysing and
evaluating attentional and topic focus control within the
presentation, and then examining its conceptual content and
structure.
EVALUATING PRESENTATIONS
(i) Task and domain knowledge analysis : A task model is
formed providing a goal hierarchy as a basis for assessing
the information content of the presentation. Each task step is
specified with one or more logical information types
required for its performance (see figure 1). These are typed
as descriptive for task objects (attributive, spatial or
qualifying types); and operational for task action (attributive,
spatial, temporal or procedural types).
EVALUATING THE METHOD
Current work is being conducted concerning the validation
of the evaluation method itself. This will address its
theoretical worth by comparing task performance with
predictions from the evaluation : would 'open cover', re-
designed as suggested, be performed more successfully in
real life ? The practical utility of evaluation must be also
assessed : will MM designers find the walkthrough useful ?
CONCLUSIONS AND FUTURE WORK
The present paper offers some tentative steps toward a
comprehension driven walkthrough method for MM
evaluation. It is hoped to place the design and evaluation of
MM presentations on a firmer theoretic base [4] : the
cognitive model emphasises comprehension and conceptual
representation which we believe to be more relevant and
economic for MM evaluation purposes.
