CHI 97 Electronic Publications: Special Interest Groups (SIGs)
Evaluation - Methodology for Telematic Application Systems: Quality for Users and Context
Dr. J.H. Erik Andriessen & Dr. Bert Arnold
Delft University of Technology
De Vries van Heystplantsoen 2
2628 RZ Delft, The Netherlands
+31 15 278 3720
andriessen@wtm.tudelft.nl / arnold@wtm.tudelft.nl
Keywords
Telematics applications, evaluation, design methodology, user requirements,
context of use, psychological impact, social impact, organizational impact.
© 1997 Copyright on this material is held by the authors.
EVALUATION OF TELEMATIC APPLICATIONS
Telematic applications are applications based on the integration of information
and communication technologies and services, e.g. tele-medicine systems,
tele-work systems, traffic management systems. New technologies in this field
are potentially very important for co-operation and exchange of information
within and between organisations and communities. There is however a critical
aspect in the development of these applications: the design and evaluation of
these new technologies are often mainly technology-driven.
The development and introduction of new technologies would progress more
adequately if they could be evaluated systematically from a users point of
view. In evaluating user aspects of telematic applications, a set of related
interacting factors should be considered: i.e. the interaction of users, tasks
and technical system in a 'context-of-use'. In this Special Interest Group
attention is given to approaches and methods to evaluate telematic applications
from a user(organisation) perspective and in the framework of the total
development process. This implies simultaneous attention to several questions:
to what extent is the interface 'usable', to what extent is interpersonal
communication adequate, and what are the effects on task performance and
(organisational) context.
In Europe interest in this area is fostered by European Community research
programs such as the Telematics Applications Program, which requires systematic
attention to user involvement, validation and field trials.
Within the context of evaluation methodology and usability testing in general
this SIG will focus on the importance of three interaction levels involved in
the process of evaluation:
- The interaction between user and a technical tool. Relevant issues are
mental effort and perceived usability.
- The interaction between user and a database at a distance. Relevant issues
are: network performance (signal quality) and quality/accessibility of the
information.
- The interaction between users, mediated by a tool. Relevant aspects are
interpersonal communication and group interaction (team effectiveness, group
dynamics and media use).
In all three cases interaction with the wider (often organisational) context
is at stake.
ISSUES AND GOALS
Essential for the development of telematics applications is the attention to
the needs and problems of the different stakeholders. To verify whether
applications are meeting the stakeholders objectives, but also to assess what
the (potential) impact on human interaction or on organisational functioning
will be, appropriate evaluation approaches have to be available. Many ideas,
principles, techniques and assessment instruments can be encountered. The
question is whether a comprehensive methodology can be developed, and what
research paradigms are relevant. This is important for designers and users of
new systems. The SIG aims at developing theory and methodology in this field.
Those interested to participate in the SIG are invited to
send a message to the
organizers. They will receive additional information on this
topic.
CHI 97 Electronic Publications: Special Interest Groups (SIGs)
