Abstract
The panel will focus on Jakob Nielsen's Discount
Usability approach and guidelines. Nielsen has single
handily restored guidelinesto CHI. After being discredited
because of the sheer impracticality ofusing 1000+
guidelines, Nielsen has been working hard to convince
practitioners that all they need to know about usability can
be summarized in 10 guidelines. This may be a real
disservice. While using10 guidelines may be better than
using none, do people who have learned Nielsen' s 10 think
that they now know all they need to know about usability?
Thepanel proposes a wide-ranging, public discussion of
these issues.
Keywords:
Discount Usability, guidelines,analysis and
evaluation techniques.
Introduction
While starting with Nielsen's writings, the panel is
expected to quickly move beyond such scholarly activities
tofocus more upon the interpretation and use of Nielsen' s
ideas by practitioners. For example, many in the CHI
community believe that different techniques and
methodologies are appropriate at different points in the
design and development life-cycle. Does adoption of
Discount Usability displace the use of other, more
knowledge and labor intensive, techniques? One reading
of Discount Usability is that empiric altechniques are
superfluous. A more fundamental concern is that interface
design may not be as easy as the Discount Usability
guidelines suggest. Perhaps design teams really need
to include a professional who is trained in empirical
evaluation, cognitive task analysis (such as GOMS or
TAG), cognitive modeling, design rationale, or claims
analysis.
MODERATOR STATEMENT
Michael E. Atwood, NYNEX Science & Technology
Our science progresses largely because we learn from our
mistakes. Problems observed with the application of the
current methods of science lead to conjectures about new
methods that alleviate these problems. These new
methods, in turn, will berejected either because they do
not alleviate observed problems or because they introduce
new problems that lead to additional conjectures
aboutmethods. Whether we view "discount usability
analysis" as "evaluation at a bargain price" or as "damaged
merchandise" depends, in large part, on what we view as
the problem we are trying to solve and what we view as a
mistake.
"Evaluation at a bargain price." The problem is that
laboratory usability studies cost too much and take too
long. As a result, they are rarely done. Relying on the 80-
20 rule,we can find 80% of the problems with only 20% of
the effort, by simplifying the communication, skill, and
equipment requirements. It is a mistake to require a
highskill level that few possess and that few will pay for.
Would you want topay $100 per hour for an artist to paint
your house when a $20 per hourpainter would do a job
you find acceptable?
"Damaged merchandise." The problem is that conducting
usability studies is not viewed as requiring specialized
skills and knowledge. As a result, they are rarely done
well. Finding ways to package these skills and knowledge
in toolsthat others can use makes apparent that specialized
skills and knowledge are required and should effectively
raise the skill level of less expert evaluators. It is a
mistake to try to downplay the skills needed for expert
performance. Would you trust a bridge built by a
"heuristic civil engineer" or be comfortable visiting
a"heuristic brain surgeon"?
PANELISTS STATEMENTS
Jacob Nielsen, SunSoft
A Bird in Hand. As the saying goes, two birds in the bush
will not provide you with much for dinner. Similarly,
perfectly polished usability techniques will not improve
your interface much unless they are in fact applied in your
project.
Discount usability engineering aims at placing a bird in
yourhand by providing methods that are so easy and cheap
to use that people will in fact use them on almost every
project. The methods have acknowledged weaknesses and
are not guaranteed to give perfect answers every time.
However, they do provide reasonable answers most of the
time, and the sereasonable answers are much better than
the guesswork that would result if designs were shipped
with zero usability involvement. Consider the issue of
statistical significance. If a result has a p-value of .2, it
means that there is 20% probability that it doesnot hold.
However, in the remaining 80% of the cases, the result
would be correct, so acting on it would bemuch better
than the 50% chance of success that would follow from a
random decision (assuming a dichotomous choice).
As organizations reach progressively higher levels of
usability maturity, they will start using a larger number of
"deluxe" usability methods. For example, many
companies eventually build even start doing formal
modeling of critical dialogues. Discount usability
engineering plays two roles in the path toward higher
usability maturity: (1) it smoothes the way by lowering the
threshold of getting started, and (2) it can be used on fast-
paced or low-budget projects even in organizations that use
a more careful approach for their high-priority projects.
For example, I recently had to perform four usability
evaluations of a World-Wide Web interface within a
single week, and some of the last icon iterations were tested
with an N of 1. Still, that was better than an N of 0 (the
alternative given the deadline to ship the design).
John M. Carroll, Virginia Tech
Flushing out the birds in the bush. The growing diversity
of evaluation methods in HCI reflects an emerging
understanding that evaluation can serve many goals in
system analysis and development, and in building a
science of HCI. As researchers, our job is to understand
these methods in terms of their various costs, benefits, and
conditions of application. As practitioners, our job is to
adapt and refine these methods, and to educate our
colleagues in their use.
Discount methods serve us as practitioners by providing a
bird in hand, as Jakob puts it in his panel statement; they
serve us as researchers by expanding the design space of
evaluation methods -- pushing on its boundswith respect to
lowered cost. If there is a downside to discount methods, it
is the blindness that all methods can engender: No method
can serve every purpose. For discount methods, as for all
methods, we need to enumerate what evaluation goals are
served and how, the costs, the benefits, the conditions of
application.
A decade ago, when performance efficiency models (like
first-generation GOMS) were ascendant in HCI, a lively
debate established that much remained to be described
about user interaction beyond ideal keypress times.
Discount methods will do us a disservice only if we again
seek general panaceas, and fail to recognize that the
problems we address are diverse, indeed open-ended, and
that our methods need to match this diversity.
Wayne D. Gray, George MasonUniversity
Discount Usability appears to be a hodgepodge of
techniques and guidelines whose exact interpretation may
vary greatly from practitioner to practitioner. While in the
hands of the master the interpretation and use of the
guidelines is influenced by years of study and thought on
HCI issues, the average practitioner does not have such an
extensive background. Herein lies the difficulty. That an
inspired designer, such as Nielsen, can use the guidelines to
produce a superior interface is not at issue. What is at issue
is how these guidelines are used by people who have not
made a career of studying interface design. While
Discount Usability may produce "bargain" interfaces, we
should be careful that we do not simply get what we paid
for.
The leading alternatives to Discount Usability (including
Claims Analysis, GOMS, TAG, Cognitive Walkthroughs,
and Participatory Design) all place a heavy emphasis on the
careful analysis of the flow of information between people
and computers (various of these approaches place varying
emphases on the social and organizational dynamics of
computer use as well). While such alternatives are by no
means "discount" approaches all promise substantial time
savings over the traditional usability lab approach. Note
that advocates of such methodologies seem to believe that
it is the application of the method and not its learning that
takes the most time. (N.B. all such techniques have been
the subject ofwell-received, 1 day, CHI tutorials.) The
time required to apply these techniques is almost totally
afunction of the amount, degree, and level of analysis
required tounderstand how the human and the computer
must interact to perform the task. Shortcutting the time
required to do these analyses may make interface design
faster but the result is no bargain.
John Long, University CollegeLondon
Human-computer interaction (HCI) is an emergent
engineering design discipline. As a discipline, it seeks
HCI knowledge to support HCI practices of diagnosis and
prescription to solve general HCI design problems.
Thescope of HCI design problems can be informally
expressed as specifying human behaviours which interact
with computer behaviours to perform work effectively.
The potential for its knowledge and practices ranges from
‘craft' engineering, like Discount Usability, which uses
experience/heuristics to support trial-and-error practice, to
‘formal' engineering which is able to specify without
iteration and with guarantee. As two old sayings have it:
you pay your money and you take your choice. However,
you cannot have your cake and eat it (and particularly not
discounted cake).
DISCUSSANT
Carolanne Fisher, US West
Carolanne Fisher is a developer who has experience in a
large corporatesetting (Wang Labs and US West) as well as
with a small startup company(Maya Designs). As
discussant, Fisher will get first dibs on providing a
developer's response to Discount Usability in general and
the panel discussion in particular.
