Name and contact information:
Janice (Ginny) Redish, Ph.D. President Redish & Associates, Inc. 6820 Winterberry Lane Bethesda, MD 20817 voice: 301-229-3039 fax: 301-229-2971 e-mail: redish@ari.netJanice (Ginny) Redish is an independent consultant, helping companies and government agencies with training and assistance in usability and documentation. She teaches workshops in usability testing and doing task analysis. She helps organizations build usability into their processes and products. She does expert reviews of user interfaces and documentation. Ginny is co-author of A Practical Guide to Usability Testing (Ablex, 1993) and has written many articles and book chapters on both research and practice in usability. Ginny was an Associate Chair for the Papers Committee for CHI '97 and serves on the Boards of both the Usability Professionals' Association and the Society for Technical Communication. She is a graduate of Bryn Mawr College and holds a Ph.D. in Linguistics from Harvard.
Most people at the workshop will be sharing a specific usability testing
experience. Rather than doing that, I believe that I can best contribute
to the workshop through my overall experience as a usability testing
consultant and as a document design specialist who works with many
major companies and organizations. In this position paper for the
workshop, therefore, I would like to contribute some issues and ideas
for the discussion of the three major themes of the workshop.
1. What are the known usability problems in Web sites, and what testing methods successfully uncover these problems?
One of the problems with the Web is that it is a medium that both allows too much and too little. Web site developers can easily use so much animation that the text is difficult to attend to. Web site developers can easily use more graphics than users with slow modems want to wait for. In many ways, the Web is a step backwards for those of us with expertise in document design and the human factors of page and screen layout. Web designers cannot control what line length or font will appear on different users' screens. Yet, usability testing of paper and traditional online documents has repeatedly shown that users have difficulty reading extensive prose from the screen. Design elements such as short chunks of information, relatively short lines, specific fonts, and functional graphics well placed with the text all make a difference in users' comprehension and attitude towards documents on the screen.
For some of the problems that are likely to appear on Web sites, usability testing methods that have worked for paper or online documentation and interfaces can work well. For example, one issue for Web site development is whether the vocabulary used for buttons and hot words will lead users where they want to go. Another is whether picture icons are meaningful to users. For issues like these, old testing techniques work well and can be done through paper prototypes storyboards before doing any HTML or Java coding. To test icons, you ask users to identify the meaning of the pictures you might use or to match pictures with brief descriptions. To test vocabulary, you ask users what information they would get by choosing a specific word or you ask users which word they would pick if they were looking for information about [name of specific topic] or if they were in a specific situation [given a certain scenario].
Usability testing of Web sites can also uncover other issues that may not have been obvious to Web site developers. For example, recent reports from naïve users indicate that many are confused by not knowing when a link is going to take them somewhere else in the same Web site and when a link is going to take them to a completely different Web site. This is a version of the "lost in hyperspace" phenomenon that emerged from usability testing of online help, but one that Web site developers may not have thought about.
2. What are the special characteristics (limitations and opportunities) of the Web as a medium, and how can we exploit those characteristics in our testing?
I would like us to discuss at least these four issues about the limitations and opportunities of the Web as a special medium with emphasis on how to test and what we can learn from usability testing about these issues:
The Web as instant global communication
One of the great benefits of the Web is that it is instantly available to anyone anywhere in the world who has access. But that means that our Web sites have to be cross-culturally sensitive and comprehensible to many for whom English is not their native language. Many organizations have been struggling with issues of making interfaces and documentation appropriate to many cultures and languages. They have generally done this by translating words and by trying to develop icons that do not offend in any culture. Some software lets users indicate the language, currency markers, punctuation, and even icons that they want to use in their interfaces. How many Web sites do that?
International usability testing and field studies have increased in the last several years. (See the presentation by Mary Beth Butler in UPA Proceedings, 1994, and the chapter by Susan Dray and Deborah Mrazak in Wixon and Ramey, Field Methods Casebook for Software Design, Wiley, 1996.) But international usability testing is still rare. Yet, it a Web site is going to have worldwide use, it should be tested worldwide. And that should be easier to do remotely, through the Web itself. It should be possible to capture the pathways that people take when working with the Web. It should be possible to have a note space for people to "think out loud" (in writing) as they work. Telephone connections and tiny cameras mounted on the user's monitor can allow more information to be collected during a remote usability test.
The Web as a replacement for traditional online help
In the last five years, documentation specialists have made tremendous strides in figuring out how to provide users with online help that is context-sensitive that is, directly linked to the application and to the specific part of the application they are in. Well prepared online help is fast and easy to access, organized and written to give users the information they need, searchable through many synonyms that users might come with, and linked to take users where they might want to go. What will happen when the Web replaces traditional online help.
The impetus for using the Web for online help is primarily technical. It eliminates issues of developing and maintaining help for several different platforms. In an intranet environment where the Web can be updated easily, it allows for rapid changes to the information. But, usability testing of new Web-based help will be essential to make sure we have not lost the aspects of traditional online help that worked well and to see if we have provided new benefits. How will Web-based help be delivered to home users who have to pay for access to their browsers? Will we simply see HTML code instead of WinHelp code and the online help will be bundled with the software, as context-sensitive and free as it is today?
The Web as an interactive medium, especially for forms and surveys
One of the major uses of the Web that has organizations excited is that it can be interactive. It can be used to collect information, and thus many organizations are planning to put forms and surveys that have traditionally been delivered in person or by mail on the Web. There are issues beyond usability in these data collection efforts. One is security of the information, particularly when sensitive, private personal or financial information is involved. But usability is a major issue. In many cases, users may be people who have not been involved with computers before. Therefore, a major issue in usability testing will be to make sure that user profiles are well thought out and that representatives of specific user groups are included in the usability testing. Another issue for which usability testing will be very helpful is whether to make Web-based forms resemble paper forms or use the "one question at a time" branching patterns of older computer-based data collection methods. Here are just a few of the other issues to consider and look at in usability testing for Web-based forms: Will users be able to stop part way through the form, save what they have done, and come back later? What types of edits will be built into the Web-based form and how will users be helped to deal with the results of the edits? For forms that users traditionally had help filling out, what types of information in what levels of layering will give most users the help they need?
The Web on different platforms, with different browsers, and different modems
Although one of the benefits of HTML is that it is not platform-specific, users access the Web through different browsers and modems that operate at different baud rates. If a usability test is done only on one browser and one baud rate, Web site developers may get a very skewed picture of the true usability of their site. The usability technique needed here is easy to define but may be less easy to set up. Testing users' perceptions and attitudes as well as behavior in different environments is essential.
3. What are the major organizational hurdles that need to be overcome to permit usability testing to take place, and how can they be addressed?
Many organizations are rushing to the Web with more fervor than thought. Organizations that were reluctant to devote resources to usability testing for distributed software are not less reluctant to find resources for usability testing of the Web. So all the traditional barriers remain in this new medium.
Furthermore, to the extent that organizations see the Web as documentation rather than as a computer interface, they may not see usability testing as a necessary part of the process. However, the same issues that have not been solved in moving people from legacy systems to GUIs or in creating online help and documentation that really answer users' questions remain in moving to the Web. The new medium is not a magic panacea.
One of the major hurdles is the faster pace of development, more rapid updating, and constant iteration that characterizes Web sites. If usability testing is perceived to be a lengthy and cumbersome process, it will seem to organization managers to be incompatible with the process of creating Web sites.
Therefore, one of the major ways of overcoming organizational hurdles is to promote and use versions of usability testing that fit into the rapid, iterative Web development process. Some of the ways of doing that are to develop
