Mary Hunter Utt

Experience:
Open Market, Inc., Software engineer, 1994-present.
UI design and implementation for Web applications, including custom, interactive Website development; and design, implementation, and usability evaluation of an interactive Web interface to a Web server log usage application. Currently part of the team developing and evaluating the buyer and merchant UIs for Open Market's transaction-processing system, OM-Transact.

Digital Equipment Corp., Software engineer, 1985-1994.
Part of a cross-functional team to design and implement a GUI-based online documentation system. Conducted extensive usability testing of prototypes for this system, including studies of screen font usability and evaluation of a graphics editor to support the system.

Digital Equipment Corp., Technical writer/editor, 1977-1985.
Wrote and edited software user's manuals for operating systems, programming languages, and other application software.

Education:
BA, English Literature, Carnegie-Mellon University
MS, Computer Science, University of Massachusetts at Amherst

Web site development at Open Market

Open Market develops software products that provide the infrastructure for secure Internet commerce. This infrastructure is a complex implementation of multiple servers, database schema and queries, firewalls, encryption and security algorithms, sales and tax computations, reporting utilities -- all of which must be rendered as user interfaces that buyers can easily navigate to make purchases and merchants can readily integrate into their business environments.

Thus, for our UI design and evaluation work, Web sites are businesses conducting commerce on the Internet. We are specifically concerned with the areas of these Web businesses that represent the "back office" -- orders for goods and subscriptions, customer service, order fulfillment, reports on business activities, setting up and managing the site.

The development of user interfaces for Web sites that are actually complex Web applications is fundamentally different from developing Web sites that deliver information or interactive entertainment. For the latter, the issues of navigation options within a hypertext are paramount.

In contrast, for Web applications the issues are more akin to those of traditional software applications, focusing on user tasks, which may well be work activities. Navigation is indeed a crucial concern, but the design issue is to focus or direct the flow through the site based on user task performance and completion.

Web applications, however, also differ from traditional software applications in that they must take into account the contraints of client-server implementations, network latency, browser differences, and statelessness. In fact, the key implementation problem for Web applications is maintaining state. But all of these constraints have user interface implications and so bear on the usability of the application.

Techniques used for usability testing

Because Open Market is still small and in many ways still a start-up, we are just getting a real usability effort under way. Because we develop web applications, we have started with a fairly traditional approach to usability. Because of budget constraints, we have so far used "discount methods." But some early successes have demonstrated the effectiveness of these methods, and we are working to make them integral to our processes as one of the early stages of software development.

We begin by learning as much as possible about the user, user contexts, and user tasks before designing the user interface of Web screens and flows. Usability of these UI designs is evaluated early on with paper prototypes.

As development progresses, we conduct user testing with the working (more or less) software. In the usual manner, we develop test scenarios, use employees or bring people in who match the user profile and who are unfamiliar with the software. The participants are observed (and videotaped) in a laboratory-like setting; their problems are noted; a report is written itemizing the problems and making recommendations for improvement.

The early design work and prototype testing help ensure a sound basic design. Iterative user testing helps show where the design requires incremental improvements; paper prototypes are also used to test the resulting redesigns.

In addition, we are experimenting with a Web-based storyboarding tool that we developed and that allows us to use screen mock-ups to assemble task scenarios. Because it is so easy to generate HTML mock-ups of screens, this tool allows us to easily walk through task scenarios and quickly modify the HTML and the screen flows to assess alternatives and improvements.

Because the project team uses the Web as a collaborative development tool, the screen mock-ups and scenarios are readily available to them to review. In fact, the functional specification for the next version of the product will be a hypertext document that links in the screens and scenarios developed to date: this will be delivered to customers to review and evaluate. While not formal testing with quantitative results, such reviews elicit early and useful evaluations of the software while it is still cheap and easy to make design alterations, and to make the alterations available for further reviews.

Effectiveness of these techniques

**Effectiveness of testing techniques used at Open Market**
Technique:	Suited for verifying:	Not suited for verifying:	Resources required:	When used:
Paper prototypes	Task flow Navigation Text/layout Forms	Network latency Graphics loading Privacy Security Statelessness Use in real work Use with real data	Minimal: screens drawn using pencil and paper or mocked up using HTML and printed.	Early in design before code is written or for redesign during development. HTML mock-ups can be hyperlinked into project documents for review by team, customers, users.
Storyboarding	Task flow Navigation Text/layout Graphics loading	Network latency Privacy Security Statelessness Use in real work Use with real data Forms	Moderate: HTML screen mock-ups and storyboarding software.	Early in design before code is written or for redesign during development. HTML mock-ups and scenarios can be hyperlinked into project documents for review by team, customers, users.
User testing	Task flow Navigation Text/layout Forms Graphics loading Privacy concerns Security concerns Statelessness	Network latency Use in real work Use with real data	Considerable: usability lab or at least two rooms for test and observers, camera, hardware and software, multiple people to administer, observe, report.	As soon as there is working software; at various points during code development (at least, at field tests) to test design/redesign.

SIGCHI Web Testing Workshop Proposal

Submitted by:

Web site development at Open Market

Techniques used for usability testing

Effectiveness of these techniques