Logo AHome
Logo BIndex
Logo CACM Copy

papersTable of Contents


Integrating Human Factors in Customer Support Systems development using a multi-level organisational approach.

Anne Miller
Customer Support Platform Usability Team
Information Technology Group
Telstra, Australia

E-mail: amiller@vitgdts1.telecom.com.au


ABSTRACT
Integrating usability into software development projects involves working across multiple organisational levels. Aligning the Customer Support Platform Usability (CSPU) Teams objectives with those of the organisation allowed more effective integration of usability activities within project teams. Primarily, corporate alignment provided a legitimate mandate for the CSPU Team to develop standards and guidelines, and to require that usability activities be undertaken by project teams. However, at the project team level, integration was achieved by definition of roles, activities and processes according to the objectives, constraints and processes of project teams. Achieving common ground in project teams involved a willingness to work with, and to actively adapt to both organisational and project based needs.

Key words
human factors; usability; corporate mandate; Graphical User Interface;
systems development life cycle; standards and guidelines; resourcing.

INTRODUCTION
The CSPU Team was formed within Telstra in August 1994 for the purpose of implementing usability in Customer Support Systems, (i.e. computer systems used by Telstra’s customer facing staff). The Teams establish-ment and development bears a striking similarity to efforts described by Billingsley (1995), at Union Pacific Railroad. Like Billingsley (1995), formation of the CSPU Team:

However, unlike Billingsley's (1995), experience at Union Pacific Railroad, Telstra:

In this climate, successful integration of usability principles within project teams required substantial cultural change across multiple levels within Telstra. This was recognised by Lindgaard (1995), as being "no mean feat".

Lindgaard (1995), states that, "if we want to ensure serious application of human factors principles in all telecommunications products, we are forced to bring about significant cultural change in design and development philosophy and approach in our organisation" (p. 362).

Effecting cultural change required:

ORGANISATIONAL CONTEXT
Historically within Telstra, 1970’s regionalisation and organisational separation into relatively autonomous business units, had resulted in poor communication and a relative lack of organisational integration. This lead to fragmented and often isolated development of customer support applications. The net result was that applications were built using a variety of platforms that were not always consistent or compatible with one another, despite similar functionality.

By the late 1980’s, senior management was addressing system inconsistency and incompatibility problems across the organisation. A key element in this process was Telstra's Information Technology Group (ITG), who in 1991, published the 'Overall Systems Architecture Strategy' (the 'Blue Book'). The 'Blue Book' set out a strategic framework for integrating technological operating systems around largely independent core business activities.

The second phase of the 'Overall Systems Architecture Strategy' was to link the separate technology bases. The most recent update of this strategy, the 'Overall Systems Architecture Phase 2 Update', 1994, defined the architectural framework for all computer systems within Telstra. Identification of opportunities for systems rationalisation through standardised convergence, and reuse was also a major objective.

In addition, Dr. Deborah Zinn, (a senior manager in ITG’s Strategy and Direction department), outlined, within the Overall Systems Architecture, a sub-strategy for Customer Support Systems. Customer Support Systems are defined as, “applications that support the interface between the Customer and Telstra for all Customer Sales, Service and Support activities” (p. 10, Overall Systems Architecture Phase 2 Update, 1994).

Within this context, Dr. Gitte Lindgaard, (a manager at TRL), argued that usable interfaces were critical to effective customer support. As a consequence, usability was identified as one of five key principles driving development of all applications used by customer facing staff. Under this principle usability specialists would be employed to:

"implement standards for usability leading to ease-of-use and a common look and feel across all systems used by customer facing staff". (p.10, Overall Systems Architecture Phase 2 Update, 1994).

This single phrase was critical. It provided a corporate mandate for usability within Customer Support System development projects within Telstra. The next step was implementation of the mandate.

THE CUSTOMER SUPPORT PLATFORM USABILITY TEAM
Dr. Lindgaard and Dr. Zinn were highly instrumental in advocating for, and forming the CSPU Team, which was initially comprised of two human factors researchers from TRL and three members from the Hiser Consulting Group (HCG). The CSPU Team later recruited four new staff members over a twelve month period. These people, with other recruits will form the permanent CSPU Team, allowing TRL and HCG personnel to be gradually phased out. Other external consultants were recruited on a needs basis.

The professional composition of the CSPU Team was an important ingredient in a multi-level organisational approach. Experimental psychologists from TRL contributed a sound understanding of Telstra's corporate culture and a strong track record in human factors research. However, these people had very little experience applying human factors methods outside of the research environment. Conversely, the HCG was known to Dr. Zinn as having considerable expertise in designing Graphical User Interfaces (GUI's), while working closely with project teams in large public and private sector organisations. The HCG contributed highly experienced human factors specialists, and an ergonomist. Later recruits contributed skill in industrial and organisational psychology, industrial and interaction design, systems analysis, computer programming, and education.

This multi-disciplinary mix of expertise allowed high levels of flexibility and responsiveness at levels of:

The CSPU Team's first goal was to demonstrate that usability could be integrated within existing project development teams.

Four strategically significant Customer Support development projects were targeted for usability intervention, involvement or activities. All projects were building applications for use by customer facing staff, and all involved the development of GUI's. One project was 'green-fields', (i.e. it had no precedent prior to conceptualisation), one was a project of high organisational significance that was near deployment, (organisational significance being defined according to the number of users affected, i.e. in excess of 2000 internal staff, and the estimated total budget of the project), and two projects in early stages of development.

One CSPU Team member was allocated to work on-site with each of the targeted project teams. Often, inexperienced CSPU Team members worked with more experienced staff, who acted as mentors. On-site Team members returned to a central location for a four hour meeting once per week to provide feedback about on-site involvement; to raise issues and to develop group strategies for solving a multiplicity of problems. Many methods and techniques used in the field, as well as screen design ideas, were developed in these sessions that often followed a workshop type format. These sessions were an important mechanism for knowledge sharing and transfer among CSPU Team members.

IMPLEMENTING SOFTWARE USABILITY IN PROJECT TEAMS
While allocating CSPU Team members to project teams was a relatively easy task, integrating usability activities within project development was another issue entirely. Project teams had not been required to include usability specialists in development teams; they did not understand the role of usability specialists and were concerned that inclusion would increase development time and expense.

Overcoming these obstacles involved a two pronged approach.

The first approach involved 'legislating' for the inclusion of usability expertise in project development. At Telstra projects pass through a project control process known as the Gating process. This is administered by ITG and ensures that all significant projects conform to the Overall Systems Architecture Strategy. The mechanism by which conformity is ensured is the allocation of funding, i.e. projects must complete key activities in order to receive funding for the next stage. By including usability tasks (e.g. user profiling; task analysis) as requirements for progression to the next gate, CSPU Team involvement became essential, as these tasks required specialist expertise.

The second approach was education. Required inclusion could not ensure that usability principles would actually be embraced by development teams. And further, mandatory requirements tended to cast the CSPU Team in the role of 'usability police'. As a result CSPU Team services were often requested too late, and recommendations involved substantial changes to work already completed. This fuelled the view that usability was too expensive and caused unnecessary delay.

To change this perception the CSPU Team organised presentations to project, as well as business unit managers. Presentations focussed on defining usability and its benefits in reduced training costs, enhanced user productivity and greater likelihood of user acceptance. The latter emerged as a key area of concern for project managers, since poor user acceptance often negatively affected the professional reputations of project team members, both from the point of view of negative user attitudes, and from the perspective of increased cost in post-deployment redesign work.

The CSPU Team also embarked on a policy of project team participation in almost every area of work. The Team strongly pushed for project team member involvement in usability activities. Project representatives were recruited as advisers, observers, as well as data collectors. This had the dual outcome of educating project members about usability activities, and by exposing these people to problems experienced by actual users, recommendations had a greater chance of being accepted as legitimate and justified.

In as much as project teams learnt about usability, CSPU Team members benefited from exposure to the day to day problems experienced by project teams. CSPU Team members knew that early involvement was critical to successfully integrating usability principles, but had only a broad understanding of development processes. The CSPU Team could not identify exactly what was required, when, and hence could not make accurate recommendations about specific usability activities or the timing of CSPU Team involvement.

Clearly the CSPU Team needed a better understanding of development processes. Drawing on Ernst & Young's Navigator System Series - Version 3.0*, (E&YNSS), documentation (a prevailing development method used at Telstra), and with the experience of HCG personnel, the CSPU Team explicitly defined usability activities in terms of phases of the systems development life cycle. This enabled the Team to:

Clearly defining the systems development life cycle also taught the CSPU Team the language of project teams. This allowed better communication of the Teams role and relevance.

An example of the effects of language differences, leading to poor communication, was the term 'task analysis'. Project managers, when asked whether they had completed a 'task analysis' as required by the Gating process, invariably replied that 'yes, they had'. When asked how this had been completed and by whom, CSPU Team members were told that business requirements had been documented by business analysts, and data flow requirements had been completed by systems analysts. This, according to Project managers, constituted a 'task analysis', but was clearly not a 'task analysis' as conducted by usability specialists. To avoid confusion the CSPU Team has since renamed its 'task analysis', to 'natural workflow analysis', which better describes and differentiates this process from other task related processes.

Systems development life cycle definition had a second outcome. The different groups comprising the CSPU Team, i.e. TRL and HCG, had very different approaches to applied research. TRL personnel preferred more experimentally 'correct' approaches, while HCG preferred a ‘user centred’ design approach. TRL expressed the need for accurately defined and well constructed studies that ensured accurate information and hence decision making. HCG emphasised the need for efficiency, project member acceptance of usability, and knowledge transfer.

At a more practical level, problems were being identified with some methods used in the field, particularly as the CSPU Team was encouraging project member participation. For example:

Identifying stages within the systems development life cycle played a key role in resolving these issues. Process definition allowed the CSPU Team to determine:

By focussing on questions like 'what information do we need at this stage and why', the CSPU Team selected methods appropriate to this question and circumvented arguments about appropriate methods based purely on ideological or philosophical grounds.

Uncertainty about the utility of different levels of measurement, e.g. population versus sample measures, also emerged as an issue. In particular, project teams had been requesting information about the cost justification of software usability. As the CSPU Team had not conducted cost-benefit analysis studies previously, there was a need to establish usability benchmarks by which benefits could be quantified early in the development life cycle.

The question became 'What variables could reflect benefits of usable applications and how should these be measured?' Possible alternatives included staff turnover and attrition rates, and formal training time; this data being collected at the level of the various customer facing staff populations. Other indices included comparisons between mean or median common task completion times prior to application development and post application development; this data being collected at the level of samples.

More recently, the number of post-deployment change requests resulting from interface inadequacies, e.g. failure of customer facing staff to perceive critical information as a result of inappropriate screen placement, has also emerged as a possible indicator of benefit. It is expected that the number of post-deployment interface change requests would be much less for projects including usability expertise, than for projects with no or limited usability involvement. Fewer change requests would reduce post-deployment cost, given that interface redesign is said to be more expensive after deployment, than if inadequacies had been identified and resolved earlier, e.g. during prototyping.

Hence, defining the systems development life cycle allowed the CSPU Team to better understand the types of information; appropriate levels of measurement and the most efficient tools and techniques required at different phases. Using this approach also circumvented heated debate about the appropriateness of different types of methods and measures.

THE ROLE OF STANDARDS.
Prior to establishing the CSPU Team, the HFRG at TRL had published the Human Factors Kit, (HFK), as a set of standards and guidelines for the design and implementation of usable technology within Telstra. The HFK is discussed in detail by Burger (1995), and Lindgaard (1995). Suffice it to say here, that the HFK was a decisive and important instrument with a range of strategic benefits for usability i.e.:

MAKING THE STANDARDS A REALITY WITHIN PROJECTS.
Despite its success the HFK had several problems:

Also prior to formation of the CSPU Team, Telstra had commissioned, on the recommendation of Dr. Zinn, the HCG to develop a set of standards, (the Customer Support Systems Usability Standards and Guidelines), specific to software design for Customer Support Systems. These standards leveraged initial work done in the HFK, but were written specifically for use by projects teams designing either Graphical or Character Based User Interfaces for customer facing staff.

However, the introduction of standards has long been difficult. For those who write or advocate them, standards embody principles that move a technological device towards an abstracted ideal, for example, 'common look and feel'.

For those required to conform to them, standards can be perceived as a set of inflexible rules, a time consuming nuisance, and a limitation on the creative input of non-human factors interface designers. Initially the task for the CSPU Team was not so much to ensure that the standards were adhered to, (the policing role), as much as it was to evangelise and make concrete the 'common look and feel'. This involved addressing questions, such as:

The answer to these questions was 'yes', but only if there was a visual embodiment of the standards, that could be tested with users and demonstrated to project teams. This involved designing a standards compliant electronic prototype, the Corporate Graphical User Interface, (CGUI).

However, if the prototype was to be successful it needed to account for the same problems that project teams encountered. Most importantly, it needed to fit the needs of users.

Early in its life, the CSPU Team worked with a 'greenfields' project for customer facing staff dealing with Commercial (small business) and Consumer (residential) customers. Involvement in this project allowed CSPU Team members to lever off already completed analysis work, and to conduct participatory design sessions similar to those described by Holtzblatt and Jones (1993), and others, (see, Constantine & Lockwood, 1994). This involvement resulted in the first iteration of the 'common look and feel' interface, and allowed informed elaboration of the first version of the Customer Support Systems Usability Standards and Guidelines.

However, as the CSPU Team became involved with a broader range of projects, the 'one-fits-all-situations' prototype began to founder as a result of different user requirements. For example, applications designed for use by customer facing staff servicing large corporate customers required far more sophisticated design solutions compared to applications designed for use by customer facing staff servicing domestic consumers.

In resolving this problem, the CSPU Team observed that applications shared some commonalities irrespective of situational complexity. For example, function and 'hot' key allocation, colour schemes, application entry points, gross navigation techniques, the look and behaviour of application and document windows, and dialogue boxes could be standardised within a skeletal framework, irrespective of the complexity of customer requirements.

Variation arose in the behaviour of on-screen elements. For example, widgets allowing customer facing staff to order single services for a domestic consumer required design solutions quite different to solutions allowing single or multiple services to be ordered for organisations divided into regions and then into individual branch outlets.

More recently, the CGUI has taken the form of a basic or skeletal framework, into which a range of design options of varying complexity can be modified and inserted according to need.

An added advantage of this approach is ability to demonstrate various design possibilities. To illustrate, this writer recently had a delightful experience, (while facilitating a design workshop with users and developers), of meeting the challenge that "the current application works perfectly well, thankyou", with a demonstration of a range of more efficient alternatives. This demonstration immediately broadened participants' horizons of what was possible. It also generated a range of sometimes 'wild' additional alternatives, and a complete change of attitude from scepticism to creative involvement on the part of participants. Demonstration allowed participants to move beyond constraints of the familiar.

Standards compliance does not, however, ensure usable interfaces. To ensure that new design solutions are usable, all prototype applications, with CSPU Team involvement, are tested for usability as part of the development process.

Another outcome, in the evolving story of the CGUI are requests from project teams for reusable on-screen objects templates. This will increasingly involve the development of libraries of re-usable on screen elements and widgets; icon libraries; and libraries of standard language and terminology. These initiatives are expected to:

The next step in development of the standards and the CGUI is cost-benefit justification studies.

By shifting the emphasis on 'Standards - a set of rules' to 'Standards - a practical and creative tool' more applications are expected to converge to a standards compliant user interface.

RESOURCING USABILITY
With greater acceptance of usability within project teams, requests for CSPU Team involvement has increased.

As a small group (the CSPU Team is not anticipated to grow beyond 8-10 permanent members), resourcing has become a problem. The Team can not place members in every project. Education in the form of skills transfer provides a solution.

Recently CSPU Team members developed a series of workshops aimed as providing project teams with a 'do-it-yourself' capability. Two of the workshops target project managers and key decision makers. These workshops reinforce usability definitions, benefits and role in the development process. Four workshops, using a 'tell-show-do' format, introduce allocated or interested project members to usability skills, e.g. conducting design workshops; natural workflow analyses, etc.

The workshop series runs in tandem with a strategy borrowed from Mrazek and Rafeld (1992), for introducing non-CSPU Team usability representatives or ‘champions’ into project teams. These people are anticipated to carry out day to day usability tasks under CSPU Team supervision. The Team is also setting in place strategies for establishing a usability special interest group and a usability network for within project usability representatives.

CONCLUSION
In conclusion integrating usability across an organisation involves establishing 'common ground' between usability professionals and:

A prime reason for CSPU Team success has been its willingness to work with and adapt to the needs of the organisation as a whole.

ACKNOWLEDGMENTS
The author acknowledges the contribution of the Telstra Research Laboratories, specifically the Human Factors Research Group and the Hiser Consulting Group to the activities undertaken by the CSPU Team. In particular the author acknowledges the support and advice offered by Helen Kieboom (HCG); Dr. John Craick (TRL) for some of the organisational context; Dr. Joe Havloujian (TRL), Daniel Szuc (CSPU Team) and Sheryl Lumb (Interactive Courseware Pty. Ltd.) for encouragement and assistance with editing; Dr Deborah Zinn, and Dr. Gitte Lindgaard for their energy and vision, without which this paper would not have been possible.

REFERENCES

  1. Burger, K. (1995). Applying Usability Standards and Guidelines within a Multi-disciplinary Team. Conference Proceedings: Human Factors in Telecommunications, 15th International Symposium on Human Factors in Telecommunications. Melbourne.

  2. Billingsley, P. (1995). Starting from scratch: Building a usability program at Union Pacific Railroad. Interactions, Oct ‘95.

  3. Constantine, L. & Lockwood, L. (1994) Essential modelling for user interface design: essential use of cases and use context. OZCHI'94 Tutorials. Melbourne: Computer Human Special Interest Group of the Ergonomics Society of Australia.

  4. Ernst & Young International. (1995). Ernst & Young Navigator Systems Series - Version 3.0: Client/Server Analysis using a 4GL/GUI. USA: Ernst & Young International, Ltd.

  5. Holtzblatt, K & Jones, S. (1993). Contextual Inquiry: A participatory technique for system design. In Schuler, D & Namioka, A. (Eds), (1993). Participatory design, principles and practice. New Jersey: Lawrence Erlbaum Associates, Inc.

  6. Lindgaard, G. (1994). Usability Testing and System Evaluation: A Guide for Designing Useful Computer Systems. London: Chapman & Hall.

  7. Lindgaard, G. (1995). Cementing human factors into product design: Moving beyond policies. Conference Proceedings: Human Factors in Telecommunications,15th International Symposium on Human Factors in Telecommunications. Melbourne.

  8. Lund, A.M. (1994). The evolution of broad-band work in Ameritech's customer interface systems and human factors department. In M.E. Wiklund (Ed.), (1994) Usability in Practice: How companies develop user-friendly products. Cambridge, Massachusetts: AP Professional.

  9. Moyal, A. (1984). Clear across Australia: a history of telecommunications. Melbourne: Thomas Nelson Australia

  10. Mrazek, D & Rafeld, M. (1992). Integrating human factors on a large scale: Product usability champions. Proceedings CHI'92. p. 565-570.

  11. Muller, M. (1993). PICTIVE: Democratizing the dynamics of the design session. In Schuler, D & Namioka, A. (Eds), (1993). Participatory design, principles and practice. New Jersey: Lawrence Erlbaum Associates, Inc.

  12. Telstra. (1994). The Overall Systems Architecture: Phase 2 Update. Melbourne: Telstra Australia