CHI 97 Electronic Publications: Technical Notes
A Factor Analysis of User Cognition and Emotion
Judith Ramsay
The Center for People and Systems Interaction
South Bank University
103 Borough Rd., London, England
Tel. + (0) 171 815 7421
E-mail: ramsayja@vax.sbu.ac.uk
ABSTRACT
Fifty two statements of cognition and emotion were gathered from computer users during breakdowns in understanding during interaction. They were reduced by factor analysis to a set of ten themes. The themes show the extent of discomfort experienced during breakdowns. The themes now form the backbone of a checklist of cognition and emotion, short enough in length to be administered during interaction. This work forms a move towards understanding and ultimately alleviating discomfort felt during human-computer interaction.
KEYWORDS
Cognition, emotion, factor analysis, human-computer interaction.
© Copyright ACM 1997
INTRODUCTION
Despite important moves towards better user interface usability, and a deeper understanding of the psychology of various types of user groups, there still exists a need to understand the subjective, emotional state of the computer user. Resistance to using a computer due to lack of knowledge can be overcome through education [1]. However, computer-related anxiety represents a more challenging problem [2, 3]. It is estimated to affect up to 30% of the U.S. workforce [3], causing substantial loss of revenue. For this reason it is important to design and develop a tool to capture and measure these thoughts and emotions. This represents the first step in a chain of interventions which might alleviate this problem.
Formal empirical research, unfortunately, has been sparse in this area [4]. What is clear however, is that computer-related distress can lead to serious performance problems such as declines in motivation and morale, increases in mistakes, absenteeism, physiological arousal and debilitating thoughts [4]. Other cognitive factors that have been associated with emotional distress include patterns of dysfunctional emotional processing [5], an increased occurrence of irrational beliefs [6], self-debasing attributional patterns [7] and increased negative self-statements [8]. As Sarason [9] has noted, such self-defeating thoughts and feelings lead to sub-optimal performance.
The goal of the checklist
The aim of the checklist is to discern those cognitions and emotions experienced during breakdowns (problems) in interaction, and their intensity. The aim of developing a brief checklist is to maximize response rates, and minimize response errors and unanswered items, thus improving its validity and reliability in use.
Method
Twelve computer users thought aloud whilst using a computer. Their cognitions and emotions were recorded when they experienced problems. Thirty three members of the general population were also interviewed (age range 17-61, average age 22). They were asked about their experiences with computers, the thoughts and emotions that they typically felt, and how these experiences compared with other stressful and non-stressful situations they experienced. In addition, three focus groups were held. Each time, the group consisted of three males and three females who discussed their experiences when using computers. The agenda for the focus groups was the same as that for the interviews. This allowed the author to amass a series of critical incidents and statements. Forty one statements resulted, along with a set of eleven statements about physiological events. This gave a total of fifty two statements.
Results
The list of fifty two statements was reduced by principal components analysis to highlight the main, underlying themes at work. The set of fifty two statements of cognition, emotion and bodily state was put into the form of a feature checklist of items, each with its own eleven point rating scale. It was then administered to thirty four computer users (21 male, 13 female) when they had problems using their computer. The checklist was indeed excessively long at this point, with many statements meaning the same thing. The aim was to obtain a rating for each checklist item from each individual user. The resulting data matrix of raw scores was then reduced by factor analysis to produce the factor matrix.
The factor analysis
For an item to qualify as a factor, it had to reach the four percent level. This is an arbitrary but standard figure which means that 73.5% of the variance was explained. All factor loadings exceeding a positive criterion level of 4% were noted, as well as all negative loadings in excess of -4%. The cut-off point in the scree test occurred after ten eigenvalues. The resultant factors were labeled with an appropriate theme label. The strength of each of the loadings onto the factor is in brackets.
Factor 1 (11.2% of the variance)
The items involved in this factor related to transient depression, thus it was termed the bad day factor.
Factor 2 (8.8% of the variance)
This factor was about going to pieces, being confused, but of not giving up hope. It was termed confused but not hopeless.
Factor 3 (8.4% of the variance)
This factor was about looking stupid in front of others.
Factor 4 (7.3% of the variance)
The sole loading item was about having a blank mind.
Factor 5 (6.6% of the variance)
This factor indicated that everything was going along well, with feelings of involvement in the task.
Factor 6 (6.5% of the variance)
The items loading onto this factor indicated feeling tense and nervous.
Factor 7 (5.7% of the variance)
The two positively loading items involved feelings of stupidity when asking others for assistance.
Factor 8(5.5% of the variance)
This factor related to being upset, and sweating.
Factor 9 (4.7% of the variance)
This concerned an increased heart rate, and feelings that the task is impossible.
Factor 10 (3.8% of the variance)
This appeared to be a second social-reflection factor, with feelings of discomfort and fears of looking silly. It is distinct from the previous social-reflection factor, i.e. factor three, as factor three indicated bad social reflection but without the accompanying anxieties of factor 10.
Validation of the factor labels
Two independent raters then each labeled the resultant factors with an appropriate theme label. The correspondence between the themes of the independent raters and those of the author were 50% (5/10) and 60% (6/10). Discussions between the two raters and the author led to representative statements being given to each of these factors or themes. These representative statements then became the content of the eventual, reduced ten-item feature checklist.
DISCUSSION
The outcome of this investigation is two-fold. Firstly, it yielded a set of ten factors or themes which are representative of usersŐ states of mind during breakdowns in understanding when interacting with computers. Secondly, when these dimensions are assigned statements or sentences that capture the essence of the dimension, they form a feature checklist. This checklist is short and concise enough to be administered in an unobtrusive manner during interaction. It is intended that the checklist be used preferably in the early stages of the design process, during usability evaluation. Used alongside other evaluation tools, it should pick up on the more subtle, qualitative nature of the interaction. Until now, this important information has proved too fine-grained for even the best user questionnaire to elucidate.
Applying the results
As we move towards the millennium, the Chi community is increasingly intent upon closing the gap between technologically poor and technologically privileged groups. Attending to sources of distress during interaction is a generic starting-point for reducing this gap. By focusing upon the nature of these groups and their respective requirements, the Chi community will advance towards a state of greater equality amongst its user groups.
REFERENCES
1. Henderson, R. D., Deane, F. P. and Ward, M. J. (1995) Occupational Differences in Computer-related Anxiety; Implications for the Implementation of a Computerized Patient Management Information System Behaviour and Information Technology, 14, (1), 23-31.
2. Harrington, K. V., McElroy, J. C. and Morrow, P. C. (1990) Computer Anxiety and Computer-based Training: A Laboratory Experiment Journal of Educational Computing Research , 6, (3), 343-358.
3. Logan, R. J. (1994) Behavioural and Emotional Usability. In M. Wiklund (ed) Usability in Practice Cambridge, Ma. Academic Press.
4. Raub, A. C. (1982) Correlates of Computer Anxiety in College Students Dissertation Abstracts International, Howard, G. and Smith, R. Computer Anxiety in Management: Myth or Reality? Communications of the ACM, 29, 611-615.
5. Clark, D. M. and Teasdale, J. D. (1982) Diurnal variation in clinical depression and accessibility of memories of positive and negative experiences. Journal of Abnormal Psychology, 91, 87-95.
6. Nelson, R. E. (1977) Irrational Beliefs in Depression. Journal of Consulting and Clinical Psychology, 45, 1190-1191.
7. Seligman, M. E. P., Abramson, L. Y., Semmel, A. and von Bayer, C. (1979) Depressive attributional style. Journal of Abnormal Psychology, 88, 242-247.
8. Hollon, S. D. and Kendall, P. C. (1980) Cognitive self-statements in depression. Development of an automatic thoughts questionnaire. Cognitive Therapy and Research, 4, 383-395.
9. Sarason, I. (1975) Anxiety and self-preoccupation. In I. Sarason and C. Spielberger (Eds.), Stress and Anxiety (Vol 2). Washington DC: Hemisphere Press.
CHI 97 Electronic Publications: Technical Notes
