|
|
David Frohlich *, Richard Hull**
In our group we have been exploring an alternative paradigm which we call 'handwriting-as-data'. Within this view the pen is used to input handwriting, drawings or gestures which remain unprocessed by the computer and yet available for manipulation by the user. The job of the computer is then to assist the user in this task. To this end we have developed an algorithm for searching electronic ink called scribble matching [2] This takes a fragment of unrecognised handwriting (or scribble) and compares it to a larger set in order to return one or more similar fragments. The algorithm currently returns a single correct match 97% of the time, from a 200 word dictionary. Scribble matching can therefore be used to perform scribble look-up in a variety of pen-based applications including notetaking, form-filling, messaging, and PIM.
Two other research groups have reported attempts to develop a similar facility within the same paradigm [3,4], but there is as yet no published data on the usability or usefulness of scribble look-up in a real application. In the rest of this paper we describe an experiment to collect such data.
Figure 1. An example fax cover sheet
For both phoning and faxing users had to retrieve a number from the phonebook. They could do this in three ways:
(a) Browsing: by clicking on the phonebook icon to open the phonebook from which they could select numbers directly before returning to the phone or fax forms.
(b) Text look-up: by clicking the keyboard icon next to the 'Full name' field on either form, typing a textual label on a soft keyboard display and then pressing the 'Look up' button.
(c) Scribble look-up: by writing the scribble label into the 'Short name' field on either form and pressing 'Look-up'.
A two factor repeated measures design was used to vary the task type (phone or fax) and the task methods (browsing, text, scribble or all) during the main session. Thus, each subject performed alternating phone and fax tasks on all 40 numbers in the phonebook. The numbers were selected in random order by the system and prompted using the corresponding textual label. The trials were organised into 4 blocks of 10 to vary the methods available. In the first three blocks subjects were constrained to use either browsing, text or scribble look-up alone (in randomised order across the group). In the final block of 10 trials subjects were allowed to chose between all these methods. A combination of objective and subjective measures were recorded for analysis.
| METHOD | Scribble | Browsing | Text |
|---|---|---|---|
| Look-up choices ('All' block) | 98 | 32 | 12 |
| Mean Rank | 1.17 | 2.25 | 2.58 |
| Mean Rating (%) | 83 | 67 | 59 |
Such preference for Scribble look-up is especially interesting in the light of the actual efficiency of each method. In fact, Scribble and Browsing were faster than Text look-up (main effect on times F=12.91, df=2, p<0.001), but Browsing was more reliable than Scribble and Text (main effect on errors c2=23.8, df=2, p<0.001). One reason subjects still preferred Scribble look-up to Browsing is that it involved less cognitive effort:
CS "I hate looking at a screen. It's much easier to have the machine do it for me".
Another reason is that Scribble look-up was seen to be more appropriate than the other two methods. This was especially noticed on the fax task where writing the recipients name was seen as part and parcel of writing the cover note:
KS "To be able to sit and extract the number and do the text all in one go.. that certainly is brilliant"
Further analysis of the reasons for errors in Scribble look-up revealed a common information retrieval problem. Subjects tended to forget their original handwritten phonebook labels, and attempted to search on the wrong target scribble. For example, they searched on Doctors for Docs., Bob C for Bob Coombes, and Budget for Car hire rtn. In these cases, comprising 17% of all Scribble look-ups, the required entry would not even appear in the top 5 nearest match list (behind the 'Correct' button) because the referent scribble was so different to the desired scribble. In another 3% of cases, the system returned the wrong match from the correct target input; leading to an 80% success rate for each Scribble look-up. However, both types of errors were usually repaired through follow-up Scribble searches. These typically involved the user re-writing the previous target label. If the input label was correct the system would usually get it right second time, whereas if it was incorrect the user would realise their own mistake on the second failed output and correct it on a third attempt. Overall this resulted in a 97% task success rate using Scribble look-up.
These findings demonstrate enthusiastic user acceptance of scribble matching whilst flagging a human memory limitation common to all retrieval methods. Within our group they have encouraged us to think of ways of minimising the effect of wrong target entries and to research other applications of scribble matching.