A Psychologist Views Human Processing: Human Errors and Other Phenomena Suggest Processing Mechanisms

A PSY<.:HOLOGIST VIEWS HUMAN PRO<.:ESSING: HUMAN t:RRORS AND OTHER PHENOMENA SUG<;EST PRO<.:ESSING MECHANISMS

Donald A. Norman

Program in ('ognit1vc Sl'icnl'c University of California . San Diego La Jolla. California 92093

when we listen first to this conversation. then to that , or when driv­ ing a car while conversing. looking first at the road, then at the passenger. The interesting cases arise when neither solution seems applicable because the several tasks interact with one another. Some kinds of mental activity can cause pupil size to increase (as cxperl poker players know) . We hear better in the direction in which we are looking (even if we do not turn the head) and, con­ versely. it is hard to ignore the sounds from the direction in which we look , even when we are trying to listen to something else [121. Novices cannot tap different rhythms with different hands, but ex­ pert musicians can. Thoughts can intrude upon actions. The initia­ tion of action can interrupt thought. Emotions -- and even such things as hunger -- can disrupt thought. We tend to remember sad things when we arc sad and happy things when we arc h:tppy. Different parts of our system intrude upon others, :tpparently in subtle. continuous ways, a point exploited by Freud.

A. Studies of attention (psychology) and time-sharing (comput~r science) do not provide helpful information

These characteristics of real behavior pose some interesting and important puzzles for students of hur.-~an information process­ ing. We know little about how such multiple goals and tasks get scheduled and accomplished. There has not been much study of this aspect of behavior. This is especially true in view of the psychological literature on simultaneous :~ttention that argues st rongly for limits on our ability to do several tasks at any one timt.: . I myself have argued such a point, telling audiences of undergradu­ ates how people are limited to doing roughly one thing at a time. Of course, while I say this , I am pacing back and forth in front of the class, avoiding the table and chair in my path, juggling a piece of chalk from hand to hand, planning the remainder of the lecture. and worrying about how I am going to get through the demands of the rest of the day. My actions contradict my speech , but in actual­ ity, it is even worse. Any one of the "si ngle'' things I am doing is a complex set of overlapping activities. The act of speaking, for ex­ ample. involves many components, many of which should really be considered separate tasks. In speaking, there is the high level plan­ ning of the utterance, the formation of the structure of the sen­ tences, the proper morphological selection and construction of the words, and the complex control of the speech organs and of the numerous muscles in the face. mouth , throat, and chest that must operate in parallel with overlapping wntrol signals. Thus, even a so-called single task is really many simultaneous tasks.

Psychologists deal with this apparent contradiction between theoretical belief and reality by talking of the distinction between automatic and non-automatic actions, stati ng that automatic acts arc not under conscious control and do not require attentional resources. As a result. there is no limit on how many of these can be done at any one time, as long as there is no conflict in the ust.: of any particular physical or psychological structure. The trouble with this explanation is that it doesn't tell us anything about how it is actually accomplished. To be polite to my field, I will make the

ABSTRACT

I argue from studies :Jf human performance. including slips of action and skilled typing that human processing struclUres are of a special son, with weak binding between functions and arguments. with strong excitatory and inhibitory interactions among simultane­ ous processes, and with the pans of action sequences neither strongly ordered nor tightly coupled. I argue that analyses of hu­ man performance imply a class of processing structures quite different than is commonly envisioned within Anificial Intelligence.

L How do people do more than one thing at a time?

An important aspect of everyday behavior is that we do several different activities at the same time , oftentimes for simul­ t<meowi (and possibly conflicting) purposes. Even when we do not attempt simultaneous actions. we still might be planning or review­ ing one set of things while performing or accomplishi ng another. We delay and defer goals or actions as needed, waiting for appropri­ ate times for them to be accomplished. This occurs for several rea­ sons. Some biological goals do not need to be satisfied at any par­ ticular instant, but within reason, can be executed at convenience (e.g., such things as eating, sleeping, or toilet activities). Some dai­ ly tasks have similar characteristics (e.g., going to the bank or post office, purchasing some needed item). Some tasks have to be de­ ferred because there is not sufficient time or information to com­ plete them during one session of work (e .g., writing a scientific pa­ per, reading a book, learning a complex task). Finally, even for tasks that arc continually active from stan to completion , they may span such a long duration that other things are also done along the way, and the individual components or the major task may have to wait for minutes or even hours. before being executed.

These problems appear to be analogous to the scheduling problems of modern real-time computers, and some of the analyses from that field arc relevant. However, the human is a special kind of biological processor. and I suspect that suprisingly little of what we know of time-shared computers applies to the human . The difficulties in doing two or more tasks at the same time are well known . There are only two ways that a system can do two or more things together at the same time. One way is to have sufficient pro­ cessing machinery that the two tasks use different resou"es and do not interact. The second way is to switch back and fonh between the two, saving the complete status of the current state before switching tasks and then restoring the state complciCly when switching back.

Which method do people usc' 1 There is clear evidence for both. Different processing structures control walking and talking. eating and seeing. The same processors are switched among tasks excuse that this explanation is still at an infant stage of develop­ ment. The statement allows us to reconcile our observations of real behavior with the theoretical belief in attentional limitations by say­ ing that, well, not everything requires attention, not once it is well learned. Regardless of what you might think of the statement and of the lack of specificity in the arguments, there is another problem with the approach; the attentional limitations are only part of the problem. We still don' t know how any organism can simultaneous­ ly perform many tasks. What kind of structures are necessary, and which are really present in the human? How can we account for the errors that people make?

Rc..,carl·h ~uppmt wa:-. provalcd hy the Oflk~ of Naval Rcsc;m:h and the Na val Air l>cvdopmcnt ( 'enter under l'O!Hfill'l N00014·79-C'-0313

( 'opyn~ht 19M 1. I >on aid /\ . Norman

Now turn to computers and, in particular, the work in Artificial Intelligence. Let me quickly assure half of my audience -­ and warn the other half -- that AI doesn't have any idea of how to handle the problem either. The relationship between the study of Artificial Intelligence and human intelligence should go two ways, and although psychologists have often taken more from AI than AI has taken from us, I think the direction of the information flow is changing. In this case we are equals; there is only a weak ebb between two stagnant pools.

Most of the intelligent programs that have so far been developed within AI are single minded, experts in their single domain of inquiry, but unable to deal with any other domain. Even when there are systems that can deal with several different domains or sub-domains of a topic, they do them in a sensible fashion, one at a time, rather than in the inelegant, cluttered human fashion of attempting to think of everything at once, mixing up the concepts of one with those of the other. The virtue for the computer is elegance and power. The virtue for the human is creativity and flexibility.

B. Human error is suggestive of a special form of mechanism

I want to argue for a different kind of processing mechanism than is usually considered by people within Artificial Intelligence. In the end, it may not be wise to model many aspects of human in­ telligence with conventional processing structures. But before I get to that, let us review the argument.

The multiple-purpose, multiple processing aspect of our behavior leads to difficulties. I have already listed some phenome­ na that imply interactions among processing structures. In this pa­ per I concentrate upon the form of human errors. Thus, we make errors. We are easily distracted by events, stopping to do things we had not intended, or we are captured by habitual acts, performing them instead of the ones intended. At times, we can be data driven, responding to sensory signals, whether we intended to or not. This can be useful, for it allows us to react appropriately to unexpected events in the environment. It is not so useful when data driven processing interrupts our intended actions, at times so distracting us from our intentions that we neglect to complete them. These errors imply that we neither separate the tasks well nor switch completely among them. As a result , we intermix com­ ponents, lose track of our status on any given task . and oftentimes do the right thing on the wrong occasion.

Errors give insight into the system, for they offer powerful clues as to the operation of the underlying mechanism. We need not agree with Freud's view that "the meaning in them is unmistak­ able, even to the dullest intelligence, and strong enough to impress even the most criticul judgment", but we can still agree that they are strongly suggestive. Errors can be divided into sever;tl different categories. I divide errors into two major classes : mistakes and slips, with the division being whether the error occurred prior to or artcr the formation of the highest level intention. I define a mis­ take to be an error in forming the intention. Thus, a mistake can result from knowledge that is erroneous or incomplete. either in the information that the person brings to the situation or that is available from the environment. The mistake can also arise in the psychological mechanisms of decision and planning that are in­ volved in the formation of the intention. I define a slip to be a failure in carrying out the intention properly. That is, the appropri­ ate action is started, but somewhere along the path of execution it is diverted or deflected.

There are several collections of slips (2,3, 9,11). The in­ stances are both humorous and informative. A business executive roared "Come in" instead of "Hello" when answering the telephone. A friend politely said "Come in" instead of "Sit down" when inviting a new person to join the two of us at a table in a hotel restaurant.' Pilots have raised the landing gear instead of the flaps. One person reported cleaning a fish and throwing the cleaned lish overboard, keeping the entrails. In preparing for a party, one person put the cake in the refrigerator and the salad in the oven. Computer users report numerous errors: typing commands into the text editor while in "insert" mode, or text while in "command" mode; deleting files instead of moving them. There are data-driven errors, in which the sight of something leads to an unintended action -- one of my students calls this the "parking spot error" : if you come across a parking spot while driving through a city, you may find yourself parked in it, even if you had no intention of stopping there. (The same student reports dashing into an elevator that hap­ pened to open its doors just as he .was walking by, even though he hadn't meant to take an elevator.) A reasonably common typing er­ ror is the "doubling error" : doubling the wrong letter in a word, yielding bokk or claas instead of book or class.

Examples of slips can be found in both speech and motor ac­ tions. One example is to select the wrong word, as in : "Wouldn't it be cheaper, I mean faster, to go that way?" I classify this as a "description" error. one that results from an error in memory re­ trieval. The word that was first retrieved shares features of the se­ mantic description of the intended word. The error can per­ severate, as in: "They have Chinee -- Japa -- Mexican food to go."

There are other forms of verbal slips. A blend occurs when two competing patterns are merged, as in "clut" which merges "close" and "shut ." In a Spoonerism, components of the words are interchanged, as in : "Ruman and Normalhart" instead of the in­ tended "Norman and Rumelhart ." (The examples come from Nor­ man (91. and Fromkin (2,3).)

Freud made the point that most errors have multiple causes, and that seems to be true of these as well. For the several people who have reported going to their bedroom to change clothes for dinner and finding themselves undressed and in bed, they may have been "captured" by performing the initial stages of a familiar habit and unconsciously completing the familiar instead of the in­ tended. but they may also have been unconsciously attempting to avoid the dinner. The invitation to "Come in" to the restaurant table could have been afl'ected by the fact that one of us was sitting in a semi -enclosed booth (and the person who made the slip so ar­ gued). In my experience, these subtle, clinical interpretations seem initially far-fetched. but are confirmed with surprising frequency by the people who make the slips. Thus the puzzle for those who wish to figure out the mechanism; how do different sources of informa­ tion interact to lead to slips (while also accounting for the fact that most of our actions are correct)?

The various phenomena I have described, plus others, imply that the parts of action seljuences are neither strongly ordered nor tightly coupled. That is, I think that the biological sysh:m is struclured so as to usc ambiguous information for memory search, 10 al­ low itself lo he responsive to mulliple sources of information, lo combine and overlap data paths. and 10 deliberately intermix what one would have thought lo be independent processing streams. Allhough these properties can lead 10 errors. I believe !hal they arc also exactly the sorl of thing thai gives us much of lhc power of human creativity and judgement, 10 allow us lo be lolcrunl of noise and of error, lo behave flexibly, lo respond in imaginative and creative ways lo novel events, and to be able lo shifl our strategies and behavior when the situation shifts.

· All th..: "iltps rt.!portt.!d in this paper have been ~.:ollcl· tcll with 'orne Gtrc a' to tll"l:U· r;u:y. omd with the ongi nal intention verified hy the perpetrator Sec the origtnal r uhll(."ation'i for details.

The basic concept is simple. We assume !hal the human in­ formation processing system is mediated by means of many separate processing structures, each of which can do only simple operations, bul each of which is coupled 10 numerous other struc­ tures. We call these structures schemas. and we allow each 10 have an activation value !hal excites or inhibits its neighboring schemas and is triggered into controlling an action sequence whenever the combination of ils activation value and the goodness-of-til of its specific trigger conditions exceed a threshold value. (For a closely related argument and description of computational structures. see (I).) For present purposes, all that is needed is I he understanding !hal there are independent processing structures, each capable of controlling action , and thai synchronization and cooperation among !hem is handled by activation and inhibition links among schemas. More discussion can be found in (7,8,9,10,13).

TABLE I CLASSIFICATION OF ACTION SLIPS (Adapted from Norman, [9))

• Slips in the formation of the intention
• Mode errors: erroneous classification of the situation
• L)c""ription errors : ambiguous or or incomplete spccifkation of the intcn­ tum
• II : Slips that resuh from faulty a~:tivation of schcmas
• Unintention<~l <~l'tivation
• I . Ci.!pturc errors: when the intended scqucrwc 1s 'iimilar to another . hcltcr learned or rnorc frc(.{ucnt sc~ucnl·c . thl' l<.tltcr OliiY g<.~in nm­ trol
• Dat<~-llnvcn ;u.:t iva tion : l.!xtcrna l cv..:nt.., ;u.:tr vatc \l'hcma'i
• J\"i"oooa ti vc activation : cur rcnlly al·tivc schcm;,1"' ;Kt ivatt: o ther.., with whil·h thi!y arc assodated
• l.o ... s o f al"tlv;ttiou
• For~cttmg an intenlion Chut con tinumg wi th the .. equcnL·c I
• Ml'iordcring the ~:on1ponent\ of a sequcm.:c
• . l . Leaving out stcrs 111 a ... cqucnre
• Kcpcati ng ~t crs in J scqucm:c
• Ill : Slith that re ... ult I rom faulty trig~cring of act ive schcmas
• Fal"ic tri~cring : a properly activated sL·hema 1s tnggcrcd at an inappropri - ate tunc
• SpooiH: ri-.;m., : rcvcr ... al ... of event l'Of111l(li1Cnt s
• Blcn~h : l"<111lhulat1Clll"' of l"Oillll<lllCilh from tw11 n1mpeting "'chcma ...
• .l Thuu~ht' leadint~. 10 a1.:tions: triggc r in~ of "'chemas only meant w hl.' thou~ht , not exCl'Uted
• Premature lrl~~crin~
• Failure to tngger
• The action was preempted by l'Ompcting schema~
• There wa .., msuffil'icnt activation
• The !rigger condition-.; failed to matr h

Action slips come in many different varie ties. I have al­ lempled the analysis shown in Table I , based upon a theoretical framework thai assumes thai actions arc caused by lhe activation and triggering or schemas.

II. Studies or skilled behavior provide more clues

A. Skilled typin.: has int~reslin.: properties

Another source of information about how people do simul ­ taneous actions comes from lhc study of ski lled !asks. One such task is expert typing, and detailed study of the typist reveals some interesting insight into the nature of skilled human performance. Typing is a single · task thai requires multiple control of I he 10 fingers and 2 hands -- there are 60 tendons and 30 joints involved simply in the movement of the fingers. Study of typing is today one of the major themes in our laboratory, and lhe analyses of typ­ ing errors and typing performance !ell us 4ui1e a bil aboul lhe na­ ture of cooperative interaction among simultaneous at·tivilies. AI this poinl I will only mention two aspects of skilled typing. One is the doubling error in which the wrong leiter in a word is doubled, so thai a word like book or mann~r is typed as bokk or maan~r. The other is the overlapping nature of the execution of the finger movements (4). The finger movements star! several leiters ahead of !heir scheduled arrival lime, oftentimes out of sequence of lhc final temporal order in which they are made. II is as if each finger starts as soon as it can towards its intended large!, and the hand ap­ pears 10 cooperate, configuring itself so as lo make maximum movement towards as many targets al a time as possible.

This !alter example is important, for il illustrates a situation in which simultaneous tasks cooperate rather than compete. This cooperation among possible competitive !asks happens frequently . Suppose you wish lo pick up several pencils and a piece of paper al lhe same time, using only one hand. The normal finger movements thai would be performed were only one object lo be picked up are modified to allow for cooperation among lhe fingers and hand to ac­ complish the multiple goal. I predict thai one of the changes thai occur in performance as a person becomes expert is a change from mutual compelilion of simullaneous actions to mutual cooperation. The behavior therefore changes from doing but a single action at a lime to overlapping, cooperative performance of several simultane­ ous acls.

TABLE 2 THE BASIC PHENOMENA OF TYPING (Adapted from Rumelharl & Norman, [14])

• The timing of keystrokes
• A People can t y~ very quickly .
• ('ro"is hand intcrstrokc intervt~ls arc -.;honer th<m those w1thm h:tntk
• C Within hand interstroke intervals appear to be a function of the rc:ll'h from one to the next.
• 1>. The t ime for a particular interstrokc interval can depend on the context in which it Ol'l'Urs.
• There is il negati ve correlation between the intervals on sun·e-.;sive strokes--especially when the a ltern;Hc strokes occur on alternate ho.111ds .
• Paucrn of Errors
• Transposition Errors
• Doubling Error
• C Altcrniltion reversal errors
• llomologous errors
• Capture errors
• Omission error..,
• ( i. Misstrokcs
• Ill . The general org;tnilation of typing
• Skilled typists move their hands towards the keys in parallel
• The units o f typing seem to he large ly at the word level or sm:.1llcr
• Sequences involvi ng ~o.·ross hand strokes seem to take longer to program than those involving only within hand strokes

Studies of typing reveal a number of phenomena that provide considerable constraints on the possible mechanisms that could be responsible for the actions. A lisl of lhe phenomena we have ex­ amined is presented in Table 2.

B. Th~ doubling error Implies that th~r~ is no type-token distinc­ tion

Consider the doubling error . How could it come aboul' 1 In our allempl lo devise a formal model of lhe typing process (141. we took special note of errors of doubling and alternation . (An aller­ nalion occurs in a word like th~s~ in which lhc ~ allcrnah.:s, hut when typed. the wrong leller is allcrnalcd, as in ths~s . ) The existencc of of doubling and alternation errors pose special problems. Consider the word book . According to our arguments, the word would be represented by schemas for each of the leiters : b o o k. It is easy to see how such a representation could lead to transposition errors (such as boko) but not to doubling errors. It would he easy lO make up a schema for a doubled leiter (so that the word would he represented by the schemas b double-o k), but this would not lead to the doubling errors either.

The doubling error turns out to have two major implications. First , it implies that there arc special schemas that signal the ex­ istence of doubled leiters, and that occasionally these schemas get applied to the wrong leiters. In a computational terms, this means that the binding between the arguments of the special schemas for doubling occasionally get made improperly. Second, the need for a special schema to mark doubled leiters implies a difficulty in having the regular leiter schema signal the double. Why isn't the word book represented by the schemas b o o k ? The reason would seem to be that this would require two instances (types) of the schema foro; the existence of the doubling error implies that such repeated tokens of a schema might not be possible .

Thus, the existence of doubling errors forced us to a pure "type" model , in which each leiter could only have a single keypress schema; the keypress schemas exist only as "types," with no "token" schemas. There must be a special schema that signals the presence of a doubled leiter. Moreover, there must be a weak binding between the special schema and the arguments upon which it operates. In our model, we let the binding be established via ac­ tivation values. with noise sometimes leading to errors in the bind­ ing. The existence of alternation errors led to the same conclusion; special schemas that signal the presence of alternating leiters, with a weak binding between the schema and its arguments.

II I. On possible psychological mechanisms

We see that there are several different aspects of skilled behavior:

1. Competition among actions, so that the doing of one thing inhibits the doing of another. For some combina­ tions of actions, the mechanisms required arc incompati­ ble, so that the competition is necessary and in these cases some sort of priority or inhibitory processes arc re­ quired.
2. Cooperation among actions. so that the operations of one action are modified to accommodate another. In this si­ tuation. most noticeable with skilled performers or with highly distinct, compatible actions, the simultaneous ac­ tions must engage in some process of "negotiation" to permit mutual performance. Thus, if one wishes to car­ ry several objects at the same time -- for example . several pencils, a piece of paper. and a cup -- the normal movements and positions of the fingers, hands, and arms will be altered to make the cooperation possible.
3. Slips of performance, so that the components of one ac­ tion sequence may get mixed up with the components of another, or the memory or the resource requirements for one will interfere with the requirements for the oth­ er. and so on .
4. Non-independence of action. so that the performance of one activity either affects or causes the performance of others , even when these other activities would appear to he quite unrelated . It is as if there were an overflow from the activation of one set of processing structures to neighboring structures. in which the major source of in­ teraction results from physical proximity of the process­ ing structures rather than from logical relationships among the activities being performed . (For an interest-

These different aspects of simultaneous performance provide hints as to the nature of the underlying mechanisms. I have al­ ready suggested that the doubling error in typing says somethinl( of the underlying representational structure. and of the possible mechanisms for binding a function to its arguments. Slips provide constraints on the nature of the underlying representational and processing mechanism. Examination or skilled typing provides another source of evidence. requiring some mechanism that can yield cooperative behavior among the fingers and hands . Studies of allcntion and of neurological deficits provide yet another source of information.

In our allempt to construct a processing model of these as­ pects of human behavior we have been forced to deviate from the more traditional processing structures. Instead. we find that a vi ­ able structure seems to require multiple. parallel units , all interm:t­ ing with one another. activating (and inhibiting) one another. with a tradeoff between activation value and the goodness to fit to trigger conditions. The scheme that we propose is a relative of production systems. but the control structure that we propose is somewhat different.

A. The role of will in the control of action

We postulate that skilled action se4uerKes arc automatil·; no conscious control of them is necessary . llowcver. hecausc people sometimes perform an action when the conditions are not mm­ pletely satisfactory. or hold hack an action even when it would oth­ erwise he appropriate some other form of control is re4uircd . In 1101. we suggest that the normal configuration of schcmas that per­ form an operation can be thought of as a hori:ontalthread of control (the name taken from the fact that the processing structure for some even sequence is often depicted as a series of horizontal pro­ cessing stages>. In normal circumstances, the horit.ontal thread suffices to carry out the action, with component schemas hcing trig­ gered when their activation values and trigger conditions arc satis­ factory . llowcver. allentional (wnscious) control is necessary when there is com:crn about the adequacy of the horiwntal thread structures (as in ill -learned tasks. novel situations , or situations per­ ceived to he dangerous>. This is done through control of the ac­ tivation values of schemas by means of vertiml thrt·tul structures. The application of allentional activation to bias the control of the horizontal thread schemas we called "will. " Thus. by the exertion of will . one can cause a schema to be triggered even if it would oth­ erwise not have been or to prevent a schema from being triggered that would otherwise have been .

The application of vertical thread activation. will, is best illus­ trated by the situation where one wishes to perform an undesirable act (such as gelling out of bed on a cold morning) or to prevent a desirable act (such as eating any more of a rich and tasty desert). In both cases will is required. in the former to increase the activa­ tion values sufficiently to cause triggering of the schema even in the absence of a sufficiently good fit of the triggering conditions. and in the Iauer case, to prevent an activity, even though the nor­ mal activation values and triggering conditions have been met. In the laHer case . continual allentional effort is required. for if allen­ lion lapses. the schema will revert to its normal activation values and triggering conditions. and the action will he performed.

The models of human processing suggested here need not he the only candidates. I mention them because they arc suggestive of the sort of processing structures required to account for human per­ formance . The important point is that conventional processing structures can not describe human behavior; a new breed of com­ putational mechanism must be developed.

ACKNOWLEDGEMENTS

The form of pro<:essing stru.:ture desnihed here was developed with my mntinual wllahorator David Rumclhart , ;md owes a lot to the developments of Jay Md 'ldland (as in I7.R, I3,1) . These in turn have heen inllucn<:cd hy the work of Geofl' !linton: sec lSI and !linton, this wnfercncc.