A Partial Taxonomy of Knowledge about Actions

I Introduction

It is commonly accepted that a computer which answers questions about a natural language text must have roughly a human reader's knowledge of the subject matter. Restricting ourselves to simple stories (like children's stories) the know­ ledge needed is that "common sense" knowledge which all members of a particular culture share. Since, by all indications even this knowledge is quite extensive, it will have to be well organized and classified before a computer will be able to make much use of it. In this paper I will be con­ cerned with classifying the kinds of facts we have about actions . Some typical facts of the sort I

will be concerned with are:

• (Fl) In giving a present the giver should give something the receiver would like to get.
• (F2) When painting some object it is a good idea to put paper on the floor under the object so as not to get paint on the floor.

Some facts which would not be included in this study, as they are intuitively not about actions, are:

• (F3) The Earth has one moon.
• (F4) Few people like modern music.

Furthermore, this study will be restricted to facts . like (Fl)-(F2) which tell us something about how to perform an action, as opposed to facts like:

• (F5) Many people break their legs while skiing.
• (F6) At playing chess, nobody is much better than Bobby Fisher.

Intuitively it seems to me that the "how to do it" type facts form a natural subclass of know­ ledge, so that by restricting the study to them I will not ·be hopelessly distorting the results. Naturally it would be preferable to have some firm evidence on this point, but short of already having a better taxonomy, it is hard to imagine what the evidence could be. Since one has to start someplace, one starts by backing one's in­ tuitions.

II What Should the Taxonomy Look Like?

To say that a particular fact is "about" some action is, at the very least, to say that the fact states some relation between the action and some­ thing else which might be another action, or per­ haps some state. What I would lik~ to find then is a list of relations such that for any action all the "how to do it" facts about it will fall under one or the other of the relations in the list. Furthermore I want the relations to be "in­ teresting" in the sense that knowing the relation­ type of a fact will give a language comprehension program information about what can be inferred from the fact, and perhaps even under what circum­

stances it should be used to make an inference.

To make all this concrete, consider the con­ cepts "pre-requisite" and "result". It should be clear that "pre-requisite" and "result" qualify as relation-types as defined in the last paragraph. After all, when one says that an action A has a result state S, on~ means that after A is performed S will hold. So result is a relation between A and something else, in this case the state S. Also both relations are "interesting" since to some degree they characterize inferences which can be drawn from the facts. In the case of pre-requi­ site (henceforth abbreviated as "pr") we have in­

ference rules like:

• (Rl) If state s is a pr of action A, then s must hold just prior to A's being done.
• (R2) If animate being P desires action A, and s is a pr of A, then that desire would con­ stitute a reason why P might desire S to hold.

Naturally, in an actual language comprehension system (Rl) and (R2) would be expressed in an appropriate formalism. However, the points I wish to make in this paper are hopefully independent of particular formalisms, so I will use informal natural language expressions, both in the state­ ment of facts, and in inferences which we might

draw from them.

Of course, pr type facts and result facts hardly exhaust the kinds of facts we have about actions, but with one or two exceptions nobody has bothered to extend the list. And while this paper will not produce a complete list, it will expand the list quite a bit.

III Justifying a Putative Fact

Ideally, a paper of this sort should proceed along the following lines : first produce a putative piece of common sense knowledge, then show that language users in a particular culture (mid­ dle class American in this paper) indeed have this piece of knowledge, and finally show how it fits, or fails to fit, in the taxonomy as developed so far, and if the latter, extend the taxonomy so it does fit. In this paper, however, I will not show that the putative pieces of common sense knowledge are indeed real, but I will rather rely on the reader's intuitions. To show what such justifi­ cation would look like, let me give an abbreviated version for the first example in the paper, repea­ ted here as (F7) .

• (F7) In giving a present the giver should give something the receiver would like to get.

In general the justification goes in two parts. First we show that something like (F7) is needed. Consider:

Fred was thinking of getting a kite to give to Bob for his birthday. Then Fred learned that Bob had a kite and did not want another.

If we were to ask whether Fred will still buy a kite, the answer would be something like "Pro­ bably not". If asked why, the response would be "Because Bob does not want a kite". If we assume a fact like (F7) these responses are easy to account for. Indeed there seems to be no other option than to assume something on the order of (F7). But need the fact be (F7) itself? That is, perhaps there are more basic facts which combine to produce (F7), and what people know are these more basic facts rather than (F7) . Showing that this is not the case is the second part of (F7) 's justification and it is in general quite difficult. All one can usually do is to examine a few poss­ ible alternatives, show them false, and then put the burden of proof on anyone who objects to the putative fact. So, for example, we can see that (F7) is a fact about present giving, rather than one about giving in general, since it would not be true for a policeman's giving somebody a parking ticket. I could go further and show that (F7) carmot even be generalized to giving "useful" things (unlike parking tickets), but will not do so here. (See (Charniak 74) for a deeper analysis of this particular fact.)

IV Strict versus Non-strict Pre-requisites

Given then that (F7) is part of our knowledge repertoire, and recognizing that it tells us some­ thing about how to go about giving a present, where should it go in our taxonomy? While it is clearly not a pr of present giving, it shares many pro­ perties of pr's. To see this, let us recast (F7) in the form of a pr.

• (F8) Receiver liking present is a pr? of giving a present.

First, as (F8) indicates, (F7) describes a relation between the action of present giving and a parti­ cular state, namely "rec e iver wanting the present". Second, (F8) obeys rule (R2) which is a rule about pr's. That is, wanting an action to happen often leads to wanting the pr's to hold. So in the case of (F8) we have examples like:

Fred bought Bob a kite for his birthday. "I hope Bob wants a kite" thought Fred.

Of course, what prevents (F8) from really being a pr is that (Rl) (pr's must hold for the action to take place) does not hold for (F8). That is, it is perfectly possible to give somebody a present he does not like. However, a rule very similar to (Rl) does apply for (F8), namely:

• (R3) If state S is a non-strict pr of action A, then it is a good idea for s to hold just prior to A's being done.

What we are doing then is to distinguish between strict pr's, which obey (Rl), and non-strict pr's, which obey (R3) . So (Rl) would now read:

• (R4) If state S is a strict pr of action A, then S must hold just prior to A's being done.

Another distinction between strict and non-strict pr's is:

• (RS) If an action A is done although a strict pr is broken we ask "how" it was done that way. If a non-strict pr is broken we ask "why" it was done that way.

Some examples of strict ,pr's are:

• (F9) Having an apple is a strict pr of eating it.
• (FlO) Not being at a location is a strict pr of going there.
• (Fll) There being a wind is a strict pr of flying a kite.
• (Fl2) The umbrella restraining strap being unhooked is a strict pr of opening the umbrella.

Some other examples of non-strict pr's are:

• (Fl3) Having permission is a non-strict pr of using an object which does not belong to you.
• (Fl4) Being invited is a non-strict pr of attending a pally.

V Logical versus Ph~ical Pre-requisites

We will be spending most of our time looking at various kinds of non-strict pr's, but let us first take a brief look at the strict variety. In particular, there might be a case for saying that there are really two kinds of strict pr's, which we might call "logical" and "physical". To see the difference, consider the difference between (F9)­ (Fl0) versus (Fll)-(Fl2). If somebody told you that he went to Chicago last year, but that he had never, ever, been outside of Chicago, you would probably accuse him of talking riddles. One might say that it simply made no sense for the speaker to say that he had gone to Chicago, if he had never been outside the city. In some sense he had con­ tradicted himself. On the other hand, there would certainly be no contradiction in terms if someone said that he had flown a kite without any wind, although you might not believe him, and ask skep­ tical questions about how he did it. That, roughly, would be the distinction between logical and phy­ sical pr's.

Now philosophers have said a lot about the existence or non-existence of distinctions like this one. However, our problem in this paper is a different one. Simply, do ordinary people in everyday conversation make such a distinction? I myself tend to doubt it, but my opinions either way are not very strong, so I will list this dis­ tinction when I sum things up later, but put a question mark beside it.

VI Social versus Suggested Pre-requisites

So far· we have only investigated one type of non-strict pr which, for reasons which will soon be apparent, we will call "social" pr. Another kind of non-strict pr is a "suggested" pr. For example:

• (Fl5) Having a tail on a kite is a suggested pr of flying it.

Reason: it makes flying the kite easier.

• (Fl6) Having paper, or otherwise uncared about material around and if possible under an object is a suggested pr of painting that object. Reason : one is less likely to get paint on other things.
• (Fl7) Being sober is a suggested prof driving . Reason: less likely to have an accident this way.

One difference between social and suggested pr's is immediately obvious. Suggested pr's are backed up with reasons for the suggestion, while social pr's, like (Fl4) or (F8) do not have reasons - things are simply "done that way". Having a reason for a suggestion allows us to relate the breaking of a suggested pr to certain difficulties experi­ enced in performing the action. For example:

If we combine (a) with - (c), or (b) with (d) the stories make sense, but the two other combinations do not make connected stories. Our comprehension model could account for such cases with a rule like:

• (R6) If a suggested pr F of an action A is broken, then associate the negation of the reason for F, if it occurs, with the breaking of the suggested pr.

Another difference between social and suggested pr's is that social pr's are much more subject to opinion than suggested pr's, and probably any re­ presentation of social pr's will have to allocate space for an indicator which shows which class of people believed in this particular pr and at which period in time. So some social pr's which at the current time divide people are:

• (Fl8) Having no foreign matter in the mouth is a social pr of talking.
• (Fl9) Wearing whites is a social pr for playing tennis.

Another distinction between the two is that social pr's have a "moral" component which we do not find in suggested pr's. For example:

Tom felt guilty using Bill's baseball without asking him first.

Mr. Jones was embarrassed playing tennis in his street clothes.

I am making the implicit claim here that embarrass­ ment and guilt spring from breaking the same kinds of rules, and the distinction is due to the parti­ cular circumstances and the strength of the feeling. This would seem to be supported by examples like:

• (F20) Being correct is a social pr of giving somebody information.

When Mr. Jones realised that he had given the driver wrong directions he felt guilty/ embarrassed.

In this last example either emotion would be reasonable, but presumably in both cases it would spring from the breaking of rule (F20). On the other hand there are clear cut cases where between the two (guilt and embarrassment) only one is appropriate. So whether one should distinguish between social and moral pr's or not is by no means clear. I have somewhat arbitrarily chosen the latter alternative.

Note that it is perfectly possible to feel guilty about breaking a suggested pr. For example, it is possible, one might even say preferable, for a person to feel guilty about driving while drunk. However, it is surely the case that this is not due simply to breaking the suggested pr, but rather that in breaking the suggested pr one is also breaking a social rule of the form "it is bad to put lives or property in danger, especially through carelessness or thoughtlessness". To see that it must be a rule like this which is responsible for guilt after breaking suggested pr's, note that it is hard to imagine feeling guilty over not putting a tail on one's kite. On the other hand, if for some strange reason, somebody's life depended on your getting the kite up quickly, it would be easy to imagine feeling guilty over forgetting to put on the tail.

Hence, another rule which distinguishes social from suggested pr's is:

• (R7) Breaking a social pr which one accepts is often a cause of guilt or embarrassment.

VII Pre-requisites and their Force

It is perhaps a good time to step back for a second and look again at what we have been doing. So far we have accumulated three kinds of pr's: social, suggested, and strict. In each case these facts mentioned some state which ought to be satis­ fied prior to an action taking place. What dif­ fered from type to type was ~ the state should be satisfied. To put this slightly differently, what differed was what would happen if we dis­ obeyed, so to speak, the pre-requisite. What we have been classifying then is "pre-requisite forc e ", that is, the "driving power" behind these various pr rules. As we will see later, these same kinds of forces apply equally well to other kinds of knowledge about actions, so really then we have been making a catalogue of "forces". That in each case we saw the force applied to a pr r e­ lation was simply for ease of presentation.

Seen in this light, it is logical to think in terms of a single pr relation but with each rule "tagged" with a force. Furthermore, we might now ask if each pr is limited to one such "tag". The answer seems to be no. So a society may take some suggestion (in my use of the word) and make it into a social rule - making virtue out of necessity, as it were. For example

(F21) Having clean hands is a pr of eating.

Force: (Social)

(Suggestion (Reason: this helps to stay healthy))

VIII Regulatory Pre-requisites

Regulations are probably the most complicated kind of force we have looked at so far, so we will start with an example which, while typical, does not exhibit all the complexities we will eventually associate with the form.

(F22) Having a driver's license is apr of driving a car.

Force: (Regulation (Authority legal)

(Punishment : fine))

Clearly (F22) is not a strict pr. Nor could it be a social pr as social pr's do not need to indicate authority (since the authority is always "society" in some sense) nor punishment (since either there isn't any, or it is just very subtle, depending on how you look at it). As for suggested pr's, (F22) would not fit there very easily either. For one thing, regulatory pr's need not have reasons be­ hind them. (In theory, of course, a regulation should have a reason behind it, but in practice that reason is not why the regulation is obeyed.)

A more interesting regulatory pr is the following:

(F23) Not being married is a pr of getting married.

FOrce: (Regulation (Authority: legal)) (Social)

What makes this rule interesting is the effect of breaking it. What happens, of course, is that the action is declared "invalid" in the sense that the end result, the two people being married, is de­ clared no longer to be the case. In some strange sense then (F23) acts like a strict pr, but there are many differences . For one thing, breaking a regulation only affects certain results. Those resulting states which are affected we will call "conventional" states. That not all results are affected is not easily seen in the case of (F23), for getting married does not have any particular end results other than being married, but we can

see this difference in:

(F24) The receiver being a legal player is a pr of completing a forward pass in football.

Force: (Regulation (Authority : Referee, also participants))

The completion of a forward pass includes the end result that the "receiver" has the ball. However, should it turn out that (F24) was broken, say an extra player sneaked out on the field, while we would still say that the forward pass was invalid, it would nevertheless be the case that the receiver had the ball at the end of the pass. What would differ is that the other results of a forward pass, like the ball being placed at the location where the receiver was finally stopped, would not hold. (Where the ball i s placed at the start of a play in football is of course part of the ru les of foot­ ball and hence conventional.)

Another difference between the effects of a broken regulation and a broken strict pr is that with the former the appropriate authority must notice the infraction before the conventional result is declared invalid. So we have the follow­ ing rule:

(R8) If F is a fact about action A with regulatory force, should F's _being broken come to the attention of the appropriate authority the conventional results of A will usually be declared not to hold.

Notice that (R8) only says the results will usually be declared invalid. To some degree this depends on the ·" importance" put on the regulation. For example, regulation (F23) will be strictly en­ forced in our culture, but at the other extreme we have a regulation like:

(F25) The ship's being at sea is a pr of the ship's captain marrying a couple.

Force: (Regulation (Authority : legal))

If, perchance, the ship had pulled into port before the ceremony started, one would not be surprised that upon later learning of this fact everybody decided to ignore it. Even formulating this notion of importance seems hard enough, but as we will see in a moment, there are cases where in spite of the regulation being important (R8) still may not be applied. So, for the time being, the circumstances under which broken regulations will be ignored is an open problem.

One regulatory pr which is interesting because it is a common one, especially in the Artificial Intelligence literature, is the following:

(F26) Owning an object (or having the owner's permission) is a pr of selling or other­ wise transferring ownership of the object.

Force: (Regulation (Authority : owner, participants or law))

The fact that you must own an object in order to sell it is often presented as a strict pr in the A.I. literature, but it seems more plausible that it is a regulation. For one thing, (F26) obeys rule (R8) concerning the invalidation of its re­ sults, rather than the corresponding rule for stritt pr 's, (R4) · So notice that it is not necessarily the case that a transfer of ownership fails if the seller does not own the object. For example, I have read one children's story whose plot went roughly as follows:

Tom was asked by Bill to help him (Bill) sell his kittens. Tom agreed, but he was not familiar with the kittens and by mista~e he sold Janet's kitten which had jumped into the

box with the rest of the kittens.

Now it is clear that (F26) was broken, but every character in the story assumed that the new "owner" was indeed the real owner, and after explaining what had happened they asked him if he would ex­ change Janet's kitten for a different one. That (F26) can be in effect ignored would seem to indi­ cate that its force is regulatory rather than strict. (F26) by the way is also the regulation which, although "important" is sometimes ignored, as this last example shows.

IX Substates and Umbrellas

So far we have been studying different kinds of force while keeping to one kind of relationship, namely pre-requisite. The rest of the paper will be devoted to studying different kinds of relation­ ships or, as we will say, different forms. To do this, we will concentrate on the representation of our knowledge about using umbrellas. It will be helpful to have common terms for the various parts of the umbrella, and not knowing the correct words (if there are any) I have rather arbitrarily chosen the ones in diagram (01). I should point out that (01) and the rest of this discussion are not inten­ ded to be accurate technical descriptions of um­

brellas and the i r workings, but simply what I happen to believe about this subject. So, for example, (01) does not show the "ribs" of the umbrella because I was not sure exactly where they went.

Roughly speaking, to use an umbrella one holds on to the handle, raises the canopy to "up" posi­ tion, and locates the canopy above one's head. While this verbal description is fine as far as everyday speech goes, it is misleading in several respects. In particular, notice that to use an umbrella it is not really necessary to raise the canopy. Afyer all, the canopy might already be up, which would make raising it unnecessary. The same is true for the umbrella being over one's head. That is to say, what is important in using an um­ brella is not the actions, but rather their resul­ ting states, "canopy up" and "canopy over head".

These two states have some resemblance to pre-requisites in that they too are states which must hold else be achieved prior to one's goal being achieved. The difference is that intuitively pr's must hold prior to doing the act, while "canopy up" .would seem to be a state which must hold during·the act of using an umbrella. To cap­ ture this similarity we will introduce the notion

of a "substate" which is a state which should hold at some point in the action. Canopy up is a stri c t substate, since it is hard to imagine using an umbrella to keep off rain without the canopy up. A suggested substate in umbrella use is that the canopy be perpendicular to the direction of the rain. Another example, this one from the action of putting on one's shoes, would be that after the shoes are on they should be tied. (Again, i f you managed to put your shoes on without first untying them, it would not be necessary to tie them again, hence the suggestion is not that one tie the shoes but rather that the shoes be in the state of being tied.) Notice that in the "shoes tied" case the substate is only desired after the rest of the action has been completed. Pr's then are simply a special kind of substate, namely those substates which are ordered prior to any other substate or subaction. My impression is that as such, pr's need not be further distinguished from other sub­ states. However in the final summary I will indi­ cate pr's as a kind of substate and put a question mark beside it.

So at one level of detail using an umbrella looks like a series of substates, as indicat~d in (02). (02) shows how PERSON can protect OBJECT from rain with an UMBRELLA. I have adopted a function-like notation where words in capitals are variables. So, "umbrella method" is a "function" of three arguments, and canopy (UMBRELLA) is a function of one argument. This notation has no particular significance. In particular I do not mean this to be a subtle argument for a "proce­ dural" representation.

1. umbrella-method (PERSON OBJECT UMBRELLA) PERSON be near OBJECT - Force : (Strict) PERSON hold stick (UMBRELLA) - Force : (Strict)

canopy (UMBRELLA) in extended position - Force : (Strict)

canopy (UMBRELLA) between rain and OBJECT - Force : (Strict)

canopy (UMBRELLA) perpendicular to direction of rain - FOrce : (Suggestion (Reason : less rain will hit OBJECT))

X Ordering and Method Statements

One deficiency in (02) is that there is nothing indicating in what order the states should be satisfied, or even if it makes any difference. In some cases it is clear that it indeed does make a difference . For example, it is hard to imagine getting the canopy up if the person is not already holding the umbrella. This would be a strict time ordering relation. In another case we have a suggestion ordering: in general it is a good idea to extend the canopy prior to putting the canopy over the object as it is generally easier to ex­ tend the canopy with the umbrella pointed out in front of one. With the ordering relations made explicit (02) will turn into (03). (I have ab­ breviated the information already expressed in (02) in order to make the new informat ion easi er to read.) When an arrow indicates that X precedes Y it does not imply that X must immediately precede Y with nothing intervening. Hence an arrow tells us what must already have happened rather than what to do next .

-slip from the normal dispenser.

• (F36) In many Italian bars one pays by first telling the cashier what one wants, giving her the money, and getting a receipt which is in turn given to the barman.

Time Orderings

• (F37) It is necessary to obtain the slip men­ tioned in (F35) after entering the shop on the current buying trip.
• (F38) In bridge the dummy is layed down after the first card is played, but before the second.

XII Other Forms

The four forms we have investigated so far hardly exhaust the possibilities. In this section we will look at two more, but more in the spirit of ongoing inquiry, rather than finished products.

Our four established forms can be put into two classes. One class, which I will call "occur­ rence statements" consists of those statements which describe some action or state which should occur. Naturally, substates and subactions fall under this heading. Another form which is also in the category is "side condition". A side condition is an event or state which in some sense describes a second state or action. For example, a side condition of moving the flange up the stick of the umbrella is that the canopy raises at the same time and to the same degree. Note that on one hand this does not qualify as a separate sub­ action, that is one does not extend the canopy separately from raising the flange, yet we cannot simply ignore this direct relation between the two events. For example, it enables us to understand how a canopy could be half extended. Another

example of a side condition is:

• (F39) Paint being left on the paint brush is a side condition of transferring paint from the brush to the 'object being painted.

Force : (Strict)

Notice that both of these side conditions have strict force. At the moment I do not know if other kinds of force are possible.

We have said that side conditions are "occur­ rence statements" along with substates and subacts. Time ordering statements and method statements on the other hand fall into the category of "control statements". Control statements tell us not what needs doing, but rather when or how to do it. The obvious analogy is with control information in programs, with ordering relations corresponding to the implicit ordering of program statements, and method statements corresponding to subroutine calls. This analogy suggests that there ought to be other kinds of control information needed to describe actions, and indeed there are. One par­ ticular i s -the "do-while" statement. So in painting (using a brush) we know to dip the brush in the paint and then transfer the paint to the

object. However, it is necessary in general to repeat these two steps until the job is done . This is a strict do-while statement, but do-wh i l e s occur with other forces also.

• (F40) In washing one's hair one rubs soap into the hair and then rinses it off. Repeat these two steps until soap lathers easily.

Force (Suggestion (Reason : makes hair cleaner))

• (F41) In bidding in bridge, one bids after the person to one's right. This is repeated until either four passes or a bid followed by three passes.

Force : (Regulation)

XIII Conclusion

What I have done in this paper then is to classify knowledge about actions according to two criteria, the force behind the rule (that is, why you should follow the rule) and the form of the rule (or what the rule looks like). As such, the paper can be summarized in diagrams (06) and (07). The question marks in the diagrams indicate dis­ tinctions which I am not sure need be made. In (06) the numbers following the force types indi­ cate rules in the paper which the force type obeys.

I have so far not mentioned other people's work in relation to the results presented in this paper. Space limitations prevent any detailed discussion, but obviously this work wa3 not done in a vacuum, and there are several papers which have influenced it. Some particular influences are: the "absolute" vs "reasonable" conditions on actions found in (Schank 74), the classification of inference types of (Rieger 74), the "infeleci­ ties" of (Austin 62), the idea of "institutional facts" found in (Searle 69), and the "frames" re­ presentation of (Minsky 74).