Toward a Detailed Model of Processing for Language Describing the Physical World
Proceedings of the Seventh International Joint Conference on Artificial Intelligence. Natural Language. Pages 1-5.
© 1981 International Joint Conferences on Artificial Intelligence
Abstract
This paper explores the problem of ju1ging wheth~r or not an English sentence could correspom to a real 'llorld situation or event which is is literally, physically plausible, and the relate~ problem of representi~ the different possible physical situ~tions. The judgement of plausibility can be made at a high level by checkisemantic marker restrictions on verb case fra~e constituents. Often, however, plausibility judgement can only be based on the results of an attempt to construct (imagine) a scene that corresponds to the sentence, and which does not violate "ccmmon sense" (i.e. relevant physical laws and expected, stereotype1 behavior). Methods are presented for constructing representations for different scenes which could correspond to a sentence. These methods incorporate (1) "subscripts" (sequences of scenes which ccmprise an event, with attachepreconditions and postconditions) to express different verb senses, (2) object representations which express properties such as shape, size, weight 1 strength, and behavior t.nder ccmmon conditions; (~I physic.'ll laws, encoded as constraints on behavior; (4) representation of context; and (5) robot problem solving-like methods to fit all this material together.
- the references below TINIAP-1 r'efers to R. Schank srrl B. Nash-webber (eds.~ Theoretical Iss~s in Nstur''il Lsnguage Processing, Arhngton, VA: ----xcr;;-W75;-anQ 'I'IINIAP-2 refers to D. Wsl tz ( ed.) Theoretic'il Issues in Natural IBng\13.ge Processing--2, New York: ACM, 1~-
- Boggess, L. c. 1979. Computstional Interpr'etation of English Spatial Prepositions. Tech Rpt T-7~. CSL, Univ. of Illinois.
- Brachman, R. J. 1979. CXl the Epistemological Status of Semantic Networks. In N. Findler (Ed.), Assocfslive Networks. New York: Acadootic.
- BUndy, A. 1978. Will It Reach the Top? Pred iction in the Mechanics World. Artifici'l.l Intelti1ence 10, 2.
- Chafe," W. L. 1979. 'l'ne flow of toought and the flow of lll!lg\13.ge. In T. Givon (Ed.) Discourse and Synt~'J New York: Academic.
- Colby, K., B. ~ught, and R. Parkinson. 1974. Psttern Matching Rules of the Recognition of Nstural Language Dialogue Ex:pressions. Memo AIM-234, Stanford AI Lab.
- Davis, R. 1978. Knowledge Aquisition in Rule-Bssed Systems. In D. Waterman and F. Hsy"ls-Roth (eds.) Psttern-directed Inference Systems. New York : Ac ademic.
- de Kleer, J. 1977. MUltiple Representations of Knowle:lge in a Mechanics Problem-solver. IJCAI-77, 197-2P3 L'31 Fillmore, C. J. 1977. 'l'ne Case for Case Reopened. In J. Kimball (Ed.) Syntax and Semantics. New York: Academic. - [
- ~1 Forbus, K. D. 1981. A Study of Qualitative and Geometric Knowledge in Reasoning about l'btion. Tech Rpt AI-TR-61 ~. MIT AI IB.b.
- Grice, H. P. 1975. Lo~ic 'l.nd Conversstion.
- In P. Cole and J. l'brgan (Eds.) Syntax m1 Semantics: Speecp ~ltsH New York: Academic.
- - -----r1 ayes, P. J. 1978. 'l'ne Naive Physics funif~sto; 'ind Nsive Physics 1: CX!tology for Liquids. Tech. Rpts., Institute for Semantic and Cognitive Studi~s, Geneva. L121 Herskovitz, A. 196). CXl the spatial uses of prepositions. Proc. of 18th Annual Meeting of the ACL, Univ. o~ Penn. ;t~iiTa. ------------ [n J Johnson-Laird, P. N. 1900. Mental models in cognitive science. Cognitive Science 4, 1, 71-115. [141
- McCarthy, J. 1968. Programs with common sense. In Sem:J.ntic Information Processing, M. Minsky ( eci.), MIT Press Cambri1ge, RA, 403=418. L151 McCarthy, J. 1977. Epistemological problems of 6 • 1r'tifici'll intellig"lnca. IJGAI-77.
- McDe rmott , D. 1974. Assimil'ltion of New Inform'ltion by •1 N• -1 tur'll L'lngu'lge-Un1erst'lnding System. T~ch ~t TR-241, ~IT AI Lab. L17
- McDermott, D. 1900. Spati. 'll Inferencoils with Ground, ~tric Formul!ls on Simple Objects. Rpt. 1TI, Y.'lle l.Jnix. Computer Science Dept.
- L18J Miller, G. A. snd P. Johnson-L!lir:l. 1976. lB.nguage and Perception,Cambridge, MA: H'lrvard Univ. Press - r191 Novak, G. S. 1976. Ccrnputer understanding of physics problems st'l ted in natural lll!lgusge. 'fuch. R"lpt. NL-~0. Dept. of Computer Science, Univ. of T"lxas, Austip. L20]
- Rie~er, C. 1975. The commonsense algorithm as a basis for computer models of human memory, inference, belief and contextu'll l11nguage comprehension. TINIAP-1, 100-1 ;1'). L21)
- Schank, R. c. and R. Abelson. 1977 . Scripts, Plans, Goals, and Un1ershnding. Hills:lale, NJ: Erlhl).l!l. ------- L22]
- Simmons, R. F. 1975. 'l'ne Clowns Microworld. _TI_N_IA--iP:,.,-,.,..1 ~ 17-9. (2~J Talmy, L. 1973. The Rel!ltion of Grsmmar to Cognition -A Synopsis. TINLAP-2, 14-24. \.241
- Waltz, D. L. arid L. Boggess. 1979a. Visual analog representstions for natural l'l.nguage understanding. IJCAI-79~ ~6-~4.
- 25J Waltz, D. L. 1979b. R9lating img_ges, concepts, and words. InN. &idler and R. &ijcsy (oos.) Reprepen?tion of ~-D Objects. Verlag, to appear. L26)
- W!iltz, IJ. ~1900. Underst'lniing scene descriptions as event simulations. Proc. 18th Annu'll Meetipg g_f the ACL, June, Univ. of Penn~ila., 7-12.
- II -waltz. D. L. 1981 • Generating an:l understanding • scene descriptions. In Joshi, Webber, and Sag (oos.), Elements of Discourse Understanding, Crunbridge Univ. Press. -----
- Wilks, Y. A. 1975. A preferential, psttern-seeking, sem"'lltios for natural l9.nguage inference. Artif!ci'll Intelligence 6, 1, 53-74.
- W1nogriia, T.-1972. Unders~~ing Natur'll LangU'lge. New York: Academic.
- Winograd, T. ~ Understanding Natur'l.l lB.nguage. New York: Academic.
References were extracted automatically from the PDF and may contain errors.
Full text extracted automatically from the PDF; figures, tables and equations may differ. The PDF is the version of record.
1. Introduction
It is clear that humans understan1 language in far greater detail than do any programs written to date; people can make fine distinctions base1 on apparently peripheral properties of objects or nuances of behavior, and can draw upon a vast number of possible inferences about any given sentence or situation. This paper examines the problem of 9dding greater detail to the internal representations and reasoning methods for a natural language understanding program which works in the dcmain of physical scenes and events. I show how we can begin to account for the fine-graine:i but important distinctions introduced by the use of alternative verbs (e.g. hit vs. graze vs. strike vs. smash) and adverbial modifiers (e.g. hit vs. hit hard vs. almost hit vs. hit squarely).
Understanding language involves (at least) the following processes:
- ( 1 ) . judging the plausibility of the langua~e; various possible readings of a sentence or text may have to be ccmpared for relative plausibility, both on the basis of the inherent content of the hnguage in isolation, snd within the context where the> hngusge is encoun tere1;
- representing meaning; a suitsble representstion of the meaning of the langusge, in context, must be constructe<i. The represents tion should be unambiguous,
WTh~s work ~s supporte3 by the Office of Nsval Research under Contract N<XX>14-75-C-0612.
TO\iARD A IETAILED MOIEL QF PROCESSING FOR LAN�UA~ ~~CRIBIN� THE PHYSICAL WORLD*
Dwii L. W~ltz
Coo~inat~1 Sci~nce Labor~tory and Electric~l Engineeri~ Department University of Illinois Urb9.Ila IL 61801
and shoul·i hsve anaphora and ellipsis resolved . Representations should be capable of disti~uis'lin.g res:iings that people regard as diff~rent; they shouli allow one to ma~e n~tural inferences about the the events or scenes described; and similar meani~s should have similar representations;
- (~) retrieving inform.~tion frcm 9Jld/ or making modifications and additions to relevant memory;
- taking action if appropriate; ~ction can be physical, linguistic (e.g. performing a s~ech act), or mental (e .g. planning a strategy).
These actions should not necess.~rily be done in any particular sequence. For example, it may be thst one h'ls to find a representation from memory before one can ju:ige its plausibility.
1.1 Representing Spatial Meaning in English
En~lish is a particul'lrly poor language on · which to b'lse a study of langu~ge lind perception. English spatial locative prepositions (on, in, at, etc.) are ambiguous a11:l irregular, so English descriptions of the perceptu~l world which use them are in general highly a~biguous. We often need to h'l.ve a great deal of 'l. priori knowleige in order to understan:i the i:iicmatic mearnl')gs which result ·N"!en English loc~tive prepositions are used with particular objects.
For example, to understand "The chair is at the desk", one must know that the ordinary relationship between desks and chairs is one where the chair faces the desk, and is partially beneath it; if a chair is facing away from the desk, or is upside down, or is on top of the desk, it cannot be said to be "at the desk". Phrases such as "at the corner", "at the store", "qt the door", and so on, also require idiosyncratic analyses, ~nd thus their meanings must be stored as separate lexical items. The same is true for other English spatial locative prepositions (see ~1 ,121).
English verbs present similar problems. T~ke for instance, the verb "cut". "Cut" can take on a wi:ie variety of spatial/temporal meanings, which depend upon the nature of its object and instrument. Compare for example, the images wd inferences evoked by "cut the board with a hand s~w", "cut my finger on broken glass", "cut through the mount'lin "'i th bulldozers", "cut the lEper with a scissors", "cut the steel with a torch" , "cut the corn with s ccmbine", lind so on. A system must be able to ju.ige that it is possible but ~culi3r to cut a grape with s jigsa..,, that it is impossible to cut a diamond with a plastic knife. Although there seem to be share1 aspects of meaning, each verb-object-instrument combin~tion must hwe separqte and. extansive me9.Iling representation scenarios.
The ~ritical problem we face is this: wh~t is ~n appropriate way to represent various possible meanings of langu~ge describing spatial relationships ancl. physical events? There seem to be four main possibilities:
- case frame representation.
- sentences of spatia~ primitixes using either predicate calculus (as in L18 and 11 J) or a conceptuql dependency- or script-lik3 form.
- (~) schematic modeling, using d~ta structures to directly represent quqlitative spatial knowlecl.ge in 2-D diagMnls [7 ,9, 19,20,22,26];
- direct geometric modeling, using ~-D coordin~te systems anp specific~tion of locations, velocities ~nd directions L1 ,17,24,29J;
It seems clear that all four ~s of representation are important, an.d that they stand in a hierarchical relationship with each other: alternative (1) is the "deepest , " i.e. closest to the representations of high-level perception, alternatives (3) and (2) are progressively more abstract, and (1) is closest to ~ surface linguistic representation. Alternatives (1) and (2) are attractive options, since their representations are bolmded, linear forms, whereas (3) and (4) require the ievelopment of novel representation schemes.
2. Biting Dachshunds
Let us look in some detail at the way sentence ( 31 ) 1'\Y dachshund bit our mailman on the ear. would be processed by a program which generates a model of the physic"ll 'OOrld correlates of the sentence*. As a general strategy, I assume that the program should start its processing at the most abstract level, and only go to deeper levels of simulation if necessary.
(Step 1) The first things to check are the case frame for the verb "bite" and any possible semantic interpretation patterns which match the 'OOrds of the sentence. The case frame for bite:
Bite (animate-entity physical-object) is satisfied with the argunents "dachshund" and "mailman's ear"~ respectively. General world knowledge*~ e.g.
goal dog (bite dog physical-object) often)
goal dog (bite dog man) sometimes)
(avoid man (bite dog man) usually) would also match the given sentence, so the general plausibility of the sentence 'OOUld be immediately established, and initial goals for the man and clog would be hypothesize]. In a quick reading or shallow understanding mode, processing might terminate here. Similarly, if general argunents were the only ones given in the sentence, as in:
(S2) The dog bit the man.
then there would be no reason for further processing, since the case frame for this sentence already matches exactly one of the semantic patterns known to the system.
(Step 2) The mention of "dachshund" and "mailman" cause the 'instantiation' of internal data structures, consisting of default postures and settings for both the
*Programs have not yet been written to handle these examples. I believe, though, that there will be no insurmountable obstacles to writing such programs.
- "*This knowledge may be listed explicitly, deduced from general facts, or deduced fran the analysis of scripts for typical actions.
'io~ an1 mailman. 'The mllilman is insbnti'ltei on .a resiiential si'iew"!lk, since this is a strongly ~ouplei def.'l.ul t, 'lnd the dog is put in the same scene with the m~ilman, since dog is weakly coupled to a number of default locations (Y'lrd, indoors, pet store; kennel, etc.) . "nle dog 'lnd mailmm qre both .qssumed to be st'l.nling. "nle scene is thus as shown in figure 1 . "nle structures for each scene entity ac tu'llly r epr esent only the cooMiMtes of its enclosing tyinllelipi.pe·l, alollP, with ll stick figure skel"lton to which n·:unad parts can be attached. "nle mailm'ln's enclosing volume is the stand~rd one for adult m!lles, and the dachshund's is found by scaling the 1imensions of the clog enclosing volLrn8 ~ppropriately. "Mi1get" or "puppy" would sele>d different enclosing volunes, as would explir.it mention of height (as in "the six- foot mailman").

(Step 3) The next step is to attempt to "run" (interpretively evaluate) the definition of "bite". "ntis definition is shown pictorially in figure 2a, and part of the definition in assertion form is shown i n figure 2b.

BITE (~imate-entity ~ysical-object)
Preconditions
Action 1
Action 2
- 1: (deformable P) 1: (closing M)
2: (and (mouth M)
(part-of A M))
3: (partially-around M P)
4: (open M)
Action 3 ----
1: defonned P)
2: not (closing M))
3: connec too M P)
4: apply-force-on M P)
5: may-not-be (open M))
1: (contact M P)
2: (complete BITE)
Postcondi tions
1: (may-not-be (partially-around M P)
2: ~open M)
3: may-be (dam~ai P))
4: time BITE (aprx 2 sec))
Figure 2b. Assertions for definition of "bite".
The definition includes preconditions, postcondixions, and a body. 'nle only postcondition we need for this example is Bite-PostC-1, which states that there is (probable) damage to the thing bitten. 'nle preconditions are of two kinds: "static preconditions" on the nature of the individual entities involved (analogous to semantic marker restrictions) and "dynamic preconditions" on the relations between the individual entities. In this case the static preconditions are Bi. te-PreC-1, which states that the object bitten must be deformable (or else we would say "bite down on" instead of "bite"., and Bi.te-PreC-2, which states that the thing that bites must be something that can close in the manner of a hinge, preferably a mouth of an animal or person*. 'nle single dynamic precondition is Bite-PreC-3 which states that the thing th~t bites must parhally surround 'the object bitten (if it totally surrounded the object, we would probably use "chew on" rather than "bite").
Checking preconditions, we see that Bi.te-PreC-1 and Bite-PreC-2 are satisfied, since ears are deformable, and the dachshund has a mouth**. Precondition Bi. te-PC-3 cannot be directly derived from general knowledge however, so "bite" tries to make it cane true. '
(Step 3.1) To this end, the pro~ram would attempt to plan a sequence of steps which could simulate reaching the goal:
(Be (Partially-around .dachshund' s-roouth mailman's-ear))).
To accomplish this goal, new preconditions must be satisfied. Basically they are P9rtially-Around-PreC-1, that the mouth be open, and P9rtiall!jAround-PreC-2, that the location of the object be locat between the ends of the "jaws" of the mouth. :Fartiall~Around-PreC-1 is trivially satisfied, since we assume t t opening a mouth is ~~thing that agents ?an alnys do unless explicit cond~b.ons have been menhoned which preclude this action. However problems arise in trying to Mtisfy precondition Partially-Around-PreC-2, since the dachshund's mouth is
Wil'he meamng of "·b~te' corresporrling to the bite of a bee or mosquito would have a different representation.
-rhese facts could be stored with the concepts of "ear" and "dachshunti" or, more likely, with their ance~tqr concepts in a "structured inherit.'lnce network" (e.g. L2J) i.e. these facts c.'l!l be derived automatically from more ge.neral facts, in this c~se, that the parts of the body of a creature, except bones and teeth, are deformable; likewise all animals have mouths.
not around the mailman's ear in the default situation instantiation (see figure 1).
' (Step 3.1.1) To achieve Partially-Around-PreC-2, one can either bring the mouth to the ear, or the ear to the mouth. Case grammar anslysis has already identified the dachshund as the agent, so the preferred method is to bring the mouth to the ear.
(Step 3.1.1.1) The first goal tried is thus:
(Reach dachshund's-mouth mailman' s-ear)
Through match of general patterns for accomplishing Reach, at least four possible subgoals can be . suggested: (reach-by-body-posture), (jump-up-to), (climb-up-using physical-object), and (make someone (move someone physical-object)). 'lhe library of dog body positions should be checked first***.
None of these postures allows the mouth to be arounti the ear, so jump-up-to is ma:ie the subgoal, and dynamic behavior of the dog checked. Using high level kno;rledge of animals' abilities attached to jump-up-to, for instance that animals can typically jump no higher than twice their longest dimension, it can be dai uced that the dachshund could not jump high enough either.
The next thing to try is (climb-up-using physical-object). Methods attached to this goal attempt to use any objects that are known to be around in order to let the dachshund climb up to the mailman's e'l.r. However, since no objects have been mentioned explicitly, and no default objects (e.g. mailbag, uniform, dog collar) are good for climbing on, this possibility fails also.
'nle last possibility is to induce someone to lower the ear so that the dachshund can reach it. 'nle only other possible agent is the mailman, and dogs can send only s very limited repertoire of messages. A dog might make sounds to induce the mailman to pet it, but other options, e.g. convince, bargain, etc. are only available to people. Other messages a dog can send, such as "threaten", would probably lead the mailman to flee rather th'ln to cane closer.
(Step 3.1.1.2) Having failed at getting the dachshun1 reach the mailman's ear, the program would then check to see if the mailman's ear might be -gotten to the dog. Here too there are two main {DSSibili ties: the mailman m'ly have put his ear within range of the dog's mouth intentionally or unintentionally. Having earlier noted the ~eneral fact:
tgoal m~ (avoid (bite dog man))) we can conclude th'lt the goal
(goal mailman (bite dachshund mailman's-ear)) has a low probability of being true, and we can thus preclude the {Dssibility that the mailman might intentionally put his ear within the dachshund's range for the purpose of having it bitten.
Another possibility is that the mailman got bitten unintentionally; this could happen if the maj.lm~ typically assuma1 {DStures that would allow the dachshun1 to reach him. To check out this {DSSibility, the script for "mailman" must be consul ted. No mention of postures other than standing, walking, getting mail out of a mail bag, and p..1tting mail in boxes are to be found in the script, so there is no reason to believe that a m'lilman would get to where a dachshund could reach him unintentionally either.
**"''iote th'l t we could Blso find range of body positions possible for a dog by simulating the movements of its joints. However, I do not believe th!it people behave this way, and for a program, it is a great deal easier to simply store ccrnmon {DStures (and {DSture sequences) directly.
The over~ll conclusion of the program would thus be that While it is plausible that a dog could bite 3 mqn, the program cannot im9gine how a dachshund could h~ve bitten a mailm~'s ear. How can we avoii possible explanations basei on the observations that people do occasionally squat or lie down, making them 'iCCessible to bites by small dogs? During informal experiments, people have suggested only possible explamtions th'lt tie in somehow with the particulars of mailmen and d~chshunds; they do not suggest explanations such as "The m'lilman w<ts a friend of the owner of the dog and W"lS lying iown on the owner's oof~ When bitten". 'lhis seems consistent with 'l policy of omitting concepts that do not occur in the sentence explicitly (e.g. sofa, owner of dog, friendship) unless they can be .'issumed as part of a relevant script (mailb'l.g, letter, dog collar, mailbox, aoo so on could thus be used ~s part of an explanation) · Knowledge about ~pparently peripheral items ( e.'g. the likelihood th'lt the mailman is a friend of the dog's owner) is not necessarily irrelevant, and may in fact be used explicitly by people i n re:J.soning ~bout event plausibility. I see no reason to believe that a system could understand natural langU/lge in any epistemologically non-trivial sense without having available a great deal of information (see aloo [15]).
If forced to produce ~ plausible explanation, the program should be able to select ways of getting the dog's mouth to the m9.ilman' s e9.r Which were not precluded, but for Which there was no support (e.g. the lll!lilmm m'lJ have squa tte.i for oome reason, there might have been an object that the dachshund could climb up oh, the mailman might h9.ve been very short, etc.).
3. Slightly different sentences, ra:l.ic"l.lly different processing
Let us now look at the ways in Which the processing of similar sentences would differ fran the processing of (31).
Let us start with
(S3) ~ dachshund bit our mailman.
As in the case of (S2) "The dog bit the man" (mentioned earlier in section 2), this sentence would be processei only to the case frame level. If pressed, of course, a system could imagine a scene to simulate this sentence, e.g. the dachshund biting the mailman on the leg.
Consider next (S4), "~ dobermm bit our mailman on the ear". 'lhe first step in processing would be exactly the same as that for (S1), i.e. the general plausibility of dogs biting people would be note1 9.long with the fact tht people usUally avoii being bitten. The seconi step would likewise be similar, but in this c"l.se the size of the dog would be much larger. In step 3, however, the processing would follow a very different line. In step 3.1.1.1, (Reach doberman's-mouth mailm9.n's-ear) would succeei via reach-through-body-posture, since the pre-stored postures of a dog would allow the mouth to be at the height of the ear When scaled appropriately for a doberman.
The progrrun would then continue the running of the event simul"l.tion for bite (see figure 2), simulating in turn the closing of the mouth on the ear, the contact between the mouth and ear, the deformation of the ear with probable damage, and the opening of the doberman's mouth again. Finally, through knowledge that only a few body postures are stable (i.e. cm be held for a long time), the program would simulate the return of the doberman fran its standing-on-two-legs posture to a standing-on-four-legs posture. In this case, then, nothing remains . to be explained, the program could judge th'lt the event 1.ni the simulated memory. described in the sentence W'l.S plausible , subevents could become p>rt of its Consiaer now (S5), "~ dachshuni bit our g'lrdener on the ear". The processing of (35) would be different from ~y of the cases above; it would par<illel the processing of (31) until the point where the mailman script is checked to see Wh'lt body postures are ordinarily assumed by mailmen in the course of their work. The script for 'l g'lrdener involves getting into 'l kneeling or sittin~ position frequently, so there would be a re.ady explanation for how the person's e'lr .::oul d h·we gotten in mnge of the d.'lchshund's mouth. 'lhe processing could then continue 'lS in the case of (S4) to completion of the biting action and return of the do~ to a st3n,! in.<_S position on the ground.
The processing of ('36), "\11y d'lchshunJ almost bit our mailmqn", is especia lly inte r es ting. Here we hwe to decide What "qlmost" could me 'ln. Refer-ring to the iefinition for "bite" shown in figure 2, we can see that the snapshot frame in wh ich the teeth m'lke contact with the object is marked "complete", me:ming that if this sh'lpshot portion ever occurs, then the qction of biting will have been completed, regardless of exactly wh~t course the action takes afterwarJs. CXl the other hand, until the action in this snapshot actually occurs, no biting can be s.'lid to h9.ve taken place . With this knowledge av~il~ble, we are . now in a position to understand (36) .
Basic.<J.lly, we know that at least some of the preconditions for bite must have been satisfied, or else "almost bite" would not be an appropriate phrase to use to describe the event ( tl}is idea is related to Grice's maxims for communication [10J). To process this sentence the way a person would, we would have to be able to generate the range of types of events that could qualify as .. <J.lmost bite"; some would be easy, e.g. cases where the d015 close:1 his mouth but f'liled to conbc t the mailman bec<J.use e ither he or the do~ moveJ , or where the dog o])3ned his mouth near the m'lilm9.n s body but ne.ver closed it on him. Generating othe r c'lses would require special knowledge of how dogs bite, as opposed to the knowledge of bi tin'S in general, which we h'lve been using so far . Speci'll knowledge of dogs coulJ al low us to judge th'lt 3 thre<J.t (such as baring the f1.ngs ·mi/ or growling) might .qu'll i fy a s "almost biting". If we knew 'l gre'it deal ~bout the pqrticular dog referred to in the sentende , we might envision arbitrary behavior that had in the past preceded biting incidents (e.g. the dog might alW3ys jerk its heaa rapidly to the left, or bark twice j ust be fore bit ing oomeone).
If the sentence h9.d been (S7) "~ dachshund tried to bite our mailman", then the envisioned possibilities would be similar to those generated for (56), al tho\Jgh we woul d strongly prefer that all the preconditions of bite be satisfied, along with all actions up to the snapshot frame marked "complete"; U iosync rat.ic or ordinary threq t sigmls would not really be appropriately describe) :1s "try to bite".
Fin.'llly for (58) , "~ dachshund bit our m.ailm:m Mrd", we would have to find a step .mere t her e w<ts w applic'ltion of force , in this case step Action) of "bite" ( see figure 2). The r epresent.'ltion of this portion of t he subscript would then be simply modifie-i, 1.n:i th".! lrldition"l.l obse rvation made that bi ting h<J.d d~ > more deform'ltion (probably)~=> more damage ( prob~bly).
- How much knowledge is necessary to do these examples?
A lot, toough not an impossibly large amount. IP-t us here take stock of the kinds of different knowle1ge necessary for processing the sentences 'lbove, 'lrrl also look at how a system could know th3.t it soould retrieve this particular knowledge at the right times.
- Case frames. G<irden variety case fra~es would suffice here, retrieved as dictionary entries; the only even mildly nonstandard feature I envision here would be to use structured inheritance for case frames (as well as most other types of information) so that, for ex3111ple, verbs that describe events that have duration would have ancestors that el!l.borated on the use of time expressions for verbs with duration (as distinct from verbs describing events which occur at a point in time such as leave or hit, or verbs which describe states rather than events).
- Semantic patterns. As I envision these, they would be similar ~n m'Uly W"iys to the patterns which form the heart of PARRY L '51 and also similar to ttB patterns usea by Wilks in his preference sem-mtics f2s1. Many of these patterns would be stored directly {e.g. <avoU 3Ilim'lte-en ti ty p-i in usu<tlly>) and others would mve to be inferred (e.g. <avoid man (bite dog man) usu'llly)). Generally these patterns would describe goals, beliefs, and combinations of concepts that would usu'llly co-occur with a verb. I believe that it will prove worthwhile to tag these patterns with probability information of a rather broad type (3.1 W!lys, usually, often, sometimes, rarely, never). 'lhe system woul~ use this information in the manner of expert systems L6l. Generally the system would begin with patterns involving as many of the entities in the sentence as possible, and attempt to find patterns that would express the goals and/or beliefs of each of the participants.
- Spatial descriptions. Each physical object known to the system soould have a spatial description, which would allow the system to judge for each object its size, weight, stable postures or orientations, positions of parts within an enclosing volume for the object, dynamic behavior (oow fast the object can move, what sequences of positions 'lre possible), etc. 'lhese descriptions soould be hierarchical, so that, for example, "jaw" would mve its own spa ti'll description which is merely pointed to by the descriptions of entities which have jaws, along with appropriate scaling information. Again here, structured inherit·mce would be useful to store infonntion about the Sp'lti:-tl world efficiently.
- "Subscripts" for verbs. In 'lrldi tion to case frames, verbs would also h'lve much more specific information about the time sequence of subpArts of the action described by the verb, including a set of preconditions, a body of actions, and a set of postconditions. 'lhe effects of the action could be found either in the general semantic p.qtterns (item (2) 11bove) or in the subscript corresponding to .a given verb sense, or both places.
- Scripts. 'lhese serve to connect goals with methods for achieving them. In addition, scripts may be used to describe typical or stereotyped sequences of events (e.g. earthquakes cause structural d3111age to buildings, may leoo to tidal waves, aftershocks, rescue of victims, repair of damage, etc.). 'lhe scripts may in turn include subscripts. In geqeral each goal may refer to a number of scripts (see [21 j).
4.1 '3ubscripts
Subscripts are at the heart of the work I 3111 describing here. 'lhey serve as a bridge between the case fr.'lffies, selection restrictions, etc. on one hand, and spatial descriptions on the other. 'lhey are important for several reasons:
- Adverbial modifier representation. Subscript'l provi:le a framework in which adverbial modification h'ls a natural represent'ltion. To take some specific examples, consi'ier "almost"; 3os suggested in the example of handling of (38) verbs cap.qble of being meaningfully modified by almost (e.g. hit, break, reach, finish, buy, etc.) each have a step in the body of their associate1 subscript which, if not completed, allows the action up until that step to be described as "almost h'lving h'lppened", whereas if th'lt step does occur, "'llmost" can no longer be used to refer to the action. Adverbs such as hard, gentle, soft, violently, etc. refer specifically to subscript steps that involve application of force, transfer of momentum, or related concepts. Adverbs such as fast, slow, smoothly, abruptly, suddenly, etc. refer to subscripts involving motion, and require th3. t the system h'lve a notion of "normal'' duration or speed of motion in order to interpret them properly. Other modifiers such as plinfully, successfully, happily, etc. emphacize effects; still others, such as gently, quietly, rudely, cleverly, etc. require special knowledge of behavior that is beyond the scope of my present work.
- Verb sense representation. Subscripts permit easy distinctions between verb senses, even where the case frame constituents do not allow for any distinctions. 'lhus, for example, ny (meaning "go by plane") can be easily distinguished from ny (meaning "pilot a plane"), because the subscripts attached to each sense represent very different sequences of actions. Th.e detail in subscripts, when comp1red with the current context, adds a great eal of information useful for selecting appropriate word senses.
- ( 3) Common sense. Subscripts provide ways of organizing pointers to relevqnt real world physic'll forces, behaviors, constraints and properties, which make it possible to bring general knowledge to bear on language understanding.
5. Rel'lted AI work
The two questions of plausibility and representation have not usu'llly been linked. Case frames 'lnd semantic markers have been used to judge plausib~li ty, as have Wilks' "preference semantics" structures 1.28] but neither is a serious knowledge representation candidate. Knowledge representation schemes (KL-ONE, NETL, SNeP3, KRL, etc.) h'lve not been much used to judge plausibility rather, they h'ive been used for storing information which is fed to them by other pr~rams. A partial exception is the work of Schank et al \on scripts as well as primitives and structures of conceptual ·dependency) which try to combine both plausibility !!J1d representation, altoough at a much more abstr'lct level than the spatial models being proposed here. 'lhe representation of adverbial modifiers is also central here. Relatively little work has been done on this topic. Ovenll, however, McDermott's work, both on :l'OPl,.E [16] and the more recent work on sp1tial inferences L 17 J, seems to me to be most simihr to what is sug~sted here. other clear intellectual predecessors are L 14 ], [29], [22 ], [ 19 ], and [ 13 ].
A k~y pr'oblem is the 1evelopuent of an adequste s.~t of spq. ti'll pr'imi ti ves. Ehg lish p;eposi tions a r~ imdequ'i t"l, so whera can llle look for gu1.dance? We have been stu.jying non-Indo-Euro~an l:mg\13.ges, st leR.st oome of which (e.g. Jingh~w. s Tibeto-Burmese lsngusge, 'lnd Tsr'iscan, s nstive American language) are far more regul'ir an1 precise thsn English in their expression of spati'll snd t81!1poral ;me'l.ning. Jinghpiw, for example, has oome fifty diff3rent verbs for 1ascribing cutting actions (compare with Ehglish examples sbove), and a like number for describing diff"lrent finger positions ani motions. More will be reported on this study in the ne'l.r future.
Suggestive and very interesting materisl on gmmmaticsl cl).l8S for spatial anj r.pornl informstion is foun:l in Tal my L 2'3] snd Fillmore [ ~ , snd on building and switching contexts in Cllafe [4]. I also intent to integrste my work on object shape description (25].
@inproceedings{ijcai1981p1,
title = {Toward a Detailed Model of Processing for Language Describing the Physical World},
author = {Waltz, David L.},
booktitle = {Proceedings of the Seventh International Joint Conference on
Artificial Intelligence, {IJCAI-81}},
publisher = {International Joint Conferences on Artificial Intelligence},
editor = {Patrick J. Hayes},
pages = {1--5},
year = {1981},
month = {8},
note = {Natural Language},
url = {https://ijcai.org/proceedings/1981/1},
}
TY - CPAPER TI - Toward a Detailed Model of Processing for Language Describing the Physical World AU - Waltz, David L. T2 - Proceedings of the Seventh International Joint Conference on Artificial Intelligence, IJCAI-81 PB - International Joint Conferences on Artificial Intelligence PY - 1981 SP - 1 EP - 5 UR - https://ijcai.org/proceedings/1981/1 ER -Copied
Machine-readable versions of this paper, for tools and AI agents: