Memory and Inference

1. Introduction

It has been &enerally reco&nized that world tnowled&e is important for makin& inferences, and that the structure and or&anization of knowled&e in memory has important consequences in determinin& what inferences are made. For example, Schank and Abelson (1977), Charniak (1978), Cullin&ford (1978), Wilensky (1978), Lehnert (1982), and Dyer (1982) are all describe systems that use a particular kind of knowled&e to make particular cl386ea of inference.

However, the overall desi&n the inference en&ine for text understand­ in& remains obscure. Most of research has focused on the nature of the knowledge needed. The theory of processin& per se has &enerally been rele&ated a secondary status, and usually developed only as far as was necessary to demonstrate the efficacy or one's tnowled&e structures.

The view taken here is that adequate theories or memory and the structure of knowled&e largely eliminates the need for a theory of inference. In this formulation, what &ets inferred is a function of what &ets accessed in memory. This, in turn, is determined by the structure of tnowled&e, and by &eneral memory acceas routines that are sensitive only to the structure and or&anization of knowledge, not to its content.

This view is parsimonious to the one espoused by Quillian (1008). Quillian's notion of representation was primitive by today's stan­ dards, however, and he was not concerned with the notion of infel'" ence per se. A more sophisticated view of inference is found in Chal'" niak (1982). His approach is similar to the one pursued here, in that both theories endorse hiply declarative representations and fairly &eneral schemes for manipulating them. However, I am concerned leas with the details of the implementation, and more with the overall architecture of the undel'6tander.

In this paper, I 81'6t present an abstract characterization of the understandin& task. Then a particular realization for this characteri­ zation is pven.

Memory and Inference

Robert Wilen.!ky

Division of Computer Science Department of EECS Univel'6ity of California, Berkeley Berkeley, CA. 94720

In this discu88ion I refer to the representation produced by the story understander as a con·trual of the input. I call the process of decid­ ing to include a particular fact in a construal determination. In addi­ tion, I shall use the term 1181Umption to refer to the kinds or infel'" ences used in story undel'6tanding, to distinguish these from the more general collection or processes that include la&ical deduction.

I. Prlnclpls ot Text Comprehenalon

I fil'6t attempt to characterize the text understanding process by characterizing those facts whose determination constitute a satisfac­ tory construal of the input. This characterization is given in terms of a set of principles. These are as follows:

The Principle of Consistent Construal - View the inputs as instantiating a set of frames that are consistent with one's world knowledge.

The Principle of Concretion (Overcommitment) - Determine as specific a fact as is possible from an input.

The Principle of Least Energy - Make only the minimal 388umptions neceMary to determine a fact.

The Principle of Exhaustion - Determine enough facts to account for all of the input.

The Principle of Poignancy - Determine those f~mes that are pertinent to the intent of the discourse.

The Principle of Consistent Construal states that, as elements of a text are viewed as instantiating &eneral frames, one should favor interpreting inputs as instantiating those frames that provide a con­ sistent construal of the il'put.

For example, suppose we were &iven the sentence "The bottle fell oil' the table and it broke." One construal of this sentence assumes that (1) the bottle hit the ftoor, (2) it was the bottle (and not the table) that broke, (3) the bottle was somewhat brittle, and (4) the bottle's hitting the ftoor, together with the bottle's brittleness, was the cause of the breaka&e . . The Principle of Consistent Construal approves of this construal because it relates individuals in the story (i. e., the fal­ ling event, the breaking event, and the bottle) to knowledge in Jon& term memory in a consistent manner.

However, the Principle of Consistent Construal does not inform us bow to decide among alternative construal&. For example, another construal of our example sentence assumes someone caught the fal­ ling bottle mentioned above and then broke it with a hammer. How­ ever, this construal seems more ad hoc. We require a way of distin­ guishing among such alternatiYes.

To address this problem, we introduce the Principle of Least Energy. Least Energy states that one should make those decisions that require the most tenable 886Umptions possible. Least Energy is meant be quantify the notion of the most tenable set of assumptions. Here we 886ume that facts are determined when they are supported by a sufficient amount of evidence. A fact that is supported by some evidence, but which is not yet determined, is said to be acliualed. In our processin! model , when the activation of a fact reaches a certain threshold, then the fact will be determined. In addition, we assume that activation is spread to "nearby" facts in memory (d. Anderson and Bower, H7l3).

Our theory assumes that the introduction and determination or a frame or instance of a frame requires a certain amount of activation "energy". At any pven point, various "activated frames" (i. e., those whose activation level is lreater than zero) may have dilferent levels or activation, and therefore some or these frames require more additional energy for determination. The Principle or Least Energy informs us that, when a choice is possible, we should determine those frames requiring the least amount of additional energy.

Least Energy allows for context sensitivity, as previous events may change the activation energy of activated frames. In addition, it allows inherent predispositions into the system. For example, if we mentioned that John ate a hamburger, we would probably assume that it was a normaJ..sized hamburger on a bun, etc. This assump­ tion is made because the frame describin! a prototypical hamburger is liven a disproportionately large initial bias.

Note, however, that so far we have no reason to promote a specific concept (such as a prototypical hamburger) when only a general con­ cept is mentioned. The Principle of Concretion is necessary to sup­ ply this direction. This principle instructs the system to &nd the most specific frame consistent with the input. This is. derived from what I have termed "the First Law of Knowledge Application, " which instructs us always to employ the most specific applicable knowled,e. For example, if the understander learns that John loves Mary, it needs to realize that this is romantic love rather than sibliD! love in order to make accurate predictions about John's behavior. Thus a form of assumption is required in which semantic knowledge from the sentence (e. g., that some sort of love relation exists) is compared against pragmatic knowledge of relationship types (i. e. , the various kinds or love), to &nd the interpretation that the speaker most likely intends.

The Principle of Exhaustion insures that an input is exploited fully in the understandin! process. For example, consider computing an explanation for the sentence "John sneued over. to the door." One explanation for this action is that John wanted to be near the door. Indeed, if the understander knew that John wanted to leave the room, such an explanation is likely to be determined. However, it is important for the understander to realize that this explanation is not sufficient. The deficiency is due to the fact that the explanation explains only the chan,e-of-location part of the action, but it does not explain the manner in which it Wall accomplished, that is, why John sneaked rather than just walked. The Principle of Exhaustion declares this an insufficient result as it leaves an unexplained residue.

The Principle of Poi,nancy distinguishes those aspects of the text that represent its essential content from those that are peripheral. Points are represented as rather abstract frames to which a certain aflective importance is attached. As with any other invoked frame, such frames may be determined if enough evidence can be found. The resulting determined, elaborated frames constitutes the essense or a text.

a. Implementation

FAUSTUS (Frame Activated Unified Story Understanding System) is an frame manipulation proyam created by Peter No_rvig that is designed to embody the principles suggested above. The idea behind FAUSTUS is that the program should make assumptions based not on specific rules of inference, but on ,eneral mechanisms for memory manipulation and on the structure of knowled!e in memory.

FAUSTUS makes a number of kinds of assumptions, including the construction of explanations and the determination of story points in accordance with the principles specified above, by using the following set of fact manipulation primitives:

1. Invocation - Initially considering a fact . Facts are currently invoked because they have been indexed under a component that occurs a11 an input, or because they are explicitly BMOCi­ ated with another invoked fact .
2. Determination - Deciding if enough evidence for an invoked fact exists to infer the fact .
3. Instantiation - Creating an instance of a determined frame to represent its occurrence in a story.
4. Elaboration - Fillin! in an empty slot of an instantiated frame. This process corresponds to the top-down processing performed extensively by systems like SAM and Ms. Malaprop.
5. Termination - Realizing that an instantiated frame is no Jon,er relevant.
6. Attrition - Allowing an invoked but undetermined fact to fade from subsequence consideration. FAUSTUS measures time in terms or the number or fact invocations that have passed. Thus if enough new facts are invoked before a previously invoked fact is determined, FAUSTUS will drop that fact from subsequent consideration.

When FAUSTUS has an input, it first looks at instantiated frames to see if the input elaborates one of them, and then looks for frames that are indexed under the input. For example, frames suggestio! the cause or an event are likely to be invoked by this process. FAUSTUS's determination process then looks for verification of these frames. Supporting evidence includes the previous mention of one or these frames or or their constituents. Thus FAUSTUS checks the story representation · and other invoked frames for corroborating items.

FAUSTUS allows invocation to spread along the constituents or invoked frames. For example, if FAUSTUS knew John want to be at a particular restaurant, and stored in the permanent data base is the fact that being at a restaurant is a precondition for eating at one, then this stage would invoke eating at that restaurant as a candidate action. In elfect, FAUSTUS will have hypothesized the explanation that John intended to eat at the particular restaurant he wanted to be at.

All the facts invoked in this manner are now subject to confirmation. If one can be determined, the others will be either eliminated by exclusion or removed soon by attrition. U not, invocation will spread out a!ain from the invoked facts. That is, if a reasonable expla.na.­ tion can be found within the story, it will be (a long, tenuous expla.­ nation may not be found due to attrition). If not, then intermediate hypothesizes are suggested until a connection with the story can be found.

4. FAUSTUS Examples

Norvig's FAUSTUS implementation uses the PHRAN natural lan,uage analyzer, written by Yigal Arens, as its front end, and the PHRED !eneration system, written by Steve Upstill and Paul Jacobe, for output (see Wilensky and Arens, 1980). These stories demo!l­ stra!es some general features or the understanding process.

·················································

INPUT TEXT:

Ann sot into her car.

She went for a drive.

She arrived at the movie theatre.

She sot a ticket.

INPUT: Why did Ann set a ticket!

OUTPUT: She wanted to see the movie.

··················································

··················································

INPUT TEXT:

Ann sot into her car.

She went for a drive.

She sot a ticket.

INPUT: Why did Ann set a ticket!

OUTPUT: She violated a traffic law.

··········································*·······

In the first version of the story, buy ins a ticket is one of the steps of the "movie-soins" frame, and thus the input is incorporated into this frame, which was previously activated by a reference to soins to the movie theatre. That is, the input is considered to be an elaboration of the "ticket-buyins" sub-frame of "movie-soins." No other interpretation of the input is considered.

In the second version, settins a ticket is not found to be part of any activated frame. lndexins on this input returns a number of frames, includins " traffic-violation" and "movie-soins" . Since traffic viol:.­ tion receives activation from the drivins frame, it is chosen; the other unrelated alternatives are discarded. Thus concretion is performed on the input by ftndins the most precise frame consistent with the input that is supported by evidence from the story.

&. Conelualona

One feature of this theory is that it ties processins very closely to representation (but not entirely, since activation can presumably spread alons non-epistemolosical lii!IIOCiations). Since the theory is heavily dependent upon the nature of repr~ntation, a substantial commitment to a particular repr~ntational scheme is required before the details of the theory can be examined or evaluated. The details of such a scheme are beyond the scope of this paper.

The examples above are certainly not beyond the capabilities of pre­ vious story understanders. The claim made herein is that the pro­ posed formulation will be conducive to the expansion of such syll­ tems. Presumably, if the theory ol processins is correct, knowledse need only be represented correctly in order to be used. While it may · be easier to build an inference eosine tiptly seared to a particular · from of knowledse without solvins th~ representation problems, I suue3t that this only defers a problem that one wiU eventually have to confront.

0. Acknowledgment.

The ideas in this paper emersed from several seminars at the Berke­ ley Artificial Intelligence Research Project and at the Berkeley Cos­ nitive Science Prosram. Participants at this seminar included, in addition to this author, Yisal Arens, Margaret Butler, David Chin, Joe Faletti, Chuck Fillmore, Paul Jacobs, Paul Kay, Marc Luria, Jim Martin, Jim Mayfield, Peter Norvia, and Lisa Rau. Independent work by Chuck Fillmore and Paul Kay and their students in the Readins Project was also responsible for some of the direction this work has taken. In addition, Peter Norvis has been workins on implementins the processins scheme alluded to above; Yisal Arens has been workins on the interaction of memory and linsuistic processes. Many ideas described herein are due directly or indirectly to their research.

'1. Reference.

1. Anderson, J. R., 6l Bower, G. H. Human A61ociative Memorv Washington: Winston 6l Sons, 11173.
2. Chamiak, E. On the use ol framed knowledge in languase comprehension. Artificial lnttlligence. Vol. D. no. 3. 11178.
3. CuUinsford, R . E. Script application: Computer understanding of newspaper stories. Yale University Computer Science Department Research Report #116, 11178.
4. Dyer, Michael G. In-Depth Understandins: A Computer Model of Integrated Processins for Narrative Comprehension. Yale University Computer Science Department Research Report #219, 1982.
5. S} Kay, P . Three Properties of the Ideal Reader. Berkeley Cosni­ tive Scienc ~ Report #3, 1982.
6. Lehnert, W. G. Plot Units: A Narrative Summarization Stra­ te&J . In W. G. Lehnert and M. H. Rinslc (eds.) Strategie1 for Natural LangutJge Proce11ing. Lawrence Erlbaum Associates, Hillsdale, New Jersey, 11182.
7. Quillian, M. R . Semantic Memory. In M L. Minsky (ed.) Semantic Information Proce1nng. MIT Press, Cambridse Masa. 1968. .
8. Schank, R. C. 6l Abelson, R. P . Script·, Plant, Goat. ontl UntlerlltJnding. Lawrence Erlbaum Associates, Hillsdale, New Jersey, 11177.
9. II) Wilensky, R. Under·tantling Goa/-Buetl Storie1. ·Yale Unive~ sity Department of Computer Sciences Research Report #140, 11178.
10. Wilensky, R . What's the Point! In Proceeding· of tile Tllirtl National Conference of llle CantJtlian Societv for llle Computa­ tional Studie1 of Intelligence. Victoria, British Columbia, May 1080.
11. Wilensky, R. Planning ond Underllanding: A Computational Approacll to Human Renoning. Addison-Wesley Publishins Company, Readins, Mass., 1983.
12. Wilensky, R. 6l Arens, Y. A Knowledse-based Approach to Natural Lansuase Processins. University of California, Berke­ ley Electronic Research Laboratory Memorandum No. UCB/ERL/~fS0/3·, Ausust 1980.