Coherence and Interpretation in English Texts

1. The Inferencing System

The data of the semantic analyzer is of two sorts -- the Text and the Lexicon. The Text may be thought of as episodic memory and the Lexicon as permanent memory . The Lexicon contains type nodes, the Text token nodes. Initially, the Text contains the information explicit in the paragraph being analyzed, expressed in a simple logical notation produced by a syntactic preprocessor (Grishman et al 1973', Hobbs and Grishman 1977i. The logical notation makes explicit the functional relation­ ships between elements in the sentence by express­ ing the information in the form of logical proposi­ tions, and distinguishes between material which is asserted and material which is grammatically sub­ ordinated, or presupposed. In the course of seman­ tic processing, certain semantic operations augment the Text by drawing inferences, interrelate it by identifying phrases in different parts of the para­ graph which refer to the same entity, and structure it by discovering relations between sentences.

The semantic operations work by searching for chains of inference in the Lexicon, which is the store of lexical and world knowledge. The Lexicon contains the "definitions" of English words, where a definition is viewed simply as the set of infer­ ences which may be drawn from the use of that word. These inferences include "superset" relations, such as the fact that a bank is a building,

and that to walk is to go from one place (z 1 l to another (z 2 ).

WALK: Nyl ( '3 z 1 ,z 2 J (walk(y) + go(y,z 1 ,z 2 l) These axioms are part of the definitions of "bank" clude lexical decompositions, e.g. that for y 1 to go from y 2 to y 3 is for y 1 ·s being at y 2 to become

and "walk", respectively. The definitions also in­ or change into y 1 ·s being at y 3 :

(In the use of higher predicates like "become", we go beyond standard predicate calculus notation.) Also included are axioms having more of a "world knowledge" flavor, such as the fact that a building

has a roof:

BUILDING: (Vy) (3zl (building(y) + roof(z,y))

No distinction is made among these various sorts of axioms, nor between "linguistic" and "non-linguis­ tic" knowledge.

Inferences are not drawn freely, but only in response to the specific demands of semantic opera­ tions. These demands take two forms, i.e. they in­ voke one of two inferencing procedures:

Forward inferencing: From p(A) try to infer something of a given pattern.

Backward inferencing: Find something in the Text from which p(A) could be inferred.

A dynamic ordering determined by context is imposed on the axioms in the Lexicon. The axioms are divided into clusters, perhaps overlapping, roughly according to topic. The clusters are given an initial measure of salience according to their anticipated relevance to the text at hand. The measures of salience are modified in the course of semantic processing in response to changes in top­ ic. When a fact in a cluster is used, the entire cluster is given maximum salience; while the facts in a cluster are not being used, its salience de­ cays. The Text itself is treated as having maxi­ mum salience.

The search for a chain of inference is then an ordered heuristic search whose evaluation function depends upon this dynamic ordering and upon the length of the chain of inference. The search is discontin.ued after the evaluation function reaches a certain threshold. For example, a chain of in­ ference of five steps might be found if the axioms involved are of high salience, whereas with axioms of very low salience the search would be cut off after just one step.

This is more than just a device for efficien­ cy. There may be more than one chain of inference satisfying the requir~~nts, each leading to a dif­ ferent interpretation of the text. The one chosen is the first one encountered in the ordered heuris­ tic search. Thus, not only the inferencing process but also the interpretations it produces are made highly dependent on global context, and context is not merely the current state of the Text, but also the current state of the dynamically ordered Lexi­

con.

The bulk of the axioms are classed as only "Normally" true, which means an inference is not drawn if it would result in a contradiction. For example, we can normally infer from "John walks" that "John balances on his feet", but if we have "John walks on his hands", we simply refrain from drawing that inference [cf. Cercone and Schubert

1975].

There are four principal semantic operations:

1. Predicate interpretation, which is de­
2. scribed in Part 2.
3. Detection of intersentence relations, de­ scribed in Pa~----
4. Knitting, which identifies and merges re­ dundancies and secondarily resolves some pronoun references.
5. Finding ~ antecedents of definite noun phrases and the remaining pronouns.

The last two operations are described else­ where [Hobbs 1975, 1976a, 1976b],

2. Interpretation and Coherence within Sentence Boundaries

The basic question addressed in this section is how to interpret words in context. To under­ stand the scope of this problem, consider the var­ iety of meanings the word "go" may take on, even in the reasonably well-behaved sublanguage of algo­ rithm descriptions. In

Go to step T4.

"go" means that the processor next executes the in­ struction at step T4. In

it means that the value of N successively equals

the integers from 1 to 100. In

Next we ~ through the linked list looking for

marked nodes

"go" means a pointer variable successively points to the ·nodes in the linked list. If we look at language in general, examples multiply.

The problem we face then is how to define words like "go" in a succinct manner in such a way as to allow this profusion of possible interpreta­ tions. Heretofore, the typical approach has been to specify a large number of the word's possible uses, for example, by means of a list of environ­ ment - interpretation pairs [cf. Riesbeck and Schank 1976]. This may be feasible for words with a small number of quite distinct meanings, such as the adjective "fair" which can mean "mediocre", "light-complexioned", or "even-handed". But for the most common words, like "go", it is utterly inadequate. First of all, it fails to capture sub­ tle similarities among the various uses. Secondly, there are too many possible uses -- the list would never end.

Our approach is rather to define a word in as general a way as possible, with the ideal of having one sense of the word cover all its uses. The method is to define the word in terms of very gen­ eral predicates. When the word is encountered in context, a semantic operation, called predicate interpretation, seeks a more specific interpreta­ tion for the general predicates.

In this method, "go" can be defined as follows:

That is, first from the explicit properties of y 1 and y 2 , try to infer "at(y 1 ,y 2 )" -- try to dis­ cover in what sense y 1 can be at y 2 , where "at" is a very general predicate in the sense that it can be inferred from many other relations between two entities. The information in the chain of infer­ ence leading to "at(y 1 ,y 2 )" provides the more specific relation between y 1 and y 2 · Then we are free to infer that that relationsh~p between y 1 and y 2 changes into the same relationship between y 1 and y ·

ior example, if (1) is being analyzed, predi­ cate interpretation must determine in what sense N could be at 1. It does so by accessing an axiom in

the definition of "variable":

That is, if a variable is equal to a number, we will say it is at that point on the number scale. "At" is thus a generalization of "equal", and when we say that a variable comes to be at a number, one of the things it could mean is that1the variable comes to equal the number. If this is the first chain of inference found in the ordered heuristic search, then this is the interpretation assumed.

Our approach overcomes the objections to the multiple-sense approach. First of all, the subtle shades of similarity and difference among various uses of a word are captured by the degree to which the general predicates are central or prevalent in the definition of the word, and on the similarities among the chains of inference interpreting the gen­ eral predicates. Secondly, there is no need to anticipate all the possible uses of a word, for its particular interpretation is determined by means of a search through the collection of world knowledge axioms which is needed anyway in a natural language understanding system.

It may be objected that (2) is ad hoc and listed in the definition of "variable" only to handle the one sentence. But in fact, (2) is an example of one of the basic principles upon which the deepest levels of the Lexicon are organized. Linguists and psychologists have frequently noted that in English and other languages, one often appeals to spatial metaphors when speaking of abstract ideas [Whorf 1956, Clark 1973, Jackendoff 1976]. These permeate language to such an extent that they generally escape notice. For example, in the previous sentence-- "permeate", "extent",

"escape notice". We speak of "high hopes", "high prices", "deep thought", "being in politics", "a book~ sociology", "gettinq the idea", "giving an example".

Now the predicate interpretation method allows us to turn the spatial metaphor into a powerful aid in the analysis of English texts.

At the base of the Lexicon are a small set of cognitive primitives with a highly spatial or vis­ ual flavor. Among these are the notion of a scale, which results from conceptualizing a "becomingn­ from one state to another, and may be thought of roughly as a partial ordering; several notions which give structure to scales, including a point being ~ a scale or a member of the partially ordered set, one point exceeding another on the scale, and three operators, Lr, Md, and Sm, which isolate imprecise overlapping-regions near the top, middle, and bottom of a scale, respectively; and the notion of an entity being at a point on a scale, or an entity being at another entity. At, scale, and the Lr, Md, andlSm operators are gener­ al predicates capable of a wide variety of possible interpretations in a given context.

The most common words are .then defined in terms of these primitives, For example, verbs of motion, like "go", have an underlying at which can get interpreted as physical location or-some meta­ phorical location, as we have seen. Many preposi­ tions also have an underlying "at"; "under" can be defined in terms of the relative positions at which its arguments are located on a real or metaphorical vertical scale -- metaphorical to handle such sen­ tences as

The GNP is under what it was last year.

A word like "greater" is defined in terms of exceed and scale. Depending on the arguments of "greater", scal;-c;n get interpreted in various ways, A word like "tall" gets defined usually in terms of the specific physical height scale but also in terms of the "Lr" operatoq the interpretation of "Lr" de­ pends on context -- whether it is a "tall building" or a "tall glass of beer".

Many interpretations are achieved by defining other relations as instances of "at", as in (2), Moreover, "at" can be related to predication in general. We take our cue from equivalences such as

John is hard at work = John is working hard, and similar "argument + preposition + predication"

combinations, such as

John is in control of the situation z John

contrOls the situation.

The czarina is under the influence of Rasputin z Rasputin in£lU;nces the czarina.

The rule is

(Vy,p) (3zl (p(y) + at(y,p(z)).

That is, if an entity is at a predication, it is possible that the entity is one of the arguments of the predication.

Jackendoff [1976] has classified word senses of verbs of motion into positional, possessional, identificational, and circumstantial modes. our approach now provides an explanation. Consider Cinderella's coach turned into a driveway.

Cinderella's coach turned into a pumpkin.

In the first, "turned into" is in the positional mode, or has a positional interpretation. In the second, it has an identificational interpretation; the coach actually becomes a pumpkin. In our ap­ proach, "turn" and "into" are defined in terms of, among other things, the predicate "at". One as­ pect of the meaning of "X turned into Y" may be re­ presented

The nature of Y is explored to determine in what sense X could be at Y, If Y is a driveway, the fact that a driveway is intended to be a path leads to the spatial interpretation of "at". If Y is a pumpkin, the spatial interpretation is not as read­ ily available as the identificational, and (3) be­ comes, by "at a prediction",

become( ··· ,pumpkin(X)).

The predicate interpretation approach also gives us a handle on the analysis of adverbials and adverbial adjectives. "Slow" can be defined as follows:

slow(y):

Required: y=move(z 1 l

Infer: on(z 2 ,Sm(z 3 )), speed(z 2 ,move(z 1 )l,

Speed-scale(z 3 )

Here "move" is a general predicate. We must find the most salient motion associated with the gram­ matical argument of "slow". Then we can infer that the speed of this motion is in an imprecisely de­ fined region at the lower end of the speed scale, which is isolated by the operator "Sm". "Sm" may itself receive a more specific interpretation in context. If we encounter "John walks slowly", we use the fact

(Vy) (walk (y) + move (y))

to interpret the motion as the forward motion of the walking. In "slow watch" we use a fact like

(Vyl (3z 1 ,z 2 l (watch(y) +

(hand(z 1 ,y)&hand(z 2 ,y)&move(z 1 )&move(z 2 lll

(A watch has hands that move) to interpret the mo­ tion as the motion of the hands. Similarly, in "slow race" the motion is the forward motion of the competitors; in "slow horse" its running at full speed, generally in a race (providing facts about horse racing are sufficiently salient in the Lexi­ con); and in "slow business" the metaphorical mo­ tion of the outflow of goods and the inflow of money.

The approach to defining and interpreting words which has been presented here is very compel­ ling on several counts. It allows us to collapse a seemingly large number of senses of a word into a very few. It is appealling as a psychological model since it rests on the close connection be­ tween visual imagery and language. It is a method which treats virtually everything as a metaphor to be interpreted in context, and thus provides a framework in which spatial metaphors, as well as much other metaphor, can be treated not as an anom­ aly but as the natural state of affairs.

3. Coherence Beyond S~ntence Boundaries

Where there is an explicit intersentence con­ nective, such as "but", "because", "i.e.", the problem of determining or verifying the relation between the sentences becomes just a special case of predicate interpretation. The connective is viewed as a higher predicate whose arguntents are the assertions of the sentences it connects. The requirements associated with the connective are just those conditions which make the sentences stand in contrastive, causal or paraphrastic rela­ tion to each other. However, since most intersen­ tence relations are implicit, a separate semantic operation is required to recognize them.

We attempt to determine intersentence rela­ tions by seeking to match the current sentence and the previous text against a small number of the most common patterns of coherence that link senten­ ces. The patterns are stated in terms of inferen­ ces that can be drawn from the current sentence and a previous sentence, and the modification to be performed on the Text if the pattern is matched. Among the patterns we have studied are Temporal Succession, Cause, Contrast, Violated Expectation, Paraphrase, Example, and Parallel. It is not claimed that this list is exhaustive, but it has been adequate for the quite diverse texts which have been examined so far.

The intersentence operation keeps these pat­ terns on a goal list while processing a sentence. Strengths are associated with each pattern, indica­ ting the urgency with which we want it matched. The strength depends on the type of text being an­ alyzed, the presence of conjunctions and certain adverbs, and the expectations created in the analy­ sis of the previous text. The heuristic search which seeks chains of inference to satisfy the pat­ terns is ordered by the ·strength of the goal as well as the length of the chain of inference and the salience of the axioms in it.

To see how the patterns are to be stated and how the pattern-matching proceeds, consider the following example:

Republicans were encouraged about their prospects. The party chairman believed that Dewey would be elected.

Intuitively, this is an instance of the Example pattern. To recognize it computationally requires us to access our knowledge of "Republican", "en­ courage", and so forth.

More precisely, let the Example pattern be stated as follows:

The predicate and arguments of the assertion of the current sentence stand in a sub­ set or element-of relation to those of the previous sentence.

Suppose "be encouraged" decomposes into "be­ lieve something good will happen". Sentences are processed from the top-down, so that in the second sentence "the party chairman believes" is process­ ed first. Since the party chairman is a member of the set of Republicans and since the "believes" matches the "believe" of "be encouraged", the first part of the Example pattern is matched . Be­ cause of this partial match, the strongest goal for the processing of the "that" clause is to show what is believed is that something good for a Re­ publican will happen. This is done by accessing the fact about a political party that one of its purposes is to win elections, and the fact about Dewey that he is a Republican.

The target representation for the analysis of a paragraph is a tree-like structure indicating the relations between sentences in the paragraph. (A brief example of the analysis of a paragraph from Newsweek, together with the target representa­ tion produced, is given in Part 4.)

The specifications for the other patterns are as follows: ·

1. Overlapping Temporal Succession, or "then". There are two varieties of this pattern. (In all the descriptions of the patterns, s 2 refers to the current sentence and s 1 to an eligiDle previous sentence. l

1. s 1 asserts a change whose final state is implicit in s 2 ·
2. Implicit in s 1 is a state which is the initial state of a change asserted by s 2 ·

The text

Walk out the door of this building. Turn left. Walk to the end of the block.

exhibits two varieties of the pattern interlocking. The first sentence asserts a change in location, the final position is implicit in the second sen­ tence and is the beginning of the change of posi­ tion described in the third. Moreover, the first sentence implies one orientation, the third sen­ tence implies another and the second asserts a change from the former to the latter.

The patterns are exhibited twice in the fol­ lowing text:

He observed a badly joined connection in the mechanism. He took it to his work­ shop to fix.

The first sentence describes a change in knowledge, a coming to know that something is broken. The second sentence, via "workshop" and "fix", implies that knowledge.

1. Cause: There are two varieties of this pattern:
2. Find a causal chain from s 1 to s 2 ·
3. Find a causal chain from s 2 to s 1 ·

The second variety is exemplified twice in the text.

He was in a foul humor. He had slept badly during the night. His electric blanket had worked only intermittantly.

The causality between the third and second senten­ ces can be established by noting that the purpose of a blanket is to enable sleep, so that a blanket failing in its purpose (not working) causes one not to sleep. The causal chain between the second and the first is that sleep enables one to overcome fatigue and fatigue frequently brings on a foul humor.

The role of this relation in narratives has been studied by Schank (1974) and Run·elhart (1975).

1. Contrast: Letting "element" refer to either the predicate or one of the arguments of a statement, the Contrast pattern may be stated as follows:

From s 1 and s 2 infer Si and s2, respectively, where

1. Si and s2 have one corresponding pair of elements which are contradictory or lie at opposite ends of some scale;
2. the other corresponding pairs of ele­ ments are identical or belong to the same small set (i.e. are "similar").

Consider the text

You are not likely to hit the hull's eye, but you're more likely to hit the bull's eye than any other equal area.

Sl is the first clause, 52 the second. From Sl, we can infer, by axioms associated with "likely", that the probability of hitting the bull's eye (call it p 1 ) is less than .5, or whatever other value counts as "likely". From s 2 , we can infer that p 1 is greater than the typical probability of hitting those other equal areas (call it p 2 ). Thus we have

The predicates of Sl' and S2' -- "<" and ">" -- are contradictory. The first arguments are identi­ cal-- "p 1 ". The second arguments -- ".5" and "p 2 " -- are similar in that both are probabilities. Hence, the Contrast pattern is matched.

In the text

Research proper brings into play clockwork­ like mechanisms; discovery has a magical essence.

"research" and "discovery" are viewed as similar elements and "mechanistic" and "magical" as lying at opposite ends of some scale.

1. Violated Expectation: This pattern holds if from s 1 one can normally infer S, but from s 2 one can infer not s. Not S is what we are expected to believe.--- ---

In

This paper is weak, but interesting

the first clause Sl would lead us to infer that the paper is not suitable for presentation, but the second 52 contradicts this. It is the infer­ ence from the second clause we are to believe.

1. Paraphrase: It is frequent for stretches of text to be attempts at the successive approxi­ mation of a meaning, or attempts to overcome the possibility of misunderstanding. The pattern may be stated

From the assertions of s 1 and s2 , infer Si and s~, respectively, where Si and s2

are the same except that either

1. an argument of s2 is more fully or pre­ cisely described than the corresponding argument of Si, or
2. s2 has adverbJ.al modification S:i lacks.

Consider the text

This immense tract of time is only sparsely illuminated by human relics. Not enough material has yet been found for us to trace the technical evolution of East Asia.

From our knowledge of "sparse" and "illuminate", we can infer from the first sentence Sl that the relics fail to cause one to know the "contents" of the immense tract of time (call this statement Sl'). From "not enough" in the second sentence 52, we can infer that the material fails to cause us to know the "contents" of the technical evolution (call this 52'). Sl' and S2' both have the form

fail(cause(A,know(we,B))), contents-of(B,C)

Sl' describes A as "relics" while 52' uses "mater­ ial", near synonyms. C, which in Sl' is described as an "immense tract of time", is specified more precisely in 52' as "technical .evolution of East Asia". Hence, the Paraphrase pattern is recog­ nized.

1. Parallel: The pattern for this relation

is From the assertions of s 1 and s 2 , infer Si and s2, respectively, where the correspond­ ing pairs of elements are identical or members of some small set.

At least one pair must be non-identical. This pat­ tern is frequently keyed by similar syntactic structures.

In

Set stack A to empty, and set link variable

P to ROOT.

the predicates and agents of both sentences are the same, A and P are both data structures and A being empty and P pointing to ROOT are both ex­ amples of plausible initial values.

4. An Exal!l.ple

In this section we examine one paragraph very briefly to see the discourse structure that is produced. A detailed analysis has been carried out by hand and is available in Hobbs (1976b). Implementation of the analysis is in progress. Unsurprisingly, it assumes state-of-the-art syn­ tactic preprocessing. Semantic analysis requires a Lexicon of at least several hundred axioms en­ coding knowledge ranging from properties of basic spatial, temporal, and mental concepts to very specific facts about American politics. The chains of inference needed to establish some of the intersentence relations are quite long in some cases and it is not yet clear what can be expected from a computer implementation.

The paragraph comes from Newsweek. The clauses and sentences are labelled for subsequent

reference.

[SA:] All of this only deepened the despond of the President's own part~ [SB:) and so inten­ sified his isolation in a time of maximum jeo­ pardy. [S :) The conviction is widespread among RepJ5licans that [S 0 :J time is running out on Operation Candor [SE:) --that Mr. Nixon must clear himself by early in the new year [SF:) or lose whatever slipping hold he has left on the party. (SG:) But they commonly doubt that the message is getting through to the President, (SH:) and now their discourage­ ment has been compounded by the news that (s 1 : J Mr. Nixon's two savviest political hands, Melvin Laird and Bryce Harlow, plan to quit as soon as Ford settles in. (SJ:) "When they go," mourned one Senate GOP leader, "we'll have no one there to talk to. (SK:) We have nothing to say to Ron Ziegler, (SL:) and Al Haig's never been in politics (SM:) --he can't even spell the word 'vote'." (Newsweek, Dec. 17, 1973, p. 25).

Between SA and SB' there is an explicit caus­ ality, keyed by "so", but it is unlikely that we can verify it. Therefore its verification is placed high on the goal list for the rest of the paragraph. It is frequent for a paragraph to pro­ vide verification for its topic sentence. SE and SF are ·linked by the "or" that introduces the cause of Nixon's obligation. Together the two are recognized as a r araphrase of s 0 , as follows: From S we infer that Operation Candor requires time tg achieve its goal. Its goal is for Nixon to clear himself, which ma~ches part of SE. The remainder of the ~~tch is achieved by detecting the correspondence between "require time" and "must ..· by early in the new year." The para­ phrase is embedded in SC, whose Expectation is Violated by SG. The expectation is that normally when one is involved in a situation, he knows it. The stretch of text from SC to SG matches the ini­ tial state of SA -- it describes the Republicans' state of discouragement. Consequently, the Tem­ poral Succession pattern is high on the goal list for the next sentence, for that would allow a match with all of SA. SH matches the pattern and the result qualifies as an example of SA, since discouragement being compounded matches despond deepening.

SJ is an Example of S because of the match between "one Senate GOP leMder mourned" and "their discouragement". SL and s might or might not be recognized as a Paraphrase~y processes that won't be described here. To recognize this as being in Parallel with SK requires us to infer from Haig's not being in politics that Haig does not engage in political activities, in particular, he does not confer with others to achieve common political goals. These sentences in turn constitute an ex­ ample of the Senate GOP leader having no one there to talk to.

The resulting discourse structure is given in Figure 1

One thing that is apparent from this tree is that it is not complete. We still have not veri­ fied the causality between S and S · The verifi­ cation is passed as a goal t~ the n~xt paragraph, and in fact that is precisely what the next para­ graph does.

Ultimately, we will want to be able to deter­ mine the message of the text as a whole, or what amounts to the same thing, construct a summary of it. This will require more than the methods of analysis presented in this paper can produce. A­ mong other things it will require information about the redundancies in the paragraph and about the obstacles that are overcome in the paragraph on the way toward the author's or the characters' goals. But the structural information discovered by the intersentence operation and illustrated in Figure 1 will be a very important element in any such construction.

5. Conclusion

What has been described in this paper are se­ veral methods by which the structure and complex meaning of a text can be built block by block from the bottom up in a regular and parsimonious manner that can be characterized. They involve close analysis of the ways in which words and sentences interact. Although they assume a great amount of world knowledge, the knowledge is encoded in a rather simple representation and the methods stress the domain-independent ways in which the knowledge is used. Thus, our approach may be viewed as com­ plementary to much current work in discourse analy­ sis, where highly structured knowledge of the situ­ ation being described is assumed and guides the analysis of the text. OUr approach consequently represents a step toward an accommodation of the richness, subtlety, and flexibility of language that make it interesting.

Acknowledgements

I have profited from discussions with Stan Rosenschein and Harry Elam on various aspects of this work. This research was partially supported by CUNY Faculty Research Award Program Grant No. 11233, and National Science Foundation Grant No. MCS 76-82164.