Implicational Molecules: A Method for Extracting Meaning from Input Sentences

Summary

A generally overlooked possibility for extract­ ing pragmatic meaning from sentence inputs is dis­ cussed, This possibility draws upon the psycho­ logy of the subjective attribution of causes to stimulus events. Application to the design of conversational programs is feasible, although the present work concerns application to the simulation of a belief system, Simple sentences kno~ by the system are grouped into categorical types, An "implicational molecule" is a meaningful set of sentence types with linked elements, for example: A does X, X causes Y 1 . A wants Y, One important way in which meaning can be ascribed to an input sentence is to use it as the basis for a molecule which is then "filled" by other information in the system, This notion is defined ~thematically, and a programming algorithm outlined, The possi­ bility of recursive use of a molecule-filling rou­ tine gives the process heuristic ~vertones,

Programs for the analysis of natural language have probed intensively into syntactic and seman­ tic problems, but we are still iUite a long way from the achievement of an "understanding machine 11 , In free on-line discourse in English between per­ sons and some of the conversational computer pro­ grams devised to date,1,2 experience has suggested to the investigators that the programs indeed miss a great deal of the "meaning" of what is said,

Although the papers in other sessions of this conference offer persuasion to the contrary, in our opinion what is lacking is not primarily in the area of clever syntactic or even semantic anal­ ysis of the input prose, Rather, the most diffi­ cult problems lie in what has been called the "pragmatics" of language -.- the penetration beyond literal decoding of sentences to the construction of plausible implications from those sentences, Recently there has been much interest in soci~l psychology in so-called "attribution theory", This refers to situations in which an individual is given information about another person's behaviors toward certain objects and must infer the appro­ priate causal influences underlying the behavior, The individual may attribute enduring personal traits to the other, or enduring incentive pro­ perties to the object, or a purposive orientation of the person toward the objects in particular situations, and so on, It is clear that the attribution process is a crucial part of the p:ag­ matic analysis of language, and that conversatlon­ al computer programs with attribution capability would be quite interesting,

A hypothetical a program might do, an on-line sentence example may clarify what such Suppose an individual types such as 1 "I went to three

IMPLICATIONAL MOLECULES! A METHOD FOR EXTRACTING MEANING FROM INPUT SENTENCES

Robert P, Abelson & Carol M, Reich Yale University

drugstores," A syntactically based program might respond, "How did you go to three drugstores?" A semantically based program might respond, "What useful things did you buy in three drugstores?" But a pragmatically based program ought to be cle­ ver enough to ask, "How come the first two drug­ stores didn't have what you wanted?" In order to achieve this response, the program would probably have to be equipped to understand that repeated actions signify persistent intention, that drug­ stores are places where things are intentionally bought if available, that this generally requires no extraordinary skill or resources from the buyer, that persistence generally connotes either marginal ability or environmental difficulty, and that there­ fore the buying intention met with availability difficulties in the drugstores,

Even such a simple example is pitted with pro~lems 1 and one should note that the inference that at least two drugstores didn't have what the person wanted might be incorrect, No pragmatic program can be correct all the time, and it seems quite unlikely that a good program would turn out to make more intelligent attributions than people would-- and people .can 1 of course, be wrong, Any such program at this stage of technology would also be at the disadvantage that its English vocab­ ulary must be sharply limited in order for its operation to be manageable,

The notion of inherent program fallibility does not coexist comfortably with the ideals of artificial intelligence work, but I leave that problem to the artificial intelligence worker, As a psychologist, I am somewhat more interested in the imitation of real intelligence than the con­ struction of ideal intelligence,

This paper is concerned with an idea that may prove extremely useful in the design of pro­ grams for pragmatic analysis, The crucial con­ cept is what I call an "implicational molecule", We have employed this concept in the study of simulated belief systems, though it could also be used in conversational programs, and would be capable in principle of responding intelligently to examples such as the 'three drugstores' prob­ lem,

Implicational Molecules

Implicational molecules are sets of sentences which are "bound together" by psychological impli­ cation, The implications which bind molecules have general applicability to classes of sentence ele­ ments rather than having a domain only of particu­ lar sentence elements, There would not be one set of implications for John and another for Joe ex­ cept insofar as John and Joe might be categorized as individuals with behaviors explainable by very different principles. If John were a "good guy" and Joe a "bad guy" 1 for example 1 then their be­ haviors might be seen subjectively as obeying dif­ ferent regularities, and the set of molecules would embody implications pertaining to Good Guys and to Bad Guys.

Verb elements as well as noun elements are grouped into categories for purposes. of molecule definition. Representative categories might be Possession, Wanting, Doing1 Liking, Prevention, Hurting, and so on 1 all representing kinds of relations between specified categories of sentence subjects and sentence objects.

A set of general sentences gains a molecular character when the sentence elements are chained between sentences so as to capture a particular pragmatic implication. For example, where A is an actor, X an action 1 and Y an outcome, the set of three sentences: A Does X 1 X Causes Y, A Wants Y constitutes a molecule expressing the idea of purp~sive action: individuals do things in order to achieve desired outcomes, The particular structure of sentence terms is important. The set: A Wants X 1 X Causes Y 1 A Does Y 1 is a col­ lection that does not make much sense, In the spe­ cification of a molecule, therefore, all the com­ ponents are interdependent. The alteration or permutation of category memberships will generally spoil the meaningful character of the sentence set, This does not preclude, however, the po~si­ bility that a given sentence type can appear 1n more than one molecule, For example, the sentence 'A Does X' from the Intention molecule could also be involved in a 'Servitude' molecule: B Bosses A, B Wants x, A Does X.

Implicational molecules may have interesting properties in connection with a nu~er of areas of human thinking: learning, memory, 1nconsistency resolution and others, 4 Many of these properties are postul~ted in Gestalt theory, particularly as formulated by Heider 1 5 with the molecule acting as a kind of cognitive �estal~. For present purposes one property -- the complet1on tendency -­ is most important. Given all sentences of a mole­ cule except one, there is a tendency for this. omitted sentence to be inferred. It is in th1s sense that I have spoken of implication or attri­ bution, If we are told that A does X1 and X causes y we are likely to infer that A wants Y. Even if we are told only that A does X 1 we may find it plausible to believe that there exists some Y which X causes and which A wants.

By means of molecule completion, therefore, it is possible for extra se~tences to be a~duced from given sentences, that 1s 1 ~or pragma:1c. meaning to be supplied to the g1vens, Th1~ 1dea is the basis for the computer program rout1ne I will describe shortly. First it will be useful to detail the setting within which the program oper­ ates.

Simulation of a Belief System

A program built to use implicational molecules of course must have access to a memory store or dictionary of molecules. It should be obvious that this store of molecules is not fixed by a ·universal principle. Some implicational molecules m~y be operative in one cognitive system but not 1n an­ other Dramatic examples may be provided by con­ siderlng implications a psychoanalyst might draw from information abou~ human actions and feelings in contrast to a layman's implications. One such funny little molecule is the fear-wish paradigm: A Fears X 1 A Wants X, Or this might be embellished to include the meaning of dreams: A Fears X, A wants X A Dreal!BY 1 Y Stands for X, As another case in 1 point 1 a politician might often interpret public events by different principles than would the Man in the Street.

The progranuner 1 then 1 must provide molecules to the system 1 appropriate to its purposes. He must also specify the elemental vocabulary of th7 system, and assign these elements to the categor1es used to define molecular sentences. The category of "actors", A 1 for example, in the foregoing illustrations, is a set of particular noun ~ements denoting the individuals, groups and institutions cognized by the system. Verb elements are also subject to categorization, so that each verb cate­ gory such as Does 1 Wants, etc,, might contain a number of specific verb forms, Finally, the memory of the system should contain a set of particu~ar sentences "known" to the system -- these are 1ts specific "beliefs".

At this stage of program development, it is not reasonable to contemplete human-sized elemental vocabularies, or sets of categories, or lists of beliefs, or stores of molecules, The full richness of unconstrained natural language cannot be handled at the same time that the pragmatic analysis prob­ lem is being tackled, It was decided to limit the range of topical concern of the system, and to standardize the syntax of stored belief sentences in the most rudimentary Subject-Verb-Object con­ struction, Even at that the memory of the system needs to be very large.

However, it is quite easy to produce a great variety of output constructions by means of keyed program exits. It might be possible to accept moderately complex input which is preprocessed into simple sentence form by some such means as the DEC and REC functions of Weizenbaum's ELIZA program,l but we have not done this, Instead, the input is limited to the simple sentence forms cognized by the central system.

In exercising a belief system such as des­ cribed, our intent has been to design a program which will operate upon input sentences in such a way as to understand or explicate them. Part of this explication would involve a response of credulity or incredulity, along with subjective reasons why the input should be believed or dis­ believed, This aspect of the program is what in previous papers 6 · 7 I have called The Credibility Test. The remaining aspect of the program consists in the Attribution Process -- the discovery or construction of pragmatic implications from cre­ dible inputs.

The output from this program can in principle serve to determine further system activities, such as changes in the stored memory structure, or the formulation of questions to elicit further infor­ mation from the input source, etc. At present, however, the output to each input sentence trig­ gers no further processes. A response is printed and the sequence terminates 1 liDless another com­ pletely independent input sentence is presented to the program.

Interaction with our program thus need not be in the conversational mode, We have found that the frustrations of attempted time-sharing, when the program involves string and list processing and a very large memory, can outweigh the potential ad­ vantages, Our present program version, written in SNOBOL3 1 is batch processed on the 7094,

The particular vocabulary, concept categories, beliefs, and molecules of one realization of the system concern the topic of United States foreign policy, There are some 500 nouns in 15 categories, 100 verbs in 11 categories, 300 beliefs, and 12 molecules, The ideology presently represented is the extreme right-wing point of view of a well­ known ex·ex-~enator, Other points of view could of course be handled in the same programming sys­ tem by changing the memory store. Our system de­ sign intention was that the replies by the auto­ mated Senator to the input statements presented him should simulate the replies the real Senator might conceivably make. Early tests of such a simulation were not notably successful, but more recently with the inclusion of the implicational molecule concept in the simulation system, a fair degree of success has been achieved, The details of this test are to be reported in a separate paper, How implicational molecules are used in the simulation program will now be described,

Using Implicational Molecules in the Simulation

The heart of the simulation program is a sub­ routine which attempts to "fill molecules", The input is a single sentence (in the simple Subject­ Verb-Object form), and the output is ~ither a filled molecule containing the input sentence and the appropriate other sentences needed to complete the molecular set, or a failure signal indicating that the system cannot explicate the input sentence, Following failure to explicate, the sub-routine may be entered again in an attempt to explicate the opposite .of the input sentence, thus enabling the system to distinguish between unacceptable and inexplicable inputs,

If the molecule-filling routine succeeds, then of course the other sentences comprising the completion of the molecule provide the verbal materials for the explication, Interestingly, the "name" of the molecule also provides valuable explicative material for use in embellishing the output, To exemplify this, consider an input sen­ tence of the form, "A Does X", Suppose it turns out that this is explicable via the purposive action molecule: that the particular x performed by this particular actor, a, causes a particular y which a wants. The name-of this molecule may be considered to be "PURPOSE (Y ,A,X)", and the program may plausibly include in its verbal output the no­ tion that y was ~'s purpose when he did x.

The details of the program concern the heuris­ tic procedures by which other sentences are collec­ ted alongside the input sentence to fill a molecule, These procedures are entirely algorithmic when cer­ tain depth-and-sequence-of-search parameters are arbitrarily fixed at simple values, although the necessity of fixing these values to achieve algor­ ithmicity indicates that the process is essentially heuristic. In presenting the program details here, we will adopt simplifications to emphasize the algorithmic aspects,

Table 1 indicates the notation, A vocabulary of V elements are each assigned to one or more of N categories. Categories of noun elements are indicated by X.(i=l 1 2 1 ,,,,I) 1 of verb or relational elements by R.ti=IT1 1 IT2 1 .,,.N), Individual noun elements are aenoted by x~(j=l, 2, ···· J), and verb elements by r;=(j=JT1 1 JTi 1 ,,,,V), Category assignment specifications (x 1 £ ~· Xl,,.,,,), (r; £ ~· R 1 ,,,,,) are assoc ated witn each element, Ordinarily each element will be a member of only one category set, but this need not neces­ sarily be the case,

There is a list L of "legal sentence types", XaR9Xc,each specifying a semantically and prag­ mat1cally reasonable subject-verb-object sequence, Nonsensical combinations of categories, such as inanimate subjects and animate verbs, do not appear, Nor do subjectively unbelievable triplets for example, our simulated right-wing politician does not allow for the possibility of a Free-World­ Nation Attacking a Free-World-Nation, and this sen­ tence type does not appear on its list L,

There is a list B of all specific belief sentences Xj rj xj in the system, This list is · a b c partitioned into blocks of belief sentences of each given legal sentence type (for example, all specific examples of Communist-Nations Subverting Free-World-Nations), This simple blocking arrange­ ment has been found to save a great amount of time during search and matching procedures using the belief list,

Finally, there is a list M of molecules, Each molecule is a particular set of s legal sentence types (most typically, s=3) together with a specification of how particular elements are to be linked across sentences, In general, the molecular set Mh may be denoted:

with specific identifications of subscripts, for example, i1=i7=the index for the category, Actors;

i 3= i 4=the index for the category, Actions; i5=ig= the index for the category, Outcomesi i2=the index for the verb category, Doing, etc,,

The abstract molecule ~ may be said to be realized by a concrete case ~k whenever all sentences in the abstract molecule specification are appropriately exemplified by sentences which are presumed true, A sentence assumes this truth value, PTRUE, either by virtue of being the input to the "fill-molecule" routine · or because it is on the belief list, or because it is calculated to be appropriate to add to the belief list, This latter calculation involves a subtlety to be dis­ cussed shortly,

In the process of attempted concrete reali­ zations of a molecule, the several categories X,R acquire elemental representative x,r. The categories may be considered "free variables" until the program algorithm assigns specific elements to create "fixed variables", This terminology is necessary because of the linkage of X-variables across sentences, It is a convention of the pro­ gram that whenever the same X-category appears in different sentences, the elements representing these categories should either be identical or highly similar, ("Highly similar", in the special sense intended here, means that one element is an in­ stance of the other, These instance relationships are stored in the ~mo~, Details have been given in previous papers, ,a, )

If all free variables in a molecule become appropriately fixed, the molecule is successfully filled, The conditions of such success are appar­ ently quite straightforward mathematically, De­ joting the contents of the concrete molecule

a realization of the abstract molecule ~ is achieved if xj ~ xj whenever ip=iq (the symbol p q

~ denoting "identical or highly similar") and a truth function t calculated on each sentence of mhk yields the value PTRUE:

The apparent straightforwardness of this condition suggests that the program algorithm might be rather simple. While there are some complications involved in trying to accommodate molecules with varying numbers of sentences and varying element linkages across sentences, and there is a certain amount of iteration over multi­ element lists, most of the essential steps are indeed straightforward. First, categories are assigned to the input elements. Then from list M a molecule is found which contains a sentence of that categorical type.

Next, the free variables of the categori;al type sentence are fixed to correspond to the ~nput, and this fixedness is transferred to the appropri­ ate element of another sentence type in the same molecule, Acceptable concrete sentences of this type are generated from the appropriate block of the belief list B, thus fixing another free variable, At this point · one remaining ·sentence in the molecule typically has both X elements fixed, and the value of the PTRUE function is calculated for it, Success either completes the molecule or, in case sentences remain in the mole­ cule specification, new free variables are fixed by concrete sentence generation and/or the PTRUE function is calculated for new fixed-variable sentences, The PTRUE function involves one or two· steps. The first is a matching procedure against the appropriate block of the B-list, looking for a known sentence with all elements "similar" to the tested sentence, ("Similar" is a more relaxed version of "highly similar", mentioned previously,) If this test fails, a second procedure is under­ taken, a procedure which lends the whole routine much greater interest, This procedure introduces recursion in the molecule-filling routine. The second way in which the function PTRUE can be satisfied for a sentence is by entering that sentence in turn as input to the filling routine, A successful exit endows the sentence with presumptive truth, The exist­ ence of this realistic program devic~ means that the attempt to complete molecules assumes an open-ended character, with partial completions leading to new searches which may have partial completions · and so on, There is undoubtedly a psychological limit to the depth such a recursion might reasonably take · and a presently arbitrary program parameter is used to define this limit. Hopefully, empirical work now in progress will shed light upon the appropriate value of this parameter, In order to make the foregoing abstract ex­ planation of the routine more concrete, Table 2 gives some of the categories, elements, and mole­ cules for the right-winger. The components shown of course represent only a small fraction of the total storage. Notice that noun elements can be compound concepts such as situations-helpful-to­ the-Communists, One of the molecules, the crucial one for present purposes · is labeled "No-Win Poli­ cy" by the system, and has the four arguments: Liberals, Western governments, situations-helpful­ to-the-Communists, and standing-up-to-the-Commu­ nists. The sentences of the molecule are1 Western governments promote situations-helpful-to-the­ Communistsi Standing-up-to-the-Communists prevents situations-helpful-to-the-Communistsi Liberals control Western governmentsi Liberals fear standing­ up·tc-th~Communists, Now suppose that the input sentence is "Britain not-interferes-with Czech invasion," The sentence type is "Western governments promotes situations-helpful-to-the-Communists." The pro­ gram consults its molecule list, and finds that the No-Win Policy molecule contains a sentence of this type. Second molecule sentences are now generated by consulting the belief list to locate sentences of the type, "standing-up-to-the-Communists pre­ vents situations-helpful-to-the-Communists." Not all sentences of this type are appropriate, how­ ever, since the specific sentence object must be similar to the fixed variable of the first sen­ tence: Czech invasions. Thus, "Allied airlift overcomes Berlin blockade" is not appropriate. A list of candidate sentences is constructed, and we suppose that the first such sentence is "U.s. shows of force stops Red invasions." For the third sentence of the molecule, the required sentence type is "Liberals control Western governments." Here, the "Western governments" variable is fixed by the input as "Britain", but "Liberals" is a free variable (because it has not occurred before). Suppose "Wilson controls Britain'' is the only appro­ priate sentence of this third type.

Passing finally to the fourth sentence of the molecule, we see that the type, "Liberals fear standing-up-to-the-Communists" has been constrained by fixing the liberal as "Wilson" and standing-up­ to-the-Communists as "U.s. shows of fprce" or something similar. Now the program instead of generating possibilities, must simply see if any­ thing at all on the belief list fits the speci­ fication. If, say, "Wilson fears u.s. nuclear actions", is present, then the molecule is success­ fully filled. The output can now be constructed by chaining together the four realized sentences of this molecule, the name of the molecule, "No-Win Policy" and suitable pre-keyed interstitial mater­ ial to produce a florid and smarmy account of why Britain does not interfere with the Czech invasion.

If, on the other hand, the belief list con­ tains nothing to suggest that "Wilson fears U.S. shows of force", then this sentence itself is in­ put to the fill-molecule routine. Another mole­ cule is found which will accommodate a sentence of this type: say, the "Liberal Weakness" molecule "Liberals fear standing-up-to-the-Communists", "Liberals oppose armaments", "Armaments promote standing-up-to-the-Communists." If the second and third sentences here can be realized, say with "Wilson opposes British troop commitments" and "British troop commitments aid u.s. shows of force", then "Wilson fears u.s. shows of force''would be considered presumptively true by the pro~am, thus filling the original molecule.

Failure of this recursion leads to an iter­ ation down the earlier list of candidate sentences in an attempt to find another suitable combination. Exhaustion of all candidate lists leads to aban­ donment of one molecule and attempts at filling others, after which the final failure exit occurs.

With a program limitation on deep recursion, this routine usually takes no more than half a minute. Of course this estimate depends on the parameters of the particular memory, and we will need more experience to derive more general estimates.

1. Weizenbaum, J. ELIZA - a computer program for the study of natural language communication be­ tween man and machine. Comm. Assoc. Computing Machinery, 1966, ~· 36-45.
2. Colby, K. M., & Enea, H. Heuristic methods for computer understanding of natural language in context-restrictedon-line dialogues. Math. Bio-sciences, 1967, !· l-25.
3. Kelley, H. H. Attribution theory in social psychology. In D. Levine (Ed.) Nebraska Sym­ posium on Motivation~ 1967. Lincoln, Nebraska: University of Nebras a Press, 1967.
4. Abelson, R. P. Psychological implication. In R. P. Abelson, E. Aronson, w. J. McGuire, T. M. Newcomb, M. J. Rosenberg & P. H. Tannenbaum (Eds.) Theories of cognitive consistency: A sourcebook. Chicago: Rand McNally, 1968.
5. Heider t F. The psychology of interpersonal relations. New York: Wiley, 1958.
6. Abelson, R. P. Computer simulation of 1 hot 1 cognition. Ins. s. Tomkins & s. Massick (Eds.) Computer simulation of personality. New York1 wiley, 1963.
7. Abelson, R. P., & Carroll, J. D. Computer simu­ lation of individual belief systems. American Behavioral Scientist, 1965 1 !· 24-30.
8. Abelson, R. P. Heuristic processes in the human application of verbal structures in new situ­ ations. Proc. XVIII International Congress of Psychology, Symposium 2s. Moscow, 1966.
9. Abelson, R. P., & Kanouse, D. E. Subjective acceptance of verbal generalizations. In s. Feldman (Ed.), Cognitive consistepcy. New York1 Academic Press, 1966.

Table l

Notation for Fill-Molecule Program

Assignment specifications! x 1 , x 2 £ x 1 ; x 3 £ x 1 , x 2 ;

r89 1 r90 £ Rl6i r9l £ Rl7 1 etc.

Belief list 1

Molecule lists M·{(X 1 R 16 x 2 , X 1 R 17 x 15 , X 2 R 20 x 15 ); (X 1 R 16 x 2 , X 1 R 22 x 3 , X 4 R 22 x 2 , X 4 R 16 x 3 ); etc.} Molecule realizations m={x 2 r 89 x 3 , x 2 r 91 x 88 ; ?}

In general, a realization of abstract molecule

is achieved by a subsumed concrete molecule

if xj~ x:l whenever i =i and x:l rj xj £ B or otherwise satisfies PTRUE and x:l rj x:lt B pq pq 123 '+56

or otherwise satisfies PTRUE 1 etc.

Table 2

A Specific Example for the Fill-Molecule Program

Noun Categories: LB1Liberals; WGIWestern governments 0 SHC1Situations Helpful to the Communists 0 SUC:

Elements: LBJ~McCa~thy 1~e~lson.,,, 0 u.s., Britain,,.,, Viet~bombi~ze~.

-~ ~ ~ding up to Communists

Verb Categories! P~ CONicontrolli~~p~r~e~v;e~n~t~i~o~;;;;~~~~~~~::==~==~

Elements: supports, helps 1 not-interferes-with 1 ,,,, 0 controls, influences 1 ,,,,;-fear~a~ids;-not­ wants , ···

Legal sentence t~(LB CON W� 0 ~B PRO SHC;~B~F~E~A~S~U~C~·~W~�~P:R~O::S=H~C~i·~·~·~·~)~::~~~~~~~==~------

Belief list: (LBJ controls u.s.; Wilson controls Britaili.:d McCarthy supports bombing halt, ....

Molecule list: (WG PRO SHC 1 SUC PRE SHC 1 LB CON WG 1 LB FEA SUCi LB FEA SUC 1 LB OPPose ARMaments,

ARM PRO SUC;,,,,)

Molecule realization: (Britain not-interferes-with Czech invasion 0 u.s, shows of force stops Red invasions 0 Wilson controls Britain 0 Wilson fears u.s. nuclear actions)

Here 1 Czech invasion is similar to (an instance of) Red invasions U,S, nuclear actions is similar to (an instance of) U,S, shows of force