Proceedings

Focus Constraints on Language Generation

Download BibTeX Cite

Proceedings of the Eighth International Joint Conference on Artificial Intelligence. Natural Language. Pages 582-586.

© 1983 International Joint Conferences on Artificial Intelligence

Abstract

Computer generation of natur al langua ge requir es th e ability to mak e r e ason e d c ho ice s from a larg e number o f possible things t o say as wel l as from a large number of exp r essive possibilities. This paper exa mi n es in d e t a il how one influ e n ce on a generated text, focus of a ttenti on , can b e us e d t o co n s train th e many possibilities that a ge neration system must consider . A computational treatm e nt of focu s of a tt e ntion i s pr e s e nt ed that ca n b e us e d to co nstr a in wh a t th e sys t e m n eeds to co n side r wh e n deciding what to say ne x t. In thi s process, information is produced th at provides constraints on whi c h words a nd syntactic structures best e xpr ess th e system's intent, thus ensuring that its resultin g t ex t is co h e r e nt . This analysis h as been us e d in the fully implemented TEX T sys t e m whi c h generates p a ragr aph length r esp onse s to questions about database structure.

Full text extracted automatically from the PDF; figures, tables and equations may differ. The PDF is the version of record.

1 . 0 INTRODUCTION

Computer generation of natural language r equir es th e ability to mak e r ea soned c hoices from a lar ge number o f possibilities and f rom a variety of knowledge sources. A system that communicates with it s users must be able to decid e what information to communicate, when to say what, and whi c h words and syntactic stru c tures among many possibiliti es bes t ex press its intent. Previous pap e r s ( e . g., [MCKEOWN 80], [MCKEOWN 82A], [MCKEOWN 828]) illustrate how th e fi nal t ex t is influ e nc e d by a variety of factor s , includ i ng commonly used rhetorical strategies, semantic information, focus of attention, and th e discour se goa l. Th is paper e xamines in d e tail how one of those sources, focus of attention, can be used to constrain th e

*-rhis wor~-;;;--p; rtially supported by National Scienc e Foundation grant #MCSB!-07290, awarded to the Computer and Informati o n Sci e nc e De pt . of the University of Pennsylvania .

many possibiliti e s that a generati on system must consider. A computational treatment of focus of attention is pres e nt e d that co nstrains wh a t informati on to communicate a nd its order, and in t he process produc es informat i o n that constrai ns whi c h words and syntacti c structur es best ex press its intent. Exampl es are given of how this analysis has bee n used in th e fully impleme nt ed TEXT sys tem, which generates paragraph length r es pons es to ques tions about database structure.

Focus constraints are only part of the TEXT system's me chanism for respond ing to a ques tion. TEXT also us es dis course plans and a mec hanism for de termini ng r e levancy [MCKEOWN 82A]. To a nswe r a question, TEXT first ci r c umsc ribes a subset of the knowledge base co nt a ining information r e levant to the given question. A discourse plan (call ed a schema [MCKEOWN 82A]) is then u~ ed t o guide the construction of an answer . The focusing mechanism aids in this process by constraining the selection of information to talk about nex t to that whi c h ti es in most appropriately with th e pr evious discour se . Thu s , focus information doesn't primarily d e termine th e content of the r es ponse, but provides c o nstraint s on th e many possibilities that must be considered and aids in shaping a coherent respons e . TEXT was implement e d using an ONR da tabase containing information about military vehicles and weapons . Exampl es are taken from this domain.

In th e following sections, va ri o u s choices that a generation s y stem must make at diff e rent phas e s of processing are first desc r ibed . How focus informati on can be us e d to influence these deci s i ons is then discussed.

2 . 0 CHOICES

One of the first steps in speaking or writing is the narrowing of attention t o knowledge relevant to the purpose at hand. Speakers and writers are c apable of ignoring information in their large body o f knowledge about the world which is not specific to the current discourse purpose. This process , called _£_],_~~~_!_ X~~ust':l_~ [GROSZ 77] is mod e l ed in TEXT by restricting the information that needs to be considered when constructing an a nswer to a subset of the knowledge base which c ontains information that could potentially be included as part of the a nswer. Although this process is not discussed further here, the fact that it does occur is critical for the success of later processes (see [MCKEOWN 82A] for further details).

Once a system has determined what information is likely to be relevant to its c urrent discourse goal , it also must be able to determine what to say first, what next, and how to close the discourse. Order of information can be crucia l to a r e ad e r's understanding of a text. Textual sequence alone can cause a reader to draw infer e nces abou t the relation between two p ropositions, including temporal sequence , c ausality, and exemplification, among o thers. While textual sequence need not always correspond to, for example, temporal sequence, the absence of textual c onnectives specifying otherwise (e.g., "when", " afte r", "whil e ") may indicate that it does. It is important, therefore, that careful attention be given to how propositions are ordered.

must At the surface level, a be able to make reasoned generator decisions about the best lexical items to use, when to use pronimal reference, and about the syntactic construction that should be used . Examples illustrating th ese choices are shown in l-3 below.

  1. Lexica l choice (bought vs. sold)
  2. Jane bought $3.00 worth of bobby socks from Michael.
  3. Michael sold $3.00 worth of bobby socks to Jane.
  4. Pronominal reference (Linda vs. she) A) Linda flew to Washington.
  5. She flew to Washington.
  6. Syntactic structure (active vs. passive)
  7. John gave the book to Mary.
  8. Mary was given the book by John.

Textual order and surface choice are both influenced by a speaker s focus of attention. In the next two sections this influence is characterized in such a wa y that it can ·be used by a language generator to resolve decisions in textual o rder and surface choic e .

3.0 IMMEDIATE FOCUS AND GENERATION

On producing a singl e utterance (controlled by a schema), TEXT narrows its focus of attention to a single object (or set of objects) in its pool of relevant information . Having made a decision about what to talk about first, it must support that decision in succeeding utterances if it wants its text to be easily understood. That is , having decided to focus on a particular object(s), its utterances constrain the set of possibilities for what can be said next if the system is to avoid jumping around from one topic to another . These are termed imme diate focus c on s t r a i n t s s i n c e they apply--l o -~allY between utterances.

TEXT uses constraints d e ve loped by Sidner [SIDNER 79] on how focus of attention can shift or be maintained from one sentence to the next . Sidner showed that a speaker can either maintain his/her current focus, shift to focus on an item just introduced , return to a previous focus, or focus on an item implicitly related to the current focus. The TEXT system uses these constraints to limit the number of possibilities it must consider when deciding what to say next. lf its discourse plan allows for s e veral next utterances, the system only considers propositions that have an element that can be focused in one of these ways.

While Sidner's constraints are sufficient for interpreting natural language, for generation a speaker may have to decide which of the constraints is better than any other at any point. An o rd ering on Sidner's constraints was developed for generation which dictates which of these is preferable (see Figure l below). The preference ordering suggests that a speaker should shift to focus on an item just introduced into conversation if s/he has something to say about it. If the speaker chooses not to do so, that item will have to be re-introduced into conversation at a later point before the additional information can be conveyed . If, on the other hand, the speaker does shift to the item just mentioned, there will be no trouble in continuing with the old conversation . In that case, the speaker is returning to a topic of previous discussion, a legal focus move.

Several consecutive moves to items just introduced are not a problem. In fact, consecu tiv e focus shifts over a sequence of sentences occurs frequently in written text. If this rule were applied indefinitely though, i t would result in nev e r- e nding side-tracking onto diff e r en t topics of co nversation . Howe ver, the model o f generation assumes th a t info rmation is being presented in order to ac h ieve a particular goal (e.g., answer a question). Only a limited amount of information is within the speaker's scope of a tt e nt io n because of its relevance to that goal (as defined by global focus). Hence only a limit e d amo unt of si d e -tracking ca n occur.

Th e s ec ond pr efe rence Indicates that a s peake r should continue talking about t he same thing rather than returning to a n ear lier t opic of conversation wher e possible . By returning to a pr e vious discussion, a speaker closes th e current topic. Th e re fo re, having introduced a topic (whi c h may entail the introduction of ot he r topic s) , one should say all that needs to be said before returning to a n ea rli e r t op ic. The second preference g uar a nt ees that a speaker will avoid implying tha t s/ he is finished talking about the current subject when in fa ct there is more to be sa id. If neither of the first two preferences apply then th e speake r must r e turn to a n ea rlier topic of discussion (preference 3).

In cases wher e a speaker must choose between two propositions with the same foc u s , the preferences desc ribed so f a r proscribe no course of action. Ra th e r than making an a rbitrary c hoi ce , a speaker tends t o group together in discourse those properties that are in some way r ela ted to eac h other. When the system has a c hoi ce between two pr oposi tions with the same focus, it chooses that proposition with the most mentions to pr ev iously me nt io ned items (pref e ren ce 4) .

Thi s ordering do es n't dictat e a bsolut e constraints on the system. Just ;5-;- sr;aker may choose to s uddenly swit c h topics, th e system may chooose to do so a lso . The ordered focus constraints are preferences which indicate the system's be st move wh e n faced with a c hoice. If the system's discourse plan indicates that no nex t choice meets these constraints, it will follow its plan making not e of the abrupt switch in fo c us. This switch can then be syntacticaly marked to ease the transition for the user.

  1. shift focus to item me ntioned in previous proposition
  2. return to topic of previous
  3. 4 . select proposition with greatest number of implicit links to previous proposition
2. maintain focus discussion FIGURE

4.0 CHOOSING SURFACE EXPRESSIONS

There are many diffe r ent wa ys in which a proposition can be expressed in English. If the system makes an arbitrary decision about which to select in a g iven situation, an i nappropriate decision co uld easily be made . For exampl e , if th e propositions shown in 1- 3 above are to be expressed as parts of discourse sequences , then one of the choices in each pair i s clearly inappropriate (4-6 below) .

  1. Jane was in a hurry to finish h e r shopping . It was a chore she particularly despised. First, { Jane bought $3 . 00 worth of bobby socks from Michael . *Michael sold $3.00 worth of bobby socks to Jane.
  2. 5 . We knew that Mary took the train t o New York with Linda, but didn't realize that { Linda flew to Washington from there . *she flew to Washington from th e re.
  3. 6 . John bought that great new book o n data s t ructures . He read the first three chapters and then { he gave the book to Mary . *Mary was given the book by John.

In these discourse sequences, th e inappropriateness of the starred choice in each case can be explained by the speaker's focus over the discourse. A number of linguists (e . g. , [HALLIDAY 6 7], [FIRBAS 66]) have discussed how themati c (or focus) information can affect the ordering of sentence constituents, suggesting that new (or unfocused) information usually occurs towards the e nd of a sentence . In order to place thi s information in its proper position in th e sentence, structures other than the unmarked active sentence may be required (for example, the passive). Structures such as it-extraposition, there-insertion, topicalization , and left-dislocation can be used to introduce new information into discourse . Pronominalization is another linguistic device associated with focused iflformation [SIDNER 79]; it is often used to signal the speaker's focus of attention.

In the example discourse sequences, focus accounts for the choices made in the following ways. In sequence (4), the focus is on Jane who is doing the action. The verb "bought " is selected over "sell" in order to allow Jane to be described as the focused participant in the action. In (5), Mary is the focused element of the factive "to know". If the second reference to Linda is pronominalized, we are likely to interpret the referent of the pronoun as the focused element, or Mary . In (6) the first sentence focuses on John . The active sentence is more approptiate since it places the focused information in surface subject position.

5.0 SURFACE CHOICE IN THE TEXT SYSTEM

Since focus information has been used to constrain the selection of propositions in the TEXT system, a record containing each proposition's focus and its potential focus list is availa ble for th e system to use when determining the specific syntactic structures that should be used in the answe r. It can examine this information to determine how a proposition is related to pre~ious discourse and therefore which of various surface-level choices should be made.

In TEXT, focus information is used in some limited situations to test whether pronominalization can be used. Part of the answer to the question "What is a ship?" where pronominalization was selected is shown in (7) below. In the first sentence of the answer, the ship is being focused on and reference to it in the following sentence can therefore be pronominalized.

7) (definition SHIP)

What is a ship?

A ship is a water-going vehicle that travels~the surface. Its surface-going capabilities a re provided by the DB attributes DISPLACEMENT and DRAFT ···

Focus information is also used to discriminate between use of the passive and active construction . The passive can be used to place the focused constituent if\ surface subject position when the logical subject of the sentence is not in focus. In TEXT, this construction may be necessary in describing binary relations in the ONR database. In the ONR database, weapons are associated through the relation "carry" with different vehicles . When answering a question about missiles, a weapon, the passive construction is used since the missile is in focus (see (8) below). When answering a question about the ECHO II, a type of submarine, the active construction is used in order to attribute information to the "ECHO II" (see (9) below)

8) (difference MISSILE TORPEDO)

What is the difference between a missile and a torpedo?

The missile's target location is indicated by the DB attribute DESCRIPTION and its flight capabilities are provided by the DB attribute ALTITUDE. Other DB attributes of the missile include PROBABILITY OF KILL, SPEED, ALTITUDE, LETHAL RADIUS & UNITS and TIME TO TARGET & UNITS. Missiles are - --- - £arried ~~ water-going vehicles and aircraft ···

9) (information ECHO-II-SUBMARINE)

What do you know about Echo II's?

There are no sub-classes of echo II in the ONR database. Echo lis carry 16 torpedoes, between 16 and-99-missiTes--and 0 _[UflS · -:-:-: ------ - -- -

The use of there-insertion by the TEXT system is shown below in (10) in a portion of the answer generated to the question "What is a guided projectile?". Use of there-insertion in this situation is one way to introduce the set of sub-classes of the guided projectile as focus into the discourse.

10) (definition GUIDED)

; What is a guided projectile?

There are 2 types of guided projectiles in the---GNR database: torp;d0es-3nd miSsile~ The missile has a tar get location in the air or on the earth's surface. The torpedo has an underwater target location ··.

6.0 FO~Q~ CO~TRA~~T IMPLEMENTATION

TEXT co nstr uc ts an answe r to a g i ve n question by r epea t e dly selecting* propositions to i ncl ud e in the a nswer , each o f which is even tu a lly translat e d to a sen t ence . A proposi ti o n consists o f a predicate a nd its arguments which a r e r ea liz ed in th e final Engl ish version as the ve rb of t he sen t e nce and its case r oles . TEXT uses its schema , o r discourse pl a n, as a guide in de termining what to i n clu d e next a nd is cons traine d i n its choice in th e two ways dis cussed so far: 1) it i s co nst r ai ne d in what it considers for inclus ion by glo bal focus and 2 ) the text co nstru cted so far constrains what it can say next. As a p r oposi t io n is adde d to t he answer, a focus record is filled out a nd add e d a l ong with it. The focus record associa t ed with eac h proposition includ es the foc us ed a r g ument and the potential focus list (other a r guments which are cand idates for a s hift in focus). This r eco rd is used bo th t o constrain wh a t proposition can fol low as wel l as for the basis of surface c hoice .

Immedia t e fo c u s co nstra i nts (the preference ordering shown in Sect ion 3 . 0 ) de termin e h ow th e focus r eco rd is fill e d out . If a ny possib l e nex t proposition ( of those th a t have been selec t ed by the schema) has an argument that can be focused such that the first preference is met (i . e ., th e a r g umen t was a memb e r of the pr evious proposi ti on's potential focus list), that proposition is selected and its a r gument is r ecorded as the cur rent focus of th e proposition. All o ther a r guments are incl ud ed in the p r o posit i on's potential focus list as th ey a r e candidates fo r a sh i f t in foc u s . If t he first preference cannot be me t, the same procedu r e is repeated for eac h of th e rema ining prefere nces until a pro pos iti o n is selected.

Surfac e c ho i ce is made on th e basis of the foc u s re cord associated with the p r opositio n. lt is us e d t o select the sente nce voice (active, passive, o r there-insertion) an d t o d e t e rmin e wh e th e r p r o nomi nalization can be us ed . On selec ti o n of a verb for the sentence, sentence voice is a l so selected . Wh e n a verb is selected t o translate a pr edica t e , the pr edica t e's arg ume nt s a r e mapped onto the case roles of th e verb (e.g., protagonist, goa l). If t h e protagonist is the focused argument, the active voice is selected; if the goal is focus e d, the passive voice is se l ec ted. The selection of voic e th e re-ins e rtion is slightly mo r e complicated as it is based on th e introduction of a set whose me mb e r s a r e focused in succeeding utt e r ances (se e [MCKEOWN 82A] for more details).

* Act ual ly by const ructing th e proposition since pro posi tion s don't exis t as a whole in th e knowledge base .

As a n example, consider th e p ropostion, consisting of a predicate, its arguments, and its focus r eco rd, shown in (llA) be low. (liB) shows the proposition at an intermediate stage of generation . Th e ve rb "to carry" has be en selec t ed t o translat e th e analogy-r e lation ON and th e a r guments of the re la tion have bee n mapped o nto th e case role s of th e verb: th e ca rrier has be e n mapp e d to the protagonist and th e weapon to the goal. Since the goa l is in focus, the passive voice is selected and th e final sentence s h own in (llC) is generated .

  • \LA) predicate= analogy -r e l a tion r e lation = ON carrier (AIR~VEHI C LE WATER-VEHICLE) weapon MISSILE current focus = MISSILE po t e ntial focus l is t = ((AIR-VEHICLE WATER-VEH ICLE) ON analogy-relation)
  • verb === ca rry
  • prot ago nist = [ co nj === a nd h ead ! === ai r c r a ft he a d2 === wa t e r-going goal === missil e voice = passive vehicle]
  • Missil e s are ca rried by water-goi ng ve hicle s an d a ir c r aft .

Pronominalization is determined on selecting a l ex i c al item to tr a nsl ate a predicate argument . When choosing pronominalization, the focus r eco rd of the last proposition is checked . If the a r gument was in focus, pronominalizati on is selected in plac e of the full ref e r e n ce for the argume nt.

7 . 0 LIMITATIONS AND UNIMPLEMENTED EFFECTS

The current formul a tion and implementation clearly show h ow focus information can be successfully use d as the basis for su r face choice . Further improvements can be made, howeve r, by encoding the tests for surface choice as part of the grammar. The grammar used in TEXT is based on Kay's functional g r a mmar formalism [KAY 79] and in fact a llows fo r the explicit encoding of focus information. Enough information available at that time to make the for an entire category (e.g., verb or phrase) instead of a lexical item, allowing for more generality test .s noun thus in determination of choice.

The implementation can also be e xtended by including tests for additional types of surface choice. The influence of focus information on lexical choice, noted earlier, is not currently implemented and would be one place to start. The use of o ther surface-level structures can also be signaled through focus information. Some of these include parallel sentence structure, subordinate sentence structure, ~nd textual connectives. Parallel sentence structure can be used to increase the cohesiveness of text when focus remains the same from one sentence to the next. When focus shifts to an item just introduced into conversation, subordinate sentence structure can be used to combine the two adjacent propositions into a single complex sentence. When there has been an abrupt shift in focus, textual connectives can be used to ease the transition for the hearer. The implementation of these uses of focus information for surface-level choices remains a topic for future work.

8 .0 CONCLUSIONS

The process of generating natural language has been shown to involve a system of choices across a wide spectrum o f knowledge sources. A method has been presented that provides a theoretical basis which constrains generation decisions . Furthermore, it illustrates how information arising from decisions about what to say can be used to constrain choices in the surface level expression. While these choices can be arbitrarily determined, an inappropriate decision could easily be made. As systems become more sophisticated, it is imperative that they produce appropriate utterances in order that they communicate effectively

with their users.

ACKNOWLEDGMENTS

I would like to thank Aravind K. Joshi, Bonnie and Kathy McCoy suggestions paper. on Webber, Michael Lebowitz, for their comments and various drafts of this

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards