Integrating Linguistic and Pragmatic Temporal Information in Natural Language Understanding: the case of "When Sentences"

1 Introduction

As Winograd had already noticed in [Winograd, 1983], the research in Natural Language Understanding (NLU) is being carried on within a new paradigm: the computational paradigm. Ils main difference with respect to the previous (generative) one stands in the "attention to process organization" and in the "relevance of non-linguistic knowledge". Recently, for instance, many works (e.g. [Webber, 1988; Moens and Steedman, 1988; Song, 1991; Lascar ides and Asher, 1991]) have underlined the importance of coping with the relations between eventualities in the world (I use eventuality as a cover term for situations of all aktionsart classes) for dealing with the temporal information conveyed by a discourse and for determining the discourse structure.

*This work was (partially) supported by Progetlo Finalizzato Sistemi Informatici e Calcolo Parallelo of CNR, under grant n. 92.0160l.PF69, and by CNR, grant 92.0006l.CT12.

However, no clear and modular account has been proposed yet of how the temporal information provided by the world knowledge (henceforth: "pragmatic" knowledge) has to be combined with "purely linguistic" temporal information (e.g. that provided by tense, aspect and temporal adverbials), since only very simple sentences (e.g. sentences in the simple past) have been generally taken into account when dealing with discourse.

The paper focuses on this problem, considering the interpretation of "when" sentences. "When" sentences are one of the most problematic cases of impact of the world knowledge on the temporal interpretation of the language, as shown by the following example from [Moens and Steedman, 1988; page 15]

• Ex.l "When they built the 39th Street bridge ....
• (a) .... a local architect drew up the plans.
• (b) .... they used the best materials.
• (c) .... they solved most of their traffic problems.

In fact, in Ex.1, the "pragmatic" world knowledge enable the hearer to establish that the eventualities described in (a), (b), (c) precede, are during and follow the eventuality described by the "when" clause respectively. To cope with this "temporal indeterminacy" of "when" sentences, their "causal implicatures" [Heinamaki, 1978] have to be considered. For example, Moens and Steedman [1988] have recently suggested that the "contingency" relations (e.g. causation and part-of relations) provide eventualities with a complex structure (called "nucleus"). Nuclei are composed by a "preparatory state", a "culmination" and a "consequent state", and explain the different temporal interpretation of "when" sentences (for instance, the (a) main clause refers to the preparatory state of "building the 39th Street Bridge", while (c) refers to the "consequent state").

However, also the pieces of temporal information provided by tense, aktionsart, aspect and temporal adverbials of the main and the "when" clause, and by the "linguistic anchoring" of the two clauses (according to rules such as the ones regulating the sequences of sentences) have to be considered. Thus, at least all of the knowledge sources shown in fig.1 have to be coped with in order to determine the temporal interpretation of "when sentences".

2 An integrated and modular approach

The paper illustrates an integrated approach in which all of the information sources in fig.1 find their proper roles. Besides devoting much attention to the "linguistic analysis", the paper also focuses on the problem of representing a KB of causal knowledge and of reasoning with it for extracting the relevant pieces of information.

One of the main goals of this work is that of developing a modular approach, in which the various knowledge sources independenLly provide temporal pieces of information, which are integrated via "standard" constraint propagation techniques [Allen, 1991]. In order to obtain such a goal, the "dishomogenity" of "pragmatic" and "purely linguistic" temporal information has to be faced. In fact, the world knowledge provides "general" relations between types of eventualities, while linguistic tools such as tense and aspect may allow the speaker to convey pieces of temporal information regarding also specific eventualities. For instance, the "pragmatic" knowledge may contain a general causal connection between RUN and KNEE ACHE, and the general temporal constraint between them (e.g. the most general temporal constraint that the beginning of the effect cannot precede the beginning of its cause [Shoham, 1987]; this constraint will be indicated as GTC in the following). On the other hand, the speaker may further specify this general constraint using linguistic tools such as the perfect or the progressive in Ex.2 and Ex.3

Ex.2 "When I had ran, my knee ached" Ex.3 "When I was running, my knee ached".

In other cases (e.g. Ex.1}, no temporal constraint is directly provided by the language, so that only the "pragmatic" constraints determine the final temporal interpretation.

The paper shows that a deep integration of the "pragmatic" and linguistic temporal constraints can be obtained while maintaining a separation of the different knowledge sources. The basic means of reaching such a goal is the distinction between the "Temporal Extent of the Eventuality per se" (TEE) and the "Attentional Temporal Extent" (ATE) of the eventualities described in the language. TEE and ATE are defined as follows:

TEE: Temporal extent of the eventuality described by a clause, not considering the speaker's perspective (i.e. the aspect). The definition is "recursive": it also applies to "composite eventualities" (e.g. in "Jo hicupped for an hour", the TEE is the temporal extent of the whole iterated event)

ATE: temporal extent "focused on" by the clause (i.e. the perspective imposed by the aspect is considered).

For instance, in the "when" clause of Ex.2, the TEE is the temporal extent of John's running action, while the ATE is a temporal extent in the consequent state of the same action.

TEE and ATE are reminescent of Reichenbach's Event Time (E) and Referent Time (R) [Reichenbach, 1947]. However, Reichenbach's R is always a point, while the ATE may be an interval as well. Moreover, TEE and ATE refer to the internal structure of eventualities (called "nucleic structure" in [Moens & Steedman, 1988]), and not only to purely tempoml features, as in [Reichenbach, 1947].

The basic intuition of the approach which is being described is that "pragmatic" and "anchoring" temporal constraints independently relates the TEEs and the ATEs of the "when" clause and the main clause respectively. The "pragmatic" relations only provide temporal constraints between the temporal extents of eventualities "per se" (TEEs) and arc insensitive of the perspectives introduced in the language. On the other hand, linguistic "anchoring" imposes constraints on the relative positions of the focused temporal perspectives (ATEs). Therefore, a modular approach to "when" sentences may be worked out, on the basis of the connections in fig.2.

(a, a') the interpretation of the "when" clause and of the main clause in isolation provides the TEE and the ATE for each clause and the temporal relations between them, depending on tense, aktionsart, aspect and temporal adverbials

(~) the "pragmatic" knowledge provides (if some contingency connection is found between the described eventualities) the temporal constraints between the TEEs of the two clauses

(y) the anchoring of tense, aspect and aktionsart of the two clauses provides the constraints between their A TEs.

Finally, the various temporal constraints are combined via standard qualitative constraint propagation, thus providing the final temporal interpretation.

In the rest of the paper, the various modules involved in the interpretation process are described. Section 3 sketches the representation of causal relations constituting (part of) the "pragmatic" knowledge, and briefly outlines the reasoning process that extracts the relevant pieces of information from the representation (see the ~ connection in fig.2). Section 4 describes the interpretation of the clauses independently of each other, specifying how the TEEs and ATEs of the clauses are determined (see a and a' in fig.2). Section 5 deals with anchoring (see y in fig.2). In this section, a further result is analysed, namely the fact that the complex temporal constraints imposed by relating tense, aspect and aktionsart of the "when" and the "main" clauses can be obtained by imposing the equality of the two A TEs. Section 6 exemplifies the propagation of the temporal constraints, to explain how the integration of linguistic and "pragmatic" constraints is obtained. Finally, in section 7, the approach is compared with others in the literature.

3 Representing and reasoning with causal information

All types of "contingency" relations arc relevant to the temporal interpretation of the discourse. However, we make the initial assumption that the pragmatic domain knowledge only consists of causal connections. Even with this restrictive assumption, the problem of representing and reasoning with the domain knowledge is a major one, since a formalism for representing causal-temporal knowledge and reasoning techniques operating on it have to be devised.

One of the main goal of the causal-temporal formalism we are developing at the University of Torino [Terenziani et al., 1990] is to provide the possibility of representing in an accurate way the temporal constraints imposed by causation. Taking also advantage of BACK [Nebel and VonLuck, 1988], a KL-ONE-like Hybrid Knowledge Representation formalism, we have proposed a classification of different definitions of causation, which depends on the temporal constraints they impose between causes and effects. The root of the causal taxonomy corresponds to the most general constraint (GTC) described above. For example, Rieger's and Grinberg's One-shot and Continuous causation [Rieger and Grinberg, 1977] are classified as specializations of such a general definition: One-shot causation (e.g. c7 in fig.3) restricts the cause to end before the end of the effect, while for Continuous causation (e.g. c6 in fig.3) the persistence of the cause is required in order to sustain the effect. Causal nets are developed and classified (by BACK's classifier) according to such a basic causal taxonomy, so that each causal connection belongs to a given type [Tcrenziani et al., 1990]. For example, in the causal net in fig.3, also NA­ Generation (Not Agentivc Generation) and T-Continuous (Threshold-Continuous) causation arc used. NA-Generation represents causal relations in which the cause and its effect have the same temporal extent (e.g. c2 in fig.3), and T­ Continuous causation represents Continuous causation in which the effect can be obtained only after the reachment of a specified threshold (e.g. cS in fig.3).

The concepts c2 .. c8 represent causal relations. The C (Cause), E (Effect), CO (COntext) and START roles connect a causal relation with its causes, effects, contextual conditions and thresholds (if any) respectively.

Given two eventualities (described by concepts in the causal net 1 ), a specialised causal-temporal reasoner [Terenziani et al., 1990] looks for a path connecting them, and determines the temporal relations between them, by propagating the temporal constraints imposed by the causal relations in the path. Since our definitions of causal relations are very specific from the temporal point of view, precise temporal constraints may be inferred by the causal· temporal reasoner. Consider, for instance, Ex.4

Ex.4 "When the air had risen, there were little droplets"

The reasoner determines the causal path c2 (type: NA· Generation; temporal constaint between cause and effect: (EQUAL)) c4 (NA-Generation; (EQUAL)), cS (T· Continuous; (CONTAINS, FINISHED-BY)), c6 Continuous; (EQUAL, STARTED-BY, FINISHED-BY, CONTAINS)), c7 (One-shot; (BEFORE, MEETS, OVERLAPS, STARTS) between AIR RISE and DROPLETS STATE (notice that hierarchical reasoning has been used for inferring that ADIABATIC COOLING is a subconcept of COOLING). After the propagation of the temporal constraints (expressed as relations in the Point· Interval Algebra [Vilain, 1982]), the ambiguous relation (BEFORE, MEETS, OVERLAPS, CONTAINS, FINISHED-BY) between AIR RISE and DROPLETS STATE is provided in output.

4 Temporal interpretation of clauses

The interpretation process operates in a compositional way, and determines the TEE and the ATE of the described eventuality. The starting point of the analysis is the "basic" aktionsart of the clause (i.e. regardless of its temporal modifiers: tense, aspect, temporal advcrbials). The aktionsart classes we adopt are those in [Moens and Steedman, 1988]: we distinguish among culminations (e.g. "Harry reached the top"; denoting punctual events leading to a new state of the world), processes (e.g. "Harry climbed"; continuous changes of the world), culminated processes (e.g. "Harry climbed to the top"; processes with an intrinsic culmination), points (e.g, "John hiccupped"; instantaneous events, not usually viewed as leading to a relevant change in the world) and states (e.g. "Harry is tall"; properties and states of affairs of the world).

1 The problem of determining the concept in the net corresponding to a linguistic description is very complex, since many forms of "accommodations" [Lewis, 73] are generally involved, and is not addressed here for the sake of brevity.

Depending on the basic aktionsart (which is determined as described in [Grasso et al., 1990]), a (durative or punctual) temporal extent is associated to the eventuality described by the clause, and constitutes its initial TEE. This extent is instantiated into the Temporal Plane (TP), which is the part of the semantic net representing the semantic content of the sentences which factors out the temporal information [Grasso et al., 1990]. The interpretation of temporal modifiers further refines this initial representation.

The interpretation is based on the approach in [Moens and Steedman, 1988]. in which the possible aktionsart changes are modelled via labelled connections among nodes (representing aktionsart classes) in a net, and temporal modifiers interpretation is dealt with by movements on the net. For instance, in

Ex.S "The air had reached the saturation level in 10 minutes"

the "in" adverbial coerces the culmination "the air reached the saturation level" to a culminated process, bringing into focus its preparatory state.

In [Grasso et al., 1990] we have introduced a set of rules which encodes the aktionsart movements depending on the temporal modifiers and which build up the temporal representation of the TEE in the TP. For instance, fig.4 shows our representation of Ex.4 in the TP: P1 represents the initial TEE and I I is the TEE obtained after the coercion triggered by the "in" adverbial.

Further rules have been introduced in order to determine the A TEs of input clauses, depending on their aspect. Roughly speaking, aspect constitutes a linguistic tool for selecting a particular perspective on an eventuality. Basically, the aspectual classes we assume arc those in [Bertinetto, 1986]: we distinguish between perfect, perfective and progressive (for the moment, we do not cope with habitual eventualities).

- perfect: (e.g. "John had ran") the eventuality is seen as indecomposable, from a point of view later than it. Perfect applies to culminations, and gives as result a consequent state. Thus, the ATE of a perfect clause is a (punctual or durative) temporal extent which is part of the consequent state of the eventuality associated to the TEE.

• perfective: (e.g. "John ran") the whole eventuality is focused upon, and it is seen as indecomposable. Perfective does not trigger any movement in the aktionsart net, and the ATE of a perfective clause is equal to the TEE.
• progressive: (e.g. "John was running") the eventuality is seen from an internal point of view. Progressive operates on processes and gives as result a (progressive) state, so that an internal view of the TEE is selected as ATE (in general all states have an internal view).

For instance, the result of the temporal interpretation of Ex.S is reported in fig.42.

fig.4 Temporal representation of the clause ''The air had reached the saturation level in 10 minutes".

Pl is the initial TEE associated to the basic culmination; 11 is the new 10-minutes-long TEE obtained after the interpretation of the "in" adverbial. 12 is the temporal extent brought into focus by the interpretation of the perfect, and the temporal extent between e and e' constitutes the ATE of the sentence.

The meaning of the PRE PARA TORY and of the CONSEQUENT arcs in fig.4 is twofold: they estabilish a temporal constraint between temporal extents, but they also represent the fact that a contingency relation holds between the related eventualities. The temporal constraints are not expressed in the figures as Vilain-like temporal relations [Vilain, 1982]. but, more intuitively, assuming time increasing from left to right. Additional constraints are reported in the figures.

Finally, notice that, at this "linguistic level" of the analysis, only the notions of "preparatory" and "consequent states" are required, whilst the details of the actual causal connections between eventualities may be avoided. In other words, at this level, the "nucleic structure" [Moens and Steedman, 1988] constitutes a suitable abstraction of the "pragmatic" world knowledge.

5 Anchoring

The determination of the "purely linguistic" temporal constraints among the two clauses of a "when" sentence is a notoriously difficult problem, since the tenses, the aktionsarts (durative or not, stative or not) and the aspects of the two clauses have to be considered. For instance, some feeling of the complexity and the variety of the constraints can be gained from Table.!, which shows the relative positions of the temporal extents of the eventualities described by "when" clauses (upper part of each rectangle) and by main clauses (lower part) in some interesting cases (without considering the effects of the relative ordering of the "when" and the main clause).

Complex rules have been devised in order to cope with this phenomenon (e.g. [Heinamaki, 1978]). On the other hand, the definition of ATE proposed in the paper allows one to Table.l Various "purely linguistic" temporal constraints between the "when" and the main clauses of "when" sentences. Question marks represent the cases of empty "purely linguistic" temporal constraint. In those cases, the only constraints are provided by the "pragmatic" knowledge (e.g. Ex.l). Dolled intervals represent "possible" temporal extensions.

2In case of perfect or progressive, some further aktionsart coercions may be required [Moens and Steedman, 1988]. Ambiguities may arise during the interpretation process, concerning the scope of the temporal modifiers and the determination of the aspect of a clause. Also inceptive readings (e.g. "At 8.15 the water boiled"; meaning: "started to boil") introduce funher ambiguities.

determine such constraints in a simple and uniform way. In fact, a detailed case analysis (which includes the cases in Table.l) has shown that the "purely linguistic" temporal constaints (when they arc present) can be obtained by simply imposing the equality of the ATEs of the two related clauses.

The resulting anchoring algorithm is straightforward:

IF (PERFECTIVE(When-clause) AND PERFECTIVE (Main-clause)) OR (PERFECT(When-clause) AND PERFECT(Main-clause))

THEN

ELSE

Nil

ATEM=ATEw

For example, let us come back to Ex.4. The interpretation of the "when" clause involves the introduction in the TP of the consequent state of "air rise" in order to interpret the perfect (sec the higher part of fig.6). The interpretation of the main clause is straightforward: since the aspect of the clause is perfect and its aktionsart is state, an internal (progressive) view is selected (see the lower part of fig.6). Anchoring imposes the equality of the two A TEs (see fig.6), thus determining the desired "purely linguistic" temporal constraint (consider the rectangle (8) in Table. I, and notice that states roughly behave like progressives with respect to anchoring). In this case, the "purely linguistic" temporal constraints are very weak. In particular, there is no constraint on which one of the two TEEs starts earlier (in fig.6, a part of 13 has been dotted to represent this fact).

An analogous solution to the anchoring problem has been proposed by Hornstein [1977], who imposes the equality between the Reichenbachian Reference Times of the main and of the "when" clause. However, Hornstein manipulates the SRE structures (i.e. the triples of Speech Time (S), Reference Time (R) and Event Time(E)) in a purely syntactic way, so that the relative ordering of the Es which is obtained after the manipulation (representing the anchoring of two clauses) has to be further interpreted. Moreover, in Hornstein's interpretation, Es are points, so that he does not cope with some of the constraints in Table. I .

6 Integrating the temporal constraints

The modules in sections 4, 5 and 3 independently provide respectively: (i) the TEEs and A TEs of each clause (see a and a.' in fig.2); (ii) the temporal relations between A TEM and A TEw (see y in fig.2); (iii) the temporal relations between TEEM and TEEw (see P in fig.2).

At the end of the analysis, the TP contains a set of temporal entities (time points and/or time intervals), temporal relations (i.e. relations in Vilain point-interval algebra [Vilain, 1982]) and contingency relations (e.g. PREPARATORY, CONSEQUENT) among them. If only the purely temporal meaning of the PREPARATORY and CONSEQUENT connections is considered, "classical" temporal constraint propagation techniques [Allen, 1991] can be applied in order to. integrate the constraints provided by the different part of the analysis.

For example, in Ex.4 the linguistic temporal constraints between the TEEs and ATEs are TEEw (BEFORE) A TEw; TEEM (CONTAINS) A TEM ; A TEw (EQUAL) ATEM (these are the representation of the temporal constraints graphically shown in fig.6 into Vilain's temporal relations), and the "pragmatic" constraint (called cl in the following) is TEEw (BEFORE, MEETS, OVERLAPS, CONTAINS, FINISHED-BY) TEEM (see section 3).

The propagation of the linguistic temporal costraints gives as results, among the others, the relation TEEw(BEFORE, MEETS , OVERLAPS, STARTS, DURING)TEEM which constitutes the "linguistic" temporal constraint between TEEw and TEEM. Constraint propagation operates the intersection of this constraint with c1, thus determining TEEw (BEFORE, MEETS, OVERLAPS) TEEM as final temporal relations between the two TEEs. Thus, both "pmgmatic" and linguistic knowledge contribute to the final temporal relations between the TEEs of Ex.4: the (linguistically acceptable) possibility that the TEEM starts before TEEw is ruled out by the causal knowledge, whilst linguistic anchoring disables the (pragmatically acceptable) possibility that the TEEM is contained into TEEw.

The case of Ex.4 is particularly complex, since constraint propagation refines both the linguistic and the "pragmatic" temporal constraints between the related TEEs, and the resulting relation is nevertheless ambiguous. On the other hand, in Ex.l (where no linguistic constraint is imposed between the two TEEs) the "pragmatic" constraints are more specific, while in Ex.2 and Ex.3 linguistic constraints are the most specific ones. In all cases, anyway, the propagation determines the integration of the constraints provided by the different knowledge sources.

A particular case is shown in Ex.6

Ex.6 "When the air rose, it had (already) expanded"

In Ex.6, the linguistic temporal information (imposing that rise follows expansion) contrasts with the "pragmatic" knowledge (stating the co-temporality of the two processes: see c2 in fig.3). However, since the main clause describes a consequent state, the causal connection between the described eventualities is not compulsory and may be neglected [Moens and Steedman, 1988]. Thus only the linguistic constraints contribute to the final interpretation.

7 Conclusions and comparisons

The paper describes a modular and integrated approach to the temporal interpretation of "when" sentences. Whilst maintaining a clear separation between linguistic and "pragmatic" knowledge sources, the approach combines the temporal constraints provided by both. With respect to the approaches to linguistic temporal interpretation (of "when" sentences or of the discourse) in the literature, the main advantages of the described approach seems to be:

(i) an accurate account of (causal) "pragmatic" knowledge. For instance, in [Moens and Steedman, 1988; Webber, 1988] the interpretation process only copes with abstract nucleic structures, and the problem of representing and reasoning with actual domain knowledge is not addressed. As a consequence, the problem of extracting precise temporal pieces of information from the world knowledge is not focused on. An explicit representation of causal knowledge has been proposed in [Lascarides and Asher, 1991]. However, they do not deal with the precise temporal constraints forced by the causal connections, since do not distinguish between different types of causal relations.

(ii) an accurate treatment of the integration of "purely linguistic" and "pragmatic" pieces of temporal information. In [Lascarides and Asher, 1991], for instance, only sentences in the perfective aspect have been explicitly discussed. Moreover, in [Lascarides & Asher, 1991] causal laws are "a mixture of world knowledge and linguistic knowledge" , and a nonmonotonic logic is used for choosing in case of contrasting pieces of information. On the contrary, in the described approach, the linguistic and the "pragmatic" knowledge modules are kept separate, and (as in [Song, 1991]) constraint propagation is used for integrating temporal information derived from the different modules.

Although the described approach copes only with "when" sentences, the problems addressed in the paper are basic in the temporal interpretation of the discourse [Webber, 1988; Lascarides and Asher, 1991]. The main difference is that, in the discourse, the temporal referent for a clause is not provided simply by the "when" clause, but has to be found in the discourse structure. So, we believe that the described work constitutes a first significant step towards the development of an integrated approach which combines linguistic information and world knowledge in the temporal interpretation of the discourse.

Acknowledgments

Most of the work reported in the paper has been developed during a stay at the Department of Computer and Information Science of the University of Pennsylvania. I am especially indebted to Prof. Bonnie Webber of that University for her constant cooperation and her valuable advice. I would like also to thank Prof. Leonardo Lesmo and Prof. Pietro Torasso, who greatly contributed to this work, and Barbara Di Eugenio and Michael White for many useful comments on an early version of the paper.