Some Basic Mechanisms for Common Sense Reasoning About Stories Environments

Giovanni Adorni, Mauro Di Manzo and Fausto Giunchiglia

Institute of Electrotechnics, University of Genoa

Viale F. Causa 13, 16145 Genoa, Italy

such a reasoning capabilities can be implemented in an "anthropomorphic" robot. The first step to­ wards the integration of what we call "visual know! edge" into a general purpose inferential system consists of building the scene sketched in a story. This generation activity requires a basic capabili­ ty of instantiating and positioning objects, which can seem a trivial proble~ but needs a surprisingly large amount of knowledge, due to the fact that most of the scene is usually not directly described in the story .

In the following we will discuss some basic inferential rules restricted to the case of static scenes and dealing with "local reasoning", that is reasoning based on knowledge which can be extracted from a single conceptualization or a single spatial relationship between two objects . This set of rules is under implementation in a conversational system, called NAUSICA, which uses Italian language as communication mean and a conceptual dependency like formalism for its internal conceptual represe~ tation (Adorni, Ansaldi , Di Manzo and Stringa, 1981) , (Adorni, Boccalatte and Di Manzo , 1982), (Schank , 1975).

II PART- OF INSTANTIATIO~ MECHANISM

The PART-OF relation is used to express a func­ tional connection between objects , as, for instance "leg PART-OF table" , "branch PART- OF tree" , "lock PART-OF door" . The existence of a PART-OF relation does not always imply a physical connection between the involved objects; however such a connection is necessary in order use the object, which is PART- OF , in accordance to its normal functional characteristics , and this gives rise to the expect~ tion of a physical connection , if there are no specific reasons to reject this hypothesis. In the following we present some examples of conditions that have been proved to be useful in order to accept or reject the implicit instantiation . From here on, if "A is PART-OF B", ~will be referred to as the sub- part object and ~ as the whole object.

A. Consistency checks

When the whole is instantiated, the position of the sub- part wi th respect to the whole can be in­ ferred f rom the structural description of the whole

itself, even within approximate constraints. The

ABSTRACT

This work is concerned with the problem of rea­ soning about physical objects and their positions in the space. The main goal is to develop a system which is able to "imagine" the scene in which a story takes place, inferring by means of sui table rules the items which are not explicitly named , in order to verify the relevance of knowledge about the physical environment in the process of stories understanding . The specific topic covered by this work i s the problem of objects instantiation and a set of rules for inferring the existence of an object is proposed.

I INTRODUCTION

The problem of processing visual information can be approached under several points of view and with different goals . Firstly, the concept itself of "visual information" can cover a wide range of information types , from elementary visual patterns to the knowledge about physical objects and their organization in complex scenes , possibly involving the interpretation of actors goals from their movements. Secondly such a knowledge can be handl­ ed, for instance, with the aim of supporting a recognition process or with the goal of verifying the role of imagination in the process of stories understanding. This second goal did not receive in the past the same attention deserved to the first one. Basic works are due to (Boggess, 1979), (Heskovits , 1980) , (Keirsey, 1978), (Lehnert, 1978~ (Sondheimer, 1976), (Waltz, 1980), and to the "gestaltisten" (Kanitza , Legrenzi and Meazzini , 1975) , but up to now the problem has been only scratched .

In this work we are faced with high level visual information, that is the representation and manipu­ lation of knowledge about physical objects , their positions in a given environment , actions involving their use and so on . Our main long term goal is to analyze how the inferential activities required to understand a story are affected by the knowledge about the physical characteristics of the environ­ ment of the story itself, what specific reasoning rules are used to handle physical objects and how position of the whole must be in turn inferred from knowledge about its typical locations in a given (or assumed) environment. Thus, for instance, the sentence "the branch on the roof" allows us to assume an outdoor environment because of the on re­ lation and the roof object , while the branch can be considered PART-OF a tree. A tree , in an outdoor environment, suggests something like a garden (a garden can be inferred from knowledge about the li~ ing environment of a tree), where its typical posi­ tion is vertical and in physical contact with the ground. From the structural description of a tree we obtain a possible position of the branch which is consistent with given that a roof is garden can be around a the relation "on the roof", PART-OF a house, and a house . We will call this kind of consistency check a POSITION check. Other consistency checks are related to object DIMENSIONs and SUPPORTING capabilities.

* Work supported by M.P.I. under Grant 27430/81.

B. Containers

When the sub- part object is in x, where x is any object having the basic function of storing things, the instantiation of t he whole object is often uncertain . Let's consider the sentence "the lock in the store- room" : even if an instantiated door can be put inside a store-room, without fail ­ ing the previously mentioned consistency checks , this is not the scene commonly imagined; it is much more likely that the listener thinks simply of a stored lock.

C. Supports

If we hear that "the engines were roaring on the track", we probably instantiate a number of cars racing or ready to start; on the contrary , it is unlikely that the sentence "the engine is roaring on the bench" suggests a car on the bench. In the second sentence the sub-part object is on an object having the function of giving support; if there are not specific reasons to do the opposite, a direct support is supposed and the whole object

is not instantiated.

III CONTAINED INSTANTIATION MECHANISM

There is a class of objects which usually need to be contained by some specific container . We will refer to objects of this class as "fluid" objects . Liquids are a typical example of fluid objects , but also solid objects made of a large number of sm-all, disconnected parts exibit similar properties, at least from t he point of view of the use of con­ tainers, and therefore they can classified as flu­ ids. When a fluid object is addressed , some specific containers must be implicitly instantiated , if there are no explicit references to t hem . The kind of co~ tainer may depend on the type of fluid or on the assumed environment or both . There are however some constraints to fulfillin order to make reason­

able the implicit instantiation of a container.

A. Consistency checks

Unlike what happens with the PART-OF in­ stantiation mechanism, if the fluid is the subject of a spatial relationship, then the supposed con­ tainer usually inherits such a relationship; therefore consistency can be checked using those interpretation rules which are typical of the in­ volved relation. For instance, tre phrase 'the beer on the table" implies a relation like "a glass on the table", which can be accepted , while in the phrase "the sugar in the lock", the interpretation rules of the in relationship reject the hypothesis of "a sugar- basin in the lock" .

B. Locations

The instantiation of a container is usually re­ jected if its inferred location is very unusual, in our everyday experience . If , for instance, we say " the coffee on the wall", probably the listener will not imagine the coffee inside a coffee-pot hung up on the wall; it is much more likely to sup­ pose that , for some accident, coffee splashed up to the wall .

C. Object characteristics

Some kind of fluid objects usually do not lead to the instantiation of specific containers. They are typically "natural fluids" which exist in outdoor environments in a free state, or the re­ mainder of some physical or manufacturing process ("the sand on the floor"). The assumed environment can play a relevant role . Some fluids, for in­ stance, are much more likely to require a specific container in an indoor than in an outdoor environ­ ment ("the water on the table" against "the water on the street" ).

IV STATE CONDITIONS

Some objects require the pr esence of other ob­ jects, often depending on the environment , in order to mantain a good physical state. Such state condi­ tions are typical of living objects; so, if we say "the fish on the table" , some water must be in­ stantiated in order to create a suitable living environment for the fish . In most cases the implic­ it instantiation is based on the assumption that the living object is in a good physical state; this assumption is acceptable if t here is no contrary evidence. Some environments, however, can lead to rejection of such an assumption , and therefore no implicit instantiation is required : an example can be "the fish in the fishmonger's shop". State cond~ tions can hold also for not living objects. The most common case is related to not- embedding assumE tions; hence, if we say "the plane is hidden in t he mountain" , a cave is instantiated .

V OTHER INSTANTIATION MECHANISMS

There is a set of basic mechanisms which can be used both to directly instantiate objects and to check instantiations made by means of PART-OF or CONTAINED relations. In this section we will briefly discuss three of them, namely the SUB-PART, the SUPPORT and the CONTENT.

A. Sub-part

When the geometric structure of the prototype of a class of objects is described, usually only few elements can be assumed to be always present in every instantiation; a number of optional vari­ ations must be taken into account, which are not instantiated unless there are specific reasons to do it . Generally, an optional sub-part (a drawer for a table, for instance) is instantiated only if it is explicitly mentioned; however, there are cases in which it must be inferred to make a spa­ tial relation reasonable. For example, if we say that "the pencil is in the table", we must infer a sub-part of the table usable as a container, other­ wise we should suppose the pencil embedded in the table; therefore, if an optional drawer is describ­ ed in the table prototype, it will be instantiat­

ed.

B. Support

The SUPPORT instantiation mechanism may be acti­ vated when a supporting action between two objects is inferred; this usually happens when an on re­ lation is asserted or assumed. To accept the sup­ port hypothesis a number of consistency checks must succeed , involving the weight of the supported ob­ ject, material and structure of the supporting one and so on. One of this check is concerned with the capability of the supported object to mantain a physical contact with the supporter; for instance, a fly on the wall has this capability , while a P:i:: ture on the wall has not . In the second case , an intermediate support must be inferred, if possible (a nail in the case of the picture on the wall).

C. Content

Many objects, expecial.ly if they define environ­ ments (living room , garden and so on) may cause expectations for some specific conten t . For example, to say "living room" suggests a particular indoor view with some typical pieces of furniture. Therefore, a sentence as "the plant in the living room" results in the implicit instantiation, for instance, of armchairs, sofa , small tables and eve~ y other object which is commonly in such a room .

VI CONCLUDING REMARKS

We will conclude giving one simple example of a coordinate use of the proposed mechanisms and of the system capabilities at the current stage of implementation.

Let us consider the phrase: "a plant in the living room". The living room, through the CONTENT mecha­ nism, results in the instantiation of a room with its typical accessories . The structure of the room

(dimensions, position of door, windows and so on)

as well as the position of its content is chosen by default; in the same way is fixed the point of view under which the whole scene is generated . The plant, because of STATE condit ions , requires some earth (in a living room a plant is supposed alive); STATE conditions give also the expected relative p~ si tions of the plant and the earth. Earth in. an in­ door environment has fluid characteristics , so it r equires a proper container (a big pot in an indoor environment), whose default location is on the floor . If another spatial relation follows ,as, for instance, "the plant is on the table" , the same assumed environment (a living room) is mantained, since it can contain the object (a table) used as new local reference. The principle of "minimum num­ ber of objects" is followed,avoiding to instantiate another plant, and possibly another table, if one instance~ s uch objects already exists;hence the new sentence simply results in removing one of the inferences done during the analysis of the first one (the pot o~ the floor), positioning again the pot in accordance to the new information.