Situated Actions and Cognition

1 Introduction

Discussions about robotics/artificial life as well as about (human) perception/mind show the prominence of the concept of cognition. In general, the process of cognition relies on many - in a certain sense autonomous - percep­ tual mechanisms. Researchers in the fields of robotics and artificial life tend to adopt a point of view about per­ ception and "mental activities" which stresses the action­ driving aspect. Such a viewpoint conflicts with what is called symbolic AI. Symbolic AI is concerned with "high-level" cognition which is usually equated with (formal) symbol-structure manipulation. This outlook ignores any necessary connection between perception and action. Our purpose is to provide a more action­ driven perspective on cognition and intentionality. We take our starting point in the field of robotics as it en­ forces a rather pragmatic approach to the action-driving aspect of cognitive creatures. One has to take the design stance, and is not so much bothered by questions about the truth/validity of cognitive hypotheses.

In the seventies, robotics passed through a flourishing period while designing technically sophisticated robots for quite pre-structured environments. However, transfer­ ring the robots into more 'real life' environments led to an incommensurable series of problems. The end of the eighties shows an offshoot in the robotics community, which is particularly marked by Brooks paper of 1986. Experimentalists acknowledged the limitations when ro­ bots have to deal with realistic environments, and con­ centrated on implementing low-level reactive systems. We denote this approach by Sensory-based Robotics Peter J. Braspenning University of Limburg Jacobstraat 6 6211 LB Maastricht The Netherlands

(Design) after [Lyons et al., 1989], or generally as "behaviour oriented Af' [Steels, 1994] . The new ap­ proach includes motor-schema's [Arkin, 1990], a notion originating from psychology. It refers to a pattern of "knowing-how" (to execute an action), and incorporates a (mental) motor-image [Decety, 1996] as well. In ro­ botics, a motor-schema comprises certain activation pat­ terns of the effectors and is triggered by sensor data. Moreover, the schema's performs rather autonomously: when triggered they invariably result into an action.

Sensor data arrive as a flow of values. Human beings (and robots designers as well) seem to have inherited perception mechanisms which emphasise patterns and disguise many irregularities. So, artificial perception is rather non-intuitive for human observers. They are easily led to misinterpretations, when they discuss other types of cognitive creatures. Hence, it makes sense to explic­ itly distinguish between frames of reference, for in­ stance, that of the designer and that of his robot. We in­ troduce the notion of cognitive basis, which helps to dis­ tinguish frames of reference in a discussion of cognition and intentionality. Moreover, using this notion, one can compare how different (types of) creatures are "carving up the world" through their sensors.

Present day technology challenges many qualitative theories of cognition by providing working mechanisms, robots, computers etc .. Indeed, one is led to the conclu­ sion that action and acting are among the concepts deeply related to cognition.

In section 2, the notion of cognitive basis is intro­ duced, and illustrated by means of an ultra sonic sensor. In section 3, we discuss perception problems, i.e., the interpretation of the sensor data. We focus on actions as an element of cognition in section 4. We connect the no­ tion of intentionality with performing a motor schema. Finally, we attempt to define more formally what we like to call a motor-schema (or motor-image) basis. There­ sult and novelty of our treatment is that (potential) ac­ tions are included into the notion of cognition.

2 Cognitive basis

2.1 Robot design

A robot consists of effectors or actuators, sensors and - what we metaphorically call - the wiring between them. The actuators are the foundation of any robotic design, because they determine (and constrain) what the robot might do, i.e. they circumscribe its 'raison d'etre'.

Our discussion stays mostly at the conceptual (logical) level so that we really address a logical actuator, which is any combination of (mechanical) hardware and soft­ ware that can transform the state of the robot and its en­ vironment. Similarly, for sensors we introduce logical sensors: any combination of hardware and software which measures the state of the system and the environ­ ment [Henderson, 1990]. The actuators and the sensors are connected through a wiring consisting of the logical (or influential) connections between them. At an ele­ mentary level a simple wiring might do. The more so­ phisticated the aims of the designer, the more complex these "wirings" might be so that one may even speak of an "intelligent" wiring. Intelligent, not because the wir­ ing itself is that smart, but because the designer looked for "smart" connections. Our analysis addresses the re­ alisation of these connections.

A motor-schema involves sensors, wiring and actua­ tors and is a building block for (the design of) a robot. It consists of rather simple perceptual and motor activities. Each motor-schema has its own dedicated sensors. A logical sensor suits a motor-schema if it enables dis­ crimination of the situations for which the schema has to become active. Complex actions may be orchestrated with the help of several motor-schema's. A complex situation is thus decomposed in line with the specific observations allowed by different sensors.

2.2 Definition

We introduce the notion of "sensor basis" analogous to the notion of "knowledge basis" in [Rauszer, 1992]. For convenience in explaining the definition we use the term "knowledge", at the end of this section we redefine the term. Suppose an agent r, has at its disposal a set of logi­ cal sensors which measure certain attributes <XE A,. The sensors can assign an attribute a with a value ve V" to an object o in the universe of discourse U.

An attribute need not discriminate between any two ob­ jects, which is expressed by a(x) = a(y). Rauszer defines an indiscernibility relation for agent r, as :

Ind(A,) is an equivalence relation and the set of equiva­ lence classes is a partitioning of the universe U. Thus, if two objects fall within the same equivalence class [ o lAr. it means that the agent cannot distinguish one from an­ other. The set of all equivalence classes :

is called the knowledge basis of an agent over the uni­ verse of discourse U, determined by its set of attributes A, as handled by its set of logical sensors. The knowl­ edge basis indicates which objects can actually be dis­ tinguished by the agent. Hence, it provides the base for any knowledge the agent can have about its environment. We discuss some further notions illustrating the useabil­ ity:

The positive knowledge of r about set X consists of the set of equivalence classes generated by the elements of X:

X is positively known by r, if the positive knowledge I'(X) = X. This means that r can (positively) distinguish each element of X from all other elements of X.

The negative knowledge I'(X-) consists of the objects in x· (X's complement) of which r can determine that they are not in X. The knowledge of r about X, K'(X) = I'(X) u I'(X); the knowledge of r about X is complete knowledge if K' (X) = U;

The idea of positive knowledge is the most relevant for our purposes. We (as designers) can describe the en­ vironment of a robot as consisting of a set X of objects, but the positive knowledge of a robot indicates which of these objects it can (possibly) observe. One has to be careful in concluding that the environment is positively known by the robot: sensors do not always discriminate objects as we would like them to do, a topic which is discussed in the next section.

In a robot design different components are brought to­ gether. Above, we distinguished sensors from the wiring; consequently the above definitions apply twice. That is, call S the set of attributes for which the logical sensors produce values, then ind(S) is the sensor basis Esens of the agent (an example is given below). We retain the title knowledge basis while applying the definition to the wiring. Calling C the set of logical connections or "in­ formation processing functions" implemented in the wiring, then ind(C) is the knowledge basis Eknow· Any robot comprises both bases.

.. ~ ...... - ~ ...... :·- Example: In [Bomhof et al., 1992] a mobile robot with an ultrasonic (US) sensor is described. The sensor readings are an angle under which the object is spotted, and a distance. The sensor basis Esens consists of cones which either are occupied from a certain distance on­ wards or empty (see Figure 1 ). Any information picked up in the environ­ ment represents a compound of such cones.

In the next subsection we treat the relationship between the sensor basis and the knowledge basis ending up in the definition of the cognitive basis.

2.3 Sensor and knowledge basis

The information flows from the sensor basis E sens to the knowledge basis Eknow· Hence, ideally E sens and Eknow fit to one another. As in information theory, one can deter­ mine how much information the sensors forward to the wiring. The basis is the number of states known to be possible at the source which is called entropy in infor­ mation theory. The amount of information forwarded is, moreover, bounded by how many of these possibilities can be excluded at the end of the channel. The source is the sensor basis Esens. the end of the channel is Eknow· For proper processing Eknow needs to respect relationships existing in E ···· ; e.g., if the sensor basis is as in

Figure I, adjacent cones of E sens need to be treated as ad­ jacent in Eknow· Formally, this is expressed by requiring that Esens maps by a homomorphism to Eknow· If, by mis­ take, there is no proper mapping, no real information can be exchanged .. Moreover, if Esens can be mapped onto Eknow (the mapping is a surjection) the sensory system provides the maximum of information which the wiring Eknow is able to process and we say that the sensor basis is suitable to the knowledge basis. The other way around, if Eknow can be mapped onto E sens the wiring can process the maximum of the information originating from the sensors.

The aim now is to define an internal cognitive struc­ ture for a robot, denoted by <I>. It should consist of those (classes) of objects OE U that can be distinguished by the sensory system as well as by the wiring. The sensors and the wiring (sensor and knowledge bases) must have "positive knowledge" (definition (iv)) about them. Tech­ nically we can define:

1 A homomorphism is a mapping that retains the relationships among the elements. In a robot it is fixed by the implementa­ tion.

(v) the cognitive basis <l>: if "t is a homomorphism from E sens to Eknow· <I> consists of the equivalence classes in­ duced by "t .

The definition says that <I> consists of just those elements of the sensor basis that can be processed by the wiring 2 . The sensors and the wiring together deliver the cognitive basis <I>.

3 Artificial perception

3.1 Sensors and Objects

In Figure I, the US sensor does not discriminate the two situations and the robot cannot find the corridor. One might try to refine the angle readings by adding an infra­ red (IR) sensor. An IR sensor can emit fairly precisely directed light beams, as reading is based on the reflection of the light (measuring distances is more complicated with this technique, and we will not consider it). On the face of it such a sensor might be used to increment the angle measurements. The aim is to extend the cognitive basis <I>; but such an extension is not straightforward at all. The behaviour of a light beam is quite different from a sound beam. Whereas cloth reflects a sound beam rather well, it can completely absorb light beams, and some objects are not detected at all by the IR sensor. Simpiy adding IR data (e.g., that no object is found) to the US data may cause inconsistency.

An elementary notion in symbolic AI is that of an (in­ dividual) object. This is true for most apparently suitable logical systems - such as [Rauszer 92] as well. Objects, however, pose a problem for artificial perception. The existence of individual objects has to be extracted from the sensor information. But this goes far beyond a simple interpretation of the sensor data. The sensors deliver only series of values of attributes, but from which objects the values originate is not detectable from the sensor signal itself. The idea of a perception basis was already given in [George, 1961]. His basis is indeed generated by simply joining the data of different sensors in extra di­ mensions. Still, this provides no answer as to how to "interpret" the sensor data.

In the definitions given above, the cognitive basis is constructed starting from external objects. The equiva­ lence classes, are sets of attribute values observed when sensing a particular object. Within the cognitive basis the "objects" are but sets of observed properties, i.e., in ad­ vance of any cognitive activity the creature has to be "taught" about the objects. The designer has to determine which attribute value combinations do indeed count as a certain object. In any case, for whatever approach, there re~ains an i.ntri~uing practical problem of determining which combmatwn of attribute values belongs to the same "object". This is essentially the problem of sensor data fusion . Any theory of cognition will be plagued by the problem of object identification.

2 t induces an isomorphism (that is a one-to-one homomor­ phism) between the sensor basis and (a part of) the knowledge basis.

3.2 The frame of reference

To recognise an object, the robot designer has to teach his robot. Here, frames-of-reference are interwoven. A robot is guided by its internal processing. Inputs from the sensors change the internal state of the creature. Call the set of possible internal states of the creature N. A certain sensor input makes the internal state v0E N of the crea­ ture change into v1E N. We use the cognitive basis to assign a meaning to this processing. The cognitive basis is stated in antropomorphic terms, the symbols in the cognitive basis are (representational) shorthands for hu­ ~an .(designer's) statements about the world. The ques­ t~on IS whether it is really legitimate to ascribe a cogni­ tive state to the creature. [Rosenschein, 1985] in devel­ oping his theory of situated automata, gives the follow­ ing definition:

(iv) agent x is said to know a proposition (<p) in a situa­ tion where its internal state is v if (<p) holds in all possi­ ble situations in which x is in state (v) .

We will not discuss whether in (iv) the term "know" is the most ideal. Our interest is in whether (iv) captures a useful cognitive notion. Let's first note that the definition covers unintended cases. E.g., consider a tree trunk: "if a tree trunk has 78 rings then it indicates that the tree is 78 years old'' [Stalnaker, 1984]. The tree trunk satisfies (iv), but we are reluctant to ascribe a cognitive state to it!

Definition (iv) has a more serious defect in that it does not lay down any causal or process-like relationship be­ tween v and <p. The consequence is that, if agent x is in state v but <p doesn't hold, the conclusion can only be that the observer made a mistake while interpreting v. In fact, the relationship between v and proposition <p is not (much) more than that between the tree trunk and the proposition that "the tree is 78 years old". Taking this route, one can, in a certain sense ascribe as much cogni­ tion to a creature as one likes, but it does not clarify the cognitive state of the creature. (We are in a certain sense commentating our own state of mind).

4 Actions

4.1 Intentionality

Above we were going through perception and ended up observing that definition (iv) is too liberal to delineate a creature's inherent cognition. The essence and the intri­ ~uing aspect of robot design is that ultimately some ac­ tions must result. Concerning robotics [Arkin, 1990] notes: "perception is meaningless without the context of motor action". We need to change the perspective a lit­ tle: we observe a robot which is acting and try to capture patterns of actions that are obviously aimed at bringing about certain results. This "purposeful" bringing about of changes in the environment is often called "intentionality".

. Exam~le: As ail illustration we consider a robot pro­ v.Ide.d with an obstacle avoidance procedure and a goal fmdmg procedure. It interprets the sensor data of figure

1 as an artificial potential field. Occupied cones are con­ sidered as a source of resistance, while the goal (given by co-ordinates in the same map) is taken as an attractive source. The magnitude of these forces decreases with increasing distances. (To visualise it one often represents the potential field as a landscape in which the repulsive sources form mountains and the attractive sources are valleys.) The robot steers itself along a route of low re­ sistance, i.e. through valleys in the landscape. This and similar procedures work rather well in implementing ob­ stacle avoidance [Khatib, 1986][Penders et al., 1994 ].

We might infer that the robot has the intention to avoid obstacles, but the avoidance of obstacles is not directly implemented. There is no point in the internal processing, where it is concluded "this is an obstacle", so "avoid it". Within the cognitive basis <I> we cannot draw this conclusion, since, as seen already, the sensor read­ ings do not allow for it. The potential field procedure is a "trick" implemented to obtain this resuit 3 ·

For convenience, we identify a motor-schema with the internal state v which it tends to bring about. For the ro­ bot using the obstacle avoidance motor-schema (implemented with the potential field procedure), v is an "equilibrium" state, from which no (new) impulses will reach the actuators. The performance of the robot is gen­ eralised in the following statement:

(vi) performing motor-schema v: in a situation where the sensor inputs do cause the mechanism to be in an internal state different from v , it performs until its same sensors cause v to occur.

Definition (vi) describes a causal process. Moreover, it refers only to internal features of the creature the defi­ nition is directed inwards the creature. How~ver, per­ forming a motor-schema does change the environment, something that is not explicitly mentioned in (vi).

As external observers (and robot designers) we want to express what happens in the environment within our frame of reference. We ascribe an intention to the motor­ schema.

(vii) A motor schema v is said to be an implementation of intention y if in a situation where y is not (yet) the case, motor-schema v is triggered and (under normal conditions) motor schema v brings about a situation in which y holds.

The robot with the potential field procedure avoids obstacles rather successfully, though "obstacle avoidance" is defined only in our frame of reference. We, external observers, ascribe the attitude to the creature, and assign a sense y to the mo­ tor schema v.

We required in (vi) that a motor-schema performs until a state v occurs. v should be caused by sensor inputs of the same type as that which caused -,v. Thus a motor-schema must ~ave an end definable in terms of its sensor inputs . The mternal state v , which characterises the motorschema, must correspond to some (for the creature itself observable) feature of the future state of the environ­ ment. If this requirement is dropped we can have, for in­ stance, a robot which releases a balloon whenever its sensor detects something. The robot has no feedback on its own ac­ tions (that is on releasing the balloon), or more precisely, it has no feedback of the type which originally forced it to per­ form. Such a robot seems rather close to the tree trunk above, which indicates the age of the tree.

3 The idea of an artificial potential field procedure is derived from physics, where its implications are rather different.

In robotics, it is straightforward to question: "What is the sense of collecting information, if it isn't used in ac­ tion?" A robot isn't but an acting device! For living spe­ cies one might ask a similar question, separating percep­ tion from action and setting perception apart as an independ­ ent phenomenon.

4.2 Motor-image

Sensory-based robotics uses motor-schema's as the primitive building blocks with which an architecture of a robot is set up. A motor-schema in robotics is the analo­ gon of the psychological concept of a motor-process (called motor-schema as well). In psychology it denotes the complex mechanism which links senses to the central nervous system and the effectors. The motor processes (incorporated in the schema) result in the execution of an action. Motor-schema's work like cybernetic loops or servomechanisms [Berthoz, 1996] .

The dual of a motor-process is a motor image. "Motor imagery can be defined as a dynamic state during which a subject mentally simulates a given action"[Decety, 1996]. A motor image implies a subject feeling himself performing an action, and imagined and executed actions share central bodily structures. The motor image is the "symbolic" content of the imagery [Annett, 1996] . Motor imagery is said to comprise the activation of an action plan. The motor image, though based on the sense and motor data, is an abstraction from these data. Motor­ schema's provide the foundation for the motor image.

Example: At the age of 4, my son was speaking to his friend (slightly older) about stairs. We were living at the second floor, our stairs were straight and very steep. The friend said "you have a long stair-case" pointing with his arm sloping upwards. My son agreed. The friend lived at the fourth floor, and their staircase rose in a spiral of eight separated staircases connected by landings. "Our stairs are like this .. ", the boy said, spiralling down with his finger and extending the movements with his arm and body. "Yes", my son shouted: "like this ... " , starting a great turning movement and while doing so, he whirled downwards, laughing.

A flight of stairs is an example of a concept consisting of motor-images; it is hard to define what a flight of stairs is without referring to bodily movements. The boys show the connection of motor-images with motor-schema's.

4.3 Motor-schema basis

We noticed earlier the difficulty of identifying objects. For analysing a robotic design the cognitive basis as de­ fined, seems sufficient because the robot designer pro­ vides the necessary frame-of-reference . However, when discussing natural (human) beings no external frame-of­ reference is available, and the ability-to-perceive distinct objects needs to be founded somewhere within the being. The importance of motor imagery in cognition suggests that motor imagery needs to be incorporated within the definition of the cognitive basis. As a preliminary at­ tempt we propose an extension to the definitions of cog­ nitive basis above.

An agent r, has at its disposal a set Ar of input chan­ nels or logical sensors and a set of motor-schema's N.­ The sensors (input channels) <XE Ar assign one or more motor-schema's to an object o in the universe of dis­ course U.

As before we can define an indiscernibility relation for a certain agent. In words, the definition will allude to: ob­ jects are distinguishable from one another only if they trigger one or more different motor-schema's.

So, we can define a motor-schema or motor-image basis as the set of equivalence classes of objects which trigger different sets of motor schema's. Hence, if two objects fall within the same equivalence class [ o lAro it means that the agent (re-)acts similarly when observing them. The motor-schema or motor-image basis is the set of equiva­ lence classes:

In (ix) the inputs are taken as mapping objects directly into motor-schema's. Each motor-schema incorporates (parts of) the perception and knowledge basis as previ­ ously defined. As in section 2 one may define mappings between the motor-schema basis and the perception, knowledge and cognitive basis. Roughly, what might be deleted of the latter bases are precisely those parts which do not trigger a motor schema.

We accent that definition (ix) is rather preliminary and may contain some (not yet foreseen) technical flaws. Be­ sides this, we have to emphasise some more principle points. The definition starts from objects, but as we have seen, objects do not have a universal (cognitive) status shared among all cognitive creatures. Another principle point is that objects must have an intentional "signi­ ficance" for the creature under consideration. By requir­ ing that a motor-schema be triggered, a rather basic type of intentionality is implicitly incorporated in definition (ix) .

For practical applications, it will be a problem to dis­ criminate between different motor-schema's and in par­ ticular motor-images. However, we achieve with defini­ tion (xi) a demonstration of a way to incorporate actions as basic elements into the concept of cognition (and "intelligence"). Suppose an agent that can only perform motor-schema's (a robot for instance) and that misses the ability to form images. This agent cannot but act on observing an object. In fact, practitioners in, for instance, biology make reference to actions when they are com­ paring intelligence of one species with another one, there are no alternatives.

4.4 Interaction

Individuals of the same species share the same types of senses and the same types of motor-schema's (and motor­ images). Thus, we have groups of agents that share much of their cognitive and motor-schema bases. In this case one can easily define a common cognitive and motor­ schema basis. The definitions of cognitive basis and motor-schema basis are indexed by the agent r; by letting the index range over the group, the shared common bases are obtained. The common cognitive and motor-schema bases are the intersection of the individual bases. The shared bases allow to define interaction and communica­ tion. Recall that, according to information theory, the amount of information carried by a signal depends on how many alter­ natives the receiver knows to be possible at the source of the signal. Creatures of the same "species" with limited bases, have few possible interpretations for observations. Moreover, if they can only perform motor-schema's (no imagery) they quite straightforwardly interact with each other. Some define such interaction as low-level communication: "A structural coupling between entities resulting in co-ordination of their actions necessary for their mutual viability"(Maturana, cited in [Stewart, 1995]).

S Conclusions

Approaching the cognition concept from the engineering (or design) perspective of robotics, we were advantageously en­ forced to be quite pragmatical. We introduced formal notions like sensor, knowledge, and cognitive bases. With them one is able to pinpoint one's frame of reference from which artificial cognitive creatures are viewed. Moreover, the whole issue of intentionality is clarified as belonging pre-dominantly to the frames of reference of the observer. or designer. The situation­ action perspective leads us to rethink the idea of a cognitive basis. The spectrum of (possible) actions becomes an integral part of a more encompassing idea of cognition. In this idea, motor-schema's (and possibly associated motor images) be­ come just as important for the designer as they are for natural scientists studying living nature. In a sense, AI has just begun to assess the importance of such notions for artifically living creatures.