Growth of Motor Coordination in Early Robot Learning

1 Introduction: developmental learning

In this paper we describe an approach to sensory-motor learning and coordination that draws from psychology rather than neuroscience. There have been many models of sensorymotor coordination but most of these have been based on specific, usually connectionist, architectures and tend to focus on a single behavioral task. We are interested in exploring mechanisms that can support not only the growth of behavior but also the transitions that are observed as behavior moves through distinct stages of competence.

Inspired by developmental psychology, we are investigating mechanisms for development in terms of stages (periods of similar behavior) and transitions (phases where new behavior patterns emerge) in very early development, in particular, the control of the limbs during the first three months of life.

Figure 1 shows our experimental robot system with two manipulator arms and a camera which is mounted on a computer-controlled pan and tilt head. In the present experiments only two joints of the arms are used, the others being held fixed. We fitted one arm with a simple probe consisting of a 10mm rod containing a small proximity sensor to detect any objects passed underneath.

2 The Motor Coordination Problem

Even before any cross-modal spatial integration can begin it is necessary to first discover the structure of the local spaces within each modality. Various stages in behavior can be discerned and during these stages the local egocentric limb space becomes assimilated into the infant's awareness and forms a substrate for future cross-modal skilled behaviors. The essential correlation between proprioceptive space and motor space seems to be a foundation stone for development, and occurs at many levels [Pfeifer and Scheier, 1997]. Sensory-motor growth in the limbs appears to precede visual development (it may begin in the womb) and even when it can continue concurrently with visual development. For this reason, in the experiments reported here we do not involve the eye system.

A two-section limb requires a motor system that can drive each section independently. The actual biological mechanisms of proprioceptive feedback are not entirely known, and there are several encoding schemes: joints encoding, shoulder encoding, body-centered encoding, and Cartesian frame.

2.1 Mappings as a Computational Substrate

We have developed a computational framework for investigating sensory-motor coordination problem based on a twodimensional mapping scheme. Each element in the map is represented by a patch of receptive area known as a field. The fields are circular, regularly spaced, and are overlapping. We assume that basic uniform map structures are produced by prior growth processes but they are not pre-wired for any spatial system. Our arm system has to learn the correlations between its sensory and motor signals and the mapping structure is the mechanism that supports this.

Every field in a map has a set of associated variables that can record state and other properties during operation: F{s, e, f,m}. These variables represent the value experienced, the current degree of stimulation of a field as a result of excitation or inhibition, the frequency the field has been accessed or visited, and the motor parameters that were in force when this field was stimulated, respectively.

The excitation level held in a map's fields is set to 1.0 for the first stimulation, but repeated stimulations are reduced by a habituation function that recovers when stimulation ceases. Also a very slow decay function causes all excitation levels to fall over time. Global variable Global excitation, is a measure of total excitation of all field excitations above a nominal lower threshold; Global familiarity, is a normalized sum of field access frequency. Such global indicators can be used to signal when changes have effectively ceased and the map has become saturated.

2.2 Constraint lifting and reflexes

Distinct stages of competence development can be achieved by lifting constraints when high competence at a level has been reached [Rutkowska, 1994]. We have several possible constraints: the availability of contact sensing, the resolution of the proprioception sense, and the parameters of the motor system. Of course, another constraint could be not having a visual sense but this very early stage of infant growth does not rely on vision. We use global state indicators to lift constraints in two ways: finer resolution sensory maps are used when global familiarity is high, and the degree of motor randomness increases with very low global excitation.

Novelty is the motivational driver and the motor system attempts to repeat actions that cause stimulation. We provide two basic reflexes to initiate the system: go to "mouth" and return to "rest"(arm being in the lateral position).

3 Experiments and results

The first trials used no contact sensing and objects on the table were either ignored or pushed out of range. Figure 2(a) illustrates behavior as traces of movements. As the stimulation levels of the mouth area fall due to the habituation function so random motor signals are introduced, which produce hand sweeps to points on the extreme boundary. When contact sensing is active, figure 2(b) then shows rest/mouth moves being interrupted by contact with an object on the path.

From these figures we see that the arm moved between mouth and rest areas first, but as these became less stimulated so random moves were introduced and fields on the boundary for the local body space were explored. Then, when contact sensing was allowed (a constraint lifted), internal fields and their neighbors were stimulated by object contact. Figure 3(a) shows map growth in terms of four "types" of fields. The number of fields

30

25

20

15

10

5

0

smallest field size middle field size largest field size mapping scale

0

20

40

60

80

100

15

10

5

0

120

number of trials Scale

(b)

Figure 3: (a): Growth of S-Mmap. (b): Transitions between three maps of different scale. Repeated visits are ignored.

observed behavior is seen as series of stages: first a "blind groping" mainly directed at the mouth area, then more groping but at the boundary, these are accompanied with unaware pushing of objects, then follows more directed and repeated "touching" of detected objects. All these behaviors, including motor babbling and the rather ballistic approach to motor action, are widely reported in young infants [Piek and Carman, 1994].

Regarding proprioception, we did not observe any clear advantage in any one encoding scheme. It is likely that the encoding scheme will matter much more when hand/eye coordination is to be learned.

From the field size experiments we see a trade off: speed of exploration versus accuracy. Figure 3(b) shows how the system started on a coarse map and progressively transitioned to a finer scale map as the global familiarity variable reached a steady plateau.

Regarding the excitation parameters, the main effects are to vary the persistent actions or number of repetitions performed on a stimulus and to alter the order in which attention is given to different objects.

4 Discussion and conclusions

Very few of sensory-motor coordination studies follow the psychological literature on development and even less deal with transitions between more than one behavioral skill pattern.

The system described here records sensory-motor schemas in topological mappings of sensory-motor events, pays attention to novel or recent stimuli, repeats successful behavior, and detects when reasonable competence at a level has been achieved. The behavior observed from the experiments displays initially spontaneous movements of the limbs, followed by more "exploratory" movements, and then directed action towards contact with objects.