Artificial Neural Network for Sequence Learning

1 Introduction

This article considers the problem of learning to predict events, i.e. to forecast the future behavior of a system us­ ing past experience. This problem has often been viewed and formalized in neural network theory as so-called tempo­ ral sequence learning. Studying such sequences is a topic of research in several domains such as robotic trajectory planning, speech or vision processing. For most of these, neural networks provide two distinct mechanisms: one for spatial information and the other for temporal information. The main mechanism stores the sequence events regardless of the temporal dependences between them. In parallel, or later, the second mechanism, the so-called short time mem­ ory (STM), extracts and learns the temporal relationships be­ tween the events. Moreover, we have shown in previous works [Gaussier and Moga, 1998; Andry et al., 2001] that the capability of learning a temporal sequence is one of most important features of a learning by imitation system. As a capability of learning by observation, imitation is a strong learning paradigm for autonomous systems. Imitation can improve and accelerate the learning of sensory/motor associ­ ations. In our work, oriented to the design of a neural network architecture allowing learning by imitation, we are involved in the first level of imitation [Whiten and Ham, 1992]. This "proto-imitation" level plays a key role in understanding the principles of the perception/action mechanisms necessary to perform higher order behaviors and it is likely that the proto­ imitation is triggered by a perception ambiguity. In our ap­ proach, the starting point for an "imitating behavior" imple­ mentation is the capability of learning temporal sequences of movements.

2 Temporal sequence learning model

Almost all neural models of sequence learning use a discrete temporal dimension by sampling the continuous time at reg-

Philippe Gaussier

ETIS I Neurocyber I U. of Cergy 6, av. Ponceau, Cergy, 95014, France gaussie r@ ensea.fr.

ular intervals. In these models, time proceeds by intervals of t:.t, and the interval between 2 items of a sequence is con­ sidered as a few units of t:.t (usually less than I 0). In this section, we introduce the model 1 for timing sequence learn­ ing with a variable and a long range time interval and we evaluate it. Our model (Fig. 1-left) is based on the idea that a prediction (P-type) neuron learns the timing between 2 items, or, to be more precise, learns to predict the end of this time in­ terval. This time interval starts with the firing of a derivation (D-type) neuron and ends with the firing of an input (E-type) neuron. The P-type neuron learns this interval using the ac­ tivity of the granular (G-type) group of neurons.

Let us consider a simple example: we present the first item and, one second later, we present the second item. The length of the interval to be learned is To = ls. The first item forces the D neuron to fire. The firing of the D neuron resets all the activity of the G neurons. Starting at this instant, the G neuron's activity is expressed by Eq. 1 . One second later, the second item forces the E neuron to fire. The firing of the E neuron enables the update of the weight between the P and the G neurons i.e. enables the learning of the To interval. Finally, when the first item is presented again the D neuron fires again. 920 milliseconds later, i.e. 80 milliseconds before T0 , the P neuron fires (Fig. !-right) and predicts an imminent firing of the E neuron.

1 The model is inspired by the functions of two brain structures involved in memory and time learning: the cerebellum and the hip­ pocampus (see [Banquet et al., 1998] for further neurobiological ref­ erences)

i - position of the neuron in the group (the ith cell of the battery); m ; and a ; -time constant and the standard deviation associated with the ith neuron; Ti - instant of the last reset of the battery.

3 Sequence learning architecture

Timing learning models are currently used by neurobiologist modelers for conditioning simulation [Bullock et at. , 1994; Grossberg and Merrill, 1992]. Alternatively, the proposed model permits the temporal sequence of events to be learned and predicted. In our context, a simple sequence is defined as an enumeration of events with the associated timing (eg. "A,B,C") and a cyclic sequence as periodic simple sequence (eg. "A,B,C,A,B,C,A, ... ") with the associated timing. The main idea is to use several batteries of G neurons for learning the timing between two consecutive events and a group of P type neurons for learning the event sequencing. The global architecture is shown in Fig. 2. The input group (CC) can be viewed as the input interface. Any neuron of this group repre­ sents a sequence event and it is ON while the corresponding input event is present; otherwise it is OFF. Each CC neuron is one-to-one linked with EC group of neurons. The EC group is made up of D-type neurons. Each EC neuron is linked with unconditional links to all neurons of a battery in the DG group. In the same way, an EC neuron is connected with un­ conditional links to all neurons of the corresponding column of the CA3 group. The DG group integrates several batteries

.. ....................... ...

of G-type neurons. A DG battery is equivalent to a Gi group of neurons shown in section 2. Each DG neuron is connected via conditional links to all neurons of the corresponding row in the CA3 group. The size of the CC group (respectively EC) is constrained by the maximum length of sequences. Alterna­ tively, the size of a DG battery is a function of the prediction precision of the time interval between two events of the se­ quence. This architecture can learn all event combinations, in other words, all possibles sequences. Consequently, the size of the CA3 group is the square of the input group (CC) . The output group (RO) is a Winner Take All neurons group. A neuron of the RO group has the same signification as a CC neuron. The RO outputs are connected to the EC inputs via one-to-one secondary unconditional links.

The architecture allows the learning of all kind of simple or cyclic sequences. The size of learned sequences is limited only by the system memory capacity.

4 Conclusion

The timing learning model and the associated neural net­ works were successfully utilized in [Gaussier and Moga, 1998; Moga, 2000] to teach an autonomous robot different "dances". The proposed model allows the correct timing to be predicted and it concords with Weber's law. Even if We­ ber's law concordance is not a prerequisite, it corresponds to a strong constraint of neurobiological inspired models of per­ ception and learning: if the prediction precision is constant then it is not possible to have reinforcement due to repetitive experiments. In addition, this model was successfully em­ ployed [Andry et al., 2001] to build a learning model based on the prediction of rhythms as a reward signal and for spatio­ temporal transition learning in autonomous robot navigation . These results prove that the proposed neural network archi­ tecture can serve both as a starting point for understanding imitation mechanisms, and as an effective learning algorithm for autonomous robotics.