Self-Organization and Segmentation with Laterally Connected Spiking Neurons

1 Introduction

Several models of the visual cortex that take into ac­ count lateral interactions between neurons have recently been proposed (see Sirosh et al. [1996b] for an overview). In the early stages of the development of the visual cor­ tex, lateral connections are believed to self-organize in synergy with the afferent connections to form a topo­ logical map of the input space. This process can be modeled computationally, showing how structures such as ocular dominance and orientation columns and pat­ terned lateral connections between them form based on input-driven Hebbian learning process (the Later­ ally Interconnected Synergetically Self-Organizing Map, or LISSOM [Miikkulainen et al., 1997; Sirosh, 1995; Sirosh and Miikkulainen, 1994; 1996; 1997; Sirosh et al., 1996a]).

Lateral connections may also play a central role in the function of the visual cortex, by modulating the spiking behavior of neuronal groups. They could cause

Risto Miikkulainen

Department of Computer Sciences The University of Texas at Austin Austin, TX 78712 USA

synchronization and desynchronization of spiking activ­ ity, thus mediating feature binding and segmentation. Such synchronization of neuronal activity emerges in the visual cortex of the cat when light bars of vari­ ous orientation are presented [Gray and Singer, 1987; Eckhorn et al., 1988; Gray et al., 1989]. Several models have been proposed to explain this phenomenon [von der Malsburg, 1987; von der Malsburg and Buhmann, 1992; Eckhorn et al., 1990; Reitboeck et al., 1993; Wang, 1996]. The model of Reitbock et al. [1993] is particularly inter­ esting because of its sophisticated model of the neuron: the synapses are leaky integrators that sum incoming signals over time with exponential decay. A network of such neurons can segment multiple objects in a scene by synchronizing neuronal activity. Spikes of neurons rep­ resenting the same object are synchronized, and those of neurons representing different objects are desynchro­ nized.

This paper shows how the leaky integrator model of the spiking neuron can be integrated with the LIS­ SOM model of self-organization. The architecture is named Spiking Laterally Interconnected Synergetically Self-Organizing Map, or SLISSOM. SLISSOM (1) forms a topological map from an initially random network through synergetic self-organization and (2) generates synchronized and desynchronized neuronal activity that can be used for segmenting multiple objects in the scene. The results suggest that lateral connections play a cen­ tral role in both the development and function of the visual cortex.

2 The SLISSOM Architecture

SLISSOM consists of two layers of interconnected neu­ rons: the "retina" and the "cortex" (figure 1a) . The overall organization of SLISSOM is based on the LIS­ SOM architecture [Miikkulainen et al., 1997; Sirosh, 1995; Sirosh and Miikkulainen, 1994; 1996; 1997; Sirosh et al., 1996a), and the neuron model on the leaky in­ tegrator neurons of Eckhorn el al. [1990] and Reitbock et al. [1993] . LISSOM provides a self-organizing princi­ ple and the leaky integrator neuron introduces temporal dynamics to the SLISSOM model.

Each cortical neuron receives afferent connections from the input layer and lateral (excitatory and inhibitory) connections from other neurons in the cortex. Each connection is a leaky integrator that performs de­ cayed summation of incoming spikes, thereby establish­ ing not only spatial summation, but also temporal sum­ mation of activity (figure 1b). Each new spike is added to the sum of the previous ones, and the sum is expo­ nentially decayed over time. The current sums are mul­ tiplied by the connection weight and added together 1 to form the net input to the neuron. The spike gen­ erator compares the net input to a threshold and de­ cides whether to fire a spike. The threshold is a sum of two factors: the base threshold () and the decayed sum of past spikes, formed by a similar leaky integra­ tor as in the input synapses. Active spiking therefore increases the effective threshold, making further spik­ ing less likely and keeping the activation of the sys­ tem within a reasonable range [Eckhorn et al., 1988; 1990] .

The overall organization of the SLISSOM model is shown in figure 1a. The cortical neurons receive input from all retinal neurons . The excitatory lateral connections are short-range (neurons marked black in the cor­ tex) , and inhibitory connections are long-range (neurons marked gray) . Each connection has a queue that stores previous spikes. In calculating the postsynaptic poten­ tial, the latest spike has the value of 1.0 and older ones are decayed by :};-, where Aq is the decay parameter, e · as they are shifted through the queue . The inhibitory feedback loop in the spike generator (figure 1b) is a sim­ ilar queue that receives spikes from the spike generator itself, with decay ;h-.

1 This differs from Eckhorn et al. [1990] and Reitbock et a!. [1993] who multiplied the weighted sums from afferent con­ nections and those from lateral connections. Multiplying ex­ erts better modulation on the neuronal activity, but disturbs self-organization by rapid fluctuation. In our experiments, modulation turned · out to be possible with additive neurons as well. ·

The input to the network consists of squares of fixed size (figure 1a) . Spikes are generated at the active reti­ nal neurons and sent through the afferent connections to the cortical neurons . The net input CJij to the spike gen­ erator of the cortical neuron at location ( i, j) at time t is calculated by summing the afferent and excitatory lat­ eral contributions and subtracting the inhibitory lateral contributions:

where /a, /e, and /i are the scaling factors for the af­ ferent, excitatory, and inhibitory contributions, ~r, ,r 2 is the decayed sum of the incoming queue from the retinal neuron at (r1, r2), /1-ij ,r,r, is the corresponding afferent connection weight, 1Jkl(t-1) is the decayed sum ofthe in­ coming queue from the map neuron at ( k, l) at timet -1, and E;j ,kl is the corresponding excitatory and l;j,kl the inhibitory lateral connection weight. The spike gener­ ator fires a spike if <Tij > (} + 1'J;j, where (} is the base threshold and 1'J;j the output of the spike generator's leaky integrator.

In the standard LISSOM model, the input is kept con­ stant while the cortical response settles to a focused, redundancy-removed activation pattern through the lat­ eral connections. SLISSOM goes through a similar set­ tling process. The input is kept constant and the cortical neurons are allowed to exchange spikes . After a while, the neurons reach a stable rate of firing, and this rate is used to modify the weights. Both the afferent and the lateral weights are modified according to the Hebbian principle:

where Wij ,mn(t) is the connection weight between neu­ rons ( i, j) and ( m, n), Wij,mn (t - 1) is the previ­ ous weight , a is the learning rate (a a for affer­ ent, aE for excitatory, and ai for inhibitory connec­ tions) , V;j and Xmn are the average spiking rates of the neurons, and N is the normalization fac­ tor, Lmn [wij,mn(t- 1) + aV;jXmnF for afferent con­ nections and Lmn [wij,mn(t- 1) + aV;jXmn] for lateral connections (cf. Sirosh and Miikkulainen [1994]) . Each neuron receives input from all receptors in the retina, and has excitatory connections with neighboring neu­ rons and inhibitory connections with a larger area of the map. 2 The radius of the lateral excitation is gradually reduced, resulting in fine tuning of the map [Miikkulai­ nen et al., 1997; Sirosh and Miikkulainen, 1997]. The weights are adapted both during self-organization and segmentation.

2 Although most long-range synapses in the cortex are ex-

3 Experiments

The SLISSOM experiment consists of two parts: (1) self-organization, and (2) object segmentation. During self-organization, lateral and afferent connection weights are adapted to form a topological map of the input space. After the network has stabilized, multiple ob­ jects (3x3 squares) are presented to the retina. The weights adapted to the input and the network segments the objects by temporally alternating the activity on the map.

The retina and the cortex both consisted of 11 x 11 units . The afferent weights were initialized to have recep­ tive fields of size 3 x 3 on the retina, centered right below each neuron, and then 65% noise was added to their val­ ues (figures 2a and 3a). The lateral connection weights were randomly initialized within [0, 1] (figure 3c). In­ hibitory connections covered the whole map, and exci­ tatory connections linked to a square area centered at each neuron (figure 1a), with initial radius of 8, gradu­ ally decreasing to 1 in 3,500 iterations. At the same time, the lateral inhibitory learning rate ai gradually increased from 0.001 to 0.1. Slow adaptation in the beginning cap­ tures long-term correlations within the inputs, which is necessary for self-organization. Fast adaptation towards

citatory, they can have inhibitory overall effects through in­ terneurons [Grinvald et al., 1994; Rata et al., 1993; Hirsch and Gilbert, 1991] . The LISSOM model predicts that such long-range inhibition is computationally necessary for self­ org~zation to occur [Sirosh and Miikkulainen, 1997] .

the end facilitates quick modulation of the activity nec­ essary for segmentation (section 4).

During self-organization, single 3 x 3 square objects were presented to the network. The retinal neurons rep­ resenting objects were spiking at each time step, and the settling consisted of 15 cycles of cortical activity update (equation 1). After settling, connection weights were modified according to equation 2, based on the average firing rate over the last 10 cycles. Each such presenta­ tion was counted as an iteration. After 5500 iterations, both the afferent and the lateral weights stabilized into smooth profiles. Afferent weights formed smooth gaus­ sian receptive fields most sensitive to input from the reti­ nal neuron right below the map neuron, as shown in figure 3b. Lateral weights formed smooth Mexican-hat profiles, as shown in figure 3d. Figure 2 shows the global organization of the map during the process. The final map {figure 2b) closely resembles the ideal map of the input space (figure 2c) .

Once the SLISSOM network had formed smooth and concentrated receptive fields and lateral interaction pro­ files, segmentation experiments were conducted on it. Several input spots (again, 3 x 3 squares) were presented to the retina at the same time. The spots constantly spiked on the retina for 500 time steps. For each spot, a separate 5 x 5 area on the map responded and the other areas remained silent. The lateral connection weights were adapted at each time step according to equation 2, with a; = 0.1, based on the average firing rate over the last 10 steps .

Segmentation is evident in the total number of spikes generated within each area per time step (i.e. the multi­ unit activity, or MUA; figure 4) . A high MUA value im­ plies that most neurons in the area are firing together , and a zero value implies that the area is silent. Ini­ tially, the three areas corresponding to the three input spots are equally active, but as time goes on, they start to alternate. The spikes within the same area become synchronized (the neurons turn on and off together), and the spikes across the different areas become desyn­ chronized (while one area is active, the other two are silent) . Such synchronized and alternating activity indi­ cates that there are three separate objects in the input; in other words, it constitutes a mechanism for binding and segmentation. This result is very robust and works repeatedly for different locations on the retina and for different numbers of objects, as long as the input spots are spatially separate {see section 4).

4 Discussion

Several studies have shown that fast adaptation of synaptic efficacy is necessary for feature binding through temporal coding [von der Malsburg, 1987; Wang, 1996). Similarly in the experiments with SLISSOM, rapid adap­ tation of lateral weights was found necessary for oscil­ latory behavior. On the other hand, self-organization requires slow adaptation so that long-term correlations can be learned. If the weights are initially random and change rapidly, they will fluctuate a lot and an ill-formed map will result. There are two possible solutions to this problem. One way is to have two sets of lateral con­ nections, one for fast adaptation and the other for slow adaptation [Wang, 1996]. The other is to vary the learn­ ing rate of the synapse. It is unknown which approach is more biologically plausible; this question has yet to be settled physiologically. In this work, the learning process starts out with a slow learning rate and grad­ ually the synapses become more plastic. This scheme does not disturb the self-organization since the activity on the map becomes more consistent and predictable as the training goes on, and the need for keeping track of the long-term correlations disappears. The two solutions are mathematically equivalent and there is no sufficient neurobiological evidence to distinguish between them at this point . The second one is simpler and was therefore chosen for this paper.

The MUAs show some overlap even when the input is successfully segmented (figure 4) . This is due to the slightly overlapping receptive fields in the model. Gray eta!. [1989] observed that in the cat visual cortex, strong phase-locking occurred when the receptive fields were clearly separate. Apparently when they overlap slightly, phase locking becomes less well defined at the edges. The overlap is unavoidable in the current small SLISSOM network, but could be reduced in larger-scale simula­ tions. Such simulations with a large number and variety of objects constitute the most immediate direction of future research. Segmentation in a more detailed self­ organized model of the visual cortex, with orientation columns and patterned lateral connections will also be studied, and it may be possible to account for phenom­ ena such as Gestalt effects based on the patterned lateral connections.

5 Conclusion

In this paper, the SLISSOM model of dynamic spiking in a synergetically self-organizing map was presented. Adapting lateral connections were shown to play an es­ sential role in both self-organization and image segmen­ tation, showing how the development and function of the visual cortex could be accounted for by a single unified architecture.

Acknowledgments

Thanks to Andrea Haessly and Joseph Sirosh for ini­ tial simulations that led to the SLISSOM architecture. This research was supported in part by the National Science Foundation under grant #IRI-9309273 and by the Texas Higher Education Coordinating Board under grant #ARP-444.