The Model of Human Visual Long-Term Memory with Ability for Generalization

THE MODEL OF HID.iArl VISUAL LON�-TEID~i MEMURY WITH ABILITY FuR GENERALIZATION.

R.~.�ranovsk:aya

Leningrad University, USSR

Summary

A model of human visual long-term memory with ability tor seneralization baa been synthesized. Perceptive structure of the system may be either parallel (multilayered retina) or successive (a network scanning a contour by tracing).

The synthesized model of retina per­ forms a number of transformation of information as in a visual analyzer - such as intensification of contrast between an object and its background, contour or outline ~xtraction, reaction to the mo­ vement of an object (in a particular di­ rection at a certain velocity).

It is the code tree that is used as a structure for storing codes in long-term memory. This structure explains the de­ pendence of the response time upon the number of alternatives and allows studying the dynamics of the following tra­ jectoria (path) reduction occuring during learning process.

A model of human long-time memory pre­ sented as a structure of a tree consisting of neuron models is described and such processes of information transfor­ mation in it as recording, storage, generalization, reading, disposition set to the given type of signals are consi­ dered. The perceptive structure of this system may be either successive (scan­ ning) or parallel (multilayered retina).

There are various kinds of scanning: a) successive, ranked, one-channel tracing of the contour and the formation of primary codes·b) distant,uneven tra­ cing on characteristic points of ob - jects and the formation of secondary co­ des· c) transition to the parallel mul­ tichannel scanning through the retina.

During the tracing the contour is di­ vided into segments according to a defi­ nite rule. The number of segments is de­ termined by the contour configuration only. Each segment is characterized by a ranked group of the primary signs (~, t, t, 8, A , E. 1 1v ) which are func tiona of its curvature. The primary code is a table in which the number of lines and columns corresponds to the number of signs used and the number of segments of the contour.

The use of the one-channel contact tracing brings us to the necessity of introducing information about the enti­ re perimeter in the form of a definite simple sequence, which reduces the rate and increases the number of limitations on the condition of perception. Digres­ sion from the contact method of perception made it possible to elaborate the corresponding extended aecondary signs of the object and to form the secondary code, out of them, The secondary codes are more generalized and shorter than the primary ones (they are one line in­ stead of a table as is the case with the primary codes) and they simply corres­ pond to an object, the sequence of pre­ senting the signs bei~ of no imp~rtance, The use of the secondary codes in the process of identification made it possi­ ble to trace characteristic points of an object irrespective of the sequence of the tracing, which reduces the number of limitations on the object under percep­ tion and makes it possible to place it into the field of vision irrespective of sequence. Besides this the characteris­ tic parts of an object may be presented, not an .· entire object, The secondary codes are invariant with respect to orthogonal, similarity, affine transformations of the contour and transformations of the central pro­ jection within definite bounds as well, The code tree is used as the long-ti­ me memory structure for the primary and the secondary codes. This structure ex­ plains to us how the time of the reac­ tion depends on the number of alternati­ ves, makes it possible to fix how often separate events occur, and to examine the dynamics of the tracing trajectory reducing while training (learning), Multichannel parallel introduction of information about visual objects in­ to memory is performed through retina in the model. This means that a certain retina structure will provide for the investigation of an object in a defini­ te sequence, division of the informa­ tion thus obtained into pieces and its investigation by these pieces.

Assuming that sensitivity of percei­ ving elements of the visual system chan­ ges so that at every moment sensitivity of only one group of elements exceeds the threshold while that in other groups does not reach it (the threshold) only those parts of an object are picked out simultaneously that are projected on the elements of high sensitivity. Hence, an object is picked out successively oy certain parts, and every part is picked out parallel to working elements. In this case we have the system working pa­ rallel-successively.

In case we have parallel, not succes­ sive introduction of information on a uniform retina it may be demonstrated that while the thresholds of the ele­ ments of retina change according to a periodic law, the information about the object is transferred from the retina to memory successively by parts, the nature of succession being dependent on the object properties. However dependence is different from that when the contour is traced. It means that the stabilized (mo­ tionless with respect to the retina) object can be perceived and introduced in memory.in these conditions, too.

Transition to such a type of percep­ tion is also a kind of tracing the clo­ sed cycle which is set by a period of threshold changes, the dynamic process of threshold changes being independent on the movements and may be set centrally, which quickens the process of percep­ tion. Periodic changes of the threshold make it possible to obtain a series of codes of one and the same object (ranked according to the law of element sensitivity changes). Formation of the group of codes of one object, the group connected with the inner period of the perceiving elements sensitivity .changes without the outer transformation of the object enab­ les one to make an approach to the expla­ nation of certain aspects of the constan­ cy of perception mechanism.

Thus, when the visual system works in a successive-parallel way the movement of the tracing system of the eye may be replaced by the dynamic regime of the perceiving neurons' sensitivity changes.

The model of the retina is a structure consisting of homogeneous elements. Each element is a multi-layered network. The first layer is the receptive field of an element. The second one consists of novelty neurons. This layer picks out brightness changes in the receptive f i ­ eld and, thereby, the object's border. The third layer consists of summating neurons. The line curvature is detected irrespective of the line ori~ntation in this layer, i.e., the element in this layer yields the output proportionate to the line curvature in the receptive field. The fourth layer consists of difference neurons and responds to the mo­ vement of the picture and performs se­ lection of the time-space code, picking out certain gradations of the curvature (Pig. 1, 2).

eld of an element. (b) The second layer­ novelty neurons. (c) The third layer - summating neurons. (d) The fourth layer­ a difference neuron. The types (b)~(d) neurons equations are shown in the figu­ re 6 of the report "The model of Human Short-Term Memory" by I.Ja.Bereznaya and R.M.Granovskaya.

(a) The first layer - an object. (b) The second layer. (c)-(h) The third layerat successive time moments t}-t 8 , when neu­ rons threshold value changes from m=8 to 111.:}.

Thus codes of object pictures at re­ tina output are invariant with respect to all the orthogonal transformations, namely, a turn, parallel transition of a pi cture and mirror reflection.

The fact that there is no division of the neuron receptive f i eld into ex­ citing and inhibiting zones is the spe­ cific feature of the given retina model.

The offered retina model makes it possible to illustrate the psychological phenomenon of illusionary movement which consists in the perception of the seeming movement of one object, although two objects are presented successively at different points. The reaction to the signal change either in space or in ti­ me is the main principle of how all the layers of retina operate. The operation of the given retina is considered as a model of separate stages of visual ob­ jects perception (Fig.}).

The memory model is either three-or two-dimensional code-trees, one- or mul­ ti-layered, dichotomous or K-tomous, built of homogeneous or heterogeneous joints and elements, what is common about them is a structure resembling a tree. The usage of the code tree as the main structure of a long-time memory has great advantages over other types of mamory arrangement, as for example the teaching matrix. Such structure makes it possible to use fewer elements than when codes are stored as an infinite sequence, which increases the rate of arriving at a decision when the code is searched in memory,

movement.

(a) The retina first layer - an object, (b)-(h) The retina fourth layer at suecessive time moments when a neuron threshold value descends from m=9 to m:3.

The process of recording in the of­ fered memory model presented as code trees is arranged so that a successively ranked set of object signs arrives at the input either directly from the outer input or from the output of the short-time memory, Each of the element after missing a certain code is inhibi­ ted while the code continues to be recorded in the upper layers of the tree, This results in the successive recording and reading of the codes,

In a situation like this a spatial code tree that makes it possible to re­ duce the number of alternatives in the tree joints is used for the storage of primary codes ·

A two-dimensional code tree is also used as a structure for the storage of secondary codes. Each gradation in such a tree will have its corresponding branch, each code - its separate way, each sign - its layer. The identical co­ des will have their identical way, the repeating codes do not alter the tree configuration,

Accordingly, the tree will have as many layers as the number of signs used and as many ways as the number of dif­ ferent codes arriving at the input of the memory system, Each way of the tree is ascribed ita weight proportionate to the number of the corresponding codes that arrived at the input of the memory system.

The only way corresponds to the full code arriving at the tree input. The end of the way is connected with the corres­ ponding index. Besides this index, in­ dexes of all the joints lying in the gi­ ven way can be counted. In case we have several ranks of the code arriving at the input, note the full code reading may be performed with but a certain de­ gree of relativity (Fig.4,5)

The tree structure provides for the only descent (simple identification) from any joint down to the tree foot while the reading takes place. If signs are different as to the number of gradations, the number of elements of the tree will depend on the succession in which signs are to be applied. The number of elements will be the smallest if the signs are applied so that the number of gradations increases. Omission of non-ramifying ends of ways in the full tree can serve as the following step to economize elemente.

tree.

When the arrival of signs in the code is ranked, the memory structure is eco­ nomized. First, the arrival of the code of the same object with a larger number of signs does not require additional co­ de recording; second, the ranking of signs according to the number of grada­ tions or to their importance makes it possible to reduce the number of steps along the tree so that the identifica­ tion is with the same degree of accuracy, i.e. the ranking is advantageous ae to the process rate.

Vfuen the ranked system of signs is used, the tree structure enables one to increase their number while the codes already in storage are not changed. This enables one to analyse objects and re­ cord codes with any degree of accuracy needed, moving from the tree foot up to its summits as it is relevant in some cases to adopt decisions at the expense of a rough classification and at the sa­ me time to reduce the time of decision without going up the tree. Such structu­ re ~kee it possible to perform a simple recording and code reading by ita index which is compared with one of its sum­ mits. Recording and reading are per­ formed without address in this structure, associatively (in so far as the code has no special address group) and the code arrangement on the tree is determined by the code properties only, the struc­ ture of its eigne.

While reading, a complete shuffling of the codes in storage is not performed~ the needed code or a group of nearby co­ des are read immediately, which quickens the process of reading. The time of se­ lection does not depend on the bulk of the information recorded, being deter­ mined by the number of layers in the tree only.

The measure of depth of the code in storage has been introduced. It is pre­ sented as weights of those branches and joints of the tree which store informa­ tion of the given code. The weight ie connected with the changes of synaptic coefficients of those elements which the codes traverse in the process of teach­ ing.

Weights accumulation in the tree structure is compared with the frequen­ cy of occurrence of the given code at the system input. If sequences with the maximum weight are read out off memory they will be the codes that have been most often presented in the history of the system.

It has been shown in the model that the optimum uneven strategy of the po­ tential hypothesis check-up can be elaborated on the basis of the statistics in the tree accumulated in the process of teaching and the trajectory of the ob­ ject investigation changes as the result of this. Thus, the brain produces a sort of reflection of an object which is ex­ pected to be on the retina: and this brings about changes in perception and speeds up the process of identification.

Arrival of information about new ob­ jects brings about changes in weight and in pseudoweight as well, i.e., there is accumulation of information in the memo­ ry structure not only about the codes that arrived earlier (weight) but also about these ones that are similar to them according to a certain criterion of similarity, i.e., codes are generalized by their similarity, the similarity be­ ing determined by the nearness of the signs arrangement in the memory structu­ re or well by the nearness of signs in the ranked sequence when they arrive at the memory input (pseudoweight).

The usage of pseudoweight makes it possible to reduce the period of teach­ ing (learning) through the reduction of the teaching sequence and to teach the system of memory with the help of the most characteristic representatives of the class, only bearing in mind that a sort of intermediate relief stretches automatically between the excited parts of memory. Such a system starts respon­ ding to objects that were not presented earlier but are part of a class of such objects and are similar to them accor­ ding to certain criteria. The system responds to the objects as if they were known to it.

The usage of peeudoweights makes it possible to optimize (reduce) the teach­ ing sequence according to the criterion needed and thereby reduce the process of teaching the system the given dictionary of objects.

The introduction of weights and peeu­ doweights makes it possible to approach the modelling of such important psycho­ logical process as the formation of a unit determined by the immediate prehis­ tory.

The usage of pseudoweights also makes it possible to extrapolate properties of nearby objects i.e. to solve the task before its exposition is over or even by way of an imaginary experiment with it.

Assuming that there is a certain tree structure, homogeneous as concerns its properties before the teaching, the structure will grow more and more hete­ rogeneous in the course of teaching in eo far as the changes in the distribu­ tion of weights of branches and joints go. The greater the number of different codes that arrived at the given memory structure, the greater the number of tree joints used, i.e., the history of the system gradually goes up the initial­ ly homogeneous tree, i.e., it sort of grow grows. The heterogeneity thus appearing is isomorphous to the group of the passed codes both by the nature and the area it spreads over and makes it possible to re­ store the history of the system with a certain degree of accuracy later on.

After the memory structure had passed through a certain sequence of codes it is a heterogeneous system consisting of ensembles of functional elements with the rat~ of processes varying in dif­ ferent parts of the system and the sig­ nals pass from one joint to another with the varying probability. Values of the processes rate in the ensembles and the distribution of weights of the branches that determine the branch the process will spread through are determined by the prehistory of the system in the pro­ cess of teaching only. The usage of a special system independent of the input sequences to control the thresholds of the functional elements of the tree in the model makes it possible to approach the explanation of how the importance of the task and its emotional colouring in­ fluence the narrowing of the solution search zone.

When such system is launched the way the given code in the tree will follow will be determined not only by the pre­ history of the memory system but also by the changed sensitivity of the upper joints of the tree. In this case the co­ de will follow another branch and will bring about another solution (it is identified with another index).

The measure of novelty of the solu­ tions thus outained is determined by the number of potential prolongations from the given joint. If there is a group of ramifications the weights of the sum of prolongations being equal, the changes in the sensitivity of one of the upper layers will essentially narrow the search zone, i.e., the task in this par­ ~cular case comes out as a factor of the search directness and determines the formation of the unit connected with the signal importance.

Reduction of the identification of printed letters of the Russian alphabet has been considered as an example of re­ duction of the process of identification Primary codes concerning sigus ..l , }> , 0 g , 1 are patterns of letters by the successive scanning. It has been shown that the greater the number of signs used the smaller the numoer of groups of the primary code and the corresponding part of the contmU' required for the simple identification. When only one sign d._ is used the alphabet falls into 18 groupe, when signs ol and f are used there are as many as 28 groupe. All the letters are mutually separate with a lar­ ger number of signs. The average number of intervals per letter necessary for the identification will be 13,3, in case all the five eigne are used the number of intervale will be 4 (Fig.6).

Further reduction of the process can be achieved at the expense of transition from the continuous tracing along the contour line to the uneven one.

Direction and value of the leap are determined by the hypothesis put forward on the basis of the analysis of the up­ per part of the tree structure. The leap must result in the hypothesis check-up (Fig. 7).

When information about letters of the Russian alphabet iA introduced in memory with the help of retina all the letters are subdivided into groups whose number depends on the number of gradations of the retina elements sensitivity. It has been shown that if the number of grada~ion tiona is six. All the letters are dis­ cerned by the retina. The smaller the number of gradations the rougher the classification i.e., several letters find themselves in one and the same group(Fig. 8).

The organization of memory according to the principle of the code tree makes it possible to perform the object iden­ tification through successive approaches while going from the foot of the tree to one of ita summits. The change of hypothesis from a more general to more par­ ticular one up to the precise identifi­ cation takes place in the process of such a movement.

a

·

·

1-'ig.B (a) Hussian letters code tree a f ter parallel multichannel scanning through the retina model when the model fourth layer neuron has 6 sensitivity change ranks. (b) Russian letters on the retina model,