Constructive Visual Imagery And Perception

Introduction

One active area of artificial Intelligence research Is the Inquiry Into the nature of human cognition. One aspect of this Investigation Is the attempt to embody theories of cognition In the form of programs which simulate both general characteristics and specific Instances of observed cognitive behavior. This paper Is a report of one such effort In this area of artificial Intel­ ligence viewed as theoretical psychology. It Is based upon research conducted and reported as a Ph.D. thesis In computer science at Carnegie­ Mellon University <1>.

Classical or atomistic theory proposed that the visual perception of form Is an "unconscious conclusion" realized by "unconscious Inferences" which are based upon the values of the smallest discriminable or homogeneous patches of the stimulus <2>. The Gestalt theory of visual form perception developed as a reaction to the ~allure of the Classical approach to adequately account for the effects of context upon the valuation (interpretation) of any part (atom) of the stim­ ulus. Gestalt theory defines several organiza­ tional principles (laws of proximity, continuity, symmetry, sImp II cIty) wh I ch are app II ed to the whole perceptual field to produce the visual form perception (J). Gestalt theory, like atomistic theory, has proven to have Its weaknesses. The laws have proven difficult to specify In quanti­ tative or operational terms. Perception of par­ tial figure regions can have significant effects upon complete figural perception <~.5>. Most notably, Gestalt theory Ignores the fact that visual form perception normally Involves multiple fixations of the eye and so provides no means for the Integration of Information from successive differing views.

The constructive theory of visual form per­ ception has developed as an alternative to the Classical and Gestalt approaches. This theory proposes that an Internal representation of the visual field Is constructed by the Integration of a succession of views of (fixations upon) the environment. This representation Is both guide for and product of visual form perception. Hebb <6> began the modern psychological formulation of this theory, describing "cell assemblies" joined together by (Into) "phase sequences" as Its basic functional elements. Hochberg <7,8> has recently continued the Investigation. He proposes "schematic maps" as the underlying struc­ tural organizations which make possible the sel­ ective attention to and the successive Integra­ tion of the visual environment.

The research which Is reported here Is a further Investigation and specification of the constructive theory of visual form perception. More speclflcal ly, the goals have been: (f) to Investigate the nature of the processes and

CONSTRUCTIVE VISUAL IMAGERY AND PERCEPTION

Arthur M. Farley University of Oregon Eugene, OR 97403

memories which are Involved In the fixation and Integration of successive views of the environ­ ment; (2) to Investigate the nature of the Inter­ nal representation (symbolic visual Image) which Is capable of embodying the necessary partial and complete perceptions; (3) to specify the results of the Investigations In the form of an operation­ al, computer-Implemented visual Imagery and per­ ception system (VIPS>. VIPS Is such a program which has been Implemented In LISP 1.6 on the CUM POP-10.

Motivation for the two Investigative goals Is abundant. The need Is best expressed by the following two statements:

"There Is little evidence to guide our thinking on how these Integrations and constructions take place. In fact, ··· , little attention has been paid to how such processes occur at al I · " ( 9,pl ?It )

"What we need Is a set of operations for defining and studying the kind of visual storage that will build up the structures of perceived forms out of momentary glimpses" ( 7 9 p322l

The third goal, that of computer Implementa­ tion of the theory, had two primary motivations. One was to force an operational specification of the theory, somethlnq which Is often elusive for concisely stated, Intuitively understood, descrip­ tive theories. The other was that features of the Implementation could be expected to be naturally applicable In extension to the realiza­ tion of an adequate, generalized computer vision system.

THE EXPERIMENTS

Two experiments were conducted to provide data from which to Infer characteristics of the visual Image representation and rules of the per­ ceptual processes. The first experiment presented subjects with a task situation which forced them to perform a perceptual actlvltv over an extended <cognitive) time frame. VIPS has been Implemented to explain this behavior. The second experiment presented task situations nearer to that of "normal" visual form perception. These results are considered In light of the theory embodied by VIPS and as bases for Its necessary extension and modification.

The maln task of Experiment I presented the subject (4 subjects) with a line drawing taped to a table which was covered by a large paper mask with a hole In It, The hole (of approx. 3 degrees visual angle) allowed the subject to view at most one vertex at a time. The subject's task was to move the hold about the drawlnq untl I being capa­ ble of verbally describing and drawing the picture. The subject was also Instructed to "think aloud"

during the hole movement sequence. This task Is an extension and modification of a partial, se­ quential viewing task described by Hochberg ( >.

Transcriptions of the video-taped protocols served as a basis for perceptual process rule specification, for the structure of recognition I ong term memory, and, together wIth the verba I descriptions and drawings, for Image representa­ tion soeclflcatlon. Figure I Is the Initial seg­ ment of a transcribed protocol. 8 1 Illustrates the view through the hole (circle> as then seen by the subject. If this Is a vertex, a label - number Is given In the lower right. C' Indicates the direction of hole movement, If any, by arrow and symbol. 0 1 Is the transcribed verbalization which occurred at that point. This example Illustrates the type of behavior this task situ­ ation produced.

'Experiment II was conducted upon a corneal reflecting eye movement tracking and recording (video-taping) system. Two subjects participated; both had been subjects for Experiment I. In the first part, e line drawing Cof approx. 20 degrees visual angle) was presented for 250 to 350 milli­ seconds whIle the subject f lxeted a pre-set pol nt of the visual field. Immediately upon removal of the picture, the subject proceeded to verabl ly describe and to drew what he now knew of the line drawing. This data served as basis for Inferring the processing that an Initial fixation receives and characteristics of the resultant representa­ tion.

In the second part, the subject was again given e pre-set Initial fixation point. A line drewln9 (approx. 20 degrees> was presented, end the subject was allowed to view <scan> the I Jne drawing until being capable of verbally describ­ Ing and drawing It from memory. The picture was removed at the subject's signal and the subject proceeded first to describe, and then to draw, the line drawing. Eye movements were videotaped and the verbal descriptions were audio­ taped. This data was transcribed as sequences of fixations specified by location end duration. Figure 2 shows one such transcription. Field A' Is fixation number, B' Is fixation duration. Initial segment of the sequence has been connected by dotted lines. The data has been used to judge the feaslbl I tty of eccomodatlng the theory of VIPS to account for eye movement data.

THE I~�E REPRESENTATION

Imagery has recently returned as a concept under Investigation In cognitive psychology. One result of this renewed Interest Is the need for an adequate scientific definition of Imagery. An I mage Is def I ned here to be an I nterna 1 , semant 1 c, symbolic representation of Information which Is capable of determining (guiding> behavior and which has an Internal modality characteristic. The Internal modality characteristic distinguishes Images fnom other forms of Internal symbolic re­ presentation. It requires that the Image be structured so that It can be (Is accessed by pro­ cesses Isomorphic to those which access the exter­ nal environment for the specific sensory modality Q.()) · The unIts of representatIon, the symbo Is and relations, are also modality specific, for vision this being visual feature symbols and spatial relations. By further specification, a visual Image Is an Internal symbolic representa­ tion of visual feature and spatial relation Infor­ mation which Is structured so as to be straight­ forwardly retrievable by Internal processes Iso­ morphic to those which access the visual environ­ ment during visual perception. A visual Image Is capable of guiding motor and cognitive behavior with regard to the visual environment that It represents. A primary example of such behavior Is visual form perception. On the other hand, visual form perception Is a primary source of visual Images. Thus, the perceptual visual Image serves as guide for Its own construction.

The basic unit of meaningful visual Informa­ tion within VIPS Is the Image chunk. An Image chunk Is a semantic structure of Interrelated symbolic elements, which represent a visual con­ ceot. Image chunks are the basic structural units from which more complex Images are then construc11ottl. VIPS uses five types of Image chunks <concept types) to construct Its representations of the line-drawing environment, these being the VERTEX, OBJECT, LINE, SIDE, and FACE types.

An Image chunk consists of one Chunk Header element, one or more Position elements, and sev­ eral Image Body elements. Properties of the Chunk Header serve to Indicate the chunk's type and general characteristics of the current Instance. The Current Reference property alwavs references one lmaoe Body element, therebv affording a means of access to the chunk's representational struc­ ture. A Position element serves to Indicate the perceived location of the Image Body elements which reference lt. This position Is In terms of a seven-by-seven coordlnate·grld of Jocatlonal areas. All Image Body elements reference a Posi­ tion element, binding the Image Body structure, and thus the visual Image, to locations In per­ ceptual <Imaginal) space. Perceptual space Is not retlnally-based, but rather Is bound to the area of current Interest within the visual field. This Indirectness plays a significant role In the maintenance of a stable visual world In spite of the differing retinal states which occur due to fixation changes during visual form perception.

A chunk's Image Body elements form a non­ hierarchical symbol structure which embodies the spatial configuration of visual features of the visual concept represented by the chunk. This structure Is a doubly linked circular Jist. The relational links between Image Body elements have directional or space-traversal (direction and dis­ tance) meaning. Thus, the allowed means of accesslnq the Image, which Is by traversal of Image Body elements according to existing struc­ tural Jinks, Is Isomorphic In meaning to a visual search, or scan of the external visual environ­ ment. This feature of the Image representation Is the embodiment of the Internal modality char­ acteristic for visual Imagery.

Five classes of Image Body elements (XJT, ANGLE, INTERNAL, END, and QUICKSEE> are used to represent the concepts embodied by Image chunks. Each element class has an associated set of features and relations that It can embody which In turn defines possible roles for It In the Image Body structure. Figure 3 Illustrates the use of these element classes In representative examples of the different chunk type Image Body structures. A VERTEX chunk consists of alterna­ ting XIT and ANGLE elements. An OBJECT chunk consists of "corner configurations" made up of INTERNAL and ANGLE elements. An INTERNAL differs from an XIT element In that It has only one In­ ternal vertex direction link to an ANGLE element. This Image representation feature embodies the figure-ground phenomenon of visual form percep­ tion and Imagery. To traverse around the "outs I de" of an object corner, another reI a ted chunk must be accessed (attended>. The LINE and SIDE chunks Introduce the use of the END element. A SIDE chunk Is always associated with an OBJECT chunk.

A visual Image Is a structure of Interrelated Image chunks. An Image Body element of one Image chunk may be linked to one of another chunk by one of two basic relation types · . One type has space traversal meaning, relating the two Image Body elements In terms of a direction and dis­ tance. The other type Is an equivalency rela­ tion. This type of link relates two Image Body elements of different chunks which represent the same aspect of the external visual field. For example, a line segment shared by two adjacent objects Is redundantly represented by both OBJECT chunks. INTERNAL elements embodying the shared II ne segment "sIde" of the objects are II nked by Inter-chunk equivalency relations. <See bottom Figure 3> Note that the line segment Is redun­ dantly represented In both OBJECT chunks.

Imooe Body Structures

The symbolic Image representations of VIPS differ from the more semantic representations of visual Information proposed by Winston ~). His representations do not satisfy the Image defini­ tion given above. Such relations as "SUPPORTED­ BY" and those accompanIed by sate I II tes, such as ""llST-NOT-ABUT", are not spatial or equivalency relations. Such semantic relations do appear to play a role In perception, possibly In the Incor­ poration process which forms more complex, "deeper" structures from Interrelated Image chunks. The process could provide perceptual hypotheses based upon the long term semantic know­ ledge to help realize the goal of efficient re­ presentation. Also, VIPS does not currently employ a "SCENE" element, but It would exist In the guise of a contextual element <1>.

THE PE~EPTUAL SYSTEM

The perceptual system of VIPS consists of six memories and four processes, as shown by Figure 4. In the figure, an arrow from a process to a memory Indicates that the process can alter the memory's contents, whl le an arrow from a memory to a process Indicates that the process can access the memory's contents. Characteristics of the system at the architectural level reflect relevant data and theoretical proposals of cogni­ tive psychology.

The Memories

Since perceptual activity Involves the Inter­ action of organism and environment, VIPS must represent both. PIC (PICture) Is the environment, defined In terms of the visual modalltv. As such, It consists of a list of vertex feature lists. Each vertex feature list represents the Inferred detectable visual features at one of the line drawing vertices <the observed points of "hole fixation">. These vertex feature lists are Inter­ related by links of directional and distance meaning. The Current Picture Pointer <CPPl Is associated with PIC and references one vertex feature list, representing the current hole posi­ tion.

The five cells of the VIsual Register <VRl serve as communication registers between the VI-SYS and AA-SYS processes. The eel Is of VR are set prior to VI-SYS activation by M-SYS. VI-SYS accesses PIC accordlnq to the VR cell specifica­ tions and accordingly alters the eel Is of VR prior to returning control to AA-SYS by deactiva­ tion.

The VI (VIsual Information) cell Is the Iconic visual Image. It Is constructed automat­ Ically as a result of the new hole positioning. The representation of visual Information In VI Is that of an Image chunk, though a simple feature list aopears favorable. It Is as yet unrelated to any existing Image contents pro­ duced by prior perceptual activity. Its contents and structure are not affected by any active per­ ceptua I goa I ·

When specified, EXP <EXPectation) and ATT <ATTention) make possible the application of pre­ attentive functions by VI-SYS to the newly accessed PIC Information. EXP can be specified either as an angle code (when ATT Is also speci­ fied) or as a vertex type and specification. Upon VI-SYS deactivation, EXP wl II be YES, NO, or CON (CONtained>, Indicating the relationship of expectation to realization. ATT can be specified as a direction symbol. l.f specified It enables VI-SYS to move through any encountered vertex which Is straight (has 180 degree angle) on the ATT side of the line being traversed.

DIR (DIRection) specifies the direction of the hole movement to be effected by VI-SYS. RAN <RANgel specifies the range of that move. RAN Is a va I ue returned by VI -SYS In VI PS (hoI e movement>; during saccadic <ballistic) eye movement behavior, It may be pre-specified by AA-SYS. The values of these cells are Incorporated Into the Image by AA-SYS. This reflects the Intrinsic role of motor <efferent> activity and symbols · In visual perception and Imagery.

Short Term Memory <STM> consists of an ordered list of nine chunks, being the limited amount of active memory available to the percep­ tual (cognitive) system for Image construction Q2~ An STM chunk Is an Image chunk or an Image chunk with a Special type element <GOL, LAST, COM, or OGOL> appended. STM chunks are accessed according to Image Body element containment of Special element appendment. STM Is the memory In which the visual Image (perception) Is con­ structed. STM and VR are the active memory com­ ponents or "mind's eye" during visual form percep­ tion. During recall and drawing Involving Imag­ ery, only STM Is active as the "mind's eye".

There are three components of long-term memory In VIPS. LTM Is the universe of symbols used In VIPS. The symbols are Interrelated, forming a semantic network of symbols and rela­ tions. The use of any symbol In an active memory or by a process rule Is an activation of that LTM element. (In the Implementation In LISP 1.6, It Is a pointer to that element.) As such, any active memory entity <VR cell or STM chunk) Is only pseudo self-contained. A symbol Instance Implies the possible use by the active process of all the symbol's LTM relations and related sym­ bols. A symbol remains effectively Imbedded In the LTM structure when activated.

RLTM (Recognition Long Term Memory) Is a mod­ Ified n-any discrimination net. The discrimina­ tion structure Is determined by angle symbols and the objects which angle configuration determine. The discrimination net Is not used to directly guide perceotual activity. Rather, It Is used to name completed object Images (If possible) or to activate an aval !able Image generating process to complete a hypothesled object Image when adequate partial Information Is available. RLTM Is acces­ sible only by REC-SYS and Is therefore only tra­ versed during REC-SYS activation.

ITM (Intermediate Term Memory) Is the memory Into which selected STM chunks are Incorporated. Meaningful confirmed results of the perceptual activity are transferred to this memory, as the current meaningful processing context. Its con­ tents are recallable Into STM for use by the per­ ceptual process. This recal I requires a ful I system cycle thus the contents are not Immediate­ ly accesslbie. As a chunk Is Incorporated Into ITM new relational properties aremded to Its Chunk Header element. These Indicate the chunk's Immediate lmeglnal context, being the Image chunks to which Its Image Body elements are linked. Tem­ poral relations between OBJECT chunks are added also. Thus, the final perceptual lamge, which Is found In ITM, Is a heterarchlcal symbol structure.

The Processes

All four processes are Implemented In the form of production systems(l)). A production system consists of an ordered list of condition­ action pairs. The system Is cyclic In operation. With each cycle, the first rule of the active process which has Its condition-half satisfied Is said to "fire", resulting In Its action-half being executed. Rule firings depend upon and can alter only active memory contents. This reflects the contextual nature of human cognition.

The Asslmllatlon-Accomodatlon process CAA-SY~ Is named for the two basic classes of behavior associated with It (14). Upon VI-SYS deactivation, this process either can assimilate the new con­ tents of VR Into the Image .In STM or must accomo­ date that current Image In light of the conflict­ Ing contents of VR. AA-SYS Is the "main" pro­ cess. It Is the source of most goals. It sets VR and activates VI-SYS so to access new PIC con­ tents. It activates REC-SYS to aid In Its Image construction activity. It activates INC-SYS to Incorporate satlsfactorl ly confirmed, meaningful segments Cchunks) of the Image Into ITM.

As the primary source of current goals and their transitions, AA-SYS embodies perceptual strategy In the VIPS Implementation. Two overall strategies have been Inferred from the protocol data, resulting In two corresponding AA-SYS Im­ plementations. One strategy Is that of success­ Ively recognizing objects untl I having represented the whole line drawing. The other Is that of first attempting to scan and represent the entire line drawing outline and then linking up unknown Inward directed exits. The second strategy di­ verts to the first In a number of specified cir­ cumstances.

Each rule of AA-SYS, REC-SYS, and INC-SYS has the name of the current goal as the primary consideration of Its condition-half. Visual form perception Is a goal directed activity which con­ sists of a sequence of goal-related episodes. Verbalizations and repeated patterns of hole move­ ments were Indications of this that can be seen In the protocols. The current goal Is a primary de­ terminant of what Image chunks are attended, what external Information Is to be accessed (fixated) next, and Into what class of lmaqe structures the new Information wl II be Integrated and by which It wl I I then be represented. Figure 5 Is a goal episode chart of the goal sequence as Inferred for the protocol segment of Figure I. Goals to the right Cln brackets> of a protocot frame number are active at (during) that "hole fixation". A goal to the left of a vertical line Is active during the hole movement sequence occurring be­ tween the protocol frames Indicated. The resul­ tant perception Is the product of the Interaction Cor Interference) between cognitive goals, partial perceptions, and visual environment.

Figure 5

The output produced by the VIPS production systems Is a trace of rule firings and resultant active memory contents. Figure 6 presents the VIPS trace corresponding to the human behavior transcribed as frames VI and V2 of Figure I. VI has been set to the Initial vertex and STM Is empty. Rule RAAO of AA-SYS fires, asslml latlng the Icon Into STM as a nee Image chunk. The goal RNO Is generated by the rule, and a new chunk Is created, marked by the special element GOL. This new chunk Is to be the sight of object Image con­ struction during the ensuing REC-SYS activation. Rule RECO of REC-SYS next fires and traverses the vertex chunk In STM (marked by LAST>, begInnIng construction of the new object accordingly. RECI next flres ·as there Is no further relevant Image Information In STM. The UQX property of the node currently referenced In RLTM activates a process to complete a square-or-rectangle Image, as Is suggested by the right angle In STM. REC-SYS then deactivdtes, control returning to AA-SYS. Rule SK021 fires, using the new object Image to specify the contents of the visual register be­ fore activating VI-SYS. Rule Vl9 fires, moving the CCP "right" to the feature list for vertex 3.

Th8 rule constructs the Icon, provides range In­ formation, end returns NO In EXP.

triangle Image and Its Immediate lmege environment are Incorporated Into ITM. In general, VIPS accounts for over 80 percent of the hole move­ ments. The system consistently has sufficient Image chunks In STM to be a source of the observed verbalizations.

(LAST V3(CR X7 PNUU P4 VSPEC (RTA 01.) sPi:C V2 TYP£ VERTEX)· ~ ~ I ~ ~

THE INITIAL FIXATION AND EYE MOVEMENTS

The behavior which wes observed and recorded as detaIn Experiment II has been considered only at a general, descriptive level. No perceptual system has yet been Implemented which produces corresponding behavior and which could then serve as an operative theoretical explanation of the behavior. This general consideration of the data has been favorable to the proposition that VIPS can be transformed Into a sufficient theory of that behavior.

The verbal descriptions and partial drawings obtained as data In Part I of Experiment II pro­ vide the basis for lnferrlno the state of the "mind's eye" following processing of the Initial fixation by the perceotual system. The data Indi­ cate that the Initial fixation serves the percep­ tual system as the source (basis) of Inferences and hypotheses. General characteristics of the total extent of the line drawing are consistently inferred, thus determining (delimiting) the cur­ rent area of perceotual Interest. Object hypoth­ eses are generated for rftglons (areas bounded by spatially disjoint features) within that extent of the visual field. Such processing of the Initial fixation Is consistent with an active, assiml latlon-accomodatlon theorv of visual form perceotlon as is embodied by VIPS.

Figure 7 presents plctoral representations of the Image chunks created by VIPS In Its account of the remainder of the protocol section of Figure I. The NO In EXP causes object Image accomodatlon. The new vertex Is assimilated as a chunk, the square Image Is altered to Include only the right and acute angles. REC-SYS Is reactivated with goal RKI and e triangle Image Is completed, as suggested by the partial object Image In STM. As ATT will be specified as the direction to the In­ s I de of the tr I an I e, VI -SYS "moves" through ver­ tex 8 during Its ensulnq activation. Since the new vertex Is not simple, It Is asslml lated Into STM and linked by equivalency to the triangle. A SIDE chunk Is created with a OUICKSEE element Included, which notes only the number of exits In the by-passed vertex. Vertex I Is next re­ encountered In the system's attempt to confirm the triangle Image. The vertex agrees with that expected so It Is not asslml fated Into STM, but Is "seen" as part of the now completely confirmed triangle Image. Being completely confirmed, the Much of the initial processing is based upon visual information lying in the periphery of the visual field. This input must be realized by processes of peripheral vision. The data Indicate that peripheral Information at eccentricities of up to twenty degrees of visual angle contributed to the Initial Inferences and hypotheses. Some object hypotheses were based upon visual Informa­ tion which must have been entirely peripheral. Though a sufficient basis for object hypotheses, peripheral vision consistently tal led to allow subjects to correctly Infer the particulars of object Interactions.

The hole movement task masked peripheral vision Information. Therefore, the representation and uti llzatlon of peripheral vision's input are not part of VIPS and are most basic and necessary extensions to the system. A class of elements which embody the position of Irregularities or discontinuities In the periphery shows promise as a possible symbolic representation. These ele­ ments would occur In VI In addition to the fove­ ally produced feature symbols. The position In­ formation would necessarily be retlnally-based <relative In direction and distance to the current fixation point>. Local differencing operators have been shown to be effective as a means of de­ tectlnq and locating discontinuities 0.5). The operators are psychologically and physiologically feasible and readily lmplementable. These ele­ ments appear to be an adequate supplement to foveal (feature> elements as bases for the observ­ ed Inferences, hypotheses, and eye movement be­ havior.

The recorded eye movement sequences obtained in Experiment II have been considered In terms of fixation durations and locations. Fixation dura­ tions varied from approximately one-eighth of a second to over a second. The variation in fixa­ tion length favors an active, asslmllation-accomo­ datlon theory of visual form perception which pre­ dicts differing amounts of Information processing per fixation. The Initial fixation was signifi­ cantly longer in duration than those Immediately following it, thus upholdln~ Its role as a highly processed source of inferences and hypotheses. In general, the eye movement sequences consisted of an initial, object-related, overall scan of the line drawing, perceptual construction relying heavl ly upon peripheral input, followed by a rescan which concentrated primarily upon object interaction areas.

For a specific example, consider the eye movement protocol of Figure 2 and Figure e, which Is the resultant verbal description provided by the subject. Part "a" of the verbal description appears to ·be a conclusion resulting from the initial segment of the protocol (that which is connected by dots>. The subject's attention is primarily focused upon the ambiguous <In depth) areas of the line drawing. The remaining sequence of eye fixations indicate that the subject scans the bottom cube and finishes with a long fixation between the remaining faces. One can infer from this sequence the construction of an image suffi­ cient to serve as basis for part "b" of the given description.

that was one of those screwy objects again

• (a) it was a

it was attempting to be two cubes on top of one another

In order to be able to draw it I said there was a normal cube on the bottom where you can see the front

• (b) front right hand side and top there were two like sides

then there was a wall sitting on top of

two I I ke wa I I s the cubes

Figure 8

TOWARD COMPUTER VISION

VIPS Is meant to be an operational theory of human visual form perception, but It has a natur­ al relevance to machine vision research. Three genera I character I s.t I cs of VIPS are app II cab I e, and lndlspenslble I believe, to the eventual re­ alization of an adequate computer vision system. One Is the system architecture of VIPS as Illus­ trated in Figure 3. It provides a basis for the appropriate modularlzatlon of memory and process involved in perceptual activity. Memories are required (I) to hold the results of the direct analysis of environmental Input, (2) In which to construct the perceptual Image, <3> to hold con­ firmed partial and the eventual complete percep­ tions, (4) to hold applicable recognition <and general) knowledge. Processes are needed (I) to analyze environmental input (i.e., perceptron and gradient operators), (2) to construct perceptual images and select environmental areas for analyt­ Ic attention, (3) to incorporate confirmed par­ tial perceptions Into an overal I representation.

The second characteristic Is the sequential, localized, 1n-depth processing of environmental Information and the cyclic asslmllation-accomoda­ tlon of the symbolic representation being con­ structed In light of that processing's results. Early attempts at computer vision found that the information load associated with the storing and processing of a whole scene at one time was as­ tronomical. Since than a main theme of vision research has necessarily been to reduce internal information flow. What VIPS and other eye move­ ment research Indicates Is that the human vision system faces the same problem and that It ap­ proaches (not always solves) the problem by the selective, in-depth processing (fixation> of local areas within the visual environment. The non-fixated region of the visual environment Is temporarily given only minor attention and sparse representation. Perception is then accomplished by the sequential application of the In-depth process. With each new fixation, new Information Is incorporated into the perception under con­ struction. The location of the next fixation is dependent upon the perceptual state following this incorporation. The perceptual state consists of the partiaf perception in active memory and the · current goa I ·

The third applicable characteristic of VIPS is the goal determined nature of its activity. The current goal plays a role in limiting infor­ mation flow by influencinq what is fixated, how It Is represented, and the determination as to when the perceptual construction Is sufficient, thus concluslng perceptual activity. Data from visual search tasks II fustrate the effects of goals upon the location of fixations during stim­ ulus scanning C16,17>. Thft now cl~sslc ~eta nre­ sented by Yarbus ( 18 > Indicate clearly the effects that question-answering Clnformatlonal> goals have upon the fixation of a presented vis­ ual scene. Other research In cognitive psychoi09Y has consistently indicated the selectivity of Information representation based upon task-rela­ ted qoals C19).

What this should tell researchers in compu­ ter vision Is that determining one perceptual strategy and one representational scheme Is not the proper target. An adequate vision system will require the availability of several strategy/ representation sub-system. Each can then be pro­ perly applied In the different goal-related situ­ ations which the associated "seelnq organism" w I I I fInd I tse If . There Is never "correct per­ ception", only "suff lclent perception" for the task at hand.

CONCLUSION

VIPS has been Implemented to directly account for four protocols. An evaluation of protocol-program trace correspondence Is favor­ able to the perceptual theory embodied In the system. The element classes of the Image repre­ sentation are shown to form a basis for Image generalization. Extensions are discussed which transform the system Into a more Inclusive per­ ceptual theory. A comparison of studies yields the proposal that human Information representa­ tion In active memory If flexible, being as suited to the task as possible In light of the existing symbols, relations, and structure of an evolving lonq term memory upon which all cogni­ tion Is based.

The research which has been reported here has resulted In a speculative, theoretical spec­ Ification of the constructive theory of visual form perception. It Is not meant to be a con­ clusion and so this report likewise does not really have one. As an Initial Implementation, VIPS can now have modifications and extensions applied Csome are discussed above> so to better embody the perceptual activity It Is meant to explain. Being a theoretical statement, It can serve as a source of experimental questions, the resulting experimental findings being the basis for subsequent specification Improvement. Fin­ ally, the model also embodies characteristics relevant to the development of a generalized machine vision system.

ABSTRACT

A computer Implementation of the construc­ tive theory of visual form perception and Imagery Is described. An II lustratlve example of the system In action and a discussion of Its corres­ pondence to human behavior are presented. Exten­ sions to the Implementation are discussed. As­ pects of the model are presented as being rele­ vant to machine vision research.