An Empirical Method that Provides a Basis for the Organization of Relaxation Labeling Processes for Vision

1. RELAXATION LABELING

Relaxation labeling is a computer vision technique that corresponds closely to neural networks in the human brain [1,2,3,4]. It is a powerful technique whereby parallel, interacting processes may be uniformly represented at many levels while incorporating context in a natural way. In designing these networks for higher-level features the choice of labels and interactions between labels is not always clear.

Neurophysiology has thus far fallen short of clarifying the operation of higher-level symbolic processes in human vision. Single cell studies do not address the ·problems of distributed processing; information .from electrically evoked potentials is at too large a scale to address issues of feature representation within layers. The psychophysical method developed by Weisstein et al. [5,6,7,8] has the advantage of capturing aspects of higher-level processes while maintaining plausible links to the underlying neurophysiological structure. It makes the connection between the logical representation of visual features in a scene and the spatio-temporal organization of the brain, and thereby . provides a basis for design specifications of relaxation labeling processes more complex than simple bar detectors.

The relaxation labeling process [4] is the iterative, parallel computation of a consistent array of location-specific labels representing features in the visual field. Compatibility functions specify which local features are most consistent with which neighboring features. For example, if we want to enhance straight line contours, we choose labels that represent straight line segments from a range of orientations and choose compatibility functions that strengthen neighboring labels of similar orientations and weaken labels of other orientations. In a higher-level system designed for office scenes [9], the compatibility functions strengthen homogeneous labeling (wall next to wall) and probable occurrences (wall next to door), and weaken improbable ones (picture next to floor). However, the system is far from being a general-purpose vision system in that the number of possible labels for surfaces is far too limited. A more general system would employ multiple levels of representation with interactions both within levels and acrose levels of the hierarchy [10].

*This work was supported in part by the Office of Energy Research of the u.s. Department of Energy under contract No. W-7405-ENG-48.

Weisstein's method described here has been designed to map out the spatio-temporal scope and strength of interactions between processes that detect specific features of the visual input [7,8]. Characteristics of the response functions obtained indicate the types of processes involved, what they represent, when they occur, and how they interact, thus narrowing the choices for possible labels and compatibility functions.

2. THE~

Two brief (10 msec.) visual stimuli, such as line segments, separated by a brief time interval are presented to a subject. The delay between the first stimulus (the target) and the second (the mask) is varied while some measure of visibility is taken [11]. Plotting the accuracy of response against delay yields a time function of the influence of the mask on the perception of the target. (See figure 1.) Usually, a minimum in this function occurs at a delay indicating the interval at which maximum masking occurs. For some stimuli enhancement rather than masking takes place. Both target and mask set up waveforms somewhere in the visual system that represent the processes enabled by their presentation. Each of these processes reaches some maximum and then decays to zero. Maximum negative interaction between target and mask processes occurs at the time interval that causes the peaks in their processing to coincide · Surprisingly, maximum masking generally occurs at a non-zero delay [12], implying that the time courses of the two processes differ. (See figure 2.)

In addition to time interactions, spatial interactions can be measured by varying spatial separation instead of delay [12]. In this way, contrast enhancement curves corresponding to lateral inhibition have been mapped out by Growney and Weisstein [13]. Figure 3 shows some preliminary experimental results which correspond to heuristic compatibility functions chosen by Zucker, Hummel, and Rosenfeld [14]. Line segments in the context of relatively straight contours are enhanced, while segments in the context of more curved or non-continuing contours are suppressed. Thus, the strength of spatial interactions between stimuli provides an experimental basis for the design of compatibility functions. The scope of interactions and the size of features in the input can be used to infer either the scale at which local interactions are taking place or the spatial scope and structure of compatibility functions. The choice of labels for nodes in the network can be narrowed down by a classification of features in the stimuli based on the characteristic masking functions they produce. For example, a number of masking stimuli whi~h appear to be three-dimensional show enhancement at zero delay, while masks with absolutely no three-dimensional cues never show enhancement (Figure 1) [8, 11]. Stimuli with fewer free endpoints and with less overall curvature bet~en connected segments have a minimum at about 90 msec., while totally unconnected stimuli show a minimum at 40 msec. Key structural features of masks can be varied gradually in order to ascertain exactly which features make up the subjective impression of three-dimensionality and connectedness. For example, three-dimensionality may be a function of the number of T-junctions in the scene, support cues, perspective lines, or apparent solidity. We can vary detailed structural features of the masking context in order to isolate the primitives that make up these higher-level global features. Of course, we would expect the relaxation labeling process to occur at many levels of representation. However, the spatial scope of a feature and its interactions gives some indication of where it is represented in the hierarchy.

3. CONCLUSION

We have described a psychophysical method that connects two areas of vision: one which addresses issues of symbolic representation, and one which deals with their underlying implementation in the brain. The sensory underpinnings to this method correspond closely t o relaxation labeling processes from computer vision. We have a method for mapping out and characterizing symbolic processes at many levels of representation and measuring their interaction in space and time, for one general-purpose working system, the human brain. The measures of interactions between symbolic processes provide a basis for the design of compatibility functions in the relaxation labeling paradigm.