On A Possible Mechanism Of Binocular Fusion

Introduction

It is assumed that corresponding sites of pair retinal images have to be found in order to provide the binocular fusion (I). This procedure seems to be rather complicated for real visual scenes (2). On the other hand soae results of the biological experiaents show that bi­ nocular fusion and stereopsis are possib­ le when corresponding details of pair images have dissimilar shape (I,3,4,5). This suggest that the human binocular fu­ sion mechanism is not based on correspon­ ding sites search.

In this paper a possible binocular fusion mechanism using a continious esti­ mation of doubling degree of image (which is formed in visual cortex by superposi­ tion of retinal projectiohs) in the cour­ se of vergence is described.

Some information processing stages rn binocular vision prece41#i obJect recoiijition

Data available allow to distinguish the following stages of complex spatial scene analysis (i.e. when a number of ob­ jects overlapping in the visual field are situated in front, middle and rear plan).

I~>coarse binocular fusion (plan extrac­ tion In depth)

Presumably at first the extraction of the most distinct regions in pair re­ tinal images of similar color or bright­ ness is carried out. ~en horizontal di­ sparity of these regions (by their cent­ res or edges) is calculated. J.fter this a divergence or convergence by corres­ ponding angle is carried out. The parts of scene located far further or nearer than plan extracted are ignored in the

x)All fUrther considerations are concer­ ned with the part of scene located in the region of central vision.

~her analysis.

2. Extraction of "subjective contours"x)

J.fter "plan extraction from the back­ ground"(image is obtained which may con­ tain several overlapping projections of objects located in the plan extracted), the boundaries among regions of different color, brightness and texture are drawn.

3. Accurate binocular fusion

In the course of fUrther vergence the doubling degree of a part of image obtained is estimated using the extracted "subjective contours". When some marginal doubling degree is attained a coamand for the eye stop is produced.

4. StereoE!is and extraction of "object f'rom the ckground"

Using the fused part as "zero point" the disparities for the other image poi­ nts are determined. Disparity distributi­ on is evaluated then and a decieion on possible 3-diaenaional object shape (or its parts) is made. If the discontinuity of disparity occurs, e.g. when a number of rigid objects is closely located in the plan extracted, some compact (with respect to disparity) set of points (an object) is extracted other being ignored in the further analysis.

The data which support a possibility of existence of these stages in binocular vision are as followsa

• (I 1 3,4,5)J
• depth perception without "subjective contours" (using disparities of dis­ tinctive regions in the stereograms)
• ex stence of coarse (gualitative) and p~ecise stereopsis (6)J inhibitor,r interaction between infor­ mation processing channels of crossed and uncrossed disparity (corresponding to different scene plans) (?)a
• reduction of "Mach bands effect" resul­ ting froa the "staircase" arrangment of test bands (in depth) (8)a
• preceding of "subjective contours" ex­ traction by the difference in color, luminance or texture to a stereopsis (I,4,9h
• suppression of untused parts of stere­ opairs in "binocular rivalry" phenome­ non (when they differ in shape signi­ ficantly) (Ih

x)The 2-nd stage is related to monqcular vision and can 1n principle start simul­ taneously with the I-st stage.

• using the "narrow slit" detectors (in­ stead of the "edge" detectors) during determination of disparity in vision (IO);
• preceding of stereopsis to a recogni­ tion (I,II).

In the following only the 3-rd stage of the process will be coAsidered, name­ ll, a mechanism of precise binocular fu­ s on. Assume that the I-st and the 2-nd stages have been accomplished.

A ~ossible mechanism 0 bfiiocular ruslon

When the vergence eye movements in the course of binocular fusion occur, cortical projections from right and left retina are shifted with respect to each other. Assume that a set ot "two ~llel lines" detectors and "single line detec­ tors with various orientation and ~;an­ ce between lines in optimal stimulus is available in visual cortex. Then the mo­ ment of fusion approaching the set of "double lines" detectors will respond with the distance between the line~ con­ tiniously decreasing. The response of so­ me "marginal set" of the detectors could be a signal tor the eye to stop. The ur­ ginal distance between the lines depend on eye movement fusion velocity and dura­ tion of earring out a stop-command. In the moment the eyes are fixed, the resul­ ting cortical image appears to be fused. A small number of responding "marginal double lines" and a large number of"sin­ gle lines" detectors responding will also correspond to this moment. A permanent evaluation of cortical image doubling de­ gree during the vergence leads to avoi­ ding the "eye overshoot" in moment of fusion.

A digital simulation of binocular fUsion mechanism

The simulation was carried out on a digital computer. Input images were ste­ reopairs of 25x25 (Pig. Ia) which repre­ sented the right and lett projections of front 0 view of a oube (subtending an angle of IS ). Initially the stereopairs have been shifted, imitating the "eye fixati­ on" being not in the cube plan. On each step computer shifted one of the stereo­ pairs by one ceil and superimposed both stereopairs which imitated the cortical projections shift during the vergence. Then the number of double and single li­ nes in resulting image was calculated. This operation was carried out by means of a set of local operators (5x5 in size) which extracted the segments of single and double lines of various orientati ons in resulting image (Pig. Ib). At each step the ratio of double lines available to single lines was calculated. The re­ lative stereopairs position in which this ratio was maximal was searched (Pig. I c). Prom this position the stereopair was shifted by two cells and the resul­ ting superimposed image was printed out (Pig. Id).

!)It is "cortical image" obtained by im­ posing the retinal projections which bea­ rs in mind.

As Fig. Id shows, stereopairs being projections of 3-diaensional object can not be completly fused in none of their relative positions. Simulation results however showed that computer correctly finds the relative position which corre­ sponds to the maximal fusion (in· our ca­ se - to the front edge cube fusion).

Conclusion

The ~ocular fusion mechanism under discussion~does not require search of co­ rresponding points in pair retinal ima­ ges. However it assuaes that "subjective contours" are initially enracted by co­ lor, luminance and texture differences. Further attempts should be directed to­ ward the study of both subjective conto­ urs extraction mechanism (particularly by texture differences) and stereopsis mechanism (which is carried out after bi­ nocular fusion of a part of iaage). The suggested visual system ability to ext­ ract "object from the background" biil.o­ cularly could be taken into considerati­ on in robot visual system design.

x)New data which support the hypothesis is obtained recently. Superiority of do­ uble parallel lines detectors with ~le distance between the lines about 0,5 was found in the visual cortex of a cat (I2) (thus indicating the existence of a certain "~~arginal detectors set") ·