A Simple Method for Recovering a Relative Depth Map in the Case of a Translating Sensor

A SIMPLE METHOD FOR RECOVERING RELATIVE DEPTH MAP IN THE CASE OF A TRANSLATING SENSOR

K. Prazdny

Computer Vision Laboratory Computer Science Center University of Maryland College Park, MD 20742

USA

on PP and the focus of expansion (FOE) [the unique point of intersection of all these straight lines]. FOE corresponds to the point where the vector along which the observer translates pierces the PP. In the case of pure translation the directions of the image velocities are independent of the sur­ face layout; the information about the spatial dis­ position of the texture elements is contained only in the magnitudes of the image motions. Because each image element moves along a known straight line on PP, the problem reduces to estimating the motion of a one dimensional brightness distribu­ tion along the line (it is assumed here that the distribution does not change significantly from frame to frame). This problem is easily solvable. Using the Taylor series expansion one can develop an expression for the image displacement 6i rela­ ting the temporal and spatial derivative of the image brightness function along the line i at a given image point (see e.g. [6]).

where I is the image brightness function, and 6i is its displacement at an image point along the line i. With a sufficiently small interframe in­ terval, 6I·di/dt, and 6~·di/dt. If one disregards the components of order 2 and higher, this approxi­ mation reduces to a linear approximation used e.g. in [1],(2],[3],[4). Observe that the linear ap­ proximation is appropriate only at points where the first spatial derivative along i is (approxi­ mately) constant and the displacement remains with­ in the extent of this linearity. The speed di/dt is related to the angular speed da/dt by

where Q is the distance of the image element on PP from the center of projection, and A is the angle between the line ~ on PP and the projection ray (Figure 1). Q and A are constants for a given FOE and reflect the fact that the projection surface is a plane which is not isotropic with respect to the image velocities. The above equations enable one to compute the relative depth directly. The derivatives di/dt, and di/d~ have to be estimated from a succession of digital images. The deriya~ tive di/dt·i can be approximated directly from successive (temporally proximal) digital images. The derivative dl/d~ of I along i is easily ob­ tainable from neighboring gray level values by interpolation. One has to assure the e~istence of both derivatives at locations where B is to be

ABSTRACT

When an optical sensor translates along a known direction in a stationary world the relative depth map of object points on the surface can be easily obtained when some simplifying assumptions are made. Under these assumptions the angular speeds of projecting rays can be obtained from (approximations to) temporal and spatial derivatives of image brightness. The relative depth of any two image points is then directly computable as a simple ratio of linear functions of their angular veloci­ ties because the directions of angular velocities are known in this special situation.

Keywords: Vision, image velocities, relative depth.

Although optical flows were shown to be a rich source of information, their detection poses a pro­ blem: in general, it is difficult to obtain image velocities directly from image brightness. Here we investigate a special situation where restricting assumptions are incorporated to compute the rela­ tive depth map from image motions. Prazdny (5] has shown that given two image points corresponding to two texture elements on a rigid object (or on two different objects moving rigidly togehter), their relative depth, Rdi 1 , defined. as the ratio of their distances (Si,S 1 ) Eo the center of (polar) projec­ tion, is determtned by

where A ·A -A is the difference of the angular velo£i~!jve!t;ts of the projecting rays of light, and Qi,Q 1 are the unit vector specifying the visual directions along which the two i~ge points lie. This is a general relation which does not depend on any restriction on (relative) motion. In the spe­ cial case when the observer translates along a known (but arbitrary) direction, the direction vectors of angular velocities ~ are known. In fact,

so t~at Ai·(vxQ 1 )/sin(ai) is the unit vector of~· and Bi is the angular speed of the ray (the mag­ nitude of the angular velocity vector (see Figure 1). In order to apply equation (1) to recover the rela­ tive depth we have to specify Bi.

In pure translation, the image elements on the projection plane (PP) move all along straight lines determined only by the position of the image points computed, i.e. the image brightness function has to be differentiable at all such points. To do this (and to suppress the noise), the individual image is low-pass filtered before estimating the derivatives. This operation virtual ly guarantees the differentiability of the image brightness {unc­ tion nearly everywhere. The requirement on the smoothness of the image brightness function is the strength and the weakness of the method (see also [6]). On the one hand, it enables one to compute image motion from equation (3). On the other hand, it is limited to areas of large brightness changes. Observe that the image motions resolvable by the method must be small with respec t to the (local) variations in the image intensity. This requirement will be satisfied near places where the gradients are large in magnitude and extent (typically, the edges).

One appealing aspect of the relative depth map as a representational structure i s its poten­ tial redundancy. For a given image point, we can compute a set of relative depth values pairwise with other image points. While these values can­ not be directly compared, the absolute depth of a given point has to be unique. Setting the "ab­ solute" depth of an arbitrary point to an (arbi­ trary) value and propagating we obtain many inde­ pendent estimates of "absolute" depth of a given point. These values can be treated as noisy esti­ mates of the true values. One can thus devise an iterative network which not only suppresses the noise but also incorporates the constraint on smoothness of the "absolute" depth values to achieve a (global) consistency.

DISCUSSION AND CONCLUSION

The present scheme is based on the assumption that the observer purely translates and that FOE is known . This restriction reduces a difficult (and still unsolved) 2D problem to an easy lD problem. The case of pure translation is impor t ant in many indust rial applications where the camera motion can be restricted and the orientation of the optical axis of the camera with respect to some fixed vec­ tor can be determined with high precision. The method presented here should be reasonably success­ ful in determining the relative depth between the places of large brightness variation. It is not intended for accurate computations of loca l sur­ face orientation. Rather, it is expected to be useful in determining the approximate relative positions of objects (e.g. for hand-eye coordina­ tion), and in determining the occlusion relations. The method is currently being tested (using a TV camera) on real world data. It is expected that it will eventually perform in real time when implemented on dedicated hardware.