Inferring Motion of Cylindrical Object From Shape Information

1. INTilODUCTION

This paper1 explores methods for inferring motion of a cylindrical object, one , of the most familiar shape

classes' for us, from an .image! sequence . .

Recent studies on computer vision have exploited methods for · interpreting changes in an iimage sequence as motion in 3-D space. Most methods find correspondence of feature point!! between frames and reconstruct J~D structure of rigid objects (Ullman, 1979)(Nagel, 1981) . The correspondence ·problem, however, · is difficult especially for curved objects . if their surfaces are without any markings.

We investigate imethods for inferring motion · of cylindrical object from two approaches. One is to analyze shading information to obtain the normals of surfaces which · are useful fo.r; finding torrespondecne ·of base planes.' The . local shading analysis (Pentland, 1982) is riot applic~ble to estimate a normal at· a point on a cylindrical surface· if ~ lighting lconditionsi are unknown! With knowledge of the lighting conditions, ·we can determine motion between frames by :an extended reflectance map method.

Another approa~h to .inferring of motion iS to use shape information. The method also needs an additional constraint. A trivial case is ' that two planes of the object are not parallel. We can easily determine correspondence of three points on the object ·between frames.

correspondence of any point oh the plane except its

Difficulties in -finding of; correspondence. appear when the two planes are parallel. Since we do not know orientation 1 of the base plane, I it is difficult to find

INFERRING MOTION OF CYLINDRICAL OBJECIT FROM Sl-IAPE INFORMATION

Minoru Asada an {I Saburo 1'suji1 Department of Control Engineering Osakil Univeristy

Toyoriaka,·· Osaka 560, Japan

centroid if the boundary is smooth. We introduce ·an assumption . that the two planes are :orthogonal, to the generating Hnes, because many man-made objects satisfy this condition. iThis strongly constrains the surface normals of the object, but itlis not sufficient; to determine the normals . The orthogonality, . however, gives another constraint between a change ·in area of the base plane and that in length of generating line from the. first to the second frames . Thus, we can estimate orientations of the base plane i'n the two frames, and the motion parameters are determinable. The method is ·robust , because· reliable parameters such as areas of base plane and lengths and directions of generating line in the two ·frames are used:

2. nASIC ASSUMPTIONS

Computational analyses in this· paper assume the following four conditions.

(1) The projection from scene to image is orthographic.

(3) TM object has a cylindrical and planar surfaces.

(2) The scene contains a moving rigid object :·

Here, a cylindrical surface is defined as a surface' covered with parallel lines (generating lines) i passing through a closed turve,

We can segment an image projected from Ia scene containing unknown ' shaped objects into spherical, cylindrical , and planar· surfaces if ( 4) is assumed.1That is, we can, test ,whether (3) is true or not.1 ( 4) The surface is Lambertian and of a constant albedo. The distribution :of illumidanation is constant (a distant point source illumidant) .

3. SHADING ANALYSIS

Pentland presented an interesting computational analysis of local !shading without knowledge of lighting conditions (Pentland, 1982). It gives an estimate . of surface· orientation for· an umbrical (with equal principal curvatures) point on a Lambertian surface, and also identifies whethet the ,surface is planar, singly or doubly curved at each point.

We use this algorithm for segmentation of image to find each surface1 and its property; spherical, cylindriCal , or planar one . Figures 1 shoW examples of ·input ' images (256 by 256 8bit digital images of a moving cylindrical object synthesized by cbmputer) and Figure 2 is the result of an<l'lysis of Figure 1 (a) . From this picture' we get information that the object has a :cylindrical and a planar surfaces. Also anothel' important feature of cylindrical surface\ a direction of generating lines in image, 1 is

obtained.

If the orientation of the base plane is known, we can estimate the shape of the base by mapping its contour in image onto a 1Hane1 parallel td the base. rrhus, 1 we can find correspondence oft. each point on the boundary of the base between consecutive f.tiames from which motion . in 3-D space is determinable .

If analysis ·of shading can determine the normals of cylindrical surfaces, the~ constrain the orientation of the base plane. The local shading analysis, however, cannot estimate surface normal of a point ori a cylindrital surface' without knowledge 1 of lighting conditions. The detailed discussions appear ~n a companion paper1 (Asada and Tsuji, 1983) .

If we know three parameters of illtiminant (albedo times illuminant intensity and two parameters ~f illumirtant direction), the surface normal IS determinable by the local shading analysis. An alternative approach is possible unden the~e conditions; an extension of the reflectance map techmque proposed by Horn (Horn, 1977). Now we examine relations between orientations of cylindrical surfaces and the base plane The analysis in the gradient . space for investigating orientations of three planes wluch meet at a Fork joint is also useful to our case. Figures 3 show an image of cylindrical object and the gradient s~ace representing orientations of surface patches of the obJect. Since we have found generating lines, the cyliindriCal surface in image is segmented into surface strips containing each generating line. Let G, be a point in p-q plane representing the gr~dient of base I?lan~. We select two arbitrary surface stnps on the cyhndncal l surface and denote corresponding po.ints in p-q plane as 1G2 and G3 · 'J;hese three points are constrained by directions o.f three lines; a generating line ahd two tangents of the base boundary at the cross points of the selected' surface strips. Thus, we need to know three unknowns, ~he location and size of the triangle G,G2G3, to determwe these surface normals. By utilizing intensity values of the surface strips and the base, these normals are determinable (Horn, 1977). The computation, however, is rather complex, because we need to find locations and sizes of triangles of which vertices are exactly located 'on conic curves specified by reflectances of surfaces and the lighting conditions.

We propose a much simpler method to determine the orientations. First, the brightest generating line in the cylindrical surface is found and its intensity is measured. Its gradient is determined in lp-q plane as points of contacts of lines with a conic' curve spec!fied.by the intensity value in the reflectance map. ifhe duectton of the tangents are gi Yen as orthogonal to the generating lines. Therefore, the gradient of the base plane · is obtained as cross points of a conic curve with the satne reflectance as base and lines from the gradients of the brightest generating lines in p-q space. The lines are orthogonal to the base boundary at the brightest generating line iri the image plane.

The companion paper (Asada and Tsuji, 1983) shows how we can estimate albedo times illuminant intensity if the illuminant direction is known. This paper does not exploit the shading analysis furt?er. ~he following section discusses on methods for mfernng motion from shape information; the direction of generating line and the boundary of base in the image.t'

4. INFEIUUNG MOTION FROM SHAPE INFOilMA TION

At first we consider a simple case for. which correspondece of points between frames is easily established. If two planes of cylinder are not parallel, we can find the longest (or shortest) generating line from the image. Thus, we can establish correspondence~ of three points, two end points of the longest generating line and the centroid of a lvisible plane, between consective frames, and: the motion of object in 3-D space is determinable from three views ,(Nagel, 1931).

Now, we deal with more complex case; the two planes are parallel and finding of correspondence is difficult . As described in the previous section three unknowns are undetermined without knowing the lighting conditions. Therefore, we need an additional constraint to estimate the orientations. Let introduce an additional assumption that the planes are orthogonal to generating lines. The orthogonality gives two constraints, thus one unknown is left undermined. The analysis in the gradient space becomes very simple (see Figure 4). A point representing the orientation of the base plane in p-q plane is on a line passing through the origin with the same slope of the generating line in the image plane . The undetermined unknown is the slant of the base plane (or a generating line) to the image plane. If we know the slant, the 3-D geometry of the object is completly determinable.

Note that the additional strong constraints' are not sufficient to determine the 3-D shape of the cylindrical object from single view. we, can, however, determine the shape from two ¥iews: A(l) and A(2), areas of the base plane and 1(1) and 1(2), lengths of generating line in the image in the first and second frames are measured . Their changes give us very useful information. Since we assume the orthogonality of generating 'line to base plane, there are strong constraints between these values. Simple computati,on yields the following relation,

R=A(2/ / A(l{. Si=lp)! l(l) o:=cos ((1-S )/ (R -s 2 ;)) 112

Where (I) is the surface slant of base at the first frame. Thus the normal · of base plane is uniquely determined. From this information, we ·can map the contour of base onto a plane parallel to it in order ' to detern'line its shape . By correlating two mapped shapes ; of the base from the two views, we can establish 'correspondence of each point on the boundary of the contour between two frames . Therefore, the 31-D motion parameters are completly determined .

Figures 5 give the result of applying the method to the .input images of Figures I. Figures 5 (c) and (d) show the mapped contours from the images, and (f) displays how these two contours are correlated. The good coincidence of two curves in shape means the precise correspondence between the two frames is established . The p~oposed method is robust because we use much more reliable shape parameters such as area of base and direction and length .of generating line than intensity value of each point used in the usual shape from shading method. Thus, the estimated oriehtation has only· a small amount of error, less than one degree for the synthesized images.

Generally speaking, \Ve do not know whether the base is orthogonal to the cylindrical surface. A simple necessary condition for the orthogonality is obtained under , a condition that ·the cylindrical surface is approximated locally as a circular cylindriCal surface .. If the circular cylindrical surface is orthogonal to the base, the line connecting A, the point of contact and· 0, the center of circle is also orthogonal to the occluding boundary AB in the image as hown in Figure 6 (a) . That is, the proposed method is not applicable if the input image sequence contains any frame in which the angle BAO is far from 90 degree as shown in Figure 6 (b). The ailgle gives another constraint between the surface normals of object. but1 two unknowns · per frame 1are left undetermined.

5. DISCUSSIONS

We have explored methods for inferring motion of cylindrical object from an image sequence. The shading analysis needs to know 1the Jightihg conditions to estimate surface normals. We argue the method is not reliable even if we have such information.

Analysis of shape information also needs an additional assumption to determine the surface normals. If the l planes are not parallel, it is easy to establish correspondence by finding ·the longest generating line . . ....._\ For the parallel case, we introduce an assumption that many man-made objects satisfy, the orthogonality of base to generating line . This constrains the surface normals \... and we can determine them lfrom two views: The method '\ is robust because we use only reliable parameters. Also ) we do not need the basic assumption (4) is strictly true . The sh'ading analysis is only fort classifying the regions and finding ,the direcitions of generating lines.