Estimating 3-0 Movement of a Rigid Object: Experimental Results

ABSTRACT

We r Eport some preliminary experimental res ults on est imating general three-dimen­ s ional ration parameters of a ri gid body from ~wo consecutive image frames taken by a television c a~era. The approach i s to first ~xL ract corner point coordinates to s ubpixei accuracy from grayvalue images, then match them between two frames , and finally solve a se t of ron- linear equations to obtain the motion rarameters. Two factors affect the ac ­ curacy cf s olutions : The magnit ud e of th a motion rarameters and the errors in the raw data . These are discussed in the analysis of the experimental result s .

I . Introduction

Three - dimensional motion es timation has applicat ions in many areas , including target tracking and robotic vision . I n addition, it is c losely related to t he problem of calibra­ t ing s tereo cameras . I n recent years, t here have been a number of papers on this topic , see , e.g., [1,2]~ However, we have not seen in the open literature work which reported the estimation of general 3-D motion parameters f r om reel-world images. The present paper is a continuation of our previous work [3,4]· It gives scme experimental result s on estimating general 3- D motion parameters from a pair of consecutive image frames taken with a televi­ sion c a~era. The approach is to first extract corner po int coordinates to s ubpixel accuracy, then match them between the two image frames , and finally solve a set of nonlinear equations to obtain the motion parameters. In [3], some computer s imulation results were given . Some experimental results on television images were presented in [4] where, however, the corner

* On leave from North China Institute of Computing Technology, Peking, China . His permanent address is "P.O. Box 619,Peking, Ch ina (PRC')".

Es timating 3- D Mnvement of a Rigid Object: Experimental Results

J .-Q. Fang* and T . S . Huang Coordinated Science Laboratory University of Illinois at Urbana- Champaign 1101 w. Spring field Avenue Urbana, Illinois 6 1801

point coordinates were determined to only pixel accuracy. The relatively poor numerica results of [4 J prompted us to develop a s ubpixel corner detector and apply it to motion estima tion, it i s the main goal of the present paper to report some preliminary results along this direction .

II. Al gorithms

Our approach to estimating 3- D motion parameters of a rigid object with s~ a ll rotation from two successive images consists of the following steps:

1. Evaluating the gradient magnitude and direction of grayvalue images by the Sobel operator .
2. Detecting corner candidates by finding the local maxima of the magnitude of the gradient direction, the detail of which is given in Fang and Huang [ 4) ·
3. Distinguishing corner points from noise points by the Hough transform after detecting the corner candidates, also given in Fang and Huang [ 4 J·
4. Det~rmining corner positions to subpixel accuracy.
5. Matching the sets of corner points in t ·he two frames by a point pattern matching method.
6. Solving a set of non-linear e quations to obtain the 3-D motion parameters using the coordinates of corresponding points as the raw data.

The algor i thm for determining corner positions to subpixel accuracy is to first find edge point positions with subpixel accuracy using gradient magnitudes, then fit an edge line through these edge points, and finally intersect the edge lines to get a corner position. We will give the details of the algorithm in a future publication.

The match algorithm we used is based on the work of Rosenfeld (5], but we modified it to accommodate the cases where the scales of the object in corresponding f rames are differ­ ent.

The basic non-linear equations we solved are of the fo r m:

' ' where (Xi, Yi) and (Xi, Yi) are the coordinates of the ith point in image frame 1 and frame 2, respectively, and the unknowns are

The translation vector is ( x, y, z). The rotation is around an axis through the origin with directional cosine (n 1 , n 2 , n 3 ). The

rotation angie is e. F is a conGtant. Algorithms for solving these equations (which are mainly the Newton and Levenberg-r~arquardt methods) have been discussed in Ref . [3 ]·

III . Results

General description

A major difficulty in estimating 3- D motion parameters from real-world images is that the non-linear equations (1) are very sensitive to errors in the raw data, i.e. the coordinates of point pairs. We have experienced this in a simulation study described in [3]. The situation was worse when we used a digital CID camera with low resolution as an input device. The point array of the camera was only 240 x 240 pixels. Some result s of motion estimation from the CI D camera images are listed in Table 1. Although these results contain fairly large errors ( 20%), they indicate the feasibility of our approach. The object in the images in our experiments was a robot arm holding a wooden cube. In Figure 1, we show some pairs of typical images we processed.

We processed dozens of picture pairs with a variety of motion. Only some of the results are listed in Table 1. In the table, "ideal" denotes ground truth. The cases "sub" and "pixel"denote the solutions of the motion equations obtained from the raw data with subpixel accuracy and pixel accuracy, respec­ tively.

Motion parameters

An important discovery in our experiments is the big influence of the magnitudes of motion parameters upon the accuracy of the solution of the non-linear equations (1).

1. A lar ~e angle of rotation caused ~low convergence or even divergence of the itera­ tion in solving the equations (1), whereas too small an angle also made the soution less accurate. The promising range of rotation angle in our experiments appeared to be 2- 4 degrees.
2. There was mutual interference between the parameters of rotation and those of translation. General speaking , the estimates of rotation parameters were more accurate than those of translation parameters. But large translation degraded the results of rotation parameters. The reason of this phenomenon seems to be that the 2- D projection of a ~otation in the 3-D space , say around the y axis, is similar to that of a translation in the x direction. This is likely to confuse human vision as wel l . So a moderate amount 6f translation (for example, x' and y' less than 0 . 5) tended to yield good results in our experiments.
3. 3 ) A large translation in the z direction (which made a big difference in scale between two frames) helped to obtain good results of motion parameters.

In summary , the magnitudes of the motion parameters have a big impact on the accuracy of the solution of equations ( 1), b'ecause these parameters determine the characteristics of the non-linear e quations (1). Investigation of this influenc e can be helpful in deciding on the time intervals between snapshots .

Accuracy of raw data

Another factor which affected the accuracy of the solution is the accuracy of the raw data, i.e., the coordinates of corner points in the two frames. For each image pair, we computed using the raw data with sub-pixel accuracy and those with pixel accuracy. In Table 1, we list these results in adjacent lines. Comparing these, one concludes that the results in the "subpixel" case are generally better than those in the "pixel" case. In theory, the results in the "subpixel" case should be much better than those listed in Table 1, but in practice, there are other factors which degrade the estimation accuracy. The distortion of lenses and the physical deformation of the corners of objects are two examples.

IV. Conclusion

We have reported in this paper some first experimental results on estimating general 3-D

~otion paramete r s of a ri~id body f rom two c onsecutive image f rames taken with a elevision camera . ~he approach is to first xt , ·~ c: corner points , then match the corners in the t wo frames , and finally so lve a se t of non - linear equation iteratively to obtain the motion parameters . Tt wa s found tha '. the solution to the non- linear equations was very s ensitive to errors in the corne r point ~oordinates , and that determininP, thes e coordinates to s ubpixel accuracy impr oved the motion parameter estimation result s .

Acknowled gement

~his work was s upport ed by National J cience Foundation Grant MCS- 8206926 .

• L 1] Dreschler , L., and ti .-H . Nagel , "Volu!Tietric tAodel and 3- D 'I'rajectory of a Moving Sar Derived from Mo nocular TV- Frame Sequences of a Street Scener , Proc . IJCAI - 8 1 , pp . 692- 697 .
• ( 2 ] l-?oach , J . VI ., and J . K. Aggarwal , " Deter!Tiining the t"iovement of Cbjects from a :3equence of Images", IEEE Trans · PAMI - 2 , Nov . 1980 , PP · 554- 562 .
• ( 3 ] C" ang , J .-Q., and ':'. S . Huang , " So lving Three Dimensional ~mall Ro tation r·otion r.: quations : Uniqueness , Al gorithms and Nume r ical f~ esul ts", submi tted to Somput e r Graphics and 1mage Processing .
• ( 4 ] ::·ang , J .-Q., and T . S . P.uang , "A Corner ~inding Algorithm for T!Tiage Analysis and Registration", Pr oceedings AAAT - 82 , Pittsburgh , Pennsylvania , Aug . 18 - 20 , 1982 , pp . 46 - 49 ·
• [ 5 ] Panade , S · and A. Rosenfeld , " Point Pattern t«Tatching by Re laxation" , PR-1 2 ( 1980) , pp . 269- 275 ·

Table 1 · Result s of estimation of 3- D motion parameters