Conics with a Common Axis of Symmetry: Properties and Applications to Camera Calibration

1 Introduction

In the domain of computer vision, conics are considered as one of the important image features similar to points and lines. Recently, the planar conics-based camera calibration has been widely investigated in some computer vision literatures of the reference-list. However, the Euclidean structures can be recovered through the projections of circles or ellipses on the image plane. The identifications of the absolute conic (AC) are the keys to recovering the Euclidean structures. In camera calibration, the imaged circular points (ICPs) are computed and used to deduce the constraints of the imaged absolute conic (IAC). Meng and Hu [Men et al., 2003] applied a calibration pattern that is made up of a circle and a set of lines passing through its center and computed the vanishing line and its intersection with the projected circle. Wu et al. [Wu et al., 2004], in their turn, proposed a quasi-affine invariance of two parallel circles for recovering the ICPs. Gurdjos et al. [Gurdjos et al., 2006] extendedWu's works and analyzed the algebraic properties of N ≥ 2 parallel circles through the generalized eigenvalues of arbitrary two circles in them. Kim et al. [Kim et al., 2005] just focused on the projected concentric circles and gave the Rank-2 constraint for deducing the projected conic dual to the circular points (CDCP) which consists of the two ICPs. Gurdjos and Kim [Gurdjos et al., 2005] then proposed how to recover the Euclidean structure from confocal conics in the same way. More generally, the principal-axes aligned (PAA) conics provide enough constraints to recover the IAC, which are deeply investigated and discussed in Ying's paper [Ying and Zha, 2007].

∗This work has been supported by French National Research Agency (ANR) through TecSan program (project DEPORRA ANR09-TECS-006).

The past works mainly focus on the researches under some special conditions, for example, all conics are circles, or conics have all principal axes aligned. In this paper, we investigate the problem of recovering the Euclidean structure from the projections of the N arbitrary conics under a more general condition. Consider a set of N ≥ 2 parallel conics, and a distinct pair of conics (C1,C2) in it. A linear family of projected conics is described as ˜C(˜λ) = ˜C1 - ˜λ ˜C2. It includes three members called degenerate conics consisting of linepairs [Gurdjos et al., 2006] corresponding to the generalized eigenvalues of ( ˜C1, ˜C2). Without any constraints, no useful information about the Euclidean structure can be deduced from the degenerate conics. Only if (C1,C2) have at least one common axis of symmetry [Wong et al., 2003], one of the three degenerate conics can deduce the image of the symmetric axis. However, this is not enough to recover the Euclidean structure that we only know the symmetric axis. The key problem is how to recover the line at infinity. Based on the degenerate conics or some geometric properties of (C1, C2), it is possible to determine the line at infinity, and then the circular point-envelope can be recovered. With it, the planar homography and the IAC can all be computed correspondingly. In this paper, we will give a more general framework on the problems of planar conics, which relaxes the problem conditions of the research works in [Gurdjos et al., 2006; Ying and Zha, 2007].

This paper is organized as follows: Section 2 briefly introduces some notations and basic equations. In Section 3, the geometric and algebraic properties of the conics with a common axis of symmetry are introduced, about the symmetric axis, the line at infinity and the degenerate conics. Two different conditions of camera calibration are proposed in Section 4. The results of synthetic and real experiments are shown in Section 5. Finally, Section 6 presents some concluding remarks.

2 Notations And Basic Equations

2.1 Pinhole Camera Model

Let M = (X, Y, Z, 1)T be the 3D homogeneous coordinates of a world point M, and ˜m = (u, v, 1)T be the homogeneous coordinates of its projection in the image plane. M and ˜m are related by the following equation in the pinhole model:

where zM is a scale factor (projection depth of M); K is the camera intrinsic matrix, with the focal length (fu, fv), the principal point (u0, v0) and the skew factor s; [R t] is the camera extrinsic matrix, that is the rotation and translation from the world frame to the camera frame; P = K[R t] is referred to the camera projection matrix. If we assume that the world space is restricted to its x-y plane, the world-toimage homography will then be expressed as:

where r1, r2 are the first two columns of R.

2.2 The Equations of Conics' Images

Any conic can be converted to another conic by some projective transformations. In the paper, we only focus on the cases of closed conics (circles and ellipses). If the world space's xy plane is restricted to the conic's supporting plane, the conic has the following form:

and its form in matrix is

with the center of conic (X0, Y0). In the image plane, the corresponding projected conic under H is of the form

with α is an unknown scale.

3 Properties of Two Conics With ACommon Axis of Symmetry

Suppose there are two central conics C1, C2 with a common axis of symmetry ls in the same plane. Generally, we assign the origin of the world space to the center of C1 and the x-axis to line ls; C2 is centered at point (d, 0), with d > 0. Then C1, C2 have the Euclidean representation in matrices

Without loss of generality, we set A1 = 1 to simplify the description. And ˜C1 , ˜C2 are denoted as the images of the two conics.

3.1 The Generalized Eigenvalues and Eigenvectors of ( ˜C1, ˜C2)

Let (˜λ, ˜V) resp. (λ,V) denote the vector of generalized eigenvalues and the matrix of generalized eigenvectors of ( ˜C1, ˜C2) resp. (C1,C2). The vector of generalized eigenvalues ˜λ has the following linear relationship with λ up to a scale factor β (computed by MAPLE):

where Π = p1p2p3p4, p1 = 1 + √A2 + √A2d, p2 = 1 + √A2-√A2d, p3 = 1-√A2+√A2d, p4 = 1-√A2-√A2d. The generalized eigenvector ˜vy corresponding to the generalized eigenvalue ˜λ3 = βλ3 is the vanishing point of y-axis. After getting the vanishing point ˜vy, the image of the symmetric axis ls can be computed by the following formulation

If the eccentricities e1, e1 of C1 and C2 are known, the scale factor β can be computed according to ˜λ3 in (6). From the eigenvalues of ˜λ, the parameter d can also be determined by the following equation:

Now, we discuss how to order the generalized eigenvalues of ˜λ. Since ˜λ1, ˜λ2 are a pair of symmetric values, it is not necessary to distinguish their differences. Therefore, what we have to do is just finding which is ˜λ3. Actually, it has something to do with the degenerate conic members of ( ˜C1, ˜C2), and only the degenerate conic with the parameter ˜λ3 consists of two real lines. A degenerate conic consists of which kind of lines, this depends on the absolute signature [Gurdjos et al., 2006] of it. If the absolute signature is equal or less than 1, then the conic's corresponding parameter is ˜λ3 .

3.2 The Degenerate Conics of ( ˜C1, ˜C2)

In the three degenerate conics of ( ˜C1, ˜C2), the one with parameter ˜λ3 is very helpful for recovering the line at infinity. It is described in the following form

Actually, if d = 0, it becomes the work described in [Ying and Zha, 2007], so this case will not be discussed. If d ̸= 0, we will discuss the problem under two conditions: e1 = e2 or not. When e1 ̸= e2, ˜C(˜λ3) consists of the images of two parallel lines perpendicular to the symmetric axis ls, and each of them is through a pair of conjugate intersection points of ( ˜C1, ˜C2) respectively. It is not enough to establish the constraints of the line at infinity by just knowing this. Therefore, some additional constraints are introduced, which will be given in Section 3.3.

When e1 = e2, we have ˜C(˜λ3) ∼ ˜l∞˜ ξT + ˜ ξ˜lT∞, where ˜l∞ is the projected line at infinity, and

The line ξ passes through a pair of conjugate intersection points of C1,C2. Obviously, the line ξ is called the radical line, if C1 and C2 are two circles. The other pair of conjugate intersection points are the absolute points IA and JA, and they have the standard form [Ying and Zha, 2007]:

where e is the eccentricity. The projected absolute points ˜IA and ˜JA lie on the vanishing line ˜l∞.

˜

To distinguish ˜l∞ and ξ, we firstly observe the generalized eigenvectors of ( ˜C1, ˜C2) corresponding to the generalized eigenvalues ˜λ1, ˜λ2. They have the following form:

We notice that the two image points ˜v1, ˜v2 are located at both sides of ˜ ξ or on it, but at the same side of ˜l∞. Therefore, based on the positions of ˜v1, ˜v2, the vanishing line ˜l∞ can be determined by the same way as described in [Gurdjos et al., 2006].

The other two degenerate conics are denoted by ˜C(˜λi)(i = 1, 2); each of them consists of a pair of conjugate complex lines. One of the two lines is through the point ˜IA and ˜vi, denoted by ˜u1 + i˜u2; the other is through the point ˜JA and ˜vi, denoted by ˜u1 - i˜u2. When C1 and C2 are circles, these complex lines are called the isotropic line.

3.3 The Vanishing Point of ls and The Line at Infinity l∞

As shown in Fig.1, C1 intersects ls with point a1 and a2, C2 intersects ls with point a3 and a4. The corresponding images of 4 intersection points are respectively ˜a1, ˜a2, ˜a3, ˜a4, which can be determined in close form according to ˜C1, ˜C2 and ˜ls. The vanishing point of x-axis, ˜vx, can then be related with ˜a1, ˜a2, ˜a3, ˜a4 in the cross-ratio equations

And p4/p1, and p2/p3 can be represented in ˜λ:

According to the equations (13,14,15,16), the image point ˜vx can be solved in the closed-form. After getting ˜vx, we can compute the image of the line at infinity, ˜l∞, by the following equation

3.4 The Conic Dual to The Absolute Points

After recovering the line at infinity, the conic dual to the absolute points C∗∞ can also be identified, which is a degenerate line conic (rank 1 or 2). In the Euclidean coordinate system, it has the following form [Ying and Zha, 2007]:

Proposition 1 Under any projective transformation H, l∞ is the null vector of C∗∞, so ˜C∗∞˜l∞ = 0.

Proof. The absolute points are defined on the line l∞, so ITAl∞ = JTAl∞ = 0. Therefore,

Proposition 2 Under any projective transformation H, the line passing through the absolute point IA (or JA) and one real finite point, lies on C∗∞. If the line is denoted by ˜w1±i˜w2, then we have the following constraints:

From [Ying and Zha, 2007], we also know that the dual conic C∗∞ is fixed under a scale and translation transformation.

Proposition 3 Once the conic C∗∞ is identified on the projective plane then projective distortion may be rectified up to an affinity. The two scaling parameters [Hartley and Zisserman, 2003] of the affinity are related with parameter e. Their ratio is equal to 1 √|e2-1| .

Actually, a rectifying homography U can be computed from the SVD-like decomposition of ˜C∗∞:

and it satisfies U ∼ H up to an affinity A. The matrix A is described as the following form:

where Q is 2 × 2 non-singular matrix, and it can be decomposed as Q = Ldiag(s1, s2)NT from the SVD. s1 and s2 are the scaling parameters of the affinity A. The ratio of them s1 : s2 is equal to 1 √|e2-1| . With knowing the parameter e, there are only 4 d.o.f in the matrix U. If e = 0, then the affinity A degenerates to a similarity S.

C∗∞ is a set of conics with parameter e. If e = 0, the degenerate conic is called the conic dual to the circular points, denoted by ˜C∗0 ∞ under a projective transformation H, which encodes the Euclidean structure in 2D projective space. It satisfies the following constraints:

where ˜x1 = ˜vy × ˜o1, ˜x2 = ˜vy × ˜o2; the image of conic center ˜oi(i = 1, 2) can be determined easily according to the the pole-polar relationship between ˜l∞ and ˜C1, ˜C2 [Hartley and Zisserman, 2003].

4 Calibration Algorithm

4.1 Calibration with The Central Conics e1 = e2

Assume that the eccentricities of C1 and C2 are equal, we can computed the scale factor β in ˜λ3. Having the scale factor β, we can determine ˜vy and ˜ls according to equation (6,7). Then the degenerate conic ˜C(˜λ3) is determined. From a SVD-like decomposition of it, the projected line at infinity ˜l∞ is recovered. Based on the constraints of (20,21,22), the conic dual to the circular points is recovered. A rectifying homography matrix U can be computed from the SVD-like decomposition ˜C∗0 ∞ = Udiag(1, 1, 0)UT , where U ∼ HS for a similarity S.

After performing the rectification, we can get the real homographyH = [h1 h2 h3], and the constraints for calibration

where ω = K-T K-1 is the IAC. At least three images of the calibration object are needed to calibrate the camera's intrinsic parameters.

4.2 Calibration with The Central Conics e1 ̸= e2

Assume that the eccentricities of C1 and C2 are all known and e1 ̸= e2, we can computed the scale factor β in ˜λ3 and the vector ˜vy. Thus ˜vx is computed from equation (13,14,15,16). Then ˜vx and ˜vy can determine the vanishing line ˜l∞. We can obtain the images of the absolute points of conic (C1,C2) by intersecting ˜l∞ and ( ˜C1, ˜C2). It is easy to find the lines through them and the conics' centers. According to the equation (20,21,22), we can get ˜C∗0 ∞. Then the homographyH can be easily recovered, which deduces the same constraints as the equation (23,24).

5 Experiments

We performed a number of experiments, both simulated and real, to assess our algorithm with respect to noise sensitivity and test its validity in the endoscope operation.

5.1 Simulated Data

The simulated camera has the following parameters: fu = fv = 900, u0 = 512, v0 = 384, and s = 0. The resolution of the simulated image is 1024 × 768. A simulated pattern consisting of a circle and an ellipse is applied to test our algorithm. The axis of symmetry coincides with the ellipse's major axis and the circle's diameter. Twenty synthetic images are captured, and Gausian noise of zero mean and σ standard deviation is added to the pixel coordinates. Varying the noise level σ from 0 to 1.5 pixels, we performed a series of tests to investigate the influence of the noise on determining the camera intrinsic parameters (s is constrained to 0). The estimated camera parameters (fu, fv, u0, v0) are compared with the ground truth, and the relative errors are computed. As shown in Fig.2, the relative errors of each camera parameters increase almost linearly with the noise level.

5.2 Endoscope Operation

In the computer aided surgery, the tool tracking of the endoscope operation is a hot research point. Before and in the process of tracking, it is necessary to calibrate the endoscope's intrinsic and extrinsic parameters. Zhang's calibration method [Zhang, 2000] has a very mature algorithm, but it is not suitable for utilizing in the endoscope operation of cavity. Some special methods for the calibration of endoscope (for example, [Yamagushi et al., 2004]) also have this kind of problems. The calibration process then can not be integrated into the mid-part of the operation process. The reason is that the calibration object can not be combined with the operation tool easily. We are in great need of a suitable calibration object, which can be put into the body cavity and lead to the intracavity calibration. Therefore, we do some research on the conics with a common axis of symmetry. Our purpose is trying to apply the new calibration object in the endoscope operation. A simulated endoscope operation experiment is performed to test the new algorithm. The experiment setup is shown in Fig.3(a). A pair of tools are put into a fake cavity, and one of them is marked with three conics, the tool's axis is also the symmetric axis of conics. The calibrated endoscope is moving and tracking the tool's position. If the initial pose of tracking is missing, we will estimate the tool's pose according to the algorithm in this paper. The estimated tool positions in the tracking are shown in Fig.3(b), where the white lines denote the tool's direction axis and position. From the images of Fig.3(b), we can see that the estimation results are good enough for the tracking.

5.3 Real Images

To fully evaluate our camera calibration algorithm, we also carried out an experiment on real images. We utilized a pattern consisting of two ellipses and a circle as the calibration object. By using a Point Grey FLEA2 camera, we took some images of the calibration object at different views. Some sample images are shown in Fig.4(a). The resolution of these images is 1024 × 768. Edges were extracted by using Canny's edge detector (Fig.4(b)) and the ellipses (Fig.4(c)) were obtained by using a least squares ellipse-fitting algorithm [Fitzgibbon et al., 1999]. The calibration results of real experiment are listed in Table.1 (s is constrained to 0), where the result from Zhang's method [Zhang, 2000] is taken as the ground truth.

From Table.1, it can be seen that the calibration results of our calibration method are near to those of Zhang's [Zhang, 2000] calibration method. Although our approach is not better than Zhang's in the real data experiment, it is still comparable to Zhang's, and can be considered as a direction to the research on camera calibration.

6 Conclusions

We discussed the geometric and algebraic properties of the projected central conics with a common axis of symmetry in this paper. According to the properties of conics via the generalized eigen decomposition, the image of the symmetric axis can be obtained. After solving the images of the conics' centers, the vanishing line is computed. Then the 2D Euclidean structure is recovered, and the constraints of IAC are deduced which are suitable for the central conics' calibration. Extensive experiments on simulated and real data were performed, and very satisfactory results were obtained. The experiment results show that our calibration technique is accurate and robust.

Acknowledgments

The author would like to thank Sandrine Voros and Philippe Cinquin for their technical supports.