A EUCLIDEAN FORMULATION OF INTERIOR ORIENTATION COSTRAINTS IMPOSED BY THE FUNDAMENTAL MATRIX

Epipolar geometry of a stereopair can be expressed either in 3D, as the relative orientation (i.e. translation and rotation) of two bundles of optical rays in case of calibrated cameras or, in case of unclalibrated cameras, in 2D as the position of the epipoles on the image planes and a projective transformation that maps points in one image to corresponding epipolar lines on the other. The typical coplanarity equation describes the first case; the Fundamental matrix describes the second. It has also been proven in the Computer Vision literature that 2D epipolar geometry imposes two independent constraints on the parameters of camera interior orientation. In this contribution these constraints are expressed directly in 3D Euclidean space by imposing the equality of the dihedral angle of epipolar planes defined by the optical axes of the two cameras or by suitably chosen corresponding epipolar lines. By means of these constraints, new closed form algorithms are proposed for the estimation of a variable or common camera constant value given the fundamental matrix and the principal point position of a stereopair. * Corresponding author


INTRODUCTION
3D epipolar geometry of an image stereopair is typically described by the coplanarity equation which requires 11 parameters for the normal pinhole camera model (i.e. when skewness and aspect ratio are not taken into consideration).These are 6 parameters for interior orientation (x01, y01, c1, x02, y02, c2) of two cameras and 5 for their relative orientation (by, bz, ω, ϕ, κ).In the case of uncalibrated cameras, on the other hand, 2D epipolar geometry is expressed by the Fundamental Matrix, which is described by 7 independent parameters and allows the estimation of the epipoles and the epipolar lines directly on the two image planes.The 2D representation of epipolar geometry corresponds to infinite 3D configurations which are known to be related by a 3D projective transformation.However, these infinite 3D configurations are constrained.Compared to the 5 independent parameters of relative orientation, when the 6 parameters of the camera interior orientations are considered to be known, the 2 additional degrees of freedom (7−5 = 2) of the fundamental matrix can be considered as constraints on the camera interior orientations.In the Computer Vision literature these constraints are expressed in the projective space, as constraints on the image of the absolute conic, through the Kruppa equations (Maybank & Faugeras, 1992;Hartley, 1997).
In the general case ≥3 images are required in order to fully calibrate a camera only from image point correspondences.However, if the principal point position is known then it is possible to estimate the camera constant from 2 images even when it is not common for their two cameras.This has been the subject of several contributions in the field of Computer Vision, where closed form solutions have been proposed for the estimation of a variable or common camera constant value from the fundamental matrix assuming known principal point.Hartley (1992) was the first who developed a rather complex algorithm for the computation of a varying camera constant value.Pan et al. (1995) derived a 3 rd degree equation in the values of c 2 .Next, they presented a linear solution in c 2 for the cases of identical and different camera constants (Newsam et al., 1996).They also found two critical geometries which do not allow the computation of varying c values from the fundamental matrix: when the optical axes are coplanar with the base or when one optical axis is perpendicular to the plane defined by the other axis and the base.An equivalent equation has been presented by Bougnoux (1998) based on the solution of the Kruppa equations, by Kanatani & Matsunaga (2000) based on constraints on the Essential Matrix and by Huang et al. (2004) through the absolute dual quadric.Sturm (2001) and Sturm et al. (2005) dealt with the case of common camera constant and formulated three different equations (one linear and two quadratic) for its determination.They also demonstrated that a common c may be calculated even when the camera axes are coplanar, as long as they are not parallel or their point of intersection is not equidistant from the two projection centres.Hartley & Kaucic (2002) gave a new geometric interpretation on the determination of different c values for the stereopair and have studied the effect of a wrong assumption about the principal point position.To address the sensitivity of all algorithms on the principal point position, Hartley & Silpa-Anan (2002) propose a new non-linear algorithm for the estimation of the fundamental matrix that leads to more stable estimations of the camera constant.With the same goal Whitehead &Roth (2002 and2004) use the DHC (dynamic hill climbing) method, while Kanatani et al. (2006) compute a new fundamental matrix from fewer point correspondences.Stewénius et al. (2005) dealt first with the simultaneous estimation of relative orientation and a single camera constant value from 6 point correspondences through the theory of Gröbner bases, and found the existence of 15, real and imaginary, solutions.A more straightforward solution was given by Li (2006) who proposed a 15 th degree polynomial using the hidden variable method.Finally, Ronda & Valdés (2007) have examined the Kruppa equations in the case of a stereopair and, based on a projective geometry theorem of the French mathematician Poncelet, propose a parameterization of all possible solutions for camera calibration.
In this contribution the constraints that the fundamental matrix imposes on the interior orientation parameters are derived in 3D Euclidean space.The epipoles, the projection centers and the epipolar lines of the principal points allow the estimation, independently on each image plane, of the dihedral angle formed by the epipolar planes of the optical axes.The equality of the estimation of this angle from the two images imposes one geometric and algebraic constraint on the interior orientation parameters.A second independent constraint is derived in a similar way from the equality of the dihedral angle of the epipolar planes that correspond to two suitably chosen epipolar lines.By means of these constraints four new closed form algorithms are developed for the computation of a common and variable camera constant from the fundamental matrix assuming known principal point.

Dihedral angle of the epipolar planes defined by the optical axes of a stereopair
The baseline and the optical axes of a stereopair, as long as they are not coplanar, define two epipolar planes Π1 and Π2 (Fig. 1).
Figure 1.Epipolar planes Π1, Π2 of the optical axes of the two cameras define a dihedral angle β.
Plane Π1 intersects the two image planes ε1, ε2 at epipolar lines lp1 and lp1′, and Π2 intersects them at lines lp2′ and lp2, respectively.These lines correspond to the epipolar lines of the two principal points p1, p2 and can be estimated by joining principal points p1, p2 with the epipoles e1, e2: [ ] where the notation: [ ] a is used to express the vector cross product as the product of a skew-symmetric matrix and a vector.
Then lp1′ and lp2′ can be found from the fundamental matrix F of the stereo pair as: On plane ε1 a new point Β1 can be constructed at the intersection of line lp2′ and a new line l, perpendicular to lp1 from principal point p1.It can be shown that: , where 1 0 0 0 1 0 0 0 0 and then Β1 is equal to: Respectively, on ε2 point Β2 can be constructed at the intersection of lp1′ with the perpendicular to lp2 from p2.This allows the formulation of two independent equations for the estimation of angle β from the fundamental matrix, the principal points p1, p2 and the camera constants c1, c2 of the two images.The equality of these equations gives one geometric constraint among the interior orientation parameters of the two cameras and their fundamental matrix.

[ ] [ ] [ ]
The position of the two image principal points p1, p2, the epipoles e1, e2 and constructed points Β1 and Β2 define on the two image planes the following distances: a1 = p1e1, b1 = p1Β1, d1 = Β1e1 on ε1 (Figure 2, left) and a2 = p2e2, b2 = p2Β2, d2 = Β2e2 on ε2 (Figure 2, right).The 3D homogenous coordinates of the vertices of tetrahedron O1Β1p1e1 are simplified if a local coordinate system is selected, centered at p1 with axes defined by lines p1e1, p1Β1 and p1Ο1.In this system p1 = [0 0 0 1] T , Β1 = [0 b1 0 1] T , e1 = [a1 0 0 1] T and  Ο 1 = [0 0 c1 1] T .Subsequently, via these points, through an analytical computation of a plane passing through 3 points, the homogeneous representations of epipolar planes Π1, Π2 can be estimated as Π1 = [0 1 0 0] T and Π2 = [c1b1 a1c1 a1b1 −a1c1b1] T and the cosine of their dihedral angle β is found by equation: ) In a similar way, planes Π1, Π2 can be independently expressed through the corresponding points of the second image plane ε2 (Figure 2, right) and the following second equation for angle β can be derived: However, the values of Eq. 6 and Eq. 7 must be equal, and so it must hold that: Eq. 8 expresses the equality of two dihedral angles in 3D Euclidean space and associates the interior orientation parameters of a stereopair with those of its fundamental matrix.This geometric as well as algebraic constraint decreases by one the 6 degrees of freedom of the interior orientation of the two cameras.

Dihedral angle of two random epipolar planes
A second independent constraint can be expressed through the computation of the dihedral angle of any two additional epipolar planes that correspond to two randomly selected epipolar lines l1 and l2 (Figure 3).Epipolar planes ΠΜ, ΠΝ are constructed on the first image plane ε1 through points Μ1, Ν1, at the intersection of lines l1, l2, respectively, with a line perpendicular to lp1 at principal point p1.

[ ] [ ] [ ]
The same planes are constructed on the second image plane ε2 through points Μ2, Ν2 at the intersection of lines l1′, l2′, with the perpendicular to lp2 at p2.Through these representations of ΠΜ and ΠΝ it is possible to estimate their dihedral angle γ:

[ ] [ ]
) A second equation for angle γ can be formulated if planes ΠΜ and ΠΝ are constructed through the respective points on the second image plane (Figure 4, right): Thus the equality of Eq. 11 and Eq. 12 gives a second constraint, additional to that of Eq. 8, among the parameters of camera interior orientations and their fundamental matrix: ) ( ) and thus the equality of angle γ allows the formulation of the following constraint: c a m n a m n a 2c a c s t c a m t n s a m n All distances referring to the second image plane are here again signed and are estimated once epipolar lines l1′, l2′ corresponding to points Μ1, Ν1 are computed via the fundamental matrix and points Μ2, Ν2.
It may seem at first that any additional epipolar line may offer further constraints in the form of Eq. 13.Such constraints, however, are not independent.Epipolar planes are defined through two bundles of epipolar lines which are in projective correspondence.As a consequence, the two families of collinear epipolar planes are also related by a projective transformation and hence they retain cross ratio.Thus, if equality of the dihedral angles of three corresponding epipolar planes is guaranteed, this equality will then hold for every additional epipolar plane.Consequently only one constraint (Eq.13 or Eq.15) is independent of Eq. 8.
As an alternative, two constraints in the form of Eq. 13 may be considered independent but in this case the constraint of Eq. 8 is no longer independent.

The case of coplanar optical axes
In case the optical axes of the two cameras of a stereo pair are coplanar, then lines lp1, lp2 are epipolar lines in correspondence, and thus the constraint of Eq. 8 is no longer valid since the epipolar planes of the optical axes coincide and their dihedral angle cannot be defined.Additionally, Eq. 13 or Eq. 15 do not only represent the equality of angle γ between planes ΠΜ and ΠΝ but also the equality of the dihedral angles formed by ΠΜ and ΠΝ with the common plane of the two optical axes.So for the two orthogonal tetrahedra O1p1Μ1e1 and O2p2Μ2e2 (Figure 3) the angles along the edges O1e1 and O2e2 should be equal.This equality allows in a way similar to section 2.2 to formulate only one constraint, of the simpler form:

CAMERA CALIBRATION ALGORITHMS
As it was shown in the previous section, in the general case of non-coplanar optical axes the fundamental matrix imposes two independent constraints on the interior orientation parameters.This is in accordance with the Computer Vision literature, in which these constraints are expressed in the projective space through the Kruppa equations (Maybank & Faugeras, 1992).
Here the two independent constraints are expressed in 3D Euclidean space through Eq. 8 and Eq. 13 or Eq. 15.
At least 3 images are required to fully calibrate a camera.The parameters of their common interior orientation can be estimated through the simultaneous solution of all the constraints imposed by the fundamental matrices of all stereo pairs, once the distances in Eq. 8 and Eq. 15 are expressed as functions of the principal point coordinates.
However, in the case of stereo pairs it is only possible to partially calibrate the cameras.Eq. 8 and Eq. 15 can be solved for any 2 out of the 6 interior orientation parameters of the two images (x01, y01, c1, x02, y02, c2) if the remaining 4 are known.Out of all possible combinations, the computation of the camera constant from a stereo pair with known principal point is of greater interest, since in most cases the latter can be assumed at the center of the image frame.
In that case, knowledge of the principal point position on the two image planes allows the estimation of all distances in Eq. 8 and Eq. 15, which can then lead to the formulation of two equations, one linear and one of 3 rd degree on to the square of the camera constant values (c1 2 , c2 2 ).In the following four new algorithms for partial camera calibration are proposed, based on the solution of these equations.

Images with different camera constants
If the location of the two principal points is known, then all distances in sections 2.1 and 2.2 can be estimated.As a consequence, the only remaining unknown variables in Eq. 8 and Eq. 15 are the values of the two camera constants c1 and c2.Solving the square of Eq. 8 for c1 2 yields equation: ( ) Then, substitution of Eq. 17 to Eq. 15 leads to the formulation of the following 3 rd degree equation on c2 2 (= w2): The analytical solution of this polynomial gives the following three solutions: These are values that satisfy simultaneously the squares of Eq. 8 and Eq. 15 and correspond to c2 2 , and therefore should be positive.Thus the first solution can be directly discarded since it is negative.From the remaining two solutions only the one that is positive and at the same time satisfies Eq. 8 and Eq. 15 (besides their squares) is kept.The camera constant of the second camera c2 can be found from the square root of the valid estimation of w2, and then c1 can be estimated from Eq. 17.
As it was previously mentioned, Eq. 8 is not valid when the two optical axes are coplanar.In such a case it is possible to formulate only one constraint among the interior orientation parameters and the fundamental matrix.The coplanarity of optical axes is, as a consequence, a critical geometry for the determination of two camera constants from a stereopair, a property that was first found by Newsam et al. (1996).

Images with common camera constant
If it is known that the two images of a stereo pair have a common camera constant, then its computation is possible even in the case of coplanar optical axes from Eq. 15 or from either Eq. 8 or Eq.16.In fact, Eq. 15 addresses the problem in general.
Conversely, the solution of Eq. 16 for a common camera constant is valid only if it is confirmed that the optical axes are coplanar, while Eq. 8 can only be used in the case of non-coplanar optical axes.In this way three different equations can be formulated, one of 3 rd degree and two linear on the square of the common camera constant value.It should be noted that the degrees of the proposed equations are in accordance to the ones suggested by Sturm (2001) and Sturm et al. (2005) which are based on the solution of the Kruppa equations through Singular Value Decomposition (SVD).

General Case
In more detail, if c1 = c2 = c, then the square of Eq. 15 gives the following 3 rd degree polynomial on c 2 (= w): ( ) where: ( ) Besides the obvious invalid solution w = 0, Eq.21 has up to three different solutions which can be estimated directly from the polynomial coefficients h1, h2, h3 and h4.The common camera constant value can then be found from the square root of the real positive solutions of w.Eq.21 is valid even in the case of images with coplanar optical axes.

Non-coplanar optical axes
If the principal point location is known, it is possible to determine whether the optical axes of the two images are coplanar or not.In the first case lines lp1 and lp2 that connect the principal point and the epipoles on the two image planes should be corresponding epipolar lines.Consequently, the two principal points p1, p2 should satisfy the epipolar constraint: So if it is confirmed from Eq. 23 that the optical axes of the two images are not coplanar, then the square of Eq. 8 can be solved for the common value c = c1 = c2 of the camera constant.This leads to the following linear solution:

Coplanar optical axes
Conversely, when the optical axes are confirmed to be coplanar Eq. 16 can be used instead, for the computation of a common camera constant: It must be noted though that, as Sturm (2001) and Sturm et al. (2005) have pointed out, the computation of a common camera constant is not possible in case the two camera projection centers are equidistant from the intersection point of the two optical axes or when the optical axes are parallel.

TESTS AND EVALUATION
To evaluate the effectiveness of the proposed algorithms tests were performed with simulated data and the results were compared to similar closed form algorithms from Computer Vision literature.For the simulations, two stereo pairs were constructed from different perspective projections of a 3D grid of dimensions 2×2×2 m 3 , consisting of 27 points (Figure 5).
Image size was set at 1024×768 pixels, the principal point was considered at the center of the image frame and camera constant was set at c1 = 800 pixels and c2 = 1000 pixels in the case of variable camera constant algorithms and at c = 900 pixels for the ones that estimate a common camera constant.The relative orientations of the two stereo pairs correspond to different B/H ra-tios, and the second stereo pair is close to a critical geometry for both algorithms, i.e. the optical axes are almost coplanar (the dihedral angle of their epipolar planes is 1.5°) and the distance of the two projection centers from the "ideal" point of intersection of the optical axes differ only by 2%.
Figure 5. Example of simulated data.

Images with different camera constants
For the estimation of a varying camera constant among the two frames three closed form algorithms were implemented and tested in the experiments: The algorithm proposed in Section 3.1 which is based on the equality of dihedral angles of epipolar planes defined independently on both images The algorithm of Newsam et al. (1996), which is based on the algebraic properties of the essential matrix The algorithm of Bougnoux (1998), which is based on the solution of the Kruppa equations An additional non-linear, self-calibrating bundle adjustment solution was also carried out without the use of control points.For initialization the results from the closed form algorithms were used.
In order to check the sensitivity of the proposed algorithms with respect to errors in the measurement of corresponding image points, normally distributed random errors of various standard deviations σxy (from 0.1 up to 1 pixel) were added to the correct image point coordinates.To further check the repeatability of the algorithms, 20 different solutions were performed for each σxy level.From them a mean (cmean) and a standard deviation (cstd) were calculated for the estimated camera constant values.
The results of all solutions are presented in Figure 6.
In all experiments, the estimations of the camera constant values from all mentioned closed form algorithms (CF in the diagrams) were identical, and at the same time very close to the bundle adjustment results.This is a confirmation that the algorithm proposed here is equivalent to the ones from the recent Computer Vision literature.The mean values of the camera constant estimations cmean are close to ground truth values with differences less than 5%.However, it is clear from the standard deviation diagrams cstd that the spread of solutions around their mean increases with the level σxy of image noise, and so does the uncertainty of estimated camera constant values.This is even worse for the second stereo pair whose image configuration is close to a critical geometry.For the estimation of a common camera constant the same experiment was repeated, but this time image coordinates for all simulations were estimated with a single camera constant c1 = c2 = 900 pixels.The 6 different methods implemented and tested are: The algorithm of Newsam et al. (1996) for common camera constant, which is based on the algebraic properties of the essential matrix (Newsam et al) The algorithm of Sturm (2001), which is based on the solution of the Kruppa equations through SVD decomposition (Sturm) The algorithm of Bougnoux (1998), where the common camera constant is estimated by the mean of the two values c1 and c2 computed by the algorithm (Bougnoux) The algorithm proposed in section 3.2.1, which is based on the equality of the dihedral angle of two suitably chosen epipolar planes defined independently on the two image planes (Alg.

3.2.1)
The linear algorithm proposed in section 3.2.2, which is based on the equality of the dihedral angle of the epipolar planes of the optical axes (Alg.

3.2.2)
A non-linear bundle adjustment solution without control points (Bundle) The results of all solutions are presented in Figure 7.
It is clear that the estimation of a common value for the camera constant of the two cameras is less stable, and in some cases their mean value does not converge to the ground truth solution, especially when image noise gets higher than 0.5 pixels.The estimations from bundle adjustment are, in general, more concentrated around their mean and closer to ground truth than the closed form solutions.A comparison of the five closed form algorithms shows that those of Newsam et al. (1996) and Sturm (2001) give equivalent results, which are very close to the mean of the two camera constant estimations of Bougnoux (1998).At the same time the algorithms proposed in this contribution give results which are closer to those from the bundle adjustment and the ground truth, even in the near-critical configuration of the second stereopair.

CONCLUDING REMARKS
2D epipolar geometry, as expressed by the Fundamental Matrix, imposes 2 independent constraints on the interior orientation parameters of the two cameras of a stereo pair.In the Computer Vision literature these are typically formulated in projective space, as constraints on the image of the absolute conic, or as algebraic constraints on the Essential Matrix.In this contribution a new formulation of these constraints is proposed in 3D Euclidean space.The main concept is that the position of the principal point and the camera constant together with the epipoles allow defining independently on the two image planes families of epipolar planes which must have common dihedral angles.Through these constraints new closed form algorithms are proposed for the estimation of a variable or common camera constant from the Fundamental Matrix and the principal point position of a stereo pair.Experimental results have shown the effectiveness of the proposed algorithms.
the right triangles Β1p1e1, Β2p2e2 and the projection centers O1, O2 form two orthogonal tetrahedra O1Β1p1e1 and O2Β2p2e2.The dihedral angles among their edges O1e1 and O2e2 defined by their non-perpendicular faces (i.e. the angles defined by the planes O1e1p1, O1e1Β1 and O2e2p2, O2e2Β2, respectively) are equal to the dihedral angle β of planes Π1, Π2.

Figure 2 .
Figure 2. Computation of the dihedral angle β of the two epipolar planes defined by the optical axes of the two cameras (planes O1e1p1, O1e1Β1 on the left image and O2e2p2, O2e2Β2 on the right).
can be simplified if points Μ1, Ν1 are suitably selected on the image plane ε1 at positions with 2D homogenous coordinatesΜ1 = [x0+y0−y1 −x0+x1+y0 1] T and Ν1 = [x0−y0+y1 x0−x1+y0 1] T insteadof selecting randomly two epipolar lines l1, l2 and then construct points Μ1, Ν1 as described above.These points belong by definition to a line perpendicular to lp1 at p1 and their signed distances to the principal point are equal to m1 = a1 and n1 = −a1.Due to this property Eq.11 takes the form of:

Figure 6 .
Figure 6.Comparison of different algorithms for the computation of two camera constant values from two simulated configurations (configuration 1 above, configuration 2 below) at different noise levels.Mean values and standard deviations are given from 20 solutions per noise level.

4. 2
Figure 7.Comparison of different algorithms for the computation of a common camera constant value from two simulated configurations (the same as in Figure 6) at different noise levels.Mean values and standard deviations are given from 20 solutions per noise level.