ON THE VISUALIZATION OF POSITIONAL PRECISION

Tasks such as image registration or pose estimation require the determination of transformations based on uncertain observations. Hence, the position of any geometric object transformed according to this estimate is also uncertain, at least in terms of precision. Often the knowledge of uncertainty changes the judgment of individuals. Thus, the visualization of this information is crucial whenever a human decision-maker is involved. In the absence of error-free reference data, we consider the estimated precision as the probably most important quantity characterizing the uncertainty. This contribution focuses on the visualization of positional precision as provided by estimated covariance matrices. Basic design principles such as coloration and contouring in 2D and 3D are presented and discussed in the context of practical applications, e.g., the superimposition of distance information as seen nowadays in sports broadcasts. As a novel contribution, we propose quartic plane curves to represent the confidence regions of the loci of conic sections.


Motivation
"One of the most challenging aspects of data visualization is the visualization of uncertainty. When we see a data point drawn in a specific location, we tend to interpret it as a precise representation of the true data value. It is difficult to conceive that a data point could actually lie somewhere it hasn't been drawn." (Wilke, 2019) This observation motivates the quantification and visualization of positional uncertainty in this contribution: Sampled spatial data often features unknown positional errors, resulting in the question of where a data point or an object should be displayed. Unfortunately, the mere addition of scalar values to convey uncertainty complicates the display already. However, a sophisticated visualization of uncertainty is crucial in many practical applications with the participation of human decision-makers. As it is generally known, the inclusion of information about uncertainty changes the judgment of individuals. More satisfaction and a lower likelihood of regret is a consequence in the decision process, cf. (Politi et al., 2007) in the context of medical applications.
Many representations of uncertainty rely on the concepts of statistics, which are not always accessible to human intuition. Furthermore, introducing information about uncertainty usually increase the data dimension considerably. For example, the positional uncertainty of a two-dimensional point, represented by a vector with expectation values and a corresponding covariance matrix, already requires five parameters which hamper the utilization of image representations. Thus, the challenge is to transform quantified positional uncertainty into a visualization that enables clear perception.

Related Work
A comprehensive overview of uncertainty visualization provide (Bonneau et al., 2014). The various sources of uncertainty are explained and discussed, a formal description inclusive of mathematical modeling is given. Different techniques for the visualization of uncertainty are present, e.g., comparison, the usage of glyphs, or an attribute modification. Examples from medical science, weather and climate, security application, and intelligence are discussed. The visualization of the uncertain spatial data is discussed in (Ślusarski and Jurkiewicz, 2020) for topographic objects.
In (Perwass and Förstner, 2006) circles are constructed by virtue of three uncertain points in any case. The propagation of the covariance matrices of the points enables the visualization of the uncertainty of the resulting circles by showing the result of sampling. For conic sections, sets of conics having probability exp(−1/2) are used for visualization.

Contribution
In this contribution, we focus on the visualization of positional uncertainty of basic geometric objects such as points, straight lines, and conic sections. Essentially, this refers to the empirical precision of the parameters. Hence, a reasonable representation of an uncertain object's position and geometry is the mean and the covariance matrix of the parameters. These quantities are usually obtained by estimation processes or via variance propagation.
Essentially, we • discuss applications in 2D and 3D, e.g., the mapping of points for image registration or the visualization of uncertain camera poses obtained by spatial resection • illustrate common design principles and visualization techniques, e.g., contouring, coloration, or uncertainty fields • define confidence regions for 2D points, conic sections, and 3D straight lines.
We propose quartic plane curves for the definition of confidence regions of conic sections, i.e., bivariate quartic equations. The zero sets of these polynomial equations define the curve's locus and hence the boundaries of the confidence regions.

REPRESENTATION AND VISUALIZATION OF BASIC UNCERTAIN 2D ENTITIES
We restrict our explanations to the geometric entities required for the applications presented in Section 3: 2D points, conic sections, and straight lines in 3D. Confidence regions for straight lines in 2D bounded by the branches of hyperbolas are discussed and shown in (Wolf, 1938) in general and in (Faugeras, 1993) for epipolar lines. Two end-points delimit an uncertain straight line segment with uncertainties depicted by two ellipses and a hyperbolic error band for the joining straight line. This yields a confidence region with the shape of an hourglass or a bone; see (Meidow et al., 2009), for instance. In 3D, the uncertainty of a plane is depicted in (Meidow et al., 2016, Fig. 5) and the confidence region of a 3D straight line in (Förstner and Wrobel, 2016, Fig. 10.16).
Notation We denote geometric 2D entities with lower-case letters and 3D entities with upper-case letters. Matrices are set sans-serif; vectors with serifs. For the representation of points, straight lines, conic sections, and transformations, we also utilize homogeneous coordinates. Homogeneous vectors and matrices are denoted with upright letters, Euclidean vectors and matrices with slanted letters. For homogeneous coordinates, '=' means an assignment or an equivalence up to a non-zero scaling factor.

Uncertain 2D Points
Uncertain 2D points with the coordinates x = [x, y] T are most frequently represented by the vector µ x of means and the covariance matrix with the correlation coefficient ρ xy ∈ [−1, 1] (Förstner and Wrobel, 2016, p. 366). A convenient two-dimensional visualization is then the confidence ellipse defined by the points fulfilling with k 2 being the inverse of the chi-square cumulative distribution function with 2 degrees of freedom and probability α. For k 2 = 1, we obtain the so-called standard error ellipse. A plotted confidence ellipse conveys the complete information provided by the point representation {µ x , Σ xx }. This visualization is chosen in combination with a vector field in Figure 1 to show the uncertainty of point positions obtained by mapping via an uncertain homography.
However, depending on the graphical ways to express the uncertainty, a single characteristic number is often desirable. Prominent scalar measures derived from the covariance matrix Σ xx of the point coordinates are Helmert's point error (Helmert, 1868) and the point error according to Werkmeister (Werkmeister, 1920). Since the shape of the ellipse is of interest, the eigendecomposition Σ xx = UΛU T of the covariance matrix is computed with Λ = Diag([λ1, λ2]) and the orthogonal matrix U.
With this, Helmert's point error is computed via considering the square root of the trace of the covariance matrix. The measure σ H is geometrically the semi-diagonal of the minimal bounding box of the standard ellipse. Note that Helmert's point error follows no distribution, thus cannot be used as a test statistic for decision-making. An alternative is the point location error according to Werkmeister which considers the determinant of the covariance matrix. We use the superscript '2' to indicate the units of these measures, i.e., σ H features the same unit as the standard deviations of the coordinates and σ 2 G features the same unit as the variance of the point coordinates. Two modifications of Werkmeister's point error (4) are common to facilitate the interpretation by human observers: Multiplication of σ 2 G by π yields the area A = πab of the confidence ellipse computed with the lengths a = λ 1 and b = λ 2 of the semi-axes. Moreover, taking the square root of which is the geometric mean of the standard deviations and motivates the subscript 'G' introduced in (4). The geometric mean is the radius of the confidence circle with the same area as the confidence ellipse (Förstner and Wrobel, 2016, p. 367).
After computing these measures for supporting points, contour lines, i.e., curves joining points of equal value, can be computed and visualized as shown exemplarily in Figure 5 below.

Conic Sections
For conic sections, we propose using quartic plane curves, i.e., bivariate quartic equations, to delimit confidence regions. With the homogeneous symmetric matrix C representing a conic section and an image point in homogeneous representation x = [x, y, 1] T , the relation holds for points on this conic section with the vectorization c = vec(C). We can utilize the six parameters a = [a, b, c, d, e, f ] T of the parametrization to rewrite (6), leading to the polynomial equation with the homogeneous vector of coefficients a and the vector of monomials u = [x 2 , xy, y 2 , x, y, 1] T also known as lifted coordinates.
Then, given a homography matrix H for the point transfer x = Hx, the transfer of a conic reads and the corresponding covariance matrix Σ c c of the result c = vec(C ) is obtained by variance-covariance propagation. The six parameters a are then extracted from the conic matrix (7). The rank of the covariance matrix Σ aa is 5 due to the homogeneous representation and the null space of Σ aa properly reflects this by the relation Σ aa µ a = 0.
Assuming a to be a Gaussian random vector with mean µ a and covariance matrix Σaa, we compute the squared Mahalanobis distance d 2 M between a and µ a . With the pseudo-inverse Σ + aa of the covariance matrix Σ aa we obtain which is chi-square-distributed with r = 5 degrees of freedom. Thus, for a percentage α of all vectors d 2 M ≤ k 2 holds where k 2 is a quantile, i.e., the inverse of the chi-square cumulative distribution function with r degrees of freedom and probability α. For a given k, the set of parameter vectors a k with equal likelihood is then given by which is equivalent to for Σ aa µ a = 0, cf. (Ochoa and Belongie, 2006) and (Hartley and Zisserman, 2000, p. 298 ff.). Considering (8) and (12), we obtain a k = k 2 Σ aa − µ a µ T a u k by comparison of coefficients and therefore the relation for points on the boundaries of the confidence region.
The graph of the polynomial equation (13) is a quartic plane curve. The branches of this curve are contour lines depicting the locus of all points with equal likelihood, i.e., the boundaries of a confidence region enclosing the conic. For k 2 = 0 the polynomial (13) becomes reducible and represents a double conic with equation µ T a u k = 0 depicting the expectation value of the conic's locus. By rearranging equation (13) we can compute the distances k 2 for points (x, y) via and u = [x 2 , xy, y 2 , x, y, 1] T . These distance values are nonnegative since Σ aa is positive semi-definite. The values can be visualized and used to generate contour lines as shown in Figure 7 for instance.

EXEMPLARY APPLICATIONS
We discuss three applications in 2D and 3D which benefit from the visualization of uncertainty: The assessment of positions in images obtained by image registration, the superimposition of distances in TV broadcasting of sporting events, and the visualization of an uncertain sensor pose. Technically, this refers to the visualization of results obtained by mapping a point via an uncertain homography, transforming an ellipse via a homography, and estimating a sensor pose.

2D Example: Point Transfer via Homography
The process of transforming different sets of image data into one coordinate system is known as image registration. We consider a linear transformation as the model for mapping image point locations by estimating and applying a homography. Figure 2 shows two aerial images capturing almost the same scene. One image is a color infrared image serving as the target or reference image, and the other is an RGB image to be transformed by image warping. Given a 2D point in homogeneous representation x and a homography matrix H, the point transfer reads with the vectorization h = vec(H), the 3×3 identity matrix I 3 , and the symbol ⊗ denoting the Kronecker product.
We utilize corresponding image points to estimate the transformation parameters. These ties points can be obtained automatically by tracking or matching procedures. The result of this registration usually depends on the spatial distribution of the utilized tie points. To demonstrate the effect of this, we interactively selected corresponding points in the image regions showing areas covered with buildings. In doing so, we assume uncertain positions of the tie points in both images.
After the estimation of the transformation parameters h, the image registration is obtained by applying the mapping. Figure 3 shows the result of this registration with a checkerboard As pointed out by the quotation in the introduction, the point locations are uncertain. However, variance propagation provides an estimate for this uncertainty: With the covariance matrices for the point coordinates and parameters of the homography, we obtain the covariance matrix cf. (Förstner and Wrobel, 2016).
To summarize, we get for the Euclidean point coordinates x = [x , y ] T the covariance matrix assuming statistical independence between point coordinates and transformation parameters.
Equation (18) provides a covariance matrix for each image point which can be visualized by an error or confidence ellipse. Visualizing these ellipses for selected points, we get a visualization of the so-called uncertainty field, see Figure 1 and (Förstner and Wrobel, 2016, Fig. 10.18). Figure 4 depicts the uncertainty field for one of the shown images. As to be expected, the plotted standard ellipses are the smallest in the vicinity of the utilized tied points. Furthermore, the major axes are pointing away from the tie points. By exchanging the roles of reference and moving image, one gets the uncertainty field in the coordinate system of the other image.  The covariance matrix can be computed for each image point, and we can consider Helmert's point error (3) as a scalar measure for evaluation. Figure 5 shows a contour plot for the measure varying from 6.5 to 82.4 pixels in the image. The values have been mapped to the hue of the image using the universal color code of traffic lights.

Transfer of a conic via Homography
As a second 2D application, we consider the transformation of a conic via a homography in which both the conic and the transformation might be uncertain. Such information is of interest in situations where we consider point distances on a plane observed in a perspective view.
As an example, consider the overlay of ellipses in a video stream showing a soccer match; see Figure 6. The largest ellipse denotes the minimum distance of 10 yards to be observed by players during a direct free kick. The homography is determined by identifying image points that correspond to points with known coordinates on the playing field. In addition or alternatively, corresponding straight lines can be used to estimate the parameters of the utilized homography. As the result of this estimation process, we obtain estimates for the transformation parameters h = vec(H) and the corresponding covariance matrix Σ hh . Without loss of generality, we assume the circle defined on the playing field to be error-free.  Figure 7 shows the result for the uncertain ellipse obtained by mapping the 10-yards-circle into the image via (9). Confidence regions for the locus of the ellipse are given by the quartic plane curves (13), which in this case decomposes into pairs of nested, not necessarily convex ovals. The precision of the ellipse's locus increases with diminishing distance to the seven interactively selected tie points.

3D Example: Pose Estimation
As a 3D example, we consider the image-based pose estimation for a calibrated camera. The pose or orientation of a camera comprises the position C of the projection center in a world coordinate system and three parameters r for the rotation from a world coordinate system to the camera coordinate system. Given measured image points corresponding to known 3D points, the six parameters of the pose can be determined by spatial resection. Figure 8 shows the utilized image captured by a camera mounted on a low-flying drone, approximately 40 m above ground. The camera's field of view covers 68°horizontally and 41.5°v ertically. The maximum likelihood estimation yields estimates for the parameters values and the covariance matrix of the parameters. The estimated confidence region for the three coordinates C of the projection center is an ellipsoid, as depicted in Figure 9. In principle, the covariance matrix Σ rr of the three rotation parameters r could be displayed similarly. However, the interpretation of rotation parameters is usually challenging, and so is the visualization of the attached precision. Therefore, we do not consider the rotation parameters but the z-axis of the camera coordinate system, which coincides approximately with the optical axis of the camera system, cf. Figure 9. This substitute allows for intuitive visualization of directional precision but provides few visual cues for the precision of the roll angle.
Given the projection center C and the matrix R(r) for the rotation form the world into the camera coordinate system, a point X on the optical axis is given by the point-direction form where λ is the Euclidean distance between the points C and X . The confidence region for the straight 3D line depicted in Figure 9 is the envelope of all confidence regions, i.e., ellipsoids, for the points along the optical axis (Förstner and Wrobel, 2016, p. 379). Starting at the projection center and moving along the optical axis, this illustrates an uncertain ray or halfline. The confidence region with the shape of an hourglass is most narrow in the vicinity of the 3D points used for the spatial resection.
Please note that the confidence region for a straight line is not a quadric in general, in particular not a hyperboloid of one sheet. However, if we fix the projection center, i.e., set Σ CC := O 3 , we obtain a double cone with an elliptical base. Conversely, if we fix the rotation, i.e., set Σ rr := O 3 , we obtain a cylinder with an elliptical base. Note that usually strong correlations exist between the parameter groups C and r if cameras with narrow fields of view are used. These correlations are not perceivable by the visualization in Figure 9.

CONCLUSIONS AND OUTLOOK
Proper visualization of the uncertainty of object positions is vital for many practical applications that demand judgment by human decision-makers. While it is comparable easy to depict positional uncertainty in the 1D case, e.g., by using box plots, it becomes challenging with an increasing number of dimensions. Thus, the desire to comprise multidimensional information in single measures to be visualized is comprehensible. However, by this action, information, e.g., about correlations, gets lost inevitably. Even with this subsumption, it is still unclear which design principle should be selected to convey the information, e.g., hue, texture, or contour lines. The exemplary twodimensional applications discussed in Section 3 illustrate some of these ways to provide this information. And in 3D the situations get worse. Here, animations or interactive virtual 3D models appear to be more beneficial, maybe even the utilization of sound (Lodha et al., 1996).
For the future, we are planning to study the impact of different visualization techniques. For this, experiments with human observers are envisaged to answer the question of which visualizations are subjectively more appealing. For applications asking for decisions, we would like to design experiments with decision-makers to compare results obtained with and without the presentation of uncertainty.