GEOMETRICAL CORRELATION AND MATCHING OF 2D IMAGE SHAPES

The problem of image correspondence measure selection for image comparison and matching is addressed. Many practical applications require image matching “just by shape” with no dependence on the concrete intensity or color values. Most popular technique for image shape comparison utilizes the mutual information measure based on probabilistic reasoning and information theory background. Another approach was proposed by Pytiev (so called “Pytiev morphology”) based on geometrical and algebraic reasoning. In this framework images are considered as piecewise-constant 2D functions, tessellation of image frame by the set of non-intersected connected regions determines the “shape” of image and the projection of image onto the shape of other image is determined. Morphological image comparison is performed using the normalized morphological correlation coefficients. These coefficients estimate the closeness of one image to the shape of other image. Such image analysis technique can be characterized as an “intensityto-geometry” matching. This paper generalizes the Pytiev morphological approach for obtaining the pure “geometry-to-geometry” matching techniques. The generalized intensity-geometrical correlation coefficient is proposed including the linear correlation coefficient and the square of Pytiev correlation coefficient as its partial cases. The morphological shape correlation coefficient is proposed based on the statistical averaging of images with the same shape. Centered morphological correlation coefficient is obtained under the condition of intensity centering of averaged images. Two types of symmetric geometrical normalized correlation coefficients are proposed for comparison of shape-tessellations. The technique for correlation and matching of shapes with ordered intensities is proposed with correlation measures invariant to monotonous intensity transformations. The quality of proposed geometrical correlation measures is experimentally estimated in the task of visual (TV) and infrared (IR) image matching. First experimental results demonstrate competitive quality and better computational performance relative to state-of-art mutual information measure.


INTRODUCTION*
Image matching is one of actual problems in remote sensing and machine vision.The selection of image matching procedure for some practical application depends on different aspects of the whole matching algorithm.Some of these aspects concern with discontinuities in object space and perspective distortions of compared images.Other aspects deal with sub-pixel precision and computational cost of matching.Many researchers have explored these aspects.We have investigated them too in our previous works (Zheltov, Sibiryakov, 1997;Zheltov, Vizilter, 2004).This paper concerns the other aspect of matching procedurethe measure of image fragments similarity (or dissimilarity) to be utilized in matching algorithm.In some practical cases the final quality and performance of image matching procedure essentially depends on quality and performance of image comparison function.So, there are some special functions for comparison of binary, grayscale, colored, multispectral and other types of images.This paper devoted to the problem of image matching "just by shape" with no dependence on the concrete intensity or radiometric pixel values.For example, one can compare images of one scene captured at different seasons, different time of day, in different weather and lighting conditions, in different spectral ranges and so on.The most popular technique for such image shape comparison utilizes the mutual information measure based on probabilistic reasoning and information theory background (Maes, 1997).The other approach was proposed by Pytiev (so called "Pytiev morphology") based on geometrical and algebraic reasoning (Pytiev, 1993).This paper develops the ideas of this geometrical approach.* Corresponding author.
In the framework of Pytiev morphology images are considered as piecewise-constant 2D functions.The tessellation of image frame by the set of non-intersected connected regions with constant intensities determines the "shape-tessellation" or simply "shape" of the image.Morphological image comparison is performed using the normalized "morphological" correlation coefficients.The motivation of reported theoretical research is following.In Pytiev approach one supposes that the "shape" F of some "class sample" f is given by supervisor, and one needs to determine whether any observed image g belongs to this class.This scheme presumes that the shape of f is more believable than the shape of g.But in practice it is not always reasonable.A lot of applications require estimating the closeness or difference between two images with the same belief of shape.So, one needs to suppose that both f and g are randomly selected from their "shape sets" F and G correspondingly.In such case they should be compared either as images ("intensity comparison" like in linear correlation) or as pure shapes ("geometrical comparison").The intermediate "intensity-geometrical" comparison like Pytiev correlation is not reasonable here.Besides, the geometrical shape comparison could provide the possibility of image matching for images segmented not only by intensity, but by other features, for example, by texture.Finally, it is useful to design some symmetric image comparison measures invariant to shape preserving intensity transforms.According to this motivation some new correlation tools for geometrical comparison and matching of shape-tessellations will be proposed based on generalization of Pytiev morphological correlation.Destination and properties of proposed geometrical correlation tools will be comparable and competitive with those of mutual information.

IMAGE COMPARISON PROBLEM AND RELATED WORKS
Let image is represented by 2D intensity function f(x,y): R, R 2 , where  is a frame region of bounded size, Rset of real numbers, R 2 -2D plane of image.The operation of "comparison" of images f(x,y) and g(x,y) presumes the estimation of their "similarity" or, conversely, "dissimilarity".Numerical estimation of "similarity" is usually performed by such similarity measures as correlation coefficients.
"Dissimilarity " is estimated by distances in some metrics.The proper formulas of such coefficients and distances are determined by image models and image spaces we use.Classical approach to image comparison considers images as elements of Hilbert space L 2 () with scalar product (f,g), But photographic linear transforms of image intensity can essentially change the distance between images and even the pair of very similar images will seem to be very "far".So, for image comparison invariant relative to linear intensity transform the following normalized linear correlation coefficient is applied In this formula images are usually centered, i.e. decreased by their constant average values.Such approach is well-known in photogrammetry as a least squares matching (Gruen, Baltsavias, 1985).
But even the normalized linear correlation is not invariant relative to more complex (nonlinear) intensity transforms occurred in practice.For image comparison under such complex and unknown intensity transforms the mutual information approach was proposed (Maes, 1997).Mutual information I(A,B) estimates the dependence of two random variables A and B by measuring the distance between the joint distribution p AB (a,b) and the distribution of complete independence p A (a)p B (b) by means of following expressions where H(A) is an entropy of A, H(B) is an entropy of B, and H(A,B) is their joint entropy.For two image intensity values a and b of a pair of corresponding pixels in the two images, required empirical estimations for the joint and marginal distributions can be obtained by normalization of the joint (2D) and marginal (1D) histograms of compared image fragments.Maximal I(A,B) value corresponds to the best geometrical matching of image fragments.This approach is based on probabilistic reasoning.It provides the robust tool for matching of images with different intensities based on their joint 2D histograms.But these histograms can not explain the geometrical idea of image "shape" in some evident form.Such mathematical "shape" formalism is given in evident form in the so-called "morphological" approach for image comparison proposed by Pytiev (Pytiev, 1993).
In the framework of this approach images are considered as piecewise-constant 2D functions where nnumber of non-intersected connected regions of tessellation F of the frame , F={F 1 ,…,F n }; f=(f 1 ,…,f n )corresponding vector of real-valued region intensities;  Fi (x,y){0,1}characteristic (support) function of i-th region: Set of images with the same tessellation F is a convex and close subspace FL 2 () called shape-tessellation or simply shape: For any image g(x,y)L 2 () the projection onto the shape F is determined as Here P F is a projective operator called projector to F. Let's note that for any image f(x,y) the description of its "shape" by a set of frame tessellation regions F={F i }, by corresponding vector subspace F and by corresponding projection operator ("shape projector") P F are practically equivalent.Pytiev morphological comparison of images f(x,y) and g(x,y) is performed using the normalized morphological correlation coefficients of the following form The first formula estimates the closeness of image g to the "shape" of image f.Second formula measures the closeness of image f to the "shape" of image f.In general K M (g,F) is not equal to K M (f,G).So, this morphological image matching score is asymmetric in contrary to symmetric linear correlation coefficient K N (f,g).

GENERALIZED INTENSITY-GEOMETRICAL CORRELATION
While the linear correlation coefficient K N (f,g) compares images just as intensity functions ("intensity-to-intensity" matching), morphological coefficient K M (f,G) compares the intensity distribution of f(x,y) with geometrical shape of g(x,y) ("intensity-to-geometry" matching).In other words, there are different types of image correlation: intensity and intensitygeometrical correlation.Are these correlation techniques principally independent or could be united in some more generic correlation frame work?In this section the answer will be given.Let some images f and g are given and some shape W is considered.Then one can measure the angle (f W  g W ). The value (f W ,g W ) characterizes the similarity of images g and f relative to shape W. In particular, if cos(f W  g W )=0, then images f and g are orthogonal or independent relative to shape W. This allows introducing the normalized linear correlation coefficient of images g and f relative to shape W: Following expressions are satisfied: So, the generalized intensity-geometrical correlation coefficient K N (f,g,W) includes both the linear correlation coefficient and the square of Pytiev correlation coefficient as its partial cases.It means that the intensity-geometrical (morphological) correlation is more generic relative to intensity (linear) correlation and includes it as a part of its own framework.Note that in this generic form K N (f,g) "structurally corresponds" to the square of morphological coefficient K M 2 (f,G).Hence, if one needs to compare the sensitivity of different correlation techniques, one should compare K N with K M 2 , but not with K M .

GEOMETRICAL SHAPE COMPARISON BASED ON STATISTICAL AVERAGING OF PROJECTED IMAGES
If one fixes some shape G and takes some different images f from shape F, there will be different values of K M 2 (f,G).Nevertheless, one can try to average these values over a set of images from F and obtain some "average effective" morphological correlation coefficient for comparison of shapes F and G. Let f(x,y) form F is a piecewise-constant 2D function described above (in section 2).Image g(x,y) from G is an analogous 2D function with m as a number of tessellation regions G={G 1 ,…,G m }; g=(g 1 ,…,g m )vector of intensity values;  Gj (x,y){0,1}support function of j-th region.Let's introduce following additional set of "S-variables": Sarea of the whole frame ; In terms of mutual information approach these S-variables correspond to following marginal and joint probabilities: where f Gj = ( i=1,..,n f i S ij )/S j , j=1,..,m.
Additionally introduce the following assumptions about the distributions of probability densities for intensity values f 1 ,…,f n : Consequently, the expectation of all intensity values in this model is zero-valued: f i  =0, and the covariation has the form With account of introduced assumptions the mean square of norm for image f of shape F has a form The mean square of projection norm for image f of shape F and fixed shape G has a form Let's determine the mean square effective morphological correlation coefficient (MSEMCC) for shapes F and G as After evident substitutions MSEMCC takes the following form where However there is one important problem with MSEMCC (1).It can not take in account the "independence" of shapes in a Pytiev sense.For such independent shapes F and G for any centered images fF, gG: K M (g,F) = K M (f,G) = 0, but MSEMCC can not provide the property K M (F,G) = K M (G,F) = 0 due to the fact that intensity of images in our reasoning above was not supposed to be centered.Let's start from the centered Pytiev morphological coefficient where f 0 =  i=1,..,n f i S i / Saverage intensity value for image f.Introduce the mean square centered morphological correlation coefficient (MSCMCC) for shapes F and G as After some substitutions MSCMCC takes the following form . 1 ) , ( Expression ( 2) is not so compact as ( 1), but due to relation MSCMCC provides both properties of (1) and important additional property This property describes the independence of shapes in a Pytiev sense (Pytiev, Chulichkov, 2010).

SYMMETRIC GEOMETRICAL SHAPE CORRELATION COEFFICIENTS
Let's try to construct symmetric modification of MSEMCC (1).Symmetric pair correlation coefficient for regions F i and G j can be defined as follows: Then modifying the expression (1) one can obtain the following symmetric geometrical correlation coefficient (SGCC): (3) SGCC satisfies both properties of (1) and symmetry property The other symmetric modification of geometric correlation (1) can be proposed based on the linear normalized pair correlation coefficient for regions F i and G j : Properties of GLCC are the same as the SGCC properties.

COMPARISON OF SHAPES WITH ORDERED INTENSITIES
Finally let's consider the problem of comparison of shapes with ordered intensities.This case corresponds to the class of intensity transforms.For simplicity we suppose that there are no equal values in intensity vectors f and g.For each region F i one can define the relative intensity function and each region G j has the analogous function  Gj (x,y).The similarity of ordered intensity sequences in "ordered shapes" F + and G + related to pair of regions F i and G j can be described by pair relative intensity correlation coefficient The use of this pair coefficient allows easy modifying the proposed geometrical correlation coefficients (1), ( 3) and (4) in order to take in account the ordering of intensity values.For example, the modified geometrical linear correlation coefficient (MGLCC) will have a form Following properties of MGLCC are satisfied: Here "-F + " means the shape with the same frame tessellation as F + , but inverse order of region intensities.In analogous way the correlation coefficients ( 1) and ( 3) can be modified too for obtaining their intensity ordering sensitive modifications.

EXPERIMENTAL RESULTS
Proposed geometrical correlation techniques were implemented in software and tested over a set of real images including remote sensing and multispectral images.The quality of proposed geometrical correlation measures was investigated relative to the quality of mutual information measure and Pytiev correlation coefficient in the task of TV (visual band) and IR (infrared band) image matching.TV and IR images were geometrically co-registered.Then some fragments of TV image were compared with all equal-sized fragments of corresponding IR image.For increasing of matching robustness the histograms of these TV and IR fragments were segmented with n=m=4 levels using least square optimal segmentation procedure.For segmented image fragments f (TV) and g (IR) at each fragment g position (x,y) following similarity measures were calculated: -mutual information MI = I(F,G); -square of centered Pytiev morphological coefficient The quality of these measures was estimated by following statistics of 2D correlation function С(x,y) -SNR (signal-tonoise ratio) and E (exceeding of first maximum to second one): where C 1global maximum of correlation value; C 2second global maximum of correlation value out of some small neighborhood of first global maximum; μmean value of correlation function; σdispersion of correlation function.Figure 1 demonstrates the example (Example 1) of experimental TV-IR data corrupted by Gaussian noise.Figure 2 demonstrates 2D-graphs (correlation fields) of different correlation measures for matching of TV etalon and IR image (for Example 1).Corresponding numerical data are listed in Table 1.Example 2 (Figures 3, 4; Table 2) presents the other TV-IR image pair corrupted by Gaussian noise with smaller size of etalon fragment.Analysis of these and other analogous TV-IR matching examples allows making following conclusions: 1.In case of relatively large fragment matching for clear and noisy images both Pytiev morphological coefficient and all proposed geometrical correlation measures (1)-(4) provide SNR and E values close to mutual information characteristics (a little bit worse or better).2. In case of relatively small fragment matching both for clear and noisy images proposed asymmetrical geometrical correlation measures (1)-( 2) provide SNR and E values slightly better than Pytiev morphological coefficient and mutual information characteristics.But proposed symmetrical geometrical correlation measures (3)-( 4) with small fragment and essential noise provide SNR and E values essentially worse than other tested measures and even can fail in global peak position (like in Example 2).These measures are not robust enough relative to noise and fragment size.3.Both shapes of correlation fields and SNR and E values for proposed measures (1) and ( 2) are exactly equal in all experiments due to the linear relation pointed in section 4 (but they will be not equal in case of inverse order of etalon and test shape arguments).

Measure
Max Totally, we can resume first experimental results as follows.
Proposed MSEMCC and MSCMCC measures are very close to mutual information by matching characteristics and shape of correlation field.In our experiments these measures are usually a little bit better than MI (not statistically proved).But, taking in account the computational simplicity of expression (1) relative to MI and MSCMCC formulas, one can conclude that MSEMCC is the best geometrical correlation measure for practical applications with matching of different band images.First experimental testing of proposed similarity measures is performed for TV and IR image matching task.Based on these experiments, MSEMCC seems to be competitive in quality and better in computational performance relative to the state-of-art mutual information measure.But proposed symmetric correlation measures are not stable enough relative to noise and should be improved.Further research will include the design of more robust symmetric geometrical correlation coefficients and massive experimental exploration of precision and robustness for all described shape correlation techniques.