ANALYSIS: IMPACT OF IMAGE MATCHING METHODS ON JITTER COMPENSATION

: The degradation of image quality caused by satellite jitter has drawn attention, and many researches have illustrated the importance of the jitter compensation. As the essential component of jitter compensation, image matching involves in the determination of jitter processing accuracy. Hence, the impact of imaging matching methods on jitter compensation is explored in this paper. At first, a framework based on imaging matching is built for jitter compensation. Two typical sub-pixel accuracy matching methods (i.e. correlation coefficient and least squares matching, as well as phase correlation matching) could then be served for the framework. The experiments are designed by using multispectral images of GF-1 satellite, and quantitative evaluations show that compared with correlation coefficient and least squares matching, phase matching method makes the accuracy of obtained jitter curve increase with the amplitude error decreasing by more than 0.012 pixel as well as phase error decreasing by more than 0.0119 rad, and makes the quality of restored images improved on both geometry and radiation. It indicates that phase matching method has better performance with respect to the framework of jitter compensation in this paper.


INTRODUCTION
Satellite jitter is a common source of satellite imaging error. Due to the space constraints of satellite platform and application design considerations, several components are unable to connect to the high-resolution camera on board using fixed joint. The disturbing force or disturbance torque activated by those dynamic structure, will lead to the deviation of camera position, so as to affect the imaging process directly (Mo et al., 2020;Zhu et al., 2022). The periodic vibration response generated by dynamic structure is the main cause of satellite jitter (Iwasaki, 2011).
Under ideal condition, there is a unique geometrically consistent image point corresponding to the object point. If the camera shifts induced by satellite platform jitter, charge aliasing will happen during the imaging process, which will not only result in image distortion but also image blur (Llaveria et al., 2020;Song et al., 2021). The distortion caused by satellite jitter could be easily observed because the geometric information has been changed. As the research of Toutin et al. (2011), the impact of satellite jitter on image geometry varies with the attitude angle, as shown in Fig. 1. The lateral shifts and scale change would be produced by satellite jitter exist in roll angle. The scan-line spacing changes would be generated due to the satellite jitter reside in pitch angle. The skew between the scan lines would be generated by satellite jitter within yaw angle. Since directions in image are usually simplified into two, it indicates that the image distortion in across-track direction comes from satellite jitter exist in roll angle, and the image distortion along the track comes from satellite jitter in pitch and yaw angle. Besides, image blur will also appear due to the motion induced by satellite jitter (Yitzhaky, 2000). As satellite jitter increases the relative motion between camera and ground object, the motion blur captured by the camera will then grow with incoming light accumulating during the exposure time of camera. Moreover, the degradation of image quality caused by satellite jitter becomes more apparent on both geometry and radiation, with the improvement of satellite spatial resolution (Mattson et al., 2010;Tong et al., 2015a). Therefore, the problem of satellite jitter should not be overlooked. (left), roll (centre) and yaw (right) (Toutin et al., 2011).
In order to ensure the quality and availability of high-resolution remote sensing images, as well as to promote the application of obtainable satellite data, there have been many researches about jitter compensation. The approaches using multispectral images outperforms others due to their economical and convenient advantages (Iwasaki, 2011;Liu et al., 2016). Multispectral bands with fixed row spacing are generally produced by the multispectral camera, and visible bands are better suited for jitter extraction with considering the similarity of them. By using only visible bands, Pan et al. (2021) reduced the coupled distortion and blur effects caused by satellite jitter simultaneously, which is the first research not considering geometric and radiometric information separately. However, as they indicate, the subpixel matching based on the correlation coefficient and least squares algorithm may limit the accuracy of jitter detection and then spread the detection error to image restoration. Hence, the impact of imaging matching methods on jitter compensation results is worthy of analysis.
The aim of this research is to explore which image matching method can result in more accurate jitter detection and image restoration. Since most common sub-pixel accuracy matching methods are correlation coefficient and least squares matching, as well as phase correlation matching, the comparison would be conducted based on these two matching methods. At first, an overall framework of jitter compensation would be established. Different matching methods are able to be applied into the framework. Then, the impact of imaging matching methods on jitter compensation results will be analysed through comparison experiments.

FRAMEWORK OF JITTER COMPENSATION
A framework of jitter compensation is built, consisting of 4 parts: image matching, acquisition of jitter curve, determination of PSF for each image line, as well as geometric correction and deblurring of images, as shown in Fig. 2. After multispectral images are processed through the framework of jitter compensation, the image degradation caused satellite jitter will be eliminated and the restored images can be obtained.

Image matching
Image matching is conducted on the overlap area between adjacent bands of multispectral images. Two matching approaches with sub-pixel accuracy are provided: correlation coefficient and least squares matching, as well as phase correlation matching. These two matching approaches could make the parallax maps produced.
(1) Correlation coefficient and least squares matching The first one takes correlation coefficient matching as coarse registration followed by least squares matching as the fine one. The correlation coefficient matching utilizes correlation coefficient as the similarity measure between template window and search window, and adopts gray values and their distribution in the local area of the image as the comparison elements. If ( , ) r c denotes the coordinate difference between the target point and candidate point, the correlation coefficient ( , ) C r c is given as follows (Potucková, 2004): where , i j g and , ' i r j c g   represents the DN values of the target window and the search window, respectively; i and j are the line number and sample number in the target window respectively; m and n are the size of the target window.
Using the initial matching point provided by the result of correlation coefficient matching, least squares matching will further make the matching accuracy reach to sub-pixel level. Same with correlation coefficient matching, least squares matching looks for corresponding points through comparing the gray values. The undetermined parameters are estimated by minimizing the sum of squares of gray difference between the template window and search window. The observation model is as follows (Bethel, 1997;Potucková, 2004): where 1 ( ) g  and 2 ( ) g  denotes the DN values corresponding to the left and right matching windows, respectively; 1 ( ) n  and 2 ( ) n  represents the noise corresponding to the left and right matching windows, respectively; ( , ) x y is the coordinate of the target point; 0 a , 1 a , 2 a , 0 b , 0 b , 1 b , 2 b are respectively the parameter of the geometric affine transformation; 0 h and 1 h are the parameters of the gray transformation.
The correlation coefficient and least squares matching method is based on image gray value with using strategy of initial value correction, and it could realize accurate point-by-point dense matching.
(2) Phase correlation matching An alternative, phase correlation matching method, is a Fourierbased matching technique, well known for its accurate and effective advantages. The phase correlation matching method could also have great potential in dense matching by using sliding window.
Based on the displacement characteristics of the Fourier transform, the normalized cross-power spectrum matrix ( , ) Q u v is expressed as (Tong et al., 2015b) 0 0 where ( , ) F u v and ( , ) G u v is the Fourier transform of the original image and shifted image, respectively;  represents the complex conjugate; 0 x and 0 y is the offset between two images in line and column directions, respectively. To simplify the problem, the image offsets in the two directions can be separated by SVD (singular value decomposition) which makes the 2-D plane in the frequency domain decomposing into two 1-D planes, and a rank-one approximation of the normalized cross-power spectrum matrix will then be obtained (Hoge, 2003). The decomposition process can be expressed as Eq. 4 indicates that image offset in each direction (i.e. 0 x or 0 y ) can be regarded as the slope of the linear phase changes. Therefore, the unwrapped phase angles show a straight line, and image offset which is presented as slop of the line can be estimated through the least square fitting.
As a typical sub-pixel matching method, phase correlation matching has the advantage of high efficiency and accuracy. According to the requirement of dense matching, the sliding window would be applied on the phase correlation matching method. Both original image and shifted image will be divided into many sub-images by the sliding window, and phase correlation matching will be conducted on the corresponding sub-images.

Acquisition of jitter curve
Parallax maps are obtained by image matching between bands of multispectral images, and jitter curve will then be acquired through the parallax maps. The acquisition procedure of jitter curve has three main components: extract parallax disparities using parallax map, disparity curve fitting by sine function, and calculate jitter curve through integration imaging model.
Parallax disparities notes the value of pixels in the parallax map. Since pixels in the same line of the image are generated simultaneously, the parallax disparities of each line could be extracted by averaging the single-line values. It has been confirmed that satellite jitter was able to be described by sine function, and relative displacements between bands of multispectral images also show sinusoidal changes (Hadar et al., 1992). Thus, disparity curve could be estimated by using the least square fitting with regard to sine function. In fact, the obtained disparity curve is the interaction result of image displacement caused by satellite jitter, integration effect of TDI CCD (time delay integration charge-coupled device), as well as time distance between corresponding points in different bands. In order to take account of those mixed factors when converting disparity curve to jitter curve, the integration imaging model is given (Ye et al., 2020): where A , f and  are parameters of jitter curve, r A , r f and r  are parameters of parallax disparities. T is the one-stage integration time of TDI CCD. 1 N and 2 N is the integration stage for adjacent bands of multispectral images, respectively. L is the fixed line distance between two bands of the multispectral image. C and D are abbreviations. Through connecting the image space and object space, the integration imaging realizes the transformation from disparity curve to jitter curve.

Determination of PSF for each image line
According to the obtained jitter curve, the jitter varies in different image lines. Therefore, PSF (point spread function) should be constructed dynamically. During the determination of PSF for each image line, the image offsets in each integration interval are extracted firstly, and then dynamic PSF is constructed using image offsets.
The jitter curve is divided into many intervals corresponding to the imaging time of integration stages in TDI CCD. As shown in Fig. 3, ( ) d t represents image offset on the jitter curve and t is time. If the TDI CCD has N integration stages, the intervals corresponding to the i th image line are marked in gray in Fig. 3, and there is one integration stage shift between the i th and ( 1) i  th image line. The image offsets in N integration intervals all contribute to the shift caused by satellite jitter. Thus, the PSF would be constructed using the image offsets in each integration interval.

Figure 3. Illustration about division of jitter curve
The PSF for i th image line is established as follows: where  is the impulse function, x d and y d is the jitter curve on the along and across track direction, respectively. It makes estimated PSF vary with the image line, so that space-variable property of image offset caused by jitter would be taken into account carefully.

Geometric correction and deblurring of images
Through applying such space-variable PSF into Wiener filtering, geometric correction and deblurring of satellite jitter images can be realized.
In order to suppress the ringing effect, a window function is applied on the image, firstly. The window function consists of nine regions, as shown in Fig. 4     The window function in different regions corresponds to different descriptions. The window function ( , ) w x y can be expressed as: Then, Wiener filtering will be conducted on the image with window function on it. The key of Wiener filtering is to minimize the mean square error. It is given by (Gonzalez and Woods, 2002): is the optimal estimation of the Fourier transform of the original image; ( , ) G u v and ( , ) H u v is the Fourier transform of the shifted image and PSF, respectively; K is the signal-to-noise ratio.
After the restored multispectral images are obtained by using Wiener filtering, the procedure of jitter compensation is done. Furthermore, through comparing the error in generated jitter curve and different restored multispectral images, the impact of imaging matching methods on jitter compensation results will be clear.

Data description
The experiments were designed using multispectral images of GF-1 satellite. Four bands are produced by the multispectral camera of GF-1 satellite, but only visible bands, i.e. B1 (blue band), B2 (green band) and B3 (red band), are adopted in experiments due to the consideration about similarity between adjacent bands. Fig. 5 shows the experimental data of GF-1 satellite and two random areas marked in yellow boxes. Because the size of the multispectral image is large, these two marked areas will be zoomed in for clearly observation in the follow-up experiment. The detailed information about the experimental data are illustrated in Table 1.

Results and analysis
In experiments, the correlation coefficient and least squares matching, as well as phase correlation matching, are separately utilized in the framework of jitter compensation. The accuracy of the detected jitter curve and quality of restored multispectral images, which were respectively obtained by different matching method, were then compared, so that the impact of image matching methods on jitter compensation would be discussed.
Using correlation coefficient and least squares matching, the obtained parallax maps and parallax disparities are shown in Fig.  6 and Fig. 7, respectively. Fig. 6(a) and (b) display the parallax map between B1 and B2, as well as that between B2 and B3, respectively. Two directions, i.e. across and along the track, are given. Parallax disparities illustrated in Fig. 7 are extracted from the parallax maps displayed in Fig. 6, where Fig. 7(a) and (b) shows the parallax disparities between B1 and B2, as well as that between B2 and B3, respectively. Similarly, parallax maps and parallax disparities obtained by using phase correlation matching are shown in Fig. 8 and Fig. 9. According to Fig. 6 and Fig. 8, compared with phase correlation matching, correlation coefficient and least squares matching pays more attention to image details and makes parallax map more likely to be accompanied with random errors. Fig. 7 and Fig. 9 indicate the obvious jitter shows in across-track direction while not in along-track direction, so that analysis would only focus on jitter in across-track direction. Based on the framework of jitter compensation, jitter curve was estimated. The detected parameters of jitter curve are shown in Table 2. In order to measure the accuracy of the acquired jitter curve, a 1-m orthophoto image was used as the reference image to obtain the truth value of the jitter curve. The obtained truth parameters of jitter curve are 1.1010 Hz frequency, 0.9940 px amplitude, 0.4993 rad phase. Then, the error of the detected parameters was calculated, which is also described in the Table  2. According to the obtained parameters of jitter curve in Table  2, the detected error through correlation coefficient and least squares matching is slight larger than that through phase correlation matching. The amplitude error decreases by more than 0.012 pixel, and phase error decreases by more than 0.0119 rad. The reason may be that random errors existed in parallax maps using correlation coefficient and least squares matching have been spread to the jitter curve, as the same with observation on the parallax maps. The stability of phase correlation matching makes its corresponding jitter curve more accurate. Based on the obtained jitter curve, the determined PSF was applied into the image restoration to realize geometric correction and deblurring of images, so that restored multispectral images were obtained. To make it easier to check the image, two areas as marked in Fig. 5 were zoomed in on display. Fig. 10 shows the images of the two areas without treatment, using correlation coefficient and least squares matching during jitter compensation, using phase correlation matching during jitter compensation, respectively. Compared with untreated images, images after jitter compensation whatever which matching method utilized have a significant visual increasement, but the visual difference of the restored images was not obvious.  (d) is the restored image of each region using correlation coefficient and least squares matching during jitter compensation, respectively. (e) and (f) is the restored image of each region using phase correlation matching during jitter compensation, respectively.  Table 3. Assessment for image quality The further quantitative assessment was conducted on the original and restored multispectral images, as shown in Table 3.
For image geometric quality assessment, the 1-m orthophoto image was used as the reference image, MAE (mean absolute error) and RMSE (root mean absolute error) are utilized to measure the relative offsets. For image radiometric quality assessment, contrast, average gradient, robert gradient and correlation are utilized to reflect the clarity of the image. It is noted that the index, correlation, refers to the similarity of local gray values in the image, and correlation will increase if the blurring happens on the image. Table 3 illustrate that phase correlation matching slightly outperforms than correlation coefficient and least squares matching, with respect to both image geometric and radiometric quality increasement. Thus, whether through the analysis of obtained jitter curves or that of restored images, it indicates that matching method has an effect on jitter compensation, and the phase matching method shows better performance under the framework in this paper.

CONCLUSIONS
In order to promote the further study about satellite jitter, an important fundamental problem is analyzed in this paper: the impact of image matching on jitter compensation. Based on the built framework of jitter compensation, two typical matching methods with sub-pixel accuracy, correlation coefficient and least squares matching, as well as phase correlation matching, are compared. Through the experimental verification, phase matching method outperforms than the correlation coefficient and least squares matching both in jitter detection and image restoration. This work will be helpful for providing the reference of selecting the image matching strategies, and the given framework of jitter compensation also shows the contribution.