HIGH-RESOLUTION OPTICAL SATELLITE IMAGES COLOR CONSISTENCY METHOD BASED ON EXTERNAL COLOR REFERENCES

Affected by factors such as season, illumination, atmospheric and sensor distortion, different satellite images often show obvious color difference, resulting in “stitching seams” at the edge of adjacent images, which seriously affects the application of satellite images. This study proposes a novel color consistency method for optical satellite images utilizing external color reference. Firstly, we improved the dark channel defogging method combining with the atmospheric distribution characteristics of satellite images, and used it to perform atmospheric correction on satellite images; Secondly, we corrected the color of atmospheric corrected satellite images through low-frequency signal replacement. Finally, we use a linear model to establish the relationship between high and low frequency signals, and stretching the high-frequency signal of images through local modelling. We selected two sets of representative experimental data for experiments, both the visual and quantitative obtained excellent results.


INTRODUCTION
With the development of deep learning, intelligent interpretation and other technologies, remote sensing technology plays more and more important roles in some fields such as global mapping, global land cover classification, etc. Therefore, the demand for large-scale satellite images has increased significantly. However, limited by the width of satellite images, large-scale ortho-images often require multiple images mosaics. Due to factors such as season, atmosphere, illumination or sensor distortion, different images often show obvious color difference, resulting in poor color continuity of adjacent images. At present, the satellite images color consistency methods mainly includes three types: the method of style conversion, the method of using image overlapping area and the method based on external reference data.
Generally, the method of style conversion often select one image in the study area with good imaging condition as the reference, and converts its color information to other images. Histogram matching is a classical non-linear method of style conversion, which changes the shape of image histogram to make it exhibit similar tone with the reference image (Helmer etc., 2005). Sun proposes using Wallis filters to make different areas of the image have similar mean and variance with the reference image (Sun, etc., 2008). The above method can achieve good processing result for satellite images with similar ground objects. However, satellite images often cover large-scale areas with different types of ground objects. Therefore, mandatory style conversion may cause texture damage and color distortion of images; The method based on image overlapping area is the mainstream technology for satellite images color consistency. Generally, this type of method uses the overlap area of adjacent images to establish a linear or non-linear relationship between adjacent images, and performing color correction of images through color * Corresponding Author propagation or global optimization. Xia uses the method of histogram matching to establish the color relationship between adjacent images, and transmit the color information of reference image to other images in the study area through minimum spanning tree (Xia etc., 2017). Canty proposed using the pseudoinvariant feature (PIF) points extracted from the overlapping area to correct the radiation quality of adjacent images (Canty etc., 2008). Yu uses the least squares global optimization method to solve the gain and offset correction parameters of images (Yu etc., 2017). The above methods can obtain good processing effect on aerial images, but they need to consider the complex spatial relationship among images and therefore have a higher degree of complexity. In addition, satellite images usually cover a large-scale area, different ground objects may have different radiation characteristics. Therefore, using the same correction parameter for the entire image often cannot obtain good results; The method of using external reference is a new way for color consistency processing of satellite images. Yu et al. proposed segmentation of the satellite image and counting the mean and variance of each area to build a color reference library (Yu etc., 2016). Zhou et al. proposed calculating the local mean value of the input and reference images, and performing Gamma correction in each part of images (Zhou etc., 2015). However, the above methods does not take into account the image texture differences between input images with the external reference image, and it is difficult to collect large-area high-resolution satellite images with good color distribution. Therefore, they are difficult to apply in practice.
This study fully analyzes the causes of color differences in satellite images and proposes a new optical satellite images color consistency method by using low-resolution external color references. The main innovations of this study are as follows: (1) This study consideres the effect of atmosphere on the color and clarity of images, and improves the dark channel prior defogging method.
(2) This study proposes using the low-frequency of images to simulate its color distribution, and performing color correction of images by low-frequency signal replacement. (3) The linear model is used to establish the relationship between high and low frequencies of image, and the high-frequency signal of images is stretched through local modeling.

Atmospheric correction based on dark channel prior
In the field of computer vision, a widely used atmospheric imaging model is as follows: where I is the observed image, J is the scene radiation, A is the atmospheric light, t is the atmospheric transmission. According to the dark channel prior, in most of fog-free image patches, there exists some pixels with low intensity at least in one color channel of RGB color space (Kaiming etc., 2011).
where J dark is the dark channel of image J, c is the color channel of RGB. Ω(x) is a local patch centered at x. Minimize both sides of Equation 1 to get: Take Equation 2 into Equation 3 to get the local atmospheric transmission of image I: Deforming Equation 1 to get the following equation: where t represents the atmosphere transmission of image I, t0 represents the transmission threshold (generally set to 0.1), and the atmospheric light A is generally determined according to the 0.1% brightest pixels of the image.

Color distribution characteristics of satellite images
Satellite images are obtained on high orbits with similar imaging angles, so the representation of same ground objects is close in different images. At different observation scales, the ground objects show different observation characteristics. From a small scale, the ground objects are changing rapidly, such as the construction and demolition of houses, the increase and decrease of vegetation. However, under large observation scales, the change of ground objects spatial distribution is slowly, such as the location and volume of mountains, cities. Therefore, this study proposes using the external low-resolution remote sensing images as color reference to perform color correction of remote sensing images. We recommend using low-resolution satellite images with good color distribution and without interference from clouds and other non-ground objects as an external reference. The Google Earth imagery with the resolution of 200meter-resolution is the ideal external color reference. As shown in Fig 1 the external reference shows obvious summer color characteristics.

MATERIALS AND METHODS
The proposed method in this paper mainly includes three parts: (1) Fast atmospheric correction of the original remote sensing image .
(2) Replacing the low-frequency signal of the atmospheric corrected image with the low-frequency signal of the up-sampled external reference I REF ; (3) Stretching the high-frequency signal of images through local modeling. The flowchart of our method is as follows.
Upsampled external reference image … High-frequency signal .
Low-frequency signal .
Low-frequency signal .
Target image … AC src I Figure 1. Flowchart of the proposed method.

Fast atmospheric correction
The atmospheric transmission of an image is related to the scene depth and atmospheric density. Unlike ordinary nature scene images, satellite imagery is imaging from high orbit, so the changes of scene depth is small relative to the imaging height. Considering the atmosphere has the characteristic of homogeneous in local areas of remote sensing images, then each local area of the image has the same atmospheric correction parameters, so this study divides the image into blocks and detects the dark pixels of each block to obtain a dark channel map corresponding to the original image, as shown in Figure 2.
where, I src is the original image, I dark is the dark channel map, Ω(i,j) represents a local area centered on pixel (i,j). From Equation 4, we can use the dark channel map I dark to obtain the atmospheric transmittance map t of original images. The atmosphere has the characteristics of continuous, so the atmospheric transmission corresponding to the original image is smooth and continuity. Therefore, we use the method of bilinear interpolation up-samplig the transmission map t to the original image size to get the atmospheric transmission for each pixels of original images.
where BL is a bilinear interpolation operator. T is the upsampled atmospheric transmission map. Then we can obtain the atmospheric corrected image according Equation 6.

Replacement the low-frequency signal of images
From the frequency domain perspective, an image can be decomposed into high and low frequencies. High-frequency signal appears as rapidly changes in pixel gray values (such as edges, details and noise), while the low-frequency signal appears as slowly changes in pixel gray values (such as background and color) (Gonzalez, 2007). Therefore, this study proposes using the low-frequency signal of an image to simulate the color distribution of the image. The mathematical representation is as follows: where I represents an image, H and L represent the high and low frequency signal of the image, respectively.
In Section 2.2, we selected a low-resolution external image as the reference data. Here, we up-sample the external reference image to the same resolution with the original image by bilinear interpolation.
where BL is a bilinear interpolation operator, is the upsampled external reference image from . According to Equation 9, the atmospheric corrected image and external reference can be decomposed into two parts:  Figure 3(b) and (d) are the low-frequency signal of I src AC and I REF respectively. This study uses Gaussian filtering to separate the high and low frequency signal of the image.

Stretching the high-frequency signal of images
In the field of radiation correction, it is generally considered that the intensity of different remote sensing images conform to a linear relationship: We assume that each local area of remote sensing image has similar radiation characteristics and different areas satisfy with different radiation relationship. So for a local area there is: where I src AC (x) and I dst AC (x) respectively represent x-centric local area Ω( ) of atmospheric corrected image I src AC and the color corrected image I dst AC . a and b represent gain and bias, respectively. According to the theory of dark channel prior, if the color corrected image is clear, then there exist dark pixels in local area of the image. That is, when ( ) → 0 has ( ) → 0. Combining the above derivation and Equation 15, we get → 0, so there is: Assuming that the intensity of the entire local area satisfy with the above model, we have: Figure 3 is a schematic of color correction for an atmosphericcorrected image. It can be found from the enlarged detail that color of the image is corrected after low-frequency signals replacement, and the texture of images are enhanced after highfrequency signal stretching.

EXPERIMENTS, EXAMPLES AND RESULTS
We selected two sets of experimental data with color difference to test the proposed method, and evaluated the results from both subjective and objective. Both Experiment 1 and Experiment 2 use GF-1 images with the resolution of 2 meters as experimental data. The external reference comes from Google Earth with the resolution of 200-meters. Both the GF-1 satellite images and the external color reference are WGS-84 coordinate systems.

Experiment 1: Hills, 159 Gaofen-1 images (2m)
The Experiment data 1 is located in Wuhan, Hubei Province, China, covers about 50,000 square kilometres. The study area is mainly hills and contains large areas of water. Figure 5(a) is the overlay schematic diagram of input images and the external reference image in the study area. Figure 5(b) is the external color reference corresponding to the study area. As shown in Figure 5(c), there is obvious color difference among images in the study area, some images are seriously disturbed by atmosphere, and individual images show significant radiation distortion. This is a representative dataset to test the effect of the color consistency algorithm. It can be found from Figure 5(d), that the color consistency of the entire study area is improved after processing by our method, and there is no stitching traces of the mosaic image. Images in the study area shows as a summer color distribution consistent with the external reference data. Detail 1 is composed of multiple images containing a large area of water. After processing by our method, the color transition of different images at the edge of the water body is natural and there is no obvious mosaic trace. Detail 2 is composed of three image, where the right image is disturbed by fog and the image texture is blurred. After processing by our method, the clarity of the right image is significantly improved, and there is no obvious difference of clarity between adjacent images, indicating that the step of atmospheric correction of images is effective.

Example 2: Gobi, 6 Gaofen-1 images (2m)
Experimental data 2 is located in Qinghai Province, China. This area is mainly gobi with sparse vegetation, which can be used to verify the effectiveness of our method in low vegetation coverage area. As shown in Figure 6(a), there is obvious color differences of images in this area. Figure 6(b) is the external reference for this area, which show in yellow tones. From Figure  6(d), it can be found that the color consistency of this area is significantly improved after processing by our method, and color distribution of this area is consistent with the external reference image. Comparing Figures 6(e) and 6(f), we can find that after processing by our method the color difference of three adjacent images are obviously reduced and the clarity of images are significantly improved. Details 2 contains a seasonal lake. It can be found from Figure 6(a) and (b) that the lake in the original image is dry, but it shows water color characteristics in the corresponding external reference image. After processing by our method, the color of original image is consistent with the external reference image, and the color of the lake is corrected as blue, and there is a natural transition at the boundary of the lake and the land, which proved the validity of our method for different ground objects.

Quantitative experiment
In order to quantitatively evaluating the experimental results, we selected four pairs of satellite images with typical color differences from Example 1, including season (Img1, Img2), atmosphere (Img3, Img4), illumination (Img5, Img6) and sensor distortion (Img7, Img8) to quantitative analysis the effect of the proposed method. Fig 7 is the processing results of the above image, Table 1 is the statistical results of Fig 7. We counted the mean and standard deviation of the overlapping areas in Fig 7. The mean value reflects the whole tone of the area, and the standard deviation reflects the clarity of the image texture, the closer the two above, the higher the color similarity.
Observing the images we can find that the visual effect of images are improved after processing by our method. The color transition at the edge of adjacent images is natural and there is no obvious mosaic traces. Both the left and right images in Scene 1 were obtained in summer and winter respectively, after processing by our method both of the above images exhibit as summer tones. The left image in Scene 2 is disturbed by fog. It can be found that the clarity of the images are significantly improved after processing by our method, which verifies the effectiveness of the atmospheric correction in section 3.1. Both the left and right images in Scene 3 were obtained in the same season with brightness difference. After processing by our method, images has same brightness and clarity, indicating that the strategy of stretching the high-frequency signal images is effective. After processing by our method the radiation distorted right image in Scene 4 is corrected.
From Table 1, it can be found that the similarity of mean value is significantly improved after processing by our method, indicating that the overall tone of adjacent images overlapping areas are close. It is worth noting that the standard deviation of images after processing by our method are obviously better than the original images, indicating that the image texture information is enhanced.

CONCLUSION
This study proposes a novel color consistency method for optical satellite images utilizing low-resolution external references images. The proposed method does not depend on the overlapping area between images and has high robustness and computational efficiency, which is suitable for color consistency processing of large-scale satellite images. The main contributions of this method are as follows: (1) This study considers the influence of atmosphere on the clarity of optical satellite images and improved the dark channel prior defog method based on the atmospheric distribution characteristic of satellite images.
(2) This study uses low-frequency signal to simulate the color spatial distribution of remote sensing images and realized color correction of images by low-frequency signals replacement.
(3) This study uses a liner model to establish the relationship between high and low frequencies of images and the highfrequency signal of the image is stretched by local modeling.