A PROPOSED FRAMEWORK TO UNMIX SCATTERING MECHANISMS OF POLARIMETRIC RADAR IMAGES USING VERY HIGH RESOLUTION OPTICAL IMAGES

A new approach has been proposed for radar and optical image fusion. Candidate images for fusion are polarimetric radar images and very high resolution stereoscopic optical images. The fused product is expected to produce more reliable large scale land cover classifications. In the first part of this paper, observation-based state-of-the-art methods for optical and radar image fusion are discussed. The idea of unmixing physical radar scattering mechanisms with the optical images has been proposed as a suitable means to improve the resolution of the polarimetric radar images. The second part focuses on a general description of the target algorithm. The suggested physical approach simulating synthetic scattering matrices with stereoscopic optical images is developed. Finally, future work to develop this unmixing algorithm is discussed.


INTRODUCTION
1.1 Designing a land cover database Land cover is a layer of information of significant interest for land management issues.Therefore, there is a crucial need to develop methods for creating and updating land cover databases.Many countries such as France have decided to design and frequently update very high resolution stereoscopic optical data repositories for topographic databases.One of the research goals is then to design a large scale land cover database from these repositories.The approach detailed in (Bordin, 2011) puts forward the maintenance of a constant terrain partition, in order to follow the evolution of land cover through time.In this context, remote sensing images are considered as a possible and rich source of information to qualify the homogeneous areas.
Until now, aerial or satellite optical remote sensing images have been widely used to compute land cover classification, as they are useful to differentiate between ground objects on the basis of their spectral reflectance.However, since the stereoscopic optical images considered are acquired at a very high resolution (20 cm to 30 cm), two limits can be taken into account to retrieve land cover information.First, using a 20 cm pixel size to derive land cover classes may not be consistent with the desired scale of land cover classification.Then, downscaling the information may imply the use of high-level image processing techniques that complicate class labelling.
Moreover, land cover classes often take into account geometrical or geophysical properties of the objects to be classified, that optical images cannot be reliably used to estimate.Therefore, the objective of this research is to explore the use of polarimetric radar images in association with optical images.Since 2006, spaceborne polarimetric sensors such as Radarsat-2 or PALSAR have offered the opportunity to estimate geometrical or geophysical properties on large areas at a rather high resolution.Polarimetric radar images provide many advantages: the resolution of sensors is consistent with the desired application; their wide swaths and their weather condition independence are useful for acquisition.In addition, the fact that microwaves penetrate vegetation and are sensitive to the geometrical or geophysical properties of the objects make these images complementary to optical data.
Polarimetric sensors observe a scattering matrix that gives information about amplitude and phase of the backscattered signal.Usual radar image processing chains tend to decompose this matrix in order to identify the occurrence of scattering mechanisms ; coherent decomposition must be applied when polarisation information needs to be preserved.Understanding these mechanisms is challenging as they are mixed inside the radar cell resolution, but it is the key to producing a reliable land cover classification.

Toward super-high resolution polarimetric images
As a consequence, fusion between observations of different remote sensors has been explored in order to distinguish between radar scattering mechanisms and to produce sharpened images.Instead of considering independent sources of information involved in the fusion process, our aim was to develop cooperative methods in which optical data is used to input information in the radar product.The final objective will be to improve the understanding of radar images and to increase their resolution, so as to retrieve more information.(Pohl and Van Genderen, 1998) argued that fused images may provide increased interpretation capabilities and more reliable results since data with different characteristics are combined.This objective can be reached by producing a super-high resolution polarimetric product, which could be used for land cover classification purposes.

Necessary hypotheses
Papers giving a review of data fusion techniques insist on the importance of the co-registration of image candidates for fusion.(Gamba and Chanussot, 2008) underline that co-registration remains a crucial step in numerous applications and still attracts a lot of attention.Co-registration is all the more important when sensors are different.(Pohl and Van Genderen, 1998) reviewing of multi-sensor image fusion in remote sensing show the importance of geometric accuracy to avoid artefacts and misinterpretation in pixel-based image fusion.Sub-pixel accuracy is then recommended.Thus co-registration is a topic of research interest.Literature about co-registration of optical and radar images can be found in (Tupin, 2011).This issue is then especially important when designing sharpening techniques.As a result, data is assumed to have been correctly co-registered.We also consider that acquisition dates of optical and radar images are closed in time.Indeed, change detection is another very interesting issue that can cause distortions in the sharpening process.

Fusion level
This work is within the framework of data fusion.Defining this framework, (Wald, 1999), (Pohl and Van Genderen, 1998) and (Gamba and Chanussot, 2008) explain that there are three common levels for fusion.The lower level is the data level, concerning the observations of the sensors and related to physical measurements.Then, at an intermediate level, there is the feature level, which deals with characteristics or objects extracted from data.Finally the highest level is the decision level, that is about classification.Nevertheless, it is also appropriate to mention the output of the fusion process.Thus, for example (Dasarathy, 1997) shows that fusion can be proceeded on the data level to produce data, features or decisions.Features can be fused to produce new features or decisions, and decisions can be fused to synthesize new decisions.
As the objective is to design super-high resolution polarimetric images, our fusion framework stands on the data level in order to produce new observations.In this first part, data-based methods used to merge optical and radar observations will be reviewed.Feature and decision-based fusion methods were explored for optical and radar fusion.For example, (Gamba and Chanussot, 2008) explain that feature extraction and combination at a geometrical level is considered a possible common practice in the near future to improve classification.The fact that fusion occurs on the the objects ensure that the approach is correct.These methods have produced reliable results for extracting building outlines in urban areas ( (Tupin and Roux, 2003)).At the decision-level, sensors are considered independently.Merging decisions is also another correct approach for fusion.

Methods taken from pan-sharpening
Most methods used to fuse radar and optical data come directly from the work carried out to develop pan-sharpened images from panchromatic and multi-spectral optical images.As a result radar images are usually considered as a way to sharpen multispectral optical images.Although there are numerous pan-sharpening algorithms for optical images, (Klonus, 2006) and (Mercer et al., 2005) observe that there are few publications mentioning high resolution radar and low resolution optical fusion.Since 2006, even if some new radar sensors have appeared, the same statement can be made.
A detailed literature review of these pan-sharpening techniques can be found in (Jinghui et al., 2010).In this paper the author divided the methods into three categories : component substitution fusion techniques, modulation-based fusion techniques and multi-resolution analysis fusion techniques.In the following, we make an inventory of those methods applied to radar and optical image fusion.
Component substitution techniques Concerning the component substitution fusion techniques, (Jinghui et al., 2010) notice that the Intensity-Hue-Saturation (IHS) and the Principal Component Analysis (PCA) transforms are the two most frequently used techniques.It appears as well that these two techniques have most often been adapted to process optical and radar image fusion.For the IHS transform a color space is used: Intensity-Hue-Saturation (IHS).For optical and radar data fusion, the IHS transform has been used by (Balik Sanli et al., 2009) or (Chen et al., 2003).In both cases, the resolution of the radar image is greater than the optical one, and as a result radar is used to sharpen the multispectral images.In these papers, the intensity of the IHS color space is directly replaced by radar intensity.The PCA transform aims at creating new uncorrelated channels from initial observations.PCA transform has been used by (Ramadan et al., 2007) to fuse Radarsat-1 image and Landsat-TM optical images.In this case, the first component of the PCA transform is replaced by radar intensity.For IHS and PCA transforms, the end of the algorithm is an inverse transformation that retrieves R, G, B values of the sharpened product.

Modulation-based fusion techniques Modulation-based fusion
techniques tend to derive a synthetic panchromatic layer from multi-spectral images.A ratio is calculated between the observed and the synthetic panchromatic image resolution in order to calculate a gain.Several methods are used to calculate this synthetic panchromatic layer.Most of them consider a linear combination of the multi-spectral channels (Brovey, or Synthetic Variable Ratio (SVR)).Other techniques based on multi-resolution analysis can also be used to determine the synthetic layer (Smoothing Filter Based Modulation (SFIM), High Pass Filter (HPF) or wavelets decomposition) Multi-resolution analysis fusion techniques Multi-resolution analysis fusion techniques aim at decomposing the high resolution and low resolution images into a sum of functions of different resolutions.The sharpening is then achieved by replacing the high frequencies terms.One of the most frequently used methods for multi-resolution analysis is wavelet decomposition.For optical and radar image fusion, these methods have most often been used along with a component substitution method.(Mercer et al., 2005) propose a method combining IHS and wavelet decomposition.(Chibani, 2007) associates as well IHS and wavelet decomposition.

Critical review of these methods
In many of the previous experiments, color distortions have been observed.(Park and Kang, 2004) state that a fundamental problem frequently occurring in existing fusion process is the distortion of spectral information.With optical images, the problem may occur when sensors are different.This problem is then all the more important when radar and optical images are considered.Moreover, the techniques developed for optical pan-sharpening are inconsistent with fusing optical and radar images.
IHS is a perceptual color space that is related to the human visual system.(Park and Kang, 2004) mention that this method is widely used because of its simplicity and high sharpening ability, while it distorts spectral properties of multi-spectral images when the panchromatic image has little correlation with the intensity image.Indeed, this method has been used to fuse panchromatic and multi-spectral optical images because it is based on a physical principle: luminous intensity can be calculated from three RGB components since RGB stimuli are integrated into the cone cells.That is also why this method is limited to three channels for the multi-spectral data.As a consequence, the IHS method should not be used for data level fusion between radar and optical images because the physical principles cannot be extended to process radar intensities.
Modulation based techniques commonly use a linear combination of the multi-spectral channels to compute the synthetic image.As explained for the IHS, there is no physical principle to express a radar intensity as a linear combination of optical multi-spectral images.
Principal Component Analysis (PCA) tends to transform multispectral observations into new observations that are uncorrelated.For optical multi-spectral images, the first component is the one containing the most information.This information usually concerns spatial details in the images.Methods identified in the literature used to substitute the first component are inappropriate.The hypothesis assuming that contour lines of optical and radar images are highly correlated is not valid.
In fact, the inadequacy of these methods to merge radar and optical data have been noticed in the literature.The fact that IHS or PCA are not appropriate has been observed by (Zhang et al., 2010).They state that classical methods considering the high resolution band as a high correlation with the low resolution band such as in IHS and PCA methods are not suitable to fuse optical and SAR imagery.(Klonus, 2006) proposes a comparison of 8 methods for optical and radar data fusion.He concludes that most of the fusion methods are not capable of integrating radar data into optical multi-spectral data without color distortions.Some research has been carried out on the compensation of color distortions.Therefore, many recently introduced algorithms are focused on solving that distortion problem.(Zhang et al., 2010) propose a block-SVR method aimed at estimating weights by local regressions, taken into count when expressing the radar intensity as a linear combination of the multi-spectral channels of the optical data.(Gungor and Shan, 2006) developed a fusion algorithm that is intended to reduce these color distortions.With constraints on the local variance and local mean, color distortions are minimized in the fused product.We think that instead of designing statistical methods to minimize distortions, methods guaranteeing maintenance of physical principle of the observations have to be designed.

Spectral and Spatial Unmixing
Spectral and spatial unmixing methods have been initially designed for optical images and are based on the concept of the mixed pixel.They assume that each pixel contains more than one land cover class.Spectral unmixing techniques have been widely used on optical hyperspectral data.The recent advances in this area are presented in a special issue on Spectral Unmixing of Remotely Sensed Data in the IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING ( (Plaza et al., 2011)).Spectral unmixing aims at decomposing a pixel into a sum of endmembers weighted by their abundances.The endmembers are a set of pure spectral signatures and the abundances are the fractions of them ( (Plaza et al., 2011)).Linear spectral unmixing is the most commonly used technique for pixel decomposition.
Whereas spectral unmixing is calculated per pixel, spatial unmixing is processed in order to sharpen resolution.The objective is then to compute a spatial decomposition of the pixels.To locate the endmembers, a more resolved source of information is needed.The spatial unmixing method has been defined by (Zhukov et al., 1999).A framework to sharpen low resolution images with co-registered high resolution images is presented.The use of high resolution images is suggested to analyze the composition of mixed pixels of the low resolution sensor.The unmixing can be constrained or unconstrained depending on whether the initial observations can be retrieved.The method proposed by (Zhukov et al., 1999) is a multi-sensor multi-resolution technique.Vocabulary has been introduced : higher resolution images are considered as the classifying instrument and lower ones as the measuring instrument.The analysis of mixed pixels of the measuring instrument is done by classifying the higher resolution images.A synthetic response is then assigned to each class.(Park and Kang, 2004) mention unmixing-based methods defined in (Zhukov et al., 1999) as a possible means for the sharpening.(Zurita-Milla et al., 2007) used this spatial unmixing on optical images from different sensors : MERIS and Landsat TM.The images of greater resolution (Landsat TM) are classified, which makes the composition analysis of lower resolution pixels possible.An estimation of a synthetic MERIS reflectance is carried out on each class of the Landsat TM Classification.
The unmixing method seems particularly interesting for the radar sharpening algorithm to be designed.First, there is an analogy with spectral unmixing.Instead of spectral signatures, the objective is to retrieve different types of scattering mechanisms that are mixed.This method is valid as a scattering matrix can be expressed as the sum of basis matrices.Each basis matrix represents an elementary scattering mechanism ( (Cloude and Pottier, 1996)).These elementary scattering matrices are in fact polarimetric signatures.The spatial unmixing is also valid as the scattering matrix is a coherent sum of the contributions of the scatterers distributed in the radar resolution cell.Our proposed approach is then a mixture of spectral and spatial unmixing adapted to radar polarimetry.

PROPOSED ALGORITHM
State-of-the-art methods have pointed out the need to produce a radar image output, so as to maintain physical principles.If not, images should be processed independently, and fusion should be applied at the feature or decision level.Data fusion methods are available to combine these features or decisions.
Literature on data-based optical and radar fusion only mentions the case of sharpening optical images with higher resolution radar images.Sharpening polarimetric radar images with higher resolution optical images seems to be a topic that has not yet been studied.Moreover, the complex nature of polarimetric radar observations has not been taken into account in the literature.To sharpen radar images, we propose to apply unmixing techniques in which optical images provide information to analyze the mixed radar observations.Simulation of radar scattering mechanisms with optical images should guarantee the physical approach.The general ideas of the algorithm are described in this section.Diagrams are presented within the UML standard ((OMG, 2011)).

Fusion framework
The unmixing algorithm aims at decomposing the observed scattering matrix.The approach is to identify spatial occurrence of these mechanisms inside the radar resolution cell.The objective is then to simulate radar behavior with optical images.Unmixing the scattering mechanisms brings in greater information necessary for sharpening.A general framework that will be used for future work is presented.shows that radar observations can take three different states : the raw complex observations, the scattering matrix and the unmixed scattering matrix.The unmixing algorithm is then the activity decomposing the scattering into a sum of synthetic scattering matrices estimated with the optical images (1).This unmixing is constrained, so that the initial scattering matrix can be retrieved from the synthetic matrices.
[S] = N i=1 ai[Si] with 0 < ai < 1 and Polarimetric representation The scattering matrix [S] observed by the radar can be very different locally for a collection of scatterers distributed in the radar resolution cell.As a consequence, speckle can be fully developed.The scattering matrix gives information about amplitude and phase of the backscattered signal.Usual radar image processing chains aim at decomposing this matrix in order to identify the occurrence of the scattering mechanisms.Coherent decomposition must be applied when polarisation information needs to be preserved.The choice of the scattering matrix as the polarimetric description used for the fusion process, rather than an incoherent description only based on intensity, ensures that all information is maintained.Second order (incoherent) representations of the information are based on local averaging.As a consequence most information concerning the phase is then lost.
Unmixing Working with the scattering matrix implies that resampling has to be avoided.That is why the unmixing algorithm has to be processed in radar geometry.Homogeneous areas provided by optical images (2) will then be re-projected in radar geometry.

Produce Orthophotos
Produce DSM

Estimate attributes
Simulate synthetic scattering matrices

The unmixing algorithm
Optical images The first step is to derive layers of information that are helpful in the simulation process from the stereoscopic optical images.Segmentation is then applied on orthophotos.Because the stereoscopic properties of the images a Digital Surface Model can be produced.

Iterative method
We propose to develop an algorithm guided by the segments extracted from the optical images.For a coarse scale of segmentation, prior information can be obtained on the composition of the mixed scattering matrices.Then, there is a loop on the segmentation scale.A threshold on the scale of segmentation is considered to stop the simulation process.(Zhukov et al., 1999) noticed that the principal limitation of this classbased unmixing is that the measuring instrument is averaged over the total area of each class.Multi-scale segmentation is a solution for this issue as the general composition of the mixed radar observation can be obtained at a coarse scale of segmentation, and progressively the simulation of synthetic scattering matrices on the smaller segments refine the estimation.Multi-scale segmentation can also provide estimation of abundances of the different behaviors occurring in the radar resolution cell.
Estimation The next step is to estimate attributes on each segment that could be used for the simulation of the [Si] matrices.
Here is a short list of them that could be extracted from the optical images to synthesize the [Si] matrices (1).Local slopes extracted while creating the DSM are an important parameter as radar-facing slopes contribute to high backscattered signal.In vegetation areas occurrence of vegetation, number and localization of trees can be estimated.In urban areas, detection of buildings might be a relevant input parameter.Other parameters could be useful.For example, we will investigate if roughness estimated by optical data is somehow related to roughness observed by radar depending on the wavelength of the microwaves and the resolution of the optical data.

Simulation
The simulation aims at calculating [Si] matrices in equation ( 1).The first objective of the simulation will be to select a method for this calculation.Two methods could be used.The first one is to set up an exhaustive library of scattering mechanisms.As a consequence, optical images would be used to identify the most likely behavior.The other method is the use of a physical decomposition of the scattering matrix (such as Pauli decomposition).As a result, proportions of each type of mechanism could be estimated.Once the method is selected, the combination of different information provided by optical data has to be taken into account.We are currently investigating the theory of evidence to take a decision on the simulation of the synthetic [Si].Indeed, parameters such as local slopes or occurrence of vegetation are very reliable, whereas other parameters can be imprecisely estimated (such as number or trees).Modelling the imprecision of the parameters estimated by optical data then becomes significant for a reliable simulation.

CONCLUSIONS AND FUTURE WORK
In this paper, a framework for observation-based data fusion when sensors are very different has been presented.As a result, the method can be used to produce sharpened radar polarimetric images integrating information from very high resolution optical images.
The unmixing algorithm will be tested on real data on two different sites in France.The first area is a radar acquisition site in Brittany (Pleine-Fougères).This is mainly an agricultural zone with gentle slopes.On this site, stereoscopic aerial photographs from the IGN (French National Geographic Institute) have been acquired at 25 cm resolution with the Ultracam-x camera.DSM will be computed because of 60% forward lap and 40% side lap. 4 multi-spectral channels are available (blue, green, red and near infra-red).These images will be combined with Radarsat-2 polarimetric images acquired with the Quad-Pol mode at an approximate 11 × 9 m resolution.The second site is in Provence (Forcalquier).Ultracam-x images at 30 cm resolution and same overlaps are also available.This is a forested and semi-natural area with possible steep slopes.Radarsat-2 polarimetric images have also been acquired with the Quad-Pol mode.This site is particularly interesting since the students of the Ecole Nationale des Sciences Géographiques (school of the IGN) carry out each year field experiments.The validity of the estimation of attributes to simulating synthetic scattering matrices can then be assessed.Future work will be carried out developing the target algorithm.
• We hope to prove that homogeneous areas contribute to determined behavior of the scattering matrix.In particular, this has to lead to an evaluation of whether or not fully developed speckle can prevent estimating this behavior.Having considered optical image segmentation as a means of providing the homogeneous parts, we also have to validate whether or not these segments can correspond to determined polarimetric radar observations.
• Estimation of synthetic matrices with simulated attributes generated from the optical reference is the key to introducing greater information in polarimetric radar images.Once the above hypothesis is validated, we will have to choose the proper method for estimation.