Title Optical and SAR data integration for automatic change patterndetection

Automatic change pattern mapping in urban and sub-urban area is important but challenging due to the diversity of urban land use pattern. With multi-sensor imagery, it is possible to generate multidimensional unique information of Earth surface features that allow developing a relationship between a response of each feature to synthetic aperture radar (SAR) and optical sensors to track the change automatically. Thus, a SAR and optical data integration framework for change detection and a relationship for automatic change pattern detection were developed. It was carried out in three steps: (i) Computation of indicators from SAR and optical images, namely: normalized difference ratio (NDR) from multi-temporal SAR images and the normalized difference vegetation index difference (∆NDVI) from multi-temporal optical images, (ii) computing the change magnitude image from NDR and ∆NDVI and delineating the change area and (iii) the development of an empirical relationship, for automatic change pattern detection. The experiment was carried out in an outskirts part of Ho Chi Minh City, one of the fastest growing cities in the world. The empirical relationship between the response of surface feature to optical and SAR imagery has successfully delineated six changed classes in a very complex urban sprawl area that was otherwise impossible with multi-spectral imagery. The improvement of the change detection results by making use of the unique information on both sensors, optical and SAR, is also noticeable with a visual inspection and the kappa index was increased by 0.13 (0.75 to 0.88) in comparison to only optical images.


INTRODUCTION
As hundreds of thousands of people are migrating from rural to urban area every year, land cover/use classes in urban and suburban areas are changing rapidly and this trend is likely to increase in future.In addition to that, several human interventions such as agricultural practice, deforestation, reforestation, dam construction etc. make a big changes in the Earth's surface.Thus, continuous monitoring is very important in several aspects including infrastructure planning and development to environmental monitoring, etc. Change information detected from the multi-temporal remote sensing images is seemed to be extremely useful (Dierking and Skriver 2002;Hayes and Sader 2001;Liao et al. 2008;Mishra and Susaki 2013;Du et al. 2013).Mainly, optical and radar images have used for change detection independently as well as in a combination with each other or with ancillary dataset.In case of optical image, the normalized difference vegetation index (NDVI) is the major index while the change in vegetation is a major concern (Lyon et al. 1998;Forkel et al. 2013).However, while considering all kinds of changes the change vector analysis (CVA) with Tesselled cap transformation is one of the most common approaches (Malila 1980) for multi-spectral images.The multi-sensor images, especially optical and SAR images, capture unique signature for each ground feature.Such information creates new research scope to enhance the change detection and labeling automatically.Accordingly, to use the complementary information from multi-sensor images, several data fusion techniques have already been in practice.Data fusion of multisensor optical imagery has been exploited widely.Majority of such fusion techniques is motivated to pan sharpening (Dong et al. 2013;Gangkofner et al. 2008;Amolins et al. 2007;Du et al. 2013;Koutsias et al. 2000).Even though, SAR and optical image fusion is not widely ex-ploited in comparison to the multi-sensor optical images, some good approaches are already in practice.The motivation behind these fusion approaches is also to enhance the spatial resolution by preserving spectral information (Du et al. 2013;Hong et al. 2009).In addition to that, SAR and optical image fusion is driven from better land cover classification or some specific structure detection.Tupin and Roux (2003) have used the SAR and optical data for building outline detection using feature based fusion approach in one of their studies.Their study showed that SAR images are capable to show the building presence and optical images are good for the shape delineation complementary information about building presence and proper shape extraction.They carried out it in two steps: first, extraction of partial potential building footprints on the SAR image and then shape detection in the optical one.Hong et al. (2009), proposed a fusion method based on wavelet-IHS transformation for SAR and optical multi-spectral (MS) images that was mainly motivated to preserve the spectral information of MS images and spatial detail of high resolution SAR image.(Hong et al. 2014), in another work for grassland and alfalfa segmentation, the same fusion technique was implemented.The fusion results gave spatial details of relatively high spatial resolution of SAR imagery and spectral detail was obtained from Moderate-resolution Imaging Spectroradiometer (MODIS) image.Major concern was again to improve the spatial resolution.As presented, several data fusion techniques are available, which allow better analysis and interpretation by making use of complementary information.Very few fusion works were inspired by the change detection (Du et al. 2012;Du et al. 2013;Hong et al. 2009), however none of them were motivated from automatic change pattern detection.Multi-class change detection based on CVA on MS images are available (Malila 1980;Johnson and Kasischke 1998)  the number of discriminated classes are limited , the CVA is good approach for making using of MS information.Consequently, it could be a very good approach for information fusion that is obtained from optical and SAR imagery.It is known that the unique signature of SAR and optical images for each land use/cover feature is stable and site independent, in the similar weather and light condition for optical imagery and same configuration in case of SAR images, it is possible to develop a relationship between them and can deploy for an automatic change pattern detection.In this study, an empirical relationship is developed by using the unique response from major features in the Earth's surface in SAR and optical imagery and deployed for automatic change pattern detection.Before that, a change area is segmented through CVA based SAR and optical information fusion.The fusion is motivated to use the complementary information without losing the inherent information that comes either from SAR or from optical images for better change detection.Specifically, it is expected to improve the sharpness of the detected feature, or be able to detect the changed features that were otherwise not possible from a single data source.The data used in this study are described in Section 2. Section 3 explains about statistical analysis.Section 4 reveals the methodology followed.The results and discussion is reported in Section 5. Finally the conclusions are presented in Section 6.

Study area
For an experimental purpose, a section of approximately 19×20 km in an outskirts part of the Ho Chi Min City was selected.Figure 1 shows the study area.The major events occurred in the area was constructions, deforestation and smoothing of agricultural land that causes the changes on agricultural land to bare land (preparation for construction), forest to bare land, bare land or agricultural land to built-up area or under-construction area.These are believed to be the major changes while expanding the urban area all over the world; therefore, study poses a sufficient generality.

Data used
HH component of two fully Polarimetric Synthetic Aperture Radar (PolSAR) images acquired by the Advanced Land Observing Satellite (ALOS) Phased Array type L-band Synthetic Aperture Radar (PALSAR) in April 2007 and April 2011 were used.Similarly, the Landsat-7 band 3 (Visible Red) and band 4 (Near InfraRed) acquired by the Enhanced Thematic Mapper Plus (ETM+) images nearly the same date as PALSAR images were considered.Table 1 shows the detail of all images used in this study.Since all images used in this study were acquired on nearly the same time of year (April), all the changes due to agricultural practices were ignored.Additionally, the different intensity of precipitation may cause the various levels of vegetation growth even in the same season of the year, thus the years (2007 and 2011) with normal precipitation records were selected for the study.Hence, all the phenological changes were also ignored and focused solely on the change due to the human intervention.The results were evaluated based on Advanced Visible and Near Infrared Radiometer type-2 (AVNIR-2) optical data acquired nearly the same time with PALSAR images and a very high-resolution (less than 1 m) QuickBird images in Google Earth.

STATISTICAL ANALYSIS
The backscattering coefficients and NDVI value of five major features (water body, bare land, grassland, forest and building) in the  Earth's surface were obtained by taking the average value of the sample obtained manually from the known area.In each feature type, the sample pixels were more than 1000, and assumed that this signature is site independent.Figure 2  While generalizing these five features, we considered grassland, forest and agricultural land (with crop plantation) as a vegetation area and identified the following possible change types.Inundation (vegetation, built-up or bare land to water bodies) and vice versa (water body to vegetation, built-up or bare land), bare land to vegetation and vice versa, bare-land to built-up and vice versa, vegetation to built-up and vice versa.Statistical analysis has been done for NDVI and SAR backscattering responses in earlier and later imagery for the above-mentioned possible change types and presented in Figure 3.Some change types are equally sensitive to the SAR and optical sensors e.g.vegetation to bare-land and vice versa, some have reverse effect such as: vegetation to builtup and vice versa and some are sensitive to one sensor whereas not in another, such as building construction in a bare land or building to bare land change.Similarly, some greenery appears in grassland or pastureland is not sensitive in some SAR sensor with relatively longer wavelength.Therefore, the complementary information available in multi-sensor images paves the way for further analysis.

METHOD
The process flow diagram for the optical and SAR image fusion for change detection and automatic pattern detection is presented  4.1.2PALSAR images -geometric correction and coregistration: All images were geometrically corrected using 30 m ASTER Global Digital Elevation Model (GDEM) using ASF Map-Ready 3.2.The images were geo-coded with Universal Transverse Mercator (UTM) system and co-registered with Landsat imageries with 19 ground control points selected manually in ENVI 5.0, where the overall error was less than a single pixel.The nearest neighbors re-sampling was used at this stage.
where, At 1 and At 2 are amplitudes of co-registered images acquired on two dates t1 and t2, respectively.

NDVI difference (∆N DV I) image:
The NDVI gives the vegetation greenness, and thus it is very useful to study the surface dynamics.NDVI at date t for Landsat TM/ETM+ is defined as Equation ( 2) where ρ3 and ρ4 are reflectance of TM/ETM+ band 3 and 4, respectively.The difference of NDVI, ∆ NDVI, is derived by Equation (3):

Fusion of NDR and ∆ NDVI for change detection
As discussed in Section 3 some changes are sensitive to both sensors however, others are sensitive in only one.Therefore, they have some complementary information, which are important for full dimensional change detection.We devise two different data fusion techniques in order to make use of complementary information that can capture all changes.4.3.2Change vector analysis (CVA): Change vector analysis is a well-established change detection method for multi-spectral images (Malila, 1980;Johnson and Kasischke, 1998).Even though the CVA is well-accepted methodology for multi-spectral images, it is new for optical and SAR integration.
For all land cover/use status, we assume that the quantity of land cover/use status in optical and SAR response, (f ), can be expressed as follows: where N denotes N DV I obtained from optical sensor and B denotes backscatter from SAR, respectively.When we take a partial derivative of Equation ( 4) with respect to t, Equation ( 5) is derived: Assuming N and B are independent to each other, amplitude of the change, A, can be written as: Now, we assume f as a simple linear function in Equation ( 7) Equation ( 6) can be rewritten as Equation ( 8): By adding another assumption that |a1| = |a2|, Equation ( 9) is derived:   2. As the NDVI and the backscattering intensity for all major land cover features are known and assumed to be stable and independent to the locations, the developed relationship is believed to be valid all over the World.

Accuracy assessment
The effectiveness of the proposed fusion method was evaluated with visual analysis, and quantitative capability.A visual comparison of the change image generated from different sensors and proposed fusion techniques and corresponding change map was done for the selected change sites, this gave the overall idea of the effectiveness of the generated change images.In addition to that, the change detection map obtained from proposed methodology was evaluated with the change map obtained from the highresolution AVNIR images and very high-resolution images from Google Earth interactively in selected areas.
In order to evaluate the results quantitatively, confusion matrix was used.This allocates the change and no change class and its expected value is derived using those in a corresponding ground reference data set.The confusion matrix allows deriving numerous summary measures of the accuracy of the allocated classes and amount of change that has occurred.The considered accuracy measures are user's accuracy, producer's accuracy, error of omission, error of commission, overall accuracy and kappa index (Foody 2010).

Change detection
The change map was generated through the proposed fusion techniques.The obtained results were compared with the results obtained from ∆NDVI, NDR and widely used multi-spectral change vector analysis (CVA) for Landsat imagery (Malila 1980;Jhnson and Kasischke 1998).Threshold values for each of the input change images was obtained with MTEP and implemented in an ENVI 5.0 that segmented the changed area from no-change area.
For the visual analysis, a false color composite of Landsat imagery was used.ation of all commission error from both sensors.In contrast to that, the NDR and NDVI integration using CVA approach is better.See in site (iv) the overestimation seen in water body while using optical imagery was reduced and the site (ii) is also reasonably better in comparison to NDR and optical imagery.The commission error in SAR is reduced in site (ii) and omission error in site (iv).Table 3 summarizes the accuracy assessment done in this study.The Kappa index is improved by 0.16 and 0.17 while using the proposed CVA based fusion approach in comparison to the NDR image, and ∆NDVI respectively.It is improved by 0.13 while comparing the CVA in Landsat imageries with significant decrements in false and missing alarm.
In general, the SAR image can detect almost all kinds of changes except the small changes that do not make much differences in the surface roughness, e.g.small vegetation; however, the performance improvement is significant.This is because the results obtained from the SAR image have lack of clear boundary line in most of the detected sites.This limitation can easily be overcome by using SAR and optical image fusion operation.In addition to that, several changes related to urban extensions are not sensitive to greenness and brightness for example bare land to built-up area and some vegetation changes such as forest to bush or grassland could not be detected properly in optical images.Similarly, some water body with different level of turbidity is also appearing as changed in the tasseled cap transformation.As a result, the false change is appeared in the generated change map.Additionally, building structure in bare land that does not alter the greenness, wetness and brightness significantly is not possible to detect.All of these errors of commissions and omissions can be reduced considerably while implementing the CVA technique with NDR and ∆NDVI.

Automatic multi-class change labelling
The change map developed through the CVA based SAR and optical information fusion approach was subjected to automatic change labeling.The results obtained from the relationship presented in Table 2 suggested that the increase or decrease in NDVI without altering NDR is very rare.Those changes, which do not alter the surface roughness significantly, such as bare land to pasture land or grassland and vice versa, which are characterized as increased or decreased vegetation are shown in Figure 9; this includes the boundary line of the change areas, mainly due to the changes in vegetation.Here, two examples are presented, (i) site 1, that is decrease in NDVI smoothing of some agricultural area that is associated with decrease in vegetation area (Zone VIII in Figure 2) and (ii) increase NDVI area, growth of small vegetation/greenness, that is associated with increase vegetation area (Zone IV as in Figure 6).Thus, these zones were merged with associated zones i.e. (Zone VIII to Zone I and Zone IV to Zone V as in Figure 6).Now we have six change classes and one no-change class as with the generalized relationship presented in Table 4.
In order to compare the results of the proposed change labeling approach with optical and SAR information, an automatic labeling with optical imageries using tasseled cap transformation brightness and greenness index (Malila 1980;Johnson and Kasischke 1998) 4) were classified in a single class 3 (decrease NDVI and increase brightness) in the optical imagery based on the brightness and greenness index obtained from the Tasseled cap transformation in Landsat 7 images.These are the major change classes in the urban extension; therefore, the change labeling using the optical information in an urban information is suffering from a poor performance.Such misclassification obtained while implementing the brightness and greenness   index is due to the lack of enough information in considered index.These indexes are highly correlated negatively (-0.8).i.e. decrease in the greenness increases the brightness.On the other hand, the NDR and ∆NDVI are linearly independent (0.33), thus, they can have more combinations of classes and poses to discriminate several classes successfully.
Regarding the other combinations of optical imagery derived indices such as brightness vs. wetness or greenness vs. wetness, all are linearly dependent to each other and they have a very good positive or negative correlation coefficient.Add to all, only the optical imagery does not provide enough information for discriminating several classes automatically.Thus, a quantitative accuracy assessment is not done for the automatic change type labeling using optical imageries.Table 5 illustrates the accuracy assessment of the change labeling using NDR and ∆NDVI.The obtained overall accuracy is 87.97% and the Kappa index is 0.78.The results obtained from optical imagerys are better in several aspects, like shape delineation or tracking vegetation dynamics and many others, but, several changes, including bare land to urban extension or forest to agricultural or bushes or pastureland changes are not detected properly.In contrast to that SAR image derived index, NDR, is very good to locate such changes.Yes, SAR images are not good at delineating the proper shape of the changed objects and do not detect these objects, which do not alter the surface roughness significantly, such as bare land to pastureland or grassland.The combined use of the SAR and optical images would be very effective to detect the change area .

CONCLUSIONS
With the availability of multi-sensor data, a multi-sources data processing and analysis technique is required to capture all changes.
The CVA technique for information fusion proved its capability of fulfilling their requirement for change detection.Given a huge potential of multi-source data, continue expansion of the quantity of diverse sensor types of remote sensing data, CVA might provide a capability of fusion of increasing demand of multi-source information for full-fledged change detection and a relationship among the responses of the Earth surface feature's to these sensors would provide a broader-dimension of change type detection.In addition to the change detection in a very complex urban sprawl area, an automatic multi-class change detection with an empirical relationship between the response of surface feature to optical and SAR imagery has shown to be effective.By further analyzing the response of each change feature to optical and SAR imagery or using ancillary dataset, this method can be further extended for disaster monitoring, crop monitoring, etc.In addition to that, an automatic adaptive thresholding would enhance the results by protecting from the human biases and error and make the system fully automatic.

Figure 1 :
Figure 1: Study area, false color combination of Landsat image.
(a) represents the NDVI for major five features, and Figure 2(b) represents the backscattering coefficients of HH polarimetric component for the same features.

Figure 3 :
Figure 3: The backscattering coefficient and NDVI in pre and post image with several land cover change classes.
4.3.1 Decision level fusion: Decision level fusion is common for multi-sensor image fusion, specifically in SAR and optical imagery and motivated from classification.In this study, we have developed a change map through thresholding of both change images independently, namely NDR image, that was derived from two multi-temporal SAR amplitude images from Equation (1), and ∆NDVI image, derived from two multi-temporal NDVI image generated from the Equation (3).Union of the detected changed area was carried out to get the final change map.The Figure 5. (a) represents the procedure for the change detection process using decision level fusion.
9) Now, ∆N = N DV I(t2)−N DV I(t1) = ∆N DV I and ∆B = B(t2) − B(t1) ≈ N DR(t1, t2), expressed by Equation (1).Then, Equation (9) can be rewritten in the form of ∆N DV I and NDR as follows: A = ∆N DV I 2 + N DR 2 (10) Equation (10) represents a change magnitude from both optical and SAR images.A threshold value in this image was identified with manual trial and error procedure that can segment change and no-change area.The overall procedure is presented in 5 (b).

Figure 5 :
Figure 5: SAR and optical information fusion procedure, (a) Decision level fusion and (b) CVA based fusion.
Figure 7 represents the false color composite of Landsat imageries in (a) 2007 and (b) 2011 and (c) and (d) are the interested zoom-in sites corresponding to the images acquired on 2007 and 2011 respectively.These figures and interested zoom-in sites were considered as a ground truth and the results obtained from each input change image were compared with a simple visual inspection.Figure 8 illustrates the change image, corresponding change map and zoom-in change map in interested sites corresponding to the interested sites in Figure 7 for all input datasets.Figure 8 (a) (c) represents the change vector magnitude (CVM) from tasseled cap transformation of Landsat-7 ETM+, corresponding change map and zoom-in map of the interested areas, similarly Figure 8 (d) (f) are for the NDVI, Figure 8 (g) (i) are for the NDR, Figure 8 (j) (l) for proposed CVM generated from ∆NDVI and NDR and Figure 8 (m) (n) are for the union of change map obtained from
was implemented.The Figure 10 (a) is the change labeling map using the proposed optical and SAR information fusion and Figure 10 (b) is the change labeling map obtained using the optical imagery only.While interpreting the resulted map visually, all the area classified as class 2, class 3 and class 4 (vegetation or bare land to built-up and decrease vegetation area according to the relationship in Table

Figure 9 :
Figure 9: Change area with no-change in NDR, (a) reference image in 2007, (b) reference image in 2011 and (c) change map with the change that is not sensitive to SAR backscattering (NDR) and interested zoom-in sites.

Figure 10 :
Figure 10: Change map with change type labeling, (a) CVA with NDR and ∆NDVI, (b) CVA with brightness and greenness obtained from tasseled cap transformation in Landsat images.
but the discriminated classes are very limited due to the lack of enough information in MS images.Even though, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume II-7, 2014 ISPRS Technical Commission VII Symposium, 29 September -2 October 2014, Istanbul, Turkey

Table 1 :
Data used

Table 3 :
Change detection accuracy assessments for several approaches.

Table 4 :
Generalized relationship between NDR and ∆NDVI with land use/cover change type.

Table 5 :
Confusion matrix for automatic change labeling in CVA -NDR and ∆NDVI.