A NOVEL APPROACH TO SUPER RESOLUTION MAPPING OF MULTISPECTRAL IMAGERY BASED ON PIXEL SWAPPING TECHNIQUE

Mixed pixels could be considered as a major source of uncertainty through classification process of satellite imagery. In this regard, the use of soft classifiers is often inevitable in order to increase the accuracy of land cover estimates. Although soft classifiers provide detailed information for each pixel, spatial arrangement of sub-pixels remains unknown. Super Resolution Mapping (SRM) has opened up a new horizon to produce finer spatial resolution maps using the outputs of soft classifiers. Wide variety of SRM algorithms has been developed. In this way, spatial optimisation techniques are the most applicable ones. However, random allocation of sub-pixels and iterative procedure of optimisation are the main limitations of current methods (e.g. Hopfield Neural Network, Simulated Annealing). This research attempts to provide an optimisation approach based on the pixel swapping technique in order to simplify the concept and to reduce the iteration procedure. In this paper, a brief survey is conducted on spatial optimisation based techniques of SRM. SVM and SMACC are used cooperatively to produce fractional maps as an input of SRM algorithm. The initial allocation of the sub-pixels is performed nonrandomised based on the highest amounts of attractiveness. An optimisation procedure is proposed to transfer the multiple allocated sub-pixels to the non-allocated ones. This procedure usually stops with minimal iterations and is time effective. The proposed method is tested on multispectral imagery (Landsat ETM+ and Quickbird) and has demonstrated precise results particularly in boundary pixels.


INTRODUCTION
Accurate and reliable information on land cover is of particular importance for environmental planning and management issues (Tatem et al., 2002).Thus, reduction of uncertainty is central in different stages of remote sensing process in order to enhance the accuracy of information derived from satellite imagery.In this way, mixed pixels affect the classification stage as a prominent source of uncertainty.Without solving the problem of mixed pixels, the full potential of remote sensing as a source of land cover information will remain unrealised (Foody, 2006).Then, soft classification techniques (e.g.linear spectral unmixing) are developed to provide detailed information within mixed pixels.Soft classifiers enhance the accuracy of classification by estimation of the fractions of different classes within a pixel.However, the spatial distribution of sub-pixels remains unknown.Therefore, while soft classification conveys more information than hard classification; the resultant predictions still contain a large degree of uncertainty (Tatem et al., 2002).Super Resolution Mapping (SRM) techniques are suggested to solve this problem (Atkinson, 1997;Tatem et al., 2002).SRM techniques provide the capability of spatial arrangement of sub-pixels in order to reduce uncertainty of sub-pixels from spatial point of view.Super Resolution Mapping (SRM) is a method that divides a coarse resolution pixel to finer pixels and prepare classified map in finer resolution (Verhoeye and De Wulf, 2002).SRM techniques based on spatial optimization (e.g.pixel swapping, spatial simulated annealing and Hopfield neural network) could be considered as well-known methods.Random initial assignment of sub-pixels and iteration process of optimization are the major problems of current methods.In this paper, a new method is proposed for non-random allocation of subpixels and its optimisation procedure.On the other hand, the accuracy of soft classification results impacts significantly the proficiency of the SRM algorithm.Therefore, in this research a method based on integrating the SVM classifier and unsupervised endmember extraction algorithm of SMACC1 (Gruninger, 2004) is proposed to extract the fractional maps.This paper is organized as follows.A brief overview of SRM techniques is presented in section 2. Section 3 presents the proposed methodology.The results of empirical implementation are presented and discussed in Section 4. The paper is finalized in Section 5 presenting some conclusions and outlooks.Atkinson (1997) introduced the SRM techniques for reduction of spatial uncertainty in the results of soft classification.The SRM algorithms provide spatial arrangement of the sub-pixels using the proportions of different classes within a pixel.So they enhance the spatial resolution of the classification results in comparison to the original image.The input of these algorithms is fractional maps produced by a soft classification method (Figure 1).Generally, accuracy of the SRM depends on the accuracy of classification method and spatial dependency between pixels is used only after finding fraction of each class (Kasetkasem, et al., 2005).Wide variety of methods has been developed for soft classification of satellite imagery.For instance, linear spectral unmixing (Foody and Cox, 1994;Garcia-Haro et al., 1996), softened maximum likelihood (Bastin, 1997), multi-layer perceptron (Atkinson et al., 1997), fuzzy c-means (Bezdek et al., 1984;Foody, 1996) and Support Vector Machines (SVM) (Brown et al., 1999) are the well-known methods in this area.The output of soft classifiers is fractional maps corresponding to each one of classification classes.Against the hard methods of classification, each pixel may be assigned to more than one label.However, the observance of two constraints of positivity and sum to unity of proportions is a challenging issue in soft classification process.These constraints are necessary and decisive factors for SRM algorithms.Towards this end, the combination of SVM and SMACC is proposed for soft classification of images as the input of SRM algorithm (Section 3.2).

Spatial Optimization Techniques
Generally, the SRM methods could be categorised in two basic types of regression and spatial optimisation algorithms (Atkinson, 2008).However, the second type is the most emphasised and applicable technique of SRM.

Regression Spatial Optimisation
Geo-statistical methods (Boucher and Kyriakidis, 2006), extensions of the linear mixture model (Verhoeye and De Wulf, 1999), feed-forward back-propagation artificial neural network (Mertens et al., 2004) Spatial pixel swapping (Atkinson, 2005), spatial simulated annealing (Atkinson, 2004), Hopfield neural network (Tatem et al., 2001) Table 1.Two main categories of SRM techniques (adopted from Atkinson (2008)) According to the first law of geography, everything is related to everything else, but near things are more related than distant things (Tobler, 1970).This theory forms the concept of spatial dependency.The spatial optimisation techniques of SRM have been mainly developed based on the maximising the spatial dependency.The class proportions obtained from the soft classification covers a certain number of sub-pixels within pixels.In spatial optimisation techniques of SRM, the subpixels have been arranged so that the spatial dependency between them maximises.Spatial pixel swapping, simulated annealing and Hopfield neural networks are the well-known methods in this area.Initial random allocation of sub-pixels and iteration procedure of the optimisation are common features of these algorithms.

Data
The implementation of proposed method will be carried out on simulated images and also multispectral imagery (Landsat and IKONOS).

Integration of SVM and SMACC
The results of the soft classification techniques used in the SRM should be such that the sum of the class proportions for each pixel becomes one and also negative proportions should not be estimated for any class (Atkinson, 1997).Regarding the limitations of existing methods of soft classification, the observance of both mentioned constraints for all of the pixels is often challengeable.In this paper, a two-step process is considered for the soft classification of images.First, the classes of interest are classified using the supervised SVM classifier.Then, with the knowledge of the number of interest classes, the rule maps produced by SVM classifier are used as the input of SMACC endmember extraction algorithm (Figure 2).The output of the SMACC algorithm satisfies both constraints of positivity and sum to unity of class proportions.
The main objectives of integrating the SVM and SMACC algorithms can be summarized as follows: a) In the case of using a soft classification algorithm (e.g.SVM) individually, two above mentioned constraints for the SRM could not be satisfied for all of the pixels.b) The SMACC algorithm is an unsupervised approach.In the case of applying this algorithm individually, a certain number of endmembers could be extracted that may be different from the interest classes.c) By integrating the SVM and SMACC algorithms, the both advantages of being supervised and satisfying the both constraints of positivity and sum to unity of class proportions could be preserved.
Figure 2. The proposed method for soft classification

Proposed SRM Method
The simple pixel swapping technique proposed by Atkinson (2005) (Goovaerts, 1997;Atkinson, 2005).The proposed pixel swapping method allocates randomly classes to sub-pixels initially and maximises the spatial dependence through an iterative procedure (Atkinson, 2005).In this paper, a SRM technique is proposed that the initial allocation of sub-pixels is non-random and the optimisation procedure is quite different.However, the basic idea is to maximise the spatial correlation between neighbouring sub-pixels under the constraint that the original pixel proportions were maintained.An image with m×n pixels is given.By applying zoom factor (Z), the number of Z 2 sub-pixels will be produced for each of the pixels.So, a certain number of sub-pixels could be allocated to the classes according to their relevant proportion within a pixel.So, the Number of Sub-pixels Per j th Class (NSPCj) for each pixel is determined.Total number of NSPCj for all of the classes in each pixel would be equivalent to Z 2 .Respectively all sub-pixel positions for each pixel numbered as i =1,2,..., z 2 , and a binary variable xi,j (sub-pixel (i) and class (j)) is defined so that the values one and zero are used respectively for the presence and the absence of the class (j) in sub-pixel (i).First, for every sub-pixel (i) the attractiveness to the class (j) is determined as a distance weighted function of its neighbours (Equation 1, Figure 3).The binary variable xi,j should follow these constraints: The constraint (3-b) prevents the presence of multiple or nonallocated sub-pixels.Keeping these assumptions in mind, the procedure of initial allocation and optimization is proposed at the rest of the paper.

Initial Non-Random Allocation of Sub-pixels
Calculating the Wi,j, a matrix W will be produced for each one of the interest classes.For initial allocation of the sub-pixels, the sub-pixels with the highest amounts of Wi,j will be chosen in every matrix W relevant to the corresponding class as many as the calculated NSPCj for the pixel.Then, the variable xi,j for the selected sub-pixels is considered one and for the others zero.For instance, we assume Z = 5 and the sub-pixel proportions of the classes A, B and C, are 10, 7 and 8 subpixels respectively within a desired pixel.Initial allocation of the sub-pixels is represented by the binary variable xi,j (see Figures 4-a, 4-b and 4-c).Through initial allocation, if every sub-pixel allocates to just one class (Equation 3-b), the SRM process for the pixel is accomplished well.However, some sub-pixels may be multiple/non-allocated (as for the above example, see Figure 4d).To solve this problem, a simple optimisation procedure is proposed in the next sub-section.The optimisation procedure is performed separately for each one of the pixels to ensure to preserve class proportions within them.First, the multiple/non allocated sub-pixels are identified.Then, the values of attractiveness Wi,j are sorted.The variable xi,j corresponding to the maximum value of Wi,j is preserved in a multiple allocated pixel (xi,j =1).The value of the variable xi,j corresponding to the second maximum value of Wi,j swaps with a non-allocated sub-pixel (1 to 0, and 0 to 1).This procedure is continued until every sub-pixel allocated to only one class (Equation 3-b).The overall trend of the proposed SRM algorithm is shown in Figure 5.
Figure 5. Proposed optimisation procedure of the multiple/nonallocated sub-pixels

EMPIRICAL RESULTS
The proposed SRM algorithm is applied on both simulated and real images.The simulated coarse resolution image is produced by degrading the original image applying a 5×5 averaging filter (Figure 6-a).Then by applying the SRM algorithm, the finer spatial resolution image is produced (Figure 6-b).The IKONOS and Landsat ETM+ imagery also are used to examine the proficiency of the proposed method.In this regard, the images are classified by hard classifier (Figure 7-a, 7-c) and the results are compared with finer spatial resolution map produced by the SRM algorithm ( In the case of simulated image, the input of SRM algorithm could be considered without any error.So, the overall accuracy of the super resolved map is determined 98.44% using the original image as reference image (just a few sub-pixels are allocated wrong.).The accurate evaluation of the proposed method on the real image requires sub-pixel ground truth.However, it is visually comprehensible that the super resolved map is considerably finer than hard classified one.

DISCUSSION AND CONCLUSIONS
SRM techniques provide land cover information in a finer spatial resolution using the outputs of soft classifiers.The outputs of soft classifiers used in SRM should satisfy both constraints of positivity and sum to the unity of class proportions within the pixels.In this paper, the integration of SVM and SMACC algorithms is considered to produce The allocation of sub-pixels could be considered as the main part of the SRM procedure.Current methods of SRM (e.g.pixel swapping) are often based on the random initial allocation and an iterative optimisation procedure.In this paper, a new method is proposed with non-randomized initial allocation.Towards this end, the number of sub-pixels corresponding to proportion of each class is determined for a certain zoom factor.Then the sub-pixels with the highest amounts of attractiveness (Wi,j) are considered as initial allocation (xi,j=1).In this situation, some of the sub-pixels may consist of more than one class or not to be allocated.So, the optimisation procedure is proposed to transfer the multiple allocated sub-pixels to the non-allocated ones.In the case of multiple allocated sub-pixels, the class with maximum value of attractiveness is preserved in the sub-pixel and other classes are transferred to the non-allocated sub-pixels with the maximum attractiveness.The proposed method is implemented on simulated and real images.The results indicate that it has a promising proficiency; especially in boundary mixed pixels, the proposed method is caused to a remarkable enhancement in spatial resolution.
The proposed SRM method is founded on maximising the spatial dependency between pixels.However, the initial allocation of sub-pixels is non-randomised and the optimisation procedure is performed with minimal iteration.This method is simple and is mostly applicable for high resolution imagery.
Another key issue to be considered is the use of noise removing filters to avoid the very partial class proportions.The results of soft classifiers most likely contain some noises which lead to a small number of sub-pixels in the image.So, the post processing techniques (e.g.majority analysis) could reduce such noises.The accuracy of SRM techniques is influenced by a variety of factors such as the accuracy of soft classified images, zoom factor, the number of neighbouring pixels, and the optimisation algorithm.Evaluation of their different aspects requires conducting more researches which are remained as open issues for the future works.

Figure 1 .
Figure 1.The overall procedure of SRM 2.1 SRM Input: Soft Classified Image Figure 3. Determination of the attractiveness of the sub-pixel (i) the to the class (j)The objective function (O) is defined as follow:

Figure 4 .
Figure 4. Initial allocation of sub-pixels according to the maximum values of Wi,j: (a) class A, (b) class B, (c) class C, (d) there are some multiple and nonallocated pixels.3.3.2Optimization of Multiple/Non-Allocated Sub-pixels
Wi,j for multiple allocated subpixels where xi,j =1 Make xi,j =1 for a non-allocated subpixel with max Wi,j of the same class fractional maps.This procedure satisfies the mentioned constraints in a supervised framework.