NOVEL CLASSIFICATION UNCERTAINTY MEASUREMENT MODEL INTEGRATING SPATIAL INFORMATION FOR REMOTE SENSING IMAGE

: Remote sensing image classification has important applications in many fields. However, the uncertainty of remote sensing image classification results will reduce its application value and reliability in these applications. Therefore, the uncertainty of remote sensing image classification results must be accurately and effectively measured. To address the shortcomings of the existing classification uncertainty measurement model in the utilization of image spatial information, this study proposes a novel uncertainty measurement model for remote sensing image classification, which considers the spatial correlation between pixels in images and the effects of local spatial heterogeneity during uncertainty measurement. Specifically, the proposed model first measures the classification uncertainty of an image at the pixel and local spatial levels on the basis of the posterior probability of image classification. Second, the local spatial heterogeneity of an image is quantified, and the proposed model uses the local spatial heterogeneity of the image as a weight to adaptively fuse the uncertainties of the pixel and local spatial levels. Accordingly, a joint uncertainty measurement index is generated for a more accurate and effective evaluation of the uncertainty of remote sensing image classification. Lastly, the classification verification experiments on three publicly available remote sensing images with different spatial resolutions confirm the validity of the proposed model. Moreover, experimental results show that the proposed model has relative superiority and better stability than the existing and commonly used uncertainty measurement models (e.g., information entropy and Eastman’s U) in improving image classification performance.


INTRODUCTION
The land cover thematic information obtained through remote sensing image classification has important application value in natural resource management (do Nascimento Bendini et al., 2019), disaster monitoring, urban planning, and decisionmaking (Cui et al., 2018;Mahdavi et al., 2019;Zhang et al., 2018;Zhang et al., 2017). However, the premise for the effective application of these land use classification products is that their reliability is sufficiently high (i.e., the classification uncertainty is sufficiently low) . In these applications, reliable classification results with low uncertainty from remote sensing images are important guarantees for scientific and reliable decision-making. Therefore, the effective and accurate evaluation of the uncertainty of remote sensing image classification results is crucial to the further application of these classification results (Chen et al., 2019;Ge, 2013;Khatami et al., 2017;Shi et al., 2015).
At present, some pixel-based indices or models are used to quantify the classification uncertainty of remote sensing images. The majority of these indices or models are proposed on the basis of the posterior probability of image classification (Bogaert et al., 2017;Giacco et al., 2010). For example, the most commonly used classification uncertainty index is the uncertainty criterion U (referred to as Eastman's U in this study) proposed by Eastman (Bogaert et al., 2017) and Shannon's information entropy (Shannon, 1948). Eastman's U measures the classification uncertainty of pixels in an image by calculating the degree to which the maximum probability deviates from the average probability in the probability vector; such a deviation depends mainly on the maximum posterior probability and the total number of classes in an image (Bogaert et al., 2017). Shannon's information entropy measures the classification uncertainty of pixels in an image from the perspective of information theory, and the large amount of information contained in the pixel implies a high classification uncertainty (and vice versa) (Shannon, 1948;Yang et al., 2017). Other classification uncertainty measures, such as probability residual and confusion index, can also be adopted (Burrough, 1986). Probability residual is the sum of all components, except the largest component, in the probability vector. The larger the residual, the greater the degree of classification uncertainty of the pixel will be. The confusion index has different calculation forms, which are often expressed as the ratio of the second largest probability to the maximum probability in the probability vector of the pixel, or the difference between the sum of all the probabilities and the difference between the maximum probability and the second largest probability (Burrough, 1986;Jiang and Zhou, 2018). In addition, Shi also defined four indices to describe the classification uncertainty of pixels, including absolute uncertainty, relative uncertainty, degree of mixing of pixels (degree of mixed pixels), and incompleteness of evidence (Shi and Ehlers, 1996).
However, the majority of these uncertainty indices or models only consider the posterior probability of a single pixel and disregard the spatial correlation between pixels in remote sensing images. Consequently, the accuracy and reliability of the measurement of image classification uncertainty may be reduced. Figure 1 shows that A and B are two different pixels in the image that have the same posterior probability output from classification. Pixel A is in a heterogeneous area while pixel B is in a homogeneous area. When using traditional uncertainty measurement models (e.g., Eastman's U and information entropy) to estimate the uncertainty of pixels A and B, the two pixels will have the same classification uncertainty. Evidently, this result is unreasonable in the uncertainty assessment of remote sensing image classification. The reason is that the probability that pixel B is correctly classified is generally higher than that of pixel A, particularly when the spatial features of the image are used to enhance the image classification (Zhang et al., 2018;. This result means that pixel B should have a lower classification uncertainty than pixel A. Figure 1. Two pixels A and B in the image (the same color represents the same classification posterior probability): pixel A is in a heterogeneous area and pixel B is in a homogeneous area.
According to the first law of geography, things or attributes that are close in geospatial space are similar (Lv et al., 2017;Tobler, 1970). In remote sensing images, this law indicates that pixels in an image often exhibit high similarity to their neighboring pixels . That is, the uncertainty of the pixel is also indirectly affected by the interaction of the adjacent pixels in the local spatial region. Therefore, when measuring the uncertainty of image classification, the spatial correlation and heterogeneity between pixels in the image must be fully considered and utilized to highlight the difference between the uncertainties of different pixels, which will improve the accuracy and reliability of the uncertainty measurement results.
To address the shortcomings of existing classification uncertainty measurement models, this study proposes a novel classification uncertainty measurement model that considers the spatial correlation and heterogeneity in remote sensing images. In particular, the proposed model first measures the classification uncertainty of the image at the pixel and local spatial levels. Thereafter, the image's local heterogeneity is used as the weight to adaptively integrate the uncertainty evaluation results at different levels. The proposed model can measure the classification uncertainty of an image more accurately and reliably than the traditional classification uncertainty measurement model. The reason is that the proposed model considers the effects of spatial correlation and heterogeneity in the image on uncertainty measurement.
The remainder of this paper is organized as follows. Section 2 proposes a novel classification uncertainty measurement model for remote sensing images. Section 3 briefly describes the validity verification schemes for the proposed uncertainty measurement model. Section 4 presents the experimental results and analysis on three test images and discusses the sensitivity of the parameters in the proposed model. Lastly, Section 5 draws the conclusions.

PROPOSED UNCERTAINTY MEASUREMENT MODEL
The majority of the traditional classification uncertainty measurement models only consider the classification posterior probability of a single pixel to evaluate its uncertainty. Moreover, these models disregard the influence of spatial correlation and heterogeneity on the classification uncertainty of pixels. Hence, the current study proposes a novel classification uncertainty measurement model that considers the spatial correlation and heterogeneity in remote sensing images. The proposed model is summarized by a detailed flowchart in Figure  2. As shown in Figure 2, the proposed model mainly involves four steps: (1) posterior probability assessment of image classification, (2) uncertainty assessment at the pixel level, (3) uncertainty assessment at the local spatial level, and (4) spatial heterogeneity-guided adaptive weighted fusion of the pixellevel and local-spatial-level uncertainties to generate the final uncertainty assessment results. The following section presents the details of each step. The evaluation of image classification uncertainty is generally based on the posterior probability output of image classification (Bogaert et al., 2017). At present, the posterior probability output of image classification (i.e., estimation of the probability of each pixel belonging to each of the classes) can be obtained through various means, such as probabilistic support vector machine (SVM), Bayesian, and maximum likelihood classifiers (Ge, 2013;Khatami et al., 2017). Among them, the probabilistic SVM classifier is one of the most commonly used methods because of its good performance in dealing with highdimensional remote sensing data with limited labeled samples (Melgani and Bruzzone, 2004). Therefore, the current study adopts probabilistic SVM to obtain the posterior probability output of image classification. The standard SVM model does not provide a posterior probability output. However, a solution was proposed for the posterior probability estimation of multi-class classification and has been implemented in LIBSVM (Chang and Lin, 2011). The probabilistic SVM classifier aims to obtain the probability ( p ) that each pixel in the image belongs to each class: where j p represents the probability that the pixel to be classified belongs to the j-th class and C represents the total number of classes in the image.
This study uses the "one against one" multi-class SVM classification strategy with RBF kernel to obtain the probability output for each class. The pairwise class probabilities ( ) of classes i and j can be obtained as follows: where f is the output of the standard SVM and A and B are estimated by minimizing the negative logarithm of the likelihood function using the training data.
The posterior probability ( coupling-based probabilistic SVM can be obtained by solving the following optimization problem: Detailed information on how to obtain a unique solution for this problem and probabilistic SVM are provided in related literature (Chang and Lin, 2011;Wu et al., 2004).

Classification Uncertainty Assessment at the Pixel Level
Eastman's U is one of the most commonly used and effective classification uncertainty index at the pixel level. Therefore, in this study, we use Eastman's U to measure the classification uncertainty of images at the pixel level. can be calculated as follows: where i pix U ranges from 0 to 1. A large i pix U implies a high classification uncertainty. That is, the corresponding pixel is more likely to be misclassified in the image classification.

Classification Uncertainty Assessment at the Local Spatial Level
As mentioned earlier, the traditional uncertainty measurement models disregard the influence of spatial correlation and heterogeneity on the measurement of pixel uncertainty, thereby decreasing the measurement accuracy of the classification uncertainty. Therefore, this study integrates spatial information into the measurement of classification uncertainty through local spatial uncertainty measurement to improve the accuracy and reliability of the uncertainty measurement. As shown in Figure  3, the uncertainty measurement at the local spatial level mainly includes three steps: (1) generating spatial information units for local spatial uncertainty evaluation, (2) assessing the posteriori probability of classification at the local spatial level, and (3) quantifying the classification uncertainty at the local spatial level. The specific process is as follows.

Generating spatial information units for local spatial uncertainty evaluation
In remote sensing image analysis, extracting regular image blocks centered on pixels is one of the simplest and most effective methods of utilizing spatial information in the image (Zhang et al., 2018). Pixels within a small image block tend to have similar spectral signatures and are generally likely to belong to the same class (Hao et al., 2019;Huang et al., 2019). Therefore, we establish spatial information units for local spatial uncertainty assessment by extracting regular image blocks (see Figure 3). For the i-th pixel i x in the image, we the image to evaluate the uncertainty of pixel i x at the local spatial level.

Posteriori probability assessment of classification at the local spatial level
To evaluate the classification uncertainty of pixel i x at the local spatial level, we must first obtain the posterior probability output of pixel   (5) where mn w represents the weight of the influence of pixel According to the first law of geography, things or attributes that are close in geospatial space are more similar than distant ones (Lv et al., 2017;Tobler, 1970). Therefore, we calculate the weight of each pixel in image block K i O according to the principle that the smaller the distance, the greater the weight will be. The calculation method of the weight is as follows:

Quantifying classification uncertainty at the local spatial level
Let the classification posterior probability of the image block where the value of i loc U ranges from 0 to 1.

Adaptive Fusion of Uncertainty Evaluation Results at the Pixel and Local Spatial Levels
The proposed model aims to adaptively fuse the uncertainty evaluation results at the pixel and local spatial levels to more accurately and reliably measure the classification uncertainty of images. Here, a weight matrix (W) is adaptively generated to guide this fusion process.
, where N is the total number of pixels in the image. The weight i W of pixel i x in weight matrix W is defined as follows: where i  represents the local spatial heterogeneity of pixel x is at will be. In general, when the heterogeneity of the local area where pixel i x is located is high, the reliability of the blockbased classification result will be reduced because the image block contains more heterogeneous information (Li et al., 2016). Therefore, a higher confidence should be given to the pixelbased classification result. This condition implies that the uncertainty evaluation results ( i pix U ) at the pixel level are more accurate and reliable. Thus, a higher weight ( i W ) should be given to the uncertainty evaluation result i pix U at the pixel level. Conversely, the higher the homogeneity of the local area, the higher the weight that should be given to the uncertainty evaluation result ( i loc U ) at the local spatial level. In this way, we can adaptively integrate the uncertainty evaluation results ( i pix U and i loc U ) obtained at the pixel and local spatial levels through weight matrix W to generate a more accurate and reliable uncertainty measurement result (i.e., a joint uncertainty measurement index). The joint uncertainty of pixel i x (i.e., its final uncertainty measure i FU ) is defined as follows: (1 )

VALIDATION SCHEME FOR THE VALIDITY OF THE MODEL
To clarify the validity of the proposed model, we test the proposed model using the classification-based verification scheme presented in the related literature . The main idea of this verification scheme is that if the proposed index (i.e., uncertainty measurement index generated by the proposed model) can be used to improve the accuracy of image classification, then the proposed index/model is beneficial and valid. Below, we present a description of this classificationbased verification scheme. In fact, the single pixel-based maximum posterior probability method can only guarantee that the obtained classification result is a mathematically optimal solution. The continuity of surface objects in geospatial requires the final classification results to satisfy spatial consistency (Cui et al., 2018), which is an inherent requirement of image classification. Therefore, each probability layer acquired by the posterior probability evaluation can be post-processed before the final classification map is generated using Equation (13 To prove the validity of the proposed model, we adjust the calculation method of the weight in traditional spatial filtering. After adjustment, the weight is determined by the magnitude of the uncertainty rather than the spatial distance. In particular, the higher the classification uncertainty, the smaller the weight will be. The new weight ( n w  ) of the n-th pixel after adjustment can be calculated by Equation (16). neighborhood. After weight adjustment, distance weighted-based spatial filtering becomes uncertainty weighted-based spatial filtering. In theory, the probability of a pixel with high uncertainty being misclassified is relatively high. Therefore, in the uncertainty weighted-based spatial filtering, the influence of uncertainty on the classification results can be reduced by giving small weights to pixels with high uncertainty, and reliability of the classification results can be enhanced to some extent. The enhancement of the reliability will contribute to the improvement of the accuracy of the classification results.
Next, we also use Equation (13) to harden the new probability output layers obtained after filtering to generate the final classification map. Lastly, the classification accuracies of the unfiltered original classification map and filtered classification map are quantitatively compared. If the accuracy of the classification map obtained by the uncertainty-based filtering is higher than that of the unfiltered original classification map and the classification map obtained by the distance-based filtering, then the proposed uncertainty measurement model and index are useful and effective.
In addition, to prove the superiority of the proposed model over the existing uncertainty models/indexes, we also compare the proposed model with the commonly used uncertainty models/indexes (i.e., information entropy and Eastman's U). Specifically, the proposed uncertainty index (i.e., FU in Equation (16) generated by the proposed model) is replaced by information entropy and Eastman's U for spatial filtering based on uncertainty weighting. Thereafter, the corresponding classification maps are generated, and the classification accuracies of these classification maps are compared and analyzed. The difference in accuracy of these classification maps will relatively reflect the differences in the validity and accuracy of these uncertainty indexes/models in measuring classification uncertainty.

Experimental Data and Settings
This section tests the validity and superiority of the proposed model on three publicly available remote sensing images with different resolutions: Pavia University, Vaihingen, and WuhanTM images. Detailed information on these three images is presented as follows.
The Pavia University image was gathered by the Reflective Optics System Imaging Spectrometer (ROSIS-03) sensor during a flight campaign over Pavia in northern Italy. The image has 610 × 340 pixels (Fauvel et al., 2013). The original image contains 103 bands, with a 1.3 m/pixel spatial resolution. In the actual experiment, the false color composite of the Pavia University image is used, which consists of channel numbers 10, 27, and 46 for red, green, and blue, respectively. Figure 4 shows the false color composite of the image and the corresponding ground reference map. Nine land-cover classes are considered in the experiment, as shown in the legend. The Vaihingen image was obtained from the Vaihingen dataset of the RS image semantic segmentation dataset, which was published by the International Society for Photogrammetry and Remote Sensing (Rottensteiner et al., 2012). The Vaihingen image has a resolution of 9 cm and consists of three bands: near-infrared, red, and green. The size of the image is 1100 × 950 pixels. In addition, the image scene has four main categories: impervious surface, buildings, low vegetation, and trees. Figure 5 shows the Vaihingen image and its ground truth. The WuhanTM image is a 30-m resolution multispectral Landsat TM image of Wuhan City, China, with a size of 400 × 400 pixels and comprising 6 bands (Ma et al., 2015). The selected region of the image is expected to contain five classes: river, vegetation, lake, bare soil, and building, which numbered 2577, 4098, 1559, 3037, and 1666, respectively, in the ground truth image. The original Wuhan TM image and its ground truth are shown in Figure 6. In the experiments, when probabilistic SVM is used to obtain the posterior probability outputs of each image, the spectral and texture features of the images are used. Spectral features here refer to all spectral bands of the experimental images. The texture features refer to the mean, variance, homogeneity, contrast, dissimilarity, entropy, angular second moment, and correlation extracted from each band of the original image in accordance with the grey level co-occurrence matrix (Agüera et al., 2008) with a window size of 3 × 3.
When classifying the three images with probabilistic SVM, 3% of the ground truth data of each image are randomly selected as the training samples, and the remaining 97% are used as the verification samples.

Experimental Results and Analysis
To prove the validity and robustness of the proposed uncertainty measurement model, we performed three experiments on the Pavia University, Vaihingen, and WuhanTM images according to the validity verification scheme described in Section 3. Figures 4(c), 5(c), and 6(c) are the original classification maps (called OCMs) obtained from the original posterior probability layers of the three images through the maximum posterior probability method. Figures 4, 5, and 6 ((d) to (g)) present the classification maps generated after spatial filtering using spatial distance, Eastman's U, information entropy, and the proposed uncertainty index as weights, respectively. These generated classification maps are correspondingly recorded as CM_Distance, CM_Eastman's U, CM_Entropy, and CM_Proposed. Table 1 presents the result of the accuracy evaluation of these classification maps by using Overall Accuracy (OA) and Kappa Coefficient (KC). Table 1 shows that in the three experiments, the accuracies (OA and KC) of the filtered classification maps are significantly better than those of the original classification map. Moreover, regardless of OA and KC, the classification accuracy of CM_Proposed is the highest, thereby implying that the classification uncertainty model and index proposed in this study are valid. The classification accuracy of CM_Proposed is higher than those of CM_Eastman's U and CM_Entropy. This result indicates that compared with the commonly used uncertainty indexes (i.e., information entropy and Eastman's U), the proposed model/index has relative superiority in terms of improving the accuracy of image classification.
As mentioned in Section 3, distance-based spatial filtering is one of the most commonly used post-processing methods for image classification. Therefore, we use CM_Distance obtained by distance-based spatial filtering as a comparison criterion to further compare the proposed uncertainty index with Eastman's U and information entropy. Table 1 shows that the proposed classification uncertainty index has stronger stability in improving the performance of image classification than Eastman's U and information entropy. The reason is that regardless of OA and KC, the classification accuracy of CM_Proposed generated by using the proposed index is higher than that of CM_Distance in all three experiments, but CM_Eastman's U and CM_Entropy are not. The classification accuracies of CM_Eastman's U and CM_Entropy are lower than that of CM_Distance in the Pavia University and WuhanTM images, but higher than that of CM_Distance in the Vaihingen image.

Discussion of the Parameter Sensitivity
In the proposed uncertainty measurement model, the K*K local spatial information unit must be established to evaluate the classification uncertainty of the local spatial area where the target pixel is located. The change in parameter K may affect the result of the uncertainty measurement. Therefore, this section tests the sensitivity of the proposed model to parameter K to further enhance the practical value of this model. Figure 7 shows the OA trend of the classification maps obtained according to the method discussed in Section 3 as parameter K changes. Figure 7 indicates that as parameter K changes, the magnitude of the change in OA is not apparent on the three experimental images with different spatial resolutions. That is, the change in parameter K has minimal effects on the proposed uncertainty measure model.

CONCLUSIONS
The reliability of remote sensing image classification results is critical to the further application of the classification results. Therefore, the uncertainty of image classification results must be accurately and effectively evaluated. At present, the measurement models of the uncertainty of remote sensing image classification results is generally based on the posterior probability output of single pixels and disregards the spatial correlation and local heterogeneity in the image. In view of the shortcomings of the existing classification uncertainty measurement models, this study proposes a novel classification uncertainty measurement model that considers spatial information of the image by effectively integrating the classification uncertainties at the pixel and local spatial levels.
In particular, the proposed model first quantifies the classification uncertainty of the image at the pixel and local spatial levels and evaluates the local spatial heterogeneity of the image thereafter. Subsequently, the proposed model adaptively fuses the uncertainty evaluation results obtained at the pixel and local spatial levels with the local spatial heterogeneity of the image as the weight and generates the final comprehensive uncertainty measurement result (i.e., a joint uncertainty measurement index). Lastly, the classification verification experiments on three public remote sensing images with different resolutions confirm the validity and superiority of the proposed model.
In future research, we will focus on developing effective uncertainty control methods or reliable remote sensing image classification strategies based on the results of uncertainty assessment to improve the reliability of remote sensing image classification.