DEVELOPMENT AND VALIDATION OF A NEW PASSIVE MICROWAVE BASED SOIL MOISTURE INDEX

Knowledge on the spatial-temporal variation of soil moisture is essential to many hydrometeorology applications. In this study, we proposed a new soil moisture index (SMI) from passive microwave observations, aiming to capture the soil moisture variability. The new SMI is developed based on the underlying physical basis that vegetation and surface roughness exert similar effects on the variation of land surface emissivity and microwave polarization difference radio (MPDI), but they act in an opposite way compared with soil moisture. Hence, we can obtain the SMI value in a two-dimensional space by combining use of land surface emissivity and MPDI to isolate the contribution of soil moisture and that of vegetation and surface roughness. We calculated the SMI by using the L-band SMAP Level-3 datasets and validated it with five well calibrated and dense soil moisture networks and also compared it with SMAP and ESA CCI soil moisture products. The results show the SMI exhibits the highest R (0.87) and lowest RMSE (0.028 m m) value after removing the systematic bias by using the cumulative distribution function (CDF) matching technique among the satellite products during the whole study period, thus demonstrating its good capability of tracking the temporal variation of soil moisture and its potential usage in various hydrometeorology applications. * Corresponding author


INTRODUCTION
Soil moisture is a key state variable that controls land surface evapotranspiration and energy and carbon transfer between the soil and atmosphere (Seneviratne et al., 2010). In situ measurements are believed to provide the most accurate soil moisture information. However, it is well known that soil moisture exerts large heterogeneity over both space and time since it is affected by many environmental factors (e.g., precipitation, vegetation, soil texture, and topography, etc.), and thus the limited in situ measurements cannot be the representative of soil moisture at large scales (Zeng et al., 2015a). Microwave remote sensing, especially the passive radiometers have been proved to be a promising and effective avenue to monitor soil moisture at global scales both temporally and spatially. Over the last few decades, various operational passive microwave-based soil moisture algorithms, such as the single channel algorithm (SCA), the dual channel algorithm (DCA), the land parameter retrieval model (LPRM), and the multitemporal dual channel algorithm (MT-DCA) (Jackson, 1993;Owe et al., 2008;Konings et al., 2016) have been developed, and the corresponding soil moisture products have been produced and released to the public. These products are derived from a variety of passive microwave satellites/sensors, including the C/X-band Advanced Microwave Scanning Radiometer-Earth Observing System (AMSR-E) and its successor AMSR2, the Windsat, and the Fengyun-3 (FY3), and the more recently soil moisture dedicated mission, i.e., the Soil Moisture and Ocean Salinity (SMOS) and Soil Moisture Active Passive (SMAP) operating at L-band (Wigneron et al., 2017;Cui et al., 2018). Extensive validation work for various satellite soil moisture products have been carried out in recent years (Brocca et al., 2011;Zeng et al., 2015aZeng et al., , 2016Al-Yaari et al., 2019;Ma et al., 2019). The results demonstrated that though the products are becoming mature, their performance varies over time and space, and there are still many uncertainties in the satellite soil moisture products. The two main perturbing factors which degrade the accuracy of soil moisture retrievals are vegetation and surface roughness (Wigneron et al., 2017). Since vegetation and surface roughness vary with time and space, auxiliary data are usually required to correct their effects in the soil moisture retrieval algorithms. The optical vegetation indices, such as normalized difference vegetation index (NDVI) or leaf area index (LAI) are often adopted to characterize the effects of vegetation. However, the optical derived vegetation water content (VWC) is not in accordance with the true VWC seen by microwave. This is because that the optical wavelength is much smaller than the microwave which can only represent the information of a very thin layer of the vegetation canopy, whereas the microwave (e.g., L-band) is able to penetrate vegetation with low to moderate coverage easily and thus denotes the information of vegetation stem and branches. Moreover, it is well known that the optical vegetation indices also easily get saturated in densely vegetated areas. Compared with vegetation, it is much more difficult to characterize the temporal dynamic of surface roughness. Since the surface roughness measurements are very scarce especially at a global scale, global or land-specific constant values are usually set for surface roughness in the soil moisture retrieval algorithms (Zeng et al., 2016). However, the constant value sets are not in line with the actual surface, and thus inevitably brings errors in the soil moisture retrievals. Therefore, how to decouple the influence of soil moisture and those of vegetation and surface roughness poses a technical challenge. On the other hand, there is an increasing interest to obtain the temporal variability of soil moisture since many previous studies have reported that for most applications (e.g., data assimilation), it is more important that the relative dynamics of soil moisture are reproduced rather than their absolute values (Reichle and Koster, 2004;Drusch et al., 2005). For instance, Koster et al. (2009) reported that the true information content of soil moisture data lies not necessarily in their absolute magnitudes but in their time variability. Under such background, the new European space agency climate change initiative (ESA CCI) soil moisture product was developed which aims to capture the temporal variations and trends of soil moisture by fusing both active and passive soil moisture products (Dorigo et al., 2017). The purpose of this paper includes three aspects: 1) to develop a new passive microwave-based soil moisture index (SMI) which aims to reproduce the temporal variations of soil moisture; 2) to use the newest L-band SMAP brightness temperature to calculate SMI since L-band is less influenced by vegetation and surface roughness compared with C/X-bands; 3) to decouple the effects of soil moisture and those of vegetation and surface roughness independent of auxiliary data as much as possible. Finally, we validated the new developed SMI with ground soil moisture measurements collected from five denselyinstrumented networks covering a wide range of ground conditions and also compared with the SMAP passive official soil moisture product and ESA CCI product.

SOIL MOISTURE INDEX
In passive microwave remote sensing of soil moisture, the three foremost perturbing factors are vegetation, surface roughness and surface temperature. For L-band SMAP and SMOS, the surface temperature is obtained from model simulations, such as Goddard Earth Observing System (GEOS)-5 model and the European Centre for Medium-Range-Weather Forecasts (ECMWF) model; while for multi-band sensors such as AMSR-E/AMSR2, the surface temperature is usually retrieved from high frequency band (e.g., the v-pol Ka-band) (Owe et al., 2008;Zeng et al., 2015b;Cui et al., 2018). Previous study found that though these surface temperature inputs have different bias in different regions, their overall accuracy is acceptable, particularly for the model simulations (Ma et al., 2019). Compared with surface temperature, it is much more difficult to characterize the effects of vegetation and surface roughness, and they are usually ignored or assumed as unchanged in previous soil moisture indices. Additionally, previous studies have found that vegetation optical depth (τ) and surface roughness (h parameter) affect the microwave emission similarly, and they can be grouped into one factor (e.g., exp(−2τ−h) in the τ-ω model) to reduce the unknown parameters (Zeng et al., 2015b). On one hand, surface roughness is considered to generally increase soil emissivity due to the increase in the surface area of the emitting surface (Wigneron et al., 2017). That is, the soil emissivity increases as the soil surface becomes rougher. Similarly, vegetation emits radiation itself though it attenuates and scatters the emission from soil surfaces, and thus enlarges the overall land surface emissivity (Mo et al., 1982). However, according to the Fresnel equations, soil reflectivity increases monotonically as soil moisture increases, and therefore the soil emissivity (1-soil reflectivity) decreases with the increase of soil moisture. On the other hand, it is found that microwave emission from a smooth flat surface at an incidence angle far from nadir is different at two polarizations (h-pol and v-pol), while the presence of surface roughness leads to the reduction of the polarization difference (Paloscia et al., 2018). Likewise, it is well documented that the polarization difference exhibits a decreasing trend as vegetation biomass increases, independently of the vegetation type (Santi et al., 2017;Paloscia et al., 2018). Consequently, the microwave polarization difference ratio (MPDI) is usually adopted to estimate the vegetation biomass in previous studies (e.g., Santi et al., 2017). Nevertheless, soil moisture significantly enlarges the polarization difference. That is the difference between v-pol and h-pol emissivity increases as soil moisture increases (Chen et al., 2018). In summary, vegetation and surface roughness have similar effects on the variation of both land surface emissivity and polarization difference ratio, but they act in an opposite direction compared with soil moisture, and thus make decoupling the effects of soil moisture from vegetation and surface roughness possible. Based on the theoretical background, we proposed a new SMI by combining use of land surface emissivity and MPDI from passive microwave observations, which aims to decouple the effects of soil moisture and those of vegetation and surface roughness to capture the temporal variation of soil moisture. Figure 1 shows the conceptual sketch of SMI, which is proposed based on two physical phenomena: 1) the MPDI exhibits a decreasing trend as vegetation and surface roughness increases (i.e., they both exert a depolarizing influence on microwave emission), while soil moisture enhances the polarization difference (see the y-axis); 2) vegetation and roughness generally impose positive effects on surface emissivity, while soil moisture and emissivity are negatively correlated (see the x-axis). Hence, we can use both MPDI and land surface emissivity in a two-dimensional space to isolate the contribution of vegetation and surface roughness. Another advantage of using MPDI and land surface emissivity is that the effects of land surface temperature are minimized. It is known that previous optical or microwave indices such as NDVI, radar vegetation index (RVI) or soil wetness index (SWI) are all ratio-based indices. In contrast, our proposed SMI is a distancebased index, calculated from emissivity and MPDI. The distance originated from e p (max) point on the x-axis to the scatters in Figure 1 denotes the SMI value. We see visually that larger distance means larger SMI value corresponding to higher soil moisture. Hence, the following mathematical formula was proposed to calculate SMI as: where the subscript p is the polarization (h or v), i denotes the original SMI value for a specific satellite pixel at i th day, e p (i) and MPDI(i) represent the corresponding emissivity and microwave polarization difference index at day i, respectively, e p (max) denotes the multiyear maximum value of e p (i) for a special satellite pixel. However, the original MPDI and emissivity have different scales (in land surface, the MPDI generally ranges from 0 to 0.1, while emissivity is in the range of 0 to 1). Therefore, we normalized the original emissivity and MPDI to a comparable scale from 0 to 1 as: where max and min denote the maximum and minimum values for e p and MPDI at a specific pixel for the entire study period.
Finally, we need a constant value C to normalize the final SMI in the range of 0 to 1, where 0 denotes the driest soil and 1 denotes the wettest soil as: where () f SMI i represents the final expression of SMI, and the subscript nor denotes the normalized parameters. Let's see the first term of equation (4)

Satellite data sets
In this study, we applied the SMI to the L-band SMAP radiometer at a grid resolution of 36 km. The SMAP L3 radiometer global daily 36 km EASE-Grid soil moisture product (SPL3SMP, V5) was adopted, which is available from the National Snow and Ice Data Center (NSIDC) (https://nsidc.org/data/SMAP/SMAP-data.html). In this product, the SMAP brightness temperature at two polarizations (h-pol and v-pol), effective soil temperature, and soil moisture (by using the SCA-V algorithm) were provided. Due to increased thermal equilibrium conditions of land surface during morning time, we used the h-pol and v-pol brightness temperature as well as the effective soil temperature (to calculate the land surface emissivity from brightness temperature) at SMAP descending pass (6 A.M. local solar time) to obtain the SMI values. We also adopted two widely used satellite-based soil moisture products for inter-comparison purpose. One is the SMAP official passive soil moisture products (36 km) retrieved by using the SCA-V algorithm, and the other is the ESA CCI soil moisture (V04.2) product which is a global daily soil moisture product with a spatial resolution of 0.25°. The ESA CCI datasets consists of three datasets, including the active, passive, and combined active and passive soil moisture products. In this study, the combined soil moisture product was used since it blends the advantage of both active and passive soil moisture products. For a consistent and fair comparison, we used the nearest interpolation method to resample the ESA CCI to a grid resolution of 36 km to be consistent with that of SMAP passive products.

Optimal polarization for SMI calculation
In equation (4), it is seen that SMI can be calculated by using the emissivity at a single polarization (h-pol or v-pol) or their combinations. To find the optimal polarization for SMI calculation, we compared the correlation coefficient (i.e., R) between the station-averaged ground observations from the five networks and SMI calculated by h-pol emissivity, v-pol emissivity, and their combinations, shown in Figure 2. It is seen that the three SMI values all correlate well with ground observations in the five networks with an averaged R value large than 0.8. Generally, SMI calculated by combining use of h-pol and v-pol emissivities exhibits marginally better performance than the SMI calculated by a single polarization. Accordingly, we adopted the SMI values calculated by combination of both h-pol and v-pol emissivities.

Validation of original SMI
We calculated the time-series SMI by using the SMAP brightness temperature and effective soil temperature. Figure 3 shows the temporal pattern of SMI and station-averaged soil moisture measurements from the five networks. We see the SMI well tracks the temporal evolution of soil moisture in all network regions which cover different land surface and climate conditions. Due to frequent rainfall, the soil moisture at the five network regions has a large dynamic range during the entire study period. Particularly, the precipitation concentrated in summer in the Tibetan Plateau, and thus we see the soil moisture increases markedly at the beginning of June and decreases steeply at the end of September in Naqu and Pali networks. It is observed that the SMI well capture this transition from wet to dry as well as from dry to wet. Table 2 lists the correlation coefficient of SMI as well as SMAP and ESA CCI soil moisture products with ground soil moisture observations in the five network regions. We see all the three datasets correlate well with in situ measurements, indicating that they can capture the temporal variability of soil moisture. The SMI exhibits the highest R value ranging from 0.71 to 0.97 with an overall averaged value of 0.87, followed by SMAP (0.80) and ESA CCI (0.69). The gray shadows displayed in Figure 3 indicate the soil frozen days. It is known that compared with the work in unfrozen soils, inferring soil moisture (more precisely, the soil liquid water content) from satellites during the frozen season remains largely unexplored. The liquid and frozen water can co-exist in the soil matrix up to several degrees below freezing point which depends on soil type, temperature and water content, and the in situ sensors are still sensitive to the liquid water in frozen soils (Wang et al., 2016). Some previous studies have confirmed the possibility of satellites to detect the soil liquid water content during frozen seasons (Wang et al., 2016;Zheng et al., 2019). It is seen that during the frozen period, the soil liquid water content is stable in Naqu and Pali networks since there was nearly no rain during this period in the Tibetan Plateau (Zeng et al., 2015a), and the SMI capture this phenomenon. To further illustrate the improved capability of SMI to capture soil liquid water content in frozen period, we calculated the correlation coefficient of SMI and MPDI which was also often used to detect soil moisture variation in previous studies (Chen et al., 2018) with soil liquid water content measured by in situ sensors during the frozen seasons at Naqu and Pali networks, shown in Figure 4. The R value for SMI is 0.70 and 0.83 in Naqu and Pali networks receptively, which is much higher than that of MPDI (0.27 and 0.51 in Naqu and Pali networks respectively), demonstrating the superiority of SMI over MPDI to capture soil liquid water content in frozen seasons. Nevertheless, it should be mentioned that due to the uncertainty of in situ measurements in frozen soils, the capability of SMI to detect soil liquid water content in frozen seasons should be further investigated and validated before drawing a solid conclusion. In addition, it is noticed that SMI and satellite soil moisture products show poorer performance in RISMA network, and the possible reasons may be that: 1) the land cover of RISMA is cropland, and previous studies found that satellite soil moisture products often showed higher accuracy in grassland region than in cropland region which may be due to the complexities of the cropland, e.g., the periodic row structure, diversity of crop types and the temporal changes in surface roughness and vegetation water content (Colliander et al., 2017); 2) the RISMA network has a cold climate, and a recent study has observed worse performance for SMAP soil moisture product in cold regions than in arid regions (Ma et al. 2017); 3) the layout of the in situ sites in RISMA is not as uniform as that in other four networks (not shown here), which may cause larger uncertainties from the ground measurements.

Validation of bias-corrected SMI
In section 4.2, we calculated the correlation coefficient between the original SMI and satellite soil moisture products with in situ measurements. Here, we used the cumulative distribution function (CDF) matching technique to further illustrate the effectiveness of the proposed SMI. The CDF matching approach is considered as an enhanced nonlinear technique for removing systematic differences between two data sets, and it has been demonstrated with various satellite soil moisture products (in unit m 3 m -3 ), and soil moisture indices (unitless ranging between 0 to 1 similar as our proposed SMI) (e.g., Brocca et al., 2011;Matgen et al., 2012). Thus, it was employed in our study to make a fair absolute comparison of SMI and satellite soil moisture products with in situ measurements. This is because systematic differences between satellite-derived soil moisture products/indices and in situ observations may exist and prevent absolute agreement between the time series of these data sets. There are several explanations for the observed systematic bias, and the most noteworthy ones are as follows (Owe et al., 2008;Zeng et al., 2015a): 1) different spatial observation scales between in situ points and satellite pixels, and 2) a mismatch between in situ soil moisture measuring depth and the microwave penetration depth. Figure 5 shows the temporal behavior of station-averaged soil moisture and biascorrected SMI (i.e., SMI-CDF) by using the CDF matching technique during the entire study period, and table 3 lists the root mean square error (RMSE) and R value of the SMI, SMAP, ESA CCI (all are bias-corrected) with ground measurements from the five networks. We see that the R values of the biascorrected SMI and satellite soil moisture products are nearly the same as those of original datasets, which is consistent with expectations since the CDF matching approach only eliminate the systematic bias between remote sensing derived and in situ data, while the temporal trends and dynamic of the satellite datasets are well preserved. The SMI-CDF agrees very well with in situ measurements with an average RMSE of 0.028 m 3 m -3 , lower than that of SMAP-CDF (0.039 m 3 m -3 ), and ESA CCI-CDF (0.052 m 3 m -3 ). The results demonstrate that the bias between two datasets can be easily corrected if they are well correlated with each other. This further highlights the importance for capturing the temporal variation of soil moisture which is the foremost purpose of SMI. In-situ_SM SMI-CDF (d) (e) Figure 5. Similar to Figure 3 but with bias-corrected SMI (i.e., SMI-CDF) by using the CDF matching technique.  Table 3. Root mean square error (RMSE, in m 3 m −3 ) and correlation coefficient (R, p-value < 0.05) of SMI, SMAP and ESA CCI with ground observations from the five networks during the whole study period. The cumulative distribution function (CDF) of all these products is rescaled to that of in situ soil moisture.

CONCLUSION
Capturing the temporal variation of soil moisture from passive microwave measurements has attracted increasing attention in recent years since many applications require information of soil moisture dynamics rather than the absolute value. In this study, we proposed a new physically-based SMI from passive microwave observations to capture the temporal variation of soil moisture. It decouples the effects of soil moisture from vegetation and surface roughness in a two-dimensional space by combining use of MPDI and land surface emissivity. The underling physical basis is that the vegetation and surface roughness exert similar effect on MPDI and emissivity, but they act in an opposite direction compared with soil moisture. It considers the temporal dynamic of vegetation and surface roughness which are often unreasonably assumed to be unchanged in other algorithms or indices. Furthermore, previous (soil moisture or vegetation) indices such as opticalbased NDVI, active microwave-based RVI, or passive microwave-based MPDI are all ratio-based indices (i.e., calculated by ratio), while the proposed SMI is an index based on distance calculation. To the best of our knowledge, this is the first distance-based index developed from passive microwave remote sensing, which may provide a new way to design indices for other applications. To demonstrate the effectiveness of the proposed SMI, we validated the SMI by using extensive in situ measurements from five dense networks covering diverse land surface conditions, and also compared with SMAP and ESA CCI soil moisture products. The results demonstrate that the SMI correlates the best with in situ soil moisture with an averaged R of 0.87 and has the lowest RMSE value of 0.028 m 3 m -3 during the whole study period after removing the systematic difference by using the CDF matching approach. In this study, we successfully applied SMI to the L-band SMAP radiometer, in the near future, we will continue this work to apply SMI to SMOS and AMSR2 to investigate the dependence of SMI on incident angle and frequency and its applicability to other satellites/sensors.