EARLY DETECTION OF FOREST DROUGHT STRESS WITH VERY HIGH RESOLUTION STEREO AND HYPERSPECTRAL IMAGERY

The project ‘Application of remote sensing for the early detection of drought stress at vulnerable forest sites (ForDroughtDet)’ is funded by the German Federal Agency of Agriculture and Food and aims to detect drought stress in an early phase using remote sensing techniques. In this project, three test sites in the south and middle part of Germany are selected. Three levels of observation and analyses are performed. In the first level, close-range stereo images and spectral information are captured with a research crane in Kranzberg forest. In the second level, three study sites are imaged twice in three years by airborne hyperspectral and stereo cameras. In the third level, the drought stress detection approach will be transferred to regional scale by satellite image. In this paper, we will briefly report our results from the first and second levels. In the first level, 3D models of the forest canopies are generated with the MC-CNN based dense matching approaches, with which the 3D shapes of the stressed and healthy trees are analysed. In addition, for the spectral analyses, different chlorophyll-sensitive indices are calculated and compared for the stressed and healthy trees. In order to further analyse the tree drought stress in the second level, a novel individual tree crown (ITC) segmentation approach is proposed and tested on the airborne stereo dataset.


INTRODUCTION
Climate change, especially global warming, is assumed to increase the frequency, intensity and duration of drought periods in Mid-Europe. The objective of the ForDroughtDet project is to identify drought stress symptoms of forest trees comprehensively and automatically by using remote sensing techniques. Drought stress is one of the main threats to the tree health. It directly affects the tree growth speed and the productivity of the forest. In addition, forest under drought stress is more sensitive to fire and various kinds of diseases (Klos et al., 2009).
Remote sensing-assisted monitoring of tree drought stress is motivated by a wide variety of applications relating to the forest management (Trumbore et al., 2015). It has been proven, that trees under drought stress will exhibit decreased water and chlorophyll content of the leaves, which will slowly influence the 3D structure of the leaves and tree crowns (Deshayes et al., 2016). In the previous studies, spectral features and spectral combinations are used to distinguish the stressed trees from healthy trees (Anderson et al., 2010;Kim et al., 2011;Buddenbaum et al., 2015). However, this procedure is quite slow, and it could be detected either in the late phase of the drought stress of large regions, or in a well-controlled laboratory or green house.
The ForDroughtDet project strives to achieve this in a way that is as pre-visual as possible, or in a still reversible state. The early detection of drought stress and understanding the performances of various trees species under drought stress will provide early warning for the forest management and give additional suggestions for the future forest stand planning. The system under development should identify forest sites with drought stress endangered trees in advance of a drop out, information required by forest management for the preferential start of restructure measures. Thus both of the spectral and 3D structure changes of trees under drought stress are monitored and analysed in this project.
The known spectral signature reaction on drought stress is regarded as main source of information and should be quantified on behalf of the physiologic morphologic measurements provided by the hosting Kranzberger Forest Roof Experiment (KROOF) (Pretzsch et al., 2014), which is an ongoing project of Technical University of Munich (TUM). In the KROOF project, trees under drought stress are located in an outdoor environment with novel, rain-controlled roof closure. Another hypothesis of the investigation concept is that water deficit will result in a change of leaf and needle orientation and shape (shrinking, rolling, leaf angle and exposure change) which might be detectable by exploring anisotropy information from appropriated air-or spaceborne sensors.
Within this project, data are recorded, merged and evaluated using various systems (hand-held stereo cameras, airborne stereo camera, and airborne hyperspectral cameras). Matching algorithms, which reflect the current state of research, are modified and extended with the aim of improving the quality and accuracy of the 3D treetop models derived from the recorded data. This development work takes into account different tree crown types and tree species. Tree-specific descriptors are derived from the highly accurate 3D models, which serve as indicators for drought stress. Furthermore, the combination of 3D models with properties derived from hyperspectral data enables a more robust tree species classification.
The research framework is covering the scale range from on site to air and space level, including basic data base generation, upscaling, system fusion, reflectance model adaptation and inversion for prognosis provision. At the 'on site' scale, field gonio-spectrometric measurements are performed with the GonioTree device across the vegetation period providing the BRDF models for beech and spruce trees under artificial drought stress (400-1000 nm), as well as phenologic spectral libraries of forest trees (400-2500nm) (TUM). The simultaneously taken stereo images of the crown segment of the GonioTree measurements are analyzed to derive biometric parameter like leaf area distribution (leaf angle and exposure, size) as well as descriptive parameters assumed to control the directional reflectance (shade and sun lit fraction. etc) (DLR). This information is employed for reflection model adaptation (UTr). The study site is imaged twice the year by hyperspectral HySpex systems (DLR, UTr) which are operated together with stereo data imaging systems (3-K, SONY low cost system). Stereo data is delivering singletree crown delineation and shade/sun lit fractions. Hyperspectral imaging is used for tree species identification and status assessment. By inversion of the developed drought stress symptom sensitive reflection models, the detection of drought stress prone patches within larger forest areas should be possible. The approach integrates existing geoinformation from the forest sector with spectral, height and anisotropy information from remote sensing systems of the latest generation in a spatial-temporal context (4D approach). Anisotropy measurements are still a fundamental challenge in remote sensing and must be considered for qualitative data evaluations. A new aspect of the ForDroughtDet project is the exploration of anisotropy features as additional, independent information source.

Test regions
Within the 'ForDroughtDet' project, two flight campaigns have been carried out in 2016 and 2018, respectively. The selected three test regions are the Kranzberg Forest, which is the main test region of KROOF, as well as one region in Gramschatzer forest and the forest near Traunstein. All three test regions are typical European forests, which are comprised mainly of European beech (Fagus sylvatica) and Norway spruce (Picea abies). The close-range photogrammetry dataset and spectral measurement are acquired by using the instruments as shown in Figure 1.

3K dataset
The airborne optical sensor system 3K (Leitloff et al., 2014) is composed of three Canon EOS 1Ds Mark III and provides multi-view, very high resolution (VHR) images (one nadir, two across track off-nadir). This allows generating VHS Digital Surface Models (DSMs). Details about the calibration and accuracy assessment of the camera system could be found in Kurz et al. (2007Kurz et al. ( , 2009). The DSM is generated using the DLR image processing platform CATENA (Krauß et al., 2013). The basic dense-matching algorithm is semi-global matching, which has been described in our previous studies (d 'Angelo, 2016).
The test data was acquired in 2016 and 2018 at the flying height of 1000 m. The 3K camera system covers an area of 2560 × 480 meters with a ground sample distance of 13 cm. Figure 2 shows the DSMs and the corresponding orthophoto over the KROOF test region. Three plots (deciduous (1), mix (2) coniferous (3)) were selected in the vicinity of the KROOF project. In the plot (2), the crane is visible in both DSM and orthophoto.

Hyperspectral dataset
The hyperspectral images were acquired by imaging spectrometer system HySpex (Köhler, 2016). HySpex system is designed by Norwegian company Norsk Elektro Optikk A/S (NEO). With two separate sensors, the system covers visible near-infrared (VNIR) and short wave infrared (SWIR) spectral domains ranging from 0.4 to 2.5 nm. Hyperspectral images are orthorectified with DSMs generated from 3K dataset. The spatial resolution of HySpex data is 0.7m for VNIR-1600 and 1.4m for SWIR-320m-e. The system is equipped with a high precision iTraceRT-F200 coupled INS/GPS navigation system that provides accurate georeferencing for the acquired data. A calibration flight was carried out after installation onto an aircraft over an area with known reference points (Köhler, 2016).

Close-range photogrammetry data
A stereo photogrammetric system, consisting of two digital high-resolution handheld cameras (SONY ILCE-5100), is used to acquire the data. The system is mounted on the experimental crane in the Kranzberg forest for a nadir-view to measure close range stereo images. A baseline of 0.25 meters is set between the two cameras, with the object distance of approximately 3 meters from the tree crown. For each camera, 19 mm lenses are used. The exposure time is set as 1/60 seconds, and the ISO speed rating is 125. The acquired images have a size of 4000 and 6000 pixels in height and width, respectively. Figure 3 is an example of the captured RGB image. Figure 3. RGB image captured from the experimental crane.

Ground measurements
In order to measure the reflective and biochemical properties of leaves and canopy, a data campaign was carried out on the 23rd of August 2018 along with the airborne Hyperspectral images acquisition. Pre-processing of the images is including Smile effects and atmospheric corrections. Four stressed and four healthy trees were selected for leaf sampling, in which 20 leaves each have been used for measurements. The following parameters were measured on the day of the flight: Chlorophyll using a SPAD and leaves reflectance using Spectral Evolution PSR 3500+ combined with a Leafclip. Leaves were also weighted and scanned.

Disparity map generation
To prove the second hypothesis of the investigation concept, a precise 3D model of the tree crown is required. 3D tree crown generation is a critical topic, due to the similar textures patterns among the leaves and shadow/wind effect. Although in computer vision many dense matching algorithms are developed and introduced to remote sensing image processing, their performance varies when different matching cost calculation approaches are adopted (Hirschmüller, 2008;Kuschk et al., 2014).
In this project, we apply Semi-Global Matching (SGM) with matching cost based on convolutional neural networks (MC-CNN) (Hirschmüller, 2008) (Zbontar and LeCun, 2016). MC-CNN is essentially a siamese network trained on pairs of image patches for a similarity measurement to calculate matching costs (Bromley et al., 1993), which has achieved state-of-the-art results. SGM is then exploited to aggregate the cost for final disparity estimation. As for the training of MC-CNN, we follow (Xia et al., 2019a) and apply a self-training strategy, due to the lack of appropriate training datasets for stereo matching of forest scenes. With a pre-trained network to generate a disparity map, a rigid left-right consistency check is used to filter most matching outliers. The remaining pixels are then used to finetune a network, which is capable of better computing the matching cost for more accurate disparity estimation. Figure 4 shows an example of the disparity maps generated from the provided stereo datasets, which can be used directly for the reconstruction of 3D models of the tree crowns if required.

Tree-leaves analyses from disparity maps
Based on the reconstructed tree model, we manually extract leaves for tree drought stress analyses. From our experience, tree leaves are apt to bend inwards to decrease the water-lose speed when suffering water shortage. Therefore, we calculate the included angle between each subarea of the leaf and a fitting plane of the whole leaf surface (Xia et al., 2019b). Figure 5 displays the included angle statistics for the selected healthy and stressed leaves. As the healthy leaves are much planar, they show relatively smaller included angles than stressed leaves.

Spectral measurement
Two 'in-situ' measurement series of spectral reflectance of forest trees are systematically performed at the KROOF site in the Kranzberger Forest: (i) ASD FieldSpec Measurements of the main tree species are collected into a spectral database to establish spectral databases of the main tree species spruce, pine, larch, beech, oak, birch, -167

Healthy leaves
Stressed leaves maple to support the approach of identifying tree species via the "phenological fingerprint" from airand spaceborne systems.
(ii) Gonio-spectro-radiometric measurements of selected drought stressedand control trees to develop a database for the approximation of the Bidirectional Reflection Distribution Function (BRDF) of the drought stress experiment tree species beech and spruce.
The first database should support tree species identification from multi-to hyperspectral data especially in multi-seasonal approaches (several observations across one vegetation period). The second databases should help correcting multi-to hyperspectral satellite data with a wide swath and data from systems with pointing capability. In the frame of the experiment these measurements are employed to test the basic hypothesis of the ForDroughtDet project, namely that a change in leaf position will happen as turgor pressure decreases and drought stress sets in. In this context, leaves are regarded as microreflectors in the canopy. This change in anisotropy behaviour is recorded by the hemispheric measurements of the goniometricspectra-radiometric data sets. Measuring morning, noon and afternoon series an approximation of the BRDF for the specific phenological stage is possible for each sun zenith angle and view direction 3 hours before to 3 hours after noon. The stereo imagery are acquired at the beginning of the Goniometer measurement series with starting position of the Gonio arms in the principal plane ( pos 1) and then turning for 90° (pos 2). Such for each tree we provide 4 images of the goniometer measurement spot. Conducting the measurements across the vegetation period the model is expanded accordingly. The challenge we set ourselves is to quantify this change as a basis for the diagnosis of the drought stress condition. In the logic of the experimental evaluation chain, approximated bidirectional reflection distribution functions (BRDF) of forest trees are integrated into physical reflection models in the "in-situ" experiment.
In the above defined experimental setup, the inversion of these models takes place using hyperspectral and multidirectional remote sensing data from "in-situ" to aircraft sensors. With respect to the operational implementation, we expect that both spectral and multidirectional dimensions can be reduced in order to diagnose drought stress on the Earth surface for larger areas using satellite data.

Hyper-spectral data based drought index analyses
To detect drought stress at leaf level different chlorophyllsensitive indices have been selected. Le Maire et al. (2004) demonstrated that Simple Ratios and Normalized Differences are optimum indices to use in empirical estimation of chlorophyll content. The modified Simple Ratio (mSR) and the modified Normalized Difference (mND) described by Sims and Gamon (2002) were selected. The modified Normalized Difference Vegetation Index (mNDVI) proposed by Gitelson and Merzlyak (1994) and the Chlorophyll Index (CI) described by Gitelson et al. (2003)  (4) for 800.9 nm and for 709.8 nm were chosen, respectively. These indices haven been selected because they are linearly proportional to chlorophyll content and they are sensitive to low chlorophyll concentrations (Le Maire. et al., 2004).  As expected and shown in Figure 6, all indices perform well in relation to chlorophyll content, the two best performing indices being mNDVI and mND with a respective correlation coefficient R of 0.82 and 0.83 followed by the CI and the mSR with 0.80 and 0.74 (all at a significance level of 0.001), respectively. The similar performances of mND and mNDVI, both "Normalized Difference" indices, show that indices based on ND seem to work the best with our data. It should be noted that a differentiation between stressed and healthy trees could not convincingly be demonstrated at leaf level (see Figure 7). However, the high correlation between chlorophyll content and the selected spectral indices suggest that they are responsive to water stress at canopy level.

Individual tree crown (ITC) segmentation
In order to enable object based analyses for single trees, ITC segmentation must be performed in the first place. ITC segmentation is difficult due to similar colour and texture of trees. Especially for clumped, deciduous tree crowns, their boundaries are sometimes hard or impossible to detect, even for human observers. We propose a segmentation approach based on the DSM, which is generated from the 3K dataset. The idea is to mitigate the drawback of watershed segmentation, namely that boundaries are located in the middle of possibly large canopy gaps. The workflow is shown in Figure 8. The steps can be described briefly as follows: 1. Global thresholding of the DSM. 2. Local histogram equalization (LHE) (Gonzalez, Woods, 2001) of the DSM, restricted to the areas above the threshold. 3. Gaussian smoothing (Gonzalez and Woods, 2001) of the LHE-DSM. 4. Active contours without edges (ACWE) (Chan and Vese, 2001) on the smoothed LHE-DSM. 5. LoG-Blob detection (Lindberg, 1994) on the LHE-DSM, restricted to the foreground areas of the ACWE level set. 6. MCWST (Vincent and Soillé, 1991) on the LHE-DSM, restricted to the foreground of the ACWE level set, using the blob centres as markers.
More details on the implementation results for each of the test plots, accuracy assessment and discussion may be found in (Kempf et al., 2019).
The obtained ITC delineation can be readily transformed to the near nadir image using known orientation parameters. This is demonstrated using the coniferous subplot in Figure 9. As it shows, most of the tree crowns are well segmented.
(a) (b) Figure 9. The extracted tree crown boundaries overlaid on the DSM (a) and original image (b).

Object-based drought index analyses
To further analyse the drought stress of single tress, the ITC segmentation results will be integrated with the airborne hyperspectral dataset to perform an object-based analyses. The averaged spectral index values are assigned to the areas inside each corresponding ITC segment. In the Kranzberg test region, the results are compared with the ground measurement data.

CONCLUSION AND FUTURE WORK
Forest drought stress detection is one of the largest challenges in forest management. In this paper, we have briefly reported the mid-term results from our research project 'ForDroughtDet'. The current work on forest drought stress detection on the first and second level prove that the changing of water content on the leaves can be observed with remote sensing techniques.
In the experiment of the spectral measurements of single leaves, high correlation coefficients between chlorophyll content and the spectral index can be observed. Although the single stressed and healthy leaves do not show a distinct difference limited to the available samples, the high correlation between chlorophyll content and the selected spectral indices suggest that they are responsive to water stress at canopy level. This should be proved in our further research. The second hypothesis has been well proved with the 3D model generated from close-range photogrammetry datasets. Clear differences could be observed between the stressed and healthy leaves. We will further analyse the 3D information at ITC level and combine it with the ground spectral analyses.