IMPROVED WATERSHED SEGMENTATION ALGORITHM FOR TREE CROWNS EXTRACTION FROM MULTI-SPECTRAL UAV-BASED AERIAL IMAGES

: Due to many problems such as diseases and pests, low fertility, and dehydration, trees need immediate actions to be taken in time of need. Since they are an important source of fruit, food, and nutrients consumed by humans, keeping track of the trees in orchards is a crucial issue in recent years. Today, drones equipped with multispectral cameras are used in precision agriculture, especially for monitoring and controlling trees. For this cause, two citrus orchards in Iran with an area of 9.2 and 2.67 hectares and a resolution of 3.6 and 0.68 cm were selected for the study area. In this study, First, tree extraction was conducted using four algorithms namely Local maxima, Image binarization, valley following, and watershed segmentation, and a proposed method that is based on the improvement of the watershed algorithm. This method achieved an overall accuracy of 87% and 81% in the two study regions which was higher than common methods. Secondly, the effect of the number of spectral bands on the accuracy of tree extraction was investigated. As a result, by adding of Red-Edge and NIR bands, the accuracy increased by about 5% and 7%. Therefore, experts suggested using NIR and Red-Edge bands besides RGB bands.


INTRODUCTION
Due to global climate change and fast population growth, trees play an important role in improving the environment and life quality.In this regard, extraction and monitoring of trees to support their growth seems necessary (Barron et al. 2016).To achieve this goal field surveying or more modern Remote Sensing methods are required.However, field observation is time-consuming and economically inefficient.Previous research has shown that the data obtained from Remote Sensing sensors are reasonable tool for automatic monitoring and detection of trees (Chianucci et al. 2016).Research into the automatic detection of trees from digital images dates back to the mid-1980s.Tree canopy extraction algorithms generally fall into seven categories including local maxima, scale analysis, region growing, binary image, template matching, watershed segmentation, and valley following (Ke and Quackenbush 2011).Recent advances in remote sensing have provided access to aerial images captured by unmanned aerial vehicles (UAVs).UAVs are capable of acquiring aerial images with a combination of very high spatial resolution and temporal resolution (Moradi et al. 2021).In addition, their operating cost and complexity are lower compared to other remote sensing platforms (Ke and Quackenbush 2011) and (Colomina and Molina 2014).Among the sensors installed on UAVs, multispectral cameras have attracted experts' interest, especially in the field of precision agriculture (PA) due to their very high spatial and spectral resolution.In recent years, these images have been widely used in research to investigate the water stress of trees (Stagakis et al. 2012), pest contamination (Näsi et al. 2015), separation of healthy and unhealthy trees (Garcia-Ruiz et al. 2013) and(DadrasJavan et al. 2019), classification of tree species (Franklin et al. 2017), Evaluation of natural indicators such as leaf area index and amount of nitrogen content (Vega et al. 2015) and (Caturegli et al. 2016), identification and extraction of tree canopy (Kolanuvada and Ilango 2021).Most of the researches that had used UAV based multispectral imagery are mainly focused on the process of radiometric outputs to identify the disease, and little research has been done on the extraction of tree crowns using geometric analysis.Therefore, in this study, we pursue two general goals, the first goal is to improve the tree crown extraction algorithm and create a geometry-wise algorithm using multispectral imagery and the second one is to study the effect of increasing the number of spectral bands on the accuracy of tree crown extraction.The rest of the structure of the article is as follows: in section two, I introduce the two site studies and the proposed method.In the third part, the accuracy of the methods is evaluated at the pixel level, and the fourth part is about discussing and analyzing the results and suggestions for future research.

MATERIALS AND METHOD
The proposed workflow according to Figure 1 consists of three main steps: 1. Obtaining data by drone 2. Preprocessing 3. Separating trees from other features and extracting tree crowns using the proposed algorithm, each part is described separately.
The first area is related to a peach orchard in Neka city with geographical coordinates (36 ° 39303 ″ N, 53 ° 17′57 ″ E WGS84) in which the crown of trees is almost the same size and trees are regularly distanced from each other, data acquisition was done by (Pourazar et al. 2019).The second area in Darab city is located at (28 ° 45′07 ″ N 54 ° 32′40 ″ E WGS84) is where extensive orchards of oranges and lemons are located.This area has trees with different crown sizes and different planting distances.The platform for data acquisition is a multirotor platform and the installed sensor was MicaSense-RedEdge.The sensor has 5 bands: blue (475 nm), red (668 nm), green (560 nm), red edge (717 nm), and infrared (740 nm).The total area covered at the Darab site was 9.2 hectares with a spatial resolution of 3.6 cm. and the Neka site has an area of 2.67 hectares with a spatial resolution of 0.68 cm.The first step is sensor calibration, in which radiometric calibration was performed by special boards pre-flight and post-flight in the field.Geometric calibration was also performed using Agisoft-PhotoScan software.After performing geometric calibration, ortho-mosaic was obtained for five bands.Different bands should be registered to each other due to the misplacement of bands in the sensor.For this cause, the Red-Edge sensor was considered as master and the other bands were slaves, then feature extraction was applied through SIFT algorithm.Using the least Euclidean distance, these images were registered with an error of fewer than 0.4 pixels.Extraction of trees from other features is conducted in two stages.In the first stage, by changing the color space to HSV, vegetation from the soil was separated.In the second step, by changing the color space from HSV to the lab, the trees were separated from other vegetation.In the final step, a fusion of watershed segmentation and K-means was used to extract tree crowns.The watershed-Segmentation algorithm considers the gray degree image as a topographic surface, then the water flows from the highest height and the holes are filled and considered as a segment.It also creates a userdefined criterion known as a barrier to avoid the merging of different segments.This algorithm is simple and fast and its main problem is the over-segmentation of features (objects).
On the other hand, The K-means algorithm also has a parameter called k that specifies the number of clusters to be obtained.In this case, k data as the center of the cluster were selected, then the distances of the rest of the data with the center of the cluster are determined and the data that are closer to the center of each cluster are placed in that cluster.
In the next step, it again selects the average of each cluster as the new center of the cluster and continues these steps until the clusters remain unchanged (Kanungo et al. 2002).This algorithm also splits the image into large segments and identifies several trees as one tree.In this research, to improve the efficiency of these two algorithms for tree crown extraction, we have proposed a fusion of two algorithms of watershed segmentation and k-means.In this way, first, all the pixels are segmented by the watershedsegmentation algorithm and in the next step, the components are updated by the k-means algorithm so that it moves from the outermost segment and focuses on the similarity of the color space of the segments on the main image.then after reaching the first point, it does not consider additional internal segments, and keeps updating until the homogeneous parts and regions do not change in two consecutive rounds.For quantitative evaluation, a confusion matrix is used, each pixel is divided into four distinct categories (Mohan et al. 2017;Ok et al. 2012;Shufelt 1999): 1.The predicted tree is the visually tree (TP) 2. The unpredicted tree is the visually tree (FN) 3. The predicted tree is not a visually tree (FP) 4. The unpredicted tree is not a visually tree (TN).Then the criterion (ACC) and precision are calculated according to equations ( 1) and ( 2), respectively, using the values obtained from the confusion matrix (Ok, 2012) and (Shufelt, 1999): (2)

RESULTS
Results from applying TE algorithms are shown in Figure 3 separately.And validation results using accuracy metrics are shown in Table 1 and 2.

CONCLUSION
In this study an improved watershed algorithm was proposed which uses a combination of watershed algorithm, k-means clustering, and changing the color space.the proposed algorithm with four common algorithms was compared in the extraction of tree crowns to evaluate its efficiency and reliability.
All algorithms are implemented on the two study sites with 3 bands i.e., Red, Green, and blue (R, G, B).As illustrated in Figure 3, the performance of the Local maximum algorithm in the Darab site is poor.The main problem is the size of the selected window, because if it is small, it will cause overs-segmentation of trees, and if it is large, small trees will not be extracted.On the other hand, at the Neka site, where the trees were about the same size, it performed better.The Valley Following algorithm also did not perform well on the Darab site.The reason for the weak performance of this algorithm is that in a part of the Darab site, the trees are dense that the overlap of the crowns prevents the formation of valleys, therefore, making it difficult to detect the trees.In another part of this site, the trees are scattered and the presence of a large space between the trees causes misdiagnosis.Also, in this site, the canopy of trees are not the same size and small trees are located next to large trees, which makes it difficult to extract small trees by this algorithm.But in the Neka site, because the trees were placed at equal distances from each other, results were more promising.Although, the watershed-Segmentation algorithm performed better on the Darab site than the other two algorithms, it resulted in the over-segmentation of trees.By visually comparing the results, it is clear that the proposed algorithm in both study sites had an acceptable performance and was able to correctly extract the largest number of trees.Using the proposed algorithm in the Darab site, we managed to extract exactly 88 trees out of 112 trees, and in the Neka site, we managed to extract 55 trees out of 64 trees.
The quantitative evaluation also confirms the results of the visual evaluation.In Table 1 the best results were marked in red.In terms of run time, the proposed algorithm takes longer to run than other algorithms, which is negligible because of its higher accuracy.
Considering that the proposed algorithm has provided better accuracy than other tree extraction (ITC) algorithms, we use this algorithm for the second purpose of this research.This algorithm is studied in two scenarios, the first scenario has three bands (R, G, B) and the second one has five bands (R, G, B, NIR, Red-Edge) in two study sites.The algorithm is implemented in a way that Red-Edge and NIR bands have larger weights, and the R, G, and B bands have the same lower weights.The quantitative evaluation results for these two data are given in Table 2.
The accuracy of the quantitative evaluation has confirmed the claim that increasing the number of spectral bands increases the geometric accuracy of tree crowns extraction.The number of tree crowns that have been extracted using the proposed algorithm is the same in both the three-band and five-band data, and this increase in accuracy is due to better vegetation features extracted from NIR and Red-Edge bands.The visual evaluation also proves this claim.As you can see in Figure 4, we put the extracted crown in four bands (Green, Blue, NIR, Red-Edge) on each other, it is obvious that the NIR and the Red-Edge bands performed better in extracting the boundary of trees.Considering the importance of greenbelt and trees, it is necessary to know the condition of the tree crowns and shrubs and detect and assess their health and changes.For this purpose, I have implemented an unsupervised and repeatable semi-automatic algorithm for processing multispectral UAV images with aim of tree extraction.its results were compared with conventional tree crown extraction algorithms in two different sites first a homogeneous and sparse while the second one was a heterogeneous garden.After evaluation and comparison, it is clear that each algorithm performed well in the first site, as for the second site, the proposed algorithm had highest performance indicating that it has provided acceptable accuracy and is scale-independent.Also, by comparing the accuracy obtained from the proposed method with other studies that have extracted tree crowns from the object-based method and deep learning, our results are close to the results of other researchers.this method claims to be simple, fast yet with promising results.Deep learning methods, despite their high accuracy, require a lot of training data and expensive GPUs, making them time-consuming and financially inefficient but suitable for dense areas which is the main drawback of traditional image processing methods.In the second part of this research, a quantitative and visual evaluation was done to assess the effect of increasing the number of spectral bands on the accuracy of geometric extraction of trees.Based on our findings, it increased the accuracy by 5% and 7% for Red-Edge and NIR bands.Thus, we suggest that researchers use bands (NIR, Red-Edge) to extract trees in their work.

Figure 1 .Figure 2 .
Figure 1.Workflow of the proposed method

Figure 4 .
Figure 4.The extracted crown in four bands (Green, Blue, NIR, Red-Edge

Table 1 .
Assessing the accuracy of tree canopy extraction.Darab site in the left column and Neka site in the right column

Table 2 .
Evaluation of tree crown extraction accuracy by increasing the number of spectral bands.Darab site in the left column and Neka site in the right column