INDIVIDUAL TREE EXTRACTION FROM UAV LIDAR POINT CLOUDS BASED ON SELF-ADAPTIVE MEAN SHIFT SEGMENTATION

Unman aerial vehicle (UAV) LiDAR has been widely used in the field of forestry. Individual tree extraction is a key step for forest inventory. Although many individual tree extraction methods have been proposed, the individual tree extraction accuracy is still low due to the complex forest environments. Moreover, many parameters in these methods generally need to be set. Thus, the degree of automation of the methods is generally low. To solve these problems, this paper proposed an automatic mean shift segmentation method, in which the kernel bandwidths can be calculated self-adaptively. Meanwhile, a hierarchy mean shift segmentation technique was proposed to extract individual tree gradually. A plot-level UAV LiDAR tree dataset was adopted for testing the performance of the proposed method. Experimental results showed that the proposed method can achieve better individual tree extraction result without any parameter setting. Compared with the traditional mean shift segmentation method, both the completeness and mean accuracy of the proposed method are higher. * Corresponding author


INTRODUCTION
Forest resources are one of the most important resources on the earth and have a major impact on human survival and development (Lim et al. 2003). In recent years, the emergence of LiDAR technology has made new breakthroughs in forest resource surveys (Yu et al. 2017). Compared with traditional optical remote sensing technology, LiDAR technology is less affected by the external environment. LiDAR technology can directly, quickly and accurately obtain three-dimensional coordinates of ground objects, and the emitted light beam can penetrate the inside of vegetation to obtain accurate internal structure (Vega et al. 2016). Thus, LiDAR technology has significant advantages in identifying forest tree types, establishing vegetation models, extracting forest structure parameters, and measuring forest volume (Liu et al. 2019).
According to different platforms, the LiDAR technology can be classified into four groups, including terrestrial LiDAR, mobile LiDAR, airborne LiDAR and UAV LiDAR. Compared to other platforms, UAV LiDAR has a good balance among convenience, spatial coverage and data quality (Hu et al. 2021). Thus, UAV LiDAR has been widely applied in forest inventories, especially in single tree extraction. The quality of single tree segmentation directly affects the accuracy of tree parameters estimation, such as tree height, crown diameter, crown height, base area, diameter at breast height (DBH), wood volume, biomass (Zhang et al. 2019). Due to the different tree species and complex tree structures, the individual tree extraction method is still challenging.
The existing individual tree extraction methods can be divided into two categories, namely CHM-based and point-based methods. The CHM-based methods interpolate the threedimensional point cloud data to generate the tree canopy high model (CHM). The CHM is the difference between the digital surface model and the digital elevation model (Bian et al.2014). After obtaining the CHM, a local maximum filter is used for treetops detection. Subsequently, the image processing algorithms can be used for single tree segmentation. Zhen et al. (2014) proposed a marker-controlled region growing method. In this method, the treetops were used as the initial seed point, and the criteria based on homogeneity, shape and region size were proposed for the region growing. Jing et al. (2012) proposed a marker-controlled Watershed segmentation algorithm. This method used multiple scales to filter the crown height model and marked the filtered local maximum. The final tree canopy map was generated by integrating the scaled segmentation map. Yang et al. (2020) combined the watershed algorithm and point cloud three-dimensional space information for better individual tree segmentation. Koch et al. (2006) proposed a Pouring algorithm, which used a filter to detect tree tops and segmented the canopies of coniferous forests and broad-leaved forests based on the assumption of the shape of the trees. Wu et al. (2016) adopted the geometric characteristics and topological relationships of trees to segment single trees. Contour lines were generated based on the CHM. The tree crown was divided into multiple levels according to the contour line. The valley following algorithm was proposed to solve the problem of tree connection.
Compared with the CHM-based methods, the point-based methods avoid interpolating errors when generating the CHM, and make full use of the three-dimensional information of point clouds. Morsdorf et al. (2004) used the local extreme points in the digital surface model as the initial seed points for K-means clustering. This method greatly improved the efficiency of single tree segmentation. Sačkov et al. (2017) applied a moving window analysis method to iteratively search for local maxima (as presumed treetops), and summarized allometric growth rules of trees by referring to the measured data. This method combined the treetops with the allometric growth rules for single tree segmentation. Zhong et al. (2017) proposed a modified normalized cut method, which can be used to segment the overlapped canopy. Compared with the Ncut method, this method can effectively separate neighboring trees. Li et al. (2012) used the relative spacing between trees to segment individual trees. This method applied a top-down regional growth strategy. Points with spacing larger than a specified threshold will be excluded from the target tree.
Among the point-based methods, the individual tree extraction using the mean shift segmentation is very popular in recent years. Ferraz et al. (2010) and (2012) modified the mean shift method by splicing the single kernel as horizontal and vertical kernels. This method can effectively divide the forest into three levels, including ground vegetation, understory vegetation and excessive vegetation (Ferraz et al. 2010;Ferraz et al. 2012). Hu et al. (2017) proposed an adaptive mean shift-based clustering approach to segment the 3D forest point clouds. In this method, the size of the bandwidth automatically changed with the crown diameter of different trees. Chen et al. (2018) combined the trunk detection technology with the mean shift method. The detected trunk information is helpful for adaptively calibrating the kernel bandwidth of the mean shift procedure and detecting the final individual trees. Yan et al. (2018) proposed a voxel-Based mean shift algorithm to perform coarse segmentation on point cloud data. An improved normalized segmentation method to segment the under-segmented clusters was proposed to identify individual trees. Dai et al. (2018) used the geometric spatial information and spectral information of the point cloud to extract individual trees. This method applied the point cloud spatial information to perform mean shift segmentation, and combines spectral information to segment the under-segmented crown clusters. Yan et al. (2020) carried out clustering based on the horizontal distance between the crown points to detect the trunk points, and used the crown diameter obtained by analysis in multiple directions as the bandwidth of the mean shift algorithm.
Although many modified mean shift methods have been proposed for individual tree extraction, the performance of this kind of methods is still affected by the kernel bandwidth setting. To achieve a better individual tree extraction result, the parameter of the kernel bandwidth needs to be tuned by trial and error. To solve this problem, this paper proposed an automatic mean shift individual tree extraction method. In this method, the kernel bandwidth can be estimated automatically. Meanwhile, a hierarchy mean shift segmentation technique was also presented to achieve higher individual tree extraction accuracy.

METHODOLOGY
The main steps of the proposed method are shown in Fig. 1. It can be found that the inputs are the filtered normalized UAV LiDAR point clouds. The UAV LiDAR point clouds are then filtered to remove the ground points using the improved morphological filtering method proposed by Hui et al. (2016).
The improved morphological filtering method is a hybrid model, which can preserve terrain details effectively. To further remove the influence of terrain trends, the left non-ground points are normalized based on the digital terrain model (DTM) built using the filtering results. Subsequently, the proposed method mainly utilizes the mean shift method for the trees segmentation. It is well known that the bandwidth parameters, especially the horizontal bandwidths have an influence on the segmentation results. To segment the trees correctly, the horizontal bandwidths are first estimated self-adaptively based on the DBSCAN clustering in this paper. Then, a hierarchy mean shift segmentation method is proposed to extract individual trees iteratively according to the calculated horizontal bandwidths. The main steps of the proposed method can be summarized as follows: Ⅰ . DBSCAN clustering for trunk points, Ⅱ . Bandwidths self-adaptive estimation and Ⅲ . Hierarchy mean shift segmentation.

DBSCAN clustering for trunk points
Generally speaking, the horizontal bandwidth should be set according to the crown size for each tree. In so doing, the points within each canopy can be clustered together. Thus, to determine the appropriate horizontal bandwidth, the crown sizes for the trees should be estimated. In general, the tree canopies are suppressed by the closest trees. In other words, if two trees are very close their crown sizes cannot be larger. Therefore, the crown radius can be estimated according to the distance between two closest trees. Compared to the airborne LiDAR system, the UAV LiDAR system usually has a relatively low flight height. Thus, the laser pulses reflected from the bottom of trees can be acquired. As shown in Fig. 2 (a), the complete point clouds for each tree can be obtained. Thus, it is easy to set a truncated height threshold to obtain the trunk points for each tree. In this paper, the truncated height threshold is set to 2.5 m.  Fig. 2 (b) shows the extracted truncated trunks for the trees. In general, the extracted results usually contain some isolated outliers that are not the points of the trunks. To calculate the distance between two closest trees and overcome the influence of the outliers, this paper first applies the DBSCAN clustering method to the extracted trunk points. DBSCAN is a famous density-based clustering method, which can be applied to the datasets with arbitrary shapes . Compared with other clustering methods, DBSCAN does not require the prior knowledge of the datasets or preset the clustering number as the K-mean method does. Moreover, DBSCAN has the ability of eliminating the influence of the outliers. Thus, it is appropriate to apply the DBSCAN for clustering the trunk points. In the DBSCAN method, two parameters need to be set, including k and Eps. k is the threshold of the number of neighboring points, which determines the outlier detection. In this paper, k is set to 20 to remove some isolated points. Eps is the neighboring radius. To separate some adjacent trees effectively, Eps is set to 1 m in this paper. The clustering results are shown in Fig. 3 (a). It is easy to find that the trunk points are clustered correctly, while some isolated points are removed effectively.

Bandwidths self-adaptive estimation
When the trunk points for different trees are obtained, it is easy to detect the tree locations. In this paper, the locations for the trees are defined as the lowest point within each cluster. Meanwhile, the tree tops can also be detected by searching the highest points within 0.5 m distance of the tree locations as shown in Fig. 3 (b). The tree tops will be used in the following hierarchy mean shift segmentation. According to the obtained tree locations, each two closest trees can be found. The distance ( d ) between two closest trees can be calculated. To estimate the crown size for each tree, this paper makes two assumptions: ⅰ the distance ( d ) between two closest trees is formed by the canopies of the two trees; ⅱ the crown sizes of the two trees are proportional to their tree heights. Based on these two assumptions, the crown size for the trees can be estimated based on the flowchart mentioned in Tab. 1. Note that the tree heights ( i h ) for each tree can be acquired by find the highest point within 0.5 m range of the horizontal position   , Tree x y for each tree. The estimated crown sizes for the trees are shown in Fig. 4. Fig. 4 (a) shows the estimated crown sizes in threedimensional (3D), while Fig. 4 (b) is the two-dimensional (2D) display. When all the crown sizes are estimated the horizontal bandwidths for the trees can be set to be equal to the crown sizes.

Hierarchy mean shift segmentation
The mean shift segmentation is a famous image segmentation method. Melzer (2007) first extended the mean shift method to segment three-dimensional point clouds. In the mean shift method, the mean shift vector which points to the direction of maximum increase in the density should be calculated iteratively (Dai et al. 2018). To each point, the mean shift vector will keep shifting until it reaches a mode. The points shifted to the similar mode will be classified as a cluster. To extract each individual tree effectively, the three-dimensional space was split into horizontal and vertical domains.
where S h = horizontal bandwidth r h = vertical bandwidth As mentioned above, the horizontal bandwidth S h affects the mean shift segmentation greatly. In this paper, vertical bandwidth r h is set as a constant, while the horizontal bandwidth S h is calculated automatically according to the principle described in subsection 2.2. Note that in this paper the bandwidths were sorted in a descending order. That is, . In terms of   S hk , the corresponding mean shift segmentation results can be achieved. In general, if an individual tree is segmented correctly, there will be only one tree top contained by these tree points. The tree tops are obtained in subsection 2.1 as shown in Fig. 3 (b). The correctly segmented tree points are removed from the point clouds gradually. The left point clouds are segmented using a smaller horizontal bandwidth. The mean shift segmentation is conducted iteratively until the smallest horizontal bandwidth is reached. When adopting the smallest horizontal bandwidth, all the left points are processed by the mean shift method to obtain the final classification results.

EXPERIMENTAL RESULTS AND ANALYSIS
To test the performance of the proposed automatic mean shift individual tree extraction method, the UAV LiDAR dataset provided by Brede et al. (2019) was adopted for the testing. The dataset was acquired using a Riegl Ricopter with VUX-1UAV. The VUX-1UAV is a survey-grade laser scanner with an acrosstrack Field Of View (FOV) of 330° (Brede et al. (2019)). The selected trees for the testing in this paper are mainly the Douglas Fir. All the individual trees are separated from each other manually using the visualization software called Cloud Compare. After filtering the ground points, the normalized trees points are shown in Fig. 5. It is easy to find that all the individual trees are separated and shown with different colors. Thus, it will be easy to test the performance of the proposed method. Three indicators, including completeness ( Com ), correctness ( Corr ) and mean accuracy ( Mean ) are adopted for accessing the individual tree extraction results. These three indicators can be calculated according to Equations (5) where _ n refer = the number of reference trees _ n extr = the number of extracted trees _ n match = the number of matched trees In this paper, if the majority of the extracted individual tree points (over 80%) are correctly identified, the tree will be considered as a matched tree (Yan et al. 2018). The performance of the proposed method was compared with that of the traditional mean shift individual tree extraction method proposed by Ferraz et al. (2012). In the traditional mean shift method, two parameters need to be adjusted. One is the horizontal bandwidth, while the other one is the vertical bandwidth. As indicated by Yan et al. (2020), compared with the vertical bandwidth, the horizontal bandwidth has a greater influence on the individual tree extraction results. Thus, both in the proposed method and the traditional mean shift method the vertical bandwidth is set to 8 m. The horizontal bandwidths can be calculated automatically by the proposed method. As a contrast, the horizontal bandwidths for the traditional mean shift method are set as 1.0 m, 1.5 m, 2.0 m, 2.5 m, 3.0 m and 3.5 m, respectively. The comparison results are shown in Fig. 6. From Fig. 6, it can be found that different horizontal bandwidths lead to different individual tree results. In terms of the traditional mean shift method, with the horizontal bandwidth changing from 1.0 m to 3.5 m, all the three indicators (completeness, correctness and mean accuracy) have a similar trend. That is the indicators are changing from low to high, then from high to low. It indicates that there will be an optimal horizontal bandwidth for achieving the best individual tree extraction result. In terms of the proposed method, in addition to the correctness, both completeness and mean accuracy are the highest when comparing with other individual tree extraction results using different horizontal bandwidths. This indicates that the proposed hierarchy mean shift technique is effective. Meanwhile, the horizontal bandwidth can be calculated automatically by the proposed method, which will avoid the parameters adjusting.

CONCLUSION
Individual tree extraction is a critical issue for the forest inventory. Due to the complicated forest environments and complex tree structures, the individual tree extraction is still challenging. To improve the accuracy and the degree of automation of the individual tree extraction, this paper proposed an automatic individual tree extraction method based on the mean shift segmentation. In the proposed method, the kernel bandwidths can be estimated automatically. Meanwhile, a hierarchy mean shift technique was proposed, in which the individual trees were detected gradually. Experimental results showed that compared with the traditional mean shift segmentation method, the proposed method can achieve a better individual tree extraction result. Meanwhile, the proposed method does not need to adjust the kernel width. Thus, the proposed method will be easy for implementation by inexperienced staffs.