AUTOMATIC TREE STEM DETECTION – A GEOMETRIC FEATURE BASED APPROACH FOR MLS

Recognition of tree stem is a fundamental task for obtaining various geometric attributes of trees such as diameter, height, stem position and so on for diverse of urban application. We propose a novel tree stem segmentation approach using geometric features corresponding to trees for high density MLS point data covering in urban environments. The principal direction and shape of point subsets are used as geometric features. Point orientation exhibits the most variance (shape of point set) of a point neighbourhood, assists to measure similarity, while shape provides the dimensional information of a group of points. Points residing on a stem can be isolated by defining various rules based on these geometric features. The shape characterization step is accomplished by estimating the structure tensor with principal component analysis. These features are assigned to different steps of our segmentation algorithm. Wrong segmentations mainly occur in the area where our rules have failed, such as vertical type objects, road poles and light post. To overcome these problems, global shape is further checked. The experiment is performed to evaluate the method; it shows that more than 90% of tree stems are detected. The overall accuracy of the proposed method is 90.6%. The results show that principal direction and shape analysis are sufficient for the tree stem recognition from MLS point cloud in a relatively complex urban area.


INTRODUCTION
The segmentation of urban vegetation (trees) is a mandatory task for various applications such as urban planning, ecomonitoring system, and autonomous navigation.The recognition of trees are well advanced in the forest context, for instance (Maas et al. 2008, Liang et al., 2012), whereas the extraction of the tree stems located in complex urban areas is still a challenge due to the high object heterogeneity of urban scenes.To date, only few attempts have been focused on urban tree detection using point clouds (Rutzinger et al. 2011).Mobile Laser Scanning (MLS) developed very recently has proven to be an efficient tool for fast and reliable capturing of three dimensional dense point clouds over large areas, so that it provides a new solution for collecting the information of urban objects.Besides, with the advancement of MLS, various attributes of a single tree such as tree height, crown coverage, diameter at breast height, detailed model and so on, can also be approximated well and such information is very supportive for various urban applications.The recognition of tree stems is the initial step for the extraction of these attributes.The manual identification of trees is a labor intensive and time consuming work.Conversely, automatic tree stem recognition often fails in dense complex urban scenes, which demands efficient and more specific solutions.The aim of this paper is to support such applications with an automatic algorithm for the tree stem detection of MLS point clouds with different geometric characteristics.
To do this, we propose an approach for segmenting MLS data in urban environment in order to extract tree stems by using geometric descriptors such as principal direction and the geometric shape of point subsets.The first step of the presented method is the extraction of point subsets referring to the trees from each point cluster approximated as tree clusters from our initial point processing step; thus this follows a cluster-wise processing strategy which assists to increase the detection accuracy.In the following step, the structure tensor with principal component analysis is utilized to derive the principal direction and the shape of each point subset.After that, tree stem segments are collected based on the direction of each point subset.Finally, a union of segments having close orientation (with adequate overlap in 2D) are accepted as the stem of a tree.
The overall structure of the paper is organized as follows: the next section (2) gives a summary of the related work on tree stem recognition from point cloud data.In section 3 and 4, the proposed method and the data used for the experiments are discussed.Section 5 comprises the evaluation of the method and discussion.The paper is concluded in section 6.

RELATED WORKS
The segmentation of laser scanner point clouds to recognize tree stems can be considered as a first step in the working scheme of tree modelling process.Afterwards, investigation of tree inventory parameters can be taken place.During the last decade, many studies have been carried out on the segmentation of point clouds to detect tree stems using the TLS data while only few contributions have been made using MLS data.The 2D Hough Transform based tree recognition method in forest environment has been proposed by Aschoff and Spiecker (2004).Pfeifer et al. (2004) extracted and modelled the structure of trees (tree stems and major branches) by fitting cylinders in to point clouds with different radii referring to stems and their branches.The evaluation results of this method have been reported by Thies and Spiecker (2004) and showed that the detection rate is fairly low.Lalonde et al. (2006) utilize geometric descriptors to identify points on a tree; however this ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume II-5/W2, 2013ISPRS Workshop Laser Scanning 2013, 11 -13 November 2013, Antalya, Turkey method has been applied only for forest context and is not sufficient for urban area.Maas et al. (2008);Tansy et al. (2009) and Bucksch et al. (2009) proposed tree detection methods for TLS data based on the 2D layer searching techniques.In these methods, tree stems have, mostly, been recognized by dividing point clouds into different slices at appropriate height intervals and then point clusters (i.e. group of points in a slice, projected to a horizontal plane) referring to the stem of a tree is found if a 2D circle can be fitted to a point cluster.The overall success rate of these methods is fairly high, for example 97.5% (Maas et al. 2008), and also the required computation time is relatively modest.Although, these methods have been well adapted to the forest context, the problem would occur for scenes which consist of heterogeneous scenario.In recent approaches, for instance, Liang et al. (2012), trees are recognized by investigating the attributes of point distribution in local neighbourhood.It has been validated with about 73% overall accuracy in stem detection.Lehtomäki et al. (2010) classify urban objects from MLS using 2D layer-wise shape characterization method.To identify tree stems, the classified objects are further analyzed based on the thickness and heights, for example 2.5m and 10m respectively; according to their results, it confirms that the 2D layer techniques match well with MLS data.Brenner (2009) also employ similar method for the purpose of pole object detection in order to assist driver guidance using MLS data.Recently, Bremer et al. (2012) present a method for structuring and classifying points into tree branches.For this the point cloud is segmented into planar faces and then segment shapes are analyzed.
In this work, our contribution is to utilize the principal direction and shape of point subsets as the geometric descriptors for differentiation of trees from other objects efficiently.

Figure 1. Sample area of study data set
The MLS data set of our study area was acquired by a Z+F scanner with a 310° x 360° field of view and 0.1mm range resolution.The study area is located in the city of Dresden, Germany.The main features of the data set are buildings, trees, bushes, terrain, street furniture (for example; light post, road signs) and some temporal objects (vehicles, pedestrians and so on), see figure 1.For the accuracy assessment of the presented approach, a sample data set containing 42 trees is selected.

SEGMENTATION OF TREE STEMS
The applied method follows a direct point cloud processing procedure and is composed of three phases.Initially, data is pre-processed for classification of ground, buildings and nonbuilding objects using the method introduced by Nalani and Maas (2012).The geometric descriptors (feature entities) based on the shape and principal direction are applied with the aim of extracting points referring to the tree stems from other isotropic objects.For this process, first, the non-building point cloud is divided into point subsets aggregating the closet points to a given point in order to speed up the stem detection.In the second step, the tree stem points are further examined using the principal direction and shape analysis and then the tree stems are located.After applying these steps, it shows that other pole type objects, light poles, have also been detected as tree stems.
To eliminate these objects, shape (in 2D) of point sets around each detected stem is characterized.Figure 2 demonstrates the entire workflow of our approach.Details of each stage are given in the following sub sections.
Figure 2. Workflow for tree stem detection

Geometric characteristics of point subsets
In our earlier work (Nalani and Maas, 2012), building and terrain points are classified by analysing the local height histogram and linear features in MLS data.The first step of our stem detection approach is to group points (clusters) referring to the non-buildings by applying connected component analysis.These clusters are then examined to isolate the most probable tree clusters from non-vegetation objects, for instance cars and pedestrians.For this, the entire extent, i.e. the height from the lowest to highest point and the number of points of clusters (size) are utilized.In our case, the minimum height and cluster size are 2m and 500 points respectively.The stem detection begins by creating point subsets (S ⊂ Q, Q is a point cluster) for each selected point cluster.With the creation of subsets, a unique label for each point is assigned.Afterwards, we used neighbourhoods within a defined sphere of a randomly chosen point ( i p ) in the cluster for forming a point subset , where i = |Q| and r is a radius).
All these neighbour points K j {k 0 ,…, k n } are used to estimate the principal direction (dp i ) and normal vector (np i ) of point p i , which represent the local shape of a surface (geometric features).In order to determine these properties of the point p i , eigenvector decomposition is applied by constructing the covariance matrix of the neighbourhood as seen in equation 1.
Generally, tree stem and their branches have, locally, approximately shape of cylinders, which means that the points in tree stem and branches represent a regular distribution of principal direction and the points from leaves have various principal directions, i.e. irregular distribution.Also, the normal ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume II-5/W2, 2013 ISPRS Workshop Laser Scanning 2013, 11 -13 November 2013, Antalya, Turkey vectors of the k j are approximately orthogonal to the stem direction.This property is considered as a rule to isolate points represented tree stem and branches from the leaves.Thereby, neighbour points of p i are aggregated into S i if dk j does not differ much from dp i (for example 12 °).All points in the subset are labelled with the same value as i p .This region growing algorithm is processed until all points are assigned into a subset.Large differences on principal direction of some of neighbour points of p i can appear even p i and all k j are belongs to a tree stem or branch.The problem mainly occurs due to occlusion, front-view scanning system of MLS and abnormal stem parts.
To eliminate this effect, we examine the percentage of points being segmented as stem points among the neighbour points of k j .Point subsets having few points are ignored for stem detection process.Once the point subsets are located, the computation of certain geometric features for each subset begins.
Geometric features such as shape and principal direction can be used for the classification of each point subset.The variance of point distribution, generally, indicates the geometric shape of points in each subset; it can be estimated based on principal component analysis (PCA) in some space.In this work, the structure tensor, which is originally initiated by Knutsson et al. (1989) (2) the vector e called an eigenvector of A associated with the eigenvalue λ The 3D structure tensor (the symmetric 3 by 3 matrix) of each subset can be expressed as shown in equation 2. After the structure tensor is computed, the eigenvalues (λ) and corresponding eigenvectors (e) are derived and then these eigenvalue decomposition results are examined to characterize point distribution (orientation) of subsets.In general, eigenvectors represent the orientation of point variance while the eigenvalues give the compactness of the local point distribution.In our classification process, different geometric features, for instance the direction, size and shape are stored to identify subsets referring to the tree stem in later process.The shape of a subset is the most important feature as it assists to characterize object as well.The shape indices and different geometric features are given in table 1.
After deriving λ and e for each 3D tensor, the Zingg Shape Indices (ZSI) is utilized in order to differentiate point subsets into two main classes (stem/branches, leaves or others) as the Zingg shape classification (see figure 3 for ZSI shape class diagram) is the most logical way to describe object shape (Szabo and Domokos, 2010).According to the given ZSI, each point subset is labelled.

Tree Stem detection
With respect to the tree stem, a point subset, probably, represents a planar and vertical shape.On the other hand, its orientation should be closely matched with the estimated tree stem orientation.The tree stem can generally be defined as a union of the connected point subsets which have been classified as stems.In our tree stem detection method, the base of the tree is first located as it is fairly easy to classify remaining upper subsets based on the known segments of which indicates the given tree (layer wise searching process).For this, a subset classified as a stem in the lowest part of the cluster (0.5m from the lowest point) is selected as a seed segment and prolong with the help of neighbour subsets.A neighbour subset is classified as a stem segment: if it's vertical shape does not differ too much from (7º) the vertical shape defined by the seed segment.Recursively, the orientation and the geometric shape are estimated in order to delineate subsequent stem segments.After that, initial line segment corresponding to its orientation is computed.Stem subsets which belong to the bounding box defined right above the current stem segment for a certain height (0.25m) are grouped based on their orientation.Subsequently, segments are accepted as a correct stem segment if the criteria for geometric shapes (see section 4.1) are fulfilled.Then the accepted segments are merged with previous stem segment if only the corresponding angle difference between directions is less than a predefined threshold and the centre points are close-to each other, so that a new line segment is added to the stem growth line (figure 4).
The process is terminated if there are no more segments to be satisfied with the defined geometric criteria.At the end, the position of tree top is located using the detected line segments.
The main stages of the stem identification are shown in figure 5.

Filtering of tree stems from other pole type objects
Some other vertical objects, such as light poles, may also be seen in the selected tree stems and they should be removed.In this case, we search 2D geometric shapes, for instance minimum coverage, convex hull shape and diameter, of tree stems with neighbouring points around them.In general, the convex hull area of the vertically projected points exhibit more compact shape for trees; however it shows more elongated or small area for road poles.To analyse these geometric shapes, point coverage and the overlap between the convex hull area and the circle area which is defined by the tree top location and the longest distance from that point to the convex hull boundary, are examined.The given stem is discarded as a road pole if the area ratio has a value lower than given threshold value; otherwise it is established as a correct detection in our algorithm.

RESULTS EVALUATION AND DISCUSSION
We tested the results of our geometric descriptor based tree stem detection approach on a randomly selected sample area with total 42 trees.The visual appearance of the obtained tree stems is shown in figure 8. To evaluate the performance of the presented algorithm, interactively estimated statistics were used as a benchmark.Sensitivity of core parameters and completeness of the results are mainly discussed in this section.

Quality of point subset detection
The angle threshold (θ), which we used to aggregate points into point subsets, is greatly influence in the accuracy of isolating tree stem-and branch-points.Points may be hard to aggregate into a subset if θ is too small.This is mainly because small θ often leads to create several small subsets (over segmentation) which then will be removed by assuming as subsets referring to tree leaves.On the other hand, points from branches and leaves may wrongly add into a subset if θ is set to be large.Similarly, the neighbourhood (kn) also has an impact on the subset creation as it controls the size of subsets.For example; if the neighbourhood is too large, then points are more likely to be sampled from various surfaces.In our case, the best parameters were 30 points neighbourhood and 15 degree angle threshold.
Figure 6 shows the parameter sensitivity results.

Impact of shape analysis parameters
In the shape characterization process, the vital parameters were identifying of the shape index values.The S/L ratio has been applied well in several works as a measure of surface flatness while elongation is described by the ratio I/L as seen in Zingg diagram.In our study, the shape indices for flatness and elongation were set less than to 0.4 and to 0.9 respectively, which work well, for almost all the stems in the test site, for example see figure 7. The selected parameters are assisted to isolate points, representing tree stem and their minor branches, from points on tree leaves automatically.Additionally, the 12° angle threshold enables clustering of all stem subsets of each corresponding tree in our data set.Table 2 summarizes the overall performance of our method.As shown in the table 2, the degree of completeness, correctness and accuracy are above 90%, which states that all most all trees can be detected without having too much false detection.
According to the results, our approach can be validated as an efficient procedure having ability to achieve high accuracy in urban contexts.In general, low-vegetation, i.e. bushes, located close to the tree stem, may have taken an error on stem detection process.However, in our case, the geometric descriptor (principal direction) based point collecting property was assisted to reduce the influence given by nearby objects as well.

Table 2. Evaluation of automatic tree stems extraction
It is noted that some of the point subsets which belong to the leaves or branches have been erroneously segmented to the tree stems due to having very similar orientation and flatness.This mostly occurs in the upper part of the stem.This needs to be improved further.Other problematic issue arises when the majority of tree stem points are not recorded by laser scanner due to the occlusion.An example of cases leading to misrecognition of trees is shown in figure 9. Approximately, 7.1% of false alarm rate is attributed to this problem.

CONCLUSION AND FUTURE WORK
In this work, we presented a fully automated tree stem detection algorithm which fits with urban scene.Principal direction and shape are used as main geometric descriptors.Performance of the method is evaluated with respect to a manually segmented sample data set of our study area containing trees together with some buildings and other urban objects as well.The completeness and correctness, as described in our results, are above 90%.This indicates that, in a dense urban area, majority of tree stems can be demarcated accurately using our method on MLS point clouds.The algorithm does not rely on the prioriknowledge of the area or data and does not depend on the characteristics of trees, for examples tree structure.However, our algorithm may fail when the data gaps on tree stem do exist.
On the other hand, over-and under-segmentation can be seen where the tree branches are connected to the main stem or branches with similar orientation (very close to the main stem).
In future work, we will plan to extend this algorithm for the construction of all branches representing individual tree skeleton and we will focus on tree model reconstruction based on these results.

(
to find the local direction in 2D for image context, with the PCA is computed as it describes the geometry of the subset in scale independently(Simon et al. 2008).For a point subset S i | Si | and i is the number of subsets in a given cluster, the expected value is

Figure 4 .
Figure 4. Stem segments and their principal directions

Figure 7 .Figure 8 .
Figure 7. Analysis of a selected tree; (a) created subsets, (b) selected flat and elongated subsets from stem and branches, (c) final result of the stem detection (dark brown-stem, brown-base) and (d) detected stem5.3 Results of stem detectionThe evaluation is done by comparing the obtained results with manually estimated statistics.The number of correctly detected trees is reported as true positive (TP) while the number of not recognized trees and the number of falsely detected results are considered as false negative (FN) and false positive (FP) errors respectively.Completeness, correctness and accuracy of the results are determined based on the following equations; Figure 9.An example of false negative results (circles denote the occluded/missing stems)