ROADSIDE TREE EXTRACTION AND DIAMETER ESTIMATION WITH MMS LIDAR BY USING POINT-CLOUD IMAGE

: Efficient management of roadside trees for local governments is important. Mobile Mapping System (MMS) equipped with a high-density LiDAR scanner has the possibility to be applied to estimate DBH of roadside trees using point clouds. In this study, we propose a method for detecting roadside trees and estimating their DBHs automatically from MMS point clouds. In our method, point clouds captured using the MMS are mapped on a 2D image plane, and they are converted into a wireframe model by connecting adjacent points. Then, geometric features are calculated for each point in the wireframe model. Tree points are detected using a machine learning technique. The DBH of each tree is calculated using vertically aligned circles extracted from the wireframe model. Our method allows robustly calculating the DBH even if there is a hump at breast height. We evaluated our method using actual MMS data measured in an urban area in Tokyo. Our method achieved a high extraction performance of 100 percent of precision and 95.1 percent of recall for 102 roadside trees. The average accuracy of the DBH was 2.0 cm. These results indicate that our method is useful for the efficient management of roadside trees.


INTRODUCTION
Roadside trees are planted by various local governments, because they have the effects such as forming street landscapes and suppressing the temperature rise in summer. Unlike other road infrastructure, roadside trees grow over the years, so it is required to monitor their sizes periodically. However, not all local governments can update the management ledger for roadside trees every year. For example, if a roadside tree collapses due to a natural disaster such as a typhoon, it may be removed immediately without updating the ledger. Since there is often a discrepancy between the actual situation and the management ledger, the number and size of roadside trees are measured by field surveys in order to investigate the situation of roadside trees at a certain point in time. However, field survey is costly and time-consuming. In addition, field surveys have problems such as variations in measurement accuracy and oversights.
Alternative methods of field surveys are investigated to solve these issues. For instance, it is possible to estimate the localization, height, the spread of branches and leaves of roadside trees from aerial photographs. However, since aerial photographs are usually taken in a direction close to vertical, it is hard to estimate the Diameter at Breast Height (DBH) of roadside trees. The cost of maintaining roadside trees varies depending on their quantity and size, and it can be calculated by certain parameters such as DBH. Therefore, besides applying aerial photographs in the project, it is still necessary to conduct * Corresponding author a field survey to measure DBH separately.
So far, many methods using a Terrestrial Laser Scanner (TLS) for measuring trees have been proposed. TLS is placed on the ground with a tripod and captures dense point clouds of trees in a field. The BDH can be precisely calculated using such dense point clouds. However, TLS is not suitable for measuring roadside trees that are widely distributed, because the location of TLS needs to be adjusted frequently. Moreover, it is necessary to obtain a road occupancy permit or impose traffic restrictions depending on where TLS is located.
On the other hand, the Mobile Mapping System (MMS) is useful for efficiently surveying roadside objects in a wide field. In particular, MMS equipped with a high-density LiDAR scanner is widely used for map creation and infrastructure management because it can acquire 3D information around the road as highdensity point clouds. The MMS enables us to conduct surveys more accurately and efficiently than conventional field surveys. Surveys using the MMS running at a legal speed do not require road occupancy permits or traffic restrictions. In addition, since trees are measured from the horizontal direction, the DBH can be stably estimated. In this study, we considered only LiDAR scanners with a single sensor, such as RIEGL VQ250, instead of LiDAR scanners with multiple sensors, such as Velodyne VLP-16. This is because we cannot generate the trajectory-based point-cloud image used in this method from a LiDAR scanner with multiple sensors.

RELATED WORK
Most methods for estimating the DBH of trees are based on cylinder fitting or 2D circle fitting to points in a certain range of height. Čerňava et al. (2017) proposed a method that uses points with maximum distance among projected points to estimate DBH. Dralle and Rudemo (1996) estimated location of trees using aerial photographs. Nevertheless, estimation of DBH remains as a difficult task.
A large number of methods have been proposed for extracting trees and estimating their DBH from TLS point clouds, for instance, Olofsson et al. (2014), Pitkänen et al. (2019), and e.g. On the other hand, point density obtained by MMS is much lower than that of TLS. Therefore, it is difficult to reliably extract trees from the MMS point cloud unless the measurement conditions are ideal. Weinmann et al. (2017a) extracted and segmented trees using 3D features from point clouds acquired by MMS. However, the extraction results include not a small quantity of erroneous extractions and over extractions, which cause problems in practical use.
In recent years, researches using deep learning are also active. Luo et al. (2021) extracted individual trees by PDE-Net and obtained 0.95% of F-measure. Zhao et al. (2018) extracted trees by using course classification, two-dimensional region growing and filtering by feature constraints and good performance has been obtained. However, some parameters need to be carefully determined. Herrero-Huerta et al. (2018) estimated parameters of trees, including DBH, from the point cloud acquired by MMS. DBH is the diameter of the trunk at a certain height from the ground, but LiDAR does not guarantee to acquire a sufficient number of points at that height. They extracted points in a specific range of height, such as (1.2 m ~ 1.4 m), or (1.0 m ~ 1.6 m), and estimated the diameter at the height of 1.3 m by projecting the points on a 2D plane. However, the adequate height ranges depend on the measurement conditions and tree species. Therefore, trial and error are necessary for determining the height range.
To solve the issues of projection-based methods, Fukano and Masuda (2015) proposed a section-based method. This method calculates cross-sections at arbitrary positions by converting point clouds into a wireframe model. This method has the advantage of being able to obtain appropriately interpolated points even at the height where points are very sparse. However, their method was aimed at man-made objects such as utility poles, and did not evaluate the DBH of trees.

PROPOSED METHOD
The process flow of our method for roadside tree extraction is shown in Figure 1. MMS exports point clouds and a trajectory. Each point in point clouds is classified into trees and others. To obtain features of each point, a point cloud is converted into a wireframe model. To obtain adjacency relationships, point clouds are mapped on an image plane so that the MMS trajectory becomes the center line of the image. We call this image as the trajectory-based point-cloud image. The wireframe is generated by connecting adjacent points on the image. Then, the wireframe is subdivided into columnar and non-columnar parts. For each non-columnar part, the histogram of angles of edges in the wireframe is calculated to determine if the part is a roadside tree.
If it is a roadside tree, the position and DBH is estimated. Each step in this process is explained in detail in the following sections.

Classification
We classify each point in point clouds into trees and others using machine learning. Geometric features are calculated at each point using multiple neighbor distances, and classified using the Random Forest (Breiman, 2001).
Our method is based the method proposed by Weinmann et al. (2017b). They proposed 3D features, such as linearity, planarity, scattering, omnivariance, anisotropy, eigentropy, the sum of eigenvalues, change of curvature, radius, density, verticality, mean height, difference of height, and standard deviation of height. We also use the same 3D features. For neighbor distances to calculate 3D features, we used the fixed 5m, 3m, 2m, and 1m distances, and the distance with the minimum eigentropy. We also use the sum of eigenvalues, ratio of eigenvalues, radius, and density as 2D features. For neighbor distance to calculate 2D features, we use 5m, 3m, 2m, and 1m distances. We also add features for the number of points, height differences, standard deviation of height, and height from the ground by projecting points onto grids defined on the XY plane. In our method, the the size of the grid was 0.25m square. Height from the ground is the relative height, whose height is lowest among the grid. In addition, the absolute height of the point is also used as a feature.
The above features are obtained only from coordinates of point clouds. In a preliminary experiment, we detected trees using these features. However, the results were not satisfactory and these features were not sufficient for classifying roadside trees.
In general, the MMS outputs reflection intensity, GPS time, acquisition order, and vehicle trajectory as well as coordinates.
The vehicle trajectory consists of a sequence of coordinates with GPS time. Such features are specific for the MMS.
We introduce intensity, height from trajectory, 2D distance from trajectory, scanline length, and scanline curvature as additional features. The height from trajectory and the 2D distance from trajectory are defined as the relative height and distance from the nearest point on the vehicle trajectory.
A scanline is a consecutively acquired ordered points. If the distance between consecutive points is less than the threshold, they are considered to be the same scanline. In our method, the threshold was set to 0.3 m. Features for scanline length and  Figure 2. The scanline curvature is the ratio of the distance from the straight line connecting the start and end points of a scanline. All points in the same scanline share the same scanline length and curvature.
For classification, the random forest is used. To detect tree points, we define 10 classes, which are ground, grass, row vegetation, high vegetation, façade, car / bike / pedestrian, wire, traffic sign, pole, and road curb. We created training data to train a random forest classifier using MMS data obtained in an area different from the evaluation area. Figure 3 shows the classification results. In this classification, leaves part of high vegetation were classified approximately properly, but trunk part of high vegetation and poles were often misclassified. In addition, some trunks are misclassified into facades due to facades with columns.
To robustly detect roadside trees, we consider points classified into the three classes of high vegetation, poles, and facade as candidates of roadside trees. Candidates points are used for further classification and DBH calculation.
The contribution of each feature can be calculated in the random forests. The top 10 features with a high contribution in this evaluation are shown in Table 2. The first, second, and fifth places are features that are specific to the MMS. Such features made a significant contribution to improve the classification accuracy.

Generation of Wireframe
We generate wireframes from point clouds in order to robustly extract 2D circles of a tree trunk. To generate a wireframe model, we use the trajectory-based point-cloud image proposed by Takahashi and Masuda (2019). It is developed from the method proposed by Guinard and Vallet (2018). Figure 4 shows an image generated using this method. Each row in the image represents points acquired during one revolution of the laser beam. The nearest points are searched in the four directions on the image, and they are connected as edges. Then, distances are measured using 3D coordinates.
As shown in Figure 5, in point clouds acquired by MMS, the distance between scanlines is very large compared to the distance between points on the same scanline. Therefore,  traditional methods such as kd-tree do not work well to detect adjacent points. Figure 6 shows the comparison between the wireframes calculated by kd-tree and our method. Figure 6 (a) shows the results for a kd-tree with 8 neighbors (k = 8), which shows a large number of disconnections inside and notches at both ends of the pole. In Figure 6 (a)(b), while kd-tree with small k fails to generate edges between scanlines, kd-tree with large k generates inadequate edges inside the trunk. It is difficult to robustly generate circle sections from such wireframes.
Since wireframe models are used to extract 2D circles for estimating DBH, edges inside the pole are not desirable, because they will adversely affect the circle extraction performance and the accuracy of the estimated DBH. Figure 6(c) shows the wireframes created by our proposed method. Since scanlines are adequately connected, section circles can be robustly calculated. In addition, our method is very efficient, because the costly neighbor search is not required. Kohira and Masuda (2017) also proposed the method for generating 2D image from MMS point clouds, but their method does not use the vehicle trajectory and therefore, roadside trees may be largely skewed or split on the border of the image.
In this method, we create the wireframe is using the positional relationships in the trajectory-based point-cloud image. Even if the point clouds density is low, this positional relationship remains unchanged. Therefore, any density of point cloud can be used to create a wire frame robustly. However, this method cannot be applied to LiDAR scanner with multiple sensors as already mentioned.

Detection of Roadside Trees
In classification using the random forest, points of trees, poles, and façade are obtained as candidate points. In the next step, poles are detected from candidates points. First, a subset containing candidate points is extracted from the wireframe. The remaining wireframe model is then split into connected components, each of which is represented a wireframe. A pole is extracted from each wireframe.
Individual objects are separated to some extent after the initial classification. After that, the objects are cut out at a certain height range. For roadside trees, each trunks are sufficiently spaced that the individual trees can be almost completely separated in this step.
First, the wireframe is subdivided using horizontal plane to generate section points. Then, 2D circles are fitted to section points using the RANSAC method (Fischler and Bolles, 1981). The horizontal planes are generated at regular intervals for multiple heights. In our method, the interval was set to 5 mm. If circle diameters are not adequate as tree trunks, the circles are rejected. In our method, the range of diameters is defined as 3 cm to 40 cm. Our method allows detecting trunks with rapidly changing diameters, unlike cylinder-fitting methods.
Pole-like objects extracted in the previous section include not only roadside trees but also utility poles, traffic signs, and curve mirrors. Therefore, each wireframe is evaluated whether it is a tree or not. Evaluation has two steps. The first is the number of points of non-pole part. When a pole is detected from a wireframe, points on the pole is extruded from the wireframe model. If the number of remaining points is less than a threshold, the wireframe is considered not a tree. The second is the bias value of the wireframe angle histogram. The wireframe is smoothed using the Taubin filter (Taubin, 1995) to align the directions of edges. Figure 7 shows smoothed edges. Then, angles from the vertical direction are calculated for all edges in the wireframe. We observe that the distribution of edge angles is significantly different from trees and man-made objects. Therefore, we generate a histogram of edge angles, and calculate In this paper, the size of each bin of the histogram was set to 5 degree.

Estimation of Tree Position and DBH
Although the DBH is the diameter at 1.2 m, the circle diameter at the height may not be reliable. Figure 8 shows an example of a roadside tree with a hump and a stem branch at breast height, and the circle was not detected at the height. In this case, even of circles can be detected at breast height, the diameter is different from the actual situation.
In order to estimate the DBH robustly, we calculate the DBH from the extracted circles in a certain height range. As shown in Figure 9, the height and diameter of each circle is plotted on a graph, and a straight line is extracted using the RANSAC method and the least-squared method. Then the DBH is calculated as a point on the line. In this example, the DBH of a tree with a hump was corrected using the interpolation among neighbor circles.

EVALUATION METHOD
We evaluated our method using actual MMS data. The evaluation area was an urban area in Tokyo. We selected a route with a particularly large number of roadside trees as the target route. The length of the route, which is a straight road with one lane in each direction, is approximately 600m. The number of roadside trees in the target area is 102. Figure 10 shows the   Measurements by MMS was made in January 2020. The MMS was a Trimble MX-8 equipped with two RIEGL VQ-250 LiDAR scanners. The appearance of the MMS is shown in Figure 11 and the specifications are shown in Table 3.
This MMS is equipped with two LiDAR scanners, L and R, whose scan lines cross each other. Figure 12 shows an image of each LiDAR scanner measuring a roadside tree. Because the measurement timings of the L and the R LiDAR scanners are different, there is a possibility that errors are included in the registration of the point clouds. Therefore, we use only the point clouds measured by the L LiDAR scanner for evaluation in this study.
The evaluation index is the extraction rate of roadside trees and the difference between the ground truth of DBH and the estimated DBH. The locations of the roadside trees used as the ground truth were obtained by plotting tree locations from aerial photographs and correcting misread or undetected roadside tree locations based on the field survey. The field survey was made in August 2020. Some of the roadside trees was cut down by the time of the field survey. Therefore, we exclude these roadside trees in this evaluation.
We collected the ground truth of DBH by measuring trunk circumference with a tape measure in the field survey. Since the growth rate of dogwood is slow, the change of DBH between the MMS measurement and the measurement of field survey is considered to be sufficiently small. Since there are errors between locations plotted by aerial photographs and measurement results from MMS, we evaluated the success score based on whether there is a ground truth near the extracted location of trees or not. If there is ground truth, it is considered a true positive. On the contrary, it is considered a false positive. False negative is defined as there are no extracted trees in the neighborhood of the ground truth. The search distance of the neighborhood was set to 2m.

EXPERIMENT RESULTS
The evaluation results of roadside tree extraction by our proposed method are shown in Table 4. The extraction results for the entire experiment area are shown in Figure 13. The overall precision was 100% over the evaluation area, and recall was 95.1%, indicating very high extraction performance. There were two types of false negatives. One was that point clouds from the LiDAR scanner was missing or in extremely small number due to parked vehicles, as shown in Figure 14. The other was when multiple trees were treated as one object, as shown in Figure 15. We believe that we can solve the second problem by adding a process to find the columnar points that are far apart. This is our future work. Michikawa et al. (2015) also quantitatively evaluated the extraction rate of roadside trees, the evaluation was conducted on 123 roadside trees. The results were 108 true positives, 12 false positives, and 15 false negatives. The results of this paper are better than the previous study.
In this study, we showed the evaluation results only using point clouds from the LiDAR scanner L. The same results were obtained using the point clouds from the LiDAR scanner R. In addition, false-negative trees from LiDAR scanner L and LiDAR scanner R were different. The reason may be the different measurement timing and direction of the L and R, as shown in Figure 12, which changes the situation of the obstructions. Therefore, if we can integrate the L and R data at an appropriate timing after removing the registration error, we can expect to improve the extraction performance. This issue needs to address in the future.
A scatter plot of the DBH of the extracted roadside trees are shown in Figure 16. We plot the ground truth on the horizontal axis and the estimated value on the vertical axis. The results obtained by our proposed method are described as proposed. The results of the DBH using a conventional method, which projects point clouds in the range of 1.2m to 0.2m on a 2D plane  Table 5. Statistics of the accuracy of DBH and fits a circle, are called conventional. Note that we omitted plot points outside the display area for the conventional method in Figure 16(b).
The statistics of the accuracy of the DBH are shown in Table 5. The accuracy of our proposed method for calculating the DBH has been greatly improved, according Figure 16 and Table 5. Herrero-Huerta et al. (2018) used the same LiDAR scanners as we were. However, the RMSE of the DBH was reported 0.049m under the best conditions. We could estimate the RMSE with higher accuracy.
In the cases of trees with tree support, shown in Figure 17, our proposed method estimates the DBH including the tree support. Therefore, we exclude roadside trees with tree support in the evaluation. In the future, we plan to study how to estimate appropriately when such tree support is included in the point clouds. In addition, we would like to study the improvement of the overall accuracy.

CONCLUSION
In this paper, we proposed a method for extracting roadside trees and estimating their DBH using point clouds captured using MMS point clouds. The procedure is as follows. First, roughly classify the point clouds by random forests. Finally, we calculate the DBH. The DBH can be robustly calculated by estimating it from the circle fitting results in the certain height range of the pole part. Our method was evaluated using actual MMS data, and achieved a high extraction performance of 100 percent of Figure 17. Roadside trees with tree support precision and 95.1 percent of recall for 102 roadside trees. Extraction failed when point clouds was missing due to parked vehicles or when multiple trees were treated as one object. The average accuracy of the DBH was 2.0 cm. These performances are higher than those of previous methods. These results indicate that our proposed method is useful for the efficient management of roadside trees to local governments.

FUTURE WORK
First, it is necessary to consider the case where the objects are connected among the unextracted roadside trees. Second, it is necessary to consider the extraction by integrating point clouds from LiDAR scanner L and LiDAR scanner R. Third, for the calculation of the DBH, we will improve the performance when tree support exists and the overall accuracy.
In this study, we used the random forests approach for rough classifications. However, since we have created the trajectorybased point-cloud image, we believe that a CNN-based approach can be applied here. This method is promising, and we would like to investigate it in the future.