DETECTION AND CLASSIFICATION OF POLE-LIKE OBJECTS FROM MOBILE MAPPING DATA

Laser scanners on a vehicle-based mobile mapping system can capture 3D point-clouds of roads and roadside objects. Since roadside objects have to be maintained periodically, their 3D models are useful for planning maintenance tasks. In our previous work, we proposed a method for detecting cylindrical poles and planar plates in a point-cloud. However, it is often required to further classify pole-like objects into utility poles, streetlights, traffic signals and signs, which are managed by different organizations. In addition, our previous method may fail to extract low pole-like objects, which are often observed in urban residential areas. In this paper, we propose new methods for extracting and classifying pole-like objects. In our method, we robustly extract a wide variety of poles by converting point-clouds into wireframe models and calculating cross-sections between wireframe models and horizontal cutting planes. For classifying pole-like objects, we subdivide a pole-like object into five subsets by extracting poles and planes, and calculate feature values of each subset. Then we apply a supervised machine learning method using feature variables of subsets. In our experiments, our method could achieve excellent results for detection and classification of pole-like objects.


INTRODUCTION
Laser scanners on a vehicle-based mobile mapping system (MMS) can capture 3D point-clouds of roads and roadside objects while running on the road.Roadside objects, such as utility poles, traffic signs, and streetlights, have to be maintained periodically.Since there are a huge number of roadside objects in residential areas, their periodic maintenance tasks are very costly and time-consuming.If 3D digital data of roadside objects can be efficiently captured using a MMS, the current statuses of roadside objects will be easily investigated.
In our previous research, we proposed a method for reliably detecting cylindrical poles and planar plates from noisy pointclouds captured by a MMS (Masuda et al., 2013).We projected points on a horizontal plane and extracted high-density regions.As shown in Figure 1, the projection of pole-like objects produces high-density regions, and therefore, the search regions for pole-like objects can be restricted only in high-density regions.
It is often required to further classify pole-like objects into utility poles, streetlights, traffic signals, traffic signs, and so on, as shown in Figure 2. Since streetlights, traffic signals, and utility poles are managed by different organizations, their classification is very important for asset management and maintenance.Identification of object classes is also required for shape reconstruction, because knowledge-based methods are typical for reconstructing 3D shapes from incomplete point-clouds (Nan et al., 2010;Nan et al., 2012;Kim et al., 2012;Lin et al., 2013).Some researchers studied shape classification for pole-like objects (Yokoyama et al., 2011;Cabo et al., 2014;Yang et al., 2015;Kamal et al., 2013;Pu et al, 2011;Li and Oude Elberink, 2013).Their methods are based on threshold values of feature values, which have to be carefully determined by experiments.For identifying new object classes, new thresholds have to be investigated.Other researchers introduced supervised machine learning methods for classifying pole-like objects (Golovinsky et al., 2009;Zhu et al., 2010;Puttonen et al., 2011;Ishikawa et al., 2012;Lai and Fox, 2009;Munoz et al, 2009;Oude Elberink and Kemboi, 2014;Weinmann et al, 2014).In machine learning, threshold values for classification are automatically determined based on training data.However, their recognition rates were not high and the numbers of classes were very limited, because polelike objects have similar shapes and therefore they have similar feature values.In this paper, we propose a new method for classifying pole-like objects into representative classes.In our method, we subdivide a pole-like object into five subsets by extracting cylinders and planes.Then we apply a supervised machine learning method to each subdivided subset.Our approach is very effective for polelike objects, because many pole-like objects are characterized using partial shapes except common poles.
We also consider a new method for detecting pole-like objects.In our previous work, thin or low poles often fails to be extracted, because their projected points is not very dense, and therefore, it is difficult to robustly separate pole parts from what are not poles.Thin and low poles can be observed mainly in urban residential areas in Japan.
In this paper, we introduce a section-based method for robustly detecting pole-like objects.We convert a point-cloud into a wireframe model and search for a sequence of circular crosssections of the wireframe model.
In the following, we describe our new extraction method for polelike objects.Then we propose a classification method for polelike objects in Section 3. In Section 4, we show experimental results, and finally we conclude our work in Section 5.

Capturing Point-Clouds of Roadside Objects
A MMS is a vehicle on which laser scanners, GPSs, IMUs, and cameras are mounted, as shown in Figure 3(a).A laser scanner emits laser beams to objects while rotating the directions of laser beams, as shown in Figure 3(b).The rotation frequency of laser beams is determined by the specification of the laser scanner.
MMS data consist of a sequence of 3D coordinates, which are optionally combined with GPS time, intensity and colors.GPS time indicates when each coordinate was captured.Since 3D coordinates are stored in a file in the order of measurement, points can be sequentially connected in the order of GPS times, as shown in Figure 4.In this paper, we call sequentially connected points as scan lines.

Rough Segmentation of Scan Lines
When a pole-like object is measured using a laser scanner, relatively short scan lines are generated.Figure 5 (a) ~ (c) show scan-lines on a wall, a road, and a pole.Most scan-lines on walls are long straight lines, and ones on roads are long horizontal lines.On the other hand, scan lines on poles are circular and relatively short.
To select scan lines on poles, we apply the principal component analysis to each scan line and calculate eigenvalues λ1, λ2, and λ3 (λ1  λ2  λ3).We select scan lines as candidates of poles when λ1 < c1 and λ1 / λ2 < c2 are satisfied.The second condition means that scan-lines on poles are not straight lines but planar curves.
Figure 6 shows scan-lines that were selected as candidates of poles.In this example, we used c1 = 50cm and c2 = 100 according to experiments.The result shows that roads and walls can be effectively eliminated using the two conditions.
We project selected scan-lines onto a horizontal plane, as shown in Figure 7(a), and convert points into triangles using the Delaunay triangulation.We generate edges between points only when two points are vertices of a triangle and their distance is shorter than a threshold value.The span of scan-lines can be estimated as v/f, where v [m/sec] is the average speed of the vehicle, and f [Hz] is the frequency of the laser scanner.In this paper, we define the threshold value as kv/f, where k is a constant value based on the variation of the vehicle speed.In our examples, we supposed v=11.1 [m/sec] and k=2.5.The value of k was determined by experiments.
We segment scan-lines connected by the Delaunay triangulation into the same group.

Connection of Neighbor Scan Lines
Segmented scan-lines are candidates of pole-like objects.However, they may include no pole-like object, or may include multiple pole-like objects.Threfore, we detect nearly-vertical poles in the scan-lines.
To stably process sparse scan-lines, we define additional edges between scan-lines, as shown in Figure 8.We generated edges between scan-lines using the method proposed by Masuda et al. (Masuda and He., 2015).
Laser scanners rotate the direction of laser beams with a constant frequency f Hz, as shown in Figure 3(b).Therefore the differences of GPS times of adjacent scan-lines are approximately 1/f second.We suppose that the GPS time of point Pi is ti second, and the rotation frequency is f Hz.Then we can expect that the nearest point is detected near point Pn with the GPS time tn that satisfies: The nearest point of Pi is searched for in the range of [n-w, n+w].The value of w is determined based on the accuracy of rotational frequency.In our examples, we used w = 8 based on experiments.
By using this method, scan-lines are converted into connected wireframe models, as shown in Figure 8, even if scan lines are relatively sparse.

Detection of Pole-Like Objects Using Section Points
We place horizontal section planes at equal intervals and calculate section points between edges and the section planes, as shown in Figure 9(a).The intervals of section planes can be arbitrary determined.In this paper, we set the interval to 10 cm.Some researchers horizontally sliced point-clouds and segmented them into subparts (Pu et al., 2011;Li and Oude Elberink, 2013;Oude Elberink and Kemboi, 2014).However their method is not robust to sparse point-clouds.Since our method converts pointclouds into wireframe models, it can generate section points at arbitrary intervals even when point-clouds are relatively sparse.Then we merge section points that are vertically placed.As shown in Figure 9(b), we select section points on adjacent two section planes and project them onto the x-y plane.Then projected points are connected using the Delaunay triangulation.In Figure 9(b), two points are connected only when their distance is less than threshold D. When the number of connected points is larger than N1, they are classified into the same group.In this paper, we set the threshold D to 10 cm, and the threshold N1 to 16 based on experiments.This process is applied to each pair of adjacent section planes.Finally, vertically placed section-points are classified into the same group.When more than N2 layers of section planes are connected, points in the group are regarded as a pole-like object.In this paper, we set N2 to 16 based on experiments.
In Figure 10  We can extract original points using section points in the same group.We suppose that a pole-like object includes section points P={pi}, and P are generated as the intersection of edges E={ei}.
Then the both end vertices of E are selected as original points on a pole-like object.Figure 11 shows original points extracted using section points in Figure 10.
Then we detect additional objects that are attached to poles.
When points on poles are eliminated from segmented scan lines shown in Figure 7 In this section, we explained the method to extract points on each pole-like object.In the next section, we discuss how pole-like objects are classified into utility poles, streetlights, traffic signs, and so on.

Classes of Pole-Like Objects
Pole-like objects include various classes of objects.Since it is tedious work to manually identify classes of a huge number of pole-like objects, it is strongly required to automatically classify pole-like objects.Figure 13 shows examples of pole-like objects, which include utility poles, traffic signals, traffic signs, direction boards, streetlights, and trees.
Machine-learning methods are useful for automatically identifying object classes.In this paper, we consider supervised machine learning methods, which study the pattern of each class using labelled training data.In typical methods, each object is represented using a sequence of values, and a machine learning system constructs the criteria of classification based on the values.We describe a sequence of values as a feature vector, and each variable as a feature variable.et al., 2009;Ishikawa et al., 2012;Yokoyama et al., 2011;Chen et al., 2007;Cabo et al.. 2014;Yang et al., 2015).However, conventional feature variables are not sufficient to distinguish pole-like objects, because pole-like objects tend to have similar feature values.Therefore, existing methods could classify pole-like objects into a single or only a few classes.
To remedy this problem, we subdivide a pole-like object into several parts by extracting poles and planes.When poles and planes are extracted and removed from a point-cloud, other parts can be separated into multiple connected components.Connected components other than poles give valuable feature variables for distinguishing pole-like objects.
In this paper, we use the random forest for classifying pole-like objects (Fukano and Masuda, 2014).The random forest is a supervised machine learning method, and it is robust to missing values and outliers in feature values (Breiman, 2001).Since point-clouds captured by a MMS include a lot of noises and missing points, the random forest is suitable for our purpose.

Subdivision of Point-Clouds
We extract poles and planes from point-clouds, and subdivide a point-cloud into five point-sets.
Figure 14 shows a subdivision process.We refer the original point-cloud of a pole-like object as P. We first detect poles using our pole detection method.We refer points on a pole as A. The remaining points are referred to as B. Then we search for planar regions in B and refer the largest continuous plane as C. The remaining points are referred to as D.
Planes can be extracted using the RANSAC method.We randomly select three points and create a plane equation through the three points.We iterate this process many times and select a plane with the maximum number of neighbor points.Then we extract the largest connected plane.When the plane is smaller than a threshold, it is discarded as a false one.
Finally we can obtain five subsets P, A, B, C, and D including the original point-set.When a plane cannot be detected, subsets C and D become null sets.

Feature Variables for Machine Learning
We calculate the following feature variables for each subset, and combine them into a single feature vector.The feature values 1~4 have been commonly used for classification of objects, but conventional methods calculated them only using all points P (Munoz et al, 2009;Weinman et al, 2014).Our method has five times more feature variables in feature vectors.Distances in the feature values 5 represent relationships between subsets, and the feature value 6 is introduced to distinguish man-made objects from trees.

Sizes of bounding box:
The bounding box of an object represents a rough shape.Since we consider pole-like objects, we create a bounding box that is perpendicular the horizontal plane.Figure 15 shows some examples of bounding boxes of pole-like objects.We calculate the two principal directions using (x, y) coordinates, and generate a rectangle so that all points are covered.We call the longer edge of the rectangle as the width, and the shorter one as the depth.The length of the bounding box in the z direction is called the height.

Eigenvalues:
We apply the principal component analysis (PCA) to each subset for calculating eigenvalues.Since eigenvalues are equivalent to variances in the principal directions, they also represent the sizes of objects.While bounding boxes are defined on the XY plane, the principal axes have arbitrary directions.

Ratios of Eigenvalues:
We describe eigenvalues as λ1, λ2, and λ3 in descending order.Ratios among eigenvalues can be used to classify linear objects and planar objects, because a linear object has a large value of λ1/λ2, and a planar object has a small λ1/λ2 and a large λ1/λ3.These ratios are added in feature vectors.
Although various feature values based on PCA eigenvalues have been proposed (Weinman et al., 2014), we use only simple combinations of eigenvalues in this paper.This is because the recognition ratios could not be improved in our experiments when other feature values based on eigenvalues were incorporated into feature vectors.

Number of subsets:
In case of objects having attachments, multiple subsets are detected in subdivision pointclouds, as shown in Figure 16.So we count detected subsets, and the numbers are added to feature variables.

Distances between subsets:
The relative distances of subsets are added to feature vectors.Distances are measured between the center positions of point-sets.

Ratios of edge directions:
A point-cloud is represented as a wireframe model.Then edges in the wireframe model are smoothed using the Taubin filter, which is a well-known lowpass filter for 3D objects (Taubin, 1995).While edges of a tree have various directions, ones of a man-made object tend to have horizontal or vertical directions.Figure 17 shows smoothed edges of a tree and a man-made object.Edges are classified into three groups based on the angles from the horizontal plane.When the angle of an edge is less than 30°, it is categorized into the horizontal group.When the angle is larger than 60°, the edge is classified into the vertical group.Other edges are categorized into the angled group.We count the number of edges for each group and calculate three ratios nh /n, nv /n and na/n, where n is the total number of points; nh, nv, and na are the numbers of horizontal, vertical, and angled edges, respectively.

Classification using Feature Vectors
In the list of feature variables, features 1~4 are calculated for each of five point-sets; feature 5 is calculated for two pairs (A, B) and (C, D); feature 6 is calculated only for P. Therefore, we totally define 55 feature variables, as shown in Table 1.
The number of our feature variables is much larger than ones of conventional methods.When the random forest is applied to feature vectors with 55 variables, it automatically constructs decision trees.If important features are included in variables of subsets A, B, C, and D, our method can improve recognition ratios compared to conventional methods, which use only variables of all points P.

Detection of Pole-Like Objects
We extracted poles from point-clouds measured in a residential district in a Japanese urban city.The number of points was 1.3 hundred millions.A lot of low pole-like objects are included in this district.In our experiments, we automatically extracted polelike objects and displayed them with the whole point-clouds on the screen.Then we visually verified extracted objects by comparing to point-clouds on the screen.
We evaluated the recall, precision, and F-measure.F-measure is a harmonic mean of recall and precision.In this paper, we call our proposed method as the section-based method.We compared this method to the projection-based method, which was proposed by Masuda et al. (Masuda et al., 2013).The projection-based method projects all points on a 2D plane and searches for circles in dense regions.
When the projection-based method was applied to point-clouds captured in a suburban city, the recall was 88% (Masuda et al., 2013).However, when we applied this method to point-clouds captured in the urban residential district, the recall was reduced to 68%, because a lot of low poles were included in this area.
We applied our section-based method to the point-clouds of the urban residential district.Our method could detect most pole-like objects, and the recall was improved to 93%.Missing poles were categorized into two cases.In the first case, points on poles were largely missing and they were not regarded as poles because of the small numbers of circles.In the other case, large objects were directly attached to poles.The precision of the section-based method was 93%.In failure cases, rainwater pipes attached to walls and columns of buildings were detected as pole-like objects.
In Table 2, the F-measure is shown with the recall and precision.
The result shows that the success rate of pole detection is largely improved by the section-based method.

Classification of Pole-Like Objects
We evaluated our classification method.We used point-clouds captured in two different areas to confirm the transferability of our method.One was captured in a residential district and the other was captured in a highway in Japan.
We automatically segmented pole-like objects and manually added their labels.Figure 18 shows numbers of pole-like objects that were used in our experiments.Then we split labelled objects into training data and validation data on halves.
Training data was used to reconstruct decision trees for the random forest.When we applied the random forest, we determined the number of decision trees as 500 and the maximum depth of trees as 15.These parameters are experimentally determined.The validation data is used for evaluating success rates of classification.
Table 3 shows  Table 4 shows the recall, precision, and F-measure when objects were classified using 55 feature variables of all subsets.For comparison, we also classified the same samples only using feature variables of all points P, and the recall, precision, and Fmeasure of the classification results are shown in Table 4.The results show that our method achieved excellent recognition ratio.
Our experimental results show that variables of subsets A, B, C, and D include important features for classifying pole-like objects.
Our subdivided subsets could produce feature variables suitable for differentiating pole-like objects.

CONCLUSION
We proposed methods for extracting and classifying pole-like objects captured by a MMS.We robustly extracted a wide variety of pole-like objects by converting point-clouds into wireframe models and calculating cross-sections between wireframe models and cutting planes.For classifying pole-like objects, we subdivided a pole-like object into five subsets by extracting poles and planes, and calculate feature values of each subset.Then we applied the random forest using feature variables of subsets.In our experiments, our method could achieve excellent extraction and classification results for pole-like objects.
In future work, we would like to investigate other feature variables, such as trajectory lines of vehicles.We would also like to apply our classification method to a wide variety of 3D objects other than poles, and investigate what feature variables would be effective for various objects.In addition, we would like to classify objects in a variety of areas, and confirm the transferability of our approach.When object classes are known, template-based shape modeling would be effective to reconstruct 3D shapes.We would like to investigate shape templates that are suitable for various classes of objects.

Figure 1 .
Figure 1.Extraction of high-density regions

Figure 5 .
Figure 3. Laser scanning by a mobile mapping system Figure 7. Segmentation of scan-lines , groups of section points are shown in different colors.Scan-lines are shown in black.Although utility poles in Figure 10(a) are located next to trees, they could be successfully extracted.In Figure 10(b), trees with curved trunks could be also extracted as pole-like objects.

Figure 8 .
Figure 8. Addition of edges between scan-lines (b), each segment is subdivided into some connected components.Since attachments of poles are placed at high positions, components at low positions are discarded.Each connected component is added to the nearest pole.Figure12shows examples of detected pole-like objects.In this figure, additional components are shown in magenta, and pole-like objects with attachments are shown in dashed lines.

Figure 11 .
Figure 11.Original points of pole-like objects Figure 13.Pole-like objects on roadsides

Figure 14 .
Figure 14.Five groups of point-sets

Figure 16 .
Figure 16.A signal having multiple subsets classification results.The numbers in the diagonal elements indicate successful classification.Each row indicates how objects in each class are classified.column shows the number of objects that are classified into each class.

Figure 18 .
Figure 18.Pole-like objects used in experiments

Table 1 .
Feature variables for pole-like objects

Table 3 .
Classification results

Table 4 .
Recognition rates for feature variables of all subsets and only P