Rule-based Segmentation of Lidar Point Cloud for Automatic Extraction of Building Roof Planes

This paper presents a new segmentation technique for LIDAR point cloud data for automatic extraction of building roof planes. Using the ground height from a DEM (Digital Elevation Model), the raw LIDAR points are separated into two groups: ground and non-ground points. The ground points are used to generate a 'building mask' in which the black areas represent the ground where there are no laser returns below a certain height. The non-ground points are segmented to extract the planar roof segments. First, the building mask is divided into small grid cells. The cells containing the black pixels are clustered such that each cluster represents an individual building or tree. Second, the non-ground points within a cluster are segmented based on their coplanarity and neighbourhood relations. Third, the planar segments are refined using a rule-based procedure that assigns the common points among the planar segments to the appropriate segments. Finally, another rule-based procedure is applied to remove tree planes which are small in size and randomly oriented. Experimental results on the Vaihingen data set show that the proposed method offers high building detection and roof plane extraction rates.


INTRODUCTION
Automatic reconstruction of buildings from aerial imagery and LIDAR (Light Detection And Ranging) data is a prerequisite for many GIS (Geographic Information System) applications, such as 3D building modelling (Zhang et al., 2005) and building change detection (Rottensteiner, 2007).
Different approaches for building roof reconstruction have been reported in the literature (Awrangjeb et al., 2013).In the model driven approach, also known as the parametric approach, a predefined catalogue of roof forms (eg, flat, saddle, etc) is prescribed and the model that best fits the data is chosen.An advantage of this approach is that the final roof shape is always topologically correct.The disadvantage, however, is that complex roof shapes cannot be reconstructed if they are not in the input catalogue.In addition, the level of detail in the reconstructed building is compromised as the input models usually consist of rectangular footprints.
In the data driven approach, also known as the generic approach (Lafarge et al., 2010) or polyhedral approach (Satari et al., 2012), the roof is reconstructed from planar patches derived from segmentation algorithms.The challenge here is to identify neighbouring planar segments and their relationship, for example, coplanar patches, intersection lines or step edges between neighbouring planes.The main advantage of this approach is that polyhedral buildings of arbitrary shape may be reconstructed (Rottensteiner, 2003).The main drawback of data driven methods is their susceptibility to the incompleteness and inaccuracy of the input data; for example, low contrast and shadow in images and low point density in LIDAR data.Therefore, some roof features such as small dormer windows and chimneys cannot be represented if the resolution of the input data is low.Moreover, if a roof is assumed to be a combination of a set of 2D planar faces, a building with a curved roof structure cannot be reconstructed.Nonetheless, in the presence of high density LIDAR and image data, curved surfaces can be well approximated (Dorninger and Pfeifer, 2008).
The structural approach, also known as the global strategy (Lafarge et al., 2010) or Hybrid approach (Satari et al., 2012), exhibits both model and data driven characteristics.For example, (Satari et al., 2012) applied the data driven approach to reconstruct cardinal planes and the model-driven approach to reconstruct dormers.A recent update on current reconstruction techniques can be found in (Haala and Kada, 2010).This paper presents an automatic and data driven approach to the extraction of building roof planes from LIDAR point cloud data.The raw LIDAR points are divided into two groups using a height threshold.The first group contains the ground points that are exploited to generate a binary building mask where buildings and trees are found as black shapes and ground and low height objects (eg, cars, road furniture etc) are represented by white areas.The second group has the non-ground points which are segmented to extract the planar segments.During segmentation, the building mask is first divided into small grid cells (1m 2 each) and the black cells are clustered based on their locality and number of black pixels.Each cluster indicates an individual building or tree and non-ground points within the cluster are coarsely segmented based on coplanarity and neighbourhood relations.The extracted coarse segments are then refined using a new rule-based procedure where common points among the segments are assigned to the appropriate segments based on their locality and neighbourhood.Finally, another rule-based procedure is followed to remove the planes extracted on trees, which are usually small in size and randomly oriented.The number of unused LIDAR points and the average height difference of the points within a tree segment are high.Experimental results show that the proposed method offers high building detection and roof plane extraction rates when applied to one of the ISPRS benchmark data sets.

RELATED WORK
In this section, a review of some of the prominent data driven methods which use raw or raster LIDAR data is presented.
Although algorithms using raster DSMs generated from the LI-DAR data (Rottensteiner andBriese, 2003, Chen et al., 2012) allow available image processing techniques for faster segmentation, they can suffer from the decreased information in the 3D point cloud.(Jochem et al., 2012) proposed a roof plane segmentation technique from raster LIDAR data using a seed point based region growing technique.Vegetation was removed using the slope-adaptive LIDAR echo ratio and the approach showed good object-based evaluation results on a large data set using a threshold-free evaluation system.
Among the methods that fully rely on raw LIDAR data, that proposed by (Perera et al., 2012) used a surface growing algorithm (Vosselman et al., 2004) for segmentation of the point cloud.
Then, a cycle graph was used to establish the topological relationship among the line segments extracted along the plane boundaries.This method failed in the absence of missing boundary lines and it displayed low geometric accuracy.(Dorninger and Pfeifer, 2008) proposed a comprehensive method for extraction, reconstruction and regularization of roof planes using LIDAR point clouds.Since the success of the proposed automated procedure was low, the authors advised manual pre-processing and post-processing steps.In the pre-processing step, a coarse selection of building regions was accomplished by digitizing each building interactively.In the post-processing step, the erroneous building models were indicated and rectified by means of commercial CAD software.Moreover, some of the algorithmic parameters were set interactively. (Sampath and Shan, 2010) presented a solution framework for segmentation (detection) and reconstruction of polyhedral building roofs from high density LIDAR data.Similar to the method in (Verma et al., 2006), the coplanarity of points was determined based on eigenvalue analysis using the Voronoi neighbourhood around each point.The normal vectors were then clustered via the fuzzy k-means algorithm.Good evaluation results for both segmentation and reconstruction were achieved.However, due to the removal of LIDAR points near the plane boundaries, the method exhibited high reconstruction errors for small planes.Furthermore, the fuzzy k-means clustering algorithm is computationally expensive (Khoshelham et al., 2005).(Kim and Shan, 2011) also segmented the normal vectors, but they applied a multiphase level set technique.(Tarsha-Kurdi et al., 2008) applied an extended robust estimation technique on the regenerated LIDAR point cloud.
After converting the original point cloud into a DSM, the missing points were estimated as the mean of the neighbouring points.Then a low-pass filter was applied and the raster point cloud was converted to the raw point cloud.As a result, the regenerated points suffered from decreased positional accuracy.Moreover, the method could not construct planes of less than 50m 2 .(Sohn et al., 2008) clustered building points first based on height similarity and then on planar similarity.They then extracted rectilinear lines and generated polyhedral models from the lines using a binary space partitioning tree.The method produced erroneous results due to improper localisation of the extracted lines or to missing lines.In addition, it failed to separate small roof planes in the clustering algorithm due to use of a predefined bin size for the height histogram.The main problem with methods using LIDAR data is that the planimetric accuracy is limited by the LIDAR point density.

PROPOSED ROOF EXTRACTION APPROACH
Figure 1 shows an overview of the proposed building roof extraction procedure.The input data consists of raw LIDAR data.
In the detection step (top dashed rectangle in Figure 1), the LI-DAR points are classified into two groups: ground points, such as ground, road furniture, cars and bushes that are below the threshold, and non-ground points, which represent elevated objects such as buildings and trees.The building mask, known as the 'ground mask', is generated using the ground points (Awrangjeb et al., 2010).Individual buildings and trees are obtained as clusters of black pixels in the building mask and trees with low density canopies are removed.The coplanarity of each individual non-ground LIDAR point is decided based on its Delaunay neighbourhood.The planar segments are extracted from the nonground LIDAR points on individual buildings and trees.The extracted LIDAR segments are then refined using a newly proposed rule-based procedure.Finally, the false planes on trees are removed using information such as area and neighbourhood, as well as any point spikes within the planar boundary.
Figures 2a presents a sample scene from the Vaihingen data set (Cramer, 2010), which will be used to illustrate the different steps of the proposed extraction method.The LIDAR point density in the Vaihingen data set varies from 4 to 6.7 points/m 2 (Rottensteiner et al., 2012).

Point Classification and Building Detection
For each LIDAR point, the corresponding DEM height (generated from the LIDAR data) is used as the ground height Hg.A height threshold T h = Hg + 1.0m is applied to divide the LI-DAR points into two groups: ground points such as ground, road furniture, cars and bushes that are below the threshold, and nonground points that represent elevated objects such as buildings and trees.The primary or building mask Mg, as shown in Figure 2b, is generated using the ground points following the procedure in (Awrangjeb et al., 2010)  the laser reflects, above the same threshold.However, it has been shown that buildings and trees are found to be thinner in Mg than in nDSM (Awrangjeb et al., 2012b).
The mask Mg is divided into 1m 2 grid cells.Figure 2c shows the grid, whose cells can be categorized into three groups.The cells in Group A shown in magenta coloured dots in Figure 2c, contain only black pixels and represent the areas inside the building or tree boundaries.In contrast, the cells in Group B (shown in cyan) contain both white and black pixels and represent the areas along the building and tree boundaries.The cells in Group C contain only white pixels and represent the ground.Group A cells are now separated into clusters, where any two Group A cells in each cluster are connected by other Group A cells belonging to the same cluster.As shown in Figure 2c, there are 5 such clusters in the sample scene.Thereafter, the Group B cells along the building boundary are added to each of the clusters.Each of the added Group B cells should have at least one Group A cell as its neighbour in the cluster.Finally, for each of the clusters, the cluster boundary is obtained using the Canny edge detector.Lines along the boundary are also extracted following the procedure in (Awrangjeb and Lu, 2008).These lines help to locate the roof planes near the building boundary.The nonground LIDAR points within the cluster boundary are assigned to the cluster.Figure 2d shows the clusters, their boundaries and non-ground LIDAR points for the sample scene.
The clustering technique described helps in the elimination of trees which are not dense (eg, tree at the bottom of Figure 2) and/or small in area.In addition, dense vegetation can be separated into small parts (eg, trees at the top-left corner of Figure 2).In a small vegetated area (eg, Cluster 4 in Figure 2d), it will be impossible to construct a large plane.Thus many planes con- structed on trees can be easily removed by applying the minimum plane size (see Section 3.3).

Extraction of Planar Segments
By using the Delaunay triangulation algorithm, a natural neighbourhood of non-ground LIDAR points can be generated for either one cluster at a time or all non-ground points at the same time.The neighbourhood of a point P consists of the points Qi, 1 ≤ i ≤ n, where each line P Qi is a side of a Delaunay triangle.In order to avoid points which are far away the following condition is applied: |P Qi| ≤ 2dmax, where dmax is the maximum point spacing in the data.The coplanarity of P is decided using its neghbouring points following the procedure in (Sampath and Shan, 2010).Points within a roof plane are found to be coplanar and those along the boundary of a plane are generally found to be non-coplanar.
For each cluster, let the two sets of the non-ground LIDAR points be S1 containing all the coplanar points and S2 containing the rest (non-coplanar).The first planar segment can now be initialised using a coplanar point P ∈ S1.Initially, P can be located along the cluster boundary using the extracted boundary lines.This new planar segment is extended using the neighbouring points from S1 and S2.Once its extension is complete, all the coplanar points in the extended planar segment are marked so that none of them is later used for initiating another planar segment.As a result, the points in S2, which mainly reside along the plane boundaries, can be used by more than one extracted plane.The second planar segment is grown by using an unused coplanar point (from S1) which may also be located along the boundary lines.Later, when no coplanar points are found along the boundary lines, an unused coplanar point is randomly selected and a new planar segment is grown.The iterative procedure continues until no coplanar point remains unused.
Figure 3(a) shows all the extracted planes from Cluster 1 of Figure 2d.There were 14 extracted planes.Many of the extracted planes overlap the neighbouring planes, ie, there may be many common points between two neighbouring planes.Most importantly, one of the planes has been wrongly extracted, as shown by red dots in Figure 3(a).This wrong plane was initialised by a coplanar point near to the ridge of two planes.As shown in the magnified part in Figure 3(a), the neighbours of the copalanar points reside on the two neighbouring planes.Consequently, the wrongly extracted plane included points from a total of six planes.
In order to refine the extracted planes, let P be a point between two neighbouring planes that may be coincident, parallel or nonparallel.The following rules are applied to decide the common  points.First, two coincident planes which share the majority of the plane points are merged.Second, for two parallel planes P is assigned to the plane with which it has a smaller normal distance.
Third, for two non-parallel planes, if P is coplanar then two angles are estimated between its normal and the two plane normals.P and its neighbours are then assigned to the plane with which the angle is smaller.Fourth, P can also be assigned based on its locality, ie, assign to the nearest plane.Finally, assign the remaining common points using the plane intersection line.
Figure 3(b) shows the planar segments after refinement.Figures 4a-b show all the extracted planar segments and their boundaries on the sample scene shown in Figure 2a.

Elimination of Trees
In order to remove false-positive planes, mostly constructed on trees, a new rule-based procedure is proposed.For an extracted LIDAR plane, its area, straight line segments along its boundary, and neighbourhood information, as well as any LIDAR spikes within its boundary, are used to decide whether it is a false alarm.For a given point on the extracted LIDAR plane, the mean height difference with its neighbouring points is also used.This height difference is large for a tree plane, but small for a roof plane.The average height difference for a plane is estimated from individual height differences within the plane.A LIDAR plane fitted on a tree is usually small in size and there may be some LIDAR spikes within its boundary.Moreover, there may be a large number of unused (ie, not on any of the extracted planes) LIDAR points within the boundary of a tree plane.The number of points used by the extracted planes is usually low on a tree cluster, but high on a building cluster.Moreover, there may be some long straight line segments (at least 3m long) along the boundary of a roof plane.
The proposed rule-based procedure works as follows: First, planes smaller than 1m 2 are removed.Second, a random point test is  (Awrangjeb et al., 2012a) and planes which show point spikes are removed.If the estimated height for a random point is smaller than 1.5m of the minimum plane height or larger than 1.5m of the maximum plane height then the point is a spike.Third, the ratio r of the number of the unused LIDAR points to the number of the used LIDAR points is calculated.If r ≥ 10% for a small plane (1 to 5m 2 ) or if r ≥ 35% for a medium plane (5 to 15m 2 ), then this plane is removed.Fourth, if the average height difference within a plane is high (more than 0.8m) then the plane is removed.Fifth, the ratio ru of the number of the used LIDAR points to the total number of the non-ground LIDAR points within a cluster boundary is calculated.If ru < 60% then all the surviving planes belonging to this cluster are further tested.If a plane does not have a long straight line segment along its boundary and its width is less than 1m, then this plane is removed.Finally, medium size planes that are neighbours of other planes that have already been removed are also removed if they have unused LIDAR points and high average height difference.Figure 4c shows all the roof planes found after the plane removal procedure for the sample test scene.
An individual building can now be easily obtained as a group of planes.All the LIDAR points from the group of planes are used together to extract the corresponding building boundary.A binary mask is formed using these points and the boundary of the building is the Canny edge around the black shape in the mask.The height of the nearest LIDAR point of the building is assigned to each edge point.Figure 4d shows the extracted building boundaries.

EXPERIMENTAL RESULTS
The Vaihingen (VH) data set (Cramer, 2010) from the ISPRS benchmark test (Rottensteiner et al., 2012) has been used in an experimental assessment and validation of the proposed roof plane extraction approach.There are three test sites in the VH data set.Area 1 is characterised by dense development consisting of historic buildings having complex shapes.Area 2 is characterised by a few high-rise residential buildings surrounded by trees, and Area 3 is purely residential with detached houses and many surrounding trees.The number of buildings (larger than 2.5m 2 ) in each of the these three areas are 37, 14 and 56, respectively, and the corresponding number of planes are 288, 69 and 235.
The results were evaluated using a threshold-based evaluation system (Rutzinger et al., 2009) and confirmed by the ISPRS commission III, Working Group 4.1 In the reference building sets, there were buildings which were as small as 2.5m 2 .
Figure 5 shows the building detection and roof plane extraction results for Area 1 of the VH data set, and Table 1 summarizes the building detection results for all three areas.Since the proposed algorithm missed some small buildings in all areas, the results for each area are similar both in object-and pixel-based evaluation.However, the large buildings were correctly extracted as evident from the completeness, correctness and quality indices for buildings over 50m 2 .This is also indicated by the pixel-based completeness, correctness and quality which are higher than their object-based counterparts.A large building in the top-middle of Area 3, although detected partially due to information missing in the LIDAR data, was found false negative by the evaluation system.That's why the completeness value for buildings larger than  1 is not 100% for Area 3. The under-segmentation cases occurred when the nearby buildings were found merged together.In Area 3, car ports were merged with the neighbouring buildings.This unexpected merging could be avoided by analysing white pixels in between the black shapes in the building mask (Section 3.1).The planimetric accuracies were close to the distance of 1 to 2 times of the maximum point spacing in the input LIDAR data.
Table 2 shows the roof plane extraction results.In the VH data set, the proposed algorithm performed better on Area 3 which comprised mainly residential buildings, with less vegetation.For all three areas, there were many under-segmentation cases where small roof structures could not be separately extracted, but might be merged with the neighbouring large planes, and some low height roof structures were missed.

CONCLUSION
A new LIDAR point cloud segmentation algorithm has been proposed for automatic extraction of 3D building roof planes from the raw LIDAR data.Firstly, the raw LIDAR data is divided into ground and non-ground points and a building mask is generated from the ground points.The black areas in the building mask represent the covered areas on the ground where no laser returns below a certain height.The building mask is divided into 1m 2 grid cells and the black cells are clustered such that each cluster represents an individual building or tree.The non-ground points within a cluster are then coarsely segmented based on their coplanarity and neighbourhood relations.The planar segments are refined using a rule-based procedure where common points between planar segments are assigned to the appropriate planes based on the relationship between the neighbouring planes and the locality of the points.Finally, a new rule-based procedure is applied to remove tree planes which are small in size and randomly oriented.In addition, within the area of an extracted plane on a tree, the average LIDAR height difference and the number of unused (not used for plane estimation) LIDAR points are usually high.
It has been shown via experimental testing that the proposed algorithm affords high building detection and roof plane extraction performance.It is not only capable of detecting small buildings but can also extract small roof planes on complex building roofs.Moreover, in most cases it can separate buildings which are surrounded by dense vegetation.
However, due to use of LIDAR data only, the planimetric accuracy of the proposed method is limited by LIDAR point density.Boundaries of the extracted planar segments are not smoothed.Future work will focus upon an integration of the available imagery with the LIDAR data so as to afford better planimetric accuracy.It will also look at development of a regularisation procedure to smooth the plane boundary.The integration of image ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume II-3/W3, 2013 CMRT13 -City Models, Roads and Traffic 2013, 12 -13 November 2013, Antalya, Turkey data will also help for better object extraction where LIDAR information is missing.

Figure 1 :
Figure 1: The proposed roof plane extraction technique.

Figure 2 :
Figure 2: A sample scene from the Vaihingen data set: (a) raw LIDAR points overlaid on an orthoimage, (b) building mask and non-ground LIDAR points, (c) grouping and clustering of the mask grid cells, and (d) clusters of non-ground LIDAR points on buildings and large trees.

Figure 4 :
Figure 4: Removal of false planes: (a) all extracted planes, (b) all plane boundaries, (c) final plane boundaries, and (d) building boundaries.

Figure 5 :
Figure 5: Building detection (a) and roof extraction results (b) on Area 1 of the VH data set.

Table 1 :
Building detection results for the Vaihingen (VH) data set.Object-based: Cm = completeness and Cr = correctness (Cm,50 and Cr,50 are for buildings over 50m 2 ) in percentage.Pixel-based: Cmp = completeness and Crp = correctness in percentage.Geometric: R b = planimetric accuracy in metre.

Table 2 :
Roof plane extraction results for the Vaihingen (VH) data set.Object-based: Cm = completeness and Cr = correctness (Cm,10 and Cr,10 are for planes over 10m 2 ) in percentage; Se = segmentation error (1 : M over-segmentation / N : 1 under-segmentation / N : M both over-and under-segmentation) in number of planes; Rp = planimetric accuracy and Rz = height error in metre.