CHANGE DETECTION IN 3D POINT CLOUDS ACQUIRED BY A MOBILE MAPPING SYSTEM

: Thanks to the development of Mobile mapping systems (MMS), street object recognition, classiﬁcation, modelling and related studies have become hot topics recently. There has been increasing interest in detecting changes between mobile laser scanning (MLS) point clouds in complex urban areas. A method based on the consistency between the occupancies of space computed from different datasets is proposed. First occupancy of scan rays ( empty , occupied , unknown ) are deﬁned while considering the accuracy of measurement and registration. Then the occupancy of scan rays are fused using the Weighted Dempster–Shafer theory (WDST). Finally, the consistency between different datasets is obtained by comparing the occupancy at points from one dataset with the fused occupancy of neighbouring rays from the other dataset. Change detection results are compared with a conventional point to triangle (PTT) distance method. Changes at point level are detected fully automatically. The proposed approach allows to detect changes at large scales in urban scenes with ﬁne detail and more importantly, distinguish real changes from occlusions.


INTRODUCTION
Change detection techniques have been developed for decades.Even though change detection using conventional remote sensing techniques is still popular nowadays, the potential for a laser scanning system for change detection in urban areas has been discussed earlier on by Murakami et al. (1998).Digital surface models (DSMs) acquired at different times were subtracted then a simple shrinking and expansion filter was utilized to remove edges of unchanged features.Technically, changes were detected by processing the differential images in 2.5 dimensions (2.5D).This methodology can be found in most later studies (Tian et al., 2013).
In terms of platforms, airborne laser scanning (ALS) has been adopted in most studies since it has many advantages, e.g.large coverage over a short time span, high accuracy in three dimensions (3D), foliage penetration, etc (Rutzinger et al., 2010).Terrestrial laser scanning (TLS), which is often utilized for a specific single object, e.g.landslides (Hsiao et al., 2004) or other deformation measurements (Monserrat and Crosetto, 2008), has smaller coverage but higher point density and better accuracy.Mobile laser scanning (MLS) has the advantages of both.More detailed features are extracted by MLS even in a complex urban environment (Zhou and Vosselman, 2012).Streets and their facilities have been detected and modelled by MMS (Pu et al., 2011).More detailed changes may be detected thanks to the development of MMS.Change detection helps maintain and update databases.Moreover, the mobility of street objects, which will facilitate the assessment of pavement accessibility, can be obtained by change detection.In addition, real time change detection can be used for motion tracking.
Several reviews have categorized change detection approaches from different perspectives.Singh (1989) generalized the approaches into two categories, i.e. classification-based comparison and direct comparison of raw multi-temporal data.In order to detect changes in ALS point clouds, most studies adopt the former approach by converting ALS point clouds to DSMs and then classifying them in 2.5D.Some researchers classify point clouds directly in 3D as urban objects, e.g.buildings, trees, then detect the changes in 3D (Xiao et al., 2012).
The second approach in which raw data are compared has been mostly used in TLS and MLS data.Monserrat and Crosetto (2008) estimated deformation parameters in TLS data using least squares 3D surface matching.Girardeau-Montaut et al. (2005) proposed a framework to semi-automatically detect changes of a construction site in terrestrial lidar data.Point clouds were directly compared using three methods, i.e. average distance, best fitting plane orientation and Hausdorff distance (the maximum distance from a point in one set to the closest point in another set).Hausdorff distance performed best.A local model for distance calculation was suggested in order to avoid density variation issues.Point-topoint (PTP) distance and similar methods are more practical for TLS and MLS data because changes can be detected directly in 3D.Nevertheless, PTP distance is very sensitive to point density.Hebel et al. (2011) adopted the concept of occupancy grids from robot mapping to detect changes from ALS data.Data were voxelized and indexed by the voxel position.Then space was defined as empty, occupied and a degree of ignorance and they were modelled by sigmoid functions.The Dempster-Shafer theory (DST) was applied to combine multiple measurements.Changes in vegetation, buildings and cars were successfully detected.However, the minimum detectable size was still large because of the low point density of ALS data.Point acquisition and registration accuracy have not been considered.There is a need for a more accurate occupancy model considering uncertainties.
The aim of this paper is to detect the changes from multi-temporal MLS point clouds in a complex street environment.An occupancy based consistency assessment method is presented.First of all, the data acquisition geometry and test sites are demonstrated in Section 2. Details of the method are presented in Section 3 including the occupancy theory and functions, intra-scan ray fusion by DST and inter-scan consistency assessment.Results are illustrated in Section 4 followed by a discussion.Finally, conclusions are drawn in Section 5 and future work is also suggested.In addition to the xyz coordinates, the origin and rotation angle of each point are known.The ray is not a perfect line but a cone-like beam with a particular footprint.The angular resolution α = 0.4 degrees and the range accuracy σm = 25mm.

ISPRS
The distance D between two scan lines varies because it depends on the inconsistent speed of the vehicle.Point densities in vertical and horizontal directions are inconsistent since the horizontal sampling depends on the vehicle speed while the vertical density depends on the object distance and the incidence angle.

Test Sites
The test street was scanned several times by the MMS.Since we concentrated on streets, road objects and pavements, only the lower parts of streets were in the scanning scope.Tree crowns and upper part of buildings were not scanned.To generate sufficient changes, some objects in the street scenes were laid out differently from one scan to another.A motorcycle, a bicycle, movable garbage bins, chairs and a table, pedestrians and some barriers were moved randomly for different scans.Figure 2 shows two sites containing larger changed objects of two scans.
Different scans of the interest site were registered by using a rigid registration method proposed by Gressin et al. (2013).According to visual inspection, the accuracy of registration σr is about 10cm.

Occupancy Definition
A scan ray reflects a point from objects, meanwhile it indicates the occupancy (empty or occupied) of space.The scene covered by the scan can be represented by occupancy.If the scene changes, the occupancy of space will change as a result.
The state of space occupancy can be represented by the universal set X = {empty, occupied} .The power set of X, 2 X = {∅, {empty}, {occupied}, {empty, occupied}}, contains all the  In scan one, there was a van and a man changing the poster on the billboard (lower right, site 1).None of these were present in scan two.On another site in scan one (upper left, site 2), there was a car coming out of the garage.In scan two, a van parked beside the pavement which was not there in scan one.
subsets of it.Due to occlusions of data acquisition, in shadow areas no information is obtained hence the occupancy is unknown.
When the occupancy of space is unknown, it can be either empty or occupied, so the state unknown is represented by set {empty, occupied} in the power set.According to DST, the mass of each element of the power set is within the range of [0, 1].Moreover, the mass of the empty set is 0 and the masses of the remaining subsets add up to a total of 1 : The mass of occupancy is denoted by m, and for each element, m({empty}), m({occupied}) and m({unknown}) are denoted by e, o and u respectively.To define the mass of occupancy of scan rays, a local reference frame is presented.Figure 3 First we define the occupancy masses along the ray direction (er , or , ur).When P is between Ot and Q, it is empty, so er = 1; or = 0.When P is at the position of Q, er = 0; or = 1.
When P is behind Q, the mass of occupancy is represented by a Gaussian function.or decreases from 1 to 0, and ur increases from 0 to 1 accordingly (Figure 4).The mass of occupancy is high near a ray while decreasing as it moves away from the ray.Thus the occupancy in the rotation and trajectory directions are represented as follows: ) 2 (3) Theoretically, the mass of occupancy in the rotation direction should be the same as in the trajectory direction.However, sampling densities are not the same in these two directions as mentioned in Section 2. To overcome this anisotropic sampling, the masses of these two directions are considered as different.The gaps between scan lines are inconstant since the vehicle speed varies.Thus λt should be large enough to avoid the gap while it is not too large either.
The overall occupancy of point Q at location P are functions of parameters r, t, θ:

Uncertainty Modelling
Uncertainties of measurement (ranging) and registration are taken into consideration since they can induce changes.Both of them are represented by normal distributions g(m) and g(r) with σm = 2mm and σr = 10cm.
In the ray direction, both of these uncertainties exist so they are both convolved with occupancy functions.The convolution of two Gaussian distributions is: Then the occupancy functions in the ray direction are convolved.
When r = 0, the point is on the edge of an object (interface of empty and occupied).Due to uncertainties, the point can be on either side of the interface hence e r = o r = 0.5 (Figure 5).The occupancy here is half empty and half occupied which is different from unknown where no information is obtained.It is more certain that the point is on the occupied side when it is behind the interface.Thus the mass of occupied increases and reaches a maximum behind the point and then deceases because of a lack of information.The shift s between the maximum value location and the original point depends on uncertainties.
In the trajectory direction, the measurement uncertainty can also be neglected.Thus, the occupancy function is: ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume II-5/W2, 2013 ISPRS Workshop Laser Scanning 2013, 11 -13 November 2013, Antalya, Turkey After modelling uncertainties, the occupancy function becomes: The three elements of occupancy are within the scope of 0 and 1 and they add up to 1 so they can be considered as barycentric coordinates.They are illustrated by a triangle (Figure 6).Occupancy around a laser beam is within this triangle (Figure 7).Three vertices represent empty (red), occupied (green) and unknown (blue) clockwise.

Intra-Scan Fusion
At a given location, all the scan rays around give evidence of occupancy, so the occupancy over the whole scene are a combination of occupancy from all the scan rays.The overall occupancy is combined by the DST.Dempster's rule of combination is as follows: where B ∈ 2 X , C ∈ 2 X and K is the conflict between two mass sets: In our case, {e} and {o} are the subsets of {u}.The combined occupancy of two rays m1 ⊕ m2 is: The combination rule is commutative and associative, so the order of combination is arbitrary.Then at a location P with I number of neighbouring rays Qi, the overall occupancy is:

Inter-Scan Consistency Assessment
To detect the changes between two datasets, we define consistency relations between their occupancies.They are conflicting (Conf ) when one dataset is empty whereas the other dataset is occupied or vice versa, consistent (Cons) when they have the same occupancy state, and uncertain (U nc) if one dataset is unknown whereas the other one is known.The consistency relations between two datasets, (E1, O1, U1) and (E2, O2, U2), are defined as: To compute the consistency relations, one simple method is by voxelizing space, computing the occupancy of each dataset on each voxel, and then comparing the occupancy values of two datasets on every voxel (Hebel et al., 2011).This method provides the consistency result to both datasets at once.However it is computationally expensive.
There is no need to compute the occupancy over the whole space since we are interested in changes (conflicts) which only happen near the acquired points.According to Section 3.1, the occupancy at a scanned point in one dataset m(P ) = (e1 , o1 , u1) = (0, 1, 0).After considering uncertainties, the maximum occupied value shifts slightly behind to Ps, m(Ps) = (e 1 , o 1 , u 1 ) (0, 1, 0) (Figure 5).Then to obtain the consistency information of this point, we only need to compare the occupancy of the other dataset at Ps, which is the combination of occupancies of neighbouring rays.The consistency relations at Ps are as follows: Conflicting means the occupancy of the point has changed with regard to the other dataset.Consistent means there is no change.Uncertain means that either the point is in the shadow or there are no counterparts in the other dataset.
For thin pole-like or hollow objects, few rays can hit them whereas most rays will miss them and reach the ground behind.Then points on these objects will have less consistent evidence but more conflicting evidence from another dataset.However, if the objects really changed, there would be no consistent evidence, which means that consistent evidence strongly suggests there are no changes.Thus we weighted the DST in favour of consistent evidence.It was given more weight than conflicting evidence when contradictions between them occurred.

Consistency Assessment Result
Each point has the three consistency elements ranging from 0 to 1.The largest value among them can be considered as dominant hence it is the final consistency result.Figure 8 shows the result of consistency assessment.The angular resolution is constantly 0.4 degrees, so λ θ is set as 0.2 degree which covers the gap and meanwhile does not overlap heavily with neighbouring rays.λt is set as 5cm according to the gaps between scan lines.To assess the result of the proposed method, we compared it with a conventional point-to-triangle (PTT) distance method.There are many sophisticated methods, e.g.comparing local point subset to meshing surface or to model, average distance of two local subsets.However, they all share the same characteristics as direct point to point comparison which suffers from irregular point density.To minimize the effect of anisotropy, we computed the distance to the nearest triangle instead of the nearest point.A certain number of nearest points (n=10) were retrieved for the target point, then a Delaunay triangulation was implemented on these points.Afterwards, the nearest triangle was found considering the distance from the target point to each triangle.If the target point project within the triangle, the distance to the projection is the PTT distance.If the projection is outside the triangle, the distance to the nearest point of the triangle is considered as the PTT distance.
As depicted in Figure 10, the changes at the two sites were also detected correctly.However, all the changes are labelled with the same color (red).Thus it is impossible to distinguish between change and occlusion or absence of counterpart.Occlusions are commonly encountered in MMS data.It is essential to distinguish real changes and those caused by them.However, they were falsely detected as changes by the PTT method as shown on site 2, points in the shadow of the van were considered the same as other changes.If the two scans do not have the same boundary, points that have no counterparts in the other dataset will also be considered as change.On site 1, all the points on the van were detected as change, including the points that were not in the scanning scope of the other dataset.In addition, the global threshold (0.3m) may not be suitable for all datasets when there are more than two multi-temporal datasets.

Experimental Results
To verify the change detection results, some urban objects were moved for different scans.As shown in Figure 11, large objects, e.g.garbage bins, pedestrians, a motorcycle, were well detected.Hollow objects, e.g. a bicycle, chairs and a table, and even a small pole-like object, about 20cm wide, were detected.The results indicate that the detection is reliable and the detectable changes can be quite small.
The minimum detectable size of changed objects depends on point density.To avoid false detections, a single point or a few points in one line are eliminated.Thus the object size should be wider than at least twice the gap between scan lines and also tall enough to be detected.Registration quality will directly affect the change detection results.With high point density, many false changes will be found even small objects can be detected if registration quality is low.
There were false detections on the edges of objects since point distribution at those locations varied even without change.These

CONCLUSIONS AND FUTURE WORK
To detect changes between two MLS datasets, a method based on the physical scanning mechanism (scanning origin and geometry) is presented.The prerequisite of this method is knowing the origin of each scanned point.The scan ray is reconstructed by the origin and the point.Then the occupancy of space is derived from rays.The WDST combines the evidence from different rays and meanwhile minimizes the contradiction of evidence.Consistency relations between point and neighbouring rays from different datasets are assessed by the concept of conflict from the DST.Compared with the conventional PTT method, this occupancy based method is informative and theoretically accurate.It is able to distinguish occlusions and points without counterparts from real changes.Urban object changes are automatically detected at point level in a complex street environment.Experimental results show the capability and reliability of the proposed method.
Future work will focus on recognising the changed points.They are to be clustered and classified.Time series change detection will also be explored.Moreover the method will be applied on Velodyne data and perhaps on synthetic data.

Figure 1 :
Figure 1: Laser scanning geometry (a) scan one and zoomed in views (b) scan two and zoomed in views

Figure 2 :
Figure2: Two scans of test sites (coloured by height).In scan one, there was a van and a man changing the poster on the billboard (lower right, site 1).None of these were present in scan two.On another site in scan one (upper left, site 2), there was a car coming out of the garage.In scan two, a van parked beside the pavement which was not there in scan one.

Figure 3 :
Figure 3: Relative position of P and Q − − → OtQ .θ, r, t represent the relative position of point Qt and P in a local cylindrical coordinate system.

Figure 4 :
Figure 4: Occupancy in ray direction

Figure 5 :
Figure 5: Occupancy in ray direction considering uncertainties In the rotation direction, the measurement uncertainty is neglected since the ranging uncertainty is mainly in the ray direction.The registration uncertainty has a proportional factor on the angle accuracy.Because sinθ θ when θ is very small, thus σ θ = σr/|| − − → OtQ||.Then the angle occupancy function becomes:

Figure 8 :Figure 9 :
Figure 8: Consistency assessment resultsFigure9shows the final result of change detection.Similar to the occupancy triangle, red, green and blue demonstrate conflicting, consistent and uncertain respectively.On site 1, the man and the billboard were detected.The van was mostly detected as change and a small part as uncertain because in the second data the MMS vehicle passed through the position of the volume that the van occupied in the other data and only a part of the volume was scanned.On site 2, the car was detected as change.And the place where the van from the second scan was parked was detected as uncertain since this part was in the shadow.

Figure 10 :
Figure 10: Point to triangle distance

Figure 11 :
Figure 11: Detected changed urban objects false changes are normally a few points irregularly distributed or within one line.Thus they can be considered as noise in the following process.