HIGH ACCURATE POINTWISE ( GEO-) REFERENCING OF A K-TLS BASED MULTI-SENSOR-SYSTEM

In recent years, the requirements in the industrial production, e.g., ships or planes, have been increased. In addition to high accuracy requirements with a standard deviation of 1mm, an efficient 3D object capturing is required. In terms of efficiency, kinematic laser scanning (k-TLS) has been proven its worth in recent years. It can be seen as an alternative to the well established static terrestrial laser scanning (s-TLS). However, current k-TLS based multi-sensor-systems (MSS) are not able to fulfil the high accuracy requirements. Thus, a new k-TLS based MSS and suitable processing algorithms have to be developed. In this contribution a new k-TLS based MSS will be presented. The main focus will lie on the (geo-)referencing process. Due to the high accuracy requirements, a novel procedure of external (geo-)referencing is used here. Hereby, a mobile platform, which is equipped with a profile laser scanner, will be tracked by a laser tracker. Due to the fact that the measurement frequency of the laser scanner is significantly higher than the measurement frequency of the laser tracker a direct point wise (geo-)referencing is not possible. To enable this a Kalman filter model is set up and implemented. In the prediction step each point is shifted according to the determined velocity of the platform. Because of the nonlinear motion of the platform an iterative extended Kalman filter (iEKF) is used here. Furthermore, test measurements of a panel with the k-TLS based MSS and with s-TLS were carried out. To compare the results, the 3D distances with the M3C2-algorithm between the s-TLS 3D point cloud and the k-TLS 3D point cloud are estimated. It can be noted, that the usage of a system model for the (geo-)referencing is essential. The results show that the mentioned high accuracy requirements have been achieved.


INTRODUCTION
Terrestrial laser scanning (TLS) is a well established method for an efficient acquisition of 3D point clouds of arbitrary objects.In the industry high accuracy requirements with a standard deviation of 1mm for the 3D point cloud are requested.In such cases, the 3D point clouds are typically captured by means of static-TLS (s-TLS).For large scale objects, e.g., planes or ships, several 3D point clouds, taken from different stand points, are necessary.Afterwards all captured 3D point clouds must be referenced to a combined point cloud of the whole object with one unique coordinate system.That leads to a very high amount of data and furthermore to many redundant detected areas.In addition, the data processing can be very time-consuming.Alternatively, the 3D acquisition can be carried out with kinematic TLS (k-TLS).In order to fulfil the mentioned high accuracy requirements, the 3D point cloud acquisition, the system calibration or the synchronisation, and the (geo-)referencing must be carried out with high accuracy.Modern terrestrial laser scanners are able to measure a 3D point cloud with an accuracy of less than 1mm (Zoller + Fröhlich GmbH, 2018).Furthermore, appropriate methods for the system calibration and synchronisation are available, see Section 2. Regarding the total accuracy of a k-TLS based multisensor-system (MSS) the (geo-)referencing can be seen as a critical issue.The tracking of the mobile platform can be carried out by internal and external sensors or with a mixture of both methods.For the (geo-)referencing with internal sensors, e.g., inertial measurement units, line scanners, odometer or camera systems are used.All acquired data are usually fused and processed in a Kalman filter, which estimates the pose (position and orientation) of the mobile platform.A well-known method in the robotics community is the the simultaneous localisation and mapping algorithm (SLAM).Examples for such indoor systems are presented in (Trimble Navigation Limited, 2017), (ViAmetris, 2018) and (Nüchter et al., 2015).A different way to (geo-)reference the mobile platform is the use of an external sensor, e.g., a total station.However, the accuracy and the tracking frequency of available total stations are limited (Lienhart et al., 2016).Examples of recently developed k-TLS based MSS with external (geo-)referencing by a total station are given in (Leica Geosystems AG, 2016) and (Keller, 2016).It can be noted, that both mentioned (geo-)referencing methods are not sufficient enough to reach an total accuracy of ±1mm for the captured point cloud.In case of the external (geo-)referencing an improvement in terms of higher accuracy and faster tracking of the mobile platform can be achieved by using a laser tracker and thereby a more highly accurate determination of the pose with a maximum frequency of 1000 Hz is possible (Hexagon Metrology, 2015).The synchronisation of the sensors is achieved by a trigger signal.At each profile a pulse is generated by the laser scanner and send to the laser tracker.Afterwards a (geo-)referencing measurement is carried out and the platform is tracked continuously.Strictly speaking only the first point of each profile is (geo-)referenced exactly.For a high accurate point wise (geo-)referencing the movement of the platform between the laser tracker measurements must be considered.This can be realized by a Kalman filter.Hereby the motion of the platform will be determined and all points can be shifted.The implementation of such a filter model is shown in Section 3. Than in Section 4 the results of a k-TLS test measurement will be compared with an high accurate s-TLS measurement.Here, the 3D distances between the k-TLS and s-TLS point clouds are estimated.Thus, it is possible to evaluate the influence and quality of the (geo-)referencing process.

A MULTI-SENSOR-SYSTEM BASED ON KINEMATIC LASER SCANNING
The principle of the k-TLS based MSS is shown in Fig. 1.Here, a terrestrial laser scanner, which operates in profile mode (2D), moves along the object on a mobile platform.A laser scanner with the highest accuracy specifications is used.The total 3D uncertainty in a maximal range of less than 10m is given with < 1mm (Zoller + Fröhlich GmbH, 2018).Further influencing factors concerning the slope distance measurement, the angle of incidence of the laser beam and the material properties of the object have to be taken into account.consists of a reflector and ten LEDs.Thus, the pose can be measured directly with its 6 degrees of freedom (DoF) measurement, consisting of three translations and rotations.The translations are calculated by the measured polar elements: distance (d), horizontal direction (φ) and vertical angle (θ).The laser tracker is equipped with an absolute interferometer (AIFM).That means, d will be measured with the accuracy of an absolute distance meter and with the speed of an interferometer.Thus, fast and accurate measurements are possible without using a bird path for the interferometer.The horizontal direction and the vertical angle are determined by the angular encoders of the laser tracker.
The accuracy, here in the form of a maximum permissible error (MPE), is given by ±15µm + 6µm/m for the 3D position (Hexagon Metrology, 2015).Additionally, the laser tracker is equipped with a camera to observe the LEDs at the T-Probe for the determination of the rotations.The accuracy for each rotation is given by an MPE of 0.01 • = 18µm/100mm (Hexagon Metrology, 2015).Finally, the laser scanner and the laser tracker must be synchronised, which can be achieved by a trigger signal generated by the laser scanner at each start of the profile and send to the laser tracker.Afterwards the (geo-)referencing measurement is carried out by the laser tracker.The 3D transformation of each measured single point Ps from the coordinate frame of the laser scanner (s−frame) into the coordinate frame of the laser tracker (l−frame), is obtained within two steps and The first step (Eq. 1) is a transformation from the s−frame to the body coordinate frame (b-frame).The origin of the b-frame is located in the coordinate system (CS) of the T-Probe.Hereby the vector t b s represents the three translations in the coordinate directions (x,y,z) and R b s the rotation matrix with the three rotation angles (yaw, pitch, roll).These 6 DoF are also known as lever arm and bore sight angles.The determination of these parameters is accomplished by means of an extrinsic calibration in advance.A suitable approach was published in (Strübing and Neumann, 2013).Here, known reference geometries (planes) are measured by the laser scanner.Afterwards the 6 DoF are estimated in an Gauss-Helmert model by the restriction, that the distance between the scanned points to the reference geometry is zero.The determined lever arm has a standard deviation of ≤ 1mm and for the boresight angles a standard deviation of a few millidegrees (Hartmann et al., 2017) and (Heinz et al., 2017) is obtained.If all sensors are stable, it can be assumed that the lever arm and the bore sight angles are constant for the whole measurement.The second transformation in Eq. 2 from the b−frame to the l−frame, is realised by the laser tracker, which is similar to the first.Hereby, the translations t(x, y, z) l b and the rotation matrix R(yaw, pitch, roll) l b are used.These translations and rotations are measured by the laser tracker.Additionally, if necessary, a third transformation from the l−frame to a superordinate reference coordinate frame (ref−frame) can be carried out.If so known reference points in the ref−frame have to be measured by the laser tracker.Afterwards all transformation parameters can be estimated by a 3D-Helmert transformation.

PRINCIPLE OF POINT WISE (GEO-)REFERENCING
As mentioned in section 2, the synchronisation of the laser scanner and the laser tracker is obtained by a trigger signal.If the platform is moving the pose of the moving platform is tracked continuously.That leads to the fact that only the first point of each profile is (geo-)referenced exactly.However, referring to the settings of the laser scanner, every profile of the laser scanner consist of a fixed amount of points.Hence, for a point wise (geo-)referencing, the movement of the platform from one pose to the other has to be considered.A simple way to do this is a linear interpolation.This was already done in test measurements of reference geometries (Stenz et al., 2017).The results show that a 3D point cloud acquisition by k-TLS in the range of 1 − 2mm is possible.These point clouds were measured with a trolley, which was moved with nearly constant speed.Within this contribution a k-TLS based MSS exemplarily based on a rope guided platform is introduced, see Fig. 2. With this platform several test measurements were made, whereby the movement of the platform was nonlinear.Hence, a linear interpolation between the pose measurements is not sufficient.Therefore, an alternative method for a point wise (geo-) referencing has to be used.An approach has to be developed where the movement of the platform can be predicted between two (geo-)referencing measurements (states).Ideally, this is carried out in real time.The prediction can be realised by means of a Kalman filter.In case of the mentioned (geo-)referencing method, the trajectory of the moving platform is defined by states.Between two consecutive states all points in one 2D profile can be (geo-)referenced with a prediction step.As a result, all points at each 2D profile are shifted, see Fig. functional relationship is given by Here + is defined as the state vector of the k-TLS based MSS and the observation vector is l (k) .The vector l (k) consist of three translations (vector t) and three rotations (vector r), determined by the laser tracker.The parameter ∆t is the time span between each point of the laser scanner profile.The index k stands for the epoch of the measurements.

Filter model -system and observation equation
We begin by assuming that the system state and the associated measurement relationship may be written in the form where f k−1 and h k are known functions, vector u (k−1) consist of the known input parameters (deterministic changes of the system, e.g., control or sensor platfrom motion), w (k−1) and v (k)  are white noise of system and measurement process.The physical knowledge about the system state, given by f k−1 (.), and captured measurements of the the state vector, given by h k (.), can be combined in an optimal way through a filtering approach.A filter in state space has usually three steps: initialisation, prediction and filtering step.The Kalman filter is a recursive optimal procedure to combine the predicted system state and the measurement vector in each epoch if the system and the measurement equations are linear.In the prediction step the new state is predicted based on the previous filtered state.In the filtering step the predicted state vector is updated by means of the captured measurements, see, e.g., (Kalman, 1960).A well known analytical approximation to handle a nonlinear system is to linearize the measurement and the system equations using Taylor series expansions (known as extended Kalman filter (EKF)).To reduce the linearization error in the measurement equations of the EKF an iterated step is needed.The filter is known as iterative EKF (iEKF), see (Simon, 2006).In order to simplify the physical model of the filter the movement of the platform should be constant and steady.That ensures on the one hand a unique resolution of the point cloud and one the other hand better results in the (geo-)referencing.However, this might be difficult due to different local conditions, e.g., large objects with difficult access.Thus, different platforms and stabilising sensors had to be used.To handle this situations, the laser scanner and the T-Probe are mounded on a carrier, which can be moved along a fixed rope, see Fig. 2. For the first testing the suspension of this carrier was kept deliberately simple.That means different forms of movements, for example vibrations, can occur.Furthermore, a constant velocity was not realised because of a missing motor control.That means higher dynamics of the k-TLS based MSS can occur.To consider all these facts in the processing an iEKF is chosen here.The state vector x (k) at epoch k is defined as whereby the translation, velocity and acceleration vectors (t, v and a), as well as the rotation angel, angular speed and the angular acceleration vectors ( r, ω and α) are integrated additionally.
As mentioned in Eq. 3, the translations t and rotations r are observed directly by the laser tracker.To consider the movement of the platform between these states the prediction step is carried out by where the transition matrix Φ (k−1) , which defines the transition between a state and the next state, the noise input matrix G (k−1) comes from the linearized system equations, Q xx, − is the variance covariance matrix (VCM) of the predicted system and Q ww is the VCM of the process noise.It has to be noted that the vector u (k−1) was not considered in the system state, see Eq. 10.After the prediction step the filter step (measurement update) is executed: End For In Eq. 11 A (k) i is the design matrix and is set up by linearizing the measurement equations h (k) (refer to Eq. 5) at the Taylor point x (k) + , if the measurement equations are nonlinear.The Kalman Gain matrix K (k) i determines the power of the correction of the prior estimate of the state vector (given in Eq. 10) by adding the appropriately weighted measurement residual indicated by Q ll .In the first iteration, predicted state corresponds to the filtered state (x If the termination condition (e.g. the relative update of the filtered state vector smaller or equal to 10 −8 ) is fulfilled, then x (k) + = x (k) i+1, + and the corresponding variance covariace matrix Q xx,i+1,+ of the filtered system state are obtained.

Filter model -implementation
For the system model the physical characteristics of the used k-TLS based MSS have to be considered.Because of the mentioned higher dynamics the angular acceleration were introduced in the state vector, see Eq. 6.This ensures that the rotations are not limited by constant rotation speeds.Thus, the transition matrix Φ (k−1) and the physical model according to (Pentenrieder, 2005) have to be extend.The system model is given by As already mentioned, three translations and three rotations are observed and defined as observations, referring to Eq. 3.That means all these components of the state vector are determined directly.The observation model is defined by Whereby v are the residuals between the predicted state and the measured observations of the new epoch.To initialise the filter start values for the state vector x(0) + as well as the corresponding VCM Q (0) xx,+ have to be set up.The state vector is defined in Eq. 6.For the first epoch (0) the translations t (0) and rotations r (0) of the first measurement were introduced.The velocity, acceleration, angular speed and the angular acceleration of the first epoch are unknown.Thus, the vectors v (0) , a (0) , ω (0) and α (0) are filled with zero values.This leads to the following state vector for the first epoch The VCM Q xx,+ is set up approximately, whereby the lowest variance was assumed for the direct observations.In addition, all covariances were assumed to be zero.Because of the fact that the starting values are improved in the filtering steps, they do not have to be chosen exactly.The VCM Q ww of the process noise of the system is unknown and can usually only be modelled by extensive investigations.It has a significant influence on the prediction step.Thus, it is necessary to carefully set up the VCM.One method is to consider the highest order of parameters (accelerations) in the system states as constant over time ∆t.This leads to uncorrelated jumps in the respective parmeters between the epochs.The parameters with low-order are given with a white noise depending on the highest-order parameter.The process noise is given by whereby ι transl and ιrot are weighting factors.In (Bar-Shalom et al., 2001) the Wiener-sequence acceleration model is mentioned.
Here, it is assumed that the acceleration increment is an independent (white noise) process.That leads to the following values for The VCM of the measurements is given by Here, all standard deviations were derived from the MPE given in the user manual (Hexagon Metrology, 2015).This was done according to the Guide to the expression of uncertainty in measurement (GUM) (JCGM, 2008).The standard deviations of the translations σ 2 t X , σ 2 t Y , σ 2 t Z are calculated by variance propagation with the standard deviations σ 2 phi , σ 2 θ , σ 2 d of the originally measured polar elements φ, θ and d.

DATA PROCESSING AND VALIDATION
In the context of test measurements a test panel was used, see Fig. 4. The test panel was captured by k-TLS and s-TLS, respectively.In order to compare the data sets, reference points were fixed at the edge of the test geometry.The coordinates were determined by a laser tracker measurement to a corner cube reflector, with an 3D uncertainty (MPE) of ±15µm + 6µm/m (Hexagon Metrology, 2015).Thus, these reference points were used to (geo-)reference the static and kinematic data set in a unique CS.The origin of these CS is shown in Fig. 4. The position of the test panel over the period of the measurement can be considered as stable.For the k-TLS based measurements a rope in a distance of 3−4 m at the height of the centre of the test panel was fixed.The suspension of the rope on one side was slightly higher than on the other.Due to the weight a downward movement of the platform is realised, which leads to a nonlinear movement.The main direction of the movement of the kinematic platform was along the X-axis of the local CS.The rotation speed in the 2D mode of the laser scanner was 50Hz.That means the time difference between to consecutive triggered (geo-)referencing measurements (states) is 20 ms.The number of points at each profile was set to 10000.This results in a constant time difference of 2 µs between two scanned points.Approximately 1000 profiles were recorded during the entire measurement process.All captured points were (geo-)referenced according to the described method in Section 3. In Fig. 5 the results of all parameters of the state vector, which was defined in Eq. 6, over the iterations (ticks) of the iEKF are shown.It can be seen in the data that the movement of the platfrom started at profile 480.Because of the inclination of the rope an acceleration occurs.This can primarily be seen in the X-and Y-direction.At the end of the measurement the platform was slowed down.After the start event a higher scattering in the accelerations of the translations a and rotations α can be seen.Because of missing damping elements a direct transmission of vibrations to the laser scanner occurs.The influence of the accelerations in the rotations have to be critically considered.They have to be taken into account here due to the fact that the influence depends on the measuring distance.To evaluate the quality of the iEKF solution, the residuals v, see Eq. 13, are calculated.It can be seen as the difference between the predicted state, which is transmitted to the observation space, and the observation data.The residuals are minimal if the predicted state and the measurement are fitting well together.Because of the mentioned high accuracy of the laser tracker, the measurement can be seen as very reliable and accurate.The residuals shown in Fig. 6 also show a start event at tick 480.In general it can be noted, that there is a higher scattering after the start event.The residuals in the translations are under ±0.5mm.The most significant differences can be seen in the y-direction.In case of the rotations, the most significant differences (±0.005rad) can be seen in the yaw-rotation.The reason for this larger deviations is probably a pendulum motion around the z-axis, which will be not adequately taken into account in the predicted state.Furthermore, the 3D point cloud (geo-)referenced with the iEKF is compared with a s-TLS 3D point cloud of the test panel.Here, the M3C2algorithm is used to calculate the distances between two point clouds with different densities, detailed information's are given by (Lague et al., 2013) and (Barnhart and Crosby, 2013).The computed distances, can be seen as an overall deviation of the k-TLS 3d point cloud with respect to the s-TLS 3D point cloud.This includes the uncertainties of the (geo-)referencing, system calibration, synchronisation and of the 3D point cloud capturing.However, in case of the presented k-TLS based MSS, the influence of the (geo-)referencing is the most significant.In order to demonstrate this, the processing was carried out in addition to the iEKF without a system model and by linear interpolation.Without a motion model means here that there is no movement considered between two consecutive (geo-)referencing measurements (states).The results of the calculated M3C2 3D distances as well as their distributions are plotted in the Fig. 7, 8 and 9.It can be seen that the distances without a system model are significantly higher than with the linear interpolation.For the case with no constant movement, the mean value is 3.07mm and the standard deviation is 2.27mm.These value are significantly reduced with the linear interpolation.Here, the mean value is 0.29mm and the standard deviation is 1.53mm.With the iEKF the mean value is 0.14mm and the standard deviation is 1.00mm.That means the filter leads an improvement of 2.93mm in the mean and 1.27mm in the standard deviation, in comparison to the (geo-)referencing without a system model.

CONCLUSIONS
In recent years, s-TLS has become a well established measurement method for the 3D point cloud capturing of large objects.
In terms of efficiency k-TLS can be seen as a promising alternative.However, current k-TLS based MSS are not able to fulfil the high accuracy requirements of ±1mm.Thus, a new k-TLS based MSS and suitable processing algorithms have been developed.In this contribution the focus lies on the (geo-)referencing.Hereby a mobile platform, which is equipped with a terrestrial laser scanner operating in profile mode, is tracked by a laser tracker.The synchronisation is realised by a trigger pulse which is send to the laser tracker at each starting point of a profile.That leads to the fact, that only the first point of each 2D profile is (geo-)referenced exactly.To (geo-)reference all points the motion of the platform must be considered.This is done with a Kalman filter.In the prediction step, each point is shifted according to the determined velocity and acceleration of the platform.Within test measurements a panel was measured with the k-TLS based MSS.Here, a rope guided platform was used.Due to the high non-linearity of this platform, the processing was made with an iEKF.For comparative purposes a high accurate 3D point cloud was captured by s-TLS.To analyse the quality of the (geo-)referencing, the 3D distances in the M3C2 between the k-TLS and s-TLS 3D point clouds were estimated.The computed distances include the whole uncertainties of the 3D point cloud capturing, system    show that a (geo-)referencing without a motion model is not sufficient enough to fulfil the mentioned accuracy requirements of ±1mm.A significant improvement can be achieved by a linear interpolation.However, this will be not sufficient enough in case of a non-linear moving platform.Here, the processing with an iEKF showes the best results.A mean value of 0.14mm and the standard deviation of 1.00mm for the computed 3D distances in the M3C2 were reached.It can be summarised, that the required accuracy can be achieved by a suitable (geo-)referencing methodology.
Figure 1.K-TLS based MSS consisting of a laser tracker (Leica AT960) and T-Probe for (geo-)referencing and laser scanner mounted on the mobile platform.

Figure 2 .
Figure 2. The k-TLS based MSS mounted on the wire rope guided construction.
Figure 3. Principe of the point wise (geo-)referencing with the k-TLS based MSS.

Figure 4 .
Figure 4. Test panel with reference points and origin of the local coordinate system.

Figure 5 .
Figure 5. Results of the state vector from the iEKF.

Figure 8 .
Figure 8. Computed 3D distance (M3C2) between s-TLS and k-TLS with linear interpolation.calibration,synchronisation and (geo-)referencing.To quantify the influence of the (geo-)referencing the processing was made without a motion model and by linear interpolation.The results show that a (geo-)referencing without a motion model is not sufficient enough to fulfil the mentioned accuracy requirements of ±1mm.A significant improvement can be achieved by a linear interpolation.However, this will be not sufficient enough in case