ON DATA ACQUISITION OF MOVING OBJECTS VIA KINEMATIC TERRESTRIAL LASER SCANNING

Terrestrial laser scanning (TLS) can be seen as an established method for geodetic data acquisition. Advantageous is its high achievable accuracy as well as its rapid, active and laminar sampling of the object space. However, problems occur if an object, the scanner itself or both are moving which is referred to as kinematic TLS (k-TLS). The cause of this issue is the sequential scanning principle of TLS which causes temporal offsets between measured points and hence describes a distinctive feature to photometric approaches. Due to this temporal shift movements of the scanner or an object lead to geometric falsification during data acquisition. If, however, one can determine current orientation and position of an object or the scanner at any point of time by applying additional sensor technology, geometrically correct and kinematic data acquisition can be derived. The contribution at hand presents a multi sensor system which applies a terrestrial laser scanner for acquisition of an object’s surface. In addition three tracking total stations are applied that monitor the object’s orientation and position within the coordinate system of the TLS. Influencing factors onto the geometric correction are the accuracy of the applied total stations for determination of six degrees of freedom (6dof) as well as the temporal synchronisation. At first an introduction into the problem domain is exemplified on a simple case. The system is then tested on a 2.5 m long ship model inside a research facility as well as outdoors on a 12 m long vessel. For the sake of assessment the results have been compared to statically acquired scans of the test objects, outside of the water respectively in position of rest, in order to derive reference models.


INTRODUCTION
The acquisition of kinematic processes is a field of application that has been dominated by photometric solutions over many decades due to its capability to quasi-simultaneously acquire information.A downside of these techniques is their dependence to sufficient contrast within the imagery which is affected by lighting conditions, contrast on the object's surface or area of interest itself and various other aspects.Within a previous study (WUJANZ et al. 2013) the outer shell of a ship had to be surveyed while being on a lake which was needed for the determination of a ship coordinate system in context of a boresight calibration.This task has been accomplished by capturing uncalibrated imagery and by generating a point cloud based on multi-view stereo (MVS) software such as BUNDLER (SNAVELY et al. 2006).The outcome was unsatisfactory due to various reasons:


The object contrast on the outer shell was mostly insufficient which lead to a sparse and heterogeneously sampled point cloud.


Reflections of the shell on the surface of the water and repetitive pattern on the ship led to false correspondences and hence to wrong coordinates.


Since the imagery had been captured from a dock, poor lighting conditions had to be accepted.


No quality measures or stochastic information was made available by the software, so that the user is forced to visually evaluate the outcome, an approach that is highly subjective and doesn't satisfy established viewpoints in surveying.

Motivation
Hence, the motivation arose to develop an approach that is capable of solving the stated problem, is insensitive to ambient lighting conditions as well as local object contrast and satisfies demands of quality assurance.Furthermore the system should be applicable indoors as well as outdoors which had to be taken into consideration during conception stage.A technology that holds for the stated requirements in terms of data acquisition is terrestrial laser scanning (TLS).Disadvantageous is however the sequential sampling process of TLS which hence leads to temporal offsets between measured points and geometric distortion if the object of interest moves during data acquisition.
If the movement of the object can be described by tracking its six degrees of freedom (6dof) during scanning corrections can be applied that yield to a geometric rectification.
ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume II-5/W2, 2013 ISPRS Workshop Laser Scanning 2013, 11 -13 November 2013, Antalya, Turkey The exact same problem needs to be solved in mobile mapping, however in an inverse case where position and orientation of a multi sensor platform changes in relation to its surroundings.A look at past and current literature, which will be given in the next section, reveals the fact that the term kinematic data acquisition is nowadays mostly used within the previously mentioned context of mobile mapping.These systems are usually established as multi sensor platforms and contain three basic functionalities which carry out laminar data acquisition, positioning of the system as well as its orientation.The required frequency of these three functionalities depends on the kinematic parameters of the relative motion.
The motion pattern that are being investigated within this contribution can be categorised as rather steady in terms of their behaviour and are hence not very demanding in this regard which is why tracking total stations are applied.It has to be mentioned that the geometric properties of all surveyed objects are considered to be stable so that a rigid body transformation is applied to describe the kinematic parameters represented by three translations and three rotations in 3D space.
Section 2 introduces the k-TLS system in detail whereas different aspects of data acquisition are also discussed.The acquisition of an oscillating motion for inspection of the temporal synchronisation is presented in section 3. Section 4 includes scenarios from naval applications and thus addresses the initially stated problem.Concluding remarks and prospective problems are discussed in the final section.

Related work
A sound overview on the history, components, processing steps and other important aspects concerning mobile mapping in a general sense are given by EL-SHEIMY (2005).Nevertheless several cases can also be defined that are suitable to capture kinematic scenarios whereas the object coordinate system (OCS), sensor coordinate system (SCS) and environment coordinate system (ECS) need to be introduced in this context: ) plans to apply a combination of TLS and photogrammetry in order to determine strain on blades of actuated wind generators.Hence this project as well as the approach described within this contribution follows, according to the previously stated definitions, the third case.The authors are not aware of any contribution that covers the fourth mentioned case.

PRINCIPLE OF THE APPLIED KINEMATIC TLS
The k-TLS approach that is presented in this contribution applies one or three total stations (2 x Leica TS 30 and 1 x Leica TCRP 1201) for the determination of kinematic parameters while a TLS (Z+F Imager 5006h) is used for data acquisition of the object's surface.Software has been implemented based on Z+F's SDK for scanner control and Leica's GEOCOM interface to communicate with all total stations.A schematic description of all necessary steps for k-TLS is summarised in the following processing chain while all procedures are discussed later in detail within this section: 1. Transformation of all sensor coordinate systems into a common coordinate system.2. Synchronisation / temporal calibration of the system.3. Data acquisition and determination of the object's kinematic parameters.4. Geometric rectification of the point cloud.
The first issue that has to be solved is the transformation into a common coordinate system, which is the only step of the process chain that is independent to the quality of the synchronisation.For this purpose artificial tie points need to be placed around the area in which the object is going to move.These targets then need to be surveyed by all applied sensors in order to derive transformation parameters into the desired coordinate system which is usually the TLS's inherent system.
In order to solve the second step of the process chain, which achieves temporal synchronisation, the system time of the applied computer serves as reference time frame.Hence, the system is applicable indoors as well as outdoors.The synchronisation of all total stations is conducted by comparing the according sensor time and system time before and after kinematic data acquisition.Due to the unknown latency during data transmission between total station and computer this temporal replication cannot be determined more accurately than 10 to 20 ms.The synchronisation of TLS data is achieved by adding timestamps of the applied computer that are set after arrival of single data packages.These packages include points that have been captured during one rotation of the deflection mirror around the tilting axis of the TLS.Since the time of circulation as well as the amount of points per rotation is known, a timestamp can be assigned to each measured point.Since this procedure involves several insecurities, an approach for temporal calibration is proposed in the following.Zoller + Fröhlich provides two methods of scanning the object space including a repetitive measurement of a profile (2D mode), which describes a rotation around the tilting axis.The second sampling approach, which is the standard method of all TLS, measures a profile that is spinning against the rotation axis (3D mode).Determination of kinematic parameters is achieved by using the tracking functionality of three applied Leica total stations whereas 360° prisms have been used.It needs to be mentioned that various combinations of sensors for 6dof determination are available.Thus total stations will be replaced in future campaigns in order to avoid instrumentation of moving objects.The last procedure, which carries out geometric rectification, is heavily dependent on the quality of synchronisation since it applies the timestamps that have been added in step two.A detailed description of this step will be given within the next section.

ANALYSIS AND CALIBRATION OF THE SYNCHRONISATION
At first a simple kinematic case is presented which includes an object that oscillates around an axis.For the implementation of this assembly, a planar board has been attached to a metal rod which represents the rotation axis.This has been connected to a metal frame by adding sockets.In order to activate an oscillating motion a pendulum rod has been attached to the board which is powered by a controllable electric motor.A prism has been connected underneath the board that enables simultaneous observation of the oscillating motion by a tracking total station and the TLS.A schematic illustration is depicted in Figure 1.As illustrated in the left part of Figure 2, the oscillating motion of the board causes an apparent sinusoidal distortion.Interestingly, variations in intensity are visible on the surface of the board that are provoked by changes of incidence angles during motion.Since the oscillation has been simultaneously acquired by both instruments, geometric correction can be conducted whose result is depicted in the right part of the illustration.Therefore an object coordinate system has to be defined, which is highlighted by coloured lines in the left part of Figure 2, whose origin lies at the outer, left end of the rotation axis.The y-axis (yellow line) runs through the rotation axis while the x-axis (red line) is computed from the cross product between location vector to the prism and an arbitrary location vector on the y-axis.The z-axis (blue line) stands perpendicular on x-and y-axis.
Due to the movement of the object of interest a change of the OCS's orientation will occur within the SCS, which is a rotation around the y-axis of the OCS in the case at hand.Now the possibility arises to determine the current orientation of the board for every point that has been captured by the total station.As the temporal resolution of TLS is considerably higher than the one of the tacheometer and moreover that the measurements haven't been acquired simultaneously, linear interpolation of the OCS's orientation for all TLS points has been conducted.Therefore two temporally adjacent points surveyed by the total station are determined for which the orientation of the board can be computed for both points in time.In order to determine the orientation of the board for every TLS-point linear interpolation is carried out based on the time stamps of the measurements.Geometric correction is achieved by transforming the coordinate system of the TLS into the OCS.Since this procedure is utilised on all acquired TLS points static areas within the point cloud appear distorted, which can notably be seen in the background of the right part of Figure 2. The motivation for the realisation of the proposed test rig was to examine the previously determined software-based synchronisation.Therefore points are selected from the distorted point cloud that are located between two cylinders.Figure 3 depicts the situation where the cylinders, that are positioned concentrically around the rotation axis (green cylinder), are represented by half transparent cylinders in red and blue.The distorted surface of the board is coloured in light grey.Between the two cylinder hulls the pink area is located from which points are selected for further processing.The selected points are then added to a two-dimensional coordinate system in dependence to their according system time (horizontal axis) whereas the distortion of the board in relation to the zero position (prism is located perpendicular under the yaxis) is mapped onto the vertical axis in degrees.Thus an approximate sine function emerges to which subsequently the total station measurements are added in the same fashion.These points have then been processed within a least squares adjustment where the functional model describes a harmonic oscillation.Rotation angles are introduced as observations while  A close look at the figure above reveals that the turning points of the waves are slightly shifted which leads to the conclusion that both system times are deviating after the applied softwarebased synchronisation.Furthermore it can be seen that the graphs are vertically misplaced which is caused by the fact that the centre of the prism does not lie within the plane of the wooden board.The existing phase shift can be converted into a temporal shift and applied to the software-based synchronisation so that a geometrically enhanced point cloud should arise.For the acquired dataset a time shift of 53 ms has been computed which has been used to calibrate all time stamps.In order to analyse the quality of the geometric correction as well as the impact of the synchronisation onto this procedure planes have been adjusted through various datasets.This has been done based on a static scan which serves as a reference and kinematically acquired data.It can clearly be seen that the oscillation of the board consequentially causes large geometric falsifications which is represented by maximum absolute residuals to the adjusted plane especially for the uncorrected dataset.After geometric correction under use of the software-based synchronisation the quality evidently increases whereas the upper and lower average deviations are still about twice as large as the ones of the statically measured reference.This result leads to the conclusion that the time frames of both instruments are not sufficiently synchronous.After temporal synchronisation and calibration of all timestamps the original geometry can almost be recovered.Consequently the achieved results can be positively assessed.

UTILISATION OF K-TLS IN NAVAL APPLICATIONS
After introduction of k-TLS on a simple example this section covers two scenarios of object's in which their spatial position and orientation need to be determined during motion.Therefore two additional tacheometers are applied that are necessary to determine six degrees of freedom of a moving ship model as well as a boat.

Acquisition of a ship model
A 2.5 m long ship model served as object of interest within the first experiment.Two datasets have been captured, one statically measured point cloud outside of an experimental basin which serves as a reference as well as a kinematically scanned dataset that has been captured while the model was on the water.For the transformation of all coordinate systems into a common one five artificial targets have been placed around the basin in which the model was moving.Subsequently the centres of all targets have been surveyed by all four sensors whereas the misclosures amount to 0.66 mm after transformation into a common coordinate system.
In order to determine the object's coordinate system which is essential for the geometric correction and computation of kinematic parameters three prisms have been mounted on top of the model.Since the water line of the applied model was only 10 cm underneath the highest point, three model containers between 11 and 16 cm have been attached on deck.Figure 5 features the applied model, the targets (circles) for the transformation into a common coordinate system as well as the prisms (dashed rectangles) for the determination of kinematic parameters.
Figure 5: Model ship with mounted prisms (circles) and targets (dashed rectangles) for the sake of transformation into a common coordinate system The origin of the coordinate system is located in the centre of the left prism; the y-axis is described by the vector between origin and outer right prism, while the z-axis stands perpendicular on the plane which is assembled by the three prism centres.Data acquisition has been carried out by using the

Acquisition of a ship
The acquisition of a 12 m long ship called "Biene" took place in a quiet side canal of the lake of Tegel in Berlin.As the canal was frozen, no crossing could have been captured as planned.
Instead the ship unmoored within a small ice free area in which it has been moved.In order to cause notable effects on the calm water a staff member translated the ship by hand and activated yaw.Before the measurements have been carried out three prisms have been attached to the boat by applying switching magnets and tension belts.Figure 8 shows the ship, which has been equipped with prisms, in the left part as well as the applied sensors (right part) with the vessel in the background.In order to carry out the transformation into a common coordinate system five artificial targets have been distributed on both banks.Three targets have been placed on tripods at the bank on which the ship moored while two additional ones were attached onto a crane located on the water's edge close to the sensors by using switching magnets.On average the misclosures of the transformation amount to 1.78 mm.Synchronisation has been conducted software-based without applying the proposed temporal calibration method since it is not sufficiently clarified how stable the sensor's time frames are in relation to each other.Due to the calm water which featured no waves at all a reference dataset has been captured via static TLS while the ship moored.The static survey has been undertaken within the 3D mode of the TLS and took about 20 seconds while no movements have been noticed during the scan.Subsequently kinematic data acquisition has been conducted, again by applying the 3D mode, which took 30 seconds. Figure 9 shows the meshed but yet uncorrected point cloud from kinematic data acquisition.In the lower part of the image effects caused by reflective and refractive interactions with the water surface are notably visible that have been excluded from subsequent processing.Kinematically acquired data is depicted in black whereas the reference dataset is coloured in green.Geometrically corrected data is illustrated at the bottom left while the uncorrected dataset can be found top right above the red dashed lines.A wavelike characteristic is noticeable within the uncorrected data that has been caused by the yaw of the ship during data acquisition.The corrected data shows unsteady areas especially in the front part of the intersection that are probably caused by outliers of tacheometrically captured points.A laminar comparison is depicted in Figure 11 where the reference is compared to uncorrected data (left part) and corrected data (right part).Before processing the absolute mean difference between reference and captured point cloud amounts to 13.5 cm while this measure reduced down to 2.6 cm after processing so that the procedure can be evaluated quite positively.

CONCLUSION AND OUTLOOK
The contribution at hand presents a novel approach for data acquisition of kinematic processes in which a TLS is applied in order to capture the surface of an object of interest while its kinematic parameters have been acquired by applying tracking total stations.Apart from a software-based synchronisation a geometrically motivated temporal calibration has been proposed which leads to more accurate results.The approach has been applied on two scenarios where ships had to be captured in motion while the results can be rated quite positively.The weak point of the system is the use of total stations since the object of interest needs to be equipped with prisms as well as the fact that their temporal resolution is comparably low which describes a limitation to problems with steady kinematic characteristics.Hence, additional studies will focus on determination of kinematic parameters based on redundant photogrammetric methods which also addresses the issue of reliability as the proposed solution based on total stations describes a minimal configuration.Another drawback which needs to be tackled is the use of artificial targets for relative orientation of all sensors which limits their use to simply accessible scenarios.All results have been derived from unprocessed tracks as measured by total stations so that more accurate results can be expected by further processing, for instance Kalman-filtering.Furthermore the assumption has been made that the objects of interest behaved as rigid bodies due to their comparably small size and low forces that were present.Since this is certainly not the case for container ships other ways of describing the essential kinematic parameters need to be explored.Thus, new challenging questions arise within this context for instance how has a freighter deformed after loading or how do ships behave while they are crossing rough waters?

Figure 1 :
Figure 1: Side view (left part) and front view (right) of the apparatus

Figure 2 :
Figure 2: Object coordinate system and intensity coded point cloud before (left) and after geometric correction (right)

Figure 3 :
Figure 3: Rotation axis (green cylinder), distorted board (light grey surface) and cylinders (red and blue transparent) for the selection of points (pink surface) the Photogrammetry, Remote Sensing and Spatial Information Sciences, VolumeII-5/W2, 2013  ISPRS Workshop Laser Scanning 2013, 11 -13  November 2013, Antalya, Turkey the time is used as fixed values.As a result from the adjustment form parameters of the harmonic oscillation are known for both datasets whereas the phase shift between the curves should be as small as possible since this measure describes the quality of the determined synchronisation.Figure4illustrates the measured and adjusted results where green points represent data which has been acquired from TLS and red crosses the prism track captured by the total station.The outcome of the adjustment based on TLS measurements is depicted by the dashed line while the continuous line represents the results derived from tacheometric measurements.

Figure 4 :
Figure 4: Captured oscillation from TLS of a sectional plane (green points) and trace of the prism as captured by the total station (red crosses).Adjusted results are represented by a dashed line (TLS) and a continuous line (total station) Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume II-5/W2, 2013 ISPRS Workshop Laser Scanning 2013, 11 -13 November 2013, Antalya, Turkey 3D mode of the TLS while the model underwent roll and yaw movements.Figure 6 depicts kinematically captured point clouds before (upper part) and after geometric correction (lower part).The dashed lines help to indicate effects that are caused by the movement of the model during data acquisition.Since a reference dataset is available represented by the statically acquired point cloud the corrected dataset has been transformed into its coordinate system by applying the iterative closest point algorithm (ICP), as introduced by BESL & MCKAY (1992), in order to being able to geometrically compare the data.

Figure 6 :
Figure 6: Point cloud of the kinematically acquired boat before (upper part) and after geometric correction (lower part)It has to be mentioned that the computed transformation parameters have influential impact onto the derived inspection map as they are sensitive to outliers (WUJANZ 2012).The quality of the correction can be rated as quite satisfactory which can be concluded based on an average deviation of 2.6 mm.The inspection map is illustrated in Figure7which shows noteworthy vertical patterns which are assumed to be caused by outliers within the kinematic parameters.The grey areas have been cut out since they were affected by waves.Due to leakage the left model container had to be removed in between scans and has hence not been considered in the figure below.

Figure 7 :
Figure 7: Colour coded inspection map between reference and geometrically corrected point cloud

Figure 8 :
Figure 8: The equipped ship (left) as well as the applied sensors (right) on the opposite bank of the ship

Figure 9 :
Figure 9: Uncorrected meshed point cloud of "Biene"Black rectangles highlight longish artefacts of the mesh that represent hoses in the figure above.These are used to inject air into the water in order to keep the immediate area around the ship free of ice.The dashed grey line represents an intersecting plane, whose outcome is depicted in Figure10, in order to vividly illustrate the "deformation" which has been caused by the movement during the scan.

Figure 10 :
Figure 10: Uncorrected dataset vs. reference (top right) and corrected dataset vs. reference (bottom left) Data acquisition can be carried out in various forms since laser scanners are capable of sampling the object space in different ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume II-5/W2, 2013 ISPRS Workshop Laser Scanning 2013, 11 -13 November 2013, Antalya, Turkey ways, as VENNEGEERTS (2011, p.19) explains.The SDK of

Table 1 :
Table 1 summarises the achieved results.Deviations of acquired point clouds in relation to an adjusted plane