CHANGE DETECTION IN PHOTOGRAMMETRIC POINT CLOUDS FOR MONITORING OF ALPINE, GRAVITATIONAL MASS MOVEMENTS

This contribution discusses the accuracy and the applicability of Photogrammetric point clouds based on dense image matching for the monitoring of gravitational mass movements caused by crevices. Four terrestrial image sequences for three different time epochs have been recorded and oriented using ground control point in a local reference frame. For the first epoch, two sequences are recorded, one in the morning and one in the afternoon to evaluate the noise level within the point clouds for a static geometry and changing light conditions. The second epoch is recorded a few months after the first epoch where also no significant change has occurred in between. The third epoch is recorded after one year with changes detected. As all point clouds are given in the same local coordinate frame and thus are co-registered via the ground control points, change detection is based on calculating the Multiscale-Model-to-Model-Cloud distances (M3C2) of the point clouds. Results show no movements for the first year, but identify significant movements comparing the third epoch taken in the second year. Besides the noise level estimation, the quality checks include the accuracy of the camera orientations based on ground control points, the covariances of the bundle adjustment, and a comparison the Geodetic measurements of additional control points and a laser scanning point cloud of a part of the crevice. Additionally, geological measurements of the movements have been performed using extensometers.


INTRODUCTION
One of the direct consequences of climate change is the melting of permafrost in alpine areas. In combination with extreme weather especially heavy rain falls gravitational mass movements in the mountains like rockfalls and hang slides are going to be a more frequently and more dangerous hazard (Clague and Stead, 2012). An important task in predicting these mass movements is the classification of the movement type based on a set of indicators. Hungr et al. (2014) classifies five different movement types: fall, topple, slide, spread, and flow. This classification needs a detailed observation of movements. Here, observing the movement rate and direction of single points over time is not sufficient to model complex changes. This modeling requires the observation of a various number of 3D points for longer time periods and leads to a classification of the movement type and an estimation of possible moving masses. Different measurement techniques are used for this purpose. Geological systems like extensometers (Malet et al., 2002;Massey et al., 2013) are used to measure changes in the width of crevices. Geodetic measurements are mainly based on tachymeter, GNSS receivers (Squarzoni et al., 2005), radar interferometry or laser scanning (Roberti et al., 2018;Delacourt et al., 2007;Kasperski et al., 2010;Oppikofer et al., 2009). All these devices have in common their high costs compared to cameras and the need of professional personnel. Surveillance of larger areas or all potential risk areas in the Alps is more or less impossible in that way. Using cameras allows handing over the job of data acquisition to non-professionals (Urban et al., 2019;Romeo et al., 2019;Mayr et al., 2019;Peppa et al., 2019;Kromer et al., 2019;Esposito et al., 2017;Hendrickx et al., 2019;Warrick * Corresponding author et al., 2016;Piermattei et al., 2016;Eltner et al., 2016;Stumpf et al., 2015;Kääb et al., 2014;Westoby et al., 2012).
This contribution focuses on the assessment of possible accuracies in 3D reconstruction and change detection based on photogrammetric image sets. Photogrammetric results are compared to Geodetic measurements using tachymeter and laser scanner.

MEASUREMENT AND PROCESSING STRATEGY
The evaluation of the accuracy and reliability in 3D reconstruction and change detection and deformation analysis is split in two parts. In the first part, the accuracy of the 3D point generation is evaluated. The second part focuses on the change detection between two point clouds.
A set of ground control points defines a local coordinate system. Camera orientations are unknown beforehand and the image set is oriented based on the ground control points. Two different sets of ground control points are used. Set one is fixed longterm ground control points defining the local coordinate system for registering the different epochs. Set two is ground control points that were individually measured for every epoch. The interior orientation of the camera is given and assumed to be fix for one epoch.

3D reconstruction
The 3D reconstruction is based on the well-known two step process of bundle adjustment of the image based on tie point and ground control points (Förstner and Wrobel, 2016) followed by a semi-global matching for dense point cloud generation (Hirschmueller, 2008). Measurements of crevices are taken with an image overlap of at least 80%. The camera is moved ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume V-2-2020, 2020 XXIV ISPRS Congress (2020 edition)  long the crevice in parallel view. The recording positions have been simulated beforehand with the calibrated camera. For large crevices with several meters width, two different camera orientation schemes are used: • Double-edge configuration ( Fig. 1): The images are ordered parallel to the crevice so that both edges of the crevice are visible in every image. The position of the camera moves parallel to the pathway of the crevice.
• Crevice configuration (Fig. 2): The camera is tilted over the edge of the crevice to record the scene in the interior of the crevice. Images now show only few or even no ground control points and are connected to the first image set via tie points. Here, the camera is also moving parallel to the pathway of the crevice.
In such cases with large crevices, it is likely that the moving side of the crevice is already unstable and cannot be accessed directly. For small crevices, both sides of the crevice are often accessible, but the crevice is to small to put the camera inside.
The bundle adjustment of the camera orientations uses image points and ground control points as observations, the interior orientation is assumed to be fixed. The exterior orientation parameters are the unknowns. As the collinearity equations (Förstner and Wrobel, 2016) lead to a non-linear equation system in the bundle adjustment, an iterative adjustment has to be done and initial values for the unknown parameters of the exterior orientation are estimated from the ground control points and their image points.
Three strategies are used to estimate the initial exterior orientation. If at least six ground control points are visible, the direct linear transform (DLT) is used (Luhmann, 2018). For three to five ground control points, the minimal-object-information strategy (MOI) (Luhmann, 2018) is used. For images with less than three ground control points, the two neighboring images with estimated exterior orientation are used to interpolate their exterior orientation parameters linear from these two neighbors with the estimated image overlaps as weights. For five and four ground control points in the MOI strategy, the optimal set of three ground control points is found by searching the combination that spans the largest triangle in the image.
The bundle adjustment uses variance components to model the different accuracy levels of measured image points and measured ground control points. Random Sample Consensus (RANSAC) (Fischler and Bolles, 1981) removes outliers in the tie point search before the bundle adjustment is performed. The resulting exterior orientations are the input for the semi-global matching (Hirschmueller, 2008).

Change detection and deformation analysis
Change detection consists of three major steps: i) coregistration; ii) noise detection; iii) detection of significant changes. As all image sets are oriented based on the same set of ground control points, one can assume the resulting point clouds to being already registered. An additional coregistration by distance minimization like Iterative Closest Point (Besl and McKay, 1992) would falsify the change detection as it minimizes the distances of all points and thus would also include moved points in the registration.
The bundle adjustment calculates the inner quality of the resulting adjusted equation system. 3D points are only generated for the tie points and the ground control points. An absolute accuracy of the dense 3D point cloud is not calculated neither in the bundle adjustment nor in the semi-global matching. Estimating the noise level is very important to distinguish noise and real changes.
The proposed strategy uses the Multiscale-Model-to-Model-Cloud method (M3C2) (Lague et al., 2013) to find for every point in one point cloud the corresponding neighbor in the other point cloud and the distance between this point pair. M3C2 is used both for noise level estimation and change detection.
Given the exterior orientation of the images from the ground control points, the registration accuracy is limited to the measurement accuracy of the ground control points. To refine the registration we assume the area with the ground control points defining the local coordinate system to be stable without changes. If so, a coregistration using Iterative Closest Point (ICP) (Fischler and Bolles, 1981) leads to a refined registration of the point clouds. However, small geometric changes caused by the works on the summit like changed position of equipment or small stone falling down during walking reduce the coregistration accuracy. These points are removed by an iterative process that generates a histogram of M3C2 distances of corresponding points in the two point clouds. The five percent points with the highest distances are cut out from the data set as points that either are wrong point pairs or regions with unexpected small geometric changes. ICP is repeated for the new reduced data set and the M3C2 distances are recalculated. This is repeated until all distances of the stable part of the summit are below a threshold. The remaining point cloud gives the parameters for the coregistration of the second point cloud to the first. These transformation parameters are now applied on the original second point cloud including all points.
To estimate the noise level in this work, the next step compares the two point clouds generated from two image sets of the same day taken with several hours difference. These point clouds include changes in lighting conditions and slight differences in the orientations of the images. As no geometric changes around and in the crevice are expected for such a short period, both point clouds should show M3C2 point distances in the level of the noise of the point cloud generation quality. The median M3C2 distance is the threshold of the noise level. The change detection uses the M3C2 distance with this threshold to remove noise and detect the geometric changes.

EXPERIMENTS: THE HOCHVOGEL CREVICE
The mountain Hochvogel (Fig. 3) is situated at the border between Germany (Bavaria) and Austria. Hochvogel is the highest mountain in the Allgaeu alps with 2592 m. Hochvogel is characterized by its steep rock slopes and fast erosion processes caused by a very choosy limestone (Scholz and Scholz, 1981;Krautblatter and Funk, 2010). The reduction of permafrost in the last decades destabilized the summit region of Hochvogel. This geologic behavior causes big mass movements and several rock falls. The summit is crossed by a big crevice of up to seven meters width and visible down to 10 meters of depth. The deeper parts of the crevice are invisible due to the debris. It has a total length of 35 meters through the whole summit from north east to south west and several narrow crevices in the south west of the summit ( Fig. 4). The outer part of the summit that is separating by the large crevice is moving in a gravitational movement to the south (Leinauer et al., 2019) with approx 2 cm per year. The 3 dimensional complex movements in the summit of the Hochvogel should be estimated.
In addition to the given ground control points used for defining the local coordinate system and for the geodetic measurements (Fig. 5), additional ground control points have been placed in the scene to have more points for the initial orientation estimation.
Images have been taken during three measurement campaign on 2018-07-09 (one image set in the morning, one image set in the afternoon for the noise level estimation), 2018-09-27, and 2019-07-19. A Sony Alpha 7RII with 42 megapixel and 15 mm focal length was used to take the images. The camera was mounted on a stick stand with a maximum length of 7 m (Fig. 6). The camera records the scene with at least 80% overlap in parallel views moving along the crevice.

RESULTS AND DISCUSSION
During the measurements, several groups from geology, geodesy and photogrammetry were working on the summit and installing ground control points and equipment. This leads to small geometric changes and changing occlusions by person during the recordings.
ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume V-2-2020, 2020 XXIV ISPRS Congress (2020 edition)   Table 2 shows the accuracy of the ground control points defining the local coordinate system measured from geodetic instruments (Wunderlich et al., 2019) and the estimated standard deviations of the ground control points after the bundle adjustment together with the remaining mean reprojection error of the ground control points in the images. One can see that the accuracies are in a range of a few millimeters.

3D reconstruction
The results for the 3D object coordinates estimated in the bundle adjustment shows that the standard deviation of the points is below 1 cm in the center of the image set and gets worse to the borders of the set ( Fig. 7 and 8. This is mainly caused by the lower number of images that see these points and by the configuration of the ground control points. The ground control points are aligned along an axis due to the formation of the crevice and accessibility of the different parts of the summit. This makes the orientation unstable to a certain point. The reprojection error for the three image sets for July 2018, September 2018, and July 2019 are 0, 225pixel, 0, 119pixel and 0, 162pixel.
Looking at figure 9, one can see the M3C2 distances of the photogrammetric point cloud from September 2018 to the corresponding point cloud taken with a terrestrial laser scanner. The standard deviation of the scanner σ is about 2mm. The mean M3C2 distance of the point clouds is 0.94mm with a standard deviation of 5.6mm. As seen in the figure 9, the main difference is in a small area where different occlusions of scanner and camera may cause  changes. Similar reasons are the small high differences which are mainly in the shadow areas of rocks.

Noise level estimation
The noise level estimation uses the two image sets July 2018 A and B taken during the first measurement campaign and compares the two generated point clouds. As we assume that there is no change in the geometry of the summit, the point clouds are coregistered using ICP to remove possible errors from the limited accuracy of the ground control point measurements. M3C2 calculates the distances of point pairs of the two point clouds. The mean distance µ of the points is 5.35 * 10 −4 m. The standard deviation σ is 0.013m.

Change detection
In change detection, two comparisons are done. The first compares the point cloud from epoch one (July 2018) to the second (September 2018) where no movements have been detected neither by geological nor geodetic measurements. In fact, using the estimated noise level, almost no significant changes are detected in the photogrammetric point clouds (Fig. 10). The left images shows the detected changes. Small rocks have moved during the summer, but no movement of the whole crevice is visible. The right images shows the cleaned point cloud with stable points only that have been used for the fine coregistration.
The mean value µ of the M3C2 distances is 9.5 * 10 −5 m and the standard deviation σ is 0.0015m. This is below the estimated noise level and thus no movement is detected.  If we have a look at the distances of the point clouds from September 2018 to July 2019 (Fig. 11, the results look totally different. On can see M3C2 differences of up to 3 cm in the left part of the crevice and that more or less all parts of the crevice are moving. If one looks at the histogram of the M3C2 distances (Fig. 12, one can see three peaks at -1.4 cm, +8 mm, and +1.6 cm. If the points are segmented using a histogram based threshold around these three peaks, one can see, that all three peaks belong to a certain characteristic (Fig. 13. The 8mm peak is more or less only on horizontal plains that are going down, whereas the 1.6 cm peak is located on more or less vertical surfaces that are moving away from the stable part of the summit. This distances and their locations describe movements of the sliding part of the summit. The peak at -1.4 cm belongs to objects that are falling into to crevice. The movement in total seems to be stronger on the left side ( Fig. 9. Characterising the type of movement, the results indicate a combination of topple and slide which fits to the geologic reference measurements.
An interesting observation can be made at the smaller crevice that is perpendicular to the big crevice. Here, we see a movement along the crevice, more or less in the same direction as the movement in the main crevice.  data set one can see a displacement of these characteristic structures.
If we compare our measurements and the interpretation with the geodetic measurements (Fig. 14, we can see that the changes of the control points on the moving part of the summit support the ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume V-2-2020, 2020 XXIV ISPRS Congress (2020 edition) results of the photogrammetric measurement and interpretation of the point cloud differences. The mean value of the movement is 2cm which fits to our estimation of the movement of the vertical surfaces away from the stable part of the summit. Table 3 shows the comparison of the geodetic measured movement and the estimated movement from the point cloud differences.

CONCLUSION AND OUTLOOK
The image acquisitions in this project were done with a terrestrial camera system. It is possible to replace this terrestrial camera by a UAV/RPAS mounted camera system. This would most likely speed up the recording process that was quite complex due to several groups working at the summit at the same time and thus generating artificial changes from persons and material that had to be removed before processing. A second limitation is the possible ground control points configuration. As the summit and crevice have an elongated shape and the moving part is hardly accessible, the ground control points for the coordinate system are also elongated which makes the registration unstable at the ends of the image sets. Nevertheless, it is possible to reach high geometric accuracies for the photogrammetric point clouds. a good ground control point set for absolute image orientation. Compared to the terrestrial laser scanner used in parallel we can see that we reach similar accuracies. The overall recording times are also comparable as the laser scanner needs a couple of positions to be placed for observing the whole crevice. Regarding the view into the crevice itself, Photogrammetry has the benefit of being able to looking down to areas occluded for the laser scanner. Airborne laser scanning systems however could reduce that position limitations of the terrestrial scanner. Mounted on aircraft, the point density will be much lower than for the photogrammetric point cloud, for UAV mounted systems the accuracies and point densities are limited. So, we propose a combination of low attitude photogrammetric UAV-based recordings and, for occluded areas, terrestrial images. Image sequences would also allow for image based change detection. This could be taken into account for additional change detection, but is limited due to its dependency on lighting conditions, especially shadow corners.
Future steps include the use of UAV/RPAS systems for carrying the camera. This has the advantage that exterior orientation parameters are measured for every image and can be used as initial values in the bundle adjustment. Nevertheless, using only these orientation will only lead to an optimized inner accuracy. It has to be investigated, whether this accuracy in combination with the noise removal and an point based coregistration can reach the same absolute accuracy as the method based in ground control points.
The interpretation of changes as specific movements can be improved by adding a segmentation to the point cloud and group points by radiometric and geometric features and detected edges as well as similar estimated movement vectors. This way, it would be possible, to track objects instead of assigning points of two point clouds to each other that show not necessarily the same object.