SEGMENTATION OF BUILDINGS BASED ON HIGH RESOLUTION PERSISTENT SCATTERER POINT CLOUDS

Integrating differential synthetic aperture radar measurements into building information modeling systems requires a mapping of these measurement points onto structural parts of the building. We use a reverse geocoding approach to project building footprints into slant-range geometry, which helps to accurately assign PS points to single building identities. By treating the deformation time series as points in a high dimensional feature space, we can use dimensional reduction and clustering techniques to extract clusters of points that show a similar movement behavior. We visualize these clusters by mapping them onto ground truth, using laser scanning point clouds. Our approach segments buildings into plausible parts.


INTRODUCTION
In the last twenty years the monitoring of deformation processes such as slow landslides, mining induced subsidence and post earthquake tectonic movements has been more and more carried out by interferometric synthetic aperture radar (InSAR) techniques (Hanssen, 2001). When it comes to monitoring urban areas, persistent scatterer interferometry (PSI) (Ferretti et al., 2001) has proven to work reliably and is becoming a standard technique in the portfolio of monitoring. The availability of open space-born SAR data made nation wide deformation mapping services possible. However, the rather low spatial resolution of such data limits their applicability for monitoring at finer scales , for example, motion patterns of individual buildings. High resolution SAR data such as TerraSAR-X are suitable for this task, but come with the downside of a limited acquisition capacity. In the future, upcoming SAR satellite constellations will provide a greater availability of high resolution SAR data, better than 1 m × 1 m (Farquharson et al., 2018). This enables officials the possibility to efficiently monitor smaller objects of interest, such as critical infrastructure like bridges and dams as well as single buildings.
The results of such monitoring can be either handled isolated or set in relation to other information related to the building. Such building information modeling (BIM) systems use sensors, meters, network infrastructure and 3D modeling to create a digital twin of the structure to inform decisions, such as predictive maintenance (Khajavi et al., 2019). PSI analysis of high resolution SAR data often covers buildings with thousands of derived PSI points in the order of 1 PS per m 2 on the façades (Schunert et al., 2012). In the following, we are exploring a method to aggregate and cluster these scatteres into groups of points that show similar movements and therefore are assumed to be located on the same structural part of the building. These parts then could be linked to actual structures of the building via a BIM and allow to augment the information with deformation histories. * Corresponding author Tanaka and Hoshuyama (2017) showed that an evaluation index derived from the correlation of displacement and topography phase shift can be used to extract rigid structures, using minimum spanning trees. A deformation history based clustering was initially used by Zhu et al. (2018) and Costantini et al. (2018), who suggest a clustering algorithm to find temporal deformation patterns in COSMO-SkyMed DInSAR results. By comparison to in-situ ground truth measurements (Schneider et al., 2020) showed that such clusters indeed reflect the actual movement of building parts, but the restricted clustering algorithm could only extract a fraction of potential clusters.
The focus of this work is to refine the allocation of PS points to single buildings and the following clustering algorithm that segments these into independent parts. By combining the results with laser scanning data, we can visualize the extracted building segments and show their plausibility. We also consider this investigation as a preliminary step towards an integration of In-SAR based city wide monitoring results into BIM Systems.
In Sec. 2 we describe the SAR and laser scanning database, followed by the methods for label transfer and a detailed description of the clustering approach. Finally we present our results in Sec. 3 along with more examples in the appendix 4.1/4.2. In Sec.4 we comment our outcome and give an outlook to future directions and applications.

ALS-Data
We are using airborne laser-scanning (ALS) data to represent the 3-dimensional structure of buildings. The ALS Data has a point density of 40 pts/m 2 and a positional accuracy of σXY Z = 0.02 m. We also derived a digital surface model (DSM) from this point cloud. The DSM served as reference elevation model for the PSI processing and the Reverse Geocoding as described in Sec.2.4, as well as for visualizing the PS-Clusters on the building in Sec. 2.5.

SAR-Data
The SAR data we are using has been acquired by the German X-Band SAR satellite TerraSAR-X (TSX). The corresponding slant range -azimuth resolution for the "High Resolution Spotlight 300Mhz" acquisition mode is 0.6 m × 1.1 m (Airbus, 2017). The 132 images were captured during a 4 years time span (September 2016 to October 2020) with 11 days repeat cycle. For the interferogram generation a master image in November 2018 was chosen (Tab. 1). Thus the maximum spatial baseline does not exceed 400m (see Fig. 1).

Persistent scatterer interferometry
The image stack was processed with L3 SARscape's Persistent scatterer interferometry -algorithm (SARMAP, 2020) that is implemented based on Ferretti et al. (2001). For interferogram generation and geocoding, we used a precise surface model, derived from the airborne laser-scanning (ALS) data. As the coherence threshold for the PS-points we choose 0.7.
Persistent scatterer interferometry (PSI) is an advanced InSAR technique. The main idea of this algorithm is the detection of temporally coherent pixels in a stack of co-registered SAR images. By analyzing the phases of such pixels in each image of the stack, relative to a master image, the line-of-sight (LOS) movement history and a 3D position of this scatterer can be estimated (Ferretti et al., 2001(Ferretti et al., , 2000. Crosetto et al. (2016) give a good overview over the history and the capabilities of PSI algorithms. For more detailed insights we highly recommend reading that article.
PSI works well for dense urban areas, since man-made structures especially metal parts located at house façades and roofs act as a good reflectors (Schunert et al., 2012). The PSI algorithm is able to estimate the scatterer's lateral positions in the order of the pixel size while the height component is typically less accurate (Chang and Hanssen, 2014).
Since PSI is analyzing time series of multiple SAR images, the displacement history of each scatterer is on of the results. For every P Sn point we obtain its relative deformation dn(tm) as a time series with a measurement for each SAR acquisition (m ∈ N | 1 ≥ m ≤ 132) (see Eq. (1)). The accuracy for each measurement can be better than 2 mm (Maccabiani et al., 2017).
The deformation time series represent the most advanced PSI product and is the base for the later clustering approach. As Gernhardt et al. (2010) and Crosetto et al. (2015) have shown, PSI time series, derived from high resolution SAR data, are able to reveal the annual movements of buildings. They confirm thermal expansion of buildings up to several millimeters in amplitude over the year. We exploit this fact for our clustering, under the assumption, that each segment of a building exhibits a characteristic movement behavior.

Label Transformation
To assign PS-points to single buildings we used Open Street Maps (OSM) building footprints. Since the PS-algorithm is able to determine a lateral position for each scatterer, a straight forward way to find PS-points in a buildings polygon would be a simple "inpolygon" query for each PS. This approach is shown in Schneider et al. (2020) by considering a buffer around each  building. To overcome the inaccuracy and issues of falsely assigned points we implemented a Reverse Geocoding approach as shown in Fig. 2. The main idea hereby is to utilize an existing pipeline (SARMAP, 2020) to geocode two lookup tables (LuTs). The LuTs contain a unique identifier for each range and azimuth cell. After geocoding with a precise DSM these LuTs are used to transfer labels respectively building footprints into the master images range-azimuth geometry. This allows precise assignment for each PS-point, without the need of the estimated UTM-coordinates. The results as shown in Fig. 3 display a sharp distinguish between neighboring buildings, which would be a challenging task with the inpolygon -buffer method. This approach is also suitable to transform training data into rangeazimuth geometry i.e. for land classification applications.

PS-Point clustering
We treat the deformation histories of each PS point as points in a M dimensional space, with a dimension for each acquisition date. Each point dn ∈ R M is defined by the M measurements d as shown Eq. (1). (2020) have shown for similar data, that "1 minus the sample correlation " (Eq. (2)) is a good heuristic metric for a densitybased spatial clustering of applications with noise (DBSCAN) clustering (Ester et al., 1996) in this space.
Nevertheless these results lead to very sparse clusters with the results being vulnerable to the choice of DBSCAN's parameter. Therefore we use a multi step clustering approach: I t-distributed stochastic neighborhood embedding (t-SNE) (van der Maaten and Hinton, 2008) to reduce the M dimensions drastically to a two-dimensional space. The metric to determine the distance between two PS deformation time series da and d b for the t-SNE embedding is set to 1 − corr(da, d b ) (Eq. (2)) (Schneider et al., 2020). The resulting two-dimensional representation should preserve the local neighborhoods respectively clusters in the original data.
II Each point in the embedded result is then characterized by a core distance (CD) density estimator (Ankerst et al., 1999). The core distance of a point is the smallest threshold such that the point is still considered a core object by the DBSCAN algorithm.
III We then exclude all points where the CD > median(CD).
IV The remaining embedded points are now clustered with DBSCAN ( = median(CD) · 1.5; minPts = 5). For each resulting DBSCAN cluster we calculate the center point d k = mean(d all ) in deformation space.
V Now we discard outliers by comparing each cluster point to the center using again the 1 − corr(d k , di) distance metric and a correlation threshold of Ct = 0.3. VI Finally we merge similar clusters by comparing there center pointsdj andd k and merge the cluster k with cluster j if 1 − corr(d k ,dj) < Ct.
The result of this consecutive clustering process are grouped PS-points that show homogeneous movement. In contrast to Schneider et al. (2020), the resulting clusters are less sparse and we can assign more than 80% of the PS-points to a cluster. The workflow, including exemplary interim results for each step, is shown in Fig. 4. To visualize the clusters on the building we are assigning each ALS-point to the closest PS-cluster via nearest neighbor, with a maximum distance of 10 m. The center points of each cluster in deformation space can be interpreted as the clusters mean movement over time.

RESULTS
For this study we evaluated several buildings with our method. In Fig. 5 we show a colored ALS point cloud. Each color represents a cluster. One can see that the PS-clusters divide the building into sections. This is especially remarkable since no spatial information about the PS-points location is considered in the clustering process. For instance, the tower in Fig. 5 is clearly divided into two clusters. The corresponding t-SNE plot is given in Fig.6. Our clustering approach was able to identify most of the visible clusters while distinguishing the independent, non-cluster points. The corresponding time-series for each cluster are presented in Fig. 7, along with the daily average temperature. The central point of each cluster is colored emphasized by color. In appendix 4.1A and 4.2B we included two more examples of segmented buildings. Both show a reasonable segmentation into sub parts.

CONCLUSION
We apply a reverse geocoding approach to map PS points onto OSM building footprints to accurately identify all scatteres on a single building. We used the deformation histories derived from high resolution SAR PSI analysis to extract clusters, which are groups of PS points that show a similar displacement behavior over time. We visualized the results by ALS point clouds of the buildings. The clusters on the structures show plausible segments of a the buildings, even though no spatial information  has been considered in the clustering process. The here shown data set consists of quite a high number of images (132) since the acquisition of such data can be very expensive it is worth to mention that previous experiments on shorter time series (30+ images) achieve similar results to some extend.
Our future and ongoing work is focusing to map such clusters onto building components provided by BIMs. Another interesting application of these clusters can be a risk assignment and stress prediction by comparing the movements of neighboring clusters.     Figure 10. The bottom plot shows the mean daily temperature. We are not sure about the origin of the "spikes" around June 2019. They could be processing artifacts due to atmospheric influence.