PREDICTION OF BUILDING FLOORPLANS USING LOGICAL AND STOCHASTIC REASONING BASED ON SPARSE OBSERVATIONS

: This paper introduces a novel method for the automatic derivation of building ﬂoorplans and indoor models. Our approach is based on a logical and stochastic reasoning using sparse observations such as building room areas. No further sensor observations like 3D point clouds are needed. Our method beneﬁts from an extensive prior knowledge of functional dependencies and probability density functions of shape and location parameters of rooms depending on their functional use. The determination of posterior beliefs is performed using Bayesian Networks. Stochastic reasoning is complex since the problem is characterized by a mixture of discrete and continuous parameters that are in turn correlated by non-linear constraints. To cope with this kind of complexity, the proposed reasoner combines statistical methods with constraint propagation. It generates a limited number of hypotheses in a model-based top-down approach. It predicts ﬂoorplans based on a-priori localised windows. The use of Gaussian mixture models, constraint solvers and stochastic models helps to cope with the a-priori inﬁnite space of the possible ﬂoorplan instantiations.


MOTIVATION AND CONTEXT
While 3D models of the exterior of buildings are meanwhile widely available, 3D building interior models (LoD4 in CityGML, Gröger and Plümer (2012)) are not yet widespread.Tasks such as rescue management, indoor navigation and guide for the blind have led to growing interest in the design and modelling of building interiors.In this context, Turner and Zakhor (2014) proposed an approach for the generation of building floorplans from laser range data based on a triangulation of a 2D sampling of wall positions.Becker et al. (2015) used shape grammars for the reconstruction of 3D indoor models from 3D point clouds.Ochmann et al. (2016) segmented a point cloud into rooms and outside area and reconstruct the scene by solving a labelling problem based on an energy minimization.For the derivation of indoor models, all mentioned approaches rely on dense observations such as 3D point clouds from laserscans or range cameras using mobile mapping systems.This requirement is often not able to be satisfied in an appropriate way so that we have to cope with an a-priori small number of observations instead.As a consequence, our central motivation is to predict unknown substructures in buildings such as floorplans, based only on few observations like the area of rooms and footprints.While footprints are already available by the use of data sources such as official data or Open Street Map, the acquisition of 3D point clouds is far costlier.This paper presents a novel approach for the automatic prediction and generation of building floorplans.Based on sparse observations, we automatically generate a limited number of best hypotheses and provide likelihoods for each solution.Dense observations like 3D point clouds are not required.We follow a modelbased top-down approach and exploit the fact that it is easier to verify or falsify hypotheses than to reconstruct models from observations in a bottom-up way.While most approaches expect observations of adequate density, characteristic for our approach is that we are able to generate best hypotheses for a floorplan based on otherwise insufficient measurements.The input consists of Figure 1: Our reasoning process restricts the space of hypotheses in three reasoning steps: incorporation of prior knowledge, Constraint Logic Programming and Bayesian Networks building footprint, exterior location of the windows and available information about rooms (area of rooms, identifying number of each room and possibly the functional use of each room).Most of these information can be acquired from building management services.The location of the windows can be derived using existing methods for the identification of building parts from point clouds or images of fac ¸ades such as, for instance, described in Dehbi et al. (2016) or Recky and Leberl (2010).The algorithm does not require any indoor images or laser scans from walls to predict nevertheless floorplans with high accuracy.Additional data may lead to a verification or falsification of models which however is less expensive than reconstructing a building interior bottom-up from measurements.
In the problem we solve, a building footprint as well as the area of each room are given.We assume that each room has a rectangular shape.Lower and upper bounds for the width and depth of each room are derived from probability density functions.The decision variant of our problem is to decide whether or not the building footprint can be partitioned into rooms that satisfy our specifications.In the special case that the building footprint is a rectangle and that for each room the lower bound is equal to the upper bound, this problem is exactly the same as Perfect-Rectangle-Packing. Since Perfect-Rectangle-Packing is known to be NP-hard (Garey and Johnson, 1979), our more general problem is NP-hard, too.
According to the Manhattan world constraint together with functional constraints like that walls do not intersect window areas as well as probability density functions (pdfs) of model parameters, the search space of possible floorplans is reduced considerably.We have to find a 1:N correspondence of one room to a-priori known windows.The dimensions of the rooms depend on the functional use and the area of the room.Furthermore, the fact that rooms with consequent room numbers are with high probability adjacent is exploited as a soft constraint and helps likewise to limit the combinatorial possibilities.We used the same paradigm for the successful prediction of fac ¸ade structures based on few observations like the fac ¸ade width (Loch-Dehbi and Plümer, 2015).The used constraints and their types for the floorplan modelling are listed in Table 1.
For floorplan design, Charman (1994) describe a knowledge-based system that generates all possible floorplans satisfying a set of geometric constraints on the rooms (non-overlap, adjacency, minimal/maximal area, minimal/maximal dimension, etc.).Therefore, they define the semi-geometric arc-consistency in order to adapt consistency techniques to geometric problems.In comparison to our method, this approach does not address the reconstruction of floorplans for existing buildings and does not take probable configurations into consideration.
In order to be able to scan huge model spaces, avoiding the pitfalls of approximate reasoning, and to exploit the potential of both observations and models, we combined Bayesian Networks (BNs) with Constraint Logic Programming (CLP).We designed a reasoning method which breaks down the problem into a feasible number of sub-problems for which exact inference can be applied.While the strength of CLP lies in solving combinatorial problems with non-linear constraint equations, BNs are used to reason with uncertain data.The implemented reasoner expects few observations as input and restricts the space of hypotheses to a few good ones.To this end, we incorporate prior knowledge in addition to BNs and CLP as depicted in Figure 1.Prior knowledge includes architectural as well as statistical constraints.The latter are characterised by probability density functions for the continuous parameters, like the width of rooms, derived from a large annotated relational building database of floorplans consisting of about 1160 rooms.Architectural constraints describe regularities such as alignments of the walls along corridors and topological relations like the non-overlapping of rooms.
Constraint programs are powerful tools to solve combinatorial problems.However, several approaches were developed that extend the framework by a stochastic component.Flerova and Dechter (2010) adapt combination and marginalization operators to find the m-best solutions for optimization tasks in graphical models.Intervals with cumulative distribution functions are used by Saad et al. (2010) in order to model a degree of knowledge for uncertain data.In order to address uncertainty, our approach combines the classical constraint propagation with Bayesian Networks and thus benefits from the strength of both paradigms.It efficiently restricts the solution space by solving the discrete problem using Constraint Logic Programming and estimates the optimal continuous parameters using a Kalman filter (Thrun et al., 2005).In order to avoid the exclusion of good floorplan hypothesis, the mbest solutions are retained following a probabilistic ranking.

LOGICAL AND STOCHASTIC REASONING
The implemented reasoner expects a few observations as input and outputs the best hypotheses, i.e. instantiations for discrete and continuous model parameters.To this end, we incorporate prior knowledge and combine Bayesian Networks with Constraint Logic Programming that results in the following three reasoning steps: 1. Incorporate prior knowledge 2. Propagate constraints 3. Calculate posterior belief Figure 1 illustrates the sequence of the reasoning steps in the presented method.The reasoner begins with an infinite number of hypotheses and narrows the space of hypotheses in each of the three reasoning steps to a small number of hypotheses.In the following, the three reasoning steps are introduced.

Incorporation of prior knowledge
Prior knowledge includes functional as well as statistical constraints.It further allows for the integration of architectural background knowledge that can be extracted from the annotated floorplans in the relational database.Domains of parameters are thus be restricted beforehand.Basic observations about buildings are available in the database with about 9 million building footprints extant in North-Rhine-Westfalia, Germany.Further background knowledge will be elaborated in the next section.
Continuous model parameters are further represented by probability density functions that can be approximated by Gaussian mixture models with n components where the ith Gaussian distribution N (µi, σ 2 i ) is weighted by wi.The use of mixture models enables us to use well-studied reasoning techniques such as Bayesian Networks where the model is defined in a directed graph with conditional probability distributions and functional dependencies (Koller and Friedman, 2009).
As an example for statistical prior knowledge, Figure 2 (top) shows the probability distribution for the width of office rooms.The prior knowledge is characterized by a Gaussian mixture with four components.In order to restrict domains in the CLP problem, components of the distributions are used to derive thresholds, i.e. they are represented by intervals of the form [µi − λσi, µi + λσi] with means µi, standard deviations σi and appropriate λ (3 or 4).Thus, the reasoner has not to deal with the a-priori infinite solution space of variables.

Constraint Logic Programming
Before applying inference techniques with Bayesian Networks we are interested in possible instantiations of the discrete parameters, e.g. the indices of the components of the Gaussian  The functional and architectural constraints that define the floorplan model together with the intervals derived from the pdfs restrict the domains of unknown parameters so that the final solution leads to a small number of qualified hypotheses.The output of the CLP component is used afterwards as evidence for statistical reasoning.CLP delivers the bilateral relations and adjacencies between the rooms as well as the correspondences of to Furthermore, CLP determines which component of a Gaussian mixture has to be considered.After this step, the floorplan model is restricted to single Gaussian distributions instead of mixtures.For further reading on constraint processing the reader is referred to Dechter (2003) and Marriott and Stuckey (1998).

Bayesian Networks
After the instantiation of the discrete parameters using CLP, the stochastic component has to reason within a specially structured Bayesian Network: a state-observation model with a n-dimensional state vector x ∈ R n representing the model parameters and a mdimensional observation vector o ∈ R m that can be described by the mapping o = M x with a measurement matrix M ∈ R n×m .For such state estimations the Kalman filter is an efficient algorithm for calculating the posterior.It assumes that state transition and measurement can be described linearly and initial beliefs are represented by multivariate Gaussian distributions: Gaussian distributions represented by µ and σ are carried over from the constraint solver of the reasoner and the posterior is computed in a correction step implemented by Kalman filter: where Q ∈ R m×m is the Gaussian noise of the observations and Id is the identity matrix.Figure 2 shows exemplarily the distri- The Gaussian mixture (red) for this continuous model parameter serves as prior knowledge and has four components.After instantiating the discrete parameters such as the component of the Gaussian mixture, the probability density function reduces to a single Gaussian mixture (green) for a floorplan at hand.The value of the continuous model parameter is finally predicted by using the correction step of the Kalman filter (blue) that yields means and variances for the continuous floorplan model parameters according to the given evidence.These parameters define shape (width, depth) and location parameters of floorplan objects such as rooms and corridors.Consequently, the output of our approach is a small set of ranked hypotheses with instantiated discrete and continuous parameters.

FLOORPLAN PREDICTION
Man-made objects such as buildings are often characterized by regularities.Geometric constraints, e.g.orthogonality or parallelity, are present in most buildings (Loch-Dehbi and Plümer, 2011).Building parameters such as the width or depth of rooms or corridors are restricted by architectural properties and can be described by probability distributions and functional dependencies.The presented approach benefits from such a prior knowledge about typical structures and distributions of building interiors that is based on an extensive data analysis.The building model is characterized by discrete as well as continuous param- Algorithm 1 depicts the approach for the prediction of indoor models using logical and stochastic reasoning based on sparse observations.Available information about the rooms (area of rooms, their identifying room number, their functional use) as well as the exterior location of windows and the building footprint is sufficient to create a floorplan that meets the reality with high probability.No further observations about the indoor model such as images or laser scans of walls are needed.At first, the footprint is rotated in such a way that its main axis is parallel to the x-axis of the underlying coordinate system.Afterwards, window intervals are extracted from their location information to narrow the domains for the CLP task.The algorithm is divided into two steps in order to acquire good models from a huge hypotheses space and to solve the non-linear problem: the constraint-based reasoning using Constraint Logic Programming (lines 1-3) and the statistical reasoning based on Bayesian Networks (lines 5-8).
One of the strength of the algorithm is the usage of prior knowledge.Model parameters such as width of rooms follow given Algorithm 1: algorithm for prediction of most probable floorplans Input: area of rooms (m 2 ), identifying room number, functional use, building footprint and location of windows, width of walls, distribution of model parameters Output: t-best floorplans with ranking // step 1: constraint-based reasoning 1 solve constraint satisfaction problem csp by querying discrete parameters i yielding a set s of solutions; 2 generate set of hypotheses h from s for floorplan models by incorporating instantiation for discrete parameters; 3 eliminate similar hypotheses in h leading to a set h of hypotheses; // step 2: statistical reasoning 4 for hi ∈ h do 5 build directed room adjacency graphs from hi; architectural restrictions.They are bounded by their probability density functions that in turn depend on the functional use of the corresponding room.For instance, a room can be functionally used as a working room in an office or as a lecture room in a university.The locations of walls are restricted by the assumption that walls cannot overlap with windows.The number of rooms that can be placed along a corridor is restricted by the width of the fac ¸ade.The fact that rooms with consequent room numbers are with high probability adjacent is used as a soft constraint and reduces the combinatorial possibilities considerably.The number of unsatisfied soft constraints is counted and must not exceed a given threshold, e.g.two.Table 1 summarizes the constraints that are used to restrict the search space.
In order to use the Bayesian Network efficiently, constraint-based reasoning is used first to solve the combinatorial problem for the prediction.This is performed by incorporating the constraints and logical facts derived from the prior knowledge.Without CLP, the adjacency of rooms and the correspondence of windows to rooms are unknown and make it impossible to define constraining relations between the variables of the neighbouring rooms.In addition, the Gaussian mixtures used as prior knowledge for the continuous variables would blow up the stochastic reasoning.To sum up, the use of constraint programming in the presented prediction process reduces the search space significantly and leads to: 1. determination of discrete states, 2. Gaussians instead of Gaussian mixtures.In this manner, instead of a multimodal distribution for each model parameter, we reduce the problem to one component of the Gaussian mixtures and thus are able to use well-studied efficient implementations.The logic program yields hypotheses consisting in the important information of adjacency of rooms, the correspondence of windows as well as the relevant component of the Gaussian mixture for the parameters of each room.Figure 3 shows the workflow of our approach with its used parameters during the reasoning process.
In a final step, the posterior distribution for each CLP hypothesis is calculated within the statistical component.Therefore, the correction step of the Kalman filter is used that yields the continuous parameters for each hypothesis.Since the constraints represented by the matrices for the Kalman filter are not fix but depend on the output of the logic program, directed adjacency graphs are used that represent the neighbourhood of rooms (right/left and top/bottom) and thus support the generic construction of the constraints for the Kalman filter efficiently in a recursive way.
The stochastic reasoning incorporates three types of constraints in order to update the Gaussian distributions for the location (x, y) and shape (depth and width) parameters of the rooms that were where floorX and rX denote the x-coordinate of the reference point of the floor and a given room respectively.Herewith, P red is the set of predecessor rooms derived from the adjacency graph.
The constraints in the y-direction are defined analogously.The third type of constraints describe the local relation between two adjacent rooms depending on the adjacency status.For example, if a room r1 is left to room r2, then their coordinates are related by: r1X + r1W idth + wallDepth − r2X = 0.
As a result the Kalman filter yields the continuous values that fit the observations and respect the constraints restricting the search space for the prediction of the floorplan.Finally, a MAP-based ranking yields the t-best floorplans.Since most building floorplans follow a regular style, the method is able to predict the interior with high accuracy.Extraordinary floorplans would force a relaxation of the constraints.Each predicted floorplan in the second row corresponds to the best ranked hypothesis found by our approach.For comparison, the reference floorplans are depicted in the first row.
Our approach can be extended in order to derive LoD4 models.
To this aim, walls can be extruded based on our predicted floorplans.However, doors have to be considered as well.Built upon our available database on building elements, in which the location of doors is labelled, a classifier can be trained in a supervised way to predict the position of doors.

CONCLUSION
This paper presented an approach for predicting and reconstructing a-priori unknown structures in building interiors.In contrast to other approaches which require sensor data like 3D point clouds, the implemented reasoner only needs few observations such as the corresponding footprint or the area of rooms to generate appropriate hypotheses with high probability.To this end, statistical reasoning was combined with Constraint Logic Programming.
Strong regularities in the appearance of man-made objects legitimate a top-down approach with a model characterized by strong constraints and distributions.Prior knowledge that supported the reasoning process was acquired by an extensive analysis of a ground truth database.Distributions of model parameters were described by non-parametric probability density functions and approximated by Gaussian mixture models.The distributions are characterized by a strongly peaked space of parameters that together with functional dependencies allowed for a generation of good hypotheses based only on few observations.Hypotheses can be refined by incorporating further observations.
We developed an algorithm that efficiently restricts the solution space by solving the discrete problem using Constraint Logic Programming and estimating the optimal continuous parameters using a Kalman filter.The reasoner provides means and variances for the continuous model parameters together with corresponding instantiations of the discrete parameters and outputs the best hypotheses given the observations.Due to the dominance of parallelism and orthogonality in man-made objects, our approach reconstructs models automatically following the Manhattan world constraints.The consideration of special objects which do not follow this constraint, will be the subject of future work.
overlap hard rooms are in the corresponding footprint hard room area = depth * width hard walls of room do not overlap with windows hard depth, width are bounded by pdf and impossible dimensions (e.g.restricted by law) hard rooms, respectively its walls, touch each other hard alignment of rooms along a corridor hard rooms with consequent room number are adjacent soft existence of at least one window in each room soft window part of one of the rooms hard

Figure 2 :
Figure 2: Probability density functions (pdfs) for the width of office rooms: Gaussian mixture as prior (top), from CLP selected component for determining the posterior of a given floorplan (bottom, green) and by Kalman filter corrected selected component (bottom, blue) butions for the width of office rooms in different reasoning steps.The Gaussian mixture (red) for this continuous model parameter serves as prior knowledge and has four components.After instantiating the discrete parameters such as the component of the Gaussian mixture, the probability density function reduces to a single Gaussian mixture (green) for a floorplan at hand.The value of the continuous model parameter is finally predicted by using the correction step of the Kalman filter (blue) that yields means and variances for the continuous floorplan model parameters according to the given evidence.These parameters define shape (width, depth) and location parameters of floorplan objects such as rooms and corridors.Consequently, the output of our approach is a small set of ranked hypotheses with instantiated discrete and continuous parameters.

Figure 3 :
Figure 3: Workflow of the prediction and used parameters during reasoning.Gm denotes a Gaussian mixture, Comp its selected component.Continuous parameters are in italics eters that follow common architectural regularities and can be constrained by non-linear functional dependencies.

6
construct matrices M, σ, Q and vectors o and µ according section 2.3; 7 calculate posterior belief for continuous model parameters by updating measurement according to equation 1; 8 M AP (µi|o) = argmax µ P (µi, o); 9 rank hypotheses h according to M AP (µ|o) and generate t-best floorplan models;

Figure 4 :
Figure 4: Examples of results of our floorplan prediction.First row: reference floorplans, second row: our predicted floorplans selected by the constraint-based reasoning.The first type of constraints are those between the shape parameters of the rooms and corridors and their corresponding floor as described by the following equation:

Figure 2
Figure2shows the prior and posterior beliefs of the filter algorithm.The beliefs are shown for three different steps.A-priori distributions (red) represented as Gaussian mixtures are the input of the algorithm.After solving the constraint satisfaction problem, the prior reduces to one component of the mixture (green) as shown at the bottom of the figure.They are specific for one solution of the problem and are given to the statistical component that integrates observations and model assumption to calculate the posterior belief (blue).As a result the reasoner yields au-

Table 1 :
(De Raedt, 2008)modelling floorplans.Soft constraints do not have to be satisfied but only a limited number of non satisfied constraints is allowed mixture.The problem is described by a set of constraints C = C1, ..., Cm on variables X = x1, ..., xn with associated domains D = D1, ..., Dn.It can thus be defined as a constraint satisfaction problem (CSP) whose solution is an instantiation of the variables, i.e. an assignment of values for each variable (x1, a1), ..., (xn, an) with (a1, ..., an) ∈ D1 × ... × Dn so that all constraints are satisfied.Constraint Logic Programming is well suited to solve those combinatorial problems with non-linear equations and is used during the reasoning process to instantiate the discrete parameters.CLP problems are defined by constraints that are expressed as logical formulas(De Raedt, 2008).