TECHNIQUES FOR MOSAICS DOCUMENTATION THROUGH PHOTOGRAMMETRY DATA ACQUISITION. THE BYZANTINE MOSAICS OF THE NATIVITY CHURCH

This paper describes a sequence of actions developed to guarantee a reliable and suitable dataset for the creation of detailed ortho images of Nativity Church mosaics. During acquisition campaigns, different photogrammetric techniques were tested, and different survey instruments were compared to improve the quality of the data obtained. The different outputs allow the adjustment of the instrument parameters and the acquisition methods, to structure a methodological process aimed at obtaining an accurate level of detail to describe the individual mosaic tile. From the realization of a reliable photomosaics an automatic vectorization system has been developed. This process, aimed at digitizing the tiles of the Church walls and the pavement mosaics, responds to a documentation and management purpose and to an objective of structuring a data acquisition method and post-production that can be replicated on other mosaic contexts.


INTRODUCTION
The study and documentation project of the Nativity Church mosaics involved the use of digitization technologies combined with non-invasive diagnostic analysis. The methodological approach illustrated here aims at devising a program of actions for the realization of reliable photogrammetric outputs at the micro-scale level. For some years now, this purpose has led to the development of several research lines, aimed at the digitization of mosaic surfaces for management and conservation (Monti, 2012). With the will to promote a documentation process for the management of mosaic tiles, such digitization technologies require a high level of metric and colorimetric accuracy, in order to develop a multilevel reading of the digital data that has been obtained. For this reason, the survey activities should be conducted at different investigation levels: one in detail, which allows to understand the geometries and materials of the individual tiles, and a larger one, useful to globally understand the design of the mosaic surface and contextualizing it within the excavation space or in a specific spatial context. (Manfredini, 2012). The achievement of a high reliability level, aimed at managing mosaic tiles, becomes possible through the structuring of an experimental procedure, repeated from time to time, based on the changes and adjustments necessary to improve the final output. The survey project for the documentation of the floor and wall mosaics of the Nativity Church has been conducted by the research team of DAda laboratory of the University of Pavia. This project is part of the documentation activities that the laboratory is carrying out since 2013, promoted by restoration company Piacenti SpA. (Parrinello, 2015). The experience of the DAda research laboratory in the Holy Land is nowadays a consolidated reality, also thanks to urban-scale projects and not only project related to the individual building (Bertocci andall, 2018. Parinello, 2015). The aim is to obtain a digital reliable ortho-mosaic representation of the floor and wall byzantine mosaics of the Basilica, in order to start a digitization processes and multiscale reading procedures of the polychrome surfaces, from the geometric characteristic of the individual tiles to their spatial arrangement. In this research project, the use of photogrammetric survey methodologies is at the center of the experimentation. The paper focuses on data analysis and output optimization, even comparing 2D image ortho-rectification and SfM photogrammetry to reach the best result for the digitization of tiles. Today, under the current floor, there is the possibility to admire portions of the mosaic carpet of the first Constantine Church, dating back to the 4th Century (Bagatti, 1952;De Girolamo, 2017). The mosaic floor, discovered in 1934 by William Harvey, was covered after a photographic documentation campaign and measurements of the main mosaics drawing borders (Madden, 2012). During the last years, several portions of the original mosaic have been uncovered, both on walls and pavement, to assess their state of conservation and to give visitors the possibility to appreciate the decorative heritage. The documentation project focused on a portion of wall mosaics related to an Angel of the procession to the altar that was totally covered with layers of plaster. The mosaic tiles, in a wide range of colored, silver, and gold-plated stones, belong to the early Christian period. The shapes, colors and iconography depicted in the big carpet of the floor mosaic refers to Christian symbolism. There are animals and vegetation representations, characteristic symbols of the Eucharistic decorations. For the huge extension, preciousness and richness of the drawings, the Nativity Church mosaics are considered important witnesses of the Christianity in the Middle East territory. These Byzantine mosaics becomes an emblem of the double observation scale, from the general to unity, i.e. from the general geometric design to the decorative tiles. For this reason, the documentation process will be more effective as much as it manages to acquire both levels of scale. The documentation and the digital drawing of each mosaic tile, intended as a minimum subdivision element, allows to give quantitative information on the number of tiles present but also qualitative information about tile significance within the general system. (Manfredini, 2011). Given these premises, it is possible to create a tool for the management and maintenance of the discovered walls and pavement surfaces to accurately document the state of conservation and to plan more suitable restoration or consolidation actions for those mosaic surfaces portions at risk of detachment. (Tucci, 2010).  Another aspect that influenced the acquisition campaign is related to religious functions being officiated in the Church during the day, besides the problems related to the shared management of the basilica space by the three cults present, Franciscan, Orthodox and Armenian. (Bagatti, 1952). Through direct observation and through the generation of mosaics 3D point clouds, with data from TLS and from SLR, Figure 5. Mosaic tiles during the restoration work the diversity of the conformation between the wall and floor mosaics tiles was noted. The floor mosaics are characterized by different tile sizes, according to their function: black and white frame tiles with a size ranging from 1 cm to 1.5 cm per side and tiles for the decorative apparatus with a size of the side less than 1 cm. Despite localized yielding going on nowadays the different dimensions, the floor tiles share parallel positioning with respect to the ground, in order to constitute a walkable decorative carpet. Wall mosaic tiles, on the other hand, have a highly diversified course, linked to the point of view of the observer in the Basilica. They are in fact inclined along the horizontal line up to 30° facing the floor of the Nativity, arranged differently on the lodging plan. Figure 6. Images of the point cloud from TLS of mosaic A, which highlights the localized subsidence of the flooring This variation of arrangement strongly influences the visual effect of the mosaic, thanks to different light reflections that change its three-dimensional perception (Tosi, 2004).

DOCUMENTATION AND SURVEY OF FLOOR AND PARIETAL MOSAICS
The documentation campaigns are configured as a pilot case to test acquisition methodologies that can be replicated on the entire investigable mosaic surface, besides a methodological protocol that can be tested on other contexts. (Cantisani, 2006).
The survey and documentation of the floor mosaics temporally follow those of the wall mosaics (XXX, 20XX). Following the discovery of the floor mosaics starting from 2018, it became necessary to create a tool that would document the restoration work and the management of the historical pavement over time.
The goal, in addition to the creation of detailed ortho-images, is the digitization the surfaces in order to perform computations on the individual tiles. The six documented floor mosaics were classified with alphabet letters in the order in which the survey was carried out. The first mosaic acquired, during the January 2019 campaign, is mosaic A, the largest restored floor system. (XXX et al, 20XX).

Data acquisition phase: testing of integrated acquisition methodologies
The Nativity Church is a complex environment in which it is necessary to carefully plan the documentation operations in order to optimize the field work. The floor mosaics, already restored and equipped with a walkable lid closure system, occupy a large area of the flooring. This requires the opening of the covers, and a reduction in the space usable by visitors.
The project was organized in order to keep the floor mosaics uncovered for as little time as possible, taking into account the strong and fast variations of natural light within the space, due to climatic conditions and the arrangement of the openings that generated shadows and variations of color perception.
To obtain a digitization of each tile of the floor mosaics, several tools have been analyzed to combine the strengths of different documentation methods.
The data acquisition has been characterized using different survey methodologies: the TLS laser scanner survey, direct survey for smaller floor mosaics, SfM methodologies and 2D photogrammetry with SLR (Adami, 2018).

TLS point cloud data comparison
The use of TLS survey allowed to obtain a topological survey of the mosaic's morphology, reducing the target assignment error when compared to a direct survey, and monitoring the geometry With the laser scanner point cloud fit has been possible to highlight differences between the wall and floor mosaics, to obtain the bases to scale the photogrammetric models but alto to analyzed some critical aspects related to the mosaics surfaces, such as the settlements that led to the misalignment of the individual tiles with respect to the flooring and the occlusions in the mortar joints between them which sometimes generate shadows and subsequent lack of data. A lot of tiles present damages and gaps, some are fragmented with mortar in the middle and many have been reconstituted during the restoration with different material in order to distinguish them from the originals. These issues create refraction variations which result in aberrations and distortions in the point cloud. To obtain a detailed ortho-mosaic, the survey was carried out for photographic campaigns, obtained by SfM technology and for photographic shootings by 2D photogrammetry, integrating the different techniques together (Kraus, 2007).

Photo shooting mode for 2D photogrammetry and SfM methodology
A detailed photogrammetric survey campaign was carried out via specific instruments, supports and lighting systems. The large extension of the mosaics, as well as the complexity of the decorations represented, required to organize the photography campaign according to an accurate and organized data archive, composed by a ordinate sequence of squared 2D images each one oriented in x, y, z coordinates, capable of easily orienting the single tile into the mosaic. The mosaics were divided into a regular square, having 15, 30 or 45 cm per side based on the complexity of the tiles drawing. The squares were manually divided on the mosaics with quoted wire and numbered during the acquisition phase. This subdivision constituted a regular grid which, distributed over the entire surface, made it possible to reference each squared with respect to the perimeter of the excavation. The photographs were taken with a SONY alpha 6000 camera, set to 24mm fixed focal length, ISO at 100 and times varying from 1/5 "to 2" thanks to the help of a photographic tripod, that has also been useful to keep the tight wire to define the reference grid. The photographs were taken with vertical shooting in reference to the plane of the mosaic, in duplicate (with numbers and without numbers), also including the side portions for a necessary overlay. The To have a textured model, readable for vector drawing, the orthomosaic generated by SfM was used as a reference. The SfM model was oriented via three-dimensional model built from the laser scanner point cloud, not suitable for a detailed reading of the edges of the tiles. Aiming at the structuring of very highdetail colorimetric representations for the vectorization of the individual mosaic tiles, the generation of otho-mosaic from SfM methods, and then the integration with 2D photogrammetry, is optimal. This is due to the possibility of orienting the models obtained by integrating with the data from TLS in space, having the possibility of texturing the model and proceeding directly on it for the tiles tracing (Remondino, 2011). The shooting mode for SfM was conducted in two different ways according to the size of the mosaics, extended over the entire portion for those of limited dimensions (with the longer sides within 4 m in length as mosaic C, E, F) and dividing into smaller portions for the mosaics with larger extension. The quoted wire used to partition the mosaics during the flat photogrammetric campaign was maintained for the SfM photogrammetry, to maintain the references between the two acquisition methods. In this way it was possible to correct the photographs taken previously on the basis of the ortho-mosaic made at this stage. The existing lighting of the central nave was used, with uniformly distributed lamps in order to obtain a diffused light. A Canon EOS 77D camera with a fixed focal length 18 mm, ISO at 150 and variable times up to 2.5", was used on a tripod. The test of vectorization process was conducted in parallel with the shooting phase, in order to correct some procedural aspects directly during the photographic campaign and improve the images obtained to facilitate the post-production process. The vectorization of orthomosaic has been conduct on SfM orthoimage and on 2D photogrammetry image, to compare the quality of the final output and to improve the data set on each survey campaign. (Manfredini, 2012) The photographic acquisitions by photogrammetry or close-up detailed photos lead to two types of survey with strongly different timings based on the extension of the mosaics, as well as strongly different methods and times of post-production phase. A further aspect that should be considered is related to the work schedule: setting up the grid on the walkable surface and creating the set of photographs for each portion of mosaic is the longest operation of the entire acquisition process. Compared to the SfM acquisition, 2D photogrammetry involves significantly more time and greater risk of making errors in the execution of the dataset. To this aspect is added the time in the adjustment of the distortion given by the lens and the correction of the single squared portion 45x45, 30x30 or 15x15 cm. For the floor mosaics, the choice to proceed with SfM in addition to the planar photogrammetry was therefore necessary to significantly reduce the amount of hours of post-production work, after analyzing that the characteristics of the ortho images were comparable. Since the tiles of the Angel mosaic were not coplanar, it was necessary to proceed with SfM photogrammetry, allowing for a Figure 12. Morphology analysis of the wall mosaic Angel with detail of the mesh methodological comparison with the floors. The 3D models obtained were elaborated to obtain mesh and texture and were scaled thanks to the surveys carried out in the field through target in common with the laser scanner point cloud. The space-oriented model allowed to export high-quality ortho-mosaic for system documentation that became the base on which to set up the semiautomatic vectorization work for the definition of the tiles borders.
Figure 13. 3D model of mosaic E, measurements, and orientation with target points, acquired by direct survey an understanding of the possible critical issues during the acquisition phase. Variations in the brightness of the pixels, which can be linked to discontinuity of the soil and therefore of distances, variation of the orientation of the tiles, properties of the variable materials and lighting from the environment lead to strong variations in the output obtained by the edge detection filter. Tests were carried out on different portions in order to put

POST-PRODUCTION: CHOOSING PARAMETERS OF PHOTOGRAPHIC OUTPUT TO MAXIMIZE EFFECTIVENESS OF EDGE DETECTION FILTERS
During the photographic acquisition phase, it has been necessary to monitor how the digital images acquired and processed had been analyzed by the software to perform the extraction of tiles borders. To digitize and automate the detection of the edges of the mosaic tiles, it is key to keep in mind that the detection is performed based on the difference in brightness among pixels and on an imposed tolerance. The edge detection processes can take advantage of different filters that calculate the brightness gradient of the image on directed along the x and y axes (Pajdla, 1993;Park 2008). Filtering occurs through the convolution of the filter kernel, applied as a sliding window function on the original pixel matrix. The first filter is applied to detect the x component of the output image and then passes to the second component Y. The filter is natively on a gray scale, and therefore acts on a single-color matrix. Working with a multichannel image gives the possibility to combine the result of the convolutions (three in the case of RGB.). Edge detection allows to highlight the pixels of a digital image in which there is a variation in light intensity. This type of procedure determines Figure 14. 3D models of mosaic B (above) and mosaic C (below) obtained by SfM photogrammetry. In case of extensive mosaic, the SfM photogrammetry has been performed inside the mosaic walkable area remedy to this problem, graphically processing photos in postproduction to increase the contrast between the mortar and the tiles making them more readable. The photos of which the shooting methodology has been described have been experimented with edge detection filters, on small portions, with different conditions. This way of proceeding, testing the postproduction in parallel with the documentation, is what allowed to obtain a correct data set for the subsequent phases. The semiautomatic vectorization of the edges of the mosaic tiles (Adobe Illustrator is the most tested software by us) generates the twodimensional drawing of the mosaic system. The fundamental steps, photography, orthophotos and vector drawing of the mosaics tiles, are what have been investigated for the future purposes of cultural enhancement of the mosaic's system.

CONCLUSIONS
The experimentation was conducted on the management and integration of the various acquisition and post-production Figure 15. Ortho-image of mosaic C obtained from SfM methodology methodologies for the mosaic works of the Nativity Church in Bethlehem. For each type of procedure performed, it assessed the accuracy of the result, the timing, and the final graphic rendering of the requested product. Digital representations, twodimensional or three-dimensional, are now widely used both for the planning of restoration processes and for the creation of digital information archives (Monti, 2012). This protocol allows to produce different types of outputs, from two-dimensional drawings to 3D models, which can be used effectively in valorization activities, but also as analysis instruments for the heritage management. The digitization of the artistic and architectural assets allows to intervene on multiple topics. The Figure 16. Semi-automatic vectorization process of the mosaic E in Adobe Photoshop, for the management of the threshold, and in Adobe Illustrator, for the definition of the tile's edges cases range from support to maintenance and management through information systems, with catalogs that can be updated on the basis of the necessary or planned interventions, allowing for digital use in augmented reality, becoming a method to allow all user categories to reach places otherwise inaccessible. The survey and documentation process carried out led to detailed documentation of the mosaic apparatus of the Basilica, in which 1: 1 scale managed ortho-images can be the basis for all types of future interventions. Through a mapping of each individual mosaic tile and 3D models of the deformation of the support structure (soil or masonry), it is possible to set interventions, restoration and maintenance work, as well as supporting tourism and information activities. The Figure 17. Semi-automatic vectorization process of white, black, and polychrome tiles, compared with ortho-image of mosaic E documentation created is an initial dataset on which to develop other projects to enhance and manage the mosaic heritage.