LOCATION DISTRIBUTION OPTIMIZATION OF PHOTOGRAPHING SITES FOR INDOOR PANORAMA MODELING

Generally, panoramas image modeling is costly and time-consuming because of photographing continuously to capture enough photos along the routes, especially in complicated indoor environment. Thus, difficulty follows for a wider applications of panoramic image modeling for business. It is indispensable to make a feasible arrangement of panorama sites locations because the locations influence the clarity, coverage and the amount of panoramic images under the condition of certain device. This paper is aim to propose a standard procedure to generate the specific location and total amount of panorama sites in indoor panoramas modeling. Firstly, establish the functional relationship between one panorama site and its objectives. Then, apply the relationship to panorama sites network. We propose the Distance Clarity function (FC and Fe) manifesting the mathematical relationship between panoramas and objectives distance or obstacle distance. The Distance Buffer function (FB)is modified from traditional buffer method to generate the coverage of panorama site. Secondly, transverse every point in possible area to locate possible panorama site, calculate the clarity and coverage synthetically. Finally select as little points as possible to satiate clarity requirement preferentially and then the coverage requirement. In the experiments, detailed parameters of camera lens are given. Still, more experiments parameters need trying out given that relationship between clarity and distance is device dependent. In short, through the function FC , Fe and FB , locations of panorama sites can be generated automatically and accurately.


INTRODUCTION 1.1 Indoor Panoramic Image Modeling
Recently, Indoor 3D modeling has been a hot spot in many research field such as smart city and Location Based Service (LB-S) for the favorable experience to present the indoor scenes.Commercial protential like Google Street View also boosts the widespread adaption and research on this issue.Methods are mainly divided into three part: traditional manual modeling, real 3D modeling based on images or laser point cloud data (Liu et al., 2005, Yinbao Zhang, 2014, Yinan Shi, 2016).
Manual modeling methods are those using algorithm or software to reform the structure and shape of objects.One successful application is 3D animation and computer game.These methods require large amount of manual operation which is timeconsuming, and the models are elaborate enough to present lively figures with precise geometry.Similarly, methods 3D modeling based on laser point clouds or depth images are accurate to record the position and geometry information.Particularly, laser point clouds and depth images are favorable to combine with manual modeling so that the 3D model can be vivid.Among these, indoor panoramic image modeling has aroused much concern since the virtual reality technology develops rapidly.It is a pseudo 3D modeling method because panoramas are still plane pictures.Compared with other indoor modeling methods, it is a fast modeling method with high degree of reduction and wealth information * Corresponding author though geometry might in a low level (Sandnes, 2016).Generally, panoramic images are reprojected onto a sphere which makes depth available when viewing the images (Zheng and Shi, 2008).Shuai Liu has summaried the applications of panoramic modeling and four main factors including stitching algorithm, acquire methods, image projection and deformation correction are mentioned to be important (Shuai et al., 2012).Lee and Tsai has make some respectful efforts in proposing a prove-of-concept in interior 3D modeling with panorama.They attached much significance to parameters of camera (Lee, 2015).In 2012, Google company put forward a combination of iterative vision-based pose estimation with user guided optimization enabling photographers to build street view outdoor, it take whole street into account (Colbert et al., 2012).In addition, there is plenty of research on image stitching, pyramid mulitiscale segmentation and image transition which requires accurate geometry information.
Most studies are devoted to data processing of panoramic images to improve the precision, optimizing a single panorama image by considering image processing algorithm and imaging ability of hardware.But dedication on specific design of panorama acquisition is in short, such as the arrangement of the location distribution of panoramic images shooting sites.Generally panoramic images are acquired by professional camera equipment or facilities based on vehicles.Images are shot along the route continuously.The lack of normative implementation standards for panoramic collection makes the panorama acquisition costly and time-consuming, not only for the instruments but also for the collection method.Finally the collection produces excessive panoramic images with bad visual effect which aggravates the expense of panorama acquisition because of the secondary collection in need.
To solve these problems, this paper is aimed to propose a standard procedure to generate the specific location of panorama sites in indoor panorama modeling.To arrange the location of panorama sites, it is important to make a balance between the panoramic image quality and the number of panoramas, given that the more the panorama sites are, the more images are available and they contribute to a higher quality and clarity of panorama modeling.The quality of panorama modeling is dependent on not only image processing driven by software but also image acquiring driven by hardware such as laser scanners and high precision GPS devices.Generally, to guarantee the quality of panorama images, more picture than the necessary amount were acquired during photographing by continuously capturing photos along the fixed roaming routes without any design.Besides, owing to the cumbersome hardware demanded to accurately locate the imagery site, such as high precision GPS devices (Colbert et al., 2012) Huang and Klette have derived formulas suitable for various sorts of stereo panorama imaging technologies by means of determining optimum parameters not only for high-accuracy panoramic image shooting and displaying with a special focus on automatic image disparity enhancement (Huang and Klette, 2010).

Evaluation of Indoor Panorama Modeling
On the basis of Huang and Klette's research, we determine the evaluation parameters capable to apply to balance the quality and quantity of panoramic images.
The quality of panorama modeling is dependent on not only the image processing with software but also the image shooting hardware.The image quality, such as clarity and distortion, affects the visual experience of users greatly.In many panoramic projects, more pictures beyond necessary were acquired during continuously photographing along the fixed routes without any design so as to guarantee the quality of panorama images.Meantime, if researchers collect panorama images in few sites the low coverage of neighboring panorama will give users a bad impact.The procedure of developing a panorama system without design is costly and time-consuming, which keep panorama modeling from widely applying (The application of panorama modeling is mainly limited in touring services such as street view because of high input in panoramic images acquisition, which slows the large expansion and fast development of panorama modeling in different business such as shopping mall).
The purpose of our study is to select as less points as possible to cover as much area as possible, meanwhile, making the image visual effect the best, which contributes a better effect of indoor panorama image modeling.In order to be able to carry out the optimization plan research, we first need to know the various factors that affect the indoor panoramic modeling system.Integrated a large number of previous research, surveys and experiments, we summarize three evaluation indexes as follows.

The Cost of Collecting Panoramic Images
The cost includes time cost and monetary cost.The layout and sequence of panorama sites are decisive to the amount of cost.In short, where to locate and in what turns to locate the panorama sites are crucial to be designed.The more sites in use, the more cost will need.

The Experience of Browsing Panoramic Images
The experience includes image clarity, image distortion, the overall stitching effect, interactivity, the brightness and saturation of panoramic images, the speed of scrolling images and the block of images at top or bottom owing to the limits of fisheye lens.The clarity reflects the resolution of panoramic images.We expect a high clarity able to distinguishing the details of interested objectives under the extreme condition that the panoramic image is at 100% zooming in or out.Obviously, the clarity and distortion are much related to the shooting distance while the overall stitching effect and interactivity are distance independent .

The Effect of Indoor Panoramic Navigation
The superiority of indoor panoramic navigation is the sense of reality and the intuitive.As the routes are linked to the panorama sites, the connectivity of panorama sites is required.Besides, the routes should be optimal to reduce the cost which signifies the importance of the location of panorama sites.
In conclusion, the location distribution optimization of photographing sites is based on the evalution above.Specifically, the coverage and clarity are two fundamental factors influencing quality and quantity of panoramic images for modeling.The Distance-Clarity function and the Distance-Buffer function are employed to make sure of the connectivity among the photographing panorama sites.Finally it is practical to obtain high quality panoramic images with less panorama sites and lower cost by means of an automatic location distribution system.

Distance Clarity Function
Divide the whole indoor area into accessible area (target) and unaccessible area (obstacle), and study the relationship between panorama quality and shooting distance.

Target-Oriented Distance Clarity Function
The Target-Oriented Distance Clarity function FC determines the optimal distance range between the panoramic site and the object by considering the clarity and distortion of images, depth of field and effective pixels.The clarity of panoramic images greatly affects the user experience.Under particular condition and application, we ought to ensure that the interested objects are recognizable even at the extremum of zooming.
We propose the following four factors to generate the relationship between the photographing distance and imaging effect from the aspect of collecting images.

Depth of field (DOF)
It can ensure the image clarity with DOF ranging from 0.5m to infinity when using fisheye lens.Nonetheless, shooting distance should be within DOF if fisheye lens is not available.

Resolution
The resolution of images should meet the requirement of identifying objects interested by users.The vast majority of the visual system in accordance with the ratio of the field of view and the number of CCD pixels to determine the resolution.Integrating the calculating of field of view, the equation of visual system resolution is determined as follows: Though users may be accustomed to the distortion, it is still adverse to view distorted images.Thus, we are supposed to minimize the distortion in the application of panorama modeling, we expect to minimize the distortion panoramic images and adopt the maximum distortion angle ratio relative to the real image as the distortion coefficient.Meantime, given that the distortion influence on panoramic image browsing greatly, parameter γ is used to adjust and exaggerate the weight of distortion.Specific process of Distance Buffer function is demonstrated as a fellows.
1. Area division Divide the whole test area into three categories, namely, inside interested area such as rooms, inaccessible area like the columns and accessible area available for panorama sites, which is outside the rooms such as aisles.Then rasterize the test area to adopt raster algebra method.
Figure 5. Original area (up) and the rasterized area (down)

Compute accessible buffer (AB)
Accessible buffer is generate with POI as buffer center, inside the buffer is the area with favourable and comprehensive view for shooting panorama images.The color saturation is used to illustrate the fitness value of buffer.As for AB, the fittest value marked with previous result of FC , describing the ability of being a panorama site.A favourable panorama site should have a relative high fitness value and cover a larger area.

Compute inaccessible buffer (IB)
Inaccessible buffer is generate with POO as buffer center, where observation would be impeded if panorama site locates.Obstacles are adverse for us to control the total number of panorama sites in test area and moreover, blocks weaken the experience for users browsing the panorama image.In other word, the IB is the area to avoid the panorama sites.The color saturation illustration of IB is similar to the AB.The value of IB describes the inadequacy of being a panorama site.A favourable panorama site should escape the impacts of IB as much as possible.
where FB = the final fitness matrix i = the AB layer or the IB layer (r,c) = the row and column location of points in layers

Selection of Panorama Site
With the final rank Distance Buffer function, we are able to evaluate the fitness of certain point to locate the panorama site.There threshold to assess whether the point rank is acceptable.

Locate seed point
Regard all the points in test area as a alternative point set CP 0 .Take the point with largest fitness value into the result point set CP , so as to start with the growth of result point.

Growth of the result point set
Considering the point with largest fitness value in the rest points of CP 0 , take the point with largest fitness value into the result point set CP likewise.Use raster algebra calculate the coverage of the points in CP and the average value of fitness matrix of the coverage area.To ensure that the coverage and clarity, a threshold is used to assess the point set CP .Besides, the coverage area of points in CP should be equivalent to the area of the whole test area.

Data
The data we used in the paper is collected by camera Nikon D800 with SIGMA fisheye lens in the panorama test field on April 14 th , 2017.In the field, mark the window of interest with red symbol.

Distance Clarity Experiment
Through the experiment, take general window as the POI and obtain the resolution, distance relationship and the actual image display as shown in the Equipments are SIGMA fisheye fixation lens and camera Nikon D800.CMOS target width 35.9mm,CCD pixel number 36350000, window size and general indoor construction window size similarity, comparable.Assume that the number of pixels of the screen device used by the user is the more common 1280 × 720.
Plot the result of experiment, the distance clarity curve is obtained as follows.At the maximum peak is the optimum distance clarity requirement where distance value in X axis is approximately 5.8 meters, in the range of 5 to 6 meters.Through the change of the distance, the influence degree of the obstacle on the panoramic image is obtained, and the curve is obtained by numerical fitting.
Notice that the result is derived from one single image with three independent experiments, so we conclude a distance range of 5 to 6 meters provided that the panoramic images are in the best viewing quality with favourable target recognition.The maximum value of distance is appropriately 5.8 meter.

Distance Buffer Function
Above all we stimulate a test data with the three categories area.
The stimulation result is shown as below.
Figure 11.Division of test area The circles represent the interested area, the square represents the inaccessible area while the rest area is accessible area available to locate panorama sites.The final buffer result superposed with the result of AB and the result of IB is shown as follows, where the highlight green points are the panorama sites of result.Considering that the Distance Clarity function is established on the basis of evaluation standard, it can be largely influenced by the changes of evaluation parameters.Besides, the researches on indoor panoramic modeling is not mature enough to carry out the assessment of our evaluation standard given that visual effect of panoramic image is large related to subjective view and the quantitative index need further examining.Thus, a common and widely acknowledged evaluating system of panoramic images is in need.This evaluation should be independent of the camera parameters as much as possible, and the one factor should be distance.
2. Different and complex indoor structure need testing We divide the test area into accessible area and inaccessible area, which is abstract from the real shopping mall building structure, because the shops usually distributes along the wall side and the whole area is spacious enough for viewing and shopping.But sometimes, the indoor structure could be more complex such as museums, there are couples of sightseeing routes crossing from each other.With more area classified, the Distance Buffer function should be modified to satisfy different circumstances.

Further mathematical validation is required
The connectivity of points in AB need further study because the algorithm in use now can not completely avoid the question.

CONCLUSION
This article propose a evaluation standard of indoor panoramic modeling, which is the basis of our research on the design of panorama site location.We take the connectivity, clarity and coverage into account to carry out a comprehensive determination of the arrangement of panorama site location.The experiment we take to obtain the Distance Clarity function prove effective to instruct the further research on Distance Buffer function.Under the guidance of this relationship, we set up the algorithm to ensure user with smooth browsing experience, where the panorama sites are as less as possible to reach a full coverage of model area.In this case, we exclude the impacts of POO and ensure the clarity view of POI.Therefore, with the final result, our algorithm prove able to model an area with a lower cost or lower consumption of time, which is much beneficial to the development and spread of indoor panoramic image modeling.
Here, we might as well to look forward to the future.In the approaching era, this technology is likely to exert a profound influence on the modeling of indoor environment.Firstly, with the development of robotics, human will be freed from the dull work of collecting image data.Instead, a kind of automated equipment is able to calculate the most appropriate locations of panorama points accurately.This progress will no doubt save us a lot time and money to a large extent.Then, this method can also be used to rebuild some indoor tourist attractions such as various museums so as to provide a vivid experience for those who have difficulty in coming in person.Furthermore, imagine a scene that we enjoy ourselves in the process of visiting malls or museums with the assistance of virtual-reality devices.Not only can we meet our demand like shopping or visiting but also gain a great deal of entertainment in the mean time.
dedicating to direct us on our study.Thank professor LI Jiansong for being our supervisor and directing us on the application of .Thank professor YAO Jian for providing us with facilities to collect panoramic images.We would also like to show our appreciation to professor LIU Jingbin for giving credit for our algorithm and reviewing the paper.

Figure 1 .
Figure 1.Explanation of DOF Figure 2. Resolution of visual system

Figure 3 .
Figure 3.The explanation of distortion

FBA
distance between certain point and POI FBA = a fitness matrix of the buffer center POI The function above is to compute the fitness value of accessible buffer of certain POI.The size of FBA is equivalent to the area of the buffer.The value of element in matrix FBA is calculated with D by FC .

Figure 6 .
Figure 6.The calculation of fitness value with POI D is the distance between certain point and POI, the fitness value of the certain point is calculated according to equation 4, related to the distance D.
Figure 7.The calculation of fitness value with POI

Figure 8 .
Figure 8.The superposition of AB layers and IB layer

FP
distance between point i and POI FP = the fitness matrix of the point set CP fP i = the element of fitness matrix T = the threshold fitness value ave = calculation to average 3. Generate the result point set When the coverage of point set CP is equivalent to the test area, and every element satisfy the threshold, the generation of result point set CP 3. EXPERIMENT

Figure 9 .FC
Figure 9. Curve of Distance Clarity function

Figure 12 .
Figure 12.The accessible buffer of POI (left), inaccessible buffer of one POO (right)

Figure 13 .
Figure 13.Final result of panorama sites distribution

Table 1 .
table (select only four values).More experiment data available in APPENDIX.Experiment data of Distance Clarity

Table 3 .
Research is supported by National Training Program of Innovation and Entrepreneurship for Undergraduates.Experiment Results of FC

Table 5 .
Experiment Results of Fe

Table 6 .
Experiment Results of