ANALYZING A BLOCK OF HRSC IMAGE STRIPS FOR A SIMULTANEOUS BUNDLE ADJUSTMENT

10 years ago the first European interplanetary mission Mars Express was launched and sent into orbit around Mars. One of the scientific instruments on board the orbiter is the High Resolution Stereo Camera (HRSC). This multi-line sensor with five panchromatic and four multispectral CCD lines was developed by the German Aerospace Center (DLR) for photogrammetric mapping purposes. It images the Martian surface with a resolution of up to 12m per pixel, depending on the altitude. The along-track stereo capability of the camera delivers image strips with three-dimensional information, which cover nearly the whole planet. For a derivation of more accurate digital terrain models and orthoimages the orientation data of the camera is improved via bundle adjustment. To map larger regions overlapping image strips can be used to form photogrammetric blocks, thus allowing a simultaneous adjustment of the different strips. Compared to the adjustment of individual strips, an adjustment of the entire block reduces not only local, but also regional inconsistencies in the data. With the growing number of HRSC image strips in this ongoing mission, number, size and complexity of potential blocks increases. To cope with these data a method for a semi-automated analysis, selection and combination of suitable strips for the design of more accurate and reliable blocks has been developed. The method takes the inhomogeneity of the HRSC data into account by adapting the processing parameters, if necessary for each strip.


INTRODUCTION
In the past 50 years many space probes have been launched into space for the exploration of our neighboring planet Mars.Orbiters as well as landers and rovers on the Martian surface were involved in these investigations.
More than 10 years ago, on December 25, 2003, a European mission reached the Martian orbit -Mars Express.It is the first interplanetary mission designed, built and managed by the European Space Agency (ESA), and is still delivering data.On board the orbtier is the High Resolutioon Stereo Camera (HRSC).This multi-line sensor with five panchromatic and four multispectral CCD lines was developed by the German Aerospace Center (DLR).It is the first camera in planetary research especially designed for photogrammetric mapping purposes (Neukum et al., 2004), (Albertz et al., 2005), (Jaumann et al., 2007).
The HRSC images the Martian surface with a resolution of up to 12m per pixel.The along-track stereo capability of the camera delivers image strips with three-dimensional information, which covers nearly the whole planet.Every single strip allows the generation of high quality orthoimages and digital terrain models (Heipke et al., 2007), (Gwinner et al., 2010).These HRSC data products are regularly released to NASAs Planetary Data System (PDS) and ESAs Planetary Science Archive (PSA) and are used by a large number of scientist i.a. for geological analysis.
Because HRSC imagery is globally available with a high coverage and simultaneously has a considerably high resolution it can additionally be used as reference data for other missions.The coregistration to HRSC orthoimages and the orthorectification with HRSC digital terrain models (DTMs) are important steps in the * Corresponding author processing of e.g.CTX or HiRISE images.Also, future missions can benefit from this valuable data source, i.e. when investigating potential new landing sites.DTMs generated from HRSC data already have contributed to the landing site selection for the Mars Science Laboratory "Curiosity" (Golombek et al., 2012).
For a derivation of more accurate orthoimages and DTMs the exterior orientation (position and attitude) of the camera as a function of time is systematically improved via bundle adjustment.This processing step uses a large number of tie points automatically generated from the panchromatic images.
The HRSC image data received in the almost 10 years from January 2004 until July 2013 have been systemically adjusted.During this time the Mars Express mission has orbited the planet 12,162 times.In 3,047 of these orbits the camera took images of the Martian surface suitable for stereo processing.In 2,626 of this strips the exterior orientation data was significantly improved by the bundle adjustment.
In many cases the area of interest is larger than the coverage of one single strip, but can be covered with multiple strips.Therefore, it is advantageous to combine these strips in order to obtain homogenous DTMs and orthoimage mosaics.For an accurate fitting of neighboring strips a photogrammetric block has to be defined by adding tie points between the neighboring strips, allowing a simultaneous adjustment.Compared to the adjustment of individual strips, an adjustment of the entire block reduces not only local, but also regional inconsistencies in the data.
The method takes the inhomogeneity of the HRSC data into account by allowing to adapt the processing parameters, if necessary for each strip.
The following chapter describes the method proposed for the selection and combination of the HRSC image strips.Besides it gives an overview of essential processing steps to achieve consistent orientation data for a regional block.In the third chapter the structure of a block consisting of 32 HRSC image strips and the obtained bundle adjustment results are presented.The last chapter concludes this paper and summarizes future work.

METHOD
For the derivation of DTM and orthoimage mosaics, with strips fitting seamlessly, the orientation data have to be adjusted as a block.For efficiency the adjustment in subdivided into two steps.First the attitude, described by the angles of exterior orientation is adjusted.In the second step the attitude is treated as constant and the relative position between the strips and the absolute position with respect to the reference frame (the Mars Orbiter Laser Altimeter (MOLA) DTM is used here) is adjusted.

Selection of image strips
When selecting a set of HRSC image strips for a block, a number of criteria has to be taken into account.The initial criterion is obviously that the target area has to be visible in the strips.Normally this area is described by a rectangle covering parts of the Martian surface.Position and extent of this area are defined either by scientific interest or using cartographic reasoning, e. g. tiles of the Mars chart systems MC 30 or MC 140 (Dumke et al., 2010).
Figure 1: The Martian surface devided into 30 or 140 quadrangles (Dumke et al., 2010) Furthermore it is reasonable to select strips based on their: • completeness of available panchromatic channels Depending on the number of available image strips in the target area these criteria should be more or less strict, so that a sufficient number of image strips remains.
From the remaining set all available images from the panchromatic channels (called ND, S1, S2, P1, P2, respectively) are processed with an approach for systematic bundle adjustment of single orbit HRSC image strips (Schmidt et al., 2008), (Bostelmann et al., 2012).Figure 2 outlines the four steps of this approach.The result of the pre-processing step are pre-rectified images with the same scale for each image within a strip.
Tie point matching: Defining the image of the nadir channel as master, tie points are selected and matched to all other images of the strip (Heipke et al., 2004).In the master image the feature points are arranged as a grid to ensure homogenous distribution.
The density of the grid, and so the maximum number of tie points is defined by default values for each strip.If neccassary values differing from the default value are used for individual strips.
Combined bundle adjustment: Using the image coordinates of the tie points, the nominal exterior orientation values and the MOLA DTM, a combined bundle adjustment is carried out in the next step (Spiegel, 2007a).
The used bundle adjustment method is based on the well known approach used in photogrammetry, by which the exterior orientation is simultaneously determined for all images.In case of HRSC the images are not acquired from individual view points but in a continuous motion.Thus, the exterior orientation must be modeled along the spacecrafts trajectory as a function of time.Three decades ago the concept of orientation points was proposed to solve this problem (Hofmann et al., 1982).Today, it is a common approach in the processing of multi-line sensor data.
To fit the photogrammetrically determined 3D coordinates of a photogrammetric bundle block to a regional or global reference system, an adequate number of ground control points is normally used in aerial or spaceborne photogrammetry.For areas without ground control points available DTMs can be used to obtain an absolute fit.On Mars the MOLA DTM provides a global reference.Therefore, a combined bundle adjustment for HRSC image data and the MOLA DTM as control information was developed, implemented and tested (Spiegel, 2007b).
The general approach of the bundle adjustment is a nonlinear least-squares adjustment.This optimization aims to find the best set of unknown model parameters to explain the observations.For a combined adjustment of HRSC images and the MOLA DTM there are four types of observations: image coordinates, orientation parameters, unknown bias and drift parameters to compensate systematic effects and DTM information.The four types of observation equations used in the bundle adjustment are described in (Bostelmann and Heipke, 2011).
Evaluation: For the evaluation of the adjustment results the nominal and the adjusted orientation data is compared.Because of the large amount of data, a single value describing the geometric quality is easier to handle for systematic analysis of the whole data set.The mean forward ray intersection error summarizes the accuracy of all object points and is a good measure for the internal consistency of the data (Gwinner et al., 2009).The evaluation step calculates the forward intersection error for all object points twice.Both times it uses the same set of tie point coordinates and only switches between nominal and adjusted orientation data, so that the resulting mean intersection error reveals the gain in quality.
Additionally, both the nominal and the adjusted orientation information is used to generate dense 3D point clouds as used for DTM generation (Gwinner et al., 2010).The intersection error for all these points is used to plot two color coded error maps.This allows a visual inspection of the count, the spatial distribution and accuracy of these points.Hence a visual detection of systematic and random errors in the data is possible.Strips where the matching did not produce enough tie points or the bundle adjustment did not reduce the mean intersection error in a satisfactory way are reprocessed with adapted processing parameters.In case still no improvement can be reached, these strips are discarded from block processing.

Combination of strips into a Block
The combined bundle adjustment with the MOLA DTM as reference also improves the orientation of the strips relative to each other.However, adjusting strips independently does not take advantage of overlap information.With tie points in these overlapping areas, common to more than one strip, neighboring strips can be adjusted together.To analyze the quality of each overlap, two overlapping strips are considered as a so called sub-block.
The maximum number of sub-blocks n sb per block depends on the number of strips ns: De facto the number of sub-blocks is much lower because most of the strips overlap only with a handful of others strips.All potential sub-blocks are determined by an automatic spatial selection.Figure 3 illustrates the concept of sub-blocks using an example of three HRSC image strips (h8601 0000, ha720 0000 and ha727 0000).In this block every strip overlaps with both of the others, so that three sub-blocks are possible.
Before the processing of these sub-blocks is started a pre-selection is done based on the: • size of the overlap relative to the size of the strips • scale similarity of the two strips • illumination angle similarity of the surface at time of imaging.

Block
Sub-block A Sub-block B Sub-block C Figure 3: Concept of sub-blocks When a suitable set of sub-blocks is found, an adaptation of the workflow presented for single orbit strips is used to analyze the stability of each sub-block.The pre-rectification is done with the mean resolution of both strips of the sub-block.This allows better matching results at large scale differences between the strips.
For the tie point matching one of the two nadir channel images is matched with the four other channels (S1, S2, P1, P2) as well as with the 5 other channels (ND, S1, S2, P1, P2) of the second strip.
In this way every tie point in the overlapping area can be found in up to 10 images.Because the number of unknown parameters in the block adjustment (see next paragraph) is quite small, the necessary number of tie points is much smaller than for the single strip adjustment.
In the subsequently bundle adjustment all sub-blocks are adjusted separately.Using the adjusted orientation of the single strips adjustment step only an offset for the position parameters of the exterior orientation for each strip is considered as unknown; the trajectory shape and the orientation angles are treated as constant.
For the evaluation of the sub-blocks only tie points found in both strips are considered, because their number and accuracy reveal how successful the matching respectively the adjustment was more clearly than the usage of all tie points of the sub-block would do.Based on this information a second selection of subblocks is done.Sub-blocks where the number of tie points is too small are rejected, and so are sub-blocks in which the mean intersection error of the tie points increases after the adjustment.The footprint of the remaining sub-blocks are then visualized.By coloring the overlapping areas with according to their MIE, an human operator is able to analyze the stability of the block.
For the final simultaneous adjustment of the entire block the tie points of all sub-blocks are used.Again the adjustment uses the orientation data resulting from the single strips adjustment as input.The result are parameters for the exterior orientation of each image strip.
For an evaluation of the block adjustment results the mean intersection error is calculated for the tie points of every overlap in the block three times; first with the nominal orientation, second with the single strips adjusted orientation and third with the block adjusted orientation data (see Figure 4).This reveals how the results improve in each step.

EXPERIMENT AND RESULTS
The described method has been tested on several blocks.A block covering a rectangular area located in the region of Elysium Mons is depicted below.It consists of 32 HRSC image strips.This area has been selected because of the coverage with image strips and the nadir image resolution of 20m or better for nearly the whole area.
Figure 5: Nadir image footprints of a Block of 32 HRSC strips

Used Data
The images have been taken in a period of almost six years.The oldest strip, h0032 0000 was taken in January 2004 and the newest h7499 in November 2009.Beside the five panchromatic channels for all but five strips also color images are available.
For this block, the nadir resolution lies between 12.5m and 30.4m. 3 strips have a resolution coarser than 20m and 14 coarser than 15m.The mean value for the image resolution is 15.8m.The length of the image strips is 65,084 image lines on average, with the shortest strip at 20,200 and the longest at 131,032 image lines.Although these characteristics looks quite inhomogeneous, compared to other sets of overlapping strips they display a higher homogeneity.

Single strip processing
According to the good quality of the data in the selected area, all image strips fulfill the selection criteria and were used for the block; the single strip processing work flow for bundle adjustment succeeded for all 32 image strips using the default processing parameters.
A grid of 10, 000 potential feature points for the matching step resulted in 1,092 -7,372 tie points per strip.In images with good matching conditions the number of successfully matched tie points was higher, whereas clouds or sandstorms visible in some of the strips obviously reduced the number of tie points.
The average number was 4,084.This number of tie points used here is quite small.In previous experiments a number of 50,000 potential tie points per strip was used.Obviously, more tie points allow using a more flexible trajectory model with a larger number of unknowns, but tests had shown that in this test case more tie points did not improve the results.Thus, the smaller number of tie points was selected here in order to keep the computational time under control.
Figure 6 shows the footprints of all nadir images.The colors correspond to the mean intersection error (MIE) of the tie points for this strip.On the left site, the nominal, and on the right site the adjusted orientation data was used.
before bundle adjustment after bundle adjustment Averaging over all strips of this block, the bundle adjustment improved the MIE from 22.5m to 15.5m.

Block Adjustment and Results
Following the strategy described in chapter 2 the next step is the processing of sub-blocks.For this block of 32 strips the number of sub-blocks is 76.
After removing 8 sub-blocks with an overlap smaller than 1% of the strip area 68 sub-blocks remained.For this set of sub-blocks the described processing workflow was used.A grid of 4,000 feature points for each sub-block resulted in a number of successfully matched tie points in the range of 432 -3,092.Test with finer grids and thus a larger number of tie points (10.000, 60,000 or 90,000) per sub-block did not produce significantly different results, but, not surprisingly, vastly increased the computation time for the adjustment.
18 sub-blocks with less than 10 tie points connecting both strips were then removed.The remaining sub-blocks showed an average number of 166 points in the overlap.In only one case (subblock h5075-h5093) the sub-block adjustment did not succeeded.
Finally the tie points of 49 sub-blocks were selected for the simultaneous bundle adjustment of the entire block.
Because of the comparably small number of tie points used for the sub-blocks it took only 24 minutes for all adjustment steps in total.The resulting parameters for the exterior orientation were evaluated as described at the end of chapter 2. Figure 7 shows the results of the evaluation.On the left the single strip adjusted orientation data was used for the calculation of the mean intersection error, on the right the block adjusted data.On average over all 49 overlapping areas the accuracy of the object points was improved from 52,0m (nominal orientation) respectively 46.9m (strips adjusted) to 16.6m.
The classification of the overlapping areas according to the mean intersection error is shown in  The bundle adjustment of HRSC image strips as a photogrammetric block is an important processing step for the derivation of high quality DTM and orhtoimage mosaics.It reliably reduces not only local but also regional inconsistencies in the orientation data.One important part of the bundle adjustment is the proper strip selection, because a single image strip or sub-block with poor data can inhibit the success of the adjustment or decrease the quality of the results.
The method presented in this paper allows to analyze the block, reveal weak spots and optimize the design for better adjustment results.This finally results in more accurate HRSC data products.The example block of 32 image strips used for this work, is an area with good data (high contrast images with good matching results and comparatively accurate nominal orientation data).Hence for all processing steps default parameters could be used, and no manual adaptation of the parameters for particular strips was needed.
One of the next steps is the systematic processing of regional Blocks in MC-30 Quadrangles (Dumke et al., 2010).In this case the HRSC strips will be selected by cartographic criteria.This will complicate the whole process and will make an extensive analyzis of the blocks necessary.Furthermore an investigation of blocks at the poles, where the strips are crossing and no MOLA DTM is available, is planned.Also blocks of strips forming a ring around the equator will be investigated.For these blocks gaps in the HRSC data have to be filled with images from other missions.

Figure 2 :
Figure 2: Workflow: Bundle adjustment of HRSC data Image pre-processing: After the reduction of image noise by a low pass filter, the images are pre-rectified using the nominal orientation data provided by the European Space Operations Centre (ESOC), and the MOLA DTM.The resolution r of the prerectified images is computed with the help of the resolutions r1 and r2 of the two stereo channels S1 and S2:

Figure 4 :
Figure 4: Evaluation of the bundle adjustment results

Figure 6 :
Figure 6: Mean intersection error of tie points per strip

Figure 7 :
Figure 7: Mean intersection error of tie points per sub-block in overlap 4 CONCLUSION AND FUTURE WORK

Table 1 :
Table 1 classifies the strips according to the achieved mean intersection error befor and after ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume II-4, 2014 ISPRS Technical Commission IV Symposium, 14 -16 May 2014, Suzhou, ChinaThis contribution has been peer-reviewed.The double-blind peer-review was conducted on the basis of the full paper.doi:10.5194/isprsannals-II-4-15-2014adjustment in to five groups.The number of image strips with the highest accuracy (MIE <10m) increased from 1 to 13, while the number of strips with lower accuracy (MIE >25m) decreased from 9 to 3. Based on these evaluation results, all strips were accepted for the next processing step.Evaluation of the single strip bundle adjustment results

Table 2 :
Evaluation of the bundle adjustment (ba) after BA as single strips after BA as block