PHOTOGRAMMETRIC PROCESSING OF OSIRIS-REX IMAGES OF ASTEROID (101955) BENNU

The principal objective of the Origins, Spectral Interpretation, Resource Identification, and Security–Regolith Explorer (OSIRIS-REx) mission is to retrieve a sample of the asteroid (101955) Bennu and return it to Earth. OSIRIS-REx arrived at Bennu in December 2018. Images of the asteroid by the OSIRIS-REx Camera Suite (OCAMS) were photogrammetrically controlled to produce a global basemap and site-specific image mosaics essential to the selection of a primary and backup sample site, which were announced in December 2019. In the control process, OCAMS images were registered to shape models created from OSIRIS-REx Laser Altimeter (OLA) data and from the process of stereophotoclinometry. This paper summarizes the photogrammetric control to date of images collected at Bennu. We briefly review the mission and the OCAMS imaging sensors. We then describe the photogrammetric control process for the global mapping campaign and targeted reconnaissance surveys of candidate sample sites. Finally, we discuss ongoing and future work.


INTRODUCTION
Following a journey of more than 2 years, NASA's Origins, Spectral Interpretation, Resource Identification, and Security-Regolith Explorer (OSIRIS-REx) spacecraft arrived at the near-Earth asteroid (101955) Bennu in December of 2018. Since then, the spacecraft and OSIRIS-REx team have been mapping Bennu and characterizing its geological, chemical, and physical properties in support of the primary mission objective to acquire a sample for return to Earth.
Characterized as a 'rubble-pile' asteroid DellaGiustina et al., 2019), Bennu has a generally spheroidal shape resembling a spinning top ( Figure 1). Equatorial radii are 282.37 m x 268.05 m and the polar radius is 249.25 m. Bennu's coordinate system, rotation model, and prime meridian are described in . OSIRIS-REx will offer further insight as to how primitive carbonaceous asteroids such as Bennu may have contributed to the beginnings of life on Earth and their role in planet formation .
To safely retrieve a sample from Bennu, OSIRIS-REx must precisely navigate its way, in a microgravity environment, to a rough surface dominated by potential hazards such as craters, ridges, and large boulders . Success depends upon having accurately characterized the surface beforehand. To this end, OSIRIS-REx Camera Suite (OCAMS; Rizk et al., 2018) images acquired within the Detailed Survey and Reconnaissance portions of the mission (Figure 2) have been photogrammetrically controlled to generate panchromatic and color image basemaps at global and local scales. Control was accomplished with a modified version of the Integrated Software for Imagers and Spectrometers (ISIS) planetary cartography package developed and maintained by the OSIRIS-REx team Sides et al., 2017).
These image products have informed the process of selecting a sample site with regard to science value, the availability of sampleable material (particles smaller than 2 cm that can be   Lauretta et al., 2017). ingested by the OSIRIS-REx sampling mechanism; Bierhaus et al., 2018), and the presence of surface features that might prevent the spacecraft from approaching safely and retrieving a sample. They have enabled the iterative reduction of approximately 50 original candidate sites to 32,16,8,4, and finally 2-a primary and a backup. The primary and backup sample sites, designated Nightingale and Osprey, respectively, were announced in December 2019.
In the following we describe the OCAMS imaging sensors and associated lens distortion models; the photogrammetric control of OCAMS data acquired in the Detailed Survey and Reconnaissance A phases of the OSIRIS-REx mission; and summarize ongoing and future work. Figure 3; Table 1), consists of three 1-megapixel imaging sensors (SamCam, PolyCam, and MapCam) that perform a variety of functions throughout the mission. The wideangle SamCam records and validates the sampling procedure, whereas image-based mapping of Bennu is accomplished primarily with MapCam and PolyCam. PolyCam is a narrowangle, high-resolution, panchromatic sensor. With variable focus (94 focus positions), it can image Bennu both as a point source from a distance of several million kilometers and at subcentimeter resolution from a distance of ~200 m. The medium angle MapCam sensor acquires panchromatic data with a broadband filter and color data with four narrow-band filters. With a field of view five times that of PolyCam, MapCam images serve as context for PolyCam images acquired within the same observation. Golish et al. (2020a) describe OCAMS pre-and in-flight calibration procedures. Ground calibrations were conducted for all three OCAMS cameras to characterize bias level, dark current rates, flat field response, stray light, and radiometric performance. MapCam interior orientation including lens distortion was determined for each filter with star field images acquired in flight. PolyCam distortion was established pre-flight at 200 m and with star field images for longer ranges. SamCam lens distortion is yet to be determined.

MapCam Distortion Model
The MapCam distortion model (Pelgrift et al., 2018) includes radial symmetric distortion that displaces image points radially from the principal point and decentering distortion (Brown, 1966) which is caused by errors in the alignment of lens components and produces both a radial and tangential displacement. An additional parameter models the temperature dependence of the focal length. Temperature dependence was determined by performing star calibrations at a range of temperatures that were anticipated during operation.

Radial distortion is given by
where is the total radial displacement, the + are coefficients of radial distortion, and r is the radial distance from the principal point such that where ( ̅ , 0) are the coordinates of an undistorted image point reduced to the principal point.
Resolution of the total radial displacement into its x and y components yields The x and y components of decentering distortion are given by where the + are the coefficients of decentering distortion.  Distorted image coordinates ( ′, 9 ) are then 9 = ̅ + 1 + 3 9 = 0 + 1 + 3 (5) Distorted image coordinates are converted to sample and line pixels ( , ) with = (1 + 1 ) 9 + = (1 + 1 ) 9 + where % models the temperature dependency of the focal length, fx, fy represent the focal length in pixels at 0º C in the image plane x and y directions, cx, cy are the pixel coordinates of the principal point, and T is the temperature in degrees Celsius.

PolyCam Distortion Model
Principal point and radial distortion parameters in the PolyCam model vary as a function of focus position. Modelled differently from MapCam (Golish et al., 2020a), radial distortion is given as Here again, is the total radial displacement and r is the radial distance from the principal point. Radial distortion coefficients are given as + to distinguish them from the MapCam radial distortion model. PolyCam focal length, principal point, and radial distortion parameters were established at 94 focus positions. Focal length and the % coefficient are completely correlated. To enable solving for focal length, the % coefficient was fixed to 0.0 in the PolyCam calibrations.

IMAGE PLANNING AND PRE-PROCESSING
Science observation planning for OSIRIS-REx (Polit et al., 2020) is accomplished primarily with the J-Asteroid geospatial information system developed by the Arizona State University Mars Space Flight Facility (Christensen et al., 2018;Figure 4). Planning for image acquisition considers, for example, pixel and footprint size; emission, phase, and incidence angles; local solar hour; target coverage; and image overlap.
Prior to photogrammetric control, images are pre-processed (Golish et al., 2020a) to remove bias, dark counts, bad pixels, and charge smear. Flat-field correction is applied and images are converted into units of reflectance (I/F). Images are then converted into ISIS cube format and additional pre-processing is performed with ISIS. In the microgravity environment of Bennu, optimal spacecraft stability can be attained only in a terminator orbit owing to the influence of solar radiation pressure. However, image quality is highly variable from a terminator orbit. Nadirlooking images in the vicinity of the terminator suffer from poor illumination. Oblique images may be better illuminated but will exhibit foreshortening, reduced resolution, and poor focus for terrain beyond the camera's depth of field. Hyperbolic flybys offer better imaging geometry . Below, following closely Bennett et al. (2020), we summarize the photogrammetric control of OCAMs PolyCam images to produce a global panchromatic basemap of Bennu. Efforts to control MapCam color images are ongoing.

Image Acquisition
PolyCam images controlled for the panchromatic basemap were obtained primarily from Flybys 3, 4A, and 4B. In each flyby, the spacecraft was ~3.5 km from Bennu and maintained a constant sub-spacecraft latitude. As the spacecraft passed over the surface, it slewed alternately north and south acquiring images in a serpentine fashion. The average pixel scale was ~5 cm and image overlaps ranged from 38% to 49% in the north-south direction and from 30% to 50% east-west ( Figure 4; Table 2). A small : Example of science observation planning for the global panchromatic basemap with the J-Asteroid geospatial information system (Christensen et al., 2018). Shown are the first two image scans for Flyby 3 in the Detailed Survey: Baseball Diamond subphase of the mission. Scan 1 (in blue) was acquired as the spacecraft slewed from south to north. Scan 2 (in red) follows immediately in the opposite direction. Ninety-two scans were acquired for Flyby 3 to achieve global coverage.
number of images from Flybys 5A and 6A were included to cover regions at high latitudes insufficiently illuminated in Flybys 3, 4A, and 4B.

Image Measurement
Tie point image measurements are acquired in ISIS by automatic interest point detection and image matching with the findfeatures application. The pointreg application was then used to refine these measurements to sub-pixel accuracy . Although outlier rejection in the matching process was highly successful, some manual editing was required to correct mismatches occurring between images with notably different geometry and illumination.
Ground control points (GCPs) were obtained from 32 local digital terrain models (LDTMs) corresponding to candidate sample sites and the location of the prime meridian. These were generated from OLA data at a resolution of ~15 cm. The LDTMs are registered to a global DTM (GDTM) generated via stereophotoclinometry at a resolution of ~80 cm Gaskell et al., 2008). The accuracy of the GDTM was estimated to be 0.52 m horizontally and 0.64 m radially. The LDTMs were utilized for GCP measurement as they were created at resolutions closer to that of the OCAMS images. GCPs were measured manually from terrain-shaded relief (TSR) maps that were created for each LDTM to facilitate visual correlation between surface features in the images and the LDTM. Because the LDTMs were only available at candidate sample sites and were not uniformly distributed over Bennu, the resulting GCP distribution was less than optimal, particularly in the southern hemisphere.

Bundle Adjustment
The final control network consisted of 3142 images, 792,217 tie points, 127 GCPs, and 3,257,452 image measurements. Bundle adjustments were performed with the ISIS program jigsaw (Edmundson et al., 2012). Initial image exterior orientation parameters were obtained from reconstructed NAIF spacecraft position and camera pointing kernels (Acton, 1996). Initial coordinates for tie points were determined by first projecting all image measurements through the sensor model to obtain the intersection with the GDTM. For each tie point, all measurement intersection points were averaged to obtain coordinates for input to the bundle adjustment.
After adjustment, the root mean square (rms) of all image measurement residuals was ~0.082 pixels. The rms of image measure residuals for GCPs only was ~1.0 pixels, equating to ~5 cm or one-third of the average resolution of the LDTMs. To estimate the spatial accuracy of the solution, we performed a separate bundle adjustment in which 11 GCPs were constrained and the remaining 116 were treated as checkpoints. The rms of checkpoint coordinate discrepancies was 23.3, 19.2, and 30.7 cm in latitude, longitude and radius respectively. To assess the fit between the adjusted point cloud and the GDTM, we compared the adjusted radius coordinate of each point to the radius at the corresponding location in the GDTM. The mean absolute error of these differences was ~47.0 cm. Approximately 90% of points lie within 1 m of the GDTM. The rest deviate from the surface as much as ~22.0 m with the highest differences around large boulders and on crater floors .

Creating the Basemap
The final basemap (Figures 5, 6) was produced with 2155 of the 3142 images in the original control network. Separate mosaics were first created for each of Flybys 3, 4A, and 4B. For each flyby mosaic, images were added in the order of acquisition, thus the more recent images were on top. The flyby mosaics were then combined to produce the global basemap. Where the flybys overlap, precedence was generally given to regions with the lowest overall emission angle while at the same time attempting to minimize abrupt seams. Images from Flybys 5A and 6A were utilized to fill areas of Flybys 4A and 4B that were completely shadowed. The basemap was photometrically normalized using a Minnaert phase angle correction (Golish et al., 2020b) Figure 5. Global panchromatic basemap of Bennu created from 2155 OCAMS PolyCam images acquired at a pixel scale of ~5 cm . Projection is equirectangular.

SURVEYS OF CANDIDATE SAMPLE SITES
The final four candidate sample sites (Nightingale, Osprey, Kingfisher, and Sandpiper) were announced in August 2019. In October, these were further evaluated for safety and sampleability during the Reconnaissance A phase of the mission . Approximately 50 PolyCam images were acquired for each site and surrounding region with an average pixel scale of ~1.4 cm (Table 3). In each case the spacecraft was ~1.015 km above the site. Image acquisition was accomplished in a manner similar to that for the global basemap. The linear camera station geometry for the Sandpiper site shown in Figure  7 was typical for all four sites. A small number of images per site (three to six) were quickly photogrammetrically controlled and mosaiced to assess the availability of sampleable material and the presence of potentially hazardous features.
Tie point image measurements were again made by automatic interest point detection and image matching followed by subpixel refinement. GCPs were obtained from LDTMs produced from OLA data at a resolution of ~5 cm. Images are registered to a GDTM with a resolution of ~80 cm.
An alternative to the use of TSR maps to facilitate GCP measurement was taken here. Utilizing the LDTM, the image exterior orientation, and a photometric model of the surface, a simulated image with backplanes containing spatial information was created for each PolyCam image (Figure 8; Golish et al., 2017;Becker et al., 2020). GCP measurements were then obtained by visual correlation between a PolyCam image and its corresponding simulated image. The TSR and simulated image approaches are similar and equally accurate in terms of feature location. However, it has proven to be easier to identify features in the simulated images as they are notably more realistic in appearance due to the accurate representation of the light source and the presence of shadows. Figure 6. Global basemap north and south views in polar stereographic projection . Latitude ranges from 0° to 90°N (left) and 0° to 90°S (right). Latitude grid is in 10° increments, starting from 0° at the outermost mark.   Bundle adjustment of these sites was complicated by the lack of diversity in camera station geometry (e.g. Figure 7) and less than optimal GCP distribution. High correlations between spacecraft position and pointing parameters for Nightingale, Osprey, and Kingfisher required suppression of one or the other in the solution to avoid ill-conditioning in the normal equations. We solved for pointing only for Nightingale; position only for Osprey and Kingfisher; and both position and pointing for Sandpiper.
Bundle adjustment statistics for the four sites are given in Table  4 and the resulting image mosaics with corresponding control networks are shown in Figure 9. Visual examination of the mosaics indicated that image-to-image registration is within one pixel. Checkpoint analyses gave accuracies in latitude and longitude of 2.1 and 5.7 cm for the Nightingale site; 4.3 and 4.5 cm for Osprey; and 11.5 and 10.5 cm for Kingfisher. As checkpoints were not available for the Sandpiper site, assessment was made by visual comparison between the updated images after adjustment and their corresponding simulated images. Here the accuracy was estimated to be ~1.5 cm in latitude and longitude.

ONGOING AND FUTURE WORK
OCAMS MapCam images also acquired in the Detailed Survey phase have been controlled to produce color global basemaps at ~24 cm/pixel. These maps provide a multispectral description of Bennu's surface at spatial scales of ~25 cm/pixel. Moving forward, we aim to globally control MapCam multispectral data to the higher-resolution PolyCam images in an effort to merge the color signatures on the asteroid with the more detailed surface morphology revealed by the monochrome PolyCam controlled mosaics.
Additionally, several localized site-specific observations have been acquired by both MapCam and PolyCam. Merging the sitespecific and globally controlled image networks is of high interest for the purposes of future geological mapping efforts and to gain context for the fine-scale processes that may influence larger-scale surface features on asteroid Bennu.
OCAMS images are currently available via the Planetary Data System (PDS; https://sbn.psi.edu/pds/resource/orex/). Higherlevel products (e.g. mosaics and shape models) will be available in the PDS one year after OSIRIS-REx departs Bennu.