CALIBRATION OF A LIGHT HEMISPHERICAL RADIANCE FIELD IMAGING SYSTEM

: A light hemispherical radiance field imaging system based on fish-eye camera was developed for the measurement of the surface incident radiance in an urban environment, which is often affected by radiometric heterogeneity problems. A linear radiometric model and a polynomial fish-eye lens model are used. A temperature-dependent dark level model is proposed to improve the dark correction for high dynamic range photography. This paper describes the calibration procedure for spectral and geometrical radiance field measurements and presents the results of the calibration. The spectral radiometric calibration error is 2.07%, 1.34%, and 0.98% for blue, green, and red bands, respectively. The mean geometrical calibration error is 2.037 pixels.


INTRODUCTION
The surface irradiance is one of the key parameters for retrieving the surface reflectance from remote sensed images. In the literature, common methods of surface reflectance retrieval are focused on the correction of atmospheric effects (Kaufman, 1984) (Moran et al., 1992) (Liang et al., 2001). Some algorithms take into account the impact of ground geometric effects (Richter, 1998) (Riano et al., 2003). For remote sensing of urban areas using high spatial resolution imagery, one major challenge of surface reflectance retrieval is the heterogeneity of the irradiance at the surface level. The presented ground objects such as buildings and trees can hide the direct solar irradiance by creating cast shadows. Shaded surface irradiance is highly sensitive to multiple reflections from surrounding surface elements (Lei et al., 2021). Only a few atmospheric correction codes are adapted for urban radiometric heterogeneity problems (Lachérade et al., 2008) (Adeline et al., 2018). In this context, it would be interesting to perform in-situ measurements in an urban environment.The measurements of the hemispherical field of the incident radiance provide more information than irradiance. These measurements make it possible to identify the contribution of each component in the irradiance to better understand the radiometric variability in the heterogeneous environment. The in-situ measurement database should include various scenes to adapt the urban environment complexity. It would be preferable for the instrument to be light, autonomous and compact to embed in a vehicle.
A digital camera is essentially an imaging radiometer. Panoramic photography can create an image that contains incident radiation from all directions. A camera with a 180 • fish-eye lens can capture the incident radiance hemisphere into a circular image. Many hemispherical imagers have been developed based on this principle for various studies. They include, but are not limited to, the canopy gap fraction estimation (Lang et al., 2010), the solar power forecasting (Urquhart et al., 2015), the sky luminance distribution (Tohsing et al., 2013), the sky spectral radiance distribution , and the in-water spectral upwelling radiance distribution (Voss and * Corresponding author Chapin, 2005). All of the mentioned works applied geometric calibration to calibrate the projection function and distortion of the fish-eye lens. Most of them performed a dark correction by subtracting a dark image. (Lang et al., 2010) performed a vignetting calibration in a radiometry laboratory. (Tohsing et al., 2013) performed an absolute radiometric calibration of luminance, but the vignetting effect was not considered. ) performed a roll-off correction of fish-eye lens with a radial function and an absolute radiometric calibration of radiance.
Our goal is to develop a light hemispherical radiance field imaging system (HRI) based on a fish-eye camera, to measure the incident radiance field of different surfaces in the environment. Since the incident radiance varies strongly outdoors, it is possible to use high dynamic range photography technology (Debevec and Malik, 1997) to expand the radiometric dynamic range of the camera. In this case, the radiometric linearity of the camera should be carefully checked. To ensure the lightness, the imager is not equipped with a cooling system like (Urquhart et al., 2015), while the sensor temperature may vary during permanent acquisition. As the dark level of the camera is proportional to the exposure time and is highly dependent on the temperature (Healey and Kondepudy, 1994), we propose a dark level model as a function of the sensor temperature and exposure time to improve the dark noise correction. This paper describes the radiometric and geometric calibration procedures of the HRI. A description of the HRI system is presented in section 2. The sensor models, including noise model, radiometric model, and geometric model are described in section 3. Details of the calibration implementation and performance evaluation are presented in sections 4, 5, and 6.

Camera
The camera CamLight (Martin et al., 2017) is a light camera developed by the french National Institute of Geographic and Forest Information (IGN). This camera was originally designed for Unmanned Aerial Vehicle (UAV) photogrammetry. The weight of the camera without lens is 160 g. The camera uses a 20 M pixel CMOSIS CMV20000 global shutter CMOS sensor (AMS, 2020). The sensor size is 32.77 mm × 24.58 mm. The image size is 5120 × 3840 pixels. The radiometry is acquired with 12 bits. A temperature sensor is included in the CMOS sensor. This camera integrates a System-on-Chip (SoC) based on a dual-core ARM and an FPGA. This system can manage the acquisition and saving of images on an SD Card. A Wi-Fi connection allows users to control the camera through a web interface. The CamLight has a modular structure; it has a GPS module and a battery module using a 7.2 V CANON LP-E6N battery for mobile usage. The CMV20000 sensor has 2 models: monochrome and Bayer color. The Bayer color model was used in this study.

Lens
A Samyang 8 mm f/3.5 fish-eye lens is used for this imager. Its focal length is 8 mm, the field of view is 180 • .The projection area of this lens is a little larger than the sensor plane: 92.3% of the pixels in the hemisphere are actually captured. Figure  1 shows the effective area of the imager, where the white area includes all pixels with an incident angle within 90 degrees.

Filter
To avoid the saturation due to high incident radiation, a neutral density (ND) filter can be added (Urquhart et al., 2015). It is difficult to attach an optical filter in front of the fish-eye lens covering the entire field of view. We therefore inserted a ND8 filter film between the lens and the CMOS sensor which reduces the radiation to 1/8 in the visible spectrum. However, this film does not suppress the near-infrared (NIR) radiation. A NIR glass filter is added complementing the ND8 filter. The transmittance spectra of the filters are presented in Figure 2.
We did not measure the transmittance spectrum of the lens. Ignoring this transmittance, the spectral response function of this imager is obtained by multiplying the quantum efficiency spectrum of the color CMV20000 sensor and the transmittance spectrum of the ND8+NIR filters. The function normalized between 0 and 1 is presented in Figure 3.

Noise model
The main sources of digital camera noise are: shot noise, dark current noise, and read noise (Healey and Kondepudy, 1994). Shot noise depends on the incident radiation and cannot be eliminated. Dark current is generated by the free electrons,  which is independent of the incident radiation but highly dependent on the temperature. The read noise is generated by the electronic device, which is independent of the incident radiation, temperature, and exposure time. Only the biases on these factors are of interest from the radiometric correction point of view, so we propose a dark level model to describe the expected value of the sum of the dark current noise and read noise as a function of the sensor temperature T and exposure time t.
where Bi,j is the camera noise in digital count at the position (i, j), t is the exposure time (s), T is the sensor temperature ( • C), Bi,j,0 is the dark level image at a very short exposure time t0 and low temperature T0. Bi,j,0 is mainly due to the read noise. The factor b is considered constant for the entire image, the factor ai,j varies for each pixel, representing its thermal current at T0.

Radiometric model
The raw pixel value of the camera is a digital readout of the electrical signal leaving the photosensitive sensor, including the photoelectrons produced by the transformation of incident radiation and the free electrons generated by the device (Healey and Kondepudy, 1994). The relation between the digital value of the pixel and the irradiance received by the photosensitive sensor is described as: where Pi,j is the raw pixel value in digital count at the position (i, j) recorded by the camera, Ei,j is the mean spectral irradiance received by the photosensitive cell (W · m −2 ·µm −1 ) in the spectral band considered, Ci,j is the radiometric coefficient of the sensor (J/count), Ai,j is the area of the photosensitive cell (m 2 ), ∆λ is the spectral band width (µm).
The transformation of radiation into electrical charge is usually described by the quantum efficiency of the photosensitive sensor, which is the ratio of the number of electrons collected to the number of photons incident on the photosensitive surface. Quantum efficiency is represented as a spectrum depending on the wavelength range of the sensor. The radiometric coefficient Ci,j is based on this principle, but quantifies the number of photons per Joule and replaces the number of electrons by the digital count including the transformation by the analog-todigital converter (ADC) module. Ci,j therefore also depends on the wavelength. For a 3-band RGB optical sensor, the radiometric coefficient of each band is considered as the integration of the spectral radiometric coefficient over its bandwidth.
The received irradiance is proportional to the incident radiance, taking into account the vignetting effect (Willson and Shafer, 1994), the follow equation describes the relationship between the received irradiance Ei,j and the incident radiance Li,j (W · m −2 ·µm −1 ·sr −1 ).
where Vi,j is the vignetting factor, fn is the f-number of the optical system, which is the ratio of the focal length to the diameter of the entrance pupil.
Therefore, the relationship between incident radiance and pixel value can be given as: We define a spatial factor Si,j to describe the spatial variation in Ai,j, Ci,j and Vi,j: Si,j = Ai,jVi,j Ci,j Cc AcVc where Ac, Cc and Vc are the values at the center of the sensor.
The equation (4) becomes: For this imager, once the choice of sensor and optical system is fixed, the parameters fn, Ac, Vc, ∆λ are constants. The final radiometric model is therefore given by the following equation.
Bi,j can be estimated using the proposed dark level model, D can be determined by radiometric response calibration, Si,j can be determined by flat-field calibration, which are described in the section below.

Geometric model
The geometric model of the imager is used to determine the incident angle relative to the optical axis for each pixel. We used the fish-eye lens model provided by the MicMac software, which is based on a rectilinear projection function and a radial distortion model in the form of a polynomial (Pierrot-Deseilligny et al., 2014). A geometric calibration is performed to determine the position of the principal point, the center of distortion, the focal length, and the parameters of the polynomial. Once these parameters are estimated, it is possible to perform geometric mapping of the radiance field from the fish-eye images.

Dark level
To determine the two factors ai,j and b in Equation (1), a set of dark images (images taken without incident light) were acquired with different sensor temperatures and exposure times. Figure 4 shows the mean value and the standard deviation of the dark images as a function of the exposure time at different sensor temperatures. The exposure time varies between 1 ms and 10 s. This figure shows that the mean value of dark noise increases linearly with the exposure time. The relationship between the dark image and the sensor temperature is plotted in Figure 5. It can be noted that the level of dark noise clearly increases with temperature at long exposure times (t > 0.1s) but not at short exposure times. This is because, at a short exposure time, the dark image contains mostly the read noise which is independent of the exposure time and the sensor temperature. T0 was fixed at 28.7 • C, the lowest measured sensor temperature. t0 was fixed empirically as 1 ms, a very short exposure time, at which the noise level is almost invariant over the entire temperature range of the sensor ( Figure 5). The basic dark image B0 was then obtained with t0 and T0. The factor b was determined to be equal to 0.1237 • C −1 using least squares regression, leading to a doubling factor every 5.6 • C, from the mean value of dark images B, with the following equation.  Once the factor b was obtained, we determined the value of ai,j and Bi,j,0 at each pixel using linear regression.

Dark image modelling performance
The difference between the mean of the measured dark image B and the mean of the modeled dark imageB was calculated as the dark subtraction error to evaluate the noise modelling performance. Figure 7 shows the evolution of error with the sensor temperature and with the exposure time. Knowing that the camera CamLight is not designed for long exposure measurements, exposure times greater than 0.1 s were not considered during the evaluation. Under normal measurement conditions, the mean values of dark images are close to 0 after dark subtraction. Figure 8 shows the histograms of a measured dark image and its dark corrected image. The dark image was measured at t = 0.1 s and T = 48 • C. The subtraction result image is presented in Figure 9. The results show the performance of the noise model for reducing the dark level to 0 and for decreasing the noise variability.

Radiometric response
The radiometric linearity of the photosensitive sensor is the basis for radiometric measurements and HDR measurements. The radiometric linearity of the camera can be evaluated by the relationship between the pixel value after dark noise subtraction P dc (= P − B) and the product of the exposure time and the in-cident radiance tL. If the light source is stable, the radiometric linearity can be evaluated by the relationship between P dc and t. We performed a set of photos with different exposure times using a Labsphere integrating sphere. Limited by the aperture size, only a small area of 200 × 200 pixels in the center of the sensor is considered as illuminated by the constant radiance of the integrating sphere.
The P dc of the red band as a function of exposure time are presented in Figure 10. We randomly selected 100 pixels in the center area of the images. This figure shows that under the constant incident light condition, the pixel value increases linearly with the exposure time until saturation.The difference between the curves is due to the spatial variation, described by the spatial factor Si,j. Based on the measurements at the center of the sensor (3 × 3 pixels) where the spatial factor is equal to 1 (Sc = 1), we determined a linear range P dc ∈ [50,3500] in which the radiometric model (Equation (7)) is valid. These measurements were also used for absolute radiometric calibration to determine the value of D. According to the radiometric model (Equation (7)), the measurements of the integrating sphere at the center of the sensor can be written as: where L is the radiance emitted by the integrating sphere corresponding to each spectral band of the camera (W · m −2 ·µm −1 ·sr −1 ), obtained by the band integration: where L(λ) is the integrating sphere spectral radiance measured by a calibrated spectroradiometer, q(λ) is the spectral response function (Figure 3).
The values of D are 8.53e-5, 7.51e-5,and 7.98e-5 for blue, green, and red bands, determined using linear regression. The regression curves for determining the factor D are presented in Figure 11.

Radiometric response performance
As shown in Figure 11, valid pixels have been verified to have a good radiometric linearity (R 2 > 0.99). The possible errors in D calibration are mainly caused by instabilities in the light source and the random noise. We calculated the relative error between the reference radiance and the modeled radiance for each spectral band. By using the HDR technology, it is possible to obtain a high pixel value for each pixel to reduce the impact of the shot noise. The relative errors of radiometric calibration are 2.07%, 1.34%, and 0.98% in the blue, green, and red band for P dc ∈ [2000,3000]. The error in the blue band is twice that of the red band because of the low emission power of the light source in the blue. Figure 11. The regression curve for determining factor D of blue, green, and red band.

Invalid pixels
Due to manufacturing defects, very few photosensitive cells of the sensor have a radiometric response incompatible with the linear model, and will therefore be considered invalid. These sensor defects are independent of the lens and detectable using an integrating sphere. Using another small diameter lens, we took a series of photos of the integrating sphere while increasing the exposure time. This lens was entirely in the sphere, so these images made it possible to verify the radiometric linearity of all pixels of the sensor.
In Figure 12, we compare the radiometric linearity of the valid pixels (green line) with the invalid pixels (blue line). We considered that the invalid pixels are the pixels whose R 2 of the radiometric linearity curve is less than 0.99. 2132 pixels have been classified as invalid pixels. We also found dead pixels through the radiometric linearity check, which did not react to the incident radiation (red line in Figure 12) . Figure 12. Radiometric linearity curves of valid pixels (green line), invalid pixels (blue) and dead pixel (red).

Flat-field
The flat-field calibration is performed by acquiring images from a uniformly illuminated scene where the incident radiance is identical for all pixels. The spatial factor Si,j can be calculated then: It is ideally to achieve the flat-field calibration with an integrating sphere (Beisl, 2006). However, the equipment is expensive and sometimes poses the problem of fitting size. As in our case, our integrating sphere is not large enough to cover the entire field of the fish-eye lens, we had to use another solution.
In very cloudy weather, we put a spherical polymer diffuser at the outside to create a homogeneous light field inside the diffuser. The camera is placed vertically below the diffuser with the lens fully inserted into the sphere (Figure 13). A series of photos were taken by gradually rotating the camera around the vertical axis. The flat-field image was obtained by fusion of the photos. Figure 13. Implementation of flat-field calibration using the spherical diffuser.
A drawback of measuring with the spherical diffuser is that it cannot ensure the uniformity of the incident radiance at a high zenith angle. Because as the zenith angle increases, the incident brightness is more affected by the surrounding environment. A series of images of the integrating sphere were acquired as the camera is rotated. These images were used to create a radial profile of the spatial factor as a reference for correcting the flatfield image obtained from the spherical diffuser. The final result is shown in Figure 14.
Although the most commonly used methods of vignetting correction are based on radial models, we prefer to use directly the spatial factor map derived from the flat-field image as it corrects also for the individual pixels sensitivity. As defined by Equation (5), the spatial factor is more complex than the vignetting factor. We can notice some non-radial features in Figure 14, like the difference in the radiometric response of the left and right parts of the CMOS sensor. We can also find small dark spots within the image which are probably due to the defective DN8 filter film. These impacts could be corrected by the spatial factor map.

Implementation
The MicMac software has been used to perform a geometric calibration based on a series of photos of a static scene with different viewing positions ( Figure 15). MicMac can automatically mark tie-points between images. By minimizing the distance between the predicted position and the observed position of tie-points in the image, MicMac can establish the geometric model.

Geometric calibration performance
The performance of the geometric calibration was evaluated by calculating the distance between the observed tie-point position and the predicted tie-point position. Figure 16 shows the distance map (in pixels) obtained using the geometric calibration images ( Figure 15). 5578 tie-points were found in the 17 calibration images by MicMac. The average error is 2.037 pixels (up to 0.09 • in zenith angle). The image sharpness is degraded by the added filters, which reduces the performance of geometric calibration. This error is not important for radiance distribution measurement. Figure 16. Distance map (in pixels) between the observed tie-point and the predicted tie-point by geometric model of MicMac.

CONCLUSION
Radiometric and geometric calibration techniques applied to a hemispherical radiance field imaging system have been described. The performance of the instrument has been evaluated.
The main contribution of the present work lies in the proposed dark level model, which is able to improve the dark noise correction for HDR acquirement and adapted for various sensor temperatures. Results demonstrate the ability of the imaging system to provide data with good accuracy. It is useful for performing surface incident radiance field measurements with a high geometrical resolution in motion due to its portability.