FIXED-WING MICRO UAV OPEN DATA WITH DIGICAM AND RAW INS/GNSS

We have recently released in the open domain data originating from a series of flights conducted with a fixed-wing micro UAV carrying high-quality small camera and navigation sensors. This data was previously used in several peer-reviewed publications. However, the data that we describe in the following is part of a larger series that will be released gradually after incorporating user feedback (e.g., on formats, description, etc.) from the first (three) released open data-sets. In the first part of this work we provide a thorough description of the common elements of these data sets, notably the UAV, its sensors, methods of time-stamping and synchronizing data streams, reference geometrical relations among them (system calibration) as well as time-invariant sensor parameters (e.g., lens distortion, non-orthogonality of inertial sensors) together with ground control points that are valid over the whole series. In the second part we describe the individual missions and provide the links to the released data sets.


Motivation
This paper introduces a new reference / benchmark data set for evaluating of UAV mapping solution. The authors feel that this is needed as drones became disruptive in many domains, including in those of mapping and monitoring at large scales. Indeed, they represent a new, flexible and cost-effective means to obtain a bird's-eye view perspective for such purposes. On one side, using a drone as a payload platform does not change the wellestablished principles of geo-referencing optical data as they are principally independent of the vehicle holding the instruments; on the other hand, the quality of on-board devices is limited by the reduced size that is required for their use with drones. This applies to both navigation and optical sensors which in turn affects the mapping performance. At the same time it provides an opportunity to revise, modify or refine some theoretical aspects of photogrammetry and their applications in terms of modeling and sensor data fusion. For such development, the access of "raw sensor data" along with reference/bench-marking values (either in sensor or spatial domain) is very important, yet not so easy to obtain. Supporting such evolution is the motivating factor behind this publication.

Related publications
The released data sets so far were directly used in the investigations of camera models (Cledat et al., 2020) and in the research related to the joint adjustment of raw inertial readings and image observations (Cucci, Skaloud, 2019b, Cucci, Skaloud, 2019a. Some data was also used in improving the attitude accuracy of direct orientation by a preflight calibration (Clausen, Skaloud, 2020).
A platform of similar type but with different physical dimensions and older payload (e.g., camera, GNSS receiver) was pre- * Corresponding author viously used in researching time calibration  and data fusion concepts (Rehak, Skaloud, 2016, Cucci et al., 2017.

Aircraft and GNSS
The drone is a second realisation of a platform described first in (Rehak, Skaloud, 2015) that is based on a foam hobby-kit from Multiplex (Fig. 1). The plane has a size of 1630 mm by 1700 mm, overall weight at full capacity of 2.8 kg and typical endurance of 45 min with a payload of 600 g. The flying speed is around 14 m/s and the stabilized and auto-mission modes are handled by an open-source autopilot (Dronecode, n.d.), ver. FMUv2 from 2016; the driver of which was modified for accepting binary messages from Javad/Topcon GNSS receivers. and an embedded PC plus highlighted coordinate frames: (i) green: the physical body-frame of IMU denoted NC0, (ii) blue: the chosen navigation frame for IMU NC1, (iii) the chosen camera-frame with x − y plane spanning image coordinates.
raw observations are stored internally on a micro-SD card at 10 Hz frequency. A master station (Javad Triumph-2) logging raw observations at the same frequency was always placed in the take-off and landing area. Coordinates of its phase-center are tied by post-processing to the Automated GNSS Network of Switzerland (AGNES) 1 .

Camera
The employed camera, conceived by IGN (Martin et al., 2014), is specially designed for close-range photogrammetric UAV acquisitions. The imaging sensor is a full frame CMOS with 5120 × 3840 square pixels with an edge of 6.4 µm. The employed camera is a panchromatic version with 12-bit resolution. The shutter speed can be programmed as fast as 1/5000 s, which -theoretically for this aircraft -allows obtaining ground resolution (GSD) of ≈1.2 cm 2 with a tolerable blur (≤ 1/3 pixel) caused by the forward motion. The camera is interfaced with Leica-M mount, to which we have installed Zeiss Biogon 35 mm, f 2.8 prime lens.
With external storage such as SSD the camera is capable of capturing many images per second. However, to limit the weight of the camera and lens within 300 g, images were stored on an internal micro-SD card. Due to that, the smallest interval between subsequent images is ≈1.3 s. Considering this plus the previously mentioned parameters the forward overlap at the limiting GSD of 1.2 cm is ≈ 70 %.

IMU
The payload includes a custom board (Kluter, 2013) with a magnetometer, a static air-pressure sensor (barometer) and two industrial-grade IMUs from the Navchip family 3 . In its first version (V1), the Navchip sensor includes one IMU plus one 3-axis magnetometer in the same enclosure. Thus, three, 3axis magnetometers are available in total. Fig. 2 gives an overview of these components within their rigid carbon fiber assembly. The IMU raw-data reading is programmable up to 1 kHz, but for more than one IMU is limited to 500 Hz. The data are stored in the internal memory and/or on an embedded computer (Raspberry-Pi) connected over USB port running a data-parsing software correcting the individual IMU data by the pre-calibrated deterministic parameters and associating the GPS-time-of-week (TOW) to each observation.

Time tagging
The GNSS receiver provides a pulse-per-second (PPS) to the autopilot as well as to the Gecko4Nav board, where it is further forwarded to each IMU for steering its internal frequency for data sampling and time-stamping to that of GPS second. Each PPS is associated with a message containing TOW, so that the offset from internal time-stamping can be determined by dataparsing software. Although the camera is equipped with an internal GNSS receiver for synchronising image-acquisition with GPS time, this one was not used. The images were triggered as a function of distance by the auto-pilot and the camera was issuing mid-exposure pulses when taking pictures. These were then TOW-tagged as events within the primary Topcon B110 receiver.

Lever-arm
The determination of lever-arm follows the technique (Rehak, Skaloud, 2015) that resolves camera to antenna phase-center vector with mm-level precision in the camera c-frame. It exploits the possibility of fixing the fuselage of a small aircraft on a tripod turned by 90 deg in roll and taking images of close targets from several positions j ∈ {1, · · · , J}. The camera poses Γ m c,j = [c m , R c m ] j , (i.e., the position and the attitude of the camera c-frame) w.r.t. mapping m-frame are then obtained via photogrammetry. The phase-center of a GNSS antenna is replaced by a tip of a pin (Fig. 3), the position of which p m j is determined (i.e., by tachymetry -theodolite) at each j. The camera-antenna lever-arm a c is the (weighted) average 4 value of all a c j = R c j,m p m j − c m j . Figure 3. Lever-arm calibration camera to GNSS antenna, phase-center of which is represented by a tip of pin (red-circle).
The lever-arm vector from camera to both IMUs, i.e., b c 0 , b c 1 is determined from the payload CAD-design or by caliper measurements. Table 1 summarizes the values of all three lever-arm vectors in the c-frame, axes of which are defined on Fig. 2 with respect to the flight direction as x-backward, y-left wing and z-down.
+0.113 +0.020 +0.000 NC1 imu Table 1. Lever-arm values of antenna-phase center a c , and imu-body navigation centers b c 0 , b c 1 in the camera-frame.

Bore-sight
The bore-sight may be resolved within bundle-adjustment as an unknown constant-parameter using navigation data within a configuration that allows to both reduce the incertitude in attitude initialisation and to de-correlate internal and external parameters of orientation parameters. The long flight of ign8 (see later on) with two flight-levels and perpendicular lines is suitable for such purpose.
The conventional approach is to use the combination of Cardan rotations to approximately align the resolved attitude of aircraft body R b ned with that of camera T c b . When using the north-eastdown (ned) local-level navigation-frame with the usual aeronautic convention (i.e., x-forward, y-right-wing, z-down) and the camera frame as defined in Fig. 2. T c b corresponds to 180 deg rotation around z-axis. Further transformation of attitude for obtaining the rotation from mapping frame to camera R c m depends on the choice of the latter and is bit complex for projections (Legat, 2006). In the most simple scenario, the mapping frame is chosen as a Cartesian frame with east-northup (enu) convention whose origin is on the navigation ellipsoid placed at the mapping area (so called tangent-plane). We define the bore-sight B b b as a matrix that together with time-constant matrices T and time-dependent matrix R b ned (t) (e.g., determined by optimal smoothing of INS/GNSS) acts as an observation of rotation from m-frame to c-frame for an image j (e.g., in the bundle-adjustment software): where the other terms were stated previously and T ned enu is a rotation matrix that exchanges the first two axes and reverses the third one. The recovered bore-sight matrix is listed in Tab

CAMERA CALIBRATION
Extensive studies have been performed to determine a suitable intrinsic calibration for the employed camera. All those are based on bundle-block adjustments with strong image geometry and dense ground control networks. Commercial photogrammetry software (Agisoft MetaShape, Pix4D Mapper and externally also Australis) has been used for the processing. Two distinct scenarios have been considered: 1. The camera was flown with a multi-rotor UAV over a small urban calibration field. The camera roll was varied to obtain converging image geometry. 16 GCPs were available. The experiment was repeated in the same conditions after 1.5 months to evaluate the camera mechanical and optical stability. The two datasets correspond to flight ignCal1 and ignCal2 in Table 6. An extensive evaluation of this scenario (including processing with Australis) can be found in (Roth, 2019).
2. The camera was flown with the fixed-wing UAV presented in Section 2.1 over a large rural calibration field (42 ha). 25 GCPs were available. Moreover, accurate absolute position/orientation control was used to improve the camera calibration observability. This correspond to flight ign8 in Table 6.
An overview of the two calibration fields, with the the obtained GCP residuals are shown in Fig. 5.
Different instances of the Brown-Conrady (Brown, 1971) model, with 10, 15 and 18 parameters, were investigated to characterize the radial and tangential distortions of the lenses. Additional parameters above 10 were found to have a marginal influence in modeling this specific camera and lenses and have been discarded. The computed parameters are presented in Table 3.  It has to be noted that the ignCal2 flight was performed 43 days after ignCal1, while several other flights have been performed in the meantime. The determined intrinsic calibration from these two flights differ only marginally. Although not shown, marginal variations of intrinsic parameters are observed also between in-flight calibrations (e.g. ign8 w.r.t. ign7), alltogether suggesting good mechanical stability of the lens system and camera.
The values obtained for the intrinsic calibration on flights ignCal2 and ign8 were evaluated in a production-like scenario in (Cledat et al., 2020) with no GCP and accurate absolute position/orientation aerial control (from GNSS/INS Kalman smoothing). A 2 km long corridor flight, corresponding to ign6xl in Table 6 was considered for which the image geometry does not allow to determine the camera calibration from  scratch. Overall, it has been found that the pre-determined camera calibration yielded acceptable residual on 24 checkpoints, which can be largely improved if the leading parameters (focal length and principal point) are re-adjusted using the determined values as initial guess. The results are summarized in Table 4; please refer to the aforesaid publication for the details as well as for a comparison with alternative (non-physical) models.
The superiority of Brown-Conrady distortion models for this camera is likely related to the combination of two elements: i) the chosen prime-lens; ii) the fact that no "internal cameracorrections" are applied on the acquired imagery. Indeed, impact of lens-distortions is often limited prior image storage directly within some smaller UAV cameras (e.g. SODA or Aeria X from senseFly, dji-Phantom, etc.) with the aim of presenting more appealing results to the user. However, such corrections are not accessible, which makes the appropriate choice of models for subsequent IO refinement or re-calibration less obvious.  Table 4. Checkpoints statistics (in mm) when the calibration from ignCal2 and ign8 is used for processing ign6xl. "Fix" means that calibration is used as it is, while "Lead" that f and cx, cy are re-adjusted. Color coding w.r.t. the best obtained values.

Deterministic
The deterministic calibration is separated in two phases. The first-phase is executed in laboratory with the help of special equipment to determine parameters that will be considered as time invariant. These are non-orthogonality angles between all inertial sensors (i.e., gyroscopes and accelerometers) as well as their scale factors. The second phase determines turn-on (random) biases on these sensors 5 . This part is executed prior the take-off or after landing. Both procedures are described in (Clausen, Skaloud, 2020). The provided data contains two files for each IMU: with and without application of deterministic calibration 6 .

Stochastic
The IMU stochastic calibration has been performed by collecting 4 hours of static data and then employing the Multi-signal Generalized Method of Wavelet Moments (M-GMWM) (Bakalli et al., 2018). GMWM (Guerrier et al., 2013) improves over the well known Allan variance technique (El-Sheimy et al., 2007) in terms of several statistical properties, please refer to the original publications for the details. On the top of this, M-GMWM allows to consider multiple realisations of the same experiment during stochastic modeling.
We model the stochastic errors affecting inertial sensors as the sum of simple stochastic processes: a white-noise and multiple first-order Gauss-Markov processes with increasing correlation time (T = 1/β). The resulting models are given in Table 5 in terms of the estimated parameters of the continuous time stochastic models 7 The units are as follows: -Gyroscope: deg/s 2 / √ Hz × 1E3 -Accelerometer: µg/s/ √ Hz 5 The biases specified by the manufacture concern only in-run stability, not turn-on/off. 6 To avoid errors in application the raw and corrected data are provided rather than parameter values. 7 The continuous time first-order Gauss-Markov noise model, as typically implemented in extended Kalman filters for navigation, isė(t) = −βe(t) + ξ(t), where ξ(t) is a continuous time white noise with Power Spectral Density (PSD) q. Note that ξ(t) adds to the time derivative of e(t), thus its units are the units of e(t) per second.  6. MISSIONS

Test zone
About 40 signalized targets (30 × 30 cm) were surveyed by carrier-phase differential GNSS (about 40 min long sessions) with respect to the aforesaid national network (AGNES) in a rural zone west of Lausanne, Switzerland as shown in Fig. 6. The shape of the test field allows to fly different configurations, (e.g. corridor -cr and block -bk) over a relatively large area (≈ 2 km 2 ). The shape of the target shown in Fig. 7 is suitable for automated image observations with an accuracy of a fraction of pixel as further detailed in Sec. 6.3. Figure 6. Test side with signalized targets of 30 × 30 cm (Fig. 7)

Overview
Tab. 6 provides an overview of exploitable missions flown in the period May -October 2018 together with their specifications: trajectory shape (block/corridor), surface covered in hectares (ha), mean altitude above ground level (AGL), ground sampling distance (GSD)/resolution of the imagery and status of data. The flights 01 -09 were taken over the rural-zone while missions 10 -11 served for taking high-converging imagery for calibrating the lens distortions. These were flown with a multirotor UAV carrying digiCAM over a designated close-range target field. The other flights were completed with a fixed-wing UAV over the previously described test-zone.

GCPs image measurements
Several points on the grounds have been surveyed with a GNSS receiver (PPK). Those have been signaled on the ground with a 30 cm black and white pattern, shown in Fig. 7, to be used either as GCPs or checkpoints. The image coordinates of such points are typically determined manually by human operators. Due to the large amount of projections of such targets in the images, a semi-automated procedure has been employed to accurately determine the image coordinates of the targets center and to match those to known 3D coordinates. This procedure works as follows: 1. The image exterior orientation of all images is determined from bundle adjustment with absolute aerial position control and no GCPs. Alternatively, an absolute aerial attitude control is also used, which would allow to skip step 2.
2. Few projections (e.g., 4 points, 3 projections per point) are manually identified and selected as GCPs. The bundle adjustment is re-run to improve the image exterior orientation.
3. The predicted image projections of all known points are generated using the image exterior orientation determined so far. These will be close to the true image projections.
4. The final sub-pixel image coordinates of the targets are determined maximizing the cross-correlation between the target template and the pixel values, in the neighbourhood of the predicted image observations. 5. All the determined image coordinates are reviewed visually to exclude any outlier.
Two examples of the cross-correlation maximization procedure are shown in Fig. 7: the blue square shows the search region, centered at the predicted image observation for that specific target and image. The red "X" marks the determined, sub-pixel center of the target and the blue lines its orientation on the image.
The described procedure permits to substantially increase the repeatability and the accuracy of target image observations. We have quantified the accuracy of those to lie between 0.1 and 0.2 pixels, which we believe to be largely below what could be obtained by a human operator.

Released data
The released data (Sec. 7.3) contain close-range calibration with converging photography ignCal2, large block flight with perpendicular flight-lines ign8 (Fig. 8b) and long corridor ign6xl (Fig. 8a). All mission have two flight-levels. More details are given in Tab. 7 with the processing results presented in (Cledat et al., 2020).

Structure
The data-sets (Sec. 7.3) are organized as follows. The main folder has README.md and FLIGHT.md files, the former containing general information on data organization, the latter, key characteristics of the flight. The sub-folder organization is depicted in Fig. 9: • There are two folders inside the root directory: 01 Observations and 02 Processed. The first contains the sensor measurements and the second the elements of trajectory estimated from navigation data using different methods.
• Inside the 01 Observations folder, there is a separate sub-folder for each sensor (Camera, GNSS rover and base station, and IMUs). A README.md file contains the information about the sensors, data format, units and other auxiliary information relevant for processing.
• Inside the 02 Processed folder, two sub-folders contain the GPS-PPK solution and the camera exterior orientation derived from ING/GNSS Kalman smoothing.

Data formats
The sensors selected for this data-set are two real and one virtual GPS/GLONASS receivers, two IMUs, and camera. The data from these sensors are provided in a format which is open and can be used across different platforms. A summary of the data formats used for different sensors is given in Tab. 8.

Data Format
Sensors GPS RIINEX 2.11 IMU .csv Camera .tif,.txt Auxiliary GCPs .txt Trajectory, EO .txt Table 8. Summary of the data formats in the data-set

Access
The data-sets are available at (Skaloud et al., 2021c, Skaloud et al., 2021b, Skaloud et al., 2021a and are distributed under the CC-BY 4.0 license 8 .

CONCLUSIONS
Drone mapping offers an opportunity to evolve the integration of navigation and optical sensors to an optimum when treating raw observations in a common adjustment. Yet gaining the access of such data is less obvious. This is for instance the case for obtaining raw inertial readings of a reasonable quality with correct time-stamping or images without on-chip distortion re-corrections by an unknown function. For these reasons we hope that the release of the described data in an open domain have value within the scientific community and commercial developers for either bench-marking traditional approaches or conceiving and testing newer and improved concepts. This release would be hardly possible without the contribution and open spirit of many to which we sincerely thank while explicitly mentioning some in the acknowledgement.