ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Download
Publications Copernicus
Download
Citation
Articles | Volume V-2-2020
ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., V-2-2020, 1005–1012, 2020
https://doi.org/10.5194/isprs-annals-V-2-2020-1005-2020
ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., V-2-2020, 1005–1012, 2020
https://doi.org/10.5194/isprs-annals-V-2-2020-1005-2020

  03 Aug 2020

03 Aug 2020

WHAT IDENTIFIES A WHALE BY ITS FLUKE? ON THE BENEFIT OF INTERPRETABLE MACHINE LEARNING FOR WHALE IDENTIFICATION

J. Kierdorf1, J. Garcke2,3, J. Behley1, T. Cheeseman4, and R. Roscher1,5 J. Kierdorf et al.
  • 1Institute of Geodesy and Geoinformation, University of Bonn, Germany
  • 2Institute for Numerical Simulation, University of Bonn, Germany
  • 3Fraunhofer Center for Machine Learning and Fraunhofer SCAI, Sankt Augustin, Germany
  • 4Happywhale and Southern Cross University
  • 5Institute of Computer Science, University of Osnabrueck, Germany

Keywords: Machine Learning, Deep Learning, Neural Networks, Interpretability, Visualization, Humpback Whales

Abstract. Interpretable and explainable machine learning have proven to be promising approaches to verify the quality of a data-driven model in general as well as to obtain more information about the quality of certain observations in practise. In this paper, we use these approaches for an application in the marine sciences to support the monitoring of whales. Whale population monitoring is an important element of whale conservation, where the identification of whales plays an important role in this process, for example to trace the migration of whales over time and space. Classical approaches use photographs and a manual mapping with special focus on the shape of the whale flukes and their unique pigmentation. However, this is not feasible for comprehensive monitoring. Machine learning methods, especially deep neural networks, have shown that they can efficiently solve the automatic observation of a large number of whales. Despite their success for many different tasks such as identification, further potentials such as interpretability and their benefits have not yet been exploited. Our main contribution is an analysis of interpretation tools, especially occlusion sensitivity maps, and the question of how the gained insights can help a whale researcher. For our analysis, we use images of humpback whale flukes provided by the Kaggle Challenge ”Humpback Whale Identification”. By means of spectral cluster analysis of heatmaps, which indicate which parts of the image are important for a decision, we can show that the they can be grouped in a meaningful way. Moreover, it appears that characteristics automatically determined by a neural network correspond to those that are considered important by a whale expert.