ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Publications Copernicus
Download
Citation
Volume V-2-2020
ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., V-2-2020, 641–648, 2020
https://doi.org/10.5194/isprs-annals-V-2-2020-641-2020
© Author(s) 2020. This work is distributed under
the Creative Commons Attribution 4.0 License.
ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., V-2-2020, 641–648, 2020
https://doi.org/10.5194/isprs-annals-V-2-2020-641-2020
© Author(s) 2020. This work is distributed under
the Creative Commons Attribution 4.0 License.

  03 Aug 2020

03 Aug 2020

ASSESSING THE SEMANTIC SIMILARITY OF IMAGES OF SILK FABRICS USING CONVOLUTIONAL NEURAL NETWORKS

D. Clermont, M. Dorozynski, D. Wittich, and F. Rottensteiner D. Clermont et al.
  • Institute of Photogrammetry and GeoInformation, Leibniz Universität Hannover, Germany

Keywords: Convolutional Neural Networks, Image similarity, Cultural heritage, Silk fabrics, Incomplete training samples

Abstract. This paper proposes several methods for training a Convolutional Neural Network (CNN) for learning the similarity between images of silk fabrics based on multiple semantic properties of the fabrics. In the context of the EU H2020 project SILKNOW (http://silknow.eu/), two variants of training were developed, one based on a Siamese CNN and one based on a triplet architecture. We propose different definitions of similarity and different loss functions for both training strategies, some of them also allowing the use of incomplete information about the training data. We assess the quality of the trained model by using the learned image features in a k-NN classification. We achieve overall accuracies of 93–95% and average F1-scores of 87–92%.