Active Learning for the Classification of Species in Underwater Images From a Fixed Observatory

Torben Moller, Ingunn Nilssen, Tim W. Nattkemper; Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, pp. 2891-2897

Vision based wildlife monitoring is an important task in the field of environmental monitoring. Wildlife monitoring activities often create large collections of data needing computational approaches to (semi-) automated detection and annotation of objects in the images/video. In this work, we consider the special case of marine wildlife monitoring using camera equipped fixed observatories. In such cases where a-priori knowledge about which species to find is limited, a standard computer vision approach, employing supervised learning, will not be applicable for detecting and classifying species (or events) in the images. In a recently proposed unsupervised learning method, image patches are extracted from a time series of underwater images that feature moving species (like starfish, etc). The patches are automatically grouped into clusters with similar morphology and a so called relevance score is assigned to each of the clusters describing the likeliness that it contains patches showing unusual changes. However, due to the unsupervised fashion (i) the categories don't have labels and (ii) do not reflect the species distribution satisfactory. In this paper, we propose an active learning method that builds upon these results and can be used to assign taxonomic categories to single patches based on a set of human expert annotations making use of the cluster structure and relevance scores. The evaluation shows that compared to traditional sampling strategies our approach uses significantly less manual labels to train a classifier. We are confident that the results are relevant for non-marine contexts as well.