Hierarchical Image Classification Using Entailment Cone Embeddings

Ankit Dhall, Anastasia Makarova, Octavian Ganea, Dario Pavllo, Michael Greeff, Andreas Krause; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2020, pp. 836-837

Image classification has been studied extensively but there has been limited work in using unconventional, external guidance other than traditional image-label pairs for training. In this work, we present a set of methods to leverage information about the semantic hierarchy induced by class labels. We first inject label-hierarchy knowledge to an arbitrary CNN-based classifier and empirically show that the availability of such external semantic information in conjunction with the visual semantics from images boosts overall performance. Taking a step further in this direction, we model more explicitly the label-label and label-image interactions using order-preserving embedding-based models governed by both Euclidean and hyperbolic geometry, prevalent in natural language and tailor them to hierarchical image classification. We empirically validate all the models on the hierarchical ETHEC dataset.