Unified Energy-based Generative Network for Supervised Image Hashing

Khoa D Doan, Sarkhan Badirli, Chandan K Reddy; Proceedings of the Asian Conference on Computer Vision (ACCV), 2022, pp. 3960-3976

Hashing methods often face critical efficiency challenges, such as generalization with limited labeled data, and robustness issues (such as changes in the data distribution and missing information in the input data) in real-world retrieval applications. However, it is non-trivial to learn a hash function in existing supervised hashing methods with both acceptable efficiency and robustness. In this paper, we explore a unified generative hashing model based on an explicit energy-based model (EBM) that exhibits a better generalization with limited labeled data, and better robustness against distributional changes and missing data. Unlike the previous implicit generative adversarial network (GAN) based hashing approaches, which suffer from several practical difficulties since they simultaneously train two networks (the generator and the discriminator), our approach only trains one single generative network with multiple objectives. Specifically, the proposed generative hashing model is a bottom-up multipurpose network that simultaneously represents the images from multiple perspectives, including explicit probability density, binary hash code, and category. Our model is easier to train than GAN-based approaches as it is based on finding the maximum likelihood of the density function. The proposed model also exhibits significant robustness toward out-of-distribution query data and is able to overcome missing data in both the training and testing phase with minimal retrieval performance degradation. Extensive experiments on several real-world datasets demonstrate superior results in which the proposed model achieves up to 5% improvement over the current state-of-the-art supervised hashing methods and exhibits a significant performance boost and robustness in both out-of-distribution retrieval and missing data scenarios.