One-Class Learned Encoder-Decoder Network With Adversarial Context Masking for Novelty Detection

John Taylor Jewell, Vahid Reza Khazaie, Yalda Mohsenzadeh; Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2022, pp. 3591-3601

Novelty detection is the task of recognizing samples that do not belong to the distribution of the target class. During training, the novelty class is absent, preventing the use of traditional classification approaches. Deep autoencoders have been widely used as a base of many novelty detection methods. In particular, context autoencoders have been successful in the novelty detection task because of the more effective representations they learn by reconstructing original images from randomly masked images. However, a significant drawback of context autoencoders is that random masking fails to consistently cover important structures of the input image, leading to suboptimal representations - especially for the novelty detection task. In this paper, to optimize input masking, we introduce a Mask Module that learns to generate optimal masks and a Reconstructor that aims to reconstruct masked images. The networks are trained in an adversarial setting in which the Mask Module seeks to maximize the reconstruction error that the Reconstructor is minimizing. When applied to novelty detection, the proposed approach learns semantically richer representations compared to context autoencoders and enhances novelty detection at test time through more optimal masking. Novelty detection experiments on the MNIST and CIFAR-10 image datasets demonstrate the proposed approach's superiority over cutting-edge methods. In a further experiment on the UCSD video dataset for novelty detection, the proposed approach achieves a frame-level Area Under the Curve (AUC) of 99.02% and an Equal Error Rate (EER) of 5.4%, exceeding recent state-of-the-art models.