Multi-Task Learning of Classification and Generation for Set-Structured Data

Fumioki Sato, Hideaki Hayashi, Hajime Nagahara; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 6741-6751

In this study we propose a multi-task learning model of classification and generation for set-structured data. The proposed model learns data generation and classification in a single neural network by integrating a classification layer into a variational autoencoder while maintaining permutation invariance and equivariance nature which are characteristics of set-structured data. The proposed model allows for semi-supervised learning in set-structured data classification and can also be applied to confidence calibration using the input data distribution estimated by the generative model. In the experiments we evaluated the performance of the proposed model in a semi-supervised classification task on set-structured datasets and compared it with a baseline model consisting only of a classifier. The results demonstrated that simultaneous learning of the classification and generation effectively improves the classification accuracy and confidence reliability for set-structured data even with a limited number of labeled data.