Fair Domain Generalization with Heterogeneous Sensitive Attributes Across Domains

Ragja Palakkadavath, Hung Le, Thanh Nguyen-Tang, Sunil Gupta, Svetha Venkatesh; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 7389-7398

Domain generalization(DG) techniques classify data from unseen domains by capitalizing on data from multiple source domains. Most methods in DG focus on improving predictive performance measures on the unseen domain. Recent studies have also attempted to enforce fairness measures on the unseen domain. However these studies assume that every domain has the same sensitive attribute including the unseen domain. In practice each domain may be required to satisfy fairness with respect to its own set of multiple sensitive attributes. Given a set of sensitive attributes (S) current methods need to train 2^n models to ensure fairness with respect to any subset of S where n = |S|. We propose a single-model solution to address this new problem setting. We learn two feature representations one to generalize the model's predictive performance and another to generalize the model's fairness. The first representation is made invariant across domains to generalize predictive performance. The second representation is kept selectively invariant i.e. invariant only across domains having the same sensitive attributes. Our single model exhibits superior predictive performance and fairness measures against the current alternative of 2^n models on unseen domains on multiple real-world vision datasets. Our code is available at https://github.com/ragjapk/SISA.