Debiasify: Self-Distillation for Unsupervised Bias Mitigation

Nourhan Bayasi, Jamil Fayyad, Ghassan Hamarneh, Rafeef Garbi, Homayoun Najjaran; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 3227-3236

Simplicity bias is a critical challenge in neural networks since it often leads to favoring simpler solutions and learning unintended decision rules captured by spurious correlations causing models to be biased and diminishing their generalizability. While existing solutions rely on human supervision obtaining annotations of the different bias attributes is often impractical. To tackle this we present Debiasify a novel self-distillation approach that works without any prior information about the nature of biases. Our method leverages a new distillation loss to distill knowledge within a network; from a deep layer where complex highly-predictive features reside to a shallow layer where simpler yet attribute-conditioned features are found in an unsupervised manner. In this way Debiasify learns robust debiased representations that generalize well across various biases and datasets enhancing worst-group performance and overall accuracy. Extensive experiments on computer vision and medical imaging benchmarks show the efficacy of our method significantly outperforming the previous unsupervised debiasing methods (e.g. a 10.13% improvement in worst-group accuracy on Wavy Hair classification in CelebA) while achieving comparable or superior performance to supervised methods. Our code is publicly available at the following link:Debiasify.