Leveling Down in Computer Vision: Pareto Inefficiencies in Fair Deep Classifiers

Dominik Zietlow, Michael Lohaus, Guha Balakrishnan, Matthäus Kleindessner, Francesco Locatello, Bernhard Schölkopf, Chris Russell; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 10410-10421

Abstract


Algorithmic fairness is frequently motivated in terms of a trade-off in which overall performance is decreased so as to improve performance on disadvantaged groups where the algorithm would otherwise be less accurate. Contrary to this, we find that applying existing fairness approaches to computer vision improve fairness by degrading the performance of classifiers across all groups (with increased degradation on the best performing groups). Extending the bias-variance decomposition for classification to fairness, we theoretically explain why the majority of fairness methods designed for low capacity models should not be used in settings involving high-capacity models, a scenario common to computer vision. We corroborate this analysis with extensive experimental support that shows that many of the fairness heuristics used in computer vision also degrade performance on the most disadvantaged groups. Building on these insights, we propose an adaptive augmentation strategy that, uniquely, of all methods tested, improves performance for the disadvantaged groups.

Related Material


[pdf] [supp] [arXiv]
[bibtex]
@InProceedings{Zietlow_2022_CVPR, author = {Zietlow, Dominik and Lohaus, Michael and Balakrishnan, Guha and Kleindessner, Matth\"aus and Locatello, Francesco and Sch\"olkopf, Bernhard and Russell, Chris}, title = {Leveling Down in Computer Vision: Pareto Inefficiencies in Fair Deep Classifiers}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, month = {June}, year = {2022}, pages = {10410-10421} }