A step towards understanding why classification helps regression

Silvia L. Pintea, Yancong Lin, Jouke Dijkstra, Jan C. van Gemert; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 19972-19981

Abstract

A number of computer vision deep regression approaches report improved results when adding a classification loss to the regression loss. Here, we explore why this is useful in practice and when it is beneficial. To do so, we start from precisely controlled dataset variations and data samplings and find that the effect of adding a classification loss is the most pronounced for regression with imbalanced data. We explain these empirical findings by formalizing the relation between the balanced and imbalanced regression losses. Finally, we show that our findings hold on two real imbalanced image datasets for depth estimation (NYUD2-DIR), and age estimation (IMDB-WIKI-DIR), and on the problem of imbalanced video progress prediction (Breakfast). Our main takeaway is: for a regression task, if the data sampling is imbalanced, then add a classification loss.

Related Material

[pdf] [supp] [arXiv]
[bibtex]
@InProceedings{Pintea_2023_ICCV, author = {Pintea, Silvia L. and Lin, Yancong and Dijkstra, Jouke and van Gemert, Jan C.}, title = {A step towards understanding why classification helps regression}, booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)}, month = {October}, year = {2023}, pages = {19972-19981} }