Rich Feature Learning via Diversification

Xi Leng, Yongqiang Chen, Xiaoying Tang, Yatao Bian; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Findings, 2026, pp. 2462-2472

Rich Feature Learning (RFL) aims to extract all beneficial features from the training distribution and shows promise for Out-of-Distribution (OOD) generalization. Despite its success, a precise and comprehensive definition of "richness" remains elusive. Through an in-depth comparison between RFL and empirical risk minimization (ERM), we identify that feature diversity is the key differentiator driving RFL's superior OOD performance. Building on this insight, we contribute a formal definition of rich features, encompassing both informativeness and diversity. Leveraging this foundation, we propose Diversity-fOunded Rich fEature lEarniNg (DOREEN), a simple yet highly effective RFL algorithm that trains multiple models with identical architectures concurrently to promote feature diversity. We theoretically demonstrate that DOREEN not only realizes the benefits of RFL but also addresses the limitations of prior RFL algorithms. Extensive experiments validate that DOREEN learns richer features and consistently enhances OOD performance across various OOD objectives.