Model Weights Reflect a Continuous Space of Input Image Domains

Simen Cassiman, Marc Proesmans, Tinne Tuytelaars, Luc Van Gool; Proceedings of the Winter Conference on Applications of Computer Vision (WACV) Workshops, 2025, pp. 644-653

In recent years there has been a growing body of work around Domain Adaptation (DA) and Domain Generalisation (DG). However most rely on empirical testing to find new ways of generating invariant features from one domain to another. In reality DA methods must deal with many more domains that vary continuously from one to the next such as coping with adverse weather conditions in autonomous driving. This work shows that it is better to have experts for each of these specific domains but this is prohibitively expensive (e.g. a model per level of rain combined with each level of darkness). To see whether these experts can be generated from only a few models this work explores the model weight space and the relationship with respect to varying input data. Analysis indicates that there is a rich and continuous structure in weight space that could be used to find such experts. However existing linear methods lack the flexibility to capture this information. We demonstrate this using intuitive synthetic data and confirm this in real-world data under adverse weather conditions. Initial steps are taken using non-linear methods which are better equipped to represent these style variations. The findings have the potential to improve model training under DG and DA as well as in distributed training and model merging.