Neuromanifold-Regularized KANs for Shape-fair Feature Representations

Mazlum Ferhat Arslan, Weihong Guo, Shuo Li; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2025, pp. 12790-12799

Traditional deep networks struggle to acquire shape-fair representations due to their high expressivity. Kolmogorov-Arnold Networks (KANs) are promising candidates as they learn nonlinearities directly, a property that makes them more adaptive. However, KANs perform suboptimally in terms of shape-fairness because of unconstrained nonlinearities, a limitation we demonstrate for the first time. On the other hand, shape-fair networks reside on a neuromanifold of low-degree. Motivated by this, we investigate neuromanifold regularization of KANs to enable learning of shape-fair feature representations. The proposed method, NeuroManifold Regularized-KANs, is a novel regularization that addresses failure modes during the acquisition of local and global shape cues, separately. This is done by constraining the degree of the neuromanifolds of two jointly trained feature extractors. Additionally, we propose a novel Style Decorrelation Loss that promotes decorrelation of intermediate representations. Our experiments demonstrate that NMR-KAN improves shape bias over baseline convolutional KANs by 14.8% while also providing robustness under image corruptions and adversarial attacks. Code is avaliable at: http://www.github.com/kaptres/NMR-KAN