MGiaD: Multigrid in all Dimensions. Efficiency and Robustness by Weight Sharing and Coarsening in Resolution and Channel Dimensions

Antonia van Betteray, Matthias Rottmann, Karsten Kahl; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2023, pp. 1292-1301

Current state-of-the-art deep neural networks for image classification are made up of 10-100 million learnable parameters, i.e. weights. Despite their high classification accuracy these networks are heavily overparameterized. The complexity of the weight count can be considered as a function of the number of channels, the spatial extent of the input and the number of layers of the network. Due to the use of convolutional layers the scaling of weight complexity is usually linear with regard to the resolution dimensions, but remains quadratic with respect to the number of channels. Active research in recent years in terms of using multigrid inspired ideas in deep neural networks have shown that on one hand a significant number of weights can be saved by appropriate weight sharing and on the other that a hierarchical structure in the channel dimension can improve the weight complexity to linear. Utilizing these findings, we introduce an architecture that establishes multigrid structures in all relevant dimensions, contributing a drastically improved accuracy-parameter trade-off. Our experiments show that this structured reduction in weight count reduces overparameterization and additionally improves performance over state-of-the-art ResNet architectures on typical image classification benchmarks.