ClusterGNN: Cluster-Based Coarse-To-Fine Graph Neural Network for Efficient Feature Matching

Yan Shi, Jun-Xiong Cai, Yoli Shavit, Tai-Jiang Mu, Wensen Feng, Kai Zhang; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 12517-12526

Abstract


Graph Neural Networks (GNNs) with attention have been successfully applied for learning visual feature matching. However, current methods learn with complete graphs, resulting in a quadratic complexity in the number of features. Motivated by a prior observation that self- and cross- attention matrices converge to a sparse representation, we propose ClusterGNN, an attentional GNN architecture which operates on clusters for learning the feature matching task. Using a progressive clustering module we adaptively divide keypoints into different subgraphs to reduce redundant connectivity, and employ a coarse-to-fine paradigm for mitigating miss-classification within images. Our approach yields a 59.7% reduction in runtime and 58.4% reduction in memory consumption for dense detection, compared to current state-of-the-art GNN-based matching, while achieving a competitive performance on various computer vision tasks.

Related Material


[pdf] [arXiv]
[bibtex]
@InProceedings{Shi_2022_CVPR, author = {Shi, Yan and Cai, Jun-Xiong and Shavit, Yoli and Mu, Tai-Jiang and Feng, Wensen and Zhang, Kai}, title = {ClusterGNN: Cluster-Based Coarse-To-Fine Graph Neural Network for Efficient Feature Matching}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, month = {June}, year = {2022}, pages = {12517-12526} }