Sem-MASt3R: Semantically Guided Feature Matching with MASt3R

Dario Tenore, Daniel Barath, Marc Pollefeys, Qunjie Zhou; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2025, pp. 130-139

Feature matching is a crucial task in computer vision, with state-of-the-art methods like MASt3R achieving impressive results by leveraging 3D geometry. However, these methods often neglect semantic information, leading to ambiguities in challenging scenarios. We introduce Sem-MASt3R, which seamlessly integrates semantic understanding into the MASt3R framework. We leverage semantic representations from DINOv2 to guide the matching process, computing a semantic similarity matrix that is refined using an NCNet and incorporated into MASt3R's cross-attention mechanism. A two-phase training strategy first trains the NCNet and then fine-tunes MASt3R's feature extraction. Experiments demonstrate that Sem-MASt3R consistently improves matching accuracy and robustness compared to MASt3R and other state-of-the-art methods, particularly when semantic information is crucial for disambiguation. Our code and trained models will be publicly available.