Research map / Architectures and efficiency / Attention and transformers

ResMatch: Residual Attention Learning for Feature Matching

AAAI 2024 · Architectures and efficiency › Attention and transformers · approach: Sparse and structured attention · 13 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ResMatch, which adds precomputed descriptor similarity to cross-attention and relative-position similarity to self-attention as residual bypasses, plus a KNN-based sparse variant sResMatch, for local feature matching.

Problem

Attention-based GNNs such as SuperGlue for local feature matching lack a clear understanding of how cross- and self-attention behave as matching and filtering operations, and full attention is O(N^2) in cost.

Approach

Reformulate cross-attention as learning residual matching functions on top of a bypassed descriptor-similarity score, and self-attention as residual filtering on top of a bypassed relative-position similarity score; sparsify attention to KNN neighborhoods chosen by these same bypass scores to get linear cost.

Key results

• On YFCC100M with RootSIFT, ResMatch reaches AUC@20 of 82.15 vs SuperGlue 80.62 and matching score 19.34 vs 17.37.
• On ScanNet with RootSIFT, ResMatch reaches AUC@20 of 57.97 vs SuperGlue 53.00.
• sResMatch uses KNN sparse attention (k=64) and achieves competitive results, e.g. AUC@20 of 82.26 on YFCC100M, while reducing cost to linear in number of points.

Measured effects reported in the paper

Surprising findings

• Removing residual cross-attention hurts more than removing residual self-attention on YFCC100M and ScanNet.
• sResMatch with sparse attention sometimes slightly outperforms full-attention ResMatch variants in ablation with many features.

Limitations

• ResMatch has O(N^2) cost from residual attention, making it the most time- and memory-hungry at large feature counts.
• sResMatch degrades on TUM with SuperPoint, attributed to unreliable dense features in low-quality indoor images.
• KNN and indexing operations are hard to optimize in practice, limiting realized speedups.

Experimental setup

Datasets: GL3D

Benchmarks: YFCC100M ScanNet FM-Bench Aachen Day-Night

Metrics: AUC@5/10/20 matching score precision %Recall #Corr localization accuracy

Baselines: NN with ratio test ConvMatch SuperGlue SGMNet ParaFormer

Builds on: SuperGlue SGMNet ResNet Realformer Sinkhorn algorithm

Closest papers by meaning

• SGAT: Learning Feature Matching with Singularity-enhanced Graph Attention Network (AAAI 2026 · similarity 0.89)
• ClusterGNN: Cluster-Based Coarse-To-Fine Graph Neural Network for Efficient Feature Matching (CVPR 2022 · similarity 0.88)
• Learning deep graph matching with channel-independent embedding and Hungarian attention (ICLR 2020 · similarity 0.87)
• Graph-Context Attention Networks for Size-Varied Deep Graph Matching (CVPR 2022 · similarity 0.87)
• Unifying Feature and Cost Aggregation with Transformers for Semantic and Visual Correspondence (ICLR 2024 · similarity 0.85)
• ResCLIP: Residual Attention for Training-free Dense Vision-language Inference (CVPR 2025 · similarity 0.84)