Research map / Architectures and efficiency / Attention and transformers
Efficient LoFTR: Semi-Dense Local Feature Matching with Sparse-Like Speed
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Revisits LoFTR to build Efficient LoFTR, a semi-dense matcher using aggregated attention with token selection and a two-stage correlation refinement, achieving ~2.5x LoFTR speed with competitive or better accuracy.
Problem
Detector-free semi-dense local feature matching (LoFTR) is accurate but slow because transformers run over the full coarse feature map; its expectation-based subpixel refinement also suffers spatial variance.
Approach
Aggregate neighboring query tokens with depthwise strided convolution and select salient key tokens by max pooling before vanilla attention (aggregated attention), plus a two-stage fine correlation that first uses MNN matching at pixel level and then expectation within a 3x3 window for subpixel refinement; also drops dual-softmax at inference.
Key results
- On MegaDepth, Ours achieves AUC@5 56.4 vs LoFTR 52.8 and AspanFormer 55.3, with 40.1 ms vs LoFTR 66.2 ms per pair on ScanNet timing.
- The efficiency-optimized variant (without dual-softmax) runs at 35.6/27.0 ms vs SuperPoint+LightGlue 31.9/30.7 ms while reaching 55.4 AUC@5 on MegaDepth.
- On HPatches homography estimation, Ours reaches 66.5 AUC@3px vs LoFTR 65.9 and on Aachen v1.1 day/night performance is on par with the best methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Efficient LoFTR instead of AspanFormer | semi-dense matching · MegaDepth | AUC@5°: 55.3 -> 56.4 | Helped |
| use Efficient LoFTR instead of AspanFormer | semi-dense matching · ScanNet | AUC@5°: 19.6 -> 19.2 | Hurt |
| use Efficient LoFTR instead of LoFTR | semi-dense matching · HPatches | Homography estimation AUC@3px: 65.9 -> 66.5 | Helped |
| remove dual-softmax at inference (ablation) | Efficient LoFTR · MegaDepth | Pose estimation AUC@5°: 56.4 -> 55.4 | Hurt |
| remove dual-softmax at inference (ablation) | Efficient LoFTR · MegaDepth | Time (ms): 139.2 -> 102.0 | Helped |
| add aggregated attention instead of LoFTR's transformer (ablation) | Efficient LoFTR · MegaDepth | Pose estimation AUC@5°: 54.7 -> 56.4 | Helped |
| add two-stage refinement instead of LoFTR's refinement (ablation) | Efficient LoFTR · MegaDepth | Pose estimation AUC@5°: 54.7 -> 56.4 | Helped |
| add second-stage refinement (ablation) | Efficient LoFTR · MegaDepth | Pose estimation AUC@5°: 55.8 -> 56.4 | Helped |
| replace ResNet with RepVGG (ablation) | Efficient LoFTR · MegaDepth | Pose estimation AUC@5°: 55.4 -> 56.4 | Helped |
| replace ResNet with RepVGG (ablation) | Efficient LoFTR · MegaDepth | Time (ms): 156.2 -> 139.2 | Helped |
Surprising findings
- Dropping the dual-softmax at inference barely hurts accuracy while substantially improving speed on high-resolution images.
- Replacing the aggregated attention with LoFTR's transformer (with RoPE) lowers both accuracy and efficiency.
Limitations
- Accuracy on ScanNet is lower than the best semi-dense baseline AspanFormer (19.2 vs 19.6 AUC@5)
- Optimized variant without dual-softmax trades some accuracy for speed
- Trained only on outdoor MegaDepth, while ROMA uses DINOv2 features with stronger indoor generalization
Experimental setup
Datasets: MegaDepth
Benchmarks: MegaDepth ScanNet HPatches InLoc Aachen v1.1
Metrics: AUC of pose error @5/10/20 degrees AUC of reprojection error @3/5/10px localization accuracy at 0.25m/0.5m/1.0m thresholds running time (ms)
Baselines: SuperPoint + NN SuperPoint + SuperGlue SuperPoint + LightGlue DRC-Net LoFTR QuadTree Attention MatchFormer TopicFM AspanFormer DKM ROMA Sparse-NCNet
Builds on: LoFTR RepVGG RoPE SuperGlue LightGlue
Closest papers by meaning
- HomoMatcher: Achieving Dense Feature Matching with Semi-Dense Efficiency by Homography Estimation (AAAI 2025 · similarity 0.88)
- ETO:Efficient Transformer-based Local Feature Matching by Organizing Multiple Homography Hypotheses (NeurIPS 2024 · similarity 0.86)
- JamMa: Ultra-lightweight Local Feature Matching with Joint Mamba (CVPR 2025 · similarity 0.85)
- Dense Match Summarization for Faster Two-view Estimation (CVPR 2025 · similarity 0.85)
- RoMa: Robust Dense Feature Matching (CVPR 2024 · similarity 0.84)
- Unifying Feature and Cost Aggregation with Transformers for Semantic and Visual Correspondence (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.