Research map / 3D vision / Depth and stereo
VSFormer: Visual-Spatial Fusion Transformer for Correspondence Pruning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VSFormer, which injects scene visual cues (cross-attention over two-view CNN features) fused with spatial correspondence features, plus a ContextFormer (KNN graph with squeeze-excitation attention followed by transformer) for correspondence pruning and camera pose estimation.
Problem
Finding correct correspondences (inliers) among putative matches is hard because inlier ratios vary across image pairs due to visual differences, and existing methods use only spatial coordinates of correspondences.
Approach
Extract scene-level visual cues via cross-attention between local image features of the two views, fuse them with correspondence features through a joint transformer-based visual-spatial fusion module, and capture local and global context with a KNN-graph attention block followed by a transformer (ContextFormer).
Key results
- On YFCC100M unknown scenes without RANSAC, VSFormer reaches 62.18 mAP5° vs 53.70 for PGFNet (2.57M params).
- On SUN3D unknown scenes without RANSAC, VSFormer reaches 20.18 mAP5° vs 19.32 for PGFNet.
- VSFusion alone improves the PointCN baseline from 43.25 to 49.63 mAP5° in the ablation on YFCC100M.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add VSFusion and ContextFormer to the two-iteration baseline (ablation) | PointCN pruning baseline · YFCC100M unknown scenes | mAP5°: 43.25 -> 62.18 | Helped |
| increase network iterations from 2 to 3 (ablation) | VSFormer · YFCC100M unknown scenes | mAP5°: 62.18 -> 55.70 | Hurt |
| add VSFusion to the ResNet encoder baseline (ablation) | PointCN pruning baseline · YFCC100M unknown scenes | mAP5°: 43.25 -> 49.63 | Helped |
| replace ResNet encoder with ContextFormer (ablation) | PointCN pruning baseline · YFCC100M unknown scenes | mAP5°: 43.25 -> 57.55 | Helped |
| add cross-attention layer (ablation) | VSFusion · YFCC100M unknown scenes | mAP5° without/with RANSAC: 57.55/61.82 -> 60.33/61.88 | Helped |
| add transformer layer (ablation) | VSFusion · YFCC100M unknown scenes | mAP5° without/with RANSAC: 57.55/61.82 -> 60.18/61.68 | Helped |
| add spatial projection layer (ablation) | VSFusion · YFCC100M unknown scenes | mAP5° without/with RANSAC: 60.18/61.68 -> 61.23/62.43 | Helped |
| add final element-wise summation (ablation) | VSFusion · YFCC100M unknown scenes | mAP5° without/with RANSAC: 61.23/62.43 -> 62.18/63.35 | Helped |
| add KNN-based graph (ablation) | ContextFormer · YFCC100M unknown scenes | mAP5° without/with RANSAC: 49.63/57.90 -> 51.53/57.78 | Mixed |
| add neighborhood aggregation (ablation) | ContextFormer · YFCC100M unknown scenes | mAP5° without/with RANSAC: 51.53/57.78 -> 53.73/59.88 | Helped |
| add graph attention block (ablation) | ContextFormer · YFCC100M unknown scenes | mAP5° without/with RANSAC: 53.73/59.88 -> 58.93/61.95 | Helped |
| add transformer (ablation) | ContextFormer · YFCC100M unknown scenes | mAP5° without/with RANSAC: 58.93/61.95 -> 59.58/62.65 | Mixed |
Surprising findings
- Increasing network iterations from two to three decreases performance (62.18 to 55.70 mAP5°), which the authors attribute to discarding inliers.
Limitations
- Redundant iterations discard inliers and hurt accuracy.
- Gains on indoor SUN3D are smaller than on outdoor YFCC100M.
Experimental setup
Datasets: YFCC100M SUN3D
Benchmarks: YFCC100M SUN3D
Metrics: mAP@5° mAP@20°
Baselines: PointNet++ PointCN DFE OANet++ ACNe T-Net LMCNet CLNet MSA-Net MS$^2$DG-Net PGFNet RANSAC
Builds on: PointCN CLNet OANet SIFT
Closest papers by meaning
- VLM-Pruner: Buffering for Spatial Sparsity in an Efficient VLM Centrifugal Token Pruning Paradigm (CVPR 2026 · similarity 0.85)
- Graph Context Transformation Learning for Progressive Correspondence Pruning (AAAI 2024 · similarity 0.85)
- SyFormer: Structure-Guided Synergism Transformer for Large-Portion Image Inpainting (AAAI 2024 · similarity 0.84)
- VGGT: Visual Geometry Grounded Transformer (CVPR 2025 · similarity 0.84)
- SplatFormer: Point Transformer for Robust 3D Gaussian Splatting (ICLR 2025 · similarity 0.84)
- Mind-the-Glitch: Visual Correspondence for Detecting Inconsistencies in Subject-Driven Generation (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.