Research map / Architectures and efficiency / RNNs and normalization
VPDETR: End-to-End Vanishing Point DEtection TRansformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VPDETR, an end-to-end DETR-style transformer that directly regresses vanishing points from images using dynamically updated anchor-point queries, plus orthogonal and cross-prediction losses for Manhattan-world data.
Problem
Vanishing point detection has relied on multi-stage pipelines (line extraction, clustering, candidate sampling and classification) that are not truly end-to-end and need post-processing; the task must also work for both Manhattan and non-Manhattan worlds.
Approach
Treat vanishing point detection as set prediction: use positional embeddings of learnable Gaussian-sphere anchor points as decoder queries, refine them layer by layer as in DAB-DETR, and add orthogonality-based cross-prediction and orthogonal losses for Manhattan scenes.
Key results
- On SU3, VPDETR reaches 92.8 AA@3 (96.4 for the 400-query variant) at 16 FPS, versus 94.4 AA@3 for NeurVPS at 0.5 FPS.
- On NYU Depth with top-k=#pred, VPDETR gets AUC@10 of 80.81, above Lin et al. at 71.62.
- Orthogonal and cross-prediction losses raise VPDETR's SU3 AA@3 from 86.0 to 88.1 (ablation on the last-layer feature map).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use VPDETR with 400 queries and bipartite matching for selection instead of standard inference selection | VPDETR · SU3 | AA@3°: 92.8 -> 96.4 | Helped |
| Add orthogonal loss (ablation) | vanilla DETR · SU3 | AA@3°: 85.2 -> 86.1 | Helped |
| Add cross-prediction loss alongside orthogonal loss (ablation) | vanilla DETR · SU3 | AA@3°: 86.1 -> 86.5 | Helped |
| Use VPDETR architecture instead of vanilla DETR (ablation) | vanishing point detection on SU3 · SU3 | AA@3°: 86.5 -> 86.0 | Helped |
| Add orthogonal loss (ablation) | VPDETR · SU3 | AA@3°: 86.0 -> 86.5 | Helped |
| Add cross-prediction loss alongside orthogonal loss (ablation) | VPDETR · SU3 | AA@3°: 86.5 -> 88.1 | Helped |
| Use C4 features instead of C5 features (ablation) | VPDETR · SU3 | AA@3°: 88.1 -> 92.2 | Helped |
| Use C3 features instead of C4 features (ablation) | VPDETR · SU3 | AA@3°: 92.2 -> 92.8 | Helped |
| Use C5 and C4 multi-scale features instead of C3 features (ablation) | VPDETR · SU3 | AA@3°: 92.8 -> 89.7 | Hurt |
| Use C5, C4, and C3 multi-scale features instead of C3 features (ablation) | VPDETR · SU3 | AA@3°: 92.8 -> 92.5 | Hurt |
| Use VPDETR instead of Lin et al. | NYU Depth · NYU Depth (top-k=#gt) | AUC @5°: 55.92 -> 54.56 | Hurt |
| Use VPDETR instead of Lin et al. | NYU Depth · NYU Depth (top-k=#gt) | AUC @10°: 69.57 -> 72.32 | Helped |
Surprising findings
- On ScanNet, the 400-query variant trails on some metrics less than expected; the base model's YUD performance is weaker than several baselines due to domain gap.
Limitations
- Weaker generalization on YUD due to domain gap.
- Manhattan-specific losses are used only for Manhattan-world datasets.
- Post-processing thresholds are still used at inference for Manhattan datasets.
Experimental setup
Datasets: SU3 ScanNet YUD SVVP NYU Depth
Benchmarks: SU3 ScanNet SVVP YUD NYU Depth
Metrics: angle accuracy (AA) at 3/5/10 degrees AUC@5/10 FPS
Baselines: J-Linkage Simon et al. Wu et al. Lu et al. Li et al. CONSAC TLC NeurVPS Lin et al. VaPiD T-Linkage
Builds on: DETR DAB-DETR Deformable DETR NeurVPS
Closest papers by meaning
- End-to-End Real-Time Vanishing Point Detection with Transformer (AAAI 2024 · similarity 0.93)
- Convex Relaxation for Robust Vanishing Point Estimation in Manhattan World (CVPR 2025 · similarity 0.87)
- Point Transformer V3: Simpler Faster Stronger (CVPR 2024 · similarity 0.85)
- DAPoinTr: Domain Adaptive Point Transformer for Point Cloud Completion (AAAI 2025 · similarity 0.84)
- Towards Implicit Aggregation: Robust Image Representation for Place Recognition in the Transformer Era (NeurIPS 2025 · similarity 0.84)
- DOCTR: Disentangled Object-Centric Transformer for Point Scene Understanding (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.