Research map / Detection and segmentation / CNN features
End-to-End Real-Time Vanishing Point Detection with Transformer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VPTR, a transformer-based end-to-end vanishing point detector that treats VP detection as point object detection on a Gaussian hemisphere with region division and a decoder bridged directly to multi-level CNN features, running at 140 FPS.
Problem
Vanishing point detection is typically line-based or relies on Manhattan assumptions and candidate sampling, which trades accuracy against speed and fails when lines are hard to detect or the number of VPs is unknown.
Approach
Model VP detection as one-stage point object detection on the Gaussian hemisphere: Fibonacci-lattice anchors define overlapping spherical caps, each query predicts a confidence and a VP location, and a deformable decoder attends directly to multi-level backbone features with no encoder.
Key results
- On SU3, VPTR reaches AA@3° 88.5 and AA@5° 92.9 at 140 FPS on one NVIDIA 3090, versus 94.4 AA@3° for NeurVPS at 1.1 FPS.
- On ScanNet, VPTR achieves AA@5° 42.2 and AA@10° 64.5, slightly above Lin et al. (42.0 and 63.7).
- Decoder bridge plus Manhattan loss and multi-level features raise SU3 AUC from 32.8 to 45.8 in the ablation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use VPTR instead of NeurVPS | vanishing point detection · SU3 | AA@3°: 94.4 -> 88.5 | Hurt |
| use VPTR instead of Lin et al. | vanishing point detection · ScanNet | AA@5°: 42.0 -> 42.2 | Helped |
| use VPTR instead of NeurVPS | vanishing point detection · Natural Scene | AA@2°: 50.3 -> 52.0 | Helped |
| use VPTR instead of NeurVPS | vanishing point detection · Natural Scene | AA@10°: 85.5 -> 84.3 | Hurt |
| use VPTR instead of Lin et al. | vanishing point detection · NYU | AA@10°: 69.5 -> 69.7 | Helped |
| add decoder bridge (ablation) | VPTR baseline · SU3 | AA@3°: 83.4 -> 87.3 | Helped |
| add Manhattan loss (ablation) | VPTR with decoder bridge · SU3 | AA@3°: 87.3 -> 88.1 | Helped |
| add multi-level features (ablation) | VPTR with decoder bridge and Manhattan loss · SU3 | AA@3°: 88.1 -> 88.5 | Helped |
| use 128 anchors instead of 256 (ablation) | VPTR · SU3 | AA@3°: 88.5 -> 87.9 | Hurt |
| use 512 anchors instead of 256 (ablation) | VPTR · SU3 | AA@3°: 88.5 -> 88.1 | Hurt |
| use 8 referring points instead of 32 (ablation) | VPTR · SU3 | AA@3°: 88.5 -> 88.2 | Hurt |
| use 128 referring points instead of 32 (ablation) | VPTR · SU3 | AA@3°: 88.5 -> 88.4 | No clear effect |
Surprising findings
- Removing the encoder improves accuracy, so the encoder was not needed in VPTR, contrary to typical DETR-style designs.
- VPTR underperforms on the York Urban Dataset due to few training images, showing limits in data-scarce settings.
Limitations
- Performance drops on datasets with very few training images such as York Urban Dataset.
- Assumes each anchor detects at most one vanishing point and that VPs are not very close.
- Accuracy on ScanNet and Natural Scene is comparable rather than superior to NeurVPS and Lin et al.
Experimental setup
Datasets: SU3 ScanNet Natural Scene dataset NYU
Benchmarks: SU3 ScanNet Natural Scene dataset NYU
Metrics: AA@3° AA@5° AA@10° AA@1° AA@2° FPS
Baselines: J-Linkage Simon et al. Li et al. CONSAC NeurVPS TLC Lin et al. Zhou et al.
Builds on: Deformable DETR NeurVPS DETR YOLO
Closest papers by meaning
- VPDETR: End-to-End Vanishing Point DEtection TRansformers (AAAI 2024 · similarity 0.93)
- Convex Relaxation for Robust Vanishing Point Estimation in Manhattan World (CVPR 2025 · similarity 0.86)
- DiskVPS: Vanishing Point Detector via Hough Transform in a Disk Region (CVPR 2025 · similarity 0.85)
- Point Transformer V3: Simpler Faster Stronger (CVPR 2024 · similarity 0.85)
- VGGT: Visual Geometry Grounded Transformer (CVPR 2025 · similarity 0.84)
- PointCFormer: A Relation-Based Progressive Feature Extraction Network for Point Cloud Completion (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.