Research map / 3D vision / Depth and stereo
ScaleLSD: Scalable Deep Line Segment Detection Streamlined
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Streamlines self-supervised deep line segment detection (HAT field with a ViT-Base/DPT backbone, HAT-induced proposal verification, and LSD-Rectifier pseudo-labeling) and scales it to 10M unlabeled SA-1B images.
Problem
Self-supervised line segment detection has been limited by small training sets (~20k images) and by the low recall and cost of homographic-adaptation pseudo-labeling, so learned detectors generalize poorly to natural images.
Approach
Replace homographic adaptation with an LSD-Rectifier that swaps the HAT field's orientation component for the classical LSD level-line orientation to generate high-recall pseudo labels, and verify proposals by HAT-induced support degree (pixel counts) instead of learned LOI scores, enabling large-scale training with a Transformer backbone.
Key results
- On zero-shot repeatability (Rep-5 strict) on YorkUrban, ScaleLSD@SA1B reaches 0.725 vs 0.711 for HAWPv3 and produces ~708 lines/image vs 225 for HAWPv3.
- On vanishing point estimation on YUD+, ScaleLSD@SA1B gets VP error 1.55 and AUC 87.1 vs 1.63 and 85.6 for DeepLSD.
- LSD-Rectifier pseudo labels yield 78 lines/image in 0.607 s per batch versus 35 lines in 3.65 s for 10-iteration homographic adaptation on Wireframe.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train ScaleLSD on SA1B at scale | ScaleLSD · YorkUrban, HPatches, RDNIM, and COCO Val2017 | Rep-5 (S): 0.711 -> 0.725; 0.322 -> 0.367; 0.295 -> 0.337; 0.644 -> 0.666 | Helped |
| Use ScaleLSD@SA1B for vanishing-point estimation | ScaleLSD · YUD+ | VP Error: 1.63 -> 1.55 | Helped |
| Use ScaleLSD@SA1B for vanishing-point estimation | ScaleLSD · NYU-VP | AUC: 69.1 (6.2) -> 70.4 (1.4) | Helped |
| Use ScaleLSD detections with GlueStick | GlueStick · ETH3D | line matching AP: 82.0 -> 85.9 | Helped |
| Use ScaleLSD for 3D line reconstruction | LiMAP · DTU Avg. | ACC-L: 0.8479 -> 0.7091 | Helped |
| Add LSD-Rectifier to pseudo-label generation (ablation) | ScaleLSD · Wireframe | Rep5 (S): 0.397 -> 0.445 | Helped |
| Add homographic adaptation to pseudo-label generation (ablation) | ScaleLSD · Wireframe | Rep5 (S): 0.397 -> 0.447 | Helped |
| Add both LSD-Rectifier and homographic adaptation (ablation) | ScaleLSD · Wireframe | Rep5 (S): 0.397 -> 0.473 | Helped |
| Replace LOI-based verification with HAT-induced verification (ablation) | ScaleLSD with Hourglass · Wireframe | Rep-5 (S): 0.356 -> 0.418 | Helped |
| Use DPT instead of Hourglass with LOI-based verification (ablation) | ScaleLSD · Wireframe | Rep-5 (S): 0.356 -> 0.263 | Hurt |
| Use DPT with HAT-induced verification (ablation) | ScaleLSD · Wireframe | Rep-5 (S): 0.418 -> 0.445 | Helped |
| Vary the number of homographic-adaptation iterations (ablation) | ScaleLSD pseudo-label generation · Wireframe | # Lines/Image: 1 iteration: 35; 2: 27; 3: 39; 4: 33; 5: 30; 6: 28; 7: 33; 8: 30; 9: 29; 10: 28 | Mixed |
Surprising findings
- Homographic adaptation with 10 iterations produces fewer lines per image than the LSD-Rectifier while costing much more time and memory.
- Under the LOI-based verification, a larger DPT backbone gives little gain over Hourglass, whereas HAT-induced verification scales with the backbone.
Limitations
- Scalability to even larger datasets was not explored, risking under-detection on test images.
- Curved structures are not explicitly modeled; curves are only represented as polylines.
- Supervised baselines such as L-CNN were excluded from zero-shot comparison due to poor generalization.
Experimental setup
Datasets: Synthetic primitives (16k) Wireframe SA-1B
Benchmarks: HPatches RDNIM YorkUrban COCO val2017 YUD+ NYU-VP ETH3D DTU
Metrics: repeatability localization error length repeatability number of lines per image VP error AUC matching precision-recall AP ACC COMP
Baselines: LSD SOLD2 HAWPv3 DeepLSD TP-LSD L-CNN GlueStick
Builds on: HAWPv3 DeepLSD LSD SOLD2 LiMAP
Closest papers by meaning
- DeepLSD: Line Segment Detection and Refinement With Deep Image Gradients (CVPR 2023 · similarity 0.92)
- Towards Light-Weight and Real-Time Line Segment Detection (AAAI 2022 · similarity 0.89)
- Learning Attraction Field Representation for Robust Line Segment Detection (CVPR 2019 · similarity 0.88)
- SOLD2: Self-Supervised Occlusion-Aware Line Description and Detection (CVPR 2021 · similarity 0.87)
- Improving Transformer Based Line Segment Detection with Matched Predicting and Re-ranking (AAAI 2025 · similarity 0.85)
- Semantic Line Combination Detector (CVPR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.