Research map / 3D vision / Face and shape
AlphaTablets: A Generic Plane Representation for 3D Planar Reconstruction from Monocular Videos
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces AlphaTablets, rectangles with alpha channels as a 3D plane representation, with differentiable rasterization and a bottom-up superpixel-initialized optimize-and-merge pipeline for planar reconstruction from monocular videos on ScanNet.
Problem
Accurate, complete and generalizable 3D planar reconstruction from monocular video; existing 2D mask, point, surfel, voxel and mesh plane representations suffer from view inconsistency, discontinuity or poor boundary modeling.
Approach
Represent each 3D plane as a textured rectangle with a learnable alpha channel, render it differentiably via NVDiffrast-based multi-layer rasterization with a modified anti-aliasing for semi-transparent layers, and build planes bottom-up from superpixel tablets by alternating photometric/geometric optimization with a KD-tree union-find merging scheme.
Key results
- On ScanNet, the method reaches F-score 0.456 vs 0.372 for PlanarRecon, the best prior method in the table.
- Plane segmentation VOI 3.468 (best) and RI 0.928 and SC 0.273 vs PlanarRecon's 3.622, 0.897, 0.248.
- Ablation: removing tablet merging drops F-score from 0.456 to 0.188.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use AlphaTablets instead of PlanarRecon | 3D planar reconstruction · ScanNetv2 validation (Atlas split) | F-Score: 0.372 -> 0.456 | Helped |
| add tablet distortion loss (ablation) | AlphaTablets with photometric and AlphaInv losses · ScanNet | F-score: 0.240 -> 0.271 | Helped |
| add normal loss (ablation) | AlphaTablets with tablet distortion loss · ScanNet | F-score: 0.271 -> 0.425 | Helped |
| add depth loss (ablation) | AlphaTablets with normal loss · ScanNet | F-score: 0.425 -> 0.456 | Helped |
| add tablet anti-aliasing (ablation) | AlphaTablets · ScanNet | F-score: 0.415 -> 0.456 | Helped |
| add tablet merging (ablation) | AlphaTablets · ScanNet | F-score: 0.188 -> 0.456 | Helped |
| add in-training merging and initialization merging (ablation) | AlphaTablets · ScanNet | F-score: 0.188 -> 0.456 | Helped |
| add in-training merging (ablation) | AlphaTablets with initialization merging · ScanNet | F-score: 0.438 -> 0.456 | Helped |
| add initialization merging (ablation) | AlphaTablets with in-training merging · ScanNet | F-score: 0.454 -> 0.456 | Helped |
| replace COLMAP initialization with Metric3D initialization (ablation) | SuGaR · ScanNet ablation subset | F-score: 0.300 -> 0.326 | Helped |
Surprising findings
- Metric3D + Seq-RANSAC achieves the best completeness (0.074) but very low precision (0.161), and PlanarRecon retains the best accuracy (0.105) despite low recall.
Limitations
- Struggles in highly non-planar regions where the single-superpixel planar assumption fails.
- No view-dependent effects modeled, so optimization relies on cross-view color consistency that non-Lambertian surfaces or lighting changes can break.
Experimental setup
Datasets: ScanNetv2 Atlas validation split
Benchmarks: ScanNet
Metrics: Comp Acc Recall Precision F-Score VOI RI SC
Baselines: NeuralRecon + Seq-RANSAC Atlas + Seq-RANSAC ESTDepth + PEAC PlanarRecon Metric3D + Seq-RANSAC SuGaR + Seq-RANSAC
Builds on: NVDiffrast SLIC superpixels mip-NeRF 360 distortion loss PlanarRecon
Closest papers by meaning
- Towards In-the-wild 3D Plane Reconstruction from a Single Image (CVPR 2025 · similarity 0.88)
- AirPlanes: Accurate Plane Estimation via 3D-Consistent Embeddings (CVPR 2024 · similarity 0.87)
- PlanarSplatting: Accurate Planar Surface Reconstruction in 3 Minutes (CVPR 2025 · similarity 0.87)
- PLANA3R: Zero-shot Metric Planar 3D Reconstruction via Feed-forward Planar Splatting (NeurIPS 2025 · similarity 0.86)
- World Tracing: Pixel-Aligned Geometry Beyond the Visible (NeurIPS 2026 · similarity 0.85)
- SPAR3D: Stable Point-Aware Reconstruction of 3D Objects from Single Images (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.