Research map / 3D vision / 3D generation
SVDTree: Semantic Voxel Diffusion for Single Image Tree Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Generates 3D trees from a single masked photo by predicting a semantic voxel structure (trunk/branch/leaf/empty) with a conditional denoising diffusion model, then reconstructing branches and foliage with a hybrid geometric algorithm.
Problem
Reconstructing realistic 3D biological tree geometry, including hidden internal branches, from a single photograph.
Approach
Represent a tree as a 64^3 Semantic Voxel Structure whose values encode trunk, branch, leaf, or empty, predict it with a semantic voxel diffusion model conditioned on ViT image features, and convert it to geometry via skeleton extraction plus space colonization for the crown.
Key results
- Full model reaches mAP 0.546 and mean absolute error 0.0134 on synthetic SVS testing, versus 0.297 mAP and 0.0468 error for cGAN.
- Trunk precision 0.538 and leaf precision 0.461 for the full model, about 40% and 33% above cGAN per the paper.
- Average runtime per image: SAM 12.6 s, SVS prediction 4.4 s, geometry reconstruction 13.7 s.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the diffusion model instead of cGAN (ablation) | SVDTree · Synthetic testing images | mAP: 0.297 -> 0.546 | Helped |
| Use the diffusion model instead of cGAN (ablation) | SVDTree · Synthetic testing images | trunk precision: 0.130 -> 0.538 | Helped |
| Use the diffusion model instead of cGAN (ablation) | SVDTree · Synthetic testing images | branch precision: 0.033 -> 0.176 | Helped |
| Use the diffusion model instead of cGAN (ablation) | SVDTree · Synthetic testing images | leaf precision: 0.137 -> 0.461 | Helped |
| Use the diffusion model instead of cGAN (ablation) | SVDTree · Synthetic testing images | mean absolute error: 0.0468 -> 0.0134 | Helped |
| Add multi-head attention to the mix block (ablation) | SVDTree · Synthetic testing images | mAP: 0.460 -> 0.546 | Helped |
| Add multi-head attention to the mix block (ablation) | SVDTree · Synthetic testing images | branch precision: 0.061 -> 0.176 | Helped |
| Add multi-head attention to the mix block (ablation) | SVDTree · Synthetic testing images | mean absolute error: 0.0169 -> 0.0134 | Helped |
Surprising findings
- Branch class precision remains low (0.176 for the full model), indicating the branch class is hard to predict.
Limitations
- Evaluation of SVS accuracy is only on synthetic procedural test data; real-world results are qualitative.
- Palm and other species hard to synthesize are excluded from training, limiting diversity.
- Relative positioning of multiple trees is arranged manually.
Experimental setup
Datasets: synthetic tree dataset generated with space colonization (4,000 models) real tree photographs (public sources and smartphone)
Benchmarks: 100 synthetic test images with procedural SVS ground truth
Metrics: precision per class (AP_T, AP_B, AP_L, AP_E) mAP absolute error mean and std
Baselines: RBV (Li et al. radial bounding volumes) cGAN (Liu et al.) NeRF
Builds on: DDPM Segment Anything Model space colonization algorithm U-Net Vision Transformer
Closest papers by meaning
- Neural Hierarchical Decomposition for Single Image Plant Modeling (CVPR 2025 · similarity 0.85)
- LT3SD: Latent Trees for 3D Scene Diffusion (CVPR 2025 · similarity 0.84)
- OAM-TCD: A globally diverse dataset of high-resolution tree cover maps (NeurIPS 2024 · similarity 0.83)
- COCOTree: A Dataset and Benchmark for Open Tree-Structured Visual Decomposition (NeurIPS 2026 · similarity 0.83)
- Reconstructing Animals and the Wild (CVPR 2025 · similarity 0.83)
- BioCLIP: A Vision Foundation Model for the Tree of Life (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.