Research map / 3D vision / 3D generation
One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Turns a single image into a textured 3D mesh in about one minute by fine-tuning Stable Diffusion to generate six tiled consistent views, lifting them with multi-view-conditioned two-stage 3D diffusion, and refining texture with TensoRF.
Problem
Image-to-3D generation that is both fast and faithful to the input image; prior per-shape optimization is slow, and Zero123-based multi-view generation is inconsistent across views, hurting feed-forward reconstruction.
Approach
Generate six absolute-pose views jointly as one tiled composite image with a fine-tuned Stable Diffusion, then lift them to 3D with a multi-view-local-feature-conditioned coarse-to-fine SDF/color volume diffusion model, followed by a lightweight TensoRF texture refinement supervised by the consistent views.
Key results
- On GSO (1,030 objects), One-2-3-45++ reaches F-Score 93.6%, CLIP-Sim 81.0 and 87.6% user preference, versus 91.6 / 73.1 / 58.6% for Zero123 XL, in about 60 s total.
- Multi-view generation on GSO achieves PSNR 22.12 and mask IoU 0.878, above Zero123 (20.32, 0.856) and SyncDreamer (21.67, 0.894 mask IoU is lower than ours in PSNR only).
- In text-to-3D on 50 DreamFusion prompts, the method gets CLIP-Sim 26.8 and 84.1% user preference, versus 25.7 for ProlificDreamer (10h+ runtime).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace consistent multi-view generation with Zero123 XL (ablation) | One-2-3-45++ · GSO | CLIP similarity: 71.9 vs. 73.4 | Hurt |
| Replace 3D diffusion reconstruction with SparseNeuS (ablation) | One-2-3-45++ · GSO | F-Score: 93.6 -> 81.2 | Hurt |
| Add texture refinement (ablation) | One-2-3-45++ · GSO | CLIP similarity: 73.4 -> 81.0 | Helped |
| Remove multi-view conditions and rely on the global CLIP feature from a single input view (ablation) | 3D diffusion module · 3D shell occupancy prediction | 3D IoU: 45.1 vs. 18.3 | Hurt |
| Use global multi-view features instead of local multi-view features (ablation) | 3D diffusion module · 3D shell occupancy prediction | 3D IoU: 45.1 vs. 24.4 | Hurt |
| Remove the global CLIP feature of the input view (ablation) | 3D diffusion module · 3D shell occupancy prediction | 3D IoU: 44.1 -> 41.4 | Hurt |
| Use predicted rather than ground-truth rendered multi-view images during training (ablation) | 3D diffusion module · 3D shell occupancy prediction | 3D IoU: 44.1 -> 41.9 | Hurt |
| Remove random projection-matrix perturbations during training (ablation) | 3D diffusion module · 3D shell occupancy prediction | 3D IoU: 45.1 -> 44.1 | Hurt |
Surprising findings
- Feeding predicted (rather than ground-truth) multi-view images during 3D diffusion training hurts performance, so ground-truth renderings are used for training with pose perturbation instead.
- Using only global CLIP features from multiple views, without multi-view local features, does not improve over single-view conditioning.
Limitations
- Geometry robustness and detail are limited, as the conclusion notes, and could be improved with additional guiding conditions.
- Training of the 3D diffusion module with predicted multi-view images degrades results, so it relies on ground-truth renderings and perturbations to handle imperfect inputs.
- Relies on fixed absolute camera elevations and six views, so coverage is limited to these poses.
Experimental setup
Datasets: Objaverse GSO
Benchmarks: GSO DreamFusion 50 text prompts
Metrics: F-Score CLIP similarity user preference runtime PSNR LPIPS mask IoU 3D IoU
Baselines: Zero123 XL One-2-3-45 SyncDreamer DreamGaussian Shap-E ProlificDreamer MVDream Wonder3D
Builds on: One-2-3-45 Zero123 LAS-Diffusion Stable Diffusion TensoRF
Closest papers by meaning
- Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Image (NeurIPS 2024 · similarity 0.92)
- Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors (ICLR 2024 · similarity 0.91)
- Cycle3D: High-quality and Consistent Image-to-3D Generation via Generation-Reconstruction Cycle (AAAI 2025 · similarity 0.91)
- Sharp-It: A Multi-view to Multi-view Diffusion Model for 3D Synthesis and Manipulation (CVPR 2025 · similarity 0.91)
- Kiss3DGen: Repurposing Image Diffusion Models for 3D Asset Generation (CVPR 2025 · similarity 0.91)
- Wonder3D: Single Image to 3D using Cross-Domain Diffusion (CVPR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.