Research map / 3D vision / 3D generation
Ouroboros3D: Image-to-3D Generation via 3D-aware Recursive Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Ouroboros3D jointly trains a video diffusion multi-view generator (SVD) and an LGM Gaussian reconstruction model in a recursive loop where rendered color and CCM maps from the reconstruction feed back as conditions to the next denoising step.
Problem
Image-to-3D pipelines that separate multi-view generation from sparse-view reconstruction suffer from multi-view 3D inconsistency and a train/inference domain gap in the reconstruction stage.
Approach
Integrates multi-view diffusion and feed-forward 3D Gaussian reconstruction into one end-to-end trainable recursive diffusion process, using a 3D-aware self-conditioning feedback (rendered RGB and canonical coordinate maps) and joint training so the reconstruction model learns from generated images.
Key results
- Ouroboros3D multi-view generation reaches PSNR 21.770 / SSIM 0.8866 / LPIPS 0.1093 on GSO, beating SV3D (21.042 / 0.8497 / 0.1296) and VideoMV (18.605).
- Image-to-3D 3DGS results reach PSNR 21.761 vs InstantMesh 19.948 and LGM 17.716 on GSO.
- Ablation: joint training plus CCM and RGB feedback raises PSNR from 20.012 to 21.761 and cuts the generated-vs-3DGS PSNR gap from 1.067 to 0.009.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Ouroboros3D instead of SV3D | Multi-view generation · GSO, multi-view images | PSNR: 21.042 -> 21.770 | Helped |
| Use Ouroboros3D instead of InstantMesh | Image-to-3D generation · GSO, reconstructed 3DGS | PSNR: 19.948 -> 21.761 | Helped |
| Add joint training without feedback (ablation) | Ouroboros3D · GSO, reconstructed 3DGS | PSNR: 20.012 -> 20.549 | Helped |
| Add CCM feedback to joint training (ablation) | Ouroboros3D · GSO, reconstructed 3DGS | PSNR: 20.549 -> 21.325 | Helped |
| Add RGB feedback to joint training (ablation) | Ouroboros3D · GSO, reconstructed 3DGS | PSNR: 20.549 -> 21.542 | Helped |
| Add both CCM and RGB feedback to joint training (ablation) | Ouroboros3D · GSO, reconstructed 3DGS | PSNR: 20.549 -> 21.761 | Helped |
| Exclude feedback for the first 10 denoising steps (ablation) | Ouroboros3D · GSO, reconstructed 3DGS | PSNR: 21.761 -> 21.633 | No clear effect |
| Exclude feedback for the first 20 denoising steps (ablation) | Ouroboros3D · GSO, reconstructed 3DGS | PSNR: 21.761 -> 20.805 | Hurt |
Surprising findings
- Excluding feedback for the first 10 of 25 steps gives comparable results, while excluding it for 20 steps degrades performance noticeably.
Limitations
- Joint training increases training time and GPU memory compared with training the two stages separately.
- Inference takes about 25 seconds per model, slower than the feed-forward LGM baseline (1.2 s).
- Output representation is 3D Gaussian Splatting; mesh output requires swapping the reconstruction module.
Experimental setup
Datasets: Objaverse GSO
Benchmarks: GSO
Metrics: PSNR SSIM LPIPS
Baselines: SyncDreamer SV3D VideoMV TripoSR LGM InstantMesh
Builds on: Stable Video Diffusion LGM self-conditioning T2I-Adapter
Closest papers by meaning
- Cycle3D: High-quality and Consistent Image-to-3D Generation via Generation-Reconstruction Cycle (AAAI 2025 · similarity 0.92)
- MVD-Fusion: Single-view 3D via Depth-consistent Multi-view Generation (CVPR 2024 · similarity 0.90)
- Sculpt3D: Multi-View Consistent Text-to-3D Generation with Sparse 3D Prior (CVPR 2024 · similarity 0.90)
- HarmonyView: Harmonizing Consistency and Diversity in One-Image-to-3D (CVPR 2024 · similarity 0.90)
- One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion (CVPR 2024 · similarity 0.89)
- Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Image (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.