Research map / 3D vision / 3D generation
MPOD123: One Image to 3D Content Generation Using Mask-enhanced Progressive Outline-to-Detail Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MPOD123, a two-stage single-image-to-3D method that first optimizes a NeRF outline shape with Zero-1-to-3 and then refines a DMTet mesh with stable diffusion using mask-rectified cross-attention (MRCA).
Problem
Single-image 3D content generation via 2D diffusion priors often yields distorted outline shapes and poor local detail, and directly combining view-conditioned and text-to-image priors entangles shape and appearance.
Approach
Disentangle outline shape and detail optimization: stage 1 uses Zero-1-to-3 SDS on a NeRF and derives 2D opacity masks; stage 2 uses Detail Appearance Inpainting with Mask Rectified Cross-Attention on the special textual-inversion token to refine a DMTet mesh while keeping the mask outline.
Key results
- On RealFusion15, MPOD123 achieves CX 1.54, CLIP 0.86, MVC 0.90, the best among compared methods.
- On TBPR-Shoes, MPOD123 achieves CX 1.29, CLIP 0.78, MVC 0.86, outperforming Magic123 (CLIP 0.71).
- Ablations show MRCA is lighter and gives better local texture than a ControlNet mask-conditioning alternative, and DAI mitigates the multi-face Janus problem.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MPOD123 instead of Magic123 | Single-image 3D generation · RealFusion15 | CX distance: 1.65 -> 1.54 | Helped |
| Use MPOD123 instead of Magic123 | Single-image 3D generation · RealFusion15 | CLIP score: 0.83 -> 0.86 | Helped |
| Use MPOD123 instead of Zero-1-to-3 | Single-image 3D generation · RealFusion15 | MVC score: 0.89 -> 0.90 | Helped |
| Use MPOD123 instead of Magic123 | Single-image 3D generation · TBPR-Shoes | CX distance: 1.33 -> 1.29 | Helped |
| Use MPOD123 instead of Magic123 | Single-image 3D generation · TBPR-Shoes | CLIP score: 0.71 -> 0.78 | Helped |
| Use MPOD123 instead of Zero-1-to-3 | Single-image 3D generation · TBPR-Shoes | MVC score: 0.84 -> 0.86 | Helped |
| Add the second-stage detail appearance optimization after outline-shape optimization (ablation) | MPOD123 · Novel views | local geometry and texture detail quality | Helped |
| Add Detail Appearance Inpainting (DAI) in the second stage (ablation) | MPOD123 · Novel views | outline shape preservation | Helped |
| Use Mask Rectified Cross-Attention instead of ControlNet for mask injection (ablation) | MPOD123 · Novel views | local texture detail quality | Helped |
Limitations
- Evaluation relies on only 15 and 17 test examples, with the new TBPR-Shoes dataset limited to shoes.
- Outline shape depends on the quality of Zero-1-to-3 priors, which the authors note as a future direction.
- Metrics are CLIP-based and CX distance; no human evaluation is reported.
Experimental setup
Datasets: RealFusion15 TBPR-Shoes
Benchmarks: RealFusion15 TBPR-Shoes
Metrics: CX distance CLIP score multi-view consistency (MVC) CLIP score
Baselines: RealFusion Make-it-3D Zero-1-to-3 Magic123
Builds on: Zero-1-to-3 DreamFusion RealFusion DMTet Instant-NGP
Closest papers by meaning
- Sculpt3D: Multi-View Consistent Text-to-3D Generation with Sparse 3D Prior (CVPR 2024 · similarity 0.85)
- GuideFlow3D: Optimization-Guided Rectified Flow For Appearance Transfer (NeurIPS 2025 · similarity 0.85)
- ViewDiff: 3D-Consistent Image Generation with Text-to-Image Models (CVPR 2024 · similarity 0.85)
- MeshGen: Generating PBR Textured Mesh with Render-Enhanced Auto-Encoder and Generative Data Augmentation (CVPR 2025 · similarity 0.85)
- DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior (ICLR 2024 · similarity 0.85)
- Kiss3DGen: Repurposing Image Diffusion Models for 3D Asset Generation (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.