Research map / Image generation / Text-to-image diffusion
Progressive3D: Progressively Local Editing for Text-to-3D Content Creation with Complex Semantic Prompts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Progressive3D decomposes complex-prompt text-to-3D generation into sequential region-constrained local editing steps, with overlapped semantic component suppression on the score-distillation guidance, and evaluates on a new 100-prompt CSP-100 set.
Problem
Text-to-3D methods fail on complex prompts describing multiple interacting objects with different attributes, causing object missing, attribute mismatch, and quality loss.
Approach
Progressive region-constrained local editing of a source 3D representation (2D masks from user bounding boxes, consistency and initialization losses) combined with suppressing the component of the target prompt's denoising direction that overlaps with the source prompt's direction (OSCS) inside SDS.
Key results
- User study with 20 participants preferred Progressive3D over DreamTime in 83.2% of cases versus 16.8%.
- Ablation on DreamTime shows L_initial and OSCS each improve metrics over baseline, and combining both gives further gains on CSP-100.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Progressive3D's progressive local editing to DreamTime | DreamTime · CSP-100 | human preference: 16.8% vs. 83.2% | Helped |
| Add the content consistency constraint to preserve content outside editable regions (ablation) | Progressive3D · CSP-100 | local editing fidelity | Helped |
| Add the content initialization constraint (ablation) | Progressive3D with DreamTime · CSP-100 | creation performance | Helped |
| Add OSCS to suppress overlapped semantic components (ablation) | Progressive3D with DreamTime · CSP-100 | creation performance | Helped |
| Add both the content initialization constraint and OSCS (ablation) | Progressive3D with DreamTime · CSP-100 | creation performance | Helped |
| Fine-tune directly with the target prompt instead of using Progressive3D local editing (ablation) | TextMesh · Progressive editing | preservation of source objects | Hurt |
| Increase the OSCS suppression weight W (ablation) | Progressive3D with OSCS · Complex semantic prompts | 3D content desirability | Helped |
| Set OSCS suppression weight W to 0.5, increasing the influence of overlapped semantics (ablation) | Progressive3D with OSCS · Complex semantic prompts | 3D content precision: W=0.5 | Hurt |
| Constrain background and foreground together rather than separating empty regions from content regions (ablation) | Progressive3D · Local 3D editing | floating artifacts | Hurt |
Surprising findings
- CLIP-based metrics fail to capture fine-grained object-attribute correspondence, while BLIP-VQA performs well on the same examples.
Limitations
- Decomposition into multiple editing steps multiplies time cost and requires more human intervention.
- Output quality is bounded by the generative capacity of the base text-to-3D method and its T2I prior.
- Requires user-defined region prompts (bounding boxes, meshes, or segmentations) for each step.
Experimental setup
Datasets: CSP-100
Benchmarks: CSP-100
Metrics: BLIP-VQA mGPT-CoT CLIP score user preference
Baselines: DreamTime TextMesh Fantasia3D MVDream Composing Energy-Based Model Attend-and-Excite
Builds on: DreamFusion DreamTime Attend-and-Excite threestudio Score Debiasing
Closest papers by meaning
- EfficientDreamer: High-Fidelity and Robust 3D Creation via Orthogonal-view Diffusion Priors (CVPR 2024 · similarity 0.89)
- VP3D: Unleashing 2D Visual Prompt for Text-to-3D Generation (CVPR 2024 · similarity 0.88)
- SceneFactor: Factored Latent 3D Diffusion for Controllable 3D Scene Generation (CVPR 2025 · similarity 0.88)
- Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation (ICLR 2024 · similarity 0.88)
- DreamDPO: Aligning Text-to-3D Generation with Human Preferences via Direct Preference Optimization (ICML 2025 · similarity 0.88)
- Interactive3D: Create What You Want by Interactive 3D Generation (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.