Research map / 3D vision / 3D generation
FocalDreamer: Text-Driven 3D Editing via Focal-Fusion Assembly
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FocalDreamer, a text-driven 3D editing framework that adds separate learnable editable parts within user-specified ellipsoid focal regions of a base mesh, using DMTet geometry, soft union, dual-path PBR texturing, and SDS with Stable Diffusion.
Problem
Text-driven 3D object editing that is separable (editable parts can be reused), precise (edits confined to desired regions), and consistent (harmonious appearance with the base shape), where prior score-distillation methods modify the whole shape or entangle geometry and texture.
Approach
Instead of directly modifying the base shape, the method generates a separate learnable part initialized as an ellipsoid in a user focal region, merges it with the frozen base SDF via a soft geometry union, and uses a geometric focal loss, collision avoidance loss, dual-path PBR texture rendering with a pixel-wise discriminative mask, and a style consistency regularizer, with two-stage geometry th
Key results
- FocalDreamer achieves CLIP_sim 0.329 and CLIP_dir 0.0519 across 15 scenes, versus 0.293 to 0.299 and 0.0178 to 0.0204 for baselines.
- User study with 65 participants gives mean 9.14 for source preservation and 8.40 for prompt relevance.
- Ablation shows removing dual-path rendering drops CLIP_sim from 0.329 to 0.313.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FocalDreamer instead of Vox-E-Global | 3D text-driven editing · Synthetic Object Dataset (15 scenes) | CLIP_sim: 0.299 -> 0.329 | Helped |
| Use FocalDreamer instead of Vox-E-Global | 3D text-driven editing · Synthetic Object Dataset (15 scenes) | CLIP_dir: 0.0204 -> 0.0519 | Helped |
| Add style consistency loss (ablation) | FocalDreamer · Synthetic Object Dataset (15 scenes; scenes requiring style consistency) | CLIP_sim: 0.312* -> 0.319* | Helped |
| Add style consistency loss (ablation) | FocalDreamer · Synthetic Object Dataset (15 scenes; scenes requiring style consistency) | CLIP_dir: 0.0402* -> 0.0495* | Helped |
| Add geometric focal loss (ablation) | FocalDreamer · Synthetic Object Dataset (15 scenes) | CLIP_dir: 0.0433 -> 0.0519 | Helped |
| Add collision avoidance loss (ablation) | FocalDreamer · Synthetic Object Dataset (15 scenes) | CLIP_dir: 0.0517 -> 0.0519 | No clear effect |
| Use dual-path rendering instead of single-path rendering (ablation) | FocalDreamer · Synthetic Object Dataset (15 scenes) | CLIP_dir: 0.0401 -> 0.0519 | Helped |
Limitations
- Evaluated only on 15 self-assembled synthetic meshes
- Relies on user-specified ellipsoid focal regions and a rough prior
- Vox-E sometimes misidentifies focal regions in baseline comparisons
Experimental setup
Datasets: Synthetic Object Dataset (15 meshes)
Benchmarks: Synthetic Object Dataset (15 scenes)
Metrics: CLIP similarity CLIP direction similarity user preference score
Baselines: Fantasia3D* Vox-E Vox-E-Global Instruct Pix2Pix ControlNet-inpainting
Builds on: DreamFusion DMTet Fantasia3D Vox-E
Closest papers by meaning
- PrEditor3D: Fast and Precise 3D Shape Editing (CVPR 2025 · similarity 0.87)
- IPDreamer: Appearance-Controllable 3D Object Generation with Complex Image Prompts (ICLR 2025 · similarity 0.86)
- Progressive3D: Progressively Local Editing for Text-to-3D Content Creation with Complex Semantic Prompts (ICLR 2024 · similarity 0.86)
- Layout-your-3D: Controllable and Precise 3D Generation with 2D Blueprint (ICLR 2025 · similarity 0.86)
- EditSplat: Multi-View Fusion and Attention-Guided Optimization for View-Consistent 3D Scene Editing with 3D Gaussian Splatting (CVPR 2025 · similarity 0.85)
- ConsistDreamer: 3D-Consistent 2D Diffusion for High-Fidelity Scene Editing (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.