Research map / Image generation / Editing and style transfer
Rethinking Score Distilling Sampling for 3D Editing and Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Unified Distillation Sampling (UDS), an SDS variant whose gradient combines an x0-based identity/reconstruction term with classifier-free guidance, enabling both text-guided 3D editing and text-to-3D generation.
Problem
SDS supports only 3D generation, while existing SDS-based editing variants (DDS, PDS) fail to generate new 3D assets well and suffer from averaging/over-smoothing and high CFG requirements.
Approach
Decompose DDS and PDS gradients into a reconstruction (identity) term and a classifier-free guidance term, then unify them into one gradient formula: a difference of Tweedie/DDIM-approximated x0 estimates plus a CFG term with a small weight (7.5), used for both editing (src vs tgt) and generation (x0 at t vs t-c, plus CFG and negative CFG).
Key results
- 3D editing: UDS CLIP score 0.2498 and 41.52% user preference vs PDS 0.2395 and 30.20% (IN2N 23.76%, DDS 4.52%).
- Text-to-3D generation: UDS (in LucidDreamer framework) CLIP 0.2984 and 48.37% preference vs LucidDreamer(ISM) 0.2897 and 41.39%.
- SVG editing: UDS LPIPS 0.3489 (lowest) with CLIP 0.2576, versus PDS LPIPS 0.3803 and SDS CLIP 0.2507.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use UDS instead of PDS | 3D editing · 3D editing | CLIP Score: 0.2395 -> 0.2498 | Helped |
| Use UDS instead of PDS | 3D editing · 3D editing | User Preference Rate: 30.20 -> 41.52 | Helped |
| Use UDS instead of ProlificDreamer | NeRF generation · 3D generation (NeRF) | CLIP Score: 0.2719 -> 0.2962 | Helped |
| Use UDS instead of ProlificDreamer | NeRF generation · 3D generation (NeRF) | User Preference Rate: 36.22 -> 42.14 | Helped |
| Use UDS instead of LucidDreamer (ISM) | 3D GS generation · 3D generation (3D GS) | CLIP Score: 0.2897 -> 0.2984 | Helped |
| Use UDS instead of LucidDreamer (ISM) | 3D GS generation · 3D generation (3D GS) | User Preference Rate: 41.39 -> 48.37 | Helped |
| Use UDS instead of SDS | SVG editing · SVG editing | CLIP Score: 0.2507 -> 0.2576 | Helped |
| Use UDS instead of PDS | SVG editing · SVG editing | LPIPS: 0.3803 -> 0.3489 | Helped |
| Use UDS instead of PDS | SVG editing · SVG editing | User Preference Rate: 26.21 -> 27.64 | Helped |
| Approximate x0 with multi-step DDIM inversion instead of single-step Tweedie's formula (ablation) | UDS editing · 3D editing | Identity preservation | Helped |
| Approximate x0 with single-step Tweedie's formula instead of multi-step DDIM inversion (ablation) | UDS editing · 3D editing | Text alignment | Helped |
| Add noise using the DDIM inverse strategy (ablation) | UDS generation · 3D generation | Generated result quality | Helped |
Surprising findings
- Even with classifier-free guidance weight 0, PDS edits alter the image (identity is not perfectly restored), attributed to random noise in the stochastic latent.
- DDS and PDS require CFG=100 for convergence, while UDS works with CFG=7.5, suggesting high CFG causes instability and over-saturation.
Limitations
- Editing fails when the gap between source scene and target prompt is large (e.g., plant to balls).
- Generation success depends on initialization, especially for 3D GS.
- Multi-step DDIM inversion approximation improves identity preservation but increases optimization time.
Experimental setup
Datasets: IN2N real-world scenes (8 scenes, 37 prompt pairs) Magic3D prompts (15) DreamFusion prompts (415) 43 prompts for 3D GS VectorFusion SVG images
Benchmarks: IN2N scenes editing set Magic3D prompts DreamFusion prompts VectorFusion SVG
Metrics: CLIP score user preference rate LPIPS
Baselines: IN2N DDS PDS DreamFusion Fantasia3D ProlificDreamer LucidDreamer SDS
Builds on: DreamFusion Delta Denoising Score Posterior Distillation Sampling Interval Score Matching ProlificDreamer
Closest papers by meaning
- DaCapo: Score Distillation as Stacked Bridge for Fast and High-quality 3D Editing (CVPR 2025 · similarity 0.93)
- Consistent3D: Towards Consistent High-Fidelity Text-to-3D Generation with Deterministic Sampling Prior (CVPR 2024 · similarity 0.91)
- DreamCatalyst: Fast and High-Quality 3D Editing via Controlling Editability and Identity Preservation (ICLR 2025 · similarity 0.91)
- Rethinking Score Distillation as a Bridge Between Image Distributions (NeurIPS 2024 · similarity 0.90)
- Text-to-3D with Classifier Score Distillation (ICLR 2024 · similarity 0.90)
- Consistent Flow Distillation for Text-to-3D Generation (ICLR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.