Research map / 3D vision / Gaussian splatting
3DitScene: Editing Any Scene via Language-guided Disentangled Gaussian Splatting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes 3DitScene, a unified scene image editing framework that lifts a single image to language-disentangled 3D Gaussians (CLIP/SAM feature distillation) with diffusion-based depth/RGB inpainting, enabling object-level and camera-level 3D editing.
Problem
Scene image editing is limited to either 2D object-level or 3D global-level editing; there is no unified approach to 3D-aware, object-granular control of scenes from a single image.
Approach
Lift an input image to 3D Gaussians refined by Stable Diffusion SDS, expand unseen regions via depth inpainting with a diffusion depth model, and distill CLIP language features (supervised by SAM masks) into the Gaussians so objects can be queried by text or box and edited independently, with random layout augmentation during optimization.
Key results
- In the user study, 3DitScene was preferred by 65.4% of human raters for consistency and 64.0% for quality, versus at most 25.1% for any baseline.
- GPT-4V preference gave 62.0% for consistency and 40.8% for quality (best in both, but quality is close to Image Sculpting at 39.2%).
- Camera control needs as few as three camera poses, versus dense trajectories required by LucidDreamer.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use 3DitScene instead of the strongest baseline, Image Sculpting | scene image editing · User study: consistency (Human) | favorite users (%): 12.7 -> 65.4 | Helped |
| Use 3DitScene instead of the strongest baseline, Image Sculpting | scene image editing · User study: consistency (GPT-4v) | favorite users (%): 31.3 -> 62.0 | Helped |
| Use 3DitScene instead of the strongest baseline, Image Sculpting | scene image editing · User study: quality (Human) | favorite users (%): 25.1 -> 64.0 | Helped |
| Use 3DitScene instead of the strongest baseline, Image Sculpting | scene image editing · User study: quality (GPT-4v) | favorite users (%): 39.2 -> 40.8 | Helped |
| Add reconstruction loss (ablation) | 3DitScene · Novel-view image quality | image quality | Helped |
| Add SDS loss (ablation) | 3DitScene · Novel-view image quality | image quality | Helped |
| Increase classifier-free guidance above the default 5 (ablation) | 3DitScene · Novel-view images | image details and consistency | Mixed |
| Add layout augmentation during optimization (ablation) | 3DitScene · Object removal / edited scenes | floating artifacts | Helped |
| Use depth inpainting instead of heuristic alignment (ablation) | 3DitScene · Novel-view depth estimation | depth alignment | Helped |
Surprising findings
- GPT-4V quality preference is only 40.8% for the proposed method, nearly tied with Image Sculpting at 39.2%.
Limitations
- Evaluation is limited to a user study with 25 users and 20 samples per method; no standard quantitative metrics reported.
- Relies on several off-the-shelf models (depth, SAM, CLIP, Stable Diffusion) whose errors propagate into the 3D representation.
- Heuristic-free depth inpainting and SDS are per-scene optimizations, so cost is not characterized.
Experimental setup
Metrics: user preference percentage (consistency) user preference percentage (quality)
Baselines: AnyDoor Object 3DIT Image Sculpting AdaMPI LucidDreamer
Builds on: 3D Gaussian Splatting LangSplat LERF Score Distillation Sampling Segment Anything Model
Closest papers by meaning
- EditSplat: Multi-View Fusion and Attention-Guided Optimization for View-Consistent 3D Scene Editing with 3D Gaussian Splatting (CVPR 2025 · similarity 0.90)
- GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting (CVPR 2024 · similarity 0.90)
- 3DGS-Drag: Dragging Gaussians for Intuitive Point-Based 3D Editing (ICLR 2025 · similarity 0.89)
- CTRL-D: Controllable Dynamic 3D Scene Editing with Personalized 2D Diffusion (CVPR 2025 · similarity 0.89)
- ProEdit: Simple Progression is All You Need for High-Quality 3D Scene Editing (NeurIPS 2024 · similarity 0.89)
- SceneSplat++: A Large Dataset and Comprehensive Benchmark for Language Gaussian Splatting (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.