Research map / Image generation / Autoregressive and latent generation
HomeDiffusion: Zero-Shot Object Customization with Multi-View Representation Learning for Indoor Scenes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HomeDiffusion, a zero-shot indoor furniture customization method on Stable Diffusion 2.1 that learns multi-view object representations (MORL) and uses pixel-aligned cross-attention on composited images (BOCL), with a new 3D-FRONT-based multi-view dataset.
Problem
Zero-shot object customization in indoor scenes loses fine object details and fails to produce viewpoints that match the background scene when only a single reference view (or asymmetric objects) is available.
Approach
Train on multi-view furniture images to predict one held-out view (MORL) with a ControlNet edge layout, then insert the reference object into a masked background using a composited image whose features are fused via pixel-aligned cross-attention, plus an HD visual encoder combining global and local DINO-V2 features.
Key results
- On ZOC-Indoor-Eval with single-view reference, achieves CLIP 89.4 and DINO 86.2, above AnyDoor (87.1/83.3) and Paint-by-Example (83.4/78.5).
- With five reference views, DINO score reaches 88.1 vs DreamBooth 85.5 (few-shot comparison).
- Human study (6 participants, 900 images): Fidelity 3.45 and Harmony 3.51 vs AnyDoor 2.70/2.46 and DreamBooth 2.18/2.57.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add MORL (ablation) | HomeDiffusion · ZOC-Indoor-Eval (one view) | DINO Score: 84.6 -> 86.2 | Helped |
| add MORL (ablation) | HomeDiffusion · ZOC-Indoor-Eval (five views) | DINO Score: 86.9 -> 88.1 | Helped |
| replace basic DINO-V2 encoder with HD visual encoder (ablation) | HomeDiffusion with masked-background guidance · ZOC-Indoor-Eval (one view) | DINO Score: 83.8 -> 85.2 | Helped |
| replace basic DINO-V2 encoder with HD visual encoder (ablation) | HomeDiffusion with masked-background guidance · ZOC-Indoor-Eval (five views) | DINO Score: 85.0 -> 86.3 | Helped |
| add composited image guidance (ablation) | HomeDiffusion with HD visual encoder · ZOC-Indoor-Eval (one view) | DINO Score: 85.2 -> 85.6 | Helped |
| add composited image guidance (ablation) | HomeDiffusion with HD visual encoder · ZOC-Indoor-Eval (five views) | DINO Score: 86.3 -> 86.9 | Helped |
| use pixel-aligned cross-attention instead of directly adding feature maps (ablation) | HomeDiffusion with composited image guidance · ZOC-Indoor-Eval (one view) | DINO Score: 85.6 -> 86.2 | Helped |
| use pixel-aligned cross-attention instead of directly adding feature maps (ablation) | HomeDiffusion with composited image guidance · ZOC-Indoor-Eval (five views) | DINO Score: 86.9 -> 88.1 | Helped |
| provide five reference views instead of one | HomeDiffusion · ZOC-Indoor-Eval | DINO Score: 86.2 -> 88.1 | Helped |
| provide more reference views (one to five) | DreamBooth · ZOC-Indoor-Eval | DINO Score: 69.8 -> 85.5 | Helped |
Limitations
- Reference object degrades when the viewpoint differs strongly from the scene orientation, e.g., color changes and hollow panels with single views.
- Benchmark and dataset are only promised for release ('plan to publicly release'), not yet available.
- Evaluation relies on CLIP/DINO similarity and a small 6-person user study.
Experimental setup
Datasets: 3D-FRONT Viton-HD
Benchmarks: ZOC-Indoor-Eval Viton-HD test
Metrics: CLIP Score DINO Score human fidelity/harmony score
Baselines: Paint-by-Example AnyDoor DreamBooth
Builds on: Stable Diffusion AnyDoor ControlNet DINO-V2 Paint-by-Example
Closest papers by meaning
- DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesis (CVPR 2024 · similarity 0.88)
- AnyDoor: Zero-shot Object-level Image Customization (CVPR 2024 · similarity 0.87)
- MOVIS: Enhancing Multi-Object Novel View Synthesis for Indoor Scenes (CVPR 2025 · similarity 0.86)
- MEAT: Multiview Diffusion Model for Human Generation on Megapixels with Mesh Attention (CVPR 2025 · similarity 0.85)
- RoomPainter: View-Integrated Diffusion for Consistent Indoor Scene Texturing (CVPR 2025 · similarity 0.85)
- Architect: Generating Vivid and Interactive 3D Scenes with Hierarchical 2D Inpainting (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.