Research map / 3D vision / 3D generation
SceneTex: High-Quality Texture Synthesis for Indoor Scenes via Diffusion Priors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
SceneTex optimizes a multiresolution hash-encoded texture field in RGB space with VSD against a depth-conditioned ControlNet prior, plus a cross-attention decoder, to synthesize style-consistent indoor scene textures.
Problem
Text-driven texture synthesis for entire indoor 3D scenes produces seams, accumulated artifacts, blurry textures, and style inconsistency across objects when using view-warping inpainting or latent-space score distillation.
Approach
Formulate scene texturing as RGB-space optimization of an implicit multiresolution texture field, decoded by a cross-attention decoder that attends to pre-sampled reference UVs per instance, and supervised by VSD with a depth-conditioned diffusion prior at 768x768 resolution.
Key results
- SceneTex achieves CLIP score 22.18 and IS 3.33, outperforming Text2Tex (20.83, 2.87), MVDiffusion (18.47, 2.83), and Latent-Paint (18.37, 1.96).
- In a 75-participant user study, SceneTex scores 4.40 visual quality and 4.29 prompt fidelity, versus 3.09/3.12 for the next best (MVDiffusion).
- Removing the multiresolution texture drops CLIP to 19.87; removing the cross-attention decoder drops CLIP to 20.94 and visual quality to 3.94.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SceneTex instead of Text2Tex | 3D scene texturing · 3D-FRONT scenes | CLIP score: 20.83 -> 22.18 | Helped |
| Use SceneTex instead of Text2Tex | 3D scene texturing · 3D-FRONT scenes | Inception Score: 2.87 -> 3.33 | Helped |
| Use SceneTex instead of Text2Tex | 3D scene texturing · 3D-FRONT scenes | Visual Quality: 2.62 -> 4.40 | Helped |
| Use SceneTex instead of Text2Tex | 3D scene texturing · 3D-FRONT scenes | Prompt Fidelity: 3.04 -> 4.29 | Helped |
| Add the texture field instead of directly optimizing an RGB texture (ablation) | SceneTex · 3D-FRONT scenes | Visual Quality: 1.23 -> 4.40 | Helped |
| Add multiresolution texture (ablation) | SceneTex · 3D-FRONT scenes | Visual Quality: 2.11 -> 4.40 | Helped |
| Use the full SceneTex configuration rather than the reported cross-attention variant (ablation) | SceneTex · 3D-FRONT scenes | Visual Quality: 3.94 -> 4.40 | Helped |
Surprising findings
- Optimizing a raw RGB tensor instead of the texture field yields noisy, unrealistic textures, and the single-resolution texture field yields an over-simplified appearance, showing the implicit field's inductive bias matters.
Limitations
- Generated textures tend to include shading effects, especially near lamps, windows, and mirrors.
- Each scene requires about 20 hours of per-scene optimization.
- Evaluated only on 10 3D-FRONT scenes with 2 prompts each.
Experimental setup
Datasets: 3D-FRONT
Benchmarks: 3D-FRONT (10 scenes, 2 prompts each)
Metrics: CLIP score Inception Score user study visual quality user study prompt fidelity
Baselines: Latent-Paint MVDiffusion Text2Tex
Builds on: Latent-Paint ProlificDreamer (VSD) ControlNet multiresolution hash encoding Text2Tex
Closest papers by meaning
- RoomPainter: View-Integrated Diffusion for Consistent Indoor Scene Texturing (CVPR 2025 · similarity 0.89)
- GenesisTex: Adapting Image Denoising Diffusion to Texture Space (CVPR 2024 · similarity 0.89)
- TriTex: Learning Texture from a Single Mesh via Triplane Semantic Features (CVPR 2025 · similarity 0.88)
- SceneCraft: Layout-Guided 3D Scene Generation (NeurIPS 2024 · similarity 0.87)
- DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesis (CVPR 2024 · similarity 0.86)
- ConTex-Human: Free-View Rendering of Human from a Single Image with Texture-Consistent Synthesis (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.