Research map / 3D vision / Face and shape
3D Paintbrush: Local Stylization of 3D Shapes with Cascaded Score Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces 3D Paintbrush, which jointly learns a localization map and texture map on meshes from text prompts, using Cascaded Score Distillation to supervise with all stages of DeepFloyd IF.
Problem
Local, text-driven stylization of semantic regions directly on meshes, producing texture maps with tight, detailed localizations; prior local editing methods use coarse localizations or do not operate on meshes.
Approach
Simultaneously optimize a neural localization field and neural texture field on the mesh, and distill scores from all stages of a cascaded diffusion model (Cascaded Score Distillation) with weighted per-stage gradients, plus a background texture loss to keep localizations tight.
Key results
- In a 39-user perceptual study over 9 meshes, the method scored 4.80 for localization versus 3D Highlighter 2.03 and SATR 1.89.
- For local edits, the method scored 4.88 versus Vox-E 2.15, Latent Paint 2.14, and its own SDS variant 4.06.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CSD instead of standard SDS (ablation) | 3D Paintbrush · Local edits | average score: 4.06 -> 4.88 | Helped |
| Use stage 1 only instead of combining cascaded stages (ablation) | 3D Paintbrush CSD · Local mesh texturing | texture detail | Hurt |
| Use stage 2 only instead of combining cascaded stages (ablation) | 3D Paintbrush CSD · Local mesh texturing | localization accuracy | Hurt |
| Combine stage 1 and stage 2 supervision (ablation) | 3D Paintbrush CSD · Local mesh texturing | texture detail | Helped |
| Increase the stage 2 weight (ablation) | 3D Paintbrush CSD · Local mesh texturing | detail and granularity | Helped |
| Optimize localization and texture simultaneously rather than in series (ablation) | 3D Paintbrush · Local mesh texturing | texture detail | Helped |
| Optimize localization and texture independently (ablation) | 3D Paintbrush · Local mesh texturing | texture-localization alignment | Hurt |
| Optimize localization first, then texture within that region (ablation) | 3D Paintbrush · Local mesh texturing | texture detail | Hurt |
Limitations
- Auxiliary semantically related components (e.g., necklaces with pharaoh headdress) can be included in the localization.
- Suffers from the Janus effect common to 2D-supervised text-to-3D methods.
Experimental setup
Metrics: average perceptual score from user study CLIP R-Precision
Baselines: 3D Highlighter SATR Latent Paint Vox-E Ours (SDS)
Builds on: Score Distillation Sampling DeepFloyd IF 3D Highlighter Latent Paint
Closest papers by meaning
- Paint-it: Text-to-Texture Synthesis via Deep Convolutional Texture Map Optimization and Physically-Based Rendering (CVPR 2024 · similarity 0.87)
- PaintHuman: Towards High-Fidelity Text-to-3D Human Texturing via Denoised Score Distillation (AAAI 2024 · similarity 0.87)
- Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Models (CVPR 2024 · similarity 0.86)
- Wukong's 72 Transformations: High-fidelity Textured 3D Morphing via Flow Models (NeurIPS 2025 · similarity 0.86)
- Single Mesh Diffusion Models with Field Latents for Texture Generation (CVPR 2024 · similarity 0.86)
- Neural 3D Strokes: Creating Stylized 3D Scenes with Vectorized 3D Strokes (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.