Research map / 3D vision / 3D generation
TexTailor: Customized Text-aligned Texturing via Effective Resampling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TexTailor, which generates view-consistent object textures from text by resampling inside a DDIM process, fine-tuning ControlNet on a few resampled images with a performance preservation loss, and adaptively adding camera viewpoints.
Problem
Text-driven texture synthesis with depth-aware diffusion inpainting across predefined viewpoints suffers from a gradual shift in texture properties and inconsistency across views, since previously synthesized textures are merged only once per timestep and viewpoints ignore mesh geometry.
Approach
Applies RePaint-style resampling within a non-Markovian DDIM process (30 steps instead of 250), fine-tunes only ControlNet on five resampled images with a performance preservation loss to prevent catastrophic forgetting, and adds camera viewpoints adaptively based on the keep/new region ratio.
Key results
- On the Objaverse subset, TexTailor achieves LPIPS 37.889 and FID 29.998, beating Paint-it (38.316 / 39.823) and Text2Tex (38.931 / 33.487).
- Fine-tuning without the performance preservation loss raises FID to 53.855 in the ablation, versus 30.567 with the loss.
- Resampling in the non-Markovian DDIM process uses 30 steps versus 250 for the original RePaint resampling.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TexTailor instead of the strongest baseline, Paint-it | Objaverse subset texture synthesis · Objaverse subset | LPIPS: 38.316 -> 37.889 | Helped |
| Use TexTailor instead of the strongest baseline, Text2Tex | Objaverse subset texture synthesis · Objaverse subset | FID: 33.487 -> 29.998 | Helped |
| Add resampling to the baseline texturing method (ablation) | TexTailor · Objaverse subset | LPIPS | Helped |
| Train on resampled textures without performance preservation loss (ablation) | TexTailor with resampling · Objaverse subset | LPIPS: 38.89 -> 39.85 | Hurt |
| Train on resampled textures without performance preservation loss (ablation) | TexTailor with resampling · Objaverse subset | FID: 30.924 -> 53.855 | Hurt |
| Add performance preservation loss during training (ablation) | TexTailor trained on resampled textures · Objaverse subset | LPIPS: 39.85 -> 38.00 | Helped |
| Add performance preservation loss during training (ablation) | TexTailor trained on resampled textures · Objaverse subset | FID: 53.855 -> 30.567 | Helped |
| Add adaptive view refinement (ablation) | TexTailor with performance preservation loss · Objaverse subset | LPIPS: 38.00 -> 37.89 | Helped |
| Add adaptive view refinement (ablation) | TexTailor with performance preservation loss · Objaverse subset | FID: 30.567 -> 29.998 | Helped |
| Set adaptive viewpoint refinement threshold β to 0.25 instead of 0.5 (ablation) | TexTailor · Objaverse, 100 meshes | LPIPS: 8.99176 -> 8.99464 | Hurt |
| Set adaptive viewpoint refinement threshold β to 0.25 instead of 0.5 (ablation) | TexTailor · Objaverse, 100 meshes | FID: 57.799 -> 58.567 | Hurt |
| Set adaptive viewpoint refinement threshold β to 0.75 instead of 0.5 (ablation) | TexTailor · Objaverse, 100 meshes | LPIPS: 8.99176 -> 8.99468 | Hurt |
Surprising findings
- Fine-tuning ControlNet on resampled images without the performance preservation loss substantially degrades image quality (FID rises from about 30.9 to 53.9).
Limitations
- Texture quality depends heavily on the quality of the five fine-tuning images, and poor choices can degrade results.
- Fine-tuning is time-intensive, about 1.5 hours per mesh.
- LPIPS does not fully capture multi-view consistency because of spatial misalignment in overlapping regions.
Experimental setup
Datasets: Objaverse subset (410 textured meshes, 225 categories) ShapeNet car
Benchmarks: Objaverse subset ShapeNet car (qualitative only)
Metrics: LPIPS FID
Baselines: Latent-Paint Text2Tex TEXTure Paint-it
Builds on: RePaint DDIM ControlNet Text2Tex
Closest papers by meaning
- ViewDiff: 3D-Consistent Image Generation with Text-to-Image Models (CVPR 2024 · similarity 0.88)
- Paint-it: Text-to-Texture Synthesis via Deep Convolutional Texture Map Optimization and Physically-Based Rendering (CVPR 2024 · similarity 0.88)
- GenesisTex: Adapting Image Denoising Diffusion to Texture Space (CVPR 2024 · similarity 0.88)
- PaintHuman: Towards High-Fidelity Text-to-3D Human Texturing via Denoised Score Distillation (AAAI 2024 · similarity 0.87)
- DreamText: High Fidelity Scene Text Synthesis (CVPR 2025 · similarity 0.87)
- TextureDreamer: Image-Guided Texture Synthesis Through Geometry-Aware Diffusion (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.