Research map / 3D vision / Face and shape
TriTex: Learning Texture from a Single Mesh via Triplane Semantic Features
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Learns a triplane-based volumetric texture field from a single textured mesh, mapping frozen Diff3F semantic features to surface colors for feed-forward texture transfer to same-category meshes.
Problem
Transferring semantic textures from a source textured mesh to other 3D meshes while preserving the source texture's appearance, faster than diffusion-based texturing methods.
Approach
Train a small triplane network on a single textured mesh: orthographically project pre-extracted Diff3F semantic features onto triplanes, process with triplane-aware 2D convolutions, and decode colors with an MLP, supervised by rendering losses (MSE plus perceptual) with augmentation, enabling feed-forward generalization to same-category shapes.
Key results
- TriTex achieves SIFID 0.22 and CLIP similarity 0.87, beating TEXTure (0.34, 0.84), MVEdit (0.38, 0.84), and EASI-Tex (0.29, 0.85).
- Inference takes about 1 minute per mesh versus 5 min for TEXTure and 15 min for EASI-Tex.
- In an AMT 2AFC user study over roughly 750 questions, raters strongly preferred TriTex over all three baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use TriTex instead of the strongest baseline, EASI-TEX | Texture transfer · Texture transfer benchmark | SIFID: 0.29 -> 0.22 | Helped |
| use TriTex instead of the strongest baseline, EASI-TEX | Texture transfer · Texture transfer benchmark | CLIP similarity: 0.85 -> 0.87 | Helped |
| use TriTex instead of TEXTure | Texture transfer · Texture transfer benchmark | Inference runtime: 5 min. -> 1 min. | Helped |
| use TriTex instead of EASI-TEX | Texture transfer · Texture transfer benchmark | Inference runtime: 15 min. -> 1 min. | Helped |
| remove the neural network and use nearest-neighbor feature matches to transfer colors (ablation) | TriTex · Texture transfer benchmark | SIFID: 0.22 -> 0.23 | Hurt |
| remove the neural network and use nearest-neighbor feature matches to transfer colors (ablation) | TriTex · Texture transfer benchmark | CLIP similarity: 0.87 -> 0.86 | Hurt |
| remove the pixel-level MSE loss (ablation) | TriTex · Texture transfer benchmark | SIFID: 0.22 -> 0.23 | Hurt |
| remove the pixel-level MSE loss (ablation) | TriTex · Texture transfer benchmark | CLIP similarity: 0.87 -> 0.86 | Hurt |
| remove the perceptual appearance loss (ablation) | TriTex · Texture transfer benchmark | SIFID: 0.22 -> 0.28 | Hurt |
| remove the perceptual appearance loss (ablation) | TriTex · Texture transfer benchmark | CLIP similarity: 0.87 -> 0.85 | Hurt |
| add preprocessing and training-time augmentations (ablation) | TriTex · Texture transfer benchmark | SIFID: 0.21 -> 0.22 | Hurt |
| add preprocessing and training-time augmentations (ablation) | TriTex · Texture transfer benchmark | CLIP similarity: 0.85 -> 0.87 | Helped |
Surprising findings
- A nearest-neighbor feature-matching baseline without the network achieves CLIP similarity 0.86, close to the full method's 0.87.
- Removing the MSE loss yields the same SIFID (0.23) as the nearest-neighbor baseline, and removing augmentation gives a slightly better SIFID (0.21) than the full model.
Limitations
- No generative capability; cannot synthesize texture details absent from the source texture.
- Relies on semantic features for correspondence, so ambiguous feature matches produce incorrect or less detailed transfers.
- Requires target shapes in the same orientation as the source; rotation augmentation reduces fine-detail reproduction.
Experimental setup
Datasets: Objaverse
Benchmarks: TriTex texture transfer benchmark
Metrics: SIFID CLIP similarity inference runtime human 2AFC preference
Baselines: TEXTure EASI-Tex MVEdit Nearest-neighbor feature transfer
Builds on: EG3D Sin3DM Diff3F TEXTure Tumanyan et al. perceptual loss
Closest papers by meaning
- Single Mesh Diffusion Models with Field Latents for Texture Generation (CVPR 2024 · similarity 0.88)
- SceneTex: High-Quality Texture Synthesis for Indoor Scenes via Diffusion Priors (CVPR 2024 · similarity 0.88)
- TextureDreamer: Image-Guided Texture Synthesis Through Geometry-Aware Diffusion (CVPR 2024 · similarity 0.87)
- Sin3DM: Learning a Diffusion Model from a Single 3D Textured Shape (ICLR 2024 · similarity 0.87)
- Paint-it: Text-to-Texture Synthesis via Deep Convolutional Texture Map Optimization and Physically-Based Rendering (CVPR 2024 · similarity 0.86)
- One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.