Research map / 3D vision / Face and shape
Wukong's 72 Transformations: High-fidelity Textured 3D Morphing via Flow Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Wukong is a training-free framework for textured 3D morphing between image or text prompts that uses Trellis's rectified-flow generator, an optimal-transport barycenter with sequential initialization for shape, and similarity-guided selective token interpolation for texture.
Problem
Textured 3D morphing between two image or text prompts is under-explored; existing methods rely on manual correspondence and deformation trajectories, are limited to untextured meshes, or lack paired 3D training data.
Approach
Treat morphing as a free-support Wasserstein barycenter over flow-model condition tokens, initialize each step recursively from the previous solution to keep the path on the data manifold, and selectively retain source texture tokens where cosine similarity is high.
Key results
- Ours achieves FID 4.01, STP-GPT 1.00, SEP-GPT 1.00, PPL 2.91, V-CLIP 0.90, outperforming 3DRM (FID 6.36, V-CLIP 0.84) and MorphFlow (FID 147.70).
- Ablation shows linear token interpolation raises FID to 14.73 versus 4.01 for the full shape method.
- Shape-only comparison on SHREC07 gives Dirichlet 4.5163 and Coverage 0.8510, better than SRIF (6.4702, 0.6418).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use WUKONG instead of 3DRM | Textured 3D morphing · Textured 3D morphing | FID: 6.36 -> 4.01 | Helped |
| Use WUKONG instead of 3DRM | Textured 3D morphing · Textured 3D morphing | STP-GPT: 0.93 -> 1.00 | Helped |
| Use WUKONG instead of 3DRM | Textured 3D morphing · Textured 3D morphing | PPL: 3.02 -> 2.91 | Helped |
| Use WUKONG instead of 3DRM | Textured 3D morphing · Textured 3D morphing | V-CLIP: 0.84 -> 0.90 | Helped |
| Use WUKONG instead of 3DRM | Headspace morphing · Headspace | FID: 6.61 -> 3.97 | Helped |
| Add recursive initialization to barycenter optimization (ablation) | WUKONG shape interpolation · Shape interpolation ablation | FID: 12.52 -> 4.01 | Helped |
| Replace direct linear interpolation with optimized barycenter interpolation (ablation) | WUKONG shape interpolation · Shape interpolation ablation | FID: 14.73 -> 4.01 | Helped |
| Use TripoSG as the backbone (ablation) | WUKONG* · Shape interpolation ablation | FID: 4.01 -> 4.16 | No clear effect |
| Use selective texture interpolation instead of copying source tokens without interpolation (ablation) | WUKONG texture morphing · Texture interpolation ablation | FID: 6.52 -> 4.20 | Helped |
| Use selective texture interpolation instead of linear texture interpolation (ablation) | WUKONG texture morphing · Texture interpolation ablation | STP-GPT: 0.92 -> 1.00 | Helped |
| Use selective texture interpolation instead of copying source tokens without interpolation (ablation) | WUKONG texture morphing · Texture interpolation ablation | PPL: 3.04 -> 2.91 | Helped |
| Vary the similarity threshold across 0.2–0.8 (ablation) | WUKONG texture interpolation · Threshold ablation | FID: 4.54; 4.20; 4.17; 4.25; 4.49 | No clear effect |
Surprising findings
- Linear interpolation of condition tokens often produces T-pose human artifacts when morphing semantically distant objects such as T-Rex to cat.
Limitations
- Struggles with extreme topological changes such as splitting or merging parts.
- Sensitive to ambiguous prompts or multi-view generation inconsistencies when input lacks structural clarity.
Experimental setup
Datasets: Headspace
Benchmarks: SHREC07
Metrics: FID STP-GPT SEP-GPT PPL V-CLIP GPT-4o visual plausibility score Dirichlet energy Coverage
Baselines: 3DRM MorphFlow DiffMorpher AID MV-Adapter Luma SRIF NeuroMorph TripoSG
Builds on: Trellis rectified flow DiffMorpher POT free-support barycenter
Closest papers by meaning
- FLOWING: Implicit Neural Flows for Structure-Preserving Morphing (NeurIPS 2025 · similarity 0.88)
- TextureDreamer: Image-Guided Texture Synthesis Through Geometry-Aware Diffusion (CVPR 2024 · similarity 0.87)
- Sin3DM: Learning a Diffusion Model from a Single 3D Textured Shape (ICLR 2024 · similarity 0.87)
- 3D Paintbrush: Local Stylization of 3D Shapes with Cascaded Score Distillation (CVPR 2024 · similarity 0.86)
- Text-Guided 3D Face Synthesis - From Generation to Editing (CVPR 2024 · similarity 0.86)
- TexTailor: Customized Text-aligned Texturing via Effective Resampling (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.