Research map / Image generation / Editing and style transfer
GuideFlow3D: Optimization-Guided Rectified Flow For Appearance Transfer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Training-free 3D appearance transfer that steers a pretrained TRELLIS rectified-flow model with part-aware appearance and self-similarity guidance losses applied to structured latents, evaluated with GPT-5-mini ranking and a user study.
Problem
Transferring texture and fine geometric detail from an appearance object (image, mesh, or text) onto a 3D input mesh whose geometry differs substantially from the appearance object, where existing methods fail and no ground-truth outputs exist for evaluation.
Approach
Applies universal-guidance-style differentiable losses interleaved with rectified flow sampling in TRELLIS structured-latent space: a PartField-based part-aware appearance loss (mesh+image) and a part-aware self-similarity contrastive loss (image/text), keeping voxel positions fixed to preserve input geometry.
Key results
- On simple-complex intra-category image conditioning, GuideFlow3D ranks best on all six GPT criteria (overall 2.12 vs. 2.62 for TRELLIS, lower is better).
- In the human study with 59 MTurk participants, GuideFlow3D received the highest preference (27.2%) among methods.
- Runtime is 96 s per object on an RTX 4090 versus 78 s for the TRELLIS baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply rectified-flow-guided optimization rather than optimizing the objective without rectified flow (ablation) | GuideFlow3D · Simple-Complex intra-category, image condition | Overall ranking: 4.50 -> 1.06 | Helped |
| replace global pooled features with SLat nearest-neighbor matching (ablation) | GuideFlow3D without rectified flow · Simple-Complex intra-category, image condition | Overall ranking: 4.50 -> 3.63 | Helped |
| use rectified flow with K-means co-segmentation over SLat features instead of PartField segmentation (ablation) | GuideFlow3D · Simple-Complex intra-category, image condition | Overall ranking: 1.06 -> 2.66 | Hurt |
| use image-conditioned structure loss instead of appearance loss (ablation) | GuideFlow3D · Simple-Complex intra-category, image condition | Overall ranking: 1.06 -> 2.03 | Hurt |
| use reference image rather than a rendered view of the appearance mesh (ablation) | GuideFlow3D · Simple-Complex intra-category, appearance-mesh conditioning | Fidelity ranking: 1.96 -> 1.08 | Helped |
| use GuideFlow3D instead of Trellis without guidance | 3D appearance transfer · Simple-Complex intra-category, image condition | Overall ranking: 2.62 -> 2.12 | Helped |
| use GuideFlow3D instead of Trellis without guidance | 3D appearance transfer · Simple-Complex intra-category, text condition | Overall ranking: 2.39 -> 1.95 | Helped |
| use GuideFlow3D rather than Trellis | 3D appearance transfer · Complex-Complex inter-category, text condition | Fidelity ranking: 2.35 -> 1.72 | Helped |
| use GuideFlow3D rather than Trellis | 3D appearance transfer · Simple-Complex inter-category, image condition | Fidelity ranking: 2.64 -> 1.99 | Helped |
| use GuideFlow3D rather than Trellis | 3D appearance transfer · Complex-Complex intra-category, image condition | Fidelity ranking: 2.82 -> 2.21 | Helped |
| use GuideFlow3D rather than Trellis | 3D appearance transfer · Text-conditioned appearance transfer | CLIP: 27.64 -> 27.23 | Hurt |
| use GuideFlow3D instead of the cited baseline runtime | Appearance transfer · Appearance transfer runtime | runtime: 78s -> 96s | Hurt |
Surprising findings
- Perceptual metrics such as CLIP and DINOv2 barely separate GuideFlow3D from TRELLIS, and CLIP even slightly favors TRELLIS in the text setting.
- Directly applying a 3D generative model (TRELLIS) fails when input and appearance geometries differ strongly, motivating the guidance approach.
Limitations
- Optimization-based and slower than the baseline, so not suited to real-time use.
- Depends on the performance of TRELLIS and PartField; failures in these propagate.
- Assumes noiseless input meshes; GPT-based evaluation may suffer from hallucination and position/contrast biases.
Experimental setup
Datasets: ABO Objaverse-XL ScanNet
Benchmarks: custom simple-complex benchmark from ABO and procedural meshes
Metrics: GPT-based pairwise ranking (fidelity, clarity, integration, DINOv2 CLIP score DreamSim user study preference
Baselines: UV Nearest Neighbor MambaST Cross Image Attention EASI-Tex TRELLIS SDXL + Cross Image Attention
Builds on: Universal Guidance TRELLIS PartField Rectified flow GPT-Eval3D
Closest papers by meaning
- 3DOT: Texture Transfer for 3DGS Objects from a Single Reference Image (NeurIPS 2025 · similarity 0.87)
- TextureDreamer: Image-Guided Texture Synthesis Through Geometry-Aware Diffusion (CVPR 2024 · similarity 0.86)
- 3DToonify: Creating Your High-Fidelity 3D Stylized Avatar Easily from 2D Portrait Images (CVPR 2024 · similarity 0.86)
- LiteReality: Graphic-Ready 3D Scene Reconstruction from RGB-D Scans (NeurIPS 2025 · similarity 0.85)
- Text-Guided 3D Face Synthesis - From Generation to Editing (CVPR 2024 · similarity 0.85)
- Wukong's 72 Transformations: High-fidelity Textured 3D Morphing via Flow Models (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.