Research map / Image generation / Autoregressive and latent generation
Mesh-RFT: Enhancing Mesh Generation via Fine-grained Reinforcement Fine-Tuning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Mesh-RFT applies fine-grained reinforcement fine-tuning to point-cloud-conditioned autoregressive mesh generation, using a topology-aware score (BER, TS, HD) to build preference pairs and Masked DPO with face-level quality masks.
Problem
Pretrained autoregressive mesh generators produce low-quality meshes with local topological defects, and global RL/DPO methods with object-level rewards and subjective human-annotated preferences fail to capture local structure quality.
Approach
Score candidate meshes with automatic topology-aware metrics (Boundary Edge Ratio, Topology Score, Hausdorff Distance) to build preference pairs, and apply DPO with a quality-aware face mask so the loss only emphasizes low-quality face tokens.
Key results
- M-DPO reduces HD by 24.6% and improves TS by 3.8% over the pretrained model (artist-mesh ablation: HD 0.3196 to 0.2411, TS 76.5 to 79.4).
- M-DPO outperforms global S-DPO with 17.4% HD reduction and 4.9% TS gain relative to the baseline's global variant.
- On artist meshes, Ours achieves CD 0.0973, HD 0.1826, TS 77.5 and 45% user-study preference vs. DeepMesh* at CD 0.1331, HD 0.2866, TS 74.9 and 22%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune the pretrained model with Mesh-RFT | pretrained mesh generator · Artist Meshes | HD: 0.2866 -> 0.1826 | Helped |
| Fine-tune the pretrained model with Mesh-RFT | pretrained mesh generator · Dense Meshes | TS: 75.8 -> 79.4 | Helped |
| Use Hausdorff Distance alone for preference scoring instead of the composite scoring system (ablation) | Mesh-RFT preference dataset construction · Dense Meshes | TS: 76.5 -> 75.7 | Hurt |
| Use Hausdorff Distance alone for preference scoring instead of the composite scoring system (ablation) | Mesh-RFT preference dataset construction · Dense Meshes | HD: 0.3196 -> 0.2919 | Helped |
| Use the composite scoring system for preference data instead of Hausdorff Distance alone (ablation) | Mesh-RFT preference dataset construction · Dense Meshes | TS: 75.7 -> 77.9 | Helped |
| Apply localized quality masks with M-DPO instead of global score-based DPO (ablation) | Mesh-RFT · Dense Meshes | HD: 0.2625 -> 0.2411 | Helped |
| Apply localized quality masks with M-DPO instead of global score-based DPO (ablation) | Mesh-RFT · Dense Meshes | US: 35% -> 40% | Helped |
Surprising findings
- Naive global DPO with only the Hausdorff Distance score (N-DPO) decreases TS relative to the pretrained model.
- The 'good' selected meshes may have inferior local performance in some regions compared to the rejected meshes.
Limitations
- Topological defects such as holes persist in complex geometries.
- Limited computational efficiency (about 40 tokens/s inference) and a basic topology reward.
- Poor generalization to snake-like data absent from training set.
Experimental setup
Datasets: ShapeNetV2 3D-FUTURE Objaverse Objaverse-XL licensed assets
Benchmarks: 100 artist-designed meshes 100 dense out-of-distribution meshes from Hunyuan2.5
Metrics: CD HD TS BER user study preference
Baselines: MeshAnythingV2 BPT DeepMesh N-DPO S-DPO
Builds on: DPO DeepMesh MeshXL Hunyuan3D 2.0 Hourglass Transformer
Closest papers by meaning
- Scaling Mesh Generation via Compressive Tokenization (CVPR 2025 · similarity 0.86)
- ARMesh: Autoregressive Mesh Generation via Next-Level-of-Detail Prediction (NeurIPS 2025 · similarity 0.86)
- Optimize Any Topology: A Foundation Model for Shape- and Resolution-Free Structural Topology Optimization (NeurIPS 2025 · similarity 0.85)
- Flow-Corrected Shape Optimization: Taming Manifold Drift in High-Dimensional 3D Models (NeurIPS 2026 · similarity 0.85)
- SILSA: Sliding-Window Slice Latents for Topology-Preserving High-Resolution 3D Generation (NeurIPS 2026 · similarity 0.85)
- HybridMQA: Exploring Geometry-Texture Interactions for Colored Mesh Quality Assessment (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.