Research map / Image generation / Text-to-image diffusion
Are We Ready for RL in Text-to-3D Generation? A Progressive Investigation
Yiwen Tang, Zoey Guo, Kaixin Zhu, Ray Zhang, Qizhi Chen, Dongzhi Jiang, Junli Liu, Bohan Zeng, Haoming Song, Delin Qu, Tianyi Bai, Dan Xu and 2 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add textual reasoning before 3D token generation in GRPO (ablation) | ShapeLLM-Omni + GRPO (HPS reward) · Toys4K (800 sampled) | CLIP score: 23.4 -> 24.0 | Improved |
| add HPS v2.1 human preference reward (ablation) | ShapeLLM-Omni + GRPO · Toys4K (800 sampled) | CLIP score: 22.7 -> 24.0 | Improved |
| add UnifiedReward on top of HPS (ablation) | GRPO reward · Toys4K (800 sampled) | CLIP score: 24.0 -> 24.6 | Improved |
| replace UnifiedReward with Qwen2.5-VL for alignment next to HPS (ablation) | GRPO reward · Toys4K (800 sampled) | CLIP score: 24.6 -> 24.2 | Hurt |
| add Qwen2.5-VL 3D consistency reward to HPS + UnifiedReward (ablation) | GRPO reward · Toys4K (800 sampled) | CLIP score: 24.6 -> 25.2 | Improved |
| add dynamic sampling to GRPO (ablation) | GRPO · Toys4K (800 sampled) | CLIP score: 25.2 -> 25.8 | Improved |
| add token-level loss averaging on top of dynamic sampling (ablation) | GRPO + dynamic sampling · Toys4K (800 sampled) | CLIP score: 25.8 -> 26.3 | Improved |
| add sequence-level optimization (GSPO) on top of dynamic sampling (ablation) | GRPO + dynamic sampling · Toys4K (800 sampled) | CLIP score: 25.8 -> 25.5 | Hurt |
| remove KL penalty (ablation) | GRPO + dynamic sampling + token averaging · Toys4K (800 sampled) | CLIP score: 26.3 -> 25.9 | Hurt |
| add decoupled clip to sampling + token averaging (ablation) | GRPO · Toys4K (800 sampled) | CLIP score: 26.3 -> 26.5 | Improved |
| scale training data 1.5x, 2x, 3x (ablation) | best RL configuration · Toys4K (800 sampled) | CLIP score: +0.4, +0.2, +0.4 | Improved |
| double training iterations (ablation) | best RL configuration · Toys4K (800 sampled) | CLIP score: +0.9 | Improved |
Closest papers by meaning
- GoT-R1: Unleashing Reasoning Capability of Autoregressive Visual Generation with Reinforcement Learning (ICLR 2026 · similarity 0.90)
- Let Language Constrain Geometry: Vision–Language Models as Semantic and Spatial Critics for 3D Generation (ICML 2026 · similarity 0.88)
- Delving into RL for Image Generation with CoT: A Study on DPO vs. GRPO (NeurIPS 2025 · similarity 0.88)
- CG-MLLM: Captioning and Generating 3D content via Multi-modal Large Language Models (ICML 2026 · similarity 0.88)
- Sketch Then Paint: Hierarchical Reinforcement Learning for Diffusion Multi-Modal Large Language Models (NeurIPS 2026 · similarity 0.87)
- MetaSpatial: Reinforcing 3D Spatial Reasoning in VLMs for the Metaverse (ICLR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
