Research map / Reinforcement learning / Offline RL
One-Step Flow Q-Learning: Addressing the Diffusion Policy Bottleneck in Offline Reinforcement Learning
Thanh Xuan Nguyen, Chang D. Yoo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train and evaluate fully in one-step regime with average velocity (OFQL) instead of 5-step DQL | DQL · MuJoCo 9 tasks | average normalized score: 87.9 -> 92.6 (+4.7) | Improved |
| apply one-step DDIM sampler to pretrained DQL at inference (ablation) | DQL · MuJoCo 9 tasks | average normalized score: 87.9 -> 11.6 (-76.3) | Hurt |
| use FBRAC flow policy with single inference step (ablation) | DQL · MuJoCo 9 tasks | average normalized score: 87.9 -> 67.1 (-20.8) | Hurt |
| distill one-step policy from multi-step flow (FQL) (ablation) | DQL · MuJoCo 9 tasks | average normalized score: 87.9 -> 79.2 (-8.7) | Hurt |
| flow ratio 0.5 instead of 1 (pure flow matching) (ablation) | OFQL · HalfCheetah-Medium-Expert | D4RL score: 38.3 -> 95.2 | Improved |
| flow ratio 0.5 instead of 0 (ablation) | OFQL · HalfCheetah-Medium-Replay | D4RL score: 10.5 -> 51.2 | Improved |
| flow ratio 0.5 instead of 0 (ablation) | OFQL · HalfCheetah-Medium | D4RL score: 63.2 -> 63.8 | Improved |
| flow ratio 0.5 instead of 0.25 (ablation) | OFQL · HalfCheetah-Medium | D4RL score: 63.76 -> 63.8 | No clear effect |
| replace DQL with OFQL | DQL · AntMaze average | D4RL normalized score: 64.6 -> 84.6 | Improved |
| replace DQL with OFQL | DQL · Kitchen average | D4RL normalized score: 61.6 -> 67.0 | Improved |
| replace DQL with OFQL | DQL · MuJoCo average | D4RL normalized score: 87.9 to 92.5 | Improved |
| use OFQL one-step training instead of 5-step DQL | DQL · MuJoCo training time over 1M steps | training hours: 11.7 hours at 5 steps -> 6.3 hours | Improved |
Closest papers by meaning
- Diffusion Policies Creating a Trust Region for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.93)
- Flow Q-Learning (ICML 2025 · similarity 0.92)
- Diffusion Actor-Critic: Formulating Constrained Policy Iteration as Diffusion Noise Regression for Offline Reinforcement Learning (ICLR 2025 · similarity 0.90)
- Flow Matching for Offline Reinforcement Learning with Discrete Actions (NeurIPS 2026 · similarity 0.89)
- Learning a Diffusion Model Policy from Rewards via Q-Score Matching (ICML 2024 · similarity 0.89)
- Learning Multimodal One-step Flow Policy via Value-weighted Optimal Transport (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
