Research map / Reinforcement learning / Policy gradient and actor-critic
SAC Flow: Sample-Efficient Reinforcement Learning of Flow-Based Policies via Velocity-Reparameterized Sequential Modeling
Yixian Zhang, Shu'ang Yu, Tonghe Zhang, Mo Guang, Haojia Hui, Kaiwen Long, Yu Wang, Chao Yu, Wenbo Ding
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| reparameterize velocity network with GRU-style gate (Flow-G) or Transformer-decoded velocity (Flow-T) instead of plain MLP velocity (ablation) | SAC Flow · Gradient norm during SAC training | gradient stability: max variation of 0.29 | Improved |
| apply SAC directly to a flow policy without velocity reparameterization (ablation) | SAC Flow naive baseline · MuJoCo from scratch and OGBench offline-to-online | return / success rate | Hurt |
| increase flow sampling steps K from 4 to 7 and 10 (ablation) | SAC Flow-T and SAC Flow-G · ablation on sampling steps | performance | No clear effect |
| use SAC Flow instead of FlowRL for from-scratch training | flow-based policy · MuJoCo from scratch | return: up to 130% improvement | Improved |
| use SAC Flow-T vs prior methods in offline-to-online | OGBench cube-triple and cube-quadruple · OGBench offline-to-online | success rate: up to 60% improvement | Improved |
| use SAC Flow with strongly regularized training (large beta) | SAC Flow-G and Flow-T · Robomimic offline-to-online | success rate | No clear effect |
| use SAC Flow instead of DIME on Humanoid | SAC Flow · MuJoCo Humanoid from scratch | return | No clear effect |
Closest papers by meaning
- ReinFlow: Fine-tuning Flow Matching Policy with Online Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
- Aligning Flow Map Policies with Optimal $Q$-Guidance (NeurIPS 2026 · similarity 0.88)
- Flow-Based Policy for Online Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
- Composite Flow Matching for Reinforcement Learning with Shifted-Dynamics Data (NeurIPS 2025 · similarity 0.87)
- Learning Multimodal One-step Flow Policy via Value-weighted Optimal Transport (NeurIPS 2026 · similarity 0.87)
- Flow-based Domain Randomization for Learning and Sequencing Robotic Skills (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
