Research map / Reinforcement learning / Policy gradient and actor-critic
Value Flows
Perry Dong, Chongyi Zheng, Chelsea Finn, Dorsa Sadigh, Benjamin Eysenbach
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| model return distribution with flow matching instead of C51 bins or CODAC quantiles | Value Flows · OGBench scene-play-singletask-task2 | 1-Wasserstein distance: 3x lower than best baseline | Improved |
| add BCFM regularization to distributional flow matching objective, with tuned coefficient (ablation) | Value Flows · OGBench puzzle-3x3 | success rate: 2.6x with best lambda | Improved |
| tune BCFM regularization coefficient to best value (ablation) | Value Flows · OGBench puzzle-3x3 | sample efficiency: 2x sample efficiency | Improved |
| reweight flow-matching loss with return-variance confidence weight at appropriate temperature (ablation) | Value Flows · two OGBench state tasks | success rate: 60% higher on average | Improved |
| use Value Flows instead of best baseline on challenging state-based tasks | Value Flows · OGBench harder state-based domains | success rate: 1.6x higher than best baseline | Improved |
| use Value Flows on image-based tasks | Value Flows · OGBench visual tasks | success rate: 1.24x over best baseline | Improved |
| use Value Flows for online fine-tuning on puzzle-4x4-play | Value Flows · OGBench puzzle-4x4-play offline-to-online | success rate: 15% higher than prior offline-to-online algorithms | Improved |
| Value Flows versus scalar-Q methods on D4RL adroit | Value Flows · D4RL adroit (12 tasks) | success rate: 50 vs 59 (ReBRAC) | No clear effect |
| use naive distributional conditional flow matching loss without bootstrapping (ablation) | Value Flows · initial experiments on some tasks | performance | Hurt |
Closest papers by meaning
- Quantile-Coupled Flow Matching for Distributional Reinforcement Learning (NeurIPS 2026 · similarity 0.88)
- Conformal Prediction Beyond the Horizon: Distribution-Free Inference for Policy Evaluation (NeurIPS 2025 · similarity 0.88)
- Beyond Expected Return: Accounting for Policy Reproducibility When Evaluating Reinforcement Learning Algorithms (AAAI 2024 · similarity 0.88)
- Designing Reinforcement Learning for Diffusion Models: A Unified Path-Space View (NeurIPS 2026 · similarity 0.87)
- Action Gaps and Advantages in Continuous-Time Distributional Reinforcement Learning (NeurIPS 2024 · similarity 0.86)
- Intrinsic Benefits of Categorical Distributional Loss: Uncertainty-aware Regularized Exploration in Reinforcement Learning (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
