AI Paper Atlas

Research map / Reinforcement learning / RL from feedback

floq: Training Critics via Flow-Matching for Scaling Compute in Value-Based RL

Bhavya Kumar Agrawalla, Michal Nauman, Khush Agrawal, Aviral Kumar

ICLR 2026 · Reinforcement learning › RL from feedback · approach: RL for LLM reasoning · 12 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace monolithic Q-network with flow-matching Q-functionFQL · OGBench 50 tasksaverage success rate (%): 46 -> 56Improved
tune flow steps and kappa per environment (Best vs Def.)floq · OGBench 50 tasksaverage success rate (%): 56 -> 59Improved
train FQL for 2M instead of 1M stepsFQL · OGBench 50 tasksaverage success rate (%): 46 -> 47No clear effect
use flow-matching critic instead of distributional IQN criticFQL · OGBench default tasksaverage success rate (%): 49 -> 64Improved
add dense supervision over all t with multi-step integration (vs t=0 only) (ablation)floq · OGBench antmaze-giantsuccess rate (%): 39 -> 86Improved
add dense supervision over all t with multi-step integration (vs t=0 only) (ablation)floq · OGBench humanoidmaze-largesuccess rate (%): 49 -> 56Improved
add dense supervision over all t with multi-step integration (vs t=0 only) (ablation)floq · OGBench cube-doublesuccess rate (%): 38 -> 72Improved
add dense supervision over all t with multi-step integration (vs t=0 only) (ablation)floq · OGBench humanoidmaze-mediumsuccess rate (%): 94 -> 98Improved
increase ensemble of monolithic critics to 8 (compute matched to 8 flow steps)FQL · OGBenchsuccess rateHurt
use HL-Gauss embedding of interpolant instead of scalar or normalized scalar (ablation)floq · OGBenchsuccess rateImproved
increase HL-Gauss sigma from {1,2,4,8} to 16 (ablation)floq · OGBenchsuccess rateImproved
use 64-dim Fourier time embedding instead of scalar t (ablation)floq · OGBenchsuccess rateImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy