AI Paper Atlas

Research map / Reinforcement learning / Offline RL

Value-Rectified Distillation for Flow-based Offline Reinforcement Learning

Ke Jiang, Wen Jiang, Yoshinobu Kawahara, Xiaoyang Tan

NeurIPS 2026 · Reinforcement learning › Offline RL · approach: Offline RL sample complexity theory · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Addresses out-of-distribution mode averaging when distilling flow-matching behavior policies in offline RL on multi-modal datasets, proposing value-rectified distillation.

Problem

Distilling flow-matching behavior policies in offline RL over multi-modal datasets makes the student policy interpolate between behavioral modes, producing invalid out-of-distribution actions.

Approach

Identifies OOD mode averaging as the failure mode of standard distillation of flow-matching policies and proposes value-rectified distillation to counter it; specific mechanism not stated in the abstract.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace FQL distillation with value-rectified distillation (VRD-R)FQL · D4RL average over all tasksnormalized score: 81.8 -> 92.8Improved
replace FQL distillation with VRD-QFQL · D4RL average over all tasksnormalized score: 81.8 -> 88.8Improved
replace FQL with VRD-QFQL · OGBench (50 tasks)success rate: 43.5 -> 56.4Improved
VRD-Q on hopper-medium vs FQLVRD-Q · D4RL hopper-medium-expertnormalized score: 105.4 -> 96.4Hurt
VRD-R geometric reshaping vs VRD-Q on locomotionVRD · D4RL MuJoCo averagenormalized score: 80.8 -> 88.5Improved
VRD-Q statistical filtering vs VRD-R on long-horizon navigationVRD · OGBench (50 tasks) averagesuccess rate: 50.8 -> 56.4Improved
quantile tau from 0.5 or 0.9 to 0.7 in VRD-Q (ablation)VRD-Q · D4RL halfcheetah-mediumnormalized score: 60.4 -> 65.7 (tau 0.5 -> 0.7); 58.1 at tau 0.9Improved
reduce inference latent scale sigma from 1 to 1e-4 in VRD-R (ablation)VRD-R · D4RL walker2d-mediumnormalized score: 53.9 -> 89.7Improved
smaller sigma below 1e-4 in VRD-R (ablation)VRD-R · D4RL halfcheetah-mediumnormalized score: 61.7 -> 61.2 (1e-4 -> 1e-5)No clear effect
use VRD instead of FQL and CQL on mixed expert-random datasetsVRD-Q and VRD-R · halfcheetah-mix and walker2d-mixscore vs random-sample ratio (50-90%)Improved

Experimental setup

Builds on: flow matching policy distillation

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy