AI Paper Atlas

Research map / Reinforcement learning / Offline RL

DEAS: DEtached value learning with Action Sequence for Scalable Offline RL

Changyeon Kim, Haeone Lee, Younggyo Seo, Kimin Lee, Yuke Zhu

ICLR 2026 · Reinforcement learning › Offline RL · approach: Offline-to-online fine-tuning

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use action sequences of length 8 instead of single-step actions (ablation)DEAS · OGBench puzzle-4x4success rate (%): 21 -> 88Improved
increase action sequence length from 8 to 16 without enlarging the actor (ablation)DEAS · OGBench puzzle-4x4success rate (%): 88 -> 51Hurt
enlarge actor to 1024x4 for H=16 (ablation)DEAS · OGBench puzzle-4x4success rate (%): 51 -> 84Improved
add distributional HL-Gaussian classification loss on top of IQL detached value learning (ablation)DEAS · OGBench puzzle-4x4success rate (%): 63 -> 88Improved
add IQL detached value learning on top of distributional HLG (ablation)DEAS · OGBench puzzle-4x4success rate (%): 75 -> 88Improved
lower intra-option discount gamma1 from 0.99 to 0.9 (ablation)DEAS · OGBench puzzle-4x4success rate (%): 81 -> 88Improved
lower gamma1 further from 0.9 to 0.8 (ablation)DEAS · OGBench puzzle-4x4success rate (%): 88 -> 87No clear effect
enlarge critic network from 256x4 to 512x4 (ablation)DEAS · OGBench puzzle-4x4success rate (%): 69 -> 88Improved
enlarge value network from 256x4 to 512x4 (ablation)DEAS · OGBench puzzle-4x4success rate (%): 88 -> 50Hurt
increase n-step horizon in FQL without action sequence criticsFQL · OGBench scene-play (1M data)success rate (%): 50 -> 36Hurt
use detached value learning with action sequences instead of coupled actor-critic (QC-FQL)Action-sequence offline RL · OGBench puzzle-3x3 (1M data)success rate (%): 62 -> 91Improved
use detached value learning with action sequences instead of coupled actor-critic (QC-FQL)Action-sequence offline RL · OGBench cube-triple (10M data)success rate (%): 83 -> 82No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy