AI Paper Atlas

Research map / Reinforcement learning / Offline RL

Action-Free Offline-To-Online RL via Discretised State Policies

Natinael Solomon Neggatu, Jeremie Houssineau, Giovanni Montana

ICLR 2026 · Reinforcement learning › Offline RL · approach: Simple practical offline RL · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add regularisation term to state-only DecQN (ablation)OSO-DecQN · Hopper-medium-expert (action-free offline)normalised return: 1.1 -> 110Improved
add regularisation term to state-only DecQN (ablation)OSO-DecQN · Hopper-medium-replay (action-free offline)discrete state difference error per step: 11.8 -> 0.99Improved
discretise state difference to Delta s instead of regressing next state s' (ablation)behaviour cloning state policy · Hopper-medium-replay (action-free offline)normalised return: 5.8 -> 29.2Improved
discretise state difference to Delta s instead of regressing continuous s'-s (ablation)behaviour cloning state policy · Hopper-medium-replay (action-free offline)normalised return: 4.9 -> 29.2Improved
discretise target to Delta s instead of s' (ablation)behaviour cloning state policy · Quadruped-Walk-medium-expert (action-free offline)discrete state difference error per step: 74.5 -> 14.7Improved
train with RL (OSO-DecQN) instead of imitating Delta sBC Delta s · Halfcheetah-medium-expert (action-free offline)normalised return: 54.7 -> 87.8Improved
train with RL (OSO-DecQN) instead of imitating Delta sBC Delta s · Walker2d-medium-replay (action-free offline)normalised return: 37.5 -> 84.8Improved
OSO-DecQN versus BC with actions on Walker2d-expertOSO-DecQN · Walker2d-expert (action-free offline)normalised return: 108.2 -> 108.1No clear effect
add guided online learning with pre-trained state policyTD3 / DecQN_N · D4RL and DeepMind Control (online, 1M steps)normalised returnImproved
add guided online learning on HopperTD3 · D4RL Hopper (online)normalised returnNo clear effect
compare Af-guide against TD3 baselineTD3 · D4RL (online)normalised returnHurt
vary IDM architecture and batch size (ablation)guided online learning · Online learning (appendix ablation)normalised returnNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy