AI Paper Atlas

Research map / Reinforcement learning / Offline RL

Belief-Based Offline Reinforcement Learning for Delay-Robust Policy Optimization

Simon Sinong Zhan, Qingyuan Wu, Philip Wang, Frank Yang, Xiangyu Shi, Chao Huang, Qi Zhu

ICLR 2026 · Reinforcement learning › Offline RL · approach: Simple practical offline RL · 3 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train belief jointly with the policy instead of pretraining it separately (ablation)DT-CORL · Hopper (delay 16)normalized return: 68.3 -> 98.5Improved
train belief jointly with the policy instead of pretraining it separately (ablation)DT-CORL · Hopper (delay 4)normalized return: 92.1 -> 101.2Improved
use transformer belief with offline RL instead of raw state augmentationCQL · Hopper-medium (deterministic delay 16)normalized return: 21.1 -> 71.8Improved
use DT-CORL instead of Belief-CQL (aligned belief and value learning)Belief-CQL · Walker2d-medium (deterministic delay 16)normalized return: 39.2 -> 86.8Improved
use DT-CORL instead of Belief-CQL at short delayBelief-CQL · HalfCheetah-medium (deterministic delay 4)normalized return: 49.2 -> 47.4No clear effect
use DT-CORL instead of online delayed RL method DBPT-SAC offlineDBPT-SAC · AntMaze umazenormalized return: 0.0 -> 76.7Improved
use belief-based policy with IQL instead of CQLBelief-IQL vs Belief-CQL · Hopper-medium (deterministic delay 16)normalized return: 42.9 -> 25.4Hurt
use DT-CORL on harder AntMaze tasksDT-CORL · AntMaze medium-play and large-playnormalized return: 2.33 vs 5.67No clear effect
use stochastic instead of deterministic delayDT-CORL · AntMaze umaze (delay 16)normalized return: 40.0 -> 67.3Improved
use stochastic instead of deterministic delayAugmented-CQL · AntMaze umaze (delay 8)normalized return: 44.0 -> 37.0Hurt
increase transformer belief size (64x10 to 256x10) (ablation)transformer belief · Hopper-medium-v2 (delay 16)belief MSE: 2.68 -> 0.32Improved
increase delay to 64 stepsDT-CORL · Hopper-medium-v2 (deterministic delay 64)normalized return: 2.77 vs 2.71No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy