AI Paper Atlas

Research map / Reinforcement learning / Offline RL

Peng's Q($\lambda$) for Conservative Value Estimation in Offline Reinforcement Learning

Byeongchan Kim, Min-hwan Oh

ICLR 2026 · Reinforcement learning › Offline RL · approach: Pessimism and uncertainty in offline RL

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Replace Bellman operator with Peng Q(lambda) in conservative value estimationCQL · MuJoCo total (D4RL)normalized score: 1010.2 -> 1252.1Improved
Replace Bellman operator with Peng Q(lambda)CQL · AntMaze total (D4RL)normalized score: 303.6 -> 397.6Improved
Replace Bellman operator with Peng Q(lambda)CQL · Adroit total (D4RL)normalized score: 93.6 -> 166.7Improved
CPQL vs strongest baselines on AdroitCPQL · Adroit totalnormalized score: 166.7 vs CSVE 168.5No clear effect
CPQL on antmaze-umaze-diverseCPQL · antmaze-umaze-diversenormalized score: 89.1 (MISA) vs 68.6Hurt
Use CPQL with small conservatism alpha (ablation)CPQL vs CQL · D4RL datasetsnormalized scoreImproved
Use Peng Q(lambda) instead of uncorrected n-step, Retrace, Tree-backup (ablation)CPQL · D4RL training curvesnormalized scoreMixed
Pretrain Q with CPQL before online PQL fine-tuningPQL · offline-to-online D4RLreturnImproved
Use multi-step operator on low-quality datasetsCPQL · low-quality datasetsperformanceHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy