AI Paper Atlas

Research map / Reinforcement learning / Offline RL

Variational Consequence-Driven Offline Reinforcement Learning

Wen Jiang, Ke Jiang, Jin Wang, Xiaoyang Tan

NeurIPS 2026 · Reinforcement learning › Offline RL · approach: Simple practical offline RL · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a variational, consequence-driven offline RL approach that constrains policies by the consequences of actions (dynamic consistency) rather than only by action-distribution divergence.

Problem

Offline RL typically handles distribution shift with divergence constraints on action distributions, which fails to ensure dynamic consistency: small action deviations can cause very different transitions that drive the agent away from the offline dataset.

Approach

Shifts the offline RL constraint from action-distribution divergence to the consequences of actions, using a variational formulation to enforce dynamic consistency with the offline data. Abstract is truncated, so details beyond this are unknown.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
constrain induced state-consequence distribution via latent divergence instead of action-distribution matchingTD3-based offline RL · D4RL MuJoCo totalnormalized score: 773.0 (SPOT) -> 811.0 (StBR)Improved
use StBR latent consequence constraintTD3 · D4RL AntMaze totalnormalized score: 501.0 (FQL) -> 534.2 (StBR)Improved
use StBR latent consequence constraintTD3 · D4RL Adroit human totalnormalized score: 83.6 (IQL) -> 110.5 (StBR)Improved
explicit next-state prediction instead of implicit latent consequence constraintENSP baseline · Adroit (45-dim)normalized score: ENSP -3.0 vs StBR 110.5 totalHurt
remove the bidirectional action IB constraint (degenerates to standard VIB) (ablation)StBR · MuJoCo, AntMaze, Adroitpercent performance differenceImproved
use a one-step generative stochastic policy (FQL-based) instead of deterministic policy (ablation)StBR · AntMaze (sparse reward)performanceImproved
use a one-step generative stochastic policy instead of deterministic policy (ablation)StBR · near-deterministic optimal-action tasksperformanceHurt
increase beta (information compression) for high-dimensional tasks (ablation)StBR · hopper vs antmaze and pennormalized score (optimal beta): optimal beta 10^-4 (hopper) -> 10^-2 (antmaze, pen)Mixed
increase environment action noise variance (ablation)StBR on hopper-medium · hopper-medium with injected noiseoptimal betaMixed
increase lambda together with beta (ablation)StBR · lambda-beta sweepoptimal betaMixed
vary latent dimension from 0.5x to 2x state dimension (ablation)StBR · latent dimension sensitivityperformanceMixed
sweep lambda from 0.015 to 0.5 (ablation)StBR vs baselines · hopper-medium-v2, antmaze-large-diverse-v2normalized scoreImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy