Research map / Learning theory and algorithms / Prediction and decision losses

Trust Guided Decision Transformer

NeurIPS 2026 · Learning theory and algorithms › Prediction and decision losses · approach: Probabilistic calibration · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Trust Guided Decision Transformer (TGDT), which filters context-suffix candidates by a calibrated rolling next-state prediction error before critic-based action ranking, improving D4RL returns over vanilla DT and critic-only suffix selection.

Problem

Decision Transformers degrade on long closed-loop rollouts because self-generated conditioning context drifts out of the offline training distribution, and existing value-guided fixes rank actions without checking whether the producing context is reliable.

Approach

Reverse the order of value-only context selection: first reject context suffixes whose rolling next-state prediction error exceeds a held-out split-conformal-style threshold, then let a frozen IQL critic pick the highest-value action among the trusted suffixes. The predictor is used only as a per-step reliability sensor, not for model-based rollouts.

Key results

• On AntMaze medium-diverse, TGDT reaches 60.0 versus 30.0 for VDT, the best sequence-modeling baseline in the table.
• On Maze2D-medium, TGDT reduces the longest violation run by roughly an order of magnitude relative to critic-only selection while obtaining the highest score (text reports about 8x and a 5-7 point gain).
• Averaged over the reported benchmarks, TGDT reports 86.3 on Gym tasks versus 84.1 for VDT, 79.1 on Maze2D versus 74.2 for VDT, and 84.7 on AntMaze versus 76.7 for VDT.

Surprising findings

• Hard reset to the shortest context lowers return relative to TGDT, showing that shorter contexts are not always better.
• Critic guidance alone gives the largest score gain among training-side fixes, yet the critic-guided models still produce long above-threshold error runs.

Limitations

• Threshold is calibrated under teacher forcing, so split-conformal coverage does not hold for closed-loop rollouts; the threshold is used only as an empirical reliability scale.
• Evaluated only on state-based D4RL; image observations and non-stationary environments are untested.
• Compute overhead grows linearly with the number of candidate suffix lengths.

Experimental setup

Datasets: D4RL Maze2D D4RL AntMaze D4RL MuJoCo D4RL Adroit D4RL Kitchen

Benchmarks: D4RL Maze2D D4RL AntMaze D4RL MuJoCo locomotion D4RL Adroit D4RL Kitchen

Metrics: normalized D4RL score violation rate longest above-threshold run intervention rate

Baselines: DT Hard reset Critic-only suffix selection VDT CGDT GDT QDT DC IQL CQL BEAR BCQ

Builds on: Decision Transformer Implicit Q-Learning TD3+BC ReBRAC Elastic Decision Transformer split conformal prediction

Closest papers by meaning

• Critic-Guided Decision Transformer for Offline Reinforcement Learning (AAAI 2024 · similarity 0.87)
• ACT: Empowering Decision Transformer with Dynamic Programming via Advantage Conditioning (AAAI 2024 · similarity 0.86)
• Rethinking Decision Transformer via Hierarchical Reinforcement Learning (ICML 2024 · similarity 0.85)
• In-Context Reinforcement Learning From Suboptimal Historical Data (ICML 2025 · similarity 0.85)
• In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thought (ICML 2024 · similarity 0.85)
• From Weak Data to Strong Policy: Q-Targets Enable Provable In-Context Reinforcement Learning (NeurIPS 2026 · similarity 0.85)