Research map / Reinforcement learning / Offline RL
StARformer: Transformer with State-Action-Reward Representations for Visual Reinforcement Learning
Jinghuan Shang, Kumara Kahatapitiya, Xiang Li, Michael S. Ryoo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use StARformer instead of Decision Transformer | DT · Atari Breakout (offline RL) | episode return: 76.9 -> 124.1 | Improved |
| use StARformer instead of Decision Transformer | DT · DMC Cheetah (offline RL) | episode return: 27.9 -> 275.3 | Improved |
| use StARformer instead of Decision Transformer | DT · Atari Seaquest (offline RL) | episode return: 1131 -> 781 | Hurt |
| use StARformer instead of Decision Transformer | DT · Atari Boxing (offline RL) | episode return: 78.3 -> 81.2 | No clear effect |
| use longer input sequences T | StARformer · Atari and DMC (offline RL) | episode return | Improved |
| use patch embeddings rather than conv in Step Transformer (ablation) | StARformer · Atari Breakout | episode return: 55.6 -> 124.1 | Improved |
| use conv rather than patch embeddings in Sequence Transformer (ablation) | StARformer · Atari Breakout | episode return: 38.0 -> 124.1 | Improved |
| add state embedding (conv) in Sequence Transformer (ablation) | StARformer · Atari Breakout | episode return: 38.7 -> 124.1 | Improved |
| use layer-wise attention connection instead of summation fusion (ablation) | StARformer · Atari Breakout | episode return: 29.2 -> 124.1 | Improved |
| use layer-wise connection instead of stacking (ablation) | StARformer · Atari Qbert | episode return: 575 -> 6968 | Improved |
| replace conv state embedding by ViT in DT (ablation) | DT · Atari Breakout | episode return: 76.9 -> 47.3 | Hurt |
| use stepwise reward instead of return-to-go (ablation) | StARformer · Atari and DMC | episode return | No clear effect |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- Decision Transformer — Decision Transformer: Reinforcement Learning via Sequence Modeling (NeurIPS 2021) · better on 12 of 13 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- SiT: Symmetry-invariant Transformers for Generalisation in Reinforcement Learning (ICML 2024 · similarity 0.86)
- Decision ConvFormer: Local Filtering in MetaFormer is Sufficient for Decision Making (ICLR 2024 · similarity 0.86)
- The Surprising Ineffectiveness of Pre-Trained Visual Representations for Model-Based Reinforcement Learning (NeurIPS 2024 · similarity 0.86)
- Decision SpikeFormer: Spike-Driven Transformer for Decision Making (CVPR 2025 · similarity 0.86)
- Efficient Reinforcement Learning Through Adaptively Pretrained Visual Encoder (AAAI 2025 · similarity 0.86)
- When Should We Prefer State-to-Visual DAgger over Visual Reinforcement Learning? (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
