Research map / Reinforcement learning / Offline RL
In-Context Compositional Q-Learning for Offline Reinforcement Learning
Qiushui Xu, Yu-Hao Huang, Yushu Jiang, Wenliang Zheng, Lei Song, Jinyu Wang, Jiang Bian
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ICQL local in-context Q-functions instead of CQL (second best) | ICQL · MuJoCo average | normalized score: 74.0 -> 80.6 | Improved |
| use ICQL instead of IQL | ICQL · Adroit average | normalized score: 33.2 -> 35.3 | Improved |
| use ICQL instead of DT on Kitchen | ICQL · Kitchen average | normalized score: 55.8 -> 66.8 | Improved |
| use ICQL instead of IQL on Hammer-Human | ICQL · Hammer-Human-v1 | normalized score: 7.2 -> 3.7 | Hurt |
| use state-similar retrieval instead of random retrieval (ablation) | ICQL · Kitchen-Partial-v0 | normalized score: 47.5 -> 61.5 | Improved |
| use state-similar retrieval instead of random retrieval (ablation) | ICQL · Hopper-Medium-v2 | normalized score: 74.1 -> 62.6 | Hurt |
| add reward filtering to state-similar retrieval (ablation) | ICQL · Walker2d-Medium-v2 / Kitchen-Complete | normalized score: 80.3 -> 83.9 (Walker2d-Medium); 79.3 -> 71.3 (Kitchen-Complete) | Mixed |
| use linear Transformer instead of standard Transformer (ablation) | ICQL · Walker2d-Medium-Replay | normalized score: 42.9 -> 81.9 | Improved |
| use linear Transformer instead of linear MLP (ablation) | ICQL · Pen-Human | normalized score: 66.6 -> 85.6 | Improved |
| use linear MLP instead of linear Transformer on Hammer (ablation) | ICQL · Hammer-Cloned | normalized score: 4.5 -> 7.9 | Improved |
| increase number of in-context layers from 4 to 20 (ablation) | ICQL · MuJoCo tasks | normalized score | Mixed |
| increase context length beyond 20 (ablation) | ICQL · Gym tasks | normalized score | Hurt |
Closest papers by meaning
- On the Complexity of Offline Reinforcement Learning with Q*-Approximation and Partial Coverage (NeurIPS 2026 · similarity 0.90)
- Model-based Offline Reinforcement Learning with Lower Expectile Q-Learning (ICLR 2025 · similarity 0.89)
- From Weak Data to Strong Policy: Q-Targets Enable Provable In-Context Reinforcement Learning (NeurIPS 2026 · similarity 0.89)
- A Perspective of Q-value Estimation on Offline-to-Online Reinforcement Learning (AAAI 2024 · similarity 0.89)
- Q-SFT: Q-Learning for Language Models via Supervised Fine-Tuning (ICLR 2025 · similarity 0.89)
- Compositional Conservatism: A Transductive Approach in Offline Reinforcement Learning (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
