Research map / Reinforcement learning / Imitation learning
Residual Q-Learning: Offline and Online Policy Customization without Value
Chenran Li, Chen Tang, Haruki Nishimura, Jean Mercat, Masayoshi Tomizuka, Wei Zhan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| customize RL prior with residual Q | soft Q-learning prior · CartPole | mean abs cart position error e_x: 0.15 -> 0.05 | Improved |
| customize IL prior with residual Q | GAIL prior · CartPole | mean abs cart position error e_x: 0.19 -> 0.07 | Improved |
| customize RL prior with residual soft actor-critic | SAC prior · Continuous Mountain Car | negative actions per episode n_neg: 42.26 -> 37.90 | Improved |
| customize IL prior with residual soft actor-critic | GAIL prior · Continuous Mountain Car | negative actions per episode n_neg: 64.40 -> 41.08 | Improved |
| customize RL prior with residual soft Q | soft Q-learning prior · Highway | avg normalized lane index: 0.46 -> 0.97 | Improved |
| customize IL prior with residual soft Q | GAIL prior · Highway | avg normalized lane index: 0.50 -> 0.99 | Improved |
| customize RL prior with residual soft actor-critic | SAC prior · Parking | non-violation rate (%): 57.61 -> 96.09 | Improved |
| customize IL prior with residual soft actor-critic | GAIL prior · Parking | non-violation rate (%): 11.27 -> 66.07 | Improved |
| customize IL prior with residual SAC (basic task side effect) | GAIL prior · Parking | basic task success rate (%): 70.91 -> 83.59 | Improved |
| customize RL prior with residual soft Q | soft Q-learning prior · Highway | basic task reward: 44.67 -> 42.23 | Mixed |
| customize RL prior with residual SAC | SAC prior · Parking | basic task success rate (%): 99.09 -> 98.73 | No clear effect |
| zero-shot residual max-entropy MCTS instead of offline residual Q | RL prior · Highway | avg normalized lane index: 0.97 -> 0.96 | No clear effect |
Closest papers by meaning
- Policy Decorator: Model-Agnostic Online Refinement for Large Policy Model (ICLR 2025 · similarity 0.89)
- Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning (NeurIPS 2024 · similarity 0.89)
- Limited Preference Aided Imitation Learning from Imperfect Demonstrations (ICML 2024 · similarity 0.88)
- Residual-MPPI: Online Policy Customization for Continuous Control (ICLR 2025 · similarity 0.88)
- Reward-free World Models for Online Imitation Learning (ICML 2025 · similarity 0.88)
- Efficient Imitation under Misspecification (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
