Research map / Reinforcement learning / Offline RL
Mildly Conservative Q-Learning for Offline Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use lambda<1 OOD pseudo-target training instead of lambda=1 (plain SAC) (ablation) | MCQ · D4RL MuJoCo | normalized score | Helped |
| lower lambda (0.3, 0.5) (ablation) | MCQ · halfcheetah-medium / hopper-medium-replay | normalized score | Hurt |
| vary lambda in 0.7 to <1 (ablation) | MCQ · D4RL MuJoCo | normalized score | Helped |
| vary number of sampled actions N (5,10,20,50) (ablation) | MCQ · halfcheetah-medium / hopper-medium-replay | normalized score | No clear effect |
| replace min with mean in OOD pseudo target (ablation) | MCQ · D4RL MuJoCo | normalized score | No clear effect |
| MCQ vs IQL on non-expert datasets | MCQ · D4RL MuJoCo non-expert average | normalized score: 59.9 -> 72.8 | Helped |
| MCQ vs IQL over all 15 datasets | MCQ · D4RL MuJoCo total | normalized score: 68.9 -> 79.2 | Helped |
| MCQ vs TD3+BC on walker2d-expert | MCQ · walker2d-expert-v2 | normalized score: 110.2±0.3 -> 107.2±1.1 | Hurt |
| offline-to-online fine-tuning with MCQ vs CQL/TD3+BC/IQL/AWAC | MCQ · 6 D4RL MuJoCo tasks | return | Helped |
Closest papers by meaning
- Doubly Mild Generalization for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.92)
- Exclusively Penalized Q-learning for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.92)
- On the Complexity of Offline Reinforcement Learning with Q*-Approximation and Partial Coverage (NeurIPS 2026 · similarity 0.92)
- A Perspective of Q-value Estimation on Offline-to-Online Reinforcement Learning (AAAI 2024 · similarity 0.91)
- Q-Distribution guided Q-learning for offline reinforcement learning: Uncertainty penalized Q-value via consistency model (NeurIPS 2024 · similarity 0.91)
- Adaptive Neighborhood-Constrained Q Learning for Offline Reinforcement Learning (NeurIPS 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.