Research map / Reinforcement learning / Offline RL
Moment Matching Q-Learning
Yiyan Liang Edgar, Sifei Liu, Weitong Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add online fine-tuning after offline training (halfcheetah-medium-replay) | MoMa QL · D4RL Gym offline-to-online | normalized score: 63.3 -> 80.9 | Improved |
| add online fine-tuning (walker2d-medium-replay) | MoMa QL · D4RL Gym offline-to-online | normalized score: 104.3 -> 117.9 | Improved |
| add online fine-tuning (hopper-medium-replay) | MoMa QL · D4RL Gym offline-to-online | normalized score: 106.5 -> 113.5 | Improved |
| add online fine-tuning (halfcheetah-medium) | MoMa QL · D4RL Gym offline-to-online | normalized score: 72.6 -> 83.1 | Improved |
| add online fine-tuning on expert datasets | MoMa QL · D4RL Gym offline-to-online (expert) | normalized score: 106.9 -> 107.1 | No clear effect |
| vary Q-learning weight eta in [0.5, 1.0] (ablation) | MoMa QL · D4RL | normalized score | No clear effect |
| increase sampling steps N beyond 2 (ablation) | MoMa QL · D4RL | normalized score | No clear effect |
| vary MMD kernel parameters a, b in [1, 8] (ablation) | MoMa QL · D4RL | normalized score | No clear effect |
| vary noise schedule parameters p_mean and p_std (ablation) | MoMa QL · D4RL | normalized score | No clear effect |
| increase sampling steps from 1 to 16 (ablation) | MoMa QL · Gym training time | seconds per 1k steps: 17.70 -> 72.56 | Hurt |
| use moment-matching MMD objective instead of diffusion (Diffusion-BC) | Diffusion-BC · D4RL Gym (behavior cloning) average | normalized score: 76.3 -> 89.8 | Improved |
| use MoMa QL instead of Diffusion-QL on Hopper | Diffusion-QL · D4RL Gym (offline RL) hopper | normalized score: 101.0 -> 99.9 | Hurt |
Closest papers by meaning
- Flow Q-Learning (ICML 2025 · similarity 0.86)
- Online Reward-Weighted Fine-Tuning of Flow Matching with Wasserstein Regularization (ICLR 2025 · similarity 0.84)
- Inductive Moment Matching (ICML 2025 · similarity 0.84)
- PRECISE: SDE-Consistent Stochastic Sampling for RL Post-Training of Flow-Matching Models (NeurIPS 2026 · similarity 0.84)
- Scaling Offline RL via Efficient and Expressive Shortcut Models (NeurIPS 2025 · similarity 0.84)
- Flow Matching for Offline Reinforcement Learning with Discrete Actions (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
