Research map / Reinforcement learning / Policy gradient and actor-critic
Q-Learning with Adjoint Matching
Qiyang Li, Sergey Levine
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the lean adjoint state instead of the basic adjoint matching objective (backprop through SDE) (ablation) | QAM · OGBench offline RL (50 tasks) | success rate (%): 35 -> 44 | Improved |
| use the critic action gradient instead of only its value (advantage weighting) | QAM · OGBench offline RL (50 tasks) | success rate (%): 8 -> 44 | Improved |
| combine QAM with FQL one-step distillation (QAM-FQL) | QAM · OGBench offline RL (50 tasks) | success rate (%): 44 -> 45 | Improved |
| combine QAM with a FEdit edit policy (QAM-EDIT) | QAM · OGBench offline RL (50 tasks) | success rate (%): 44 -> 46 | Improved |
| add SOC formulation with optimal-policy guarantee vs FBRAC direct backprop | BAM · OGBench offline RL (50 tasks) | success rate (%): 11 -> 35 | Improved |
| compare BAM (backprop through SDE) with FQL distilled one-step policy | BAM · OGBench offline RL (50 tasks) | success rate (%): 35 vs 36 | No clear effect |
| enable gradient clipping (ablation) | QAM-EDIT · OGBench offline RL (50 tasks) | success rate (%) | Improved |
| use large critic ensemble K=10 (ablation) | QAM-EDIT · OGBench offline RL (50 tasks) | success rate (%) | Improved |
| increase flow steps T beyond 10 (ablation) | QAM-EDIT · OGBench offline RL (50 tasks) | success rate (%) | No clear effect |
| tune temperature tau (ablation) | QAM-EDIT · OGBench offline RL (50 tasks) | success rate (%) | Improved |
| train on noisy datasets (cube-triple-noisy) instead of play | QAM-EDIT · OGBench cube-triple-noisy | success rate (%) | Mixed |
| train on stitch datasets instead of navigate for locomotion | QAM-EDIT · OGBench stitch navigation | success rate (%) | No clear effect |
Closest papers by meaning
- Trust Region Q Adjoint Matching (NeurIPS 2026 · similarity 0.91)
- Scalable Maximum Entropy Reinforcement Learning for Diffusion Policies via Adjoint Matching (NeurIPS 2026 · similarity 0.89)
- Diffusion-based Reinforcement Learning via Q-weighted Variational Policy Optimization (NeurIPS 2024 · similarity 0.88)
- Flow Q-Learning (ICML 2025 · similarity 0.88)
- Flow Matching for Offline Reinforcement Learning with Discrete Actions (NeurIPS 2026 · similarity 0.87)
- Efficient Adjoint Matching for Fine-tuning Diffusion Models (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
