Research map / Reinforcement learning / Policy gradient and actor-critic
XQC: Well-conditioned Optimization Accelerates Deep Reinforcement Learning
Daniel Palenicek, Florian Vogt, Joe Watson, Ingmar Posner, Jan Peters
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use batch normalization instead of layer normalization in the critic (ablation) | XQC critic · DMC dog-trot Hessian condition number | condition number: orders of magnitude smaller | Improved |
| use categorical cross-entropy critic loss instead of MSE (ablation) | XQC critic · DMC dog-trot Hessian condition number | condition number | Improved |
| combine BN, weight normalization and CE loss (ablation) | SAC critic · DMC dog-trot | return at 1M steps / conditioning | Improved |
| replace BN with LN in XQC (ablation) | XQC · 55 proprioceptive tasks | aggregate normalized return | Hurt |
| halve learning rate for LN variant (ablation) | XQC with LN · 55 proprioceptive tasks | aggregate normalized return | Hurt |
| replace CE critic loss with MSE (ablation) | XQC · 55 proprioceptive tasks | aggregate normalized return | Hurt |
| remove weight normalization (ablation) | XQC · 55 proprioceptive tasks | aggregate normalized return | Hurt |
| add skip connections at investigated depths (ablation) | XQC · depth scaling | performance | No clear effect |
| vary batch size (128, 512 vs 256), add weight decay, action repetition (ablation) | XQC · ablations | performance | No clear effect |
| increase update-to-data ratio (ablation) | XQC · UTD scaling | AUC of IQM normalized return | Improved |
| remove weight normalization (growing parameter norms) (ablation) | XQC · plasticity analysis | effective learning rate | Hurt |
Closest papers by meaning
- Bigger, Regularized, Optimistic: scaling for compute and sample efficient continuous control (NeurIPS 2024 · similarity 0.88)
- CrossQ: Batch Normalization in Deep Reinforcement Learning for Greater Sample Efficiency and Simplicity (ICLR 2024 · similarity 0.88)
- Hyperspherical Normalization for Scalable Deep Reinforcement Learning (ICML 2025 · similarity 0.87)
- Quality-Diversity Actor-Critic: Learning High-Performing and Diverse Behaviors via Value and Successor Features Critics (ICML 2024 · similarity 0.87)
- Actor-Free Continuous Control via Structurally Maximizable Q-Functions (NeurIPS 2025 · similarity 0.87)
- Actor-Critics Can Achieve Optimal Sample Efficiency (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
