Research map / Reinforcement learning / Policy gradient and actor-critic
Composing Value Functions in Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Compose base Q-functions via maximum (-OR- composition) in entropy-regularised RL | Blue and Purple base Q-functions · Video game PurpleOrBlue task | qualitative value function and returns | No clear effect |
| Low-temperature limit of entropy regularisation (max over Q-functions) for standard RL composition | Standard RL -OR- composition with Blue and Purple tasks · Video game | qualitative | No clear effect |
| Averaging library Q-functions (-AND- composition) instead of learning the intersection task | Blue and Square base policies · Video game BlueSquare task | IQR of returns from 50k episodes | Mixed |
| Assume stochastic dynamics instead of deterministic | -OR- composition theorem · Three-state counterexample MDP | return of composed policy: composed policy returns 0.9 vs optimal 1 | Hurt |
Closest papers by meaning
- Utilizing Prior Solutions for Reward Shaping and Composition in Entropy-Regularized Reinforcement Learning (AAAI 2023 · similarity 0.91)
- A Boolean Task Algebra for Reinforcement Learning (NeurIPS 2020 · similarity 0.89)
- Composing Entropic Policies using Divergence Correction (ICML 2019 · similarity 0.89)
- Generalisation in Lifelong Reinforcement Learning through Logical Composition (ICLR 2022 · similarity 0.89)
- When Do Skills Help Reinforcement Learning? A Theoretical Analysis of Temporal Abstractions (ICML 2024 · similarity 0.87)
- Convergence Theorems for Entropy-Regularized and Distributional Reinforcement Learning (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.