Research map / Reinforcement learning / MDPs and planning
Reinforcement Learning with Non-Markovian Rewards
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use L* instead of EDSM to learn reward automata | Q-learning and R-max on non-Markovian multi-armed bandit tasks · Non-Markovian MAB | average reward | Helped |
| Learn reward automata with EDSM rather than L* | R-max in robot world · Robot world, reward scheme 1 | policy reward | Helped |
| Learn reward automata with L* rather than EDSM | Q-learning in robot world · Robot world, reward scheme 3 | policy reward | Helped |
| Use EDSM to learn reward automata | Q-learning in robot world · Robot world, reward scheme 2 | policy reward | Helped |
| Treat the reward as Markovian without learning an automaton | Vanilla Q-learning and R-max · Robot world, reward scheme 3 | policy reward | Hurt |
| Use R-max rather than Q-learning as the underlying RL algorithm | RL with automata learning · Robot world, reward scheme 1 | policy reward | Helped |
| Use EDSM rather than L* to learn reward automata | RL with automata learning · Robot world | policy reward | Mixed |
| Provide the correct reward automaton ahead of time | Q-learning and R-max · Non-Markovian MAB | steps to converge to optimal policy: less than 100,000 steps | Helped |
| Use L* rather than EDSM for automata learning | Q-learning and R-max · Non-Markovian MAB | average reward | Helped |
| Assume Markovian rewards instead of learning reward automata | Vanilla Q-learning and R-max · Non-Markovian MAB | average reward: 0 reward | Hurt |
Closest papers by meaning
- LTLf/LDLf Non-Markovian Rewards (AAAI 2018 · similarity 0.89)
- Efficient Reinforcement Learning in Probabilistic Reward Machines (AAAI 2025 · similarity 0.89)
- Zeroth-Order Policy Gradient for Reinforcement Learning from Human Feedback without Reward Inference (ICLR 2025 · similarity 0.88)
- A Theoretical Framework for Partially-Observed Reward States in RLHF (ICLR 2025 · similarity 0.88)
- Hybrid Reinforcement Learning Breaks Sample Size Barriers In Linear MDPs (NeurIPS 2024 · similarity 0.87)
- A PAC Learning Algorithm for LTL and Omega-Regular Objectives in MDPs (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.