Research map / Reinforcement learning / RL from feedback
Non-Markovian Reward Modelling from Trajectory Labels via Interpretable Multiple Instance Learning
Joseph Early, Tom Bewley, Christine Evers, SArvapali Ramchurn
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace Markovian Instance Space NN with Instance Space LSTM | MIL reward model · Timer | return MSE: 130.8 -> 7.313 | Improved |
| Replace Markovian Instance Space NN with CSC Instance Space LSTM | MIL reward model · Moving | return MSE: 22.24 -> 5.307 | Improved |
| Add concatenated skip connection to Instance Space LSTM (ablation) | Instance Space LSTM · Charger | return MSE: 0.628 -> 0.125 | Improved |
| Add concatenated skip connection to Instance Space LSTM (ablation) | Instance Space LSTM · Timer | reward MSE: 0.263 -> 0.073 | Improved |
| Add concatenated skip connection to Instance Space LSTM (ablation) | Instance Space LSTM · Lunar Lander | return MSE: 1.223 -> 0.501 | Improved |
| Add concatenated skip connection to Instance Space LSTM (ablation) | Instance Space LSTM · Key | return MSE: 0.488 -> 0.391 | Mixed |
| Use Embedding Space LSTM with post hoc reward computation instead of instance space reward | Embedding Space LSTM · Timer | reward MSE: 0.263 -> 101.8 | Hurt |
| Use Embedding Space LSTM instead of CSC Instance Space LSTM | Embedding Space LSTM · Lunar Lander | reward MSE: 7.365 -> 120.2 | Hurt |
| Use Embedding Space LSTM instead of CSC Instance Space LSTM | Embedding Space LSTM · Lunar Lander | return MSE: 0.501 -> 0.416 | Improved |
| Use CSC Instance Space LSTM reward model for RL agent training instead of oracle reward and hidden states | SAC / DQN agents · Timer and Lunar Lander | oracle return | Improved |
| Train RL agents without hidden state information | RL agents with oracle reward · five RL tasks | oracle return | Hurt |
| Increase label swap noise from 0 to 0.5 | CSC Instance Space LSTM · five RL tasks | return and reward prediction | Hurt |
Closest papers by meaning
- On the Expressivity of Markov Reward (NeurIPS 2021 · similarity 0.88)
- On the Theory of Reinforcement Learning with Once-per-Episode Feedback (NeurIPS 2021 · similarity 0.88)
- A Theoretical Framework for Partially-Observed Reward States in RLHF (ICLR 2025 · similarity 0.87)
- DUO: Diverse, Uncertain, On-Policy Query Generation and Selection for Reinforcement Learning from Human Feedback (AAAI 2025 · similarity 0.87)
- Leveraging Sub-Optimal Data for Human-in-the-Loop Reinforcement Learning (ICLR 2025 · similarity 0.86)
- Reinforcement Learning from Imperfect Corrective Actions and Proxy Rewards (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
