Research map / Multi-agent systems and games / Games and equilibria
Consistent Aggregation of Objectives with Diverse Time Preferences Requires Non-Markovian Rewards
Silviu Pitis
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Aggregate work and play rewards using fixed Markovian weights despite their different discount factors | Work/play objectives in Procrastinator's Peril · Procrastinator's Peril | trajectory value: 3.20 vs 1.00 | Mixed |
| Use history-dependent aggregation weights updated by each objective's discount factor | Multi-objective aggregation · Procrastinator's Peril | trajectory value: 3.00 -> 3.20 | Improved |
| Increase historical discounting factor from η=0.00 to η=1.00 | Historical discounting · Procrastinator's Peril with shifting preferences | first-step discounted reward: 2.635 -> 3.200 | Improved |
| Use historical discounting with η=0.30 rather than η=0.00 | Historical discounting · Procrastinator's Peril with shifting preferences | first-step discounted reward: 2.635 -> 2.635 | No clear effect |
| Use fixed Markovian stationary-policy optimization instead of a history-dependent policy | Procrastinator's Peril · Procrastinator's Peril | trajectory value: 3.2 -> 3.0 | Hurt |
Closest papers by meaning
- Policy Aggregation (NeurIPS 2024 · similarity 0.88)
- An Analytical Study of Utility Functions in Multi-Objective Reinforcement Learning (NeurIPS 2024 · similarity 0.86)
- AI Alignment with Changing and Influenceable Reward Functions (ICML 2024 · similarity 0.85)
- Beyond Scalar Rewards: An Axiomatic Framework for Lexicographic MDPs (NeurIPS 2025 · similarity 0.85)
- Projection Optimization: A General Framework for Multi-Objective and Multi-Group RLHF (ICML 2025 · similarity 0.85)
- Towards Theoretical Understanding of Sequential Decision Making with Preference Feedback (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
