Research map / Reinforcement learning / Imitation learning
Meta-Inverse Reinforcement Learning for Mean Field Games via Probabilistic Context Variables
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PEMMFIRL, a meta inverse RL method for mean field games that infers rewards from multi-task expert demonstrations via probabilistic context variables, without knowing the contexts or modifying the MFG.
Problem
Inverse reinforcement learning in mean field games assumes homogeneous agents, but real demonstrations come from heterogeneous agents with distinct and unknown reward contexts (e.g., taxi drivers with different preferences).
Approach
Attach a probabilistic context variable externally to a family of MFGs rather than to agents inside one MFG, and combine mean-field adversarial IRL with a learned context inference model q(m|tau) trained via mutual information regularisation, using a variational estimate of the context-conditioned mean field.
Key results
- On the taxi pricing problem with one-month data, the learned policy raised average driver profit by 2.8% (+$0.1308 per ride) with a 0.4% drop in served passengers at eta=5.
- For a higher decay proportion (0.7%), average driver profit rose by 3.1% (+$0.1448 per ride).
- On simulated MFG tasks, PEMMFIRL policies deviate only slightly from the expert while PLIRL and MF-AIRL show large deviations, with about 32% lower average variance in returns and policy deviations.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply PEMMFIRL's learned pricing policy instead of the empirical MFG model's optimal policy | Taxi drivers on NYC passenger-demand data · NYC taxi rides, η=5 | average profit: +2.8% | Helped |
| Apply PEMMFIRL's learned pricing policy instead of the empirical MFG model's optimal policy | Taxi drivers on NYC passenger-demand data · NYC taxi rides, η=10 | average profit: +2.3% | Helped |
| Apply PEMMFIRL's learned pricing policy instead of the empirical MFG model's optimal policy | Taxi drivers on NYC passenger-demand data · NYC taxi rides, η=15 | average profit: +3.4% | Helped |
| Apply PEMMFIRL's learned pricing policy instead of the empirical MFG model's optimal policy | Taxi drivers on NYC passenger-demand data · NYC taxi rides, η=20 | average profit: +3.1% | Helped |
| Use the learned pricing policy | Taxi service on NYC passenger-demand data · NYC taxi rides, η=5 | served passengers: - 0.4% | Hurt |
| Use the learned pricing policy | Taxi service on NYC passenger-demand data · NYC taxi rides, η=10 | served passengers: - 0.5% | Hurt |
| Use the learned pricing policy | Taxi service on NYC passenger-demand data · NYC taxi rides, η=15 | served passengers: - 0.6% | Hurt |
| Use the learned pricing policy | Taxi service on NYC passenger-demand data · NYC taxi rides, η=20 | served passengers: - 0.7% | Hurt |
| Vary η from 5 to 20 | PEMMFIRL taxi pricing policy · NYC taxi rides | average profit increase: +2.8%, +2.3%, +3.4%, +3.1% | Mixed |
Limitations
- Context variable is discrete and the number of contexts is assumed known/small (e.g., m in {1,2}).
- Runtime is slightly higher than baselines due to the additional context inference model.
- Real-world evaluation relies on an empirical MFG model fitted in prior work and a learned profit model, so gains are measured under modelled rather than observed profits.
Experimental setup
Datasets: New York Yellow Taxi Dataset Virus infection simulated MFG Malware spread simulated MFG Investment in product quality simulated MFG
Benchmarks: Virus infection Malware spread Investment in product quality New York Yellow Taxi spatial pricing
Metrics: policy deviation (KL divergence) expected return under ground-truth reward average driver profit passenger decay rate
Baselines: PLIRL MF-AIRL
Builds on: MF-AIRL meta-IRL adversarial IRL entropy-regularised mean field games PEARL
Closest papers by meaning
- Recent Advancements in Inverse Reinforcement Learning (AAAI 2024 · similarity 0.88)
- Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective (ICML 2024 · similarity 0.88)
- On Feasible Rewards in Multi-Agent Inverse Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
- Offline Inverse RL: New Solution Concepts and Provably Efficient Algorithms (ICML 2024 · similarity 0.87)
- Non-Adversarial Inverse Reinforcement Learning via Successor Feature Matching (ICLR 2025 · similarity 0.87)
- Rethinking Inverse Reinforcement Learning: from Data Alignment to Task Alignment (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.