Research map / Multi-agent systems and games / Human-AI and goal-conditioned
MAGELLAN: Metacognitive predictions of learning progress guide autotelic LLM agents in large goal spaces
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MAGELLAN, which uses an LLM agent's own representation plus an MLP to predict competence and absolute learning progress over natural-language goals, enabling online curriculum learning without expert goal groupings.
Problem
Estimating learning progress (LP) for goal prioritization in autotelic LLM agents trained with online RL over large, discrete, high-dimensional natural-language goal spaces, where per-goal evaluation is costly and expert goal groupings are brittle.
Approach
Learn a goal-conditioned competence estimator on top of the agent's own LLM (LoRA adapters plus MLP head), so semantically similar goals share competence estimates; ALP is computed by comparing estimates from current and past stored estimator weights, and goals are sampled proportionally to ALP.
Key results
- MAGELLAN matches the competence-estimation accuracy of Eval-ALP and expert-group methods while using online estimates with no extra evaluation episodes.
- MAGELLAN is the only method without expert groupings that reaches at least 90% success rate in all Little-Zoo goal categories within 500k episodes.
- On held-out unseen goals, MAGELLAN's mean competence prediction error is 0.11 versus 0.53 for Online-ALP and 0.09 for EK-Online-ALP.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MAGELLAN instead of Online-ALP to estimate competence on unseen goals | Flan-T5 248M agent · Little-Zoo held-out test, Mean | competence estimation error: 0.53 ± 0.06 -> 0.11 ± 0.06 | Helped |
| Use MAGELLAN instead of Online-ALP to estimate competence on unseen goals | Flan-T5 248M agent · Little-Zoo held-out test, Grasp | competence estimation error: 0.98 ± 0.00 -> 0.01 ± 0.00 | Helped |
| Use MAGELLAN instead of Online-ALP to estimate competence on unseen goals | Flan-T5 248M agent · Little-Zoo held-out test, Grow plant | competence estimation error: 0.78 ± 0.07 -> 0.05 ± 0.03 | Helped |
| Use MAGELLAN instead of Online-ALP to estimate competence on unseen goals | Flan-T5 248M agent · Little-Zoo held-out test, Grow herbivore | competence estimation error: 0.34 ± 0.18 -> 0.08 ± 0.05 | Helped |
| Use MAGELLAN instead of Online-ALP to estimate competence on unseen goals | Flan-T5 248M agent · Little-Zoo held-out test, Grow carnivore | competence estimation error: 0.00 ± 0.00 -> 0.30 ± 0.16 | Hurt |
| Use MAGELLAN instead of Online-ALP to prioritize goals | Flan-T5 248M agent · Little-Zoo, 25k training goals | success rate | Helped |
| Use MAGELLAN instead of EK-Online-ALP to prioritize goals | Flan-T5 248M agent · Little-Zoo, 25k training goals | time to category mastery | Hurt |
| Use separate LoRA adapters for the policy and MAGELLAN (ablation) | MAGELLAN · Little-Zoo training, non-Grasp goal categories | learning progress | Helped |
| Share LoRA adapters between the policy and MAGELLAN (ablation) | MAGELLAN · Little-Zoo training, Grow carnivore | goal mastery | Hurt |
| Use the frozen pretrained LLM representation for MAGELLAN (ablation) | MAGELLAN · Little-Zoo training | goal mastery | Hurt |
| Train shared adapters using only the policy loss (ablation) | MAGELLAN · Little-Zoo training | goal mastery | Hurt |
Surprising findings
- Keeping the LLM frozen for the competence estimator leads to poor results, requiring a dynamic representation space.
- EK-Online-ALP learns faster than MAGELLAN despite similar competence estimation, because expert groups let it discard impossible goals more easily during exploration.
Limitations
- Experiments limited to small-scale LLM (Flan-T5 248M) and a controlled synthetic textual environment.
- Impossible goals make up 80% of the goal space, which is a designed choice rather than natural-language distributions.
- LLM agent does not perfectly generalize to unseen goals, causing estimation error especially for Grow carnivore goals.
Experimental setup
Datasets: Little-Zoo OpenR1-Math-220k BabyAI-Text
Benchmarks: Little-Zoo OpenR1-Math-220k (simulated) BabyAI-Text
Metrics: success rate competence estimation error competence estimation cost time to mastery
Baselines: Eval-ALP EK-Eval-ALP Online-ALP EK-Online-ALP Uniform
Builds on: SAC-GLAM ALP-GMM Playground environment LoRA
Closest papers by meaning
- Position: Truly Self-Improving Agents Require Intrinsic Metacognitive Learning (ICML 2025 · similarity 0.86)
- ReMA: Learning to Meta-Think for LLMs with Multi-agent Reinforcement Learning (NeurIPS 2025 · similarity 0.85)
- MAGE: Multi-Agent Self-Evolution with Co-Evolutionary Knowledge Graphs (NeurIPS 2026 · similarity 0.85)
- Adaptive Collaboration with Humans: Metacognitive Policy Optimization for Multi-Agent LLMs with Continual Learning (ICLR 2026 · similarity 0.85)
- Meta-Cognitive Memory Policy Optimization for Long-Horizon LLM Agents (NeurIPS 2026 · similarity 0.85)
- Latent Learning Progress Drives Autonomous Goal Selection in Human Reinforcement Learning (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.