Research map / Multi-agent systems and games / Human-AI and goal-conditioned
A Behavioural and Representational Evaluation of Goal-Directedness in Language Model Agents
Raghu Arghal, Phoebe Chen, Niall Dalton, Evgenii Kortukov, Calum McNamara, Angelos Nalmpantis, Moksh Nirvaan, Gabriele Sarti, Mario Giulianelli
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase grid size and obstacle density | GPT-OSS-20B agent · MiniGrid navigation | per-action accuracy | Hurt |
| Apply iso-difficulty transforms (reflect, rotate, swap start/goal, transpose) | GPT-OSS-20B agent · MiniGrid navigation | behavioural metrics | No clear effect |
| Add a functionless key (door removed) | GPT-OSS-20B agent · KeyNoDoorEnv | per-action accuracy (%): 98.7 -> 97.2 | Hurt |
| Add vestigial key on one of two optimal paths | GPT-OSS-20B agent · 2PathKeyEnv | success rate (%): 75.5 -> 71.4 | Hurt |
| Add vestigial key on one of two optimal paths | GPT-OSS-20B agent · 2PathKeyEnv | per-action accuracy (%): 74.3 -> 76.0 | Improved |
| Use MLP probe instead of linear probe for cognitive maps (ablation) | probe on layer-15 pre-reasoning activations · 11x11 grids | cell-type accuracy (%): 39.1 -> 75.7 | Improved |
| Decode after reasoning instead of before (ablation) | cognitive map MLP probe · grid cognitive map | overall probe accuracy (%): 75 -> 60 | Hurt |
| Decode plans from post-reasoning instead of pre-reasoning activations (N=1) (ablation) | 2-layer plan decoder · plan decoding | prefix accuracy (%): 41.5 -> 66.49 | Improved |
| Decode plans from pre-reasoning instead of post-reasoning activations for long prefixes (ablation) | 2-layer plan decoder · plan decoding, N>=5 | prefix accuracy | Improved |
| Increase plan decoder depth from 2 to 4 layers (ablation) | plan decoder · plan decoding | prefix accuracy | No clear effect |
| Evaluate actions against top-k decoded grids instead of top-1 (ablation) | decoded cognitive maps · action optimality | action accuracy | Improved |
| Patch activations at a single layer on chat template tokens (ablation) | GPT-OSS-20B · activation patching | action distribution | No clear effect |
Closest papers by meaning
- Do LLMs Build World Representations? Probing Through the Lens of State Abstraction (NeurIPS 2024 · similarity 0.89)
- From Assumptions to Actions: Turning LLM Reasoning into Uncertainty-Aware Planning for Embodied Agents (ICLR 2026 · similarity 0.88)
- What Do Agents Learn from Trajectory-SFT: Semantics or Interfaces? (ICML 2026 · similarity 0.88)
- Embodied Task Planning via Graph-Informed Action Generation with Large Language Models (ICML 2026 · similarity 0.88)
- Expanding LLM Agent Boundaries with Strategy-Guided Exploration (NeurIPS 2026 · similarity 0.87)
- General agents need world models (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
