Research map / Multi-agent systems and games / LLM agents
No More, No Less: Task Alignment in Terminal Agents
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces TAB, a benchmark of 89 transformed Terminal-Bench tasks where a necessary cue and a co-located irrelevant distractor sit in the environment, and measures cue utilization, distraction resistance, and task alignment across ten frontier terminal agents and six prompt-injection defenses.
Problem
Terminal agents must act on relevant environmental instructions while ignoring irrelevant or misleading ones, but existing capability benchmarks and robustness benchmarks measure these behaviors in isolation.
Approach
Task alignment is defined as the product of cue utilization (completing the task after observing a necessary cue) and distraction resistance (not executing a co-located distractor), and is measured by converting Terminal-Bench tasks into underspecified variants where cue and distractor share a surface.
Key results
- GPT-5.5 reaches 73% Terminal-Bench resolution but only 23% task alignment, due to 27% distraction resistance.
- Claude Opus 4.7 reaches 72% task alignment with 94% distraction resistance.
- Sanitizer-style defenses such as SIC cut distractor execution (0.65 to 0.03 on GPT-5.4 mini) but also collapse task capability (0.36 to 0.01).
Surprising findings
- The most capable Terminal-Bench agent (GPT-5.5) has lower task alignment than several weaker open-weight agents.
- Presence of a useful cue raises distractor execution by 9 to 19 points.
Limitations
- Distractors are plausible but not explicitly adversarial or harmful; difficulty and harm variation left to future work.
- Only terminal tasks evaluated; generality to other agent domains not tested.
- Six tasks excluded because abstracted instructions remained solvable; abstraction relies on Claude Opus 4.7 as proxy.
Experimental setup
Datasets: Terminal-Bench 2.1
Benchmarks: TAB Terminal-Bench 2.1
Metrics: cue utilization distraction resistance task alignment resolution rate distractor execution rate
Baselines: Spotlighting RUP SIC PromptArmor Task Shield Firewall
Builds on: Terminal-Bench Terminus-2 AgentDojo Spotlighting CaMeL
Closest papers by meaning
- Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces (ICLR 2026 · similarity 0.88)
- What Do Agents Learn from Trajectory-SFT: Semantics or Interfaces? (ICML 2026 · similarity 0.86)
- SePO: Self-Evolving Prompt Agent for System Prompt Optimization (NeurIPS 2026 · similarity 0.86)
- AgentSynth: Scalable Task Generation for Generalist Computer-Use Agents (ICLR 2026 · similarity 0.86)
- ECHO: Terminal Agents Learn World Models for Free (NeurIPS 2026 · similarity 0.86)
- Just Do It!? Computer-Use Agents Exhibit Blind Goal-Directedness (ICLR 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.