Research map / Multi-agent systems and games / LLM agents

Agent MechSuits : Mechanistic Subspace Safety Steering for Multi-Turn CLI Agents

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Agent safety and attack benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Agent MechSuits, a white-box runtime safety framework that probes step-level hidden states of CLI agents to detect harmful steps and steers a 10-dimensional subspace at one layer to suppress harmful actions, evaluated on a new MAS benchmark of 194 tasks.

Problem

Safety risks in multi-turn CLI agents accumulate across interaction steps, while existing defenses are external guardrails and mechanistic safety methods are limited to single-turn or jailbreak QA settings.

Approach

Train a single step-level linear probe on last-token hidden states of the CLI agent, reuse it across all steps for detection, and steer the agent by adding a scaled vector on the top-10 probe-weight dimensions at one layer, with the steering strength chosen by an LLM controller.

Key results

• Agent MechSuits reaches 99.15% current-step accuracy on LLaMA-3.1-8B, 97.46% on Qwen-2.5-7B and 95.35% on Gemma-2-9B, beating the best LLM-judge and guardrail baselines.
• Mitigation reduces average ASR from 85.99% to 13.36%, a 72.63-point absolute reduction, but raises average collapse to 18.54% versus 2.24% for vanilla.
• Benign task success stays at 91-96% across the three models with the full pipeline enabled, versus 100% for vanilla.

Surprising findings

• Lookahead detection by LLM judges is highly unstable, with Gemini-3-Pro and DeepSeek-V4 recalling as little as 7.69% of lookahead steps on LLaMA-3.1-8B.
• Reversing the steering direction on refusal steps converts refusals into malicious commands, which the authors read as causal evidence for the subspace.

Limitations

• Small benchmark with sparse lookahead subsets (13 to 31 positive examples) and imbalanced labels.
• Steering introduces generation collapse (18.54% average) and depends on an external LLM judge for alpha search.
• Probe does not reliably trigger on prompt-injection distribution shift, so detection generalization is limited.

Experimental setup

Datasets: AdvCUA Safe-OS AgentBench

Benchmarks: MAS

Metrics: accuracy F1 lookahead recall ASR collapse rate task success rate

Baselines: GPT-4o GPT-5.2 Claude Sonnet-4.6 Claude Opus-4.6 Gemini-3-Pro DeepSeek-V4-Flash DeepSeek-V4-Pro LLaMA-Guard-4-12B AGrail RepE Self Reminder

Builds on: linear representation hypothesis ReAct RepE AGrail

Closest papers by meaning

• Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agents (ICML 2026 · similarity 0.89)
• LITMUS: Benchmarking Behavioral Jailbreaks of LLM Agents in Real OS Environments (NeurIPS 2026 · similarity 0.89)
• Speculative Safety Honeypot: Toward Proactive Defense Against Multi-turn Agent Attacks (ICML 2026 · similarity 0.86)
• SENTINEL: A Multi-Level Formal Framework for Safety Evaluation of Foundation Model-based Embodied Agents (NeurIPS 2026 · similarity 0.86)
• OpenAgentSafety: A Comprehensive Framework For Evaluating Real-World AI Agent Safety (ICLR 2026 · similarity 0.86)
• PROACT-Agent: Progressive Runtime Oversight and Active Circuit-breaking for Real-Time Safety (NeurIPS 2026 · similarity 0.86)