Research map / Multi-agent systems and games / LLM agents

PROACT-Agent: Progressive Runtime Oversight and Active Circuit-breaking for Real-Time Safety

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Agent safety and attack benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PROACT-Agent, a pre-LLM runtime guard trained via SFT (LoRA) on 155,780 bilingual prefix-level states (PROACT-Bench) with progressive unrolling, rationale augmentation, and monotonic label rectification, reaching 91.46% unsafe F1 under source holdout and cutting AgentDojo non-DoS ASR from 20.82% to 0.40%.

Problem

Agent guardrails are retrospective (judging finished trajectories) and lack large-scale, causally consistent prefix-level data for deciding when to block the next LLM inference; existing step-level datasets suffer from label "safety drift" and are English-centric.

Approach

Unroll logs into non-Markovian prefix states, recover missing rationales, label via a multi-model committee with arbitration, enforce monotonic causal consistency by truncating after the first unsafe prefix, and localize tool contexts to Chinese with verification checks; then train a Qwen2.5-7B guard to block before the next inference.

Key results

• Under complete source holdout, PROACT-Agent reaches 91.46% unsafe F1 and 90.63% exact-boundary detection, versus 71.38% F1 for Qwen3Guard-Gen-8B and 57.99% exact boundary for Qwen3Guard.
• On AgentDojo non-DoS targeted attacks, overall ASR drops from 20.82% (no guard) to 0.40% with PROACT-Agent.
• Under strict root grouping, PROACT-Agent reaches 96.34% F1 versus 91.20% for Qwen3Guard-Gen-8B and 84.70% for TS-Guard.

Surprising findings

• Under complete source holdout, the released Qwen3Guard-Gen-8B and TS-Guard drop to about 71-72% F1, indicating weak cross-source transfer of existing guards.
• Many agent guard frameworks (AgentDoG variants, Safiron) show very high recall but low precision, i.e., frequent false blocks.

Limitations

• Localization depends on predefined cultural knowledge base, requiring manual updates as tool ecosystems evolve.
• Reasoning capacity bounded by base LLM; not tested against adaptive multi-agent adversarial attacks.
• No systematic human annotation of first-unsafe boundaries; labels are model-adjudicated, and the guard cannot undo already-executed tool calls.

Experimental setup

Datasets: R-Judge ToolSafety ATBench AgentAlign AgentDojo

Benchmarks: PROACT-Bench AgentDojo

Metrics: accuracy precision recall F1 exact-boundary detection first-unsafe recall targeted attack success rate false-block rate

Baselines: Qwen3Guard-Gen-8B TS-Guard Llama-Guard-3-8B AgentDoG-Qwen3-4B AgentDoG-Qwen2.5-7B AgentDoG-Llama3.1-8B ShieldAgent Safiron Qwen3-Max Gemini 3 Flash DeepSeek-V3.2 Kimi-K2.5

Builds on: LLaMA-Factory LoRA AgentDoG TS-Guard AgentDojo

Closest papers by meaning

• ProAct: A Benchmark and Multimodal Framework for Structure-Aware Proactive Response (ICML 2026 · similarity 0.90)
• Building a Foundational Guardrail for General Agentic Systems via Synthetic Data (ICLR 2026 · similarity 0.89)
• SENTINEL: A Multi-Level Formal Framework for Safety Evaluation of Foundation Model-based Embodied Agents (NeurIPS 2026 · similarity 0.88)
• UniClawBench: A Universal Benchmark for Proactive Agents on Real-World Tasks (NeurIPS 2026 · similarity 0.88)
• Think Twice Before You Act: Enhancing Agent Behavioral Safety with Thought Correction (ICML 2026 · similarity 0.88)
• LiSA: Lifelong Safety Adaptation via Conservative Policy Induction (NeurIPS 2026 · similarity 0.88)