Research map / Multi-agent systems and games / LLM agents

SafeHarbor: Defining Precise Decision Boundaries via Hierarchical Memory-Augmented Guardrail for LLM Agent Safety

ICML 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent collaboration and safety · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SafeHarbor, a training-free hierarchical memory guardrail for LLM agents that retrieves harm prohibitions and benign exemptions, with a small contrastive safety projector and LLM judgment, to cut over-refusal while keeping high refusal on harmful agent requests.

Problem

LLM agent guardrails either over-refuse benign but ambiguous tool-use requests or miss harmful agentic attacks; static classifiers draw coarse decision boundaries that fail to adapt to context.

Approach

Builds a self-organizing hierarchical memory of harm-prohibition and benign-exemption rule pairs, generated by an LLM from adversarially enhanced harmful trajectories and contrasted with nearby benign samples; routes queries via a contrastive MLP projector score and benign-similarity fast path, with LLM judgment for ambiguous cases. Node splitting/merging is driven by an entropy-based information

Key results

• On AgentHarm with GPT-4o, SafeHarbor reaches a benign score of 63.6% (vs 61.3% for A-Mem, 52.4% for LlamaGuard) while keeping harmful refusal at 93.2%.
• On Qwen2.5-7B, SafeHarbor reduces benign refusal to 9.1% vs 22.7% for LlamaGuard, with harmful refusal of 89.2%.
• Average latency is 306.67 ms versus 6433.09 ms for GuardAgent and 379.30 ms for LlamaGuard, at 14 GB VRAM on a single 7B model.

Surprising findings

• Removing benign exemption rules raises harmful refusal slightly (94.3% vs 89.2%) but triples benign refusal, showing exemptions drive the utility gain.
• On Qwen3-32B, SafeHarbor increases benign refusal from 1.1% to 17.6% while raising harmful refusal from 40.8% to 94.3%, showing a notable utility cost on that model.

Limitations

• Benign refusal rises on Qwen3-32B (1.1% to 17.6%), indicating a utility trade-off on some backbones.
• Most evaluations use AgentHarm and AgentSafetyBench plus a memory built from a single dataset (AgentAlign); generalization to other threats is not shown in detail.
• Reported results rely on LLM judges and an LLM-based judgment stage; the paper's own fine-grained analysis of judge sensitivity is in an appendix table.

Experimental setup

Datasets: AgentAlign AgentHarm AgentSafetyBench

Benchmarks: AgentHarm AgentSafetyBench

Metrics: harmful score full score refusal rate non-refusal score intent match noise ratio latency ASR

Baselines: Rule Traverse GuardAgent RAG A-Mem LlamaGuard AgentAlign (SFT)

Builds on: RAG A-Mem GuardAgent AgentAlign

Closest papers by meaning

• A-MemGuard: A Proactive Defense Framework For LLM-Based Agent Memory (ICML 2026 · similarity 0.89)
• DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agents (NeurIPS 2025 · similarity 0.89)
• Safeguarding LLMs via Model-Agnostic Latent Safety Signals from Dark Knowledge (NeurIPS 2026 · similarity 0.89)
• Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents (ICLR 2025 · similarity 0.88)
• Jailbreak Antidote: Runtime Safety-Utility Balance via Sparse Representation Adjustment in Large Language Models (ICLR 2025 · similarity 0.88)
• Metis: Learning to Jailbreak LLMs via Self-Evolving Metacognitive Policy Optimization (ICML 2026 · similarity 0.88)