Research map / Large language models / Language, safety and interpretability

Context-Aware Hierarchical Learning: A Two-Step Paradigm towards Safer LLMs

NeurIPS 2025 · Large language models › Language, safety and interpretability · approach: Safety alignment of LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Tool-Completion Attack (TCA), a prompt-injection attack hidden in fake tool-completion contexts, and the Tool-Completion benchmark, and proposes Context-Aware Hierarchical Learning (CAHL), which adds segment summarization and global self-attention on top of ISE to improve instruction hierarchy robustness.

Problem

LLMs process all tokens uniformly and lack role-based instruction hierarchy, making them vulnerable to prompt injections, especially injected instructions hidden in tool-call responses that appear as legitimate task completions.

Approach

Segment query embeddings with a segment-masked cross-attention summarization per instruction role, followed by global self-attention propagation, added to token and instruction-segment embeddings to build a context-aware instruction hierarchy; combined with a new tool-completion attack that creates a false task completion before injecting the instruction.

Key results

• On the Tool-Completion benchmark, CAHL (clean) reduces ASR to 44.89% vs 56.72% for Delimiter and 57.53% for ISE, with capability 77.12 vs 72.09 for Delimiter.
• Table of TCA results shows o3-mini 99.4%, GPT-4o 91.4%, and DeepSeek-R1 99.3% ASR on the TCA-attacked benchmark.
• On the clean StruQ benchmark, CAHL reduces average ASR to 26.32% from 33.41% for ISE, and worst-case ASR to 37.98% from 61.54%.

Measured effects reported in the paper

Surprising findings

• Large reasoning models such as o3-mini and DeepSeek-R1 show near-perfect ASR under TCA, suggesting chain-of-thought does not defend against it.
• Clean-only fine-tuned ISE shows a marginal drop in injection ignorance versus the Delimiter baseline on the Tool-Completion benchmark.

Limitations

• Study covers only a single prompt injection vector in single-turn and multi-turn settings.
• Training restricted to supervised fine-tuning, which may reduce cross-domain generalizability.
• Attack evaluation uses only a low-risk 'print Hacked' target instruction with keyword matching; the adversarial training set is used as an optimization upper bound.

Experimental setup

Datasets: Alpaca-Cleaned AlpacaFarm BUTTON Tensor Trust

Benchmarks: StruQ benchmark Tool-Completion benchmark GCG attack evaluation

Metrics: Attack Success Rate (ASR) Capability (AlpacaEval win rate) Injection Ignorance Rate (IIR) manual inspection ASR

Baselines: Text (vanilla delimiter) Delimiter (StruQ) ISE o3-mini GPT-4o DeepSeek-R1 Llama-3.1-8B-Instruct Qwen2.5-7B-Instruct

Builds on: Instructional Segment Embedding (ISE) StruQ BUTTON Circuit Breakers

Closest papers by meaning

• Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchy (ICLR 2025 · similarity 0.90)
• Scaling Instruction-tuned LLMs to Million-token Contexts via Hierarchical Synthetic Data Generation (ICLR 2025 · similarity 0.88)
• BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models (NeurIPS 2025 · similarity 0.87)
• Simulate and Eliminate: Revoke Backdoors for Generative Large Language Models (AAAI 2025 · similarity 0.87)
• Evaluating the Zero-shot Robustness of Instruction-tuned Language Models (ICLR 2024 · similarity 0.87)
• RigorLLM: Resilient Guardrails for Large Language Models against Undesired Content (ICML 2024 · similarity 0.87)