Research map / Adversarial robustness and security / Attacks on text and graphs

Learning to Inject: Automated Prompt Injection via Reinforcement Learning

NeurIPS 2026 · Adversarial robustness and security › Attacks on text and graphs · approach: LLM jailbreak and injection defense · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AutoInject, a black-box GRPO-based RL framework that learns adversarial suffixes for prompt injection, using a learned comparison-based feedback model to densify the binary success reward, and reports gains over template, GCG, TAP and evolutionary baselines on AgentDojo.

Problem

Automated prompt injection attacks on LLM agents require emitting specific tool calls with correct arguments; the binary success signal is sparse, so random suffixes rarely succeed and standard jailbreak optimizers lack a useful gradient.

Approach

Score each candidate suffix against the best suffix seen so far with a learned LLM comparison (preference probability), turning the binary success signal into a dense reward used for GRPO training of a small suffix-generating policy; supports online query-based attacks and offline transferable suffixes.

Key results

• AutoInject reaches 58.00% ASR on Gemini-2.5-flash vs. under 25% for the best template attack (full AgentDojo, n=949).
• AutoInject achieves 21.88% ASR on Meta-SecAlign-70B where all template attacks are at or near 0%.
• In the GPT-5-nano ablation, removing the learned feedback model drops ASR from 36.57% to 25.00%, and removing GRPO drops it to 19.44%.

Surprising findings

• Learned suffixes containing the nonsensical token 'allelujah' transfer across several production models, compromising up to 70 task pairs on Gemini-2.5-flash.
• On Gemini-2.0-flash and GPT-4o-mini, utility under attack for AutoInject exceeds the no-attack baseline.

Limitations

• Relies on GPT-4o-mini feedback model and API victim queries, so cost and dependence on closed APIs; results are limited to AgentDojo.
• Claude-3.5-Sonnet results are on the banking suite only and are affected by Bedrock Guardrails filtering via OpenRouter.
• Cross-task transfer substantially degrades utility relative to the no-attack baseline.

Experimental setup

Datasets: AgentDojo

Benchmarks: AgentDojo Meta-SecAlign-70B hardened subset

Metrics: attack success rate utility under attack benign utility McNemar test Mann-Whitney U test

Baselines: Direct Instruction Ignore Previous Important Instructions InjecAgent System Message Tool Knowledge GCG TAP Adaptive Attack Evolutionary Search RL-Hammer

Builds on: GRPO GCG TAP RL-Hammer AgentDojo

Closest papers by meaning

• Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks (NeurIPS 2026 · similarity 0.88)
• Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Use (ICML 2026 · similarity 0.87)
• ChatInject: Abusing Chat Templates for Prompt Injection in LLM Agents (ICLR 2026 · similarity 0.87)
• Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks (NeurIPS 2024 · similarity 0.87)
• GASP: Efficient Black-Box Generation of Adversarial Suffixes for Jailbreaking LLMs (NeurIPS 2025 · similarity 0.86)
• Claudini: Autoresearch Discovers State-of-the-Art Adversarial Attack Algorithms for LLMs (NeurIPS 2026 · similarity 0.86)