Research map / Multi-agent systems and games / LLM agents

MetaCipher: A Time-Persistent and Universal Multi-Agent Framework for Cipher-Based Jailbreak Attacks for LLMs

AAAI 2026 · Multi-agent systems and games › LLM agents · approach: Agent safety and attack benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MetaCipher, an RL-guided multi-agent framework that selects among 21 cipher encodings of malicious keywords to jailbreak LLMs in few queries.

Problem

Jailbreak research is costly because querying top-tier black-box LLMs is expensive and effective attacks quickly lose potency as safety updates roll out; existing cipher attacks are single-shot or non-adaptive.

Approach

Encrypt only the malicious keywords (not the full prompt) with a pool of 21 ciphers, adding decryption guidance and placeholder questions, and use a Q-learning cipher selector with Jaccard-based soft sharing of rewards plus a judge agent to iteratively choose ciphers within a small query budget.

Key results

• On JailbreakBench with Falcon3-10B-Instruct, MetaCipher reaches 88% ASR at 10 attempts versus 73% for WordGame+.
• On HarmBench with Falcon3-10B, MetaCipher achieves 83.0% ASR using about 3.5 queries and 0.05 h, versus 7.0% for PiF-T and 1.1% for ArrAttack.
• On ChatGPT-4o web text-to-image, the acrostic-cipher attack yields 41.0% overall success on VBCDE-100 with 0% on pornographic content.

Surprising findings

• Varying ciphers on only the malicious keywords with a fixed template already achieves high ASR, and removing placeholder questions hurts some victims but slightly helps Claude-3.7.
• Reasoning model gemini-2.5-pro reaches 45% ASR under 10 attempts on JailbreakBench only with the adaptive method, while single-cipher attacks are near zero.

Limitations

• Relies on victims decrypting ciphers, so it is less effective against strong reasoning models like gemini-2.5-pro and claude.
• Judge agent may differ from benchmark criteria (false positives/early exits), and ASR on MaliciousInstruct uses the authors' own judge.
• Reported T2I case study is a single cipher on one service and is qualitative in scope.

Experimental setup

Datasets: AdvBench JailbreakBench MaliciousInstruct HarmBench StrongREJECT VBCDE-100

Benchmarks: JailbreakBench MaliciousInstruct HarmBench StrongREJECT VBCDE-100

Metrics: ASR average score queries per prompt time per prompt

Baselines: ACE LACE WordGame+ PiF ArrAttack

Builds on: WordGame ACE KROP SEAL

Closest papers by meaning

• JailPO: A Novel Black-Box Jailbreak Framework via Preference Optimization Against Aligned LLMs (AAAI 2025 · similarity 0.91)
• Lifelong Safety Alignment for Language Models (NeurIPS 2025 · similarity 0.90)
• Metis: Learning to Jailbreak LLMs via Self-Evolving Metacognitive Policy Optimization (ICML 2026 · similarity 0.90)
• Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignment (CVPR 2025 · similarity 0.90)
• Improved Techniques for Optimization-Based Jailbreaking on Large Language Models (ICLR 2025 · similarity 0.89)
• Systematic Scaling Analysis of Jailbreak Attacks in Large Language Models (NeurIPS 2026 · similarity 0.89)