Research map / Large language models / Language, safety and interpretability
RepeatLeakage: Leak Prompts from Repeating as Large Language Model Is a Good Repeater
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that LLMs repeating context rely on the prompt rather than parametric knowledge, and uses this to design RepeatLeakage, a template-based attack that extracts system prompts and conversation contexts.
Problem
Stealing confidential system prompts of LLM applications (e.g., GPT Store apps) where developers cannot confirm leaked prompts, so the authenticity of stolen prompts is hard to verify.
Approach
When an LLM is asked to repeat, it copies from context rather than parametric knowledge; the authors validate this by contaminating noun embeddings and restoring intermediate layers, then design a repeat-based template (place, format, encouragement) to leak prompts with high confidence.
Key results
- RepeatLeakage reached 71% (Random Leakage), 58% (Targeted Leakage) and 77.8% (System Prompt Leakage) at 99%-100% BERTScore on Llama-3 and Phi-3 results, outperforming Zhang et al. and Perez et al. baselines.
- On Llama-3, repetitive-task RRD ranged 0.0060 to 0.2364 versus knowledge-based tasks, indicating repetition does not depend on parametric knowledge.
- RepeatLeakage was also tested on the ChatGPT web interface, leaking system prompts and conversation contexts.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RepeatLeakage instead of the strongest manual leakage baseline | Llama-3 · Random Leakage (ShareGPT) | 99%-100% BertScore rate: 6.0% -> 63.0% | Helped |
| Use RepeatLeakage instead of the strongest manual leakage baseline | Llama-3 · Targeted Leakage (ShareGPT) | 99%-100% BertScore rate: 2.0% -> 36.0% | Helped |
| Use RepeatLeakage instead of the strongest manual leakage baseline | Llama-3 · System Prompt Leakage (GPTs) | 99%-100% BertScore rate: 44.4% -> 68.9% | Helped |
| Use RepeatLeakage instead of the strongest manual leakage baseline | Phi-3 · Random Leakage (ShareGPT) | 99%-100% BertScore rate: 5.0% -> 71.0% | Helped |
| Use RepeatLeakage instead of the strongest manual leakage baseline | Phi-3 · Targeted Leakage (ShareGPT) | 99%-100% BertScore rate: 7.0% -> 58.0% | Helped |
| Use RepeatLeakage instead of the strongest manual leakage baseline | Phi-3 · System Prompt Leakage (GPTs) | 99%-100% BertScore rate: 30.0% -> 77.8% | Helped |
| Use RepeatLeakage instead of the strongest manual leakage baseline | Qwen2 · Random Leakage (ShareGPT) | 99%-100% BertScore rate: 3.0% -> 64.0% | Helped |
| Use RepeatLeakage instead of the strongest manual leakage baseline | Qwen2 · Targeted Leakage (ShareGPT) | 99%-100% BertScore rate: 9.0% -> 30.0% | Helped |
| Use RepeatLeakage instead of the strongest manual leakage baseline | Qwen2 · System Prompt Leakage (GPTs) | 99%-100% BertScore rate: 34.4% -> 72.2% | Helped |
Surprising findings
- Qwen2 on interrogative questions showed anomalous behavior, attributed to confusion between repeating and answering.
Limitations
- Relies on BERTScore similarity thresholds rather than exact-match verification of leaked prompts
- Qwen2 anomaly on interrogative repeat tasks shows sensitivity to phrasing
- Ground truth for GPT Store and ChatGPT leakage is only available from public datasets
Experimental setup
Datasets: Known Facts Dataset Simple Questions Wikidata GPT Store leaked system prompts ShareGPT conversations
Benchmarks: GPT Store leaked system prompts ShareGPT ChatGPT conversations
Metrics: BERTScore similarity ranges contaminated degree restored degree accuracy of first token
Baselines: Zhang et al. 2024 Perez et al. 2022
Builds on: Perez et al. 2022 prompt leaking Zhang et al. 2024 conversation prompt leakage
Closest papers by meaning
- Auditing Prompt Caching in Language Model APIs (ICML 2025 · similarity 0.86)
- On the Worst Prompt Performance of Large Language Models (NeurIPS 2024 · similarity 0.86)
- OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics (NeurIPS 2025 · similarity 0.86)
- OR-Bench: An Over-Refusal Benchmark for Large Language Models (ICML 2025 · similarity 0.86)
- Keeping an Eye on LLM Unlearning: The Hidden Risk and Remedy (NeurIPS 2025 · similarity 0.86)
- Adjacent Words, Divergent Intents: Jailbreaking Large Language Models via Task Concurrency (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.