Research map / Large language models / Language, safety and interpretability
Copyright Traps for Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes injecting synthetic copyright trap sequences into books to enable document-level membership inference on a 1.3B LLM (CroissantLLM) trained from scratch that does not naturally memorize.
Problem
Existing document-level membership inference relies on naturally occurring memorization, which fails for medium-size models such as a 1.3B LLM, leaving content creators unable to verify whether their copyrighted content was used in training.
Approach
Insert purposefully generated trap sequences (synthetic or real excerpts, varied in length, perplexity, and repetition count) into original documents before training, then run sequence-level MIAs on the trap as a proxy for document membership, using a randomized controlled setup with a model trained from scratch.
Key results
- Short/medium trap sequences (25-50 tokens) repeated 100 times are not detectable (AUC about 0.49-0.52), while 100-token sequences repeated 1,000 times reach AUC 0.748 with the Ratio attack.
- Existing document-level MIAs perform near chance on the 1.3B target model (AUC 0.513 and 0.524), versus 0.86 and 0.88 in their original setups.
- Detectability persists under reduced precision, with AUC 0.697 at int4 for the 100-token, 1,000-repetition setting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use injected copyright traps instead of relying on naturally occurring memorization | CroissantLLM 1.3B document membership inference · Books without traps vs. injected traps | AUC: 0.513 ± 0.021; 0.524 ± 0.003; up to 0.748 | Helped |
| Increase synthetic trap repetitions from 1 to 100 (ablation) | Ratio MIA on 25-token synthetic traps · Synthetic traps, 25 tokens | AUC: 0.490 -> 0.524 | No clear effect |
| Increase synthetic trap repetitions from 1 to 1,000 (ablation) | Ratio MIA on 25-token synthetic traps · Synthetic traps, 25 tokens | AUC: 0.490 -> 0.557 | No clear effect |
| Increase synthetic trap repetitions from 100 to 1,000 (ablation) | Ratio MIA on 50-token synthetic traps · Synthetic traps, 50 tokens | AUC: 0.521 -> 0.627 | Helped |
| Increase synthetic trap length from 50 to 100 tokens at 1,000 repetitions (ablation) | Ratio MIA · Synthetic traps, 1,000 repetitions | AUC: 0.627 -> 0.748 | Helped |
| Use a 50-token excerpt from the document as the trap, repeated 100 times | Ratio MIA · Real traps, 50 tokens, 100 repetitions | AUC: 0.492 | No clear effect |
| Add 100 tokens of preceding document context (ablation) | Ratio MIA on synthetic traps · 25-token traps, 1,000 repetitions | AUC: 0.557 -> 0.603 | Helped |
| Add 100 tokens of preceding document context (ablation) | Ratio MIA on synthetic traps · 50-token traps, 100 or 1,000 repetitions | AUC: 0.521 -> 0.581; 0.627 -> 0.685 | Helped |
| Add 100 tokens of preceding document context (ablation) | Ratio MIA on synthetic traps · 100-token traps, 1,000 repetitions | AUC: 0.748 -> 0.739 | No clear effect |
| Load target model at reduced parameter precision (ablation) | Ratio MIA on 100-token synthetic traps repeated 1,000 times · CroissantLLM 1.3B | AUC: 0.748 -> 0.745 -> 0.738 -> 0.697 | Hurt |
| Increase trap-sequence perplexity (ablation) | Ratio MIA on 100-token synthetic traps repeated 1,000 times · Perplexity buckets | AUC: approximately 0.65 -> 0.8 | Helped |
| Increase the number of training steps and exposures to traps (ablation) | Ratio MIA on synthetic traps repeated 1,000 times · Intermediate CroissantLLM checkpoints | AUC | Helped |
Surprising findings
- Repeating a trap sentence 100 times is not enough for detection by any method, contrary to the intuition from training-data extraction literature.
- Higher-perplexity sequences are more detectable (Pearson 0.715), and naturally repeated sequences in The Pile have lower perplexity, suggesting perplexity is a confounder in post-hoc memorization studies.
Limitations
- Traps must be long and repeated up to 1,000 times to be detectable, which may hurt readability of the content.
- Only a single 1.3B model was trained, so generality across model sizes and training setups is not established.
- Attacker-side assumption of non-trivial traps; the paper notes that deduplication or malicious developers could evade detection.
Experimental setup
Datasets: Project Gutenberg public-domain books (9,542 books, excluding PG-19) PG-19 The Pile (non-copyrighted version) CroissantLLM pretraining corpus
Benchmarks: document-level MIA on Project Gutenberg books synthetic trap sequence MIA
Metrics: AUC
Baselines: Loss attack Min-K% Prob Ratio attack document-level MIA (Shi et al.) document-level MIA (Duan et al.)
Builds on: Min-K% Prob Ratio membership inference attack document-level membership inference (Shi et al. / Duan et al.) CroissantLLM
Closest papers by meaning
- Can Watermarking Large Language Models Prevent Copyrighted Text Generation and Hide Training Data? (AAAI 2025 · similarity 0.89)
- Evaluating Copyright Takedown Methods for Language Models (NeurIPS 2024 · similarity 0.89)
- DE-COP: Detecting Copyrighted Content in Language Models Training Data (ICML 2024 · similarity 0.89)
- Positional Fragility in LLMs: How Offset Effects Reshape Our Understanding of Memorization Risks (NeurIPS 2025 · similarity 0.88)
- Quantifying and Analyzing Entity-Level Memorization in Large Language Models (AAAI 2024 · similarity 0.88)
- Detecting Pretraining Data from Large Language Models (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.