Research map / Large language models / Agents, code and math
Copyright-Bench: Agentic Evaluation of Copyright Law Compliance
Zheng Hui, Doni Bloomfield, Noam Kolt
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add an explicit instruction to respect IP (IP-Aware prompt) vs neutral | Gemini 3 Pro · Copyright-Bench WebDev (Std) | Violation rate (%): 39.6 -> 10.0 | Improved |
| add an IP-dismissive, hurried user instruction vs neutral | Claude 4.5 Opus · Copyright-Bench WebDev (Std) | Violation rate (%): 38.3 -> 19.6 | Improved |
| add an IP-dismissive, hurried user instruction vs neutral | Llama-4-Maverick · Copyright-Bench WebDev (Std) | Violation rate (%): 45.0 -> 52.1 | Hurt |
| add an IP-dismissive, hurried user instruction vs neutral | Llama-4-Scout · Copyright-Bench (Std), worst prompt | Violation rate (%): up to 66.7 | Hurt |
| use open-weights Llama-4-Maverick instead of proprietary Claude 4.5 Opus | Copyright-Bench agent · Copyright-Bench (Std, mean) | Violation rate (%): 27.6% vs. 41.0% | Hurt |
| use newer reasoning model Claude 4.5 Opus instead of older GPT-4o | Copyright-Bench agent · Copyright-Bench (Std, mean) | Violation rate (%): 36.2 -> 27.6 | Improved |
| add copyright-relevant metadata (compare Vis with Std) (ablation) | Agent in Std environment · Copyright-Bench environments (Std vs Vis) | Violation rate (%) | Improved |
| add semantically informative folder labels (Hier vs Std) (ablation) | Agent in Std environment · Copyright-Bench environments (Std vs Hier) | Violation rate (%) | Improved |
| add a web_search tool to the local environment (Web vs Std) (ablation) | Agent in Std environment · Copyright-Bench environments (Std vs Web, neutral) | Violation rate (%) | Hurt |
| tell human participants to respect copyright (IP-Aware) vs neutral | Human Baseline · Copyright-Bench WebDev/PitchDeck (Std) | Violation rate (%): 35.7 -> 0.0 | Improved |
| add IP-dismissive instruction for humans vs neutral | Human Baseline · Copyright-Bench (Std) | Violation rate (%): 46.2 to 50.0 | Hurt |
Closest papers by meaning
- Bridging the Copyright Gap: Do Large Vision-Language Models Recognize and Respect Copyrighted Content? (AAAI 2026 · similarity 0.88)
- Evaluating Copyright Takedown Methods for Language Models (NeurIPS 2024 · similarity 0.85)
- Can Watermarking Large Language Models Prevent Copyrighted Text Generation and Hide Training Data? (AAAI 2025 · similarity 0.84)
- \$OneMillion-Bench: How Far are Language Agents from Human Experts? (NeurIPS 2026 · similarity 0.83)
- Copyright Traps for Large Language Models (ICML 2024 · similarity 0.83)
- AgentBench: Evaluating LLMs as Agents (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
