Research map / Large language models / AI and society
Position: A Safe Harbor for AI Evaluation and Red Teaming
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
The paper argues that major generative AI developers should voluntarily commit to legal and technical safe harbors protecting good-faith public-interest AI evaluation and red teaming from litigation and account suspension.
Problem
Terms of service and account enforcement by generative AI companies deter good-faith independent evaluation and red teaming, creating legal fear and chilling effects on safety research.
Approach
Proposes two voluntary commitments: a legal safe harbor modeled on vulnerability disclosure practice (scoped to good-faith research, not at companies' sole discretion) and a technical safe harbor that delegates access authorization to trusted third parties and provides transparent appeals against account suspension.
Key results
- Table 3 finds only partial or no satisfaction of enforcement, justification, and appeal criteria across major AI systems, with only OpenAI, Inflection, and Midjourney showing evidence of an appeals process.
- The authors report that account suspensions of public-interest researchers have occurred for OpenAI, Anthropic, Inflection, and Midjourney, with Midjourney the most prolific.
- The paper reports that the bulk of independent evaluation concentrates on Meta Llama-2 and OpenAI ChatGPT models.
Surprising findings
- Only 25% of policy enforcement and evaluation criteria were satisfied on average across companies, per the Foundation Model Transparency Index, with no company providing reproducible evaluations of risk mitigations.
- OpenAI expanded its safe harbor to include model vulnerability and academic model safety research in response to an early draft of the proposal.
Limitations
- Proposals are framed within US legal perspectives and may not transfer to other jurisdictions.
- Potential for malicious actors to exploit publicly shared vulnerabilities or abuse safe harbor protections.
- Based on informal discussions and experiences of the authors, a sample skewed toward researchers who chose to conduct safety research.
Experimental setup
Baselines: OpenAI researcher access program Google bug bounty Anthropic safe harbor Meta Llama 2 license
Builds on: Abdo et al. 2022 social media safe harbor algorithmic bug bounties structured access DOJ good-faith security research policy
Closest papers by meaning
- Position: Near to Mid-term Risks and Opportunities of Open-Source Generative AI (ICML 2024 · similarity 0.86)
- Position: Require Frontier AI Labs To Release Small "Analog" Models (NeurIPS 2025 · similarity 0.85)
- SafetyAnalyst: Interpretable, Transparent, and Steerable Safety Moderation for AI Behavior (ICML 2025 · similarity 0.84)
- AI Control: Improving Safety Despite Intentional Subversion (ICML 2024 · similarity 0.84)
- Machine Unlearning Doesn't Do What You Think: Lessons for Generative AI Policy and Research (NeurIPS 2025 · similarity 0.84)
- ART: Automatic Red-teaming for Text-to-Image Models to Protect Benign Users (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.