Research map / Large language models / AI and society

Position: In-House Evaluation Is Not Enough. Towards Robust Third-Party Evaluation and Flaw Disclosure for General-Purpose AI

ICML 2025 · Large language models › AI and society · approach: Human-centered trust and agency

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

A position paper arguing that general-purpose AI needs third-party evaluation infrastructure, including standardized AI flaw reports, good-faith rules of engagement, legal safe harbors with flaw disclosure programs, and a disclosure coordination center.

Problem

Flaw reporting and third-party evaluation infrastructure for general-purpose AI systems is underdeveloped compared with software security, leaving flaws reported ad hoc, with legal chilling effects on independent researchers.

Approach

Adapts coordinated vulnerability disclosure and bug bounty practices from software security to GPAI, proposing a standardized AI flaw report schema, good-faith rules of engagement, provider safe harbors with ToS exemptions, and a centralized disclosure coordination center that routes reports by tags to stakeholders.

Key results

• Proposes a standardized AI Flaw Report schema (Figure 2, Table A1) adapted from AI Incident Database, MITRE AI incident form, OECD incident form, and AI Vulnerability Database fields.
• Proposes a legal safe harbor template and ToS exemption for good-faith GPAI evaluation, plus a moderation-exempt research access option.
• Recommends an AI Disclosure Coordination Center that routes flaw reports to subscribed stakeholders via tags, modeled on CISA incident reporting.

Surprising findings

• Many GPAI safety flaws are transferable across providers' systems, so disclosure to one provider alone is insufficient.

Limitations

• No empirical evaluation of the proposed report schema, safe harbor, or coordination center.
• Acknowledges unresolved disagreements over flaw responsibility and the governance of a disclosure coordination center.

Experimental setup

Builds on: coordinated vulnerability disclosure bug bounty programs disclose.io safe harbor template AI Incident Database

Closest papers by meaning

• To Err Is AI: A Case Study Informing LLM Flaw Reporting Practices (AAAI 2025 · similarity 0.86)
• MOSAIC-Bench: Measuring Compositional Vulnerability Induction in Coding Agents (NeurIPS 2026 · similarity 0.85)
• AI Control: Improving Safety Despite Intentional Subversion (ICML 2024 · similarity 0.82)
• Position: A Safe Harbor for AI Evaluation and Red Teaming (ICML 2024 · similarity 0.82)
• Evaluation and Incident Prevention in an Enterprise AI Assistant (AAAI 2025 · similarity 0.82)
• CTRL-ALT-DECEIT Sabotage Evaluations for Automated AI R&D (NeurIPS 2025 · similarity 0.82)