Research map / Large language models / AI and society

Position: AI Safety Must Embrace an Antifragile Perspective

ICML 2025 · Large language models › AI and society · approach: Human-centered trust and agency · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

A position paper arguing AI safety should adopt an antifragile, time-evolving view of robustness, formalized via a two-timescale dynamic-regret framework over a perceived-vs-true MDP, with ethical and practical guidelines.

Problem

Static benchmarks and one-shot robustness tests fail to capture evolving environments, rare black swan events, and maladaptation (reward hacking, over-alignment) in deployed AI systems.

Approach

Reframes robustness as a dynamic property where exposure to stressors expands adaptive capacity; defines fragility and antifragility via whether dynamic regret over community model updates increases or decreases over time, and argues black swan gaps between perceived and true MDPs are inevitable.

Key results

• Theorems show that in a multi-state (|S|>=3), multi-step (T>=2) MDP with distorted perception, a non-zero robustness gap is unavoidable, so black swan events are inevitable.
• Corollary gives a lower bound on the robustness gap when c† > δ† + ε*, implying the gap cannot vanish under a static perceived model.
• Defines anti-fragile systems as those whose dynamic regret decreases over time and fragile systems as those whose regret increases.

Surprising findings

• Argues that no amount of fine-tuning on already-known data removes the robustness gap if black swan states remain discounted as impossible.

Limitations

• Conceptual and high-level; no empirical validation of the antifragility framework or the proposed guidelines.
• Practical measurement of antifragility (dynamic regret in real systems) is acknowledged as tricky and left largely open.
• Relies on the MDP abstraction, which the authors note is a conceptual tool rather than a literal implementation requirement.

Experimental setup

Builds on: robust MDPs online learning with dynamic regret prospect theory distortion functions adversarial training red teaming

Closest papers by meaning

• Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress? (NeurIPS 2024 · similarity 0.88)
• Towards a Science of AI Agent Reliability (ICML 2026 · similarity 0.86)
• Position: Require Frontier AI Labs To Release Small "Analog" Models (NeurIPS 2025 · similarity 0.86)
• GT-HarmBench: Benchmarking AI Safety Risks Through the Lens of Game Theory (NeurIPS 2026 · similarity 0.85)
• From Gambits to Assurances: Game-Theoretic Integration of Safety and Learning for Interactive Robotics (AAAI 2025 · similarity 0.85)
• LiSA: Lifelong Safety Adaptation via Conservative Policy Induction (NeurIPS 2026 · similarity 0.85)