Research map / Large language models / AI and society
Position: On the Societal Impact of Open Foundation Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Position paper that defines open foundation models, identifies five distinctive properties, weighs their societal benefits and risks, proposes a six-step marginal-risk assessment framework, and makes recommendations to developers, researchers, policymakers, and competition regulators.
Problem
Debate over whether to release foundation models openly lacks conceptual clarity and empirical grounding about their benefits and marginal misuse risks relative to closed models and pre-existing technologies.
Approach
Decomposes open-model societal impact into five distinctive properties (broader access, customizability, local inference, inability to rescind access, inability to monitor) and a six-step marginal-risk framework (threat identification, existing risk, existing defenses, marginal risk evidence, ease of defense, uncertainty) applied to misuse vectors.
Key results
- Llama 2 training with 3.3 million A100 GPU hours is estimated at about $6 million at $1.8/GPU-hour, illustrating barriers to entry for foundation model development.
- Of seven prior misuse analyses of open foundation models assessed under the framework, six are incomplete (Table 1).
- The authors judge the current marginal risk of open models for automated vulnerability detection to be low, while the marginal risk for digitally altered non-consensual intimate imagery is considerable at present.
Surprising findings
- Two recent studies found language model access does not significantly increase access to biosecurity-relevant information compared to internet access alone.
- Current evidence for marginal risk is weak for several misuse vectors, with prior studies mostly incomplete under the framework.
Limitations
- Framework does not provide a mechanism to trade off marginal benefits against marginal risks or consider opportunity costs of not releasing.
- Does not account for unknown unknown risks.
- Binary open vs. closed dichotomy is reductive and capabilities of future models may change the risk picture.
Experimental setup
Builds on: threat modeling in computer security structured access (Shevlane 2022) Foundation Model Transparency Index
Closest papers by meaning
- The Foundation Model Transparency Index (NeurIPS 2026 · similarity 0.87)
- Modeling the Economic Impacts of AI Openness Regulation (NeurIPS 2025 · similarity 0.82)
- Position: Near to Mid-term Risks and Opportunities of Open-Source Generative AI (ICML 2024 · similarity 0.81)
- On the Consideration of AI Openness: Can Good Intent Be Abused? (AAAI 2025 · similarity 0.80)
- Holistically Evaluating the Environmental Impact of Creating Language Models (ICLR 2025 · similarity 0.80)
- Position: AI Safety should prioritize the Future of Work (ICML 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.