Research map / Large language models / AI and society
Position: Open-Endedness is Essential for Artificial Superhuman Intelligence
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Position paper arguing open-endedness, formally defined via novelty and learnability relative to an observer, is essential for artificial superhuman intelligence and can be pursued by combining open-ended algorithms with foundation models, with safety implications discussed.
Problem
Current foundation models trained on static internet-scale data are not open-ended, and there is no formal definition of open-endedness to guide research toward self-improving, ever-novel AI capable of superhuman discovery.
Approach
Defines open-endedness formally as a system producing artifacts that are both novel (increasingly unpredictable to an observer's learned model) and learnable (conditioning on longer history makes artifacts more predictable), argues this is essential for ASI, and proposes foundation models plus open-ended algorithms (RL, self-improvement, task generation, evolution) as a path.
Key results
- Proposes a formal definition of open-endedness requiring artifacts to be both novel and learnable with respect to an observer's statistical model.
- Argues contemporary foundation models trained on fixed datasets are not open-ended with respect to any observer who can model their training data.
- Reviews AlphaGo, AdA, and POET as open-ended examples, noting AdA's novelty plateaus after about one month of training.
Surprising findings
- Contemporary foundation models, despite broad capabilities, are argued not to be open-ended with respect to any observer who can model their training dataset.
Limitations
- Position paper with no empirical validation of the definition or proposed paths.
- Authors note that directing open-ended systems toward useful objectives while preserving open-endedness is an open problem.
- Open-ended systems may produce safety risks such as goal misgeneralization and dual-use artifacts.
Experimental setup
Builds on: Soros and Stanley open-endedness conditions Sigaud et al. open-endedness definition unsupervised environment design POET FunSearch
Closest papers by meaning
- OMNI: Open-endedness via Models of human Notions of Interestingness (ICLR 2024 · similarity 0.86)
- OMNI-EPIC: Open-endedness via Models of human Notions of Interestingness with Environments Programmed in Code (ICLR 2025 · similarity 0.84)
- Position: Near to Mid-term Risks and Opportunities of Open-Source Generative AI (ICML 2024 · similarity 0.83)
- The Generative AI Paradox: “What It Can Create, It May Not Understand” (ICLR 2024 · similarity 0.83)
- Position: AI Safety Must Embrace an Antifragile Perspective (ICML 2025 · similarity 0.82)
- Position: Explainable AI Cannot Advance Without Better User Studies (ICML 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.