Research map / Large language models / AI and society
Asymptotically Unambitious Artificial General Intelligence
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use episodic reward maximization in a sealed room where opening the door ends the episode | BoMAI · Boxed interaction | outside-world ambition | Helped |
| Penalize intra-episode model space with a prior proportional to \beta^\ell | BoMAI · Boxed interaction, as \beta \to 0 | probability of eventual benignity | Helped |
| Choose exploitation actions to maximize current-episode reward under the MAP world-model | BoMAI · General task interaction | asymptotic reward relative to human mentor | Helped |
| Set exploration probability proportional to expected information gain | BoMAI · General task interaction | sum of squared exploration probabilities | Helped |
| Assume prior support for the true environment and human mentor policy, with finite-entropy priors | BoMAI · General task interaction | asymptotic reward relative to human mentor | Helped |
| Assume accurate non-benign models require more intra-episode space than the true model | BoMAI · Boxed interaction, as \beta \to 0 | probability of eventual benignity | Helped |
| Replace the episodic, boxed design with reward maximization that permits actions to influence future outside-world reward | Reward-maximizing RL agents · General task interaction | incentive to gain outside-world power | Hurt |
Closest papers by meaning
- Position: Stop treating `AGI' as the north-star goal of AI research (ICML 2025 · similarity 0.83)
- The Alignment Problem from a Deep Learning Perspective (ICLR 2024 · similarity 0.82)
- The AI Race: Why Current Neural Network-based Architectures are a Poor Basis for Artificial General Intelligence (AAAI 2025 · similarity 0.81)
- Position: Levels of AGI for Operationalizing Progress on the Path to AGI (ICML 2024 · similarity 0.80)
- Position: Open-Endedness is Essential for Artificial Superhuman Intelligence (ICML 2024 · similarity 0.79)
- Position: Stop Making Unscientific AGI Performance Claims (ICML 2024 · similarity 0.79)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.