Research map / Multi-agent systems and games / Human-AI and goal-conditioned
The World Is Bigger: A Computationally-Embedded Perspective on the Big World Hypothesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces a computationally-embedded view of agents as automata inside a universal-local environment, defines an algorithmic-complexity-based interactivity objective, and uses a meta-gradient RL method on a self-predicting synthetic benchmark showing deep linear policies sustain interactivity better than deep nonlinear ones.
Problem
Continual learning is motivated by the big world hypothesis, but explicit constraints on agents are ad hoc and hard to enforce or scale; the paper seeks an implicit constraint from embedding the agent in its environment and a measure of continual adaptability.
Approach
Model an embedded agent as an automaton simulated by a universal-local Markov environment (so it is always capacity-constrained and equivalent to a stateful policy in a POMDP), and define interactivity as the algorithmic-complexity difference of future behaviour with and without conditioning on past behaviour, approximated by TD prediction errors and maximized via model-based meta-gradients.
Key results
- Deep nonlinear policies fail to sustain interactivity, while deep linear policies sustain higher interactivity.
- Increasing depth of deep linear networks yields a large increase in sustained interactivity; increasing width gives only a marginal increase.
- Theory bounds maximum interactivity between roughly (1-alpha)C(A) and C(A)+O(1), where C is automaton capacity.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use a deep linear policy instead of a deep nonlinear policy | interactivity-seeking agent · Behavioral self-prediction | interactivity | Helped |
| increase deep linear policy width (ablation) | deep linear policy · Behavioral self-prediction | interactivity | Helped |
| increase deep linear policy depth (ablation) | deep linear policy · Behavioral self-prediction | interactivity | Helped |
Surprising findings
- Deep nonlinear policies learn actions with no discernible predictable structure, while deep linear policies learn non-stationary wave-like actions.
Limitations
- Experiments use shallow networks (depth up to 4) with meta-gradient horizon T=10, so nonlinear failure may stem from meta-gradient instability rather than nonlinearity itself.
- Meta-gradient computation scales poorly with horizon and depth, limiting scaling experiments.
- Evaluation of interactivity is online and does not fit standard fixed-model held-out evaluation.
Experimental setup
Benchmarks: self-prediction interactivity benchmark (synthetic)
Metrics: agent-relative interactivity
Baselines: deep nonlinear policy (ReLU)
Builds on: AIXI successor features cross-prop model-agnostic meta-learning
Closest papers by meaning
- Continual Reinforcement Learning by Planning with Online World Models (ICML 2025 · similarity 0.88)
- From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems (ICML 2024 · similarity 0.87)
- Scaling Large Language Model-based Multi-Agent Collaboration (ICLR 2025 · similarity 0.86)
- Continual Learning Bench: Evaluating Frontier AI Systems in Real-World Stateful Environments (NeurIPS 2026 · similarity 0.86)
- Revisiting Multi-Agent World Modeling from a Diffusion-Inspired Perspective (NeurIPS 2025 · similarity 0.86)
- Adaptive Collaboration with Humans: Metacognitive Policy Optimization for Multi-Agent LLMs with Continual Learning (ICLR 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.