Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Dynamic Knowledge Injection for AIXI Agents
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces DynamicHedgeAIXI, an AIXI approximation that maintains an exact Bayesian mixture over dynamically changing sets of predicate-based environment models (Φ-BCTW specialists) using a DynamicHedge weighting variant, evaluated on biased RPS, Taxi, and epidemic control.
Problem
Approximations of AIXI use a fixed, predefined model class, leaving irreducible model bias that cannot be fixed by more data; the paper targets incorporating new candidate models supplied online by a human operator.
Approach
Combine specialist aggregation (Hedge with abstention) and GrowingHedge-style weight initialization for contiguous specialists, so newly arriving Φ-BCTW environment models enter with path-dependent prior weights and old ones are removed, yielding an adaptive exact Bayesian mixture with value-convergence bounds.
Key results
- DynamicHedgeAIXI and HedgeAIXI vastly outperform U-Tree and PARSS on Taxi and Epidemic Control domains.
- DynamicHedgeAIXI's performance picks up faster than HedgeAIXI on Taxi, an effect especially pronounced there.
- On biased RPS, DynamicHedgeAIXI and HedgeAIXI are more unstable than U-Tree initially but DynamicHedgeAIXI eventually converges to optimal performance.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DynamicHedgeAIXI instead of U-Tree | AIXI agents · Taxi | performance | Helped |
| Use DynamicHedgeAIXI instead of PARSS | AIXI agents · Taxi | performance | Helped |
| Use DynamicHedgeAIXI instead of U-Tree | AIXI agents · Epidemic Control | performance | Helped |
| Use DynamicHedgeAIXI instead of PARSS | AIXI agents · Epidemic Control | performance | Helped |
| Reinitialize a model's weight when a newly entering model replaces an older model (ablation) | HedgeAIXI · Taxi | adaptation speed | Helped |
| Reinitialize a model's weight when a newly entering model replaces an older model (ablation) | HedgeAIXI · Epidemic Control | adaptation speed | Helped |
| Use DynamicHedgeAIXI instead of U-Tree | AIXI agents · Biased RPS | stability | Hurt |
| Use DynamicHedgeAIXI instead of U-Tree | AIXI agents · Biased RPS | performance | Mixed |
Surprising findings
- On biased RPS, U-Tree learned faster than DynamicHedgeAIXI because the environment needs only two predicates, and DynamicHedgeAIXI's weight redistribution caused initial instability.
Limitations
- Transfers a large burden of predicate design to the human expert.
- Initial instability on biased RPS due to frequent weight redistribution to newly entering models.
- Value bounds depend on number of switches and total models seen; experiments use only five seeds.
Experimental setup
Datasets: contact network dataset with 1133 nodes and 5451 edges biased rock-paper-scissors Taxi (2x5 grid variant)
Benchmarks: biased Rock-Paper-Scissors Taxi Epidemic Control over Contact Networks
Metrics: learning curve (reward) mean and standard deviation over five seeds model weights over time
Baselines: U-Tree PARSS-DT HedgeAIXI
Builds on: AIXI Φ-AIXI-CTW GrowingHedge Hedge Φ-BCTW
Closest papers by meaning
- Bayesian Exploration Networks (ICML 2024 · similarity 0.85)
- Sequential Probabilistic Uncertainty Estimation for Parallel Multi-Agent Reasoning Systems (NeurIPS 2026 · similarity 0.85)
- NestRL: A Nested Training Regime for Mutual Adaptation in Human–AI Teaming (NeurIPS 2026 · similarity 0.85)
- Bayesian Ego-graph inference for Networked Multi-Agent Reinforcement Learning (NeurIPS 2025 · similarity 0.85)
- KABB: Knowledge-Aware Bayesian Bandits for Dynamic Expert Coordination in Multi-Agent Systems (ICML 2025 · similarity 0.85)
- Modeling Boundedly Rational Agents with Latent Inference Budgets (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.