Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Intrinsic Goals for Autonomous Agents: Model-Based Exploration in Virtual Zebrafish Predicts Ethological Behavior and Whole-Brain Dynamics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces 3M-Progress, a model-based intrinsic drive that tracks KL divergence between a fixed ethological prior world model and an online world model, and shows embodied virtual zebrafish agents trained purely on this intrinsic goal reproduce futility-induced passivity behavior and whole-brain neural-glial dynamics.
Problem
Existing model-based intrinsic motivation methods yield inconsistent, non-stationary exploration that does not converge to a stable exploratory policy, and neuroscience lacks goal-driven models of the neural basis of autonomous, task-independent behavior such as futility-induced passivity in larval
Approach
Compare a frozen ethological world-model memory pretrained in a head-free niche with an online world model learned in the new environment; use the absolute deviation of an exponentially filtered model-memory mismatch (KL divergence) as intrinsic reward, yielding a stationary, niche-partitioning exploration objective.
Key results
- 3M-Progress agents achieve the highest model-behavior alignment among intrinsic drives, reproducing stable active-passive cycling that ICM, RND, Disagreement and gamma-Progress fail to capture.
- 3M-Progress agents capture nearly all of the explainable variance in whole-brain neural and astrocytic activity under One-to-One mapping, outperforming baselines.
- 3M-Progress is the only model nearly fully aligned with both behavioral and neural-glial alignment, and its PCA latent dynamics mirror astrocyte accumulation and neuronal transient signals.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use 3M-Progress instead of ICM, RND, Disagreement, or γ-Progress | Embodied zebrafish agent · Zebrafish futility-induced passivity | model-behavior alignment | Helped |
| Use 3M-Progress instead of existing intrinsic motivation algorithms or baseline controls | Embodied zebrafish agent · Zebrafish whole-brain neural-glial dynamics | neural-glial alignment | Helped |
| Use ICM, RND, Disagreement, or γ-Progress instead of 3M-Progress | Embodied zebrafish agent · Zebrafish futility-induced passivity | model-behavior alignment | Hurt |
| Use ICM, RND, Disagreement, or γ-Progress instead of 3M-Progress | Embodied zebrafish agent · Zebrafish whole-brain neural-glial dynamics | neural-glial alignment | Hurt |
| Train with 3M-Progress rather than other intrinsic drives | Embodied zebrafish agent · Reward-free exploration | stability of behavioral transitions | Helped |
Surprising findings
- Baseline intrinsic drives (ICM, RND, Disagreement, gamma-Progress) converge to stable complete passivity or activity rather than cycling.
- Inter-animal alignment is nearly 100%, so a One-to-One mapping suffices for whole-brain alignment instead of full linear regression.
Limitations
- Analyses limited to constrained virtual environments with limited ecological realism.
- Body model lacks muscles and motor circuits; low-level controller unconstrained by biomechanics.
- Model abstracts away biochemical signaling and detailed astrocyte anatomy.
Experimental setup
Datasets: whole-brain larval zebrafish neural-glial calcium imaging (open-source DANDI dataset 000350) virtual zebrafish environment (MuJoCo, dm-control swimmer)
Benchmarks: zebrafish futility-induced passivity behavioral transitions whole-brain neural-glial calcium imaging alignment
Metrics: Pearson correlation noise-corrected One-to-One neural predictivity swim power correlation
Baselines: ICM RND Disagreement gamma-Progress Homeostatic agent Maximum entropy agent Random agent Gaussian-Process PID controller
Builds on: PPO Intrinsic Curiosity Module Random Network Distillation gamma-Progress Disagreement
Closest papers by meaning
- Point process latent variable models of larval zebrafish behavior (NeurIPS 2018 · similarity 0.87)
- Deep RL Needs Deep Behavior Analysis: Exploring Implicit Planning by Model-Free Agents in Open-Ended Environments (NeurIPS 2025 · similarity 0.87)
- SEEA-R1: Tree-Structured Reinforcement Fine-Tuning for Self-Evolving Embodied Agents (NeurIPS 2025 · similarity 0.86)
- Inverse Reinforcement Learning with Switching Rewards and History Dependency for Characterizing Animal Behaviors (ICML 2025 · similarity 0.85)
- Behavioral Exploration: Learning to Explore via In-Context Adaptation (ICML 2025 · similarity 0.85)
- Discovering Mechanistic Models of Neural Activity: System Identification in an in Silico Zebrafish (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.