Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Self-Improving World Modelling with Latent Actions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SWIRL, which learns world models from state-only sequences by treating actions as latent variables and alternating GRPO-based updates of a forward world model and an inverse dynamics model, each rewarded by the other's log-probability.
Problem
Learning internal world models for LLMs and VLMs normally requires costly action-labelled trajectories; inverse dynamics is also ambiguous when labels are sparse.
Approach
Alternates two GRPO phases: the forward model is rewarded by the frozen inverse model's log-likelihood of the latent action (framed as variational CMI maximisation), and the inverse model is rewarded by the frozen forward model's log-likelihood of the observed next state (framed as ELBO coordinate ascent).
Key results
- On Aurora-Bench, SWIRL (Iterative) raises the average GPT-4o score from 4.36 (Liquid-SFT) to 5.06.
- On ByteMorph, SWIRL (Iterative) improves the average from 43.38 to 53.77.
- On StableToolBench, Qwen-2.5-3B-SWIRL reaches 14.61 average BLEU versus 12.85 for SFT.
Surprising findings
- Shared-weight FWM/IDM training is less stable, with a performance drop at iteration 3.
- Repeated self-improvement does not help on WorldPredictionBench; performance peaks at the first iteration.
Limitations
- Gains on camera zoom/motion are limited because unlabelled in-the-wild videos are mostly static.
- Shared-parameter training introduces instability and gradient interference between visual generation and linguistic action inference.
- Liquid-SFT remains far behind proprietary and specialised editing models on general image editing.
Experimental setup
Datasets: Pico-banana-400K Aurora UCF-101 Movement-in-Times Kinetics700 VidGen-1M ScienceWorld Mind2Web StableToolBench
Benchmarks: Aurora-Bench ByteMorph WorldPredictionBench ScienceWorld Mind2Web StableToolBench GEdit-Bench
Metrics: GPT-4o judge score DiscEdit CLIP score BERTScore ROUGE-L BLEU
Baselines: Liquid-SFT Test-time Verification Bootstrap GoT SmartEdit InstructPix2Pix BAGEL OmniGen2 UniWorld-V1 Qwen-2.5-32B-Instruct
Builds on: GRPO Liquid Qwen-2.5-3B-Instruct
Closest papers by meaning
- AdaWorld: Learning Adaptable World Models with Latent Actions (ICML 2025 · similarity 0.88)
- Beyond Prediction: Steering VLM Agents with Retrospective World Modeling (NeurIPS 2026 · similarity 0.87)
- VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agents (NeurIPS 2025 · similarity 0.86)
- World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry (NeurIPS 2026 · similarity 0.86)
- Open-World Reinforcement Learning over Long Short-Term Imagination (ICLR 2025 · similarity 0.86)
- Zero-shot World Models via Search in Memory (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.