Research map / Video / Video generation
stable-worldmodel: A Platform for Reproducible World Modeling Research and Evaluation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Presents stable-worldmodel (swm), an open-source platform unifying data storage (Lance), world model baselines, planning solvers, and a perturbable environment suite for reproducible world model research and OOD evaluation.
Problem
World model research is fragmented across custom codebases, with slow video data loading and no standardized generalization benchmarks, hindering reproducibility and fair comparison.
Approach
A single modular framework combining a Lance-based columnar data layer with format converters, reimplemented baselines (DINO-WM, LeWM, PLDM, TD-MPC2, GCBC, GCIQL/GCIVL) and planning solvers (CEM, MPPI, GD, etc.), plus controllable factors of variation and visual wrappers applied to many environments for zero-shot robustness evaluation.
Key results
- On Push-T in-distribution planning, LeWM reaches 94% success and DINO-WM 92%, while TD-MPC2 reaches only 12% in the offline setting.
- Planning success rates drop sharply under mild visual perturbations for most models, and success decays quadratically with the number of visual distractor squares.
- Lance achieves the highest data loading throughput in the Push-T benchmark, outperforming local HDF5, MP4, and LeRobot formats and even remote HDF5 streaming from S3.
Surprising findings
- Prediction error correlates poorly with planning success; success and failure MSE distributions largely overlap even under strong distribution shift.
- TD-MPC2 performs poorly in the offline setting, conjectured to be due to OOD actions fooling the predictor.
Limitations
- Evaluation is limited to simulation; sim-to-real transfer is not addressed.
- Case study focuses mainly on Push-T; broader per-environment results are in the appendix.
- Several baseline results are reproduced from original papers and TD-MPC2 underperforms in offline setting, which may reflect implementation or setting differences.
Experimental setup
Datasets: Push-T expert dataset (DINO-WM) Two-Room LeRobot datasets Atari (ALE) OGBench DeepMind Control Suite Craftax CartPole
Benchmarks: Push-T OGB-Cube DeepMind Control Suite Atari Craftax CartPole
Metrics: success rate data loading throughput (samples/sec) prediction MSE disk storage usage
Baselines: TD-MPC2 GCBC LeWM PLDM DINO-WM
Builds on: DINO-WM PLDM LeWorldModel TD-MPC2 Gymnasium LeRobot
Closest papers by meaning
- LoopNav: Benchmarking Spatial Consistency in World Models (NeurIPS 2026 · similarity 0.88)
- WorldModelBench: Judging Video Generation Models As World Models (NeurIPS 2025 · similarity 0.88)
- Trimming the Long-Tail of Visual World Modeling Evaluation (NeurIPS 2026 · similarity 0.87)
- World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry (NeurIPS 2026 · similarity 0.86)
- Dyn-O: Building Structured World Models with Object-Centric Representations (NeurIPS 2025 · similarity 0.86)
- From Forecasting to Planning: Policy World Model for Collaborative State-Action Prediction (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.