Research map / Reinforcement learning / Imitation learning

World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry

NeurIPS 2026 · Reinforcement learning › Imitation learning · approach: Imitation combined with RL · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes World Action Verifier (WAV), which verifies action-conditioned world-model predictions via a forward-inverse cycle (video-trained subgoal generator, sparse inverse dynamics model, forward rollout) to select informative interactions for self-improving world models.

Problem

World models must stay reliable over broad, suboptimal action distributions that are underrepresented in action-labeled robot data; existing forward-error-based verification is unreliable in under-explored regimes where it is most needed.

Approach

Decompose action-conditioned forward prediction into state plausibility (from an action-free subgoal prior trained on video) and action reachability (from a sparse inverse dynamics model on action-relevant features), and use their cycle consistency as a verifier to pick informative transitions for exploration.

Key results

• Sparse IDM outperforms world model at action inference, with the gap largest in low-data regimes and as object count grows (MiniGrid).
• WAV reports about 2x sample efficiency in exploration per the abstract, and about 22% downstream policy improvement, including on OOD Can variants.

Surprising findings

• Forward-model error is most pronounced in low-data and high-object-count regimes, where inverse models remain stable.
• Goal-oriented ordering (subgoal first, forward rollout last) is argued to be more robust than action-oriented ordering (forward rollout first).

Limitations

• Current instantiation requires three inference passes per step, making it more expensive than prior exploration methods.
• Still relies on environment feedback to correct action-conditioned errors; not fully self-improving without real interaction.
• Theory is linear-Gaussian and informal, with stylized assumptions.

Experimental setup

Datasets: MiniGrid custom tasks (Key Delivery, Ball Delivery, Object Matching) RoboMimic (Lift, Can, Square) ManiSkill (PullCube, PokeCube, LiftPeg)

Benchmarks: MiniGrid custom tasks RoboMimic ManiSkill OOD Can variants

Metrics: prediction error (MSE) dynamics accuracy Action Following Score Spearman rank correlation task reward after imagination-based refinement

Baselines: Random (RND) Uncertainty Progress Vanilla IDM Oracle

Builds on: Dreamer-v3 CLAM SAILOR diffusion policy

Closest papers by meaning

• Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation (ICLR 2025 · similarity 0.89)
• From Forecasting to Planning: Policy World Model for Collaborative State-Action Prediction (NeurIPS 2025 · similarity 0.89)
• Action Images: End-to-End Policy Learning via Multiview Video Generation (NeurIPS 2026 · similarity 0.89)
• AdaWorld: Learning Adaptable World Models with Latent Actions (ICML 2025 · similarity 0.89)
• EA-WM: Event-Aware Generative World Model with Structured Kinematic-to-Visual Action Fields (NeurIPS 2026 · similarity 0.88)
• RoboAlign-R1: Distilled Multimodal Reward Alignment for Robot Video World Models (NeurIPS 2026 · similarity 0.88)