Research map / Video / Video generation

LoopNav: Benchmarking Spatial Consistency in World Models

NeurIPS 2026 · Video › Video generation · approach: World-model video generation

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces LoopNav, a 250-hour Minecraft loop-navigation dataset and benchmark with an object-centric Scene Graph Consistency Score (SGCS) to evaluate long-horizon spatial consistency of world models, finding that four existing world models perform poorly.

Problem

World models often fail to keep spatial consistency over long rollouts, and existing datasets and pixel-level metrics (SSIM, LPIPS, FVD) neither enforce nor reliably measure revisitation-based spatial consistency.

Approach

Loop-style (ABA and ABCA) Minecraft navigation trajectories with curriculum sequence lengths, and an explore-then-generate protocol scored by SGCS, which matches category-wise SAM3 object masks by centroid distance with Hungarian matching and area-weighted Dice aggregation, so it is tolerant to small pixel misalignment.

Key results

• Metric validation: SGCS stays stable under geometric perturbations (color, translation, rotation, scaling) while dropping sharply under object deletion and swapping, where SSIM and LPIPS may still score high.
• Across four baselines, MineWorld achieves the best SGCS, followed by NWM and DIAMOND, while Oasis performs worst; SGCS decreases as navigation range grows (e.g., MineWorld-5 ABA 0.54 vs MineWorld-50 ABA 0.24).
• Baselines are unsatisfactory: MineWorld and NWM stay roughly consistent for about 20 frames before failing, while Oasis and DIAMOND collapse into artifacts.

Surprising findings

• SSIM, LPIPS and FVD fail to reflect the relative ranking of baselines or the increasing difficulty with navigation range.
• MineWorld and DIAMOND score higher on ABCA length-50 than length-30, attributed to sparse visually distinctive objects letting models generate generic grass and sky scenes that inflate SGCS.

Limitations

• Data is limited to Minecraft; real-world generalization and out-of-distribution transfer are untested.
• Environments are static (mobs disabled), so dynamic-object spatial consistency is not covered.
• SGCS uses simple object-level matching and can be inflated by visually sparse scenes.

Experimental setup

Datasets: LoopNav

Benchmarks: LoopNav

Metrics: SGCS SSIM LPIPS FVD

Baselines: Oasis MineWorld DIAMOND Navigation World Model

Builds on: Navigation World Model DIAMOND Oasis MineWorld SAM 3 Mineflayer

Closest papers by meaning

• stable-worldmodel: A Platform for Reproducible World Modeling Research and Evaluation (NeurIPS 2026 · similarity 0.88)
• WorldMem: Long-term Consistent World Simulation with Memory (NeurIPS 2025 · similarity 0.87)
• WorldCoder-Bench: Benchmarking Physically Grounded 3D World Synthesis (NeurIPS 2026 · similarity 0.86)
• SpatialBench: Is Your Spatial Foundation Model an All-Round Player (NeurIPS 2026 · similarity 0.86)
• TerraVis: Towards Evaluation of World-Grounded Visual Consistency in Text-to-Image Generation via MLLM Workflows (NeurIPS 2026 · similarity 0.86)
• Video World Models with Long-term Spatial Memory (NeurIPS 2025 · similarity 0.86)