Research map / 3D vision / Gaussian splatting
STORM: Spatio-TempOral Reconstruction Model For Large-Scale Outdoor Scenes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes STORM, a feed-forward transformer that predicts pixel-aligned 3D Gaussians and velocities from sparse posed multi-timestep images, aggregating them via self-supervised scene flow with motion tokens for dynamic outdoor scene reconstruction, trained with reconstruction losses only.
Problem
Dynamic outdoor scene reconstruction from sparse multi-timestep posed images typically needs per-scene optimization, dense observations, and pseudo-label motion supervision (masks, optical flow, trajectories), leading to slow fitting and poor generalization.
Approach
A ViT-based feed-forward model predicts per-frame 3D Gaussians with velocities; Gaussians from all frames are transported to a target timestep and aggregated into an amodal scene, and learnable motion tokens decode shared velocity bases combined by SAM-style attention weights, so dynamics are learned purely from reconstruction losses.
Key results
- On Waymo dynamic regions, STORM reaches 22.10 PSNR and 0.624 SSIM, versus 17.79 PSNR for the best per-scene baseline EmerNeRF in dynamic regions, with inference in 0.18 s.
- On Waymo scene flow, STORM achieves EPE3D 0.276 m and Acc5 81.12% using camera images only, versus 0.698 m and 42.17% for NSFP.
- STORM processes a 20-second video in 0.5 s on a single A100 with batch inference and bf16.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use STORM instead of the strongest per-scene method | Waymo dynamic scene reconstruction · Waymo Open Dataset, dynamic regions | PSNR: 17.79 -> 22.10 | Helped |
| Use STORM instead of GS-LRM | Waymo dynamic scene reconstruction · Waymo Open Dataset, dynamic regions | EPE3D: 0.698 -> 0.276 | Helped |
| Use STORM instead of NSFP++ | Waymo scene flow estimation · Waymo Open Dataset | Acc5: 53.10 -> 81.12 | Helped |
| Use STORM instead of GS-LRM | STORM · Waymo Open Dataset | inference time: 0.02s -> 0.18s | Hurt |
| Use STORM instead of GS-LRM | NuScenes reconstruction · NuScenes, full image | PSNR: 24.53 -> 24.90 | Helped |
| Use STORM instead of GS-LRM | Argoverse2 reconstruction · Argoverse2, full image | PSNR: 24.49 -> 24.80 | Helped |
| Add velocity regularization with coefficient 5 × 10^-3 (ablation) | STORM · Waymo Open Dataset | dynamic PSNR | Helped |
| Remove velocity regularization (ablation) | STORM · Waymo Open Dataset | training stability | Hurt |
| Set motion token count to 16 (ablation) | STORM · Waymo rendering and flow estimation | performance | Helped |
| Train and test with four input timesteps rather than other tested counts (ablation) | STORM · Waymo reconstruction | performance | Helped |
| Increase the number of input timesteps during training (ablation) | STORM · Waymo reconstruction | performance | Helped |
| Train with a single input timestep (ablation) | STORM · Waymo dynamic regions | PSNR: 2 to 4 PSNR improvement | Helped |
Surprising findings
- Training without the velocity regularization loss often causes gradient explosion and NaN loss.
- Trained with a single input timestep, STORM acts as a future predictor yet still beats per-scene methods using 4 input timesteps by 2 to 4 dynamic PSNR.
Limitations
- Token sequence length grows with resolution, cameras and timesteps (about 7k tokens), raising cost; 32k tokens only tried in preliminary experiments.
- Requires known camera intrinsics and extrinsics as input.
- Latent variant trades some pixel sharpness for human and occluded-region modeling, and default model performs weaker on human modeling.
Experimental setup
Datasets: Waymo Open Dataset NuScenes Argoverse2
Benchmarks: Waymo Open Dataset NuScenes Argoverse2
Metrics: PSNR SSIM Depth RMSE EPE3D Acc5 Acc10 angular error inference time
Baselines: EmerNeRF 3DGS PVG DeformableGS LGM GS-LRM NSFP NSFP++
Builds on: 3D Gaussian Splatting SAM mask decoder Vision Transformer LGM GS-LRM
Closest papers by meaning
- Flux4D: Flow-based Unsupervised 4D Reconstruction (NeurIPS 2025 · similarity 0.88)
- Dynamic 3D Gaussian Fields for Urban Areas (NeurIPS 2024 · similarity 0.87)
- DGS-LRM: Real-Time Deformable 3D Gaussian Reconstruction From Monocular Videos (NeurIPS 2025 · similarity 0.86)
- DeRainGS: Gaussian Splatting for Enhanced Scene Reconstruction in Rainy Environments (AAAI 2025 · similarity 0.85)
- FreeTimeGS: Free Gaussian Primitives at Anytime Anywhere for Dynamic Scene Reconstruction (CVPR 2025 · similarity 0.85)
- Adaptive Flow Matching for Resolving Small-Scale Physics (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.