Research map / Video / Video segmentation and tracking

MESSENGER: Memory-Enhanced Sequential Scene Flow Estimation via Autoregressive Next-Frame Forecasting

NeurIPS 2026 · Video › Video segmentation and tracking · approach: Stochastic video prediction

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MESSENGER, a memory-buffer-based autoregressive next-frame forecasting pipeline for sequential LiDAR scene flow estimation and future prediction, with uncertainty-aware reweighting of stored history flows and states.

Problem

Multi-frame scene flow estimation suffers from growing compute with more input frames and degraded long-horizon future flow prediction due to compounding errors and weak motion propagation.

Approach

A coarsest-stage memory buffer storing multiple history flows and GRU hidden states is retrieved via cross-attention to forecast the current initial flow, and a per-point uncertainty derived from the residual between the forecast and refined flow reweights the stored flows to suppress error accumulation.

Key results

• On nuScenes future prediction, EPE3D at H+4 is 0.025 vs GenFlow3D 0.088 (about 71.6% reduction).
• On Argoverse 2 future prediction, Acc3DS at H+4 is 0.555 vs GenFlow3D 0.130, with EPE3D 0.062 vs 0.192 (about 67.7% reduction).
• Scene flow estimation on nuScenes with 3 input frames: SF_1 EPE3D 0.0154 vs GenFlow3D 0.0161.

Surprising findings

• Removing the memory buffer more than triples EPE3D at the 4th future step (0.088 to 0.025 gap in the reported ablation direction), showing reliance on explicit memory.
• Gains over GenFlow3D in 3-frame flow estimation are small, while the advantage grows with more input frames and longer horizons.

Limitations

• Forecasting capability is limited to a short future frame length given attention-based autoregression.
• Gains over GenFlow3D are small in the 3-frame estimation setting.

Experimental setup

Datasets: nuScenes Argoverse 2

Benchmarks: nuScenes Argoverse 2

Metrics: EPE3D Acc3DS Out3D

Baselines: PointPWC-Net FLOT PV-RAFT Bi-PointFlowNet PT-FlowNet DifFlow3D GenFlow3D

Builds on: GenFlow3D DifFlow3D FlowNet3D RAFT-style GRU refinement

Closest papers by meaning

• DeltaFlow: An Efficient Multi-frame Scene Flow Estimation Method (NeurIPS 2025 · similarity 0.89)
• StreamFlow: Streamlined Multi-Frame Optical Flow Estimation for Video Sequences (NeurIPS 2024 · similarity 0.88)
• MemFlow: Optical Flow Estimation and Prediction with Memory (CVPR 2024 · similarity 0.88)
• DifFlow3D: Toward Robust Uncertainty-Aware Scene Flow Estimation with Iterative Diffusion-Based Refinement (CVPR 2024 · similarity 0.87)
• DiffSF: Diffusion Models for Scene Flow Estimation (NeurIPS 2024 · similarity 0.86)
• Zero-Shot Monocular Scene Flow Estimation in the Wild (CVPR 2025 · similarity 0.86)