Research map / Probabilistic and scientific modeling / Molecules and proteins
DiffSF: Diffusion Models for Scene Flow Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Formulates scene flow estimation as a conditional denoising diffusion process on a GMSF-based transformer, yielding state-of-the-art accuracy and multi-hypothesis uncertainty estimates.
Problem
Scene flow estimation from source and target point clouds lacks robustness under occlusion and noise and provides no reliability/uncertainty measure for safety-critical use.
Approach
Treat scene flow as a conditional diffusion process: the ground-truth flow field is noised and a transformer denoiser (GMSF-based, with an added initial-estimate warping stage) recovers it conditioned on source and target point clouds; sampling multiple times from different noise seeds yields hypotheses whose standard deviation serves as per-point uncertainty.
Key results
- On FlyingThings3D (non-occluded), DiffSF reaches EPE3D 0.0062 and ACC_S 99.54 vs GMSF 0.0090 and 99.18.
- On Waymo-Open, DiffSF achieves EPE3D 0.080 and ACC_S 76.0 vs GMSF 0.083 and 74.7.
- Standard deviation over 20 sampled hypotheses correlates with EPE; precision-recall break-even at about 55% for outlier detection on F3D_o.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace GMSF with DiffSF | Scene flow estimation · F3D_s | EPE_3D: 0.0090 -> 0.0062 | Helped |
| Replace GMSF with DiffSF | Scene flow estimation · F3D_o | ACC_S: 95.0 -> 96.7 | Helped |
| Replace GMSF with DiffSF | Scene flow estimation · KITTI_o | ACC_S: 91.6 -> 94.5 | Hurt |
| Replace GMSF with DiffSF | Scene flow estimation · Waymo-Open | EPE_3D: 0.083 -> 0.080 | Helped |
| Add the improved architecture to retrained GMSF (ablation) | GMSF on F3D_o-all · F3D_o-all | EPE_3D: 0.039 -> 0.037 | Helped |
| Add the diffusion process without the improved architecture (ablation) | GMSF on F3D_o-all · F3D_o-all | EPE_3D: 0.039 -> 0.061 | Hurt |
| Add the diffusion process to the improved architecture (ablation) | DiffSF on F3D_o-all · F3D_o-all | ACC_S: +10.3% | Helped |
| Use 14 rather than 8 global-cross transformer layers (ablation) | DiffSF · F3D_o-all | EPE_3D: 0.0439 -> 0.0361 | Helped |
| Use 16 rather than 14 global-cross transformer layers (ablation) | DiffSF · F3D_o-all | EPE_3D: 0.0361 -> 0.0383 | Hurt |
| Increase feature channels from 32 to 128 (ablation) | DiffSF · F3D_o-all | EPE_3D: 0.0612 -> 0.0361 | Helped |
| Use 2@20 rather than 1@5 diffusion steps (ablation) | DiffSF · F3D_o-all | EPE_3D: 3.608 -> 3.576 | Helped |
| Use 1@100 rather than 2@20 diffusion steps (ablation) | DiffSF · F3D_o-all | EPE_3D: 3.576 -> 3.678 | Hurt |
Surprising findings
- Diffusion alone without the architectural change degrades performance over the GMSF baseline (EPE 0.061 vs 0.039 on F3D_o-all), and the gain appears only when combined with the improved architecture.
- Results are stable across many training and sampling step counts, with the best at 2 training steps and 20 sampling steps, suggesting fewer steps are needed than in image generation.
Limitations
- Training requires annotated scene flow ground truth, which is hard to obtain for real-world data.
- Transformer-based global matching limits the maximum number of points that can be processed.
Experimental setup
Datasets: FlyingThings3D KITTI Scene Flow Waymo-Open
Benchmarks: FlyingThings3D KITTI Scene Flow Waymo-Open
Metrics: EPE3D ACC_S ACC_R Outliers
Baselines: FlowNet3D HPLFlowNet PointPWC-Net FLOT Bi-PointFlow 3DFlow MSBRN DifFlow3D GMSF FESTA FH-Net
Builds on: DDPM GMSF FLOT DGCNN DDIM
Closest papers by meaning
- DifFlow3D: Toward Robust Uncertainty-Aware Scene Flow Estimation with Iterative Diffusion-Based Refinement (CVPR 2024 · similarity 0.91)
- FlowDiffuser: Advancing Optical Flow Estimation with Diffusion Models (CVPR 2024 · similarity 0.89)
- SceneDiffuser: Efficient and Controllable Driving Simulation Initialization and Rollout (NeurIPS 2024 · similarity 0.88)
- Floxels: Fast Unsupervised Voxel Based Scene Flow Estimation (CVPR 2025 · similarity 0.87)
- DiffBEV: Conditional Diffusion Model for Bird’s Eye View Perception (AAAI 2024 · similarity 0.87)
- Diffusion Models Without Attention (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.