Research map / Probabilistic and scientific modeling / Dynamical systems
Sparse Diffusion Autoencoder for Test-time Adapting Prediction of Complex Systems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SparseDiff, a codebook-based sparse probe encoder with a graph neural ODE predictor and diffusion decoder, using test-time re-encoding to adapt the probe topology during long-term prediction of PDE systems.
Problem
Long-term autoregressive prediction of complex spatiotemporal systems, where latent encodings lose spatial structure and fail to track emerging dynamics patterns.
Approach
Replace dense latent vectors with a sparse probe graph derived from a learned codebook of dynamical patterns (one probe per hit codeword, ~1% of grid points), model probe dynamics with a edge-weighted graph neural diffusion ODE, decode with a guided diffusion model, and re-encode the topology at test time when a latent consistency score drops below a threshold.
Key results
- Average RMSE reduction of 49.99% over baselines, with the best RMSE on Lambda-Omega (2.912e-2 vs 5.138e-2 for FNO) and on Swift-Hohenberg (7.628e-2 vs 8.214e-2 for G-LED).
- Achieves the best RMSE on four of five settings in Table 1 while using probes on about 1% of grid points (around 150 probes on Swift-Hohenberg).
- Generalizes to out-of-distribution Reynolds numbers (trained on Re<500) with consistently higher SSIM than baseline on cylinder flow.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SparseDiff instead of the strongest reported baseline | Long-term prediction · Lambda-Omega | RMSE × 10^-2: 5.138\pm 0.557 -> 2.912\pm 0.187 | Helped |
| Use SparseDiff instead of the strongest reported baseline | Long-term prediction · Navier-Stokes | RMSE × 10^-2: 12.334\pm 0.485 -> 11.130\pm 3.241 | Helped |
| Use SparseDiff instead of the strongest reported baseline | Long-term prediction · Swift–Hohenberg | RMSE × 10^-2: 8.214\pm 0.937 -> 7.628\pm 3.102 | Helped |
| Use SparseDiff instead of the strongest reported baseline | Long-term prediction · Cylinder-Flow | RMSE × 10^-2: 10.021\pm 0.873 -> 8.544\pm 0.239 | Helped |
| Use SparseDiff instead of the strongest reported baseline | Long-term prediction · Real-world | RMSE × 10^-2: 10.304\pm 2.009 -> 7.957\pm 1.207 | Helped |
| Use SparseDiff instead of the strongest reported baseline | Long-term prediction · Cylinder-Flow | SSIM × 10^-1: 7.583\pm 0.525 -> 7.392\pm 0.171 | Hurt |
| Replace region-aware edge weights with an unweighted graph (ablation) | SparseDiff · Navier-Stokes | Prediction performance | Hurt |
| Use spatially uniform probe selection instead of codebook-based selection (ablation) | SparseDiff · Swift–Hohenberg | Prediction performance | Hurt |
| Use random probe selection instead of codebook-based selection (ablation) | SparseDiff · Swift–Hohenberg | Prediction performance | Hurt |
| Increase codebook size (ablation) | SparseDiff · Navier-Stokes | SSIM | No clear effect |
| Increase Gaussian observation noise (ablation) | SparseDiff · Real-world climate dataset | Prediction performance | Hurt |
| Use 10% observations instead of full-grid input | SparseDiff · Navier-Stokes | RMSE × 10^-2: 11.39 -> 12.27 | Hurt |
Surprising findings
- SparseDiff with 10% random observations (12.27 RMSE) is nearly as accurate as with full-grid input on Navier-Stokes, though still worse than full-grid SparseDiff (11.39).
Limitations
- Edge-weight calculation rules are heuristic and strongly influence long-term prediction quality.
- Probe initialization by average aggregation may lose high-frequency information at a few grid points.
- Requires regular-grid input because the UNet diffusion decoder needs regular grids; irregular domains need interpolation or zero-filling.
Experimental setup
Datasets: Lambda-Omega simulations Navier-Stokes simulations Swift-Hohenberg simulations Cylinder flow LBM simulations SEVIR ir069
Benchmarks: Lambda-Omega Navier-Stokes Swift-Hohenberg Cylinder-Flow SEVIR
Metrics: RMSE SSIM
Baselines: FNO ConvLSTM UNet G-LED GKN GRAND + kNN GRAND + spectral
Builds on: VQ-VAE Graph Neural Diffusion (GRAND) G-LED Bayesian diffusion posterior sampling
Closest papers by meaning
- eXponential FAmily Dynamical Systems (XFADS): Large-scale nonlinear Gaussian state-space modeling (NeurIPS 2024 · similarity 0.87)
- Thermalizer: Stable autoregressive neural emulation of spatiotemporal chaos (ICML 2025 · similarity 0.86)
- CSG-ODE: ControlSynth Graph ODE For Modeling Complex Evolution of Dynamic Graphs (ICML 2025 · similarity 0.86)
- Generating Full-field Evolution of Physical Dynamics from Irregular Sparse Observations (NeurIPS 2025 · similarity 0.86)
- P$^2$C$^2$Net: PDE-Preserved Coarse Correction Network for efficient prediction of spatiotemporal dynamics (NeurIPS 2024 · similarity 0.86)
- Diffusion Models are Secretly Exchangeable: Parallelizing DDPMs via Auto Speculation (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.