Research map / Video / Video segmentation and tracking
Advection Augmented Convolutional Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ADRNet, a physically inspired network that augments CNNs with a semi-Lagrangian advection (push) operator plus reaction and diffusion components, evaluated on spatio-temporal scientific and video prediction datasets.
Problem
CNNs underperform on spatio-temporal prediction tasks that require long-range transport (advection) of information, and existing methods lack explainability.
Approach
Model spatio-temporal dynamics as a reaction-advection-diffusion process: a sparse but non-local semi-Lagrangian push operator, with a learned velocity field from history frames, handles advection, while diffusion and reaction are implemented with convolutional residual components solved via operator splitting.
Key results
- On PDEBench SWE, ADRNet reaches NRMSE 1.3e-4 versus 2.2e-3 for the best listed baseline (MPP-AVIT-L).
- On CloudCast, ADRNet reports SSIM 0.83 and PSNR 38.17 versus 0.66 and 8.06 for AE-ConvLSTM.
- On Moving MNIST, ADRNet attains MSE 16.1, second behind IAM4VP (15.3); on KTH it trails the state of the art in SSIM (0.808 vs 0.930 for MSPred) and PSNR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ADRNet instead of MPP-AVIT-L | PDEBench-SWE prediction · PDEBench-SWE | NRMSE: 2.2e-3 -> 1.3e-4 | Helped |
| Use ADRNet instead of AE-ConvLSTM | CloudCast prediction · CloudCast | SSIM: 0.66 -> 0.83 | Helped |
| Use ADRNet instead of AE-ConvLSTM | CloudCast prediction · CloudCast | PSNR: 8.06 -> 38.17 | Helped |
| Use ADRNet instead of FNO | SWE long-range prediction · SWE 10 → 5, 10 → 10, 10 → 20, 10 → 50 | MSE: 4.0e-07 -> 9.2e-08; 5.8e-07 -> 1.5e-07; 6.7e-07 -> 2.1e-07; 1.4e-06 -> 8.5e-07 | Helped |
| Use ADRNet instead of IAM4VP | Moving MNIST prediction · Moving MNIST | MSE: 15.3 -> 16.1 | Hurt |
| Use ADRNet instead of IAM4VP | Moving MNIST prediction · Moving MNIST | MAE: 49.2 -> 50.3 | Hurt |
| Use ADRNet instead of the strongest listed baseline | KITTI prediction · KITTI t+3 | MS-SSIM (× 10⁻²): 78.01 -> 83.62 | Helped |
| Use ADRNet instead of the strongest listed baseline | KITTI prediction · KITTI t+1 and t+3 | LPIPS (× 10⁻²): 10.70 -> 7.54; 19.27 -> 9.26 | Helped |
| Use ADRNet instead of MSPred | KTH Action prediction · KTH Action | SSIM: 0.930 -> 0.808 | Hurt |
| Use ADRNet instead of TAU | TaxiBJ prediction · TaxiBJ | MSE: 0.344 -> 0.445 | Hurt |
| Use ADRNet instead of IAM4VP | TaxiBJ prediction · TaxiBJ | SSIM: 0.983 -> 0.975 | Hurt |
| Use color-preserving rather than mass-preserving semi-Lagrangian transport (ablation) | ADRNet · Reported ablation experiments | model performance | No clear effect |
Surprising findings
- On KTH Action and TaxiBJ, ADRNet is competitive but does not clearly beat the strongest baselines, which the authors attribute to limited generative capacity needed for unseen scenarios.
Limitations
- Performs best on scientific datasets where the future is determined by history; less suited to video requiring generative power for new content.
- Results on KTH Action and TaxiBJ trail the strongest video prediction baselines.
- Some reported baseline numbers are compared across inconsistent setups and metrics.
Experimental setup
Datasets: PDEBench-SWE CloudCast Moving MNIST KITTI KTH Action TaxiBJ
Benchmarks: PDEBench-SWE CloudCast Moving MNIST KITTI KTH Action TaxiBJ
Metrics: NRMSE MSE MAE SSIM PSNR MS-SSIM LPIPS nRMSE
Baselines: UNET PINN MPP-AVIT ORCA-SWIN-B FNO AE-ConvLSTM MD-GAN TVL1 MSPred MAU PhyDNet SimVP
Builds on: FNO UNet PDE-inspired CNN networks (parabolic/hyperbolic)
Closest papers by meaning
- CALM-PDE: Continuous and Adaptive Convolutions for Latent Space Modeling of Time-dependent PDEs (NeurIPS 2025 · similarity 0.85)
- Space and time continuous physics simulation from partial observations (ICLR 2024 · similarity 0.83)
- PARCv2: Physics-aware Recurrent Convolutional Neural Networks for Spatiotemporal Dynamics Modeling (ICML 2024 · similarity 0.83)
- Symmetric Basis Convolutions for Learning Lagrangian Fluid Mechanics (ICLR 2024 · similarity 0.83)
- CViT: Continuous Vision Transformer for Operator Learning (ICLR 2025 · similarity 0.82)
- Augmented Lagrangian Predictive Coding (NeurIPS 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.