Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Neuroexplicit Diffusion Models for Inpainting of Optical Flow Fields
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Combines a PDE-based edge-enhancing diffusion inpainting process for optical flow with a learned U-Net diffusion tensor module, trained end-to-end, to inpaint sparse flow fields from random masks.
Problem
Inpainting sparse optical flow fields from random masks: purely PDE methods need many manually tuned parameters and fail in low-contrast regions, while deep models (FlowNetS, WGAIN, probabilistic diffusion) need large data and generalize poorly out of domain.
Approach
A neuroexplicit architecture where a U-Net Diffusion Tensor Module predicts per-pixel diffusion tensor eigenvalues/eigenvectors and discretization parameter alpha that drive a stable, coarse-to-fine explicit anisotropic diffusion process, replacing the structure tensor heuristic.
Key results
- On Sintel at 5% density, the neuroexplicit model reaches EPE 0.40 vs 0.52 for EED, 0.51 for AMLE-type baseline and 0.57 for FlowNetS-type baseline (best among compared).
- Averaged across mask densities the method outperforms explicit baselines by 11-27%, the GAN baseline by 47% and the probabilistic diffusion baseline by 42% (abstract).
- Trained on only 194 samples, it reaches competitive performance with the network trained on the full dataset.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace learned eigenvalues with explicit eigenvalues (ablation) | Neuroexplicit Diffusion · FlyingThings3D test, 1% | EPE: +0.95 | Hurt |
| Replace learned eigenvalues with explicit eigenvalues (ablation) | Neuroexplicit Diffusion · FlyingThings3D test, 5% | EPE: +0.28 | Hurt |
| Replace learned eigenvalues with explicit eigenvalues (ablation) | Neuroexplicit Diffusion · Sintel test, 1% | EPE: +0.17 | Hurt |
| Replace learned eigenvectors with explicit eigenvectors (ablation) | Neuroexplicit Diffusion · Sintel test, 5% | EPE: +0.06 | Hurt |
| Replace learned spatially varying alpha with constant explicit alpha (ablation) | Neuroexplicit Diffusion · Sintel test, 10% | EPE: +0.02 | Hurt |
| Learn the second discretization parameter beta (ablation) | Neuroexplicit Diffusion · FlyingThings3D and Sintel | EPE: -0.04; -0.03; -0.01; -0.02; -0.01; -0.01 | No clear effect |
| Learn the finite-difference operators (ablation) | Neuroexplicit Diffusion · FlyingThings3D and Sintel | EPE: +0.00; -0.01; +0.03; +0.01; -0.01; +0.00 | No clear effect |
Surprising findings
- Explicit EED outperforms the purely data-driven baselines on the Sintel and training-domain tests.
- Learning the second discretization parameter beta gives insignificant gains and breaks the stability restriction, so it is not recommended.
Limitations
- Only the regularization/inpainting component is studied; not embedded in an end-to-end optical flow estimator.
- Explicit EED converges slowly and leaks information in low-contrast regions.
- Probabilistic diffusion and WGAIN generalize poorly; PD was not evaluated on KITTI due to resolution mismatch.
Experimental setup
Datasets: FlyingThings3D Sintel KITTI 2015
Benchmarks: FlyingThings3D test Sintel KITTI 2015
Metrics: endpoint error (EPE) Fl outlier rate
Baselines: Edge-Enhancing Diffusion AMLE Laplace-Beltrami FlowNetS WGAIN Probabilistic Diffusion with RePaint
Builds on: edge-enhancing diffusion nonstandard finite difference discretization fast semi-iterative scheme RePaint WGAIN
Closest papers by meaning
- Video Diffusion Models Are Strong Video Inpainter (AAAI 2025 · similarity 0.87)
- RAD: Region-Aware Diffusion Models for Image Inpainting (CVPR 2025 · similarity 0.86)
- Image Neural Field Diffusion Models (CVPR 2024 · similarity 0.86)
- Neural Diffusion Models (ICML 2024 · similarity 0.86)
- Image Inpainting via Iteratively Decoupled Probabilistic Modeling (ICLR 2024 · similarity 0.86)
- InpDiffusion: Image Inpainting Localization via Conditional Diffusion Models (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.