Research map / Video / Video segmentation and tracking
EulerMormer: Robust Eulerian Motion Magnification via Dynamic Filtering within Transformer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EulerMormer, a Transformer-based Eulerian video motion magnification model with a dynamic sparse cross-covariance filter and multi-scale gating, trained on a synthetic dataset with noise and blur variants.
Problem
Video motion magnification suffers from photon noise (flicker in static regions) and spatial inconsistency (blur and distortion in dynamic regions), which existing CNN-based and hand-crafted Eulerian methods handle poorly.
Approach
Disentangle texture and shape with Transformer encoders, compute motion as inter-frame shape difference, and apply a dynamic filter (top-k sparse cross-covariance attention plus multi-scale dual-path gating) to denoise motion and recoupled features during magnification and generation.
Key results
- On Synthetic-I, EulerMormer reaches RMSE 0.0594, PSNR 25.49 dB, SSIM 0.9536, LPIPS 0.0535, beating the best prior method MDLMM (0.0615, 24.84 dB, 0.9173, 0.1228).
- On real-world MANIQA, EulerMormer scores 0.6920 / 0.6760 / 0.7316 on Static / Dynamic / Fabric, above the best baseline Anisotropy (0.6872 / 0.6634 / 0.7288).
- Ablation shows removing the filter in Phase 2 raises LPIPS from 0.0535 to 0.1170, and top-k with k=7 gives the lowest RMSE of 0.0594.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use EulerMormer instead of MDLMM | Video motion magnification · Synthetic-I | RMSE: 0.0615 -> 0.0594 | Helped |
| Use EulerMormer instead of MDLMM | Video motion magnification · Synthetic-II (Poisson noise) | PSNR: 24.53 -> 25.04 | Helped |
| Use EulerMormer instead of MDLMM | Video motion magnification · Synthetic-III (Gaussian blur) | LPIPS: 0.4205 -> 0.4077 | Helped |
| Use EulerMormer instead of Anisotropy | Video motion magnification · Static | MANIQA: 0.6872 -> 0.6920 | Helped |
| Use EulerMormer instead of Anisotropy | Video motion magnification · Dynamic | MANIQA: 0.6634 -> 0.6760 | Helped |
| Use EulerMormer instead of Anisotropy | Video motion magnification · Fabric | MANIQA: 0.7288 -> 0.7316 | Helped |
| Add the dynamic filter in Phase 3 (ablation) | EulerMormer · Synthetic-I | RMSE: 0.0747 -> 0.0594 | Helped |
| Add the dynamic filter in Phase 2 (ablation) | EulerMormer · Synthetic-I | LPIPS: 0.1170 -> 0.0535 | Helped |
| Add DMF and MGR to both phases (ablation) | EulerMormer · Synthetic-I | RMSE: 0.0708 -> 0.0594 | Helped |
| Add disentangled representation loss Ldr (ablation) | EulerMormer · Synthetic-I | PSNR: 22.05 -> 24.91 | Helped |
| Add LoG edge loss (ablation) | EulerMormer · Synthetic-I | PSNR: +0.58 dB | Helped |
| Use LoG edge loss instead of Sobel loss (ablation) | EulerMormer · Synthetic-I | PSNR: +0.43 dB | Helped |
Surprising findings
- Top-k with k=11 (near-dense attention) gives a larger error (RMSE 0.0621) than k=7, showing that some sparsity helps.
Limitations
- Synthetic dataset is used for quantitative evaluation, and real-world evaluation relies on no-reference MANIQA only.
- Hyperparameters such as top-k and loss weights are tuned on Synthetic-I.
Experimental setup
Datasets: Synthetic-I Synthetic-II Synthetic-III Static dataset Dynamic dataset Fabric dataset DIS5K backgrounds StickPNG objects
Benchmarks: Synthetic-I Synthetic-II Synthetic-III Static dataset Dynamic dataset Fabric dataset
Metrics: RMSE PSNR SSIM LPIPS MANIQA
Baselines: Linear Phase Acceleration Jerk-Aware LBVMM Anisotropy LNVMM MDLMM
Builds on: Restormer Charbonnier loss Laplacian of Gaussian edge detection
Closest papers by meaning
- Frequency Decoupling for Motion Magnification via Multi-Level Isomorphic Architecture (CVPR 2024 · similarity 0.90)
- DynamicScaler: Seamless and Scalable Video Generation for Panoramic Scenes (CVPR 2025 · similarity 0.87)
- Dynamic Gaussian Splatting from Defocused and Motion-blurred Monocular Videos (NeurIPS 2025 · similarity 0.86)
- DynaMoDe-NeRF: Motion-aware Deblurring Neural Radiance Field for Dynamic Scenes (CVPR 2025 · similarity 0.85)
- 4D Gaussian Splatting in the Wild with Uncertainty-Aware Regularization (NeurIPS 2024 · similarity 0.85)
- 4D3R: Motion-Aware Neural Reconstruction and Rendering of Dynamic Scenes from Monocular Videos (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.