Research map / Video / Video segmentation and tracking
Frequency Decoupling for Motion Magnification via Multi-Level Isomorphic Architecture
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FD4MM, a frequency-decoupling Transformer with a multi-level isomorphic architecture, sparse high/low-pass filters, a sparse frequency mixer, and contrastive regularization for video motion magnification.
Problem
Video motion magnification must reveal subtle motion while avoiding noise-induced undesired motion, ringing and artifacts; prior Eulerian hand-crafted filters and learning-based representation/multi-domain methods suffer from artifacts, disentanglement errors, and high compute.
Approach
Decouple spatial features into low- and high-frequency bands with a recursive multi-level isomorphic encoder, model motion from the stable low-frequency band, and use ReLU-sparse attention filters and a sparse mixer to suppress noise during magnification and recoupling, plus a VGG-feature contrastive loss.
Key results
- FD4MM achieves average MANIQA 0.6780 on real-world videos vs 0.6551 for MDLMM (previous best).
- On the synthetic dataset FD4MM reaches SSIM 0.9104 and LPIPS 0.1376, versus SSIM 0.8864 without the proposed loss combination.
- FD4MM uses 24.9 GFLOPs versus 65.4 for MDLMM, cutting FLOPs by about 1.63x, and runs at 82.9 ms vs 222.1 ms per frame pair at 720x720.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FD4MM instead of MDLMM | Motion magnification on Real-world Datasets · Real-world Datasets (Avg.) | MANIQA: 0.6551 vs. 0.6780 | Helped |
| Use FD4MM instead of MDLMM | Motion magnification model (720×720) · 720×720 inference | FLOPs (G): 65.4 vs. 24.9 | Helped |
| Use FD4MM instead of MDLMM | Motion magnification model (720×720) · 720×720 inference | inference time (ms): 222.1 vs. 82.9 | Helped |
| Add middle and deep frequency bands (three-level architecture) (ablation) | FD4MM · Synthetic Dataset | SSIM: 0.8582 -> 0.9104 | Helped |
| Add an extra frequency-decoupling band (ablation) | FD4MM three-level architecture · Synthetic Dataset | SSIM: 0.9104 -> 0.9008 | Hurt |
| Add the sparse low-pass filter, high-pass filter, and frequency mixer (ablation) | FD4MM MIA baseline · Synthetic Dataset | SSIM: 0.7651 -> 0.8177 -> 0.8521 -> 0.9104 | Helped |
| Replace the proposed Transformer components with Swin-T or Restormer (ablation) | FD4MM · Synthetic Dataset | SSIM: 0.9104 vs. 0.8864; 0.9104 vs. 0.8973 | Hurt |
| Remove the sparse strategy (ablation) | FD4MM · Synthetic Dataset | SSIM: 0.9088 vs. 0.9104 | Hurt |
| Use LoG edge loss instead of Sobel edge loss (ablation) | FD4MM · Synthetic Dataset | SSIM: 0.8901 -> 0.8937 | Helped |
| Add contrastive regularization to LoG edge loss (ablation) | FD4MM · Synthetic Dataset | SSIM: 0.8937 -> 0.9104 | Helped |
Surprising findings
- Sparse ReLU attention gives only a small gain over softmax-style full attention (SSIM 0.9104 vs 0.9088).
Limitations
- Real-world videos lack ground truth, so evaluation relies on a no-reference quality metric (MANIQA) and a self-built synthetic benchmark.
- Parameter count (1.47M) is higher than MDLMM (0.12M), trading parameters for speed.
- Hyperparameters such as the number of layers per band were chosen empirically.
Experimental setup
Datasets: Training dataset of Oh et al. (LBVMM, ECCV'18) DIS5K StickPNG
Benchmarks: Real-world motion magnification videos (12 videos) Synthetic test dataset (10 videos)
Metrics: MANIQA SSIM LPIPS FLOPs Params inference time
Baselines: Acceleration Jerk-Aware Anisotropy LBVMM LNVMM MDLMM Swin-T Restormer
Builds on: Eulerian video magnification Restormer MDLMM Swin Transformer
Closest papers by meaning
- EulerMormer: Robust Eulerian Motion Magnification via Dynamic Filtering within Transformer (AAAI 2024 · similarity 0.90)
- Motion4D: Learning 3D-Consistent Motion and Semantics for 4D Scene Understanding (NeurIPS 2025 · similarity 0.85)
- Generative Perception of Shape and Material from Differential Motion (NeurIPS 2025 · similarity 0.84)
- Degrees of Freedom Matter: Inferring Dynamics from Point Trajectories (CVPR 2024 · similarity 0.84)
- Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting (ICLR 2024 · similarity 0.84)
- HiCoM: Hierarchical Coherent Motion for Dynamic Streamable Scenes with 3D Gaussian Splatting (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.