Research map / Video / Video segmentation and tracking
Harnessing Meta-Learning for Improving Full-Frame Video Stabilization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Applies MAML-style meta-learning so pixel-synthesis full-frame video stabilizers (DMBVS, DIFRINT) can adapt at test time to each input video with a single adaptation pass, improving stability and quality.
Problem
Pixel-synthesis full-frame video stabilization models use fixed parameters and generalize poorly across videos with diverse motion profiles and content; test-time optimization methods are too slow.
Approach
Meta-train feed-forward stabilizers with MAML so they can be fast-adapted at test time using a self-supervised inner loss (global-optical-flow stability against rigid-affine-aligned frames plus perceptual, gram and contextual losses) that needs no ground-truth stable frames; outer loop uses DeepStab stable videos.
Key results
- Meta-trained DIFRINT with one adaptation pass over all sequences reaches 0.8786 stability on NUS, the best among compared methods, versus 0.7904 baseline DIFRINT.
- Adaptation gives about 5% average absolute stability gain for DMBVS and about 8% for DIFRINT with a single adaptation pass on test videos.
- Meta-trained DMBVS improves stability from 0.7372 to 0.7867 with recurrent frames while distortion stays high.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add 1-step test-time adaptation using 100 sampled sequences | DMBVS · NUS dataset | stability: 0.7372 -> 0.7532 | Helped |
| increase test-time adaptation from 1 to 5 steps using 100 sampled sequences (ablation) | DMBVS · NUS dataset | stability: 0.7532 -> 0.7852 | Helped |
| adapt on all sequences instead of 100 sampled sequences (ablation) | DMBVS with 1-step adaptation · NUS dataset | stability: 0.7532 -> 0.7760 | Helped |
| add frame recurrence after 1-step adaptation on all sequences (ablation) | DMBVS · NUS dataset | stability: 0.7760 -> 0.7867 | Helped |
| add 1-step test-time adaptation using 100 sampled sequences | DIFRINT · NUS dataset | stability: 0.7904 -> 0.8428 | Helped |
| increase test-time adaptation from 1 to 5 steps using 100 sampled sequences (ablation) | DIFRINT · NUS dataset | stability: 0.8428 -> 0.8528 | Helped |
| adapt on all sequences instead of 100 sampled sequences (ablation) | DIFRINT with 1-step adaptation · NUS dataset | stability: 0.8428 -> 0.8786 | Helped |
| increase the weight of the stability loss during adaptation (ablation) | Meta-trained DMBVS · QuickRotation6, Running14, Crowd3, Zooming20 | stability | Helped |
| increase the weight of the quality loss during adaptation (ablation) | Meta-trained DMBVS · QuickRotation6, Running14, Crowd3, Zooming20 | distortion | Helped |
| use category-specific stability and quality loss weights (ablation) | Meta-trained DMBVS · NUS dataset categories | stability: 1∼2% | Helped |
| fine-tune the baseline model instead of using meta-trained initialization (ablation) | DMBVS · worst-performing evaluation videos | stability | Hurt |
Surprising findings
- Fine-tuning the baseline for many iterations yields only minuscule gains, whereas meta-inference with a single adaptation pass gives significant improvement.
Limitations
- Test-time adaptation still adds per-video compute compared with purely feed-forward inference.
- Stability gains depend on the loss weights and category-specific ratios chosen per motion type.
- Some baselines (StabNet, Adobe Premiere Pro) fail on some videos, so averages are computed over subsets.
Experimental setup
Datasets: DeepStab NUS dataset (Liu et al.)
Benchmarks: NUS dataset
Metrics: stability cropping distortion
Baselines: DMBVS DIFRINT L1 stabilizer Bundled camera paths Adobe Premiere Pro 2020 warp stabilizer StabNet FuSta Yu and Ramamoorthi et al. Zhang et al.
Builds on: MAML DMBVS DIFRINT DeepStab
Closest papers by meaning
- Instant Video Models: Universal Adapters for Stabilizing Image-Based Networks (NeurIPS 2025 · similarity 0.86)
- BIVDiff: A Training-Free Framework for General-Purpose Video Synthesis via Bridging Image and Video Diffusion Models (CVPR 2024 · similarity 0.84)
- Rebalancing Reference Frame Dominance to Improve Motion in Image-to-Video Models (NeurIPS 2026 · similarity 0.84)
- Align-A-Video: Deterministic Reward Tuning of Image Diffusion Models for Consistent Video Editing (CVPR 2025 · similarity 0.84)
- AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformers (CVPR 2025 · similarity 0.84)
- Ctrl-Adapter: An Efficient and Versatile Framework for Adapting Diverse Controls to Any Diffusion Model (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.