Research map / Video / Video segmentation and tracking
Generalizable Fourier Augmentation for Unsupervised Video Object Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Generalizable Fourier Augmentation (GFA), which samples Fourier amplitude statistics from Gaussians and EMA-updates phase features inside a SegFormer-based UVOS network to improve out-of-distribution generalization.
Problem
Unsupervised video object segmentation degrades severely on out-of-distribution test videos due to domain shift (scene style shift and semantic shift).
Approach
Apply FFT to intermediate encoder features, augment amplitude (style) statistics by Gaussian sampling with reparameterization and AdaIN-like transform, and smooth phase (semantic) features across past frames with an exponential moving average so the model learns domain-invariant features.
Key results
- GFA (SegFormer) reaches 88.2 J&F on DAVIS-2016 and 82.4 J on FBMS, versus 86.7 J&F for HFAN on DAVIS-2016.
- GFA (SegFormer) achieves 74.7 average J on YouTube-Objects, the best overall average in Table 1 against HFAN's 73.4.
- Adding GFA to the SegFormer baseline improves J&F on DAVIS-2016 by 2.9 points and J on FBMS by 3.2 points, and lowers IIW from 5.2E-06 to 2.8E-06.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add generalizable Fourier augmentation with SegFormer (ablation) | UVOS model · DAVIS-2016 | J &F: 85.3 -> 88.2 | Helped |
| Add generalizable Fourier augmentation with SegFormer (ablation) | UVOS model · FBMS | J: 79.2 -> 82.4 | Helped |
| Add amplitude feature augmentation via Gaussian sampling (ablation) | Transformer-based baseline · DAVIS-2016 | J &F: +2.2% | Helped |
| Add amplitude feature augmentation via Gaussian sampling (ablation) | Transformer-based baseline · FBMS | J: +2.3% | Helped |
| Add phase feature online update (ablation) | GFA · DAVIS-2016 and FBMS | evaluation metrics: about 1% | Helped |
Limitations
- Gains from the phase EMA component are about 1 point, and the paper relies on the SegFormer backbone for best results.
- Evaluation of domain shift relies on the IIW metric and qualitative distribution plots rather than dedicated OOD benchmarks.
Experimental setup
Datasets: DAVIS-2016 YouTubeVOS-2018
Benchmarks: DAVIS-2016 FBMS YouTube-Objects
Metrics: J F J&F IIW
Baselines: AGS AGNN COSNet MATNet GraphMem WCS-Net AMCNet RTNet HFAN TMO
Builds on: SegFormer AdaIN RAFT
Closest papers by meaning
- Generalizable Multi-Camera 3D Object Detection from a Single Source via Fourier Cross-View Learning (ICML 2025 · similarity 0.86)
- Fourier-basis Functions to Bridge Augmentation Gap: Rethinking Frequency Augmentation in Image Classification (CVPR 2024 · similarity 0.85)
- EFDTR: Learnable Elliptical Fourier Descriptor Transformer for Instance Segmentation (ICML 2025 · similarity 0.84)
- beta-FFT: Nonlinear Interpolation and Differentiated Training Strategies for Semi-Supervised Medical Image Segmentation (CVPR 2025 · similarity 0.84)
- Dual Prototype Attention for Unsupervised Video Object Segmentation (CVPR 2024 · similarity 0.84)
- Matching Anything by Segmenting Anything (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.