Research map / Video / Video generation
FRAG: Frequency Adapting Group for Diffusion Video Editing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FRAG, a training-free, model-agnostic plug-in that uses frequency-adaptive filtering and temporal grouping to set dynamic receptive fields for diffusion video editing, reducing blur and flicker.
Problem
Diffusion-based text-driven video editing suffers from blurring and flickering, attributed to a "high-frequency leak" where the denoising process fails to synthesize high-frequency components accurately.
Approach
A receptive field branch that adaptively sets frame groups per denoising step: an adaptive frequency pass filter that tracks the spatial frequencies synthesized so far, followed by hierarchical clustering of frames with a logistic-style group scheduler, so early steps use large groups (low frequencies) and later steps use many small groups (high frequencies).
Key results
- Applied to TAV, FateZero, FLATTEN and TokenFlow, FRAG improves frame-consistency FVD on DAVIS/TGVE, e.g. TokenFlow FVD 2984 to 2841 and PSNR 15.1 to 18.2.
- Human preference for TAV rises from 0.27 to 0.73 with FRAG (36 participants).
- Gains are reported across all four editing systems without any additional training.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add FRAG to TAV | TAV · DAVIS and TGVE | fidelity PSNR: 13.7 -> 14.9 | Helped |
| Add FRAG to FateZero | FateZero · DAVIS and TGVE | fidelity PSNR: 13.9 -> 15.3 | Helped |
| Add FRAG to FLATTEN | FLATTEN · DAVIS and TGVE | fidelity PSNR: 14.2 -> 15.3 | Helped |
| Add FRAG to TokenFlow | TokenFlow · DAVIS and TGVE | fidelity PSNR: 15.1 -> 18.2 | Helped |
| Add FRAG to TAV | TAV · DAVIS and TGVE | FVD: 3452 -> 3251 | Helped |
| Add FRAG to FateZero | FateZero · DAVIS and TGVE | FVD: 3241 -> 3119 | Helped |
| Add FRAG to FLATTEN | FLATTEN · DAVIS and TGVE | FVD: 3002 -> 2951 | Helped |
| Add FRAG to TokenFlow | TokenFlow · DAVIS and TGVE | FVD: 2984 -> 2841 | Helped |
| Add frequency adaptive refinement to FRAG with temporal grouping (ablation) | FateZero + FRAG · video editing quality, especially high-frequency details | high-frequency detail quality | Helped |
| Use a fixed large receptive field (ablation) | video editing quality enhancement modules · video editing | visual quality | Hurt |
| Use a fixed small receptive field (window size of 2 ~ 4) (ablation) | video editing quality enhancement modules · video editing | temporal consistency | Hurt |
Surprising findings
- FRAG helps propagation-based methods most, and is less effective with prior-guidance (optical flow) methods because the guidance prior already enforces consistency.
Limitations
- Sensitivity to the underlying consistency module; gains smaller with optical-flow guidance methods.
- Temporal grouping relies only on frequency characteristics and does not account for scene semantics, which may limit robustness on longer, more diverse videos.
- Evaluation limited to short clips (32-128 frames) from TGVE and DAVIS, with a 36-person human study.
Experimental setup
Datasets: TGVE DAVIS UCF-101
Benchmarks: TGVE DAVIS
Metrics: CLIP score FVD PSNR SSIM human preference
Baselines: Tune-A-Video FateZero FLATTEN TokenFlow
Builds on: DDIM Stable Diffusion Tune-A-Video TokenFlow FateZero
Closest papers by meaning
- FADE: Frequency-Aware Diffusion Model Factorization for Video Editing (CVPR 2025 · similarity 0.87)
- High-Fidelity Diffusion-Based Image Editing (AAAI 2024 · similarity 0.84)
- FDS: Frequency-Aware Denoising Score for Text-Guided Latent Diffusion Image Editing (CVPR 2025 · similarity 0.84)
- FreqPrior: Improving Video Diffusion Models with Frequency Filtering Gaussian Noise (ICLR 2025 · similarity 0.83)
- Schedule Your Edit: A Simple yet Effective Diffusion Noise Schedule for Image Editing (NeurIPS 2024 · similarity 0.83)
- Re-Attentional Controllable Video Diffusion Editing (AAAI 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.