Research map / Probabilistic and scientific modeling / Molecules and proteins

Balancing Frequencies and Pixels in Flow Matching

NeurIPS 2026 · Probabilistic and scientific modeling › Molecules and proteins · approach: Flow matching generative modeling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a Focal Log-Frequency Loss plus a sigmoid schedule that switches from frequency-domain to pixel v-loss, speeding up convergence of pixel-space flow matching on ImageNet.

Problem

Pixel-space flow matching objectives weight spatial errors uniformly, so low frequencies dominate the gradient signal (the 1/f^2 spectrum of natural images), delaying learning of high-frequency detail and slowing convergence.

Approach

A spectrally balanced frequency-domain loss (focal weighting of log-compressed DFT residuals, interpreted as a continuous Laplacian-pyramid generalization) applied early in training, then scheduled via a sigmoid to the standard pixel v-loss for spatial refinement; drop-in with no architecture change.

Key results

• VFLoss XL/16 reaches FID 2.13 vs 2.21 for JiT-XL/16 at 750k steps (unguided, 256px).
• With REPA and perceptual losses, VFLoss reaches FID 1.83 in 500k steps, matching PixelGen's 1.83 at 800k steps, with IS 323.4 vs 293.6.
• On PixelDiT, VFLoss reaches FID 10.18 at 200k steps vs 15.09 for the v-loss reproduction, roughly 2x faster convergence.

Surprising findings

• Frequency loss alone plateaus later in training and is overtaken by pixel v-loss, and a reversed schedule (pixel first, frequency later) performs among the worst.
• Applying the frequency loss in latent SiT-B x-prediction gives only a modest FID gain and does not reach the v-prediction baseline.

Limitations

• Origin of the plateau of frequency-only loss and why pixel loss helps late is not understood.
• Adds about 14% wall-clock overhead from per-step 2D FFT and dual-loss backprop.
• Switch point chosen by crossing-point rule on validation FID rather than systematic search; latent-space benefit is unclear.

Experimental setup

Datasets: ImageNet

Benchmarks: ImageNet 256x256 ImageNet 512x512

Metrics: FID FDD IS precision recall density coverage

Baselines: JiT-XL/16 JiT-H/16 JiT-G/16 EPG PixelFlow DiP PixNerd DeCo PixelGen PixelDiT SiT-B

Builds on: JiT Focal Frequency Loss flow matching REPA PixelDiT

Closest papers by meaning

• Asymmetric Flow Models (NeurIPS 2026 · similarity 0.87)
• Improving Progressive Generation with Decomposable Flow Matching (NeurIPS 2025 · similarity 0.86)
• PixelART: Image-to-Layer Decomposition without Latents or Text-to-Image Pretraining (NeurIPS 2026 · similarity 0.86)
• Blockwise Flow Matching: Improving Flow Matching Models For Efficient High-Quality Generation (NeurIPS 2025 · similarity 0.86)
• Adaptive Flow Matching for Resolving Small-Scale Physics (ICML 2025 · similarity 0.86)
• Value Gradient Guidance for Flow Matching Alignment (NeurIPS 2025 · similarity 0.86)