Research map / Image generation / Autoregressive and latent generation
Representation Fréchet Loss for Visual Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows Fréchet Distance can be optimized directly as a training loss by decoupling a large feature population (queue or EMA moments) from the gradient batch, and proposes FDr^K, a multi-representation metric.
Problem
FID is used only as an evaluator because optimizing Fréchet Distance as a loss requires populations (~50k) far larger than training batches, making batch-wise FD estimates noisy and degrading generators.
Approach
Decouple the population size used for FD estimation (queue of 50k features or EMA of feature moments) from the batch size used for gradients (1024), back-propagating only through the current batch; combine FD across multiple frozen representation spaces with normalized losses.
Key results
- Under Inception FD, pMF-H one-step generator reaches 0.72 FID on ImageNet 256 (from 2.29), and JiT-H repurposed to one-step reaches 0.75 FID (from 1.97 at 200 NFE).
- FD-SIM post-training drives pMF-H to FDr^6 1.89 versus 6.87 for the base model.
- Queue size 50k with batch 1024 gives FID 0.89; EMA beta 0.999 gives FID 0.81 for pMF-B/16, versus 3.84 FID when statistics use the current batch only.
Surprising findings
- Optimizing Inception FD improves FID but can worsen modern-representation metrics; ViT-based representations worsen FID yet improve FDr^6 more.
- Over-optimizing Inception metrics with 100x learning rate yields 2.09 FID and 660 IS but FDr^6 rises to 50.66 with visible artifacts.
Limitations
- Results depend on the choice of representation spaces and reference statistics; other domains may need different feature sets.
- FDr^K inherits Gaussian moment-matching assumption of Fréchet distance and may be hackable by optimizing the chosen representations.
- Experiments focus on ImageNet class-conditional generation plus one text-to-image demo.
Experimental setup
Datasets: ImageNet-1k BLIP3o-Pretrain-Long-3M BLIP3o-GPT4o-60k
Benchmarks: ImageNet 256x256 ImageNet 512x512 ImageNet class-conditional generation with SD3.5 text-to-image
Metrics: FID IS Precision Recall FDr^6 human preference
Baselines: pMF iMF JiT SiT-XL/2 MAR VAR BAR RAE REPA-E Drift
Builds on: FID MoCo pixel-MeanFlow improved MeanFlow JiT
Closest papers by meaning
- Rethinking FID: Towards a Better Evaluation Metric for Image Generation (CVPR 2024 · similarity 0.90)
- Fréchet Wavelet Distance: A Domain-Agnostic Metric for Image Generation (ICLR 2025 · similarity 0.89)
- Distance Marching for Generative Modeling (NeurIPS 2026 · similarity 0.86)
- On the Content Bias in Frechet Video Distance (CVPR 2024 · similarity 0.86)
- STREAM: Spatio-TempoRal Evaluation and Analysis Metric for Video Generative Models (ICLR 2024 · similarity 0.85)
- Improving Progressive Generation with Decomposable Flow Matching (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.