Research map / Image generation / Autoregressive and latent generation
Fréchet Wavelet Distance: A Domain-Agnostic Metric for Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Fréchet Wavelet Distance (FWD), a generative-image metric that computes per-packet Fréchet distances on level-4 Haar wavelet packet coefficients, avoiding pretrained-network domain bias.
Problem
FID and DINOv2-based FD are biased toward ImageNet-like content and specific generators, leading to misleading rankings on out-of-domain datasets and vulnerability to manipulation; they are also costly to compute.
Approach
Replace pretrained-network features with Haar wavelet packet transform coefficients, compute a Fréchet distance per packet, and average across packets, giving a deterministic, network-free, interpretable metric that captures both spatial and frequency information.
Key results
- On CelebA-HQ, FID prefers ProjGAN (6.358 vs 7.641) while FWD prefers DDGAN (0.408 vs 1.388), matching a user study where DDGAN had higher human-realism rate (32.5% vs 20.0%).
- FWD is about 1923 images/sec versus 53 for FD-DINOv2 and 526 for FID, with 0.006 GFLOPs versus 15.566 for FD-DINOv2.
- Optimizing sample weights to lower FID by ~52.5% lowered FD-DINOv2 by only 1.47% and made FWD slightly worse (+17.75%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FWD instead of ImageNet-pretrained FID to compare generators | generator evaluation · CelebAHQ, FFHQ, Agriculture, Sentinel | generator ranking by distance: CelebAHQ: ProjGAN 1.388 vs DDGAN 0.408; FFHQ: ProjGAN 0.651 vs StyleGAN2 0.312; Agriculture: ProjGAN 1.442 vs DDGAN 1.35 | Mixed |
| Use FWD instead of FD-DINO for evaluation | generator evaluation · Agriculture | generator ranking by distance: ProjGAN 1.442; DDGAN 1.357 | Helped |
| Use FWD instead of FD-DINO for evaluation | metric computation · single image and 50k images | GFLOPs and throughput: 15.566 -> 0.006 GFLOPs; 53 -> 1923 imgs/sec | Helped |
| Optimize sample weights for FID | StyleGAN2 samples on FFHQ · FFHQ | FWD: 0.338 ± 0.017 -> 0.398 ± 0.009; +17.75% | Hurt |
| Optimize sample weights for FID | StyleGAN2 samples on FFHQ · FFHQ | FID: 4.278 ± 0.019 -> 2.031 ± 0.005; -52.53% | Helped |
| Optimize sample weights for FID | StyleGAN2 samples on FFHQ · FFHQ | FD-DINO: 420.223 ± 0.563 -> 414.048 ± 0.905; -1.47% | Helped |
| Fine-tune InceptionV3 on Agriculture for FID | generator evaluation · Agriculture | generator ranking by FID (DNDD): ProjGAN 20.937; DDGAN 52.521 | Hurt |
| Evaluate generated samples with FWD | Stable Diffusion model ranking · Conceptual Captions validation | alignment with user rating: StableDiffusion-3.5 (Large): FWD 4.979, User Rating 61%; StableDiffusion-1.5: FWD 17.498, User Rating 14% | Helped |
Surprising findings
- FID ranks ProjGAN above DDGAN on CelebA-HQ even though visual inspection shows more deformations in ProjGAN samples, attributed to InceptionV3 class-distribution bias.
- Retraining InceptionV3 on the small agriculture dataset (DNDD) did not fix FID's preference for ProjGAN.
Limitations
- Relies on user studies with a small number of volunteers (about 50) and limited comparison pairs.
- Several key experiments use self-trained generators on small custom datasets, so generalization is not fully established.
- Speculated that LVD-142M may contain satellite imagery; DINOv2 training data is closed, limiting analysis.
Experimental setup
Datasets: CelebA-HQ FFHQ Agriculture (DNDD) Sentinel-1/2 (ROIs_2017_Winter) LSUN Churches LSUN Bedrooms ImageNet CelebA Conceptual Captions
Benchmarks: CelebA-HQ FFHQ Agriculture Sentinel LSUN Churches LSUN Bedrooms ImageNet 64
Metrics: FWD FID FD-DINOv2 KID throughput GFLOPs human preference rate
Baselines: FID FD-DINOv2 KID SWD FID_inf IS Clean-FID
Builds on: Fréchet Inception Distance DINOv2-Fréchet Distance Sliced Wasserstein Distance Kernel Inception Distance
Closest papers by meaning
- Representation Fréchet Loss for Visual Generation (NeurIPS 2026 · similarity 0.89)
- Rethinking FID: Towards a Better Evaluation Metric for Image Generation (CVPR 2024 · similarity 0.87)
- STREAM: Spatio-TempoRal Evaluation and Analysis Metric for Video Generative Models (ICLR 2024 · similarity 0.85)
- On the Content Bias in Frechet Video Distance (CVPR 2024 · similarity 0.85)
- Distance Marching for Generative Modeling (NeurIPS 2026 · similarity 0.83)
- Improving Progressive Generation with Decomposable Flow Matching (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.