Research map / Image generation / Autoregressive and latent generation
The Fault in Our Metrics: Revisiting Generative Model Evaluation with Chamfer Distance
Chirag Vashist, Vaibhav Santhosh, Tristan Engst, Yanshu Zhang, Ke Li
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Revisits generative image model evaluation, arguing FID and KNN-ball precision/recall metrics are misaligned with real data geometry, and proposes Chamfer-distance-based evaluation instead.
Problem
Widely used generative image metrics (FID and KNN-ball precision/recall) rely on assumptions misaligned with the geometry of real feature distributions, so they may fail to reliably assess generative models.
Approach
FID reduces feature distributions to a Gaussian moment summary and is therefore blind to differences beyond the first two moments; the paper proposes evaluating generative models using Chamfer distance to better match the geometry of real data.
Experimental setup
Metrics: FID precision recall Chamfer distance
Baselines: FID precision/recall (KNN-ball)
Builds on: FID precision/recall for generative models
Closest papers by meaning
- Efficient Precision and Recall Metrics for Assessing Generative Models using Hubness-aware Sampling (ICML 2024 · similarity 0.89)
- Rethinking FID: Towards a Better Evaluation Metric for Image Generation (CVPR 2024 · similarity 0.88)
- A Unifying Information-theoretic Perspective on Evaluating Generative Models (AAAI 2025 · similarity 0.87)
- Distance Marching for Generative Modeling (NeurIPS 2026 · similarity 0.87)
- STREAM: Spatio-TempoRal Evaluation and Analysis Metric for Video Generative Models (ICLR 2024 · similarity 0.86)
- Representation Fréchet Loss for Visual Generation (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
