Research map / Learning theory and algorithms / Prediction and decision losses
Informative Features for Model Comparison
Wittawat Jitkrittum, Heishiro Kanagawa, Patsorn Sangkloy, James Hays, Bernhard Schölkopf, Arthur Gretton
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase Rel-UME test locations from J10 to J40 (ablation) | Rel-UME · CelebA GAN comparison 4 (S, N, RN) | rejection rate: 0.57 -> 1.0 | Improved |
| Increase test locations from J10 to J20 (ablation) | Rel-UME · CelebA GAN comparison 4 (S, N, RN) | rejection rate: 0.57 -> 0.97 | Improved |
| Increase test locations from J20 to J40 (ablation) | Rel-UME · CelebA GAN comparison 4 (S, N, RN) | rejection rate: 0.97 -> 1.0 | Improved |
| Use optimized local test locations and kernel bandwidth rather than the Rel-MMD median-heuristic bandwidth | Rel-UME · Blobs | test power | Improved |
| Increase the number of test locations from J=1 to J=5 (ablation) | Rel-UME · Blobs | test power | Improved |
| Use density and log-density-gradient information rather than sample-only comparison | Rel-FSSD · RBM, epsilon=0.35, n=2000 | test power | Improved |
| Increase test locations from J=1 to J=5 (ablation) | Rel-UME · CelebA GAN comparison 4 (S, N, RN) | rejection rate: 0.57 -> 1.0 | Improved |
| Use a calibrated statistical threshold rather than treating raw FID ordering as a test | Relative model comparison · CelebA GAN comparisons 1 and 2 | false rejection rate: 0.53; 0.7 | Improved |
| Train the LSGAN from 15 to 17 epochs | LSGAN · MNIST digit 6 | relative goodness-of-fit | Improved |
| Train the LSGAN from 15 to 17 epochs | LSGAN · MNIST digits 3 and 9 | relative goodness-of-fit | Hurt |
| Use LSGAN rather than WGAN-GP after ten epochs | MNIST GAN comparison · MNIST | relative goodness-of-fit | Improved |
| Use LSGAN rather than the original GAN after 40 epochs | MNIST GAN comparison · MNIST digits 1 and 5 | relative goodness-of-fit | Improved |
Closest papers by meaning
- Quantitatively Evaluating GANs With Divergences Proposed for Training (ICLR 2018 · similarity 0.88)
- Are GANs Created Equal? A Large-Scale Study (NeurIPS 2018 · similarity 0.87)
- On the Statistical Properties of Generative Adversarial Models for Low Intrinsic Data Dimension (JMLR 2025 · similarity 0.86)
- A Domain Agnostic Measure for Monitoring and Evaluating GANs (NeurIPS 2019 · similarity 0.86)
- SMaRt: Improving GANs with Score Matching Regularity (ICML 2024 · similarity 0.85)
- Direct Discriminative Optimization: Your Likelihood-Based Visual Generative Model is Secretly a GAN Discriminator (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
