Research map / Image generation / Autoregressive and latent generation
Conformal Reliability: A New Evaluation Metric for Conditional Generation
Yachen Gao, Xinwei Sun, Yikai Wang, Ye Shi, Jingya Wang, Jianfeng Feng, Yanwei Fu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| calibrate jointly in a latent space instead of using directional quantile regression alone | CReL prediction set · nonlinear synthetic, alpha=0.10 | area of prediction set: 287.4 -> 232.7 | Improved |
| calibrate in latent space rather than output space (Feldman et al.) | CReL prediction set · nonlinear synthetic, alpha=0.10 | area of prediction set: 234.5 -> 232.7 | No clear effect |
| use CReL instead of PCP for prediction set | conformal calibration · nonlinear synthetic, alpha=0.1 | area of prediction set: 854.24 -> 232.70 | Improved |
| calibrate in latent space with linear programming instead of grid discretization in output space | conformal calibration · calibration runtime at r=12 | runtime (s): 122,795.0 -> 63.54 | Improved |
| replace average metric with CReL worst-case reliability score | image-to-text ranking (CLIP-SIM) · MS-COCO, alpha=0.1 | model ranking: BLIP-base: last by CLIP (0.23304) to first by CReL-CLIP (0.00701) | Mixed |
| replace average metric with CReL worst-case reliability score | text-to-image ranking (DINO-SIM) · MS-COCO, alpha=0.1 | model ranking: SD3-M: highest average DINO 0.46151 to lowest CReL-DINO -0.14804 | Mixed |
| run projected gradient descent from several random starting points (ablation) | reliability score optimization · image-to-text BLIP-base CLIP-SIM, alpha=0.1 | std over 10 runs: std 0.00027 | Improved |
| reduce VAE reconstruction quality (large reconstruction error) (ablation) | latent generative model · coverage sensitivity analysis | coverage | Hurt |
Closest papers by meaning
- Generative Conformal Prediction with Optimized Coverage Allocation (NeurIPS 2026 · similarity 0.89)
- Conformal Generative Modeling with Improved Sample Efficiency through Sequential Greedy Filtering (ICLR 2025 · similarity 0.87)
- A Unified Comparative Study with Generalized Conformity Scores for Multi-Output Conformal Regression (ICML 2025 · similarity 0.87)
- Position: All Current Generative Fidelity and Diversity Metrics are Flawed (ICML 2025 · similarity 0.87)
- A Unifying Information-theoretic Perspective on Evaluating Generative Models (AAAI 2025 · similarity 0.86)
- Conformal Prediction Beyond the Seen: A Missing Mass Perspective for Uncertainty Quantification in Generative Models (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
