Research map / Image generation / Autoregressive and latent generation

RA-ClipScore: Making Generative Model Evaluation More Interpretable

NeurIPS 2026 · Image generation › Autoregressive and latent generation · approach: Synthetic training data from generators · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RA-CLIPScore, a training-free CLIP-based metric using dual positive/negative prompts and local patch tokens (from layer L-1) to measure attribute and spatial distribution alignment of generative image models, with SaD, PaD and R-SaD divergences.

Problem

Scalar generative metrics like FID give little diagnostic insight, and existing CLIP-based attribute metrics (CLIPScore, HCS) suffer from softmax competition among attributes and global-only CLIP embeddings, so they cannot reveal attribute-wise or spatial biases.

Approach

Reformulate each attribute as an independent binary presence/absence prompt pair, compute patch-level scores by bypassing the final attention layer for local tokens (keeping the global CLS token), refine them with attention from earlier layers, and aggregate them to get attribute scores and per-patch spatial divergences (R-SaD).

Key results

• On CelebA attribute classification, RA-CLIPScore reaches 79.75% accuracy and 55.20 F1 under the irrelevant-attribute setting, versus HCS 79.30 accuracy and 54.20 F1 under that setting.
• In a user study with 29 participants, R-SaD correlated r=1.0 with human diversity judgments, while FID had r=0.5 and Coverage r=0.29.
• In the ablation on CelebA, the full method reaches Cohen's d of 3.16 for Male and 3.03 for Eyeglasses, compared with 1.55 and 2.01 for HCS.

Surprising findings

• RA-CLIPScore correlates only r=0.6 with human diversity judgments, lower than the spatial R-SaD built on it.
• Recall and Coverage correlate least with human judgments of diversity among the tested metrics.

Limitations

• CLIP as the sole feature extractor inherits CLIP's known limitations; alternative encoders are not explored.
• RA-CLIPScore is less correlated with human diversity judgments (r=0.6) than R-SaD, and global features may override spatial signals.
• The user study used 29 participants and 20 attributes, a small-scale study.

Experimental setup

Datasets: FFHQ CelebA COCO ImageNet

Benchmarks: CelebA attribute classification FFHQ generative model evaluation ImageNet spatial evaluation COCO injection experiment

Metrics: accuracy F1 Cohen's d Hellinger distance 1-OVL FID KID precision recall density

Baselines: CLIPScore HCS FID KID Precision/Recall Density/Coverage LPIPS

Builds on: CLIPScore Heterogeneous CLIPScore (HCS) dual prompts Precision/Recall FID

Closest papers by meaning

• Attribute Based Interpretable Evaluation Metrics for Generative Models (ICML 2024 · similarity 0.89)
• Revisiting text-to-image evaluation with Gecko: on metrics, prompts, and human rating (ICLR 2025 · similarity 0.86)
• Demystifying CLIP Data (ICLR 2024 · similarity 0.86)
• Compositional Scene Understanding through Inverse Generative Modeling (ICML 2025 · similarity 0.86)
• Does CLIP’s generalization performance mainly stem from high train-test similarity? (ICLR 2024 · similarity 0.86)
• STREAM: Spatio-TempoRal Evaluation and Analysis Metric for Video Generative Models (ICLR 2024 · similarity 0.85)