Research map / Image generation / Text-to-image diffusion

MaSC: A Masked Similarity Metric for Evaluating Concept-Driven Generation

NeurIPS 2026 · Image generation › Text-to-image diffusion · approach: Inference-time T2I improvement · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MaSC, a training-free personalization metric that uses externally supplied foreground masks and a single frozen SigLIP2 forward pass to compute concept preservation (masked max-cosine over patches) and prompt following (background-only attention pooling with subject-name stripping).

Problem

Existing personalization metrics (CLIP-I, DINO-I, CLIP-T) use global pooling and correlate poorly with human judgments, conflating subject identity with background/scene content, and LLM judges are API-bound and non-differentiable.

Approach

Decompose images into subject and background regions with a foreground mask: CP uses foreground-reference patches' max cosine to any output patch; PF pools only background tokens via the SigLIP2 attention pooler's suppression mask and compares to a subject-stripped prompt embedding, all from one shared SigLIP2 forward pass.

Key results

• MaSC reaches Krippendorff alpha +0.471 on DreamBench++ concept preservation, beating all non-LLM baselines and GPT-4V (+0.432), trailing GPT-4o (+0.499) by 0.028.
• On ORIDa, MaSC achieves AUC 0.992 versus 0.986 for GPT-4o and 0.954 for the same-backbone global pool.
• MaSC prompt-following alpha +0.354 beats CLIP-T (+0.327) on DreamBench++ but trails VQAScore (+0.504) and ImageReward (+0.441).

Surprising findings

• Replacing masked max-cosine with mutual-nearest-neighbor matching drops alpha by 0.462, indicating the aggregator matters more than the encoder.
• Pooling only the foreground for prompt following yields negative alpha (-0.108), inverting the signal.

Limitations

• Depends on externally supplied foreground masks; segmentation failures or small/absent subjects affect scores and filtering.
• PF does not reach state of the art; VQAScore and ImageReward outperform it.
• Subject-name stripping is unsuitable for style prompts; ORIDa lacks human ratings; single-concept scope only.

Experimental setup

Datasets: DreamBench++ ORIDa

Benchmarks: DreamBench++ ORIDa

Metrics: Krippendorff alpha Spearman rho AUC latency

Baselines: CLIP-I DINO-I DINOv3 DreamSim AM-RADIO C-RADIOv4-SO400M DIFT-SDXL GPT-4V GPT-4o CLIP-T VQAScore ImageReward HPSv3

Builds on: SigLIP2 SAM3 DreamBench++ CLIP-T DINO

Closest papers by meaning

• The Many Senses of Visual Similarity: A Text-Prompted Image Perceptual Metric (NeurIPS 2026 · similarity 0.86)
• MMIG-Bench: Towards Comprehensive and Explainable Evaluation of Multi-Modal Image Generation Models (NeurIPS 2025 · similarity 0.86)
• DreamMatcher: Appearance Matching Self-Attention for Semantically-Consistent Text-to-Image Personalization (CVPR 2024 · similarity 0.86)
• ConceptBed: Evaluating Concept Learning Abilities of Text-to-Image Diffusion Models (AAAI 2024 · similarity 0.86)
• Semantic Surgery: Zero-Shot Concept Erasure in Diffusion Models (NeurIPS 2025 · similarity 0.86)
• CoRe: Context-Regularized Text Embedding Learning for Text-to-Image Personalization (AAAI 2025 · similarity 0.85)