Research map / Vision-language and multimodal / Multimodal LLMs

Can Machines Understand Composition? Dataset and Benchmark for Photographic Image Composition Embedding and Understanding

CVPR 2025 · Vision-language and multimodal › Multimodal LLMs · approach: Interleaved image-text generation with MLLMs · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces PICD, a 36,857-image photographic composition dataset with 24 expert-verified categories and 355 scenes, and benchmarks specialized composition embedding models and MLLMs using a new Composition Discrimination Accuracy (CDA) metric.

Problem

Existing composition datasets are small, have few categories, noisy labels, and limited scene diversity, so the composition embedding ability of specialized models and the composition understanding of MLLMs cannot be reliably evaluated.

Approach

A large, expert-voted composition label system (points/lines/shapes crossed with arrangements, refined to 24 categories) plus a simple three-choice triplet task (CDA) that serves as a lightweight proxy for retrieval mAP and suits MLLM evaluation.

Key results

• Across ten specialized composition/aesthetic models, avg-CDA1 stays below 0.5 (best GANC 0.478), and mAP@10 below 0.5 for all models, showing weak composition embeddings.
• avg-CDA1 correlates with avg-mAP@10 across specialized models with Spearman 0.758 (p=0.011), supporting CDA as a lightweight proxy.
• MLLMs score near chance on composition triplet distinction (avg-CDA1 mostly around 0.33; best Qwen-VL-max 0.462 and Idefics2 0.458) and all fall below 50% on element counting and arrangement recognition.

Measured effects reported in the paper

Surprising findings

• Specialized models' avg-CDA2 under semantic interference is consistently lower than avg-CDA1, indicating semantic content couples with composition in embeddings.
• Many MLLMs score below chance-like levels on element counting and arrangement recognition despite the tasks being simple for humans.

Limitations

• Specialized-model and MLLM results are reported without detailed hardware or release information; the benchmark's reproducibility artifacts are not described in the text.
• Element counting and arrangement tasks use a subset of 1,593 manually annotated images from only seven categories.
• Label-quality comparison uses only 18 participants and 150 images.

Experimental setup

Datasets: KUPCP CADB LODB AADB AVA OpenImage PARA COCO2017 TAD66k VOC2007 VOC2012 Unsplash

Benchmarks: PICD

Metrics: Composition Discrimination Accuracy (CDA) mAP@k Silhouette score Davies-Bouldin index element counting accuracy arrangement recognition accuracy Spearman correlation

Baselines: SAMP PCP HLGCN MUSIQ BAID EAT CAC CGS GANC S2C Qwen-VL-max Qwen-VL-plus

Builds on: KUPCP CADB LODB

Closest papers by meaning

• Not Only Text: Exploring Compositionality of Visual Representations in Vision-Language Models (CVPR 2025 · similarity 0.87)
• The Photographer's Eye: Teaching Multimodal Large Language Models to See, and Critique Like Photographers (CVPR 2025 · similarity 0.87)
• Visual Data-Type Understanding does not emerge from scaling Vision-Language Models (ICLR 2024 · similarity 0.86)
• VidComposition: Can MLLMs Analyze Compositions in Compiled Videos? (CVPR 2025 · similarity 0.86)
• Progressive Compositionality in Text-to-Image Generative Models (ICLR 2025 · similarity 0.85)
• ConceptBed: Evaluating Concept Learning Abilities of Text-to-Image Diffusion Models (AAAI 2024 · similarity 0.85)