Research map / Vision-language and multimodal / Multimodal LLMs

Can Vision–Language Models Assess Graphic Design Aesthetics? A Benchmark, Evaluation, and Dataset Perspective.

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces AesEval-Bench, a 12-indicator graphic design aesthetics benchmark with three tasks (judgment, region selection, bbox localization), evaluates 10 VLMs, and fine-tunes Qwen2.5-VL-7B on a 30k indicator-grounded reasoning dataset built with human-guided VLM labeling.

Problem

Whether vision-language models can assess graphic design aesthetic quality comparably to humans, given narrow benchmarks, coarse evaluation protocols, few systematic VLM comparisons, and lack of training data.

Approach

A quantifiable benchmark built by injecting controlled perturbations into Crello designs and checking them with human review, plus a training set whose reasoning paths tie each design indicator to bounding-box regions rather than generic reasoning.

Key results

• GPT-5 achieves the best aesthetic judgment overall accuracy of 0.7252 among evaluated models, and 0.6989 on region selection.
• Fine-tuning Qwen2.5-VL-7B on AesEval-Train raises overall aesthetic judgment accuracy by 5.97 points (0.6390 to 0.6987) and precise localization IoU by 17.17 points (0.0514 to 0.2231), surpassing GPT-5's 0.1993.
• Reasoning-augmented models (GPT-o1, GPT-o3, Gemini-2.5-Pro) show no clear advantage over non-reasoning counterparts.

Surprising findings

• Reasoning-augmented VLMs offer no clear advantage over non-reasoning models on design aesthetics.
• Removing positive (good) samples from training collapses aesthetic judgment accuracy to about 0.21, far below the base model.

Limitations

• Benchmark is built only from Crello, so it excludes infographics and mobile UIs.
• Indicator taxonomy is not fully disentangled, and subjective aspects like creativity are excluded.
• Human-guided VLM labels may not be perfectly accurate; RL for reasoning is left for future work.

Experimental setup

Datasets: Crello

Benchmarks: AesEval-Bench

Metrics: accuracy IoU

Baselines: GPT-4o GPT-5 GPT-o1 GPT-o3 Gemini-2.5-Pro Qwen-VL-72B Intern-VL3-14B LLaVA-13B AesExpert-7B UNIAA-LLaVA

Builds on: Crello dataset Qwen2.5-VL

Closest papers by meaning

• VisJudge-Bench: Aesthetics and Quality Assessment of Visualizations (ICLR 2026 · similarity 0.89)
• Bridging Cognitive Gap: Hierarchical Description Learning for Artistic Image Aesthetics Assessment (AAAI 2026 · similarity 0.89)
• AesthetiQ: Enhancing Graphic Layout Design via Aesthetic-Aware Preference Alignment of Multi-modal Large Language Models (CVPR 2025 · similarity 0.88)
• Unlocking the Essence of Beauty: Advanced Aesthetic Reasoning with Relative-Absolute Policy Optimization (ICLR 2026 · similarity 0.87)
• ArtiMuse: Fine-Grained Image Aesthetics Assessment with Joint Scoring and Expert-Level Understanding (CVPR 2026 · similarity 0.87)
• Aligning Vision Models with Human Aesthetics in Retrieval: Benchmarks and Algorithms (NeurIPS 2024 · similarity 0.87)