Research map / Vision-language and multimodal / Multimodal LLMs
AesthetiQ: Enhancing Graphic Layout Design via Aesthetic-Aware Preference Alignment of Multi-modal Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AAPA, which applies DPO-style preference alignment to an MLLM layout predictor using a VILA judge's pairwise aesthetic preferences, plus quality-based data filtering and a judge win-rate metric, reporting gains on Crello and WebUI.
Problem
Content-aware graphic layout generation is typically trained with cross-entropy on coordinate tokens, which penalizes aesthetically valid alternative layouts and ignores rendered aesthetic quality.
Approach
Aligns an MLLM layout predictor to aesthetic preferences by DPO over pairs of layouts ranked by an MLLM judge (VILA-7B), with alignment/overlap-based filtering of training layouts and an MLLM-judged win rate versus ground truth.
Key results
- AesthetiQ-8B reaches 42.83% mean IoU (All) and 17.19% judge win rate on Crello, versus 25.74% IoU and 5.58% win rate for LayoutNUWA.
- On WebUI, AesthetiQ-8B achieves 48.29% mean IoU and 24.48% win rate, compared to 32.16% and 15.28% for LayoutNUWA.
- In user study 1, 78.41% of preferences went to AesthetiQ layouts; VILA agreed with human preferences 78.8% of the time.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase model scale from 1B to 8B (ablation) | AesthetiQ · Crello | Mjudge Win Rate (%): 2.43% to 17.19% | Helped |
| Add pretraining on 80,000 templates (ablation) | AesthetiQ-8B without alignment or filtering · Crello All | Mean IoU (%): 37.64% to 40.81% | Helped |
| Add quality-based data filtering (ablation) | AesthetiQ-4B with pretraining and VILA alignment · Crello All | Mean IoU (%): +4.97% | Helped |
| Add quality-based data filtering (ablation) | AesthetiQ-4B with pretraining and VILA alignment · Crello All | Mjudge Win Rate (%): +2.56% | Helped |
| Add quality-based data filtering (ablation) | AesthetiQ-8B with pretraining and VILA alignment · Crello All | Mean IoU (%): 39.18 -> 42.83 | Helped |
| Add VILA preference alignment (ablation) | AesthetiQ-1B without pretraining or filtering · Crello All | Mean IoU (%): 22.06 -> 17.45 | Hurt |
| Add VILA preference alignment (ablation) | AesthetiQ-1B without pretraining or filtering · Crello All | Mjudge Win Rate (%): 2.18 -> 1.64 | Hurt |
| Use AAPA preference alignment (ablation) | AesthetiQ-8B with pretraining and filtering · Crello All | Mjudge Win Rate (%): 16.13 -> 17.19 | Helped |
| Use AAPA preference alignment (ablation) | AesthetiQ-4B with pretraining and filtering · Crello All | Mjudge Win Rate (%): 12.18 -> 14.74 | Helped |
| Use AAPA preference alignment (ablation) | AesthetiQ-2B with pretraining and filtering · Crello All | Mjudge Win Rate (%): 4.93 -> 6.13 | Helped |
| Use AAPA preference alignment (ablation) | AesthetiQ-1B with pretraining and filtering · Crello All | Mjudge Win Rate (%): 2.02 -> 2.43 | Helped |
| Use GPT-4o instead of VILA for training and evaluation judging (ablation) | AesthetiQ-8B · Crello All | Mjudge Win Rate (%): 17.19 -> 19.41 | Helped |
Surprising findings
- Judge win rate for AesthetiQ-1B shows limited gains from AAPA, which the authors attribute to poor sampled layouts limiting preference optimization.
Limitations
- Relies on an MLLM judge (VILA-7B) whose agreement with humans is only 78.8%, below GPT-4o's reported 88.6%.
- Layout rendering for WebUI uses bounding boxes only, so the judge sees simplified renderings.
- Gains are small for the 1B model, where sampled layouts are often low quality.
Experimental setup
Datasets: Crello WebUI 80k online design templates (pretraining)
Benchmarks: Crello WebUI
Metrics: mean IoU judge win rate user study agreement
Baselines: FlexDM LACE PosterLLaVa LayoutNUWA Desigen SmartText+ Typography LMM
Builds on: Direct Preference Optimization InternVL VILA Desigen
Closest papers by meaning
- Seeing is Improving: Visual Feedback for Iterative Text Layout Refinement (CVPR 2026 · similarity 0.89)
- Towards Aligned Layout Generation via Diffusion Model with Aesthetic Constraints (ICLR 2024 · similarity 0.88)
- Can Vision–Language Models Assess Graphic Design Aesthetics? A Benchmark, Evaluation, and Dataset Perspective. (ICLR 2026 · similarity 0.88)
- Retrieval-Augmented Layout Transformer for Content-Aware Layout Generation (CVPR 2024 · similarity 0.88)
- Graphic Design with Large Multimodal Model (AAAI 2025 · similarity 0.87)
- PosterO: Structuring Layout Trees to Enable Language Models in Generalized Content-Aware Layout Generation (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.