Research map / Vision-language and multimodal / Multimodal LLMs
Beyond Atomic Layouts: Compositional Design Understanding with Vision-Language Models
Yiyang Huang, Zhaowen Wang, Simon Jenni, Jing Shi, Yitian Zhang, Yizhou Wang, Yun Fu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add multimodal alignment (grounding) to direct finetuning (ablation) | Direct Finetune on Qwen2.5-VL 7B · CoDeLayout (type-balanced) | weighted accuracy: 80.21 -> 82.03 | Improved |
| add structural perception (spatial relationship metadata) to direct finetuning (ablation) | Direct Finetune on Qwen2.5-VL 7B · CoDeLayout (type-balanced) | weighted accuracy: 80.21 -> 83.85 | Improved |
| combine MA and SP (full MASON) versus either alone (ablation) | Qwen2.5-VL 7B · CoDeLayout (type-balanced) | weighted accuracy: 83.85 -> 84.11 | Improved |
| add MA on Morphing category (ablation) | Direct Finetune · CoDeLayout Morphing (type-balanced) | accuracy: 72.34 -> 70.21 | Hurt |
| train MASON on 30% type-balanced data instead of Direct Finetune on full data | Qwen2.5-VL 7B · CoDeLayout test | weighted accuracy: 88.80 -> 89.32 | Improved |
| MASON versus Direct Finetune across data scales 10-100% (ablation) | Qwen2.5-VL 7B · CoDeLayout test | weighted accuracy | Improved |
| replace GPT-4o grounding generator with Qwen3-VL script-based grounding (ablation) | MASON · CoDeLayout test | weighted accuracy: 91.6 -> 91.1 | No clear effect |
| remove visual input at test time (ablation) | MASON · CoDeLayout test | weighted accuracy: 91.6 -> 65.1 | Hurt |
| fine-tune MASON on full data versus 30% balanced data on Morphing | MASON · CoDeLayout Morphing | accuracy: 72.34 -> 70.21 | Hurt |
| post-train Qwen2.5-VL 7B with MASON versus zero-shot GPT-o3 | Qwen2.5-VL 7B · CoDeLayout Clipping | accuracy: 63.64 -> 90.91 | Improved |
Closest papers by meaning
- Seeing is Improving: Visual Feedback for Iterative Text Layout Refinement (CVPR 2026 · similarity 0.87)
- LayoutNUWA: Revealing the Hidden Layout Expertise of Large Language Models (ICLR 2024 · similarity 0.87)
- Visual Layout Composer: Image-Vector Dual Diffusion Model for Design Layout Generation (CVPR 2024 · similarity 0.87)
- AesthetiQ: Enhancing Graphic Layout Design via Aesthetic-Aware Preference Alignment of Multi-modal Large Language Models (CVPR 2025 · similarity 0.87)
- A Simple yet Effective Layout Token in Large Language Models for Document Understanding (CVPR 2025 · similarity 0.86)
- Read, Parse, Describe: Unified Document Parsing with Visual Element Description Generation (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
