Research map / Vision-language and multimodal / Multimodal LLMs

Seeing is Improving: Visual Feedback for Iterative Text Layout Refinement

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Interleaved image-text generation with MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes VFLM, a Qwen2.5-VL-7B layout generator trained with SFT on teacher-distilled generation-render-reflection trajectories and then GRPO with a learned layout reward model plus OCR-based rewards, enabling iterative visual-feedback refinement of SVG text layouts.

Problem

Existing MLLM layout generators are code-only and open-loop: they emit layout code without seeing the rendered result, so readability and aesthetics are not ensured.

Approach

Close the loop: the model renders its SVG layout, reflects on the rendered image, and iteratively refines it, trained by SFT on multi-round trajectories and RL with only outcome-level rewards (layout reward model + OCR), which the authors find more stable than process rewards.

Key results

• On TextLayout, VFLM reaches Char-F 0.9376 and GPT-4o overall 8.59, versus 0.8672 Char-F for the best MLLM baseline Claude 3.7 (overall 8.31).
• Outcome-only reward RL improves over its step-1 output (Char-F 0.9071 to 0.9376, RM score 0.5415 to 0.6018), while process-reward RL improves less (RM 0.4936 to 0.5241) and degrades after 250 steps.
• Reward model reaches 97.4% pairwise accuracy on held-out preference data.

Surprising findings

• Complex process-oriented rewards led to reward hacking and performance degradation after ~250 steps, while a simple outcome-only reward kept improving.
• Adding visual feedback to GPT-4o and Qwen2.5-VL-7B without training yields only marginal gains, with some metrics declining.

Limitations

• VFLM has the highest inference latency compared with OpenCOLE and IGD.
• Reward model and evaluation rely on GPT-4o judgments and in-house data, which may limit reproducibility.
• Image editing baselines and some metrics are evaluated on a single in-house test split of 1K samples.

Experimental setup

Datasets: TextLayout (in-house ~200K samples) Crello DESIGNERINTENTION

Benchmarks: TextLayout Crello DESIGNERINTENTION

Metrics: Char-F (OCR) R_ali R_ove R_com RM score GPT-4o judge scores (text, harmony, quality, meaning, overal

Baselines: GPT-4o Claude 3.7 Doubao-Seed-1.6 Qwen2.5-VL-72B Qwen2.5-VL-7B GPT-4o(edit) Qwen-Image-Edit FLUX-Kontext OpenCOLE IGD

Builds on: GRPO REINFORCE++ Qwen2.5-VL RLHF

Closest papers by meaning

• AesthetiQ: Enhancing Graphic Layout Design via Aesthetic-Aware Preference Alignment of Multi-modal Large Language Models (CVPR 2025 · similarity 0.89)
• LayoutNUWA: Revealing the Hidden Layout Expertise of Large Language Models (ICLR 2024 · similarity 0.88)
• Rethinking The Training And Evaluation of Rich-Context Layout-to-Image Generation (NeurIPS 2024 · similarity 0.88)
• DesignDiffusion: High-Quality Text-to-Design Image Generation with Diffusion Models (CVPR 2025 · similarity 0.87)
• PosterO: Structuring Layout Trees to Enable Language Models in Generalized Content-Aware Layout Generation (CVPR 2025 · similarity 0.87)
• Towards Aligned Layout Generation via Diffusion Model with Aesthetic Constraints (ICLR 2024 · similarity 0.87)