AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

Figma2Code: Automating Multimodal Design to Code in the Wild

Yi Gui, Jiawan Zhang, Yina Wang, Tianran Ma, Yao Wan, Shilin He, Dongping Chen, Zhou Zhao, Wenbin Jiang, Xuanhua Shi, Hai Jin, Philip S. Yu

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Interleaved image-text generation with MLLMs · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Figma2Code, a multimodal design-to-code task and 213-sample benchmark built from Figma community files (image + JSON metadata + assets), benchmarking ten MLLMs and an agentic workflow on visual fidelity, responsiveness, and maintainability.

Problem

Image-only design-to-code forces MLLMs to infer UI details such as icons, hierarchy, styles, and layout from screenshots, and ignores the metadata and assets present in real Figma design files. Existing benchmarks lack this multimodal information and code-quality evaluation.

Approach

Pairs Figma design images with cleaned JSON metadata and assets collected from the Figma community, and evaluates generated Tailwind code on visual fidelity (VES, MAE) plus reference-free responsiveness (RUR, APR) and maintainability (STR, AVU) metrics, with an F2CAgent ReAct-style critic/refiner loop.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add Figma metadata to image inputDirect Prompting ERNIE 4.5 424B VL · Figma2CodeVES: 0.5653 -> 0.6801Improved
add design image to metadata inputDirect Prompting ERNIE 4.5 424B VL · Figma2CodeVES: 0.6801 -> 0.6923Improved
add F2CAgent workflow on image + metadataERNIE 4.5 424B VL · Figma2CodeVES: 0.6923 -> 0.7990Improved
add F2CAgent workflow on image + metadataERNIE 4.5 424B VL · Figma2CodeAPR (%): 2.83 -> 13.57Hurt
add metadata to image inputDirect Prompting ERNIE 4.5 424B VL · Figma2CodeAPR (%): 0.01 -> 5.01Hurt
use Template Conversion instead of MLLM prompting on metadataFigmaToCode plugin · Figma2CodeAPR (%): 5.01 -> 58.0Hurt
add text-augmentation to image-only promptingDesign2Code ERNIE 4.5 424B VL · Figma2CodeVES: 0.5653 -> 0.5683No clear effect
add style attributes to metadata (ablation)Direct Prompting ERNIE 4.5 424B VL · Figma2CodeVES: below 0.55 -> approximately 0.65Improved
add image assets to metadata (ablation)Direct Prompting ERNIE 4.5 424B VL · Figma2CodeVESImproved
add geometry and hierarchy to metadata (ablation)Direct Prompting ERNIE 4.5 424B VL · Figma2CodeRURImproved
add style or geometry to metadata (ablation)Direct Prompting ERNIE 4.5 424B VL · Figma2CodeSTR / AVUHurt
add image to metadata-only input in ablationsDirect Prompting ERNIE 4.5 424B VL · Figma2CodeVES / APRImproved

Surprising findings

Limitations

Experimental setup

Datasets: Figma community design files (crawled)

Benchmarks: Figma2Code

Metrics: VES MAE RUR APR STR AVU

Baselines: Design2Code direct prompting Text-Augmented prompting FigmaToCode template conversion Llama 4 Maverick Llama 4 Scout Qwen 2.5 VL ERNIE 4.5 424B VL Nova Pro v1 Gemini 2.5 Pro Grok4 Claude Opus 4.1 GPT-4o

Builds on: Design2Code ReAct FigmaToCode DINOv2

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy