Research map / Vision-language and multimodal / Multimodal LLMs
Deterministic Component Mining for Multi-Framework UI2Code Generation
Zixiong Yang, Linxiao Li, Jiaye Lin, Binrui Wu, Xiaoyu Kang, Jiechao Gao
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a method for generating UI code across multiple front-end frameworks (React, Vue, Angular) from UI images using deterministic component mining, rather than only single HTML/CSS output.
Problem
Existing MLLM-based UI2Code approaches focus mainly on single HTML/CSS output and do not systematically handle front-end frameworks such as React, Vue, and Angular; their outputs are often value-truncated or incomplete (abstract cut off).
Approach
Extends UI2Code to multiple front-end frameworks by mining deterministic UI components to structure generation; details unavailable since only the truncated abstract was provided.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DCM pipeline instead of end-to-end Gemini-2.5-Pro | DCM · WebCode2M-Short (React) | Visual score: 0.72 -> 0.79 | Improved |
| use DCM pipeline instead of end-to-end Gemini-2.5-Pro | DCM · WebCode2M-Short (Angular) | Visual score: 0.62 -> 0.68 | Improved |
| use DCM pipeline instead of Gemini-2.5-Pro | DCM · WebCode2M-Short (Vanilla HTML) | Visual score: 0.75 -> 0.82 | Improved |
| use DCM instead of Gemini-2.5-Pro | DCM · WebCode2M-Long (React) | compilation success rate: 0.74 -> 0.73 | No clear effect |
| use DCM instead of Gemini-2.5-Pro | DCM · WebCode2M-Long (React) | reusability rate: 0.68 -> 0.67 | No clear effect |
| use DCM instead of Gemini-2.5-Pro | DCM · WebCode2M-Long (Vanilla HTML) | CLIP: 0.80 -> 0.78 | Hurt |
| use DCM instead of Gemini-2.5-Pro | DCM · WebCode2M-Short (React) | compilation success rate: 0.88 -> 0.91 | Improved |
| use DCM instead of Qwen2.5-VL-7B | DCM · WebCode2M-Short (Angular) | compilation success rate: 0.54 -> 0.76 | Improved |
| use DCM instead of GPT-4o in blind human comparison | DCM · human pairwise preference | win rate: 48.0% vs 17.6% | Improved |
| target component frameworks instead of Vanilla HTML (baselines degrade) | GPT-4o · WebCode2M-Long | Visual score: 0.65 -> 0.52 | Hurt |
Closest papers by meaning
- LayoutNUWA: Revealing the Hidden Layout Expertise of Large Language Models (ICLR 2024 · similarity 0.85)
- Widget2Code: From Visual Widgets to UI Code via Multimodal LLMs (CVPR 2026 · similarity 0.85)
- JanusCoder: Towards a Foundational Visual-Programmatic Interface for Code Intelligence (ICLR 2026 · similarity 0.83)
- CodeMMR: Bridging Natural Language, Code, and Image for Unified Retrieval (CVPR 2026 · similarity 0.83)
- UI2Code^N: UI-to-Code Generation as Interactive Visual Optimization (ICML 2026 · similarity 0.82)
- Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
