Research map / Vision-language and multimodal / Multimodal LLMs

JanusCoder: Towards a Foundational Visual-Programmatic Interface for Code Intelligence

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Interleaved image-text generation with MLLMs

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces a data toolkit and the 800K-sample JanusCode-800K multimodal code corpus, used to train JanusCoder (text-only, Qwen3 backbones) and JanusCoderV (VLM, Qwen2.5-VL/InternVL3.5 backbones) for visual-programmatic code generation, plus the DTVBench benchmark.

Problem

Multimodal code intelligence (code tied to visual outputs such as charts, web UIs, animations) is held back by scarce high-quality multimodal code data and by specialized models that do not generalize across tasks.

Approach

A data synthesis toolkit that combines guided evolution, re-contextualization, reverse instruction and bidirectional translation with execution validation (Python, Wolfram, Manim, Playwright) and VLM/LLM reward filtering, producing a unified corpus that trains one model across text-centric and vision-centric coding tasks.

Key results

• JanusCoder-14B reaches 9.7% incorrect code on PandasPlotBench (vs 9.7% for GPT-4o) with task score 86 vs 85 for GPT-4o.
• JanusCoderV-8B scores 69.5 overall on ChartMimic direct mimic vs GPT-4o at 60.9 (per the reported tables).
• JanusCoder-14B achieves 41.10 average on ArtifactsBench vs 37.97 for GPT-4o.

Surprising findings

• Removing the algorithm data improved ChartMimic (68.74 to 70.16) and InteractScience results, suggesting not all data subsets help every task.

Limitations

• Judge-based reward and evaluation rely on VLM/LLM judges, with possible circularity (partly addressed by cross-model adjudication).
• DTVBench is small (52 Manim and 50 Wolfram tasks) and the subjective faithfulness score uses college-student annotators.
• JanusCoderV-8B underperforms on some metrics such as PandasPlotBench visual scores and SVG artifacts.

Experimental setup

Datasets: JanusCode-800K VisCoder VisCodex CodeEvo 3Blue1Brown video dataset MMSVG TheStackV2 WebCode2M WebDev Arena Wolfram Demonstrations

Benchmarks: PandasPlotBench ArtifactsBench DTVBench ChartMimic DesignBench WebCode2M InteractScience BigCodeBench LiveCodeBench

Metrics: incorrect code rate visual score task score CLIP similarity TreeBLEU VLM-judge score pass rate CMS MLLM score

Baselines: GPT-4o Qwen2.5-Coder-14B-Instruct Qwen3-8B Qwen3-14B Qwen2.5-VL-7B-Instruct InternVL3.5-8B MiniCPM-V-2-6 Llama-3.2-11B-Vision-Instruct

Builds on: VisCoder LLaMA-Factory FlashAttention-2 DeepSpeed

Closest papers by meaning

• VisCodex: Unified Multimodal Code Generation via Merging Vision and Coding Models (ICLR 2026 · similarity 0.91)
• Thinking with Programming Vision: Towards a Unified View for Thinking with Images (CVPR 2026 · similarity 0.89)
• CodeMMR: Bridging Natural Language, Code, and Image for Unified Retrieval (CVPR 2026 · similarity 0.89)
• Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs (NeurIPS 2024 · similarity 0.88)
• Breaking the SFT Plateau: Multimodal Structured Reinforcement Learning for Chart-to-Code Generation (ICLR 2026 · similarity 0.87)
• VCG-Bench: Towards A Unified Visual-Centric Benchmark for Structured Generation and Editing (ICML 2026 · similarity 0.87)