Research map / Vision-language and multimodal / Multimodal LLMs

Hint2Gen: Bridging Understanding and Generation via Code-structured Hints

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes code-structured (SVG/HTML) visual hints that encode reasoning steps on the image, builds a ~120K hint dataset and the Reason2Gen benchmark, and trains Hint2Gen (FLUX.1 Kontext + hint conditioning) for reasoning-intensive image generation.

Problem

Unified and text-to-image generation models fail on reasoning-intensive generation tasks (mazes, tangrams, puzzles) even though VLMs/LLMs can solve them, because they lack a structured interface to render reasoning as precise visual output.

Approach

Use VLM/LLM-generated SVG/HTML overlays rasterized onto the image as explicit hint conditioning; train FLUX.1 Kontext to take the hint latent added to the noised input latent, with a pipeline that auto-generates and filters hints from natural-image edits and web-rendered puzzles.

Key results

• Hint2Gen achieves 31.04% overall accuracy on Reason2Gen, above Nano Banana Pro (17.96%) and Nano Banana Pro with GPT-5 reasoning text (22.77%).
• Adding a hint image without retraining raises Nano Banana Pro overall accuracy from 17.96% to 18.99% and GPT-Image from 11.33% to 11.67%.
• On GenEval, Ours* (hint input) reaches 0.91 overall, matching BAGEL* and above GPT-Image* (0.89).

Surprising findings

• Fine-tuning FLUX.1 Kontext on the same dataset with hint images removed gives no significant change, indicating the hints themselves drive gains.
• Using a weaker VLM (Qwen2.5-VL) as hint generator yields comparable results to Gemini-2.5 Pro, suggesting the structured SVG/HTML format matters more than generator capability.

Limitations

• Accuracy remains well below 100% on Reason2Gen, with many categories near zero for all models.
• Evaluation relies on GPT-5 as LLM judge, which is only moderately aligned with human judgments and may be unreliable.
• Hint generation depends on closed-source GPT-5 and Gemini for data construction.

Experimental setup

Datasets: natural-image editing pairs GPT-5-generated web-rendered puzzle data GenEval

Benchmarks: Reason2Gen GenEval

Metrics: accuracy elo pearson correlation with human elo

Baselines: GPT-Image Nano Banana Nano Banana Pro Seedream 4.0 Seedream 4.5 FLUX.1 Kontext Bagel Bagel-Think Qwen-Image Step1X-Edit OmniGen2 UniWorld-V2

Builds on: FLUX.1 Kontext MetaQuery BAGEL

Closest papers by meaning

• CodeDance: A Dynamic Tool-integrated MLLM for Executable Visual Reasoning (CVPR 2026 · similarity 0.89)
• GENOME: Generative Neuro-Symbolic Visual Reasoning by Growing and Reusing Modules (ICLR 2024 · similarity 0.89)
• GIR-Bench: Versatile Benchmark for Generating Images with Reasoning (ICLR 2026 · similarity 0.88)
• SLVR: Structured Latent Visual Reasoning via Human-like Reasoning Flows (NeurIPS 2026 · similarity 0.88)
• Multi-step Visual Reasoning with Visual Tokens Scaling and Verification (NeurIPS 2025 · similarity 0.88)
• MMFineReason: Closing the Multimodal Reasoning Gap via Open Data-Centric Methods (NeurIPS 2026 · similarity 0.88)