Research map / Vision-language and multimodal / Multimodal LLMs

Learning from Reasoning Failures via Synthetic Data Generation

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Interleaved image-text generation with MLLMs · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Generates a 553k-example multimodal instruction-tuning dataset by having GPT-4o diagnose LLaVA-1.5-7B's reasoning failures and propose targeted synthetic QA pairs (real or FLUX-generated images), then filtering with GPT-4o as judge.

Problem

Synthetic multimodal training data for LMMs is typically generated broadly without regard to the target model's specific reasoning weaknesses, and often depends on real images, making it inefficient and hard to apply in low-resource domains.

Approach

Use a frontier LMM to analyze a weaker LMM's failures on benchmark training samples (where the weak model is wrong and the frontier model is right), then have it propose new image descriptions, questions and answers targeting those failure modes; images are either real (reused) or generated by a text-to-image diffusion model, and an LMM judge filters the output.

Key results

• On OK-VQA, LLaVA trained with 124,922 synthetic samples reaches 61.5 EM vs 54.3 for LLaVA_real trained on equivalent real OK-VQA data and 57.0 for the baseline.
• With 125k synthetic samples, Vicuna-7B LLaVA scores 33.2 on InfoVQA, 61.1 on OK-VQA and 73.0 on ScienceQA, exceeding ALLaVA (28.8/49.4/66.5) and CoSyn (29.6/57.7/70.9).
• Continued finetuning of Qwen2.5-VL-7B-Instruct on the 124K synthetic set raises ScienceQA from 67.9 to 85.5 and OK-VQA from 42.4 to 53.0.

Surprising findings

• Training on the synthetic data for only ~1,000 failures (e.g., 607 OK-VQA failures generating 9,009 samples) yielded larger gains than training on the equivalent amount of real OK-VQA data.
• Substituting up to 25% of the original LLaVA data with purely synthetic-image data matched or beat the baseline on many tasks.

Limitations

• Relies on a frontier model (GPT-4o) for failure analysis and generation, so cost and access depend on proprietary APIs.
• Filtering rejects a large share of synthetic images (81% for VizWiz), and some samples remain ambiguous or incorrect.
• Gains on some benchmarks (e.g., Qwen2.5-VL InfoVQA) are flat, and a few OOD scores drop slightly (e.g., MMBench for some settings).

Experimental setup

Datasets: VizWiz InfoVQA ScienceQA OK-VQA LLaVA-Instruct (665K mix) LLaVA 558k pretraining

Benchmarks: InfoVQA ScienceQA OK-VQA TextVQA OCR-Bench VQAv2 MMBench MMMU HallusionBench LLaVA-Bench-COCO POPE Math-Vision

Metrics: exact match accuracy

Baselines: ALLaVA CoSyn-400K SimVQA Img-Diff LLaVA_real

Builds on: LLaVA-1.5 AgentInstruct DISCERN

Closest papers by meaning

• Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Sampling (ICLR 2025 · similarity 0.88)
• ORACLE: Optimizing Reasoning Abilities of Large Language Models via Constraint-Led Synthetic Data Elicitation (AAAI 2026 · similarity 0.88)
• Learning from Synthetic Data Improves Multi-hop Reasoning (ICLR 2026 · similarity 0.88)
• LOKI: A Comprehensive Synthetic Data Detection Benchmark using Large Multimodal Models (ICLR 2025 · similarity 0.87)
• It's All Training: A Fully Synthetic Single-Stage Recipe for LLMs (NeurIPS 2026 · similarity 0.87)
• GGBench: A Geometric Generative Reasoning Benchmark for Unified Multimodal Models (CVPR 2026 · similarity 0.87)