Research map / Image generation / Text-to-image diffusion
A Matched-Budget Audit Framework for Recaptioned Image-Text Supervision Distributions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a matched-budget, five-axis audit framework for recaptioned image-text supervision distributions, using claim-level CBUs judged against images, and applies it to seven paired comparisons across five public source corpora while releasing a ~490M-caption recap corpus.
Problem
Recaptioned image-text corpora used for T2I training are hard to audit at corpus scale: length proxies miss caption-register artifacts, and downstream T2I benchmarks entangle captioning data with training choices.
Approach
Treat each recaptioned corpus as an audit target D_{pi,V_c,C} and audit it at a fixed text budget B=64 with five axes (text budget, prompt-pool support, claimed density, surface concentration, image-conditioned support/risk), using controllable basic units (CBUs) caption-only extracted and re-asked against the paired image by two judges.
Key results
- Across four cross-corpus pairs, the released surface raises supported CBU per caption by +3.39 to +6.36 under both Qwen and Gemma judges, with risk dropping by 0.05 to 0.16.
- On CC12M, Ours reaches 14.60 supported CBU/cap (Qwen judge) with risk 0.030, versus 9.84 and 0.069 for CC12M-LLaVA-NeXT.
- Human verification finds 0 explicit-no judgments on Ours versus 15.4% on PixelProse; Qwen and Gemma judges agree with humans at 84.8% and 84.2%.
Surprising findings
- Against the same captioner, a naive detailed-caption prompt produces a surface with 1.8x longer captions and higher concentration, indicating policy rather than captioner drives the gap.
- A short tag-style reference (CC12M-Qwen3-VL) has the highest CBU per 100 lex and lowest risk, so density per token and total yield diverge (a long-vs-dense frontier).
Limitations
- Risk aggregates captioner, extractor, and judge errors; the extractor stays on the Qwen checkpoint in both Judge settings.
- Human verification covers only 137 sampled CC12M claims; count and text-rendering claims show lowest judge agreement.
- Prompt pools cover natural-language T2I prompts; most captions exceed the 77-token CLIP context, so image-output comparisons confound encoder and augmentation.
Experimental setup
Datasets: DataComp CC12M LAION-pop PD12M Danbooru CommonCatalog LAION-Aesthetics LAION-HighRes-Aesthetic Megalith-CC0
Benchmarks: DataComp recaptioned pairs (cross-corpus) CC12M long-vs-dense frontier
Metrics: claimed CBU per caption supported CBU per caption unsupported risk prompt-mass support n-gram JSD top-100 prefix mass distinct-3
Baselines: Recap-DataComp CC12M-LLaVA-NeXT PixelProse CC12M-Qwen3-VL LAION-pop-Llama PD12M released Danbooru-Florence
Builds on: DCScore Recap-DataComp Brack et al. ablations
Closest papers by meaning
- Painting with Words: Elevating Detailed Image Captioning with Benchmark and Alignment Learning (ICLR 2025 · similarity 0.86)
- A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captions (CVPR 2024 · similarity 0.86)
- What If We Recaption Billions of Web Images with LLaMA-3? (ICML 2025 · similarity 0.86)
- CapsFusion: Rethinking Image-Text Data at Scale (CVPR 2024 · similarity 0.85)
- BalCapRL : A Balanced Framework for RL-Based MLLM Image Captioning (NeurIPS 2026 · similarity 0.85)
- CAPability: A Comprehensive Visual Caption Benchmark for Evaluating Both Correctness and Thoroughness (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.