Research map / Vision-language and multimodal / Visual question answering

ViUniT: Visual Unit Tests for More Robust Visual Programming

CVPR 2025 · Vision-language and multimodal › Visual question answering · approach: Visual reasoning diagnostics · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ViUniT, which generates visual unit tests (image-caption plus expected-answer pairs rendered by diffusion models) to verify visual programs, and uses them for best-program selection, answer refusal, re-prompting, and unsupervised RL rewards.

Problem

Visual programs often produce correct answers for the wrong reasons (only 33% of correct programs on GQA were actually correct), and existing visual unit tests only check return types rather than logical correctness.

Approach

Generate unit tests as image descriptions and expected answers using an LLM, select a coverage-maximizing subset (by answer, then by caption embedding diversity), render them to images with text-to-image diffusion, and score candidate programs by how many tests they pass.

Key results

• Best program selection with unit tests improves average accuracy by 11.4 points over the base setup across GQA, Winoground and SugarCREPE.
• CodeLlama-7B with ViUniT selection averages 48.67 vs 41.92 for gpt-4o-mini base setup, a 7.7-point average gain over gpt-4o-mini.
• Unit-test-based RL reward outperforms supervised correctness reward by an average of 1.3 points.

Measured effects reported in the paper

Surprising findings

• Including program implementation in unit test generation helps little at low unit-test counts and hurts as counts grow; query-only generation was more effective.
• Human inspection found 65% of right-for-wrong-reason programs in ITM were unreliable from the start.

Limitations

• Human evaluation found 25% of generated unit tests incorrect, most often due to missing objects.
• Results are reported on small subsets (502 GQA validation examples, 1600 Winoground), and the method adds cost from image generation per query.
• Limited to VQA and image-text matching tasks.

Experimental setup

Datasets: GQA SugarCREPE Winoground

Benchmarks: GQA Winoground SugarCREPE

Metrics: accuracy F1 score for answer refusal human-judged program correctness

Baselines: gpt-4o-mini base setup most common answer selection error re-prompting supervised correctness reward reverting on error

Builds on: ViperGPT Khan et al. visual program RL property-based visual unit testing LM Guided Diffusion

Closest papers by meaning

• CodeV: Code with Images for Faithful Visual Reasoning via Tool-Aware Policy Optimization (CVPR 2026 · similarity 0.88)
• Self-Training Large Language Models for Improved Visual Program Synthesis With Visual Reinforcement (CVPR 2024 · similarity 0.87)
• ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models (NeurIPS 2025 · similarity 0.87)
• When Open-Vocabulary Visual Question Answering Meets Causal Adapter: Benchmark and Approach (AAAI 2025 · similarity 0.87)
• Not Another Text Benchmark: Putting the “Visual" Back in Visual Question Answering for Large Video Models (NeurIPS 2026 · similarity 0.87)
• Look as You Think: Unifying Reasoning and Visual Evidence Attribution for Verifiable Document RAG via Reinforcement Learning (AAAI 2026 · similarity 0.87)