Research map / Vision-language and multimodal / Human and embodied modeling
Beyond Task-Specific Reasoning: A Unified Conditional Generative Framework for Abstract Visual Reasoning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UCGS, a unified conditional generative framework that reformulates multiple abstract visual reasoning tasks as predictability estimation on panel images, instantiated as a Transformer-based VQ model (UCGS-T) trained once on RAVEN and PGM and evaluated zero-shot on O3, VAP and SVRT.
Problem
Existing deep abstract visual reasoning (AVR) solvers use task-specific architectures or parameters, so each new AVR task typically requires retraining or retuning, raising cost.
Approach
Proves that RPM, VAP, odd-one-out and SVRT can all be solved by estimating the predictability p(x | context panel images) of a conditional generative model, with training-free judgment functions turning the shared estimator into task-specific answers; the estimator is a VQ-based Transformer that predicts discrete patch codes of the target image from context concepts.
Key results
- UCGS-T reaches 64.6% selection accuracy on RAVEN and 38.1% on PGM, above the task-specific solvers RAISE (54.5%, 14.0%) and GCA (37.3%, 31.7%) under the same training setup.
- UCGS-T achieves zero-shot accuracy of 33.6% on O3-ID, 35.8% on VAP-ID and 84.6% on SVRT-ID, with 52.8% on OOD SVRT versus 50.0% random guess.
- On the GPT-4o subsets, UCGS-T scores 60.9% on RAVEN versus GPT-4o's 12.6%, while GPT-4o is better on OOD G1-set (52.9% vs 30.4%) and SVRT (64.2% vs 52.8%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the unified conditional generative solver rather than TF + Patch | AVR reasoning · RAVEN | selection accuracy: 21.0 -> 64.6 | Helped |
| use the unified conditional generative solver rather than TF + Patch | AVR reasoning · PGM | selection accuracy: 15.6 -> 38.1 | Helped |
| use the unified conditional generative solver rather than TF + Patch | AVR reasoning · O3-ID | selection accuracy: 35.8 -> 33.6 | Hurt |
| use the unified conditional generative solver rather than TF + Patch | AVR reasoning · VAP-ID | selection accuracy: 15.5 -> 35.8 | Helped |
| use the unified conditional generative solver rather than TF + Patch | AVR reasoning · SVRT-ID | selection accuracy: 71.8 -> 84.6 | Helped |
| use the unified conditional generative solver rather than TF + OCL | AVR reasoning · G1-set | selection accuracy: 33.7 -> 30.4 | Hurt |
| use the unified conditional generative solver rather than ANP + Patch | AVR reasoning · VAP | selection accuracy: 30.9 -> 28.8 | Hurt |
| use the unified conditional generative solver rather than TF + Patch | AVR reasoning · SVRT | selection accuracy: 51.4 -> 52.8 | Helped |
| use the patch backbone with ANP solver rather than monolithic backbone with ANP solver (ablation) | ANP conditional generative solver · RAVEN | selection accuracy: 6.0 -> 10.5 | Helped |
| use the patch backbone with Transformer solver rather than monolithic backbone with Transformer solver (ablation) | Transformer conditional generative solver · RAVEN | selection accuracy: 11.2 -> 21.0 | Helped |
| replace the patch backbone with the object-centric backbone under the Transformer solver (ablation) | Transformer conditional generative solver · RAVEN | selection accuracy: 21.0 -> 7.4 | Hurt |
| replace the patch backbone with the monolithic backbone under the Transformer solver (ablation) | Transformer conditional generative solver · RAVEN | selection accuracy: 21.0 -> 11.2 | Hurt |
Surprising findings
- Zero-shot OOD performance on G1-set and VAP is only around 30%, just 5-8 points above random guessing, so generalization to unseen rules is limited.
- GPT-4o beats UCGS-T on the OOD zero-shot tasks G1-set and SVRT, while UCGS-T wins on in-distribution tasks.
Limitations
- Accuracy remains below state-of-the-art selective RPM solvers.
- Not evaluated on real-world scenes, which may need more powerful image tokenizers and decoders.
- Zero-shot performance on OOD tasks is weak (around 30% on G1-set and VAP).
Experimental setup
Datasets: RAVEN PGM
Benchmarks: RAVEN PGM O3-ID VAP-ID SVRT-ID G1-set VAP SVRT
Metrics: selection accuracy
Baselines: ANP + Mono ANP + OCL ANP + Patch TF + Mono TF + OCL TF + Patch GPT-4o PrAE NVSA GCA ALANS RAISE
Builds on: VQVAE Attentive Neural Processes STSN RAVEN
Closest papers by meaning
- One Self-Configurable Model to Solve Many Abstract Visual Reasoning Problems (AAAI 2024 · similarity 0.91)
- Slot Abstractors: Toward Scalable Abstract Visual Reasoning (ICML 2024 · similarity 0.88)
- VisuRiddles: Fine-grained Perception is a Primary Bottleneck for Multimodal Large Language Models in Abstract Visual Reasoning (ICLR 2026 · similarity 0.87)
- GenVP: Generating Visual Puzzles with Contrastive Hierarchical VAEs (ICLR 2025 · similarity 0.87)
- VisRes Bench: On Evaluating the Visual Reasoning Capabilities of VLMs (CVPR 2026 · similarity 0.87)
- StemBind: When MLLMs Get Lost Between Rules and Instances in Abstract Visual Reasoning (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.