Research map / Vision-language and multimodal / Human and embodied modeling
Abstract visual reasoning benchmarks
Benchmarks few-shot and systematic visual reasoning using object-centric abstraction and analogical learning.
63 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2017–2026) · an approach in Human and embodied modeling, keyword cluster “computer vision · humans · abstract”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Still moving The best reported result is still improving on 2 of 2 tracked benchmarks.
21 papers in 2023–24 and 17 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 3.9% to 2.5%.
Open questions the data points to
- I-RAVEN (accuracy): best went from 97.62 (up to 2024) to 99.4 in 2025 — DARR: A Dual-Branch Arithmetic Regression Reasoning Framework for Solving Machine Number Reasoning (AAAI 2025 · 3 citations)
- RAVEN-FAIR (accuracy): best went from 97.58 (up to 2024) to 99.2 in 2025 — DARR: A Dual-Branch Arithmetic Regression Reasoning Framework for Solving Machine Number Reasoning (AAAI 2025 · 3 citations)
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2017 · Learning to See Physics via Visual De-animation (NeurIPS 2017 · 117 citations)
- 2018 · Who Let the Dogs Out? Modeling Dog Behavior From Visual Data (CVPR 2018 · 55 citations)
- 2019 · Interpretable and Fine-Grained Visual Explanations for Convolutional Neural Networks (CVPR 2019 · 147 citations)
- 2020 · Machine Number Sense: A Dataset of Visual Arithmetic Problems for Abstract and Relational Reasoning (AAAI 2020 · 23 citations)
- 2021 · Stratified Rule-Aware Network for Abstract Visual Reasoning (AAAI 2021 · 69 citations)
- 2022 · Visual Abductive Reasoning (CVPR 2022 · 54 citations)
- 2023 · Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence? (NeurIPS 2023 · 21 citations)
- 2024 · InstructDiffusion: A Generalist Modeling Interface for Vision Tasks (CVPR 2024 · 63 citations)
- 2025 · LogicAD: Explainable Anomaly Detection via VLM-based Text Feature Extraction (AAAI 2025 · 19 citations)
- 2026 · Can Vision Language Models Learn Intuitive Physics from Interaction? (ICML 2026 · 0 citations)
Earliest papers here
- Learning to See Physics via Visual De-animation (NeurIPS 2017 · 117 citations)
- Sherlock: Scalable Fact Learning in Images (AAAI 2017 · 24 citations)
- Who Let the Dogs Out? Modeling Dog Behavior From Visual Data (CVPR 2018 · 55 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
I-RAVEN · accuracy (higher is better)
5 papers, 2021–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2021 | 60.8 | Stratified Rule-Aware Network for Abstract Visual Reasoning (AAAI 2021 · 69 citations) |
| 2023 | 96.5 | Neural Prediction Errors enable Analogical Visual Reasoning in Human Standard Intelligence Tests (ICML 2023) |
| 2024 | 97.62 | Learning Visual Abstract Reasoning through Dual-Stream Networks (AAAI 2024 · 9 citations) |
| 2025 | 99.4 | DARR: A Dual-Branch Arithmetic Regression Reasoning Framework for Solving Machine Number Reasoning (AAAI 2025 · 3 citations) |
RAVEN-FAIR · accuracy (higher is better)
4 papers, 2023–2025 · still improving
| Year | Best so far | Set by |
|---|---|---|
| 2023 | 97.1 | Neural Prediction Errors enable Analogical Visual Reasoning in Human Standard Intelligence Tests (ICML 2023) |
| 2024 | 97.58 | Learning Visual Abstract Reasoning through Dual-Stream Networks (AAAI 2024 · 9 citations) |
| 2025 | 99.2 | DARR: A Dual-Branch Arithmetic Regression Reasoning Framework for Solving Machine Number Reasoning (AAAI 2025 · 3 citations) |
What has been tried
53 papers here report 558 measured changes (a component added, removed or swapped, compared with a baseline): helped 70%, no clear effect 4%, hurt 22%, mixed 4%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Use highly detailed instructions instead of coarse task labels and fixed colors | HumanArt · average precision | 7.0 -> 51.4 | Helped | InstructDiffusion: A Generalist Modeling Interface for Vision Tasks (CVPR 2024 · 63 citations) |
| Add the Adaptive Layer | DeepCAD test, expert prompts (L3) · Arc F1 | 27.18 -> 36.03 | Helped | Text2CAD: Generating Sequential CAD Designs from Beginner-to-Expert Level Text Prompts (NeurIPS 2024 · 27 citations) |
| Add Guided CoT | MVTec LOCO AD · AUROC | 23.4 -> 48.5 | Helped | LogicAD: Explainable Anomaly Detection via VLM-based Text Feature Extraction (AAAI 2025 · 19 citations) |
| Increase ViT depth from 12 to 16 | PGM Held-Out Attribute Pairs · accuracy | 93.7% to 89.7% | Hurt | Learning Visual Abstract Reasoning through Dual-Stream Networks (AAAI 2024 · 9 citations) |
| Use SAL instead of RN in SCAR | G-set · test accuracy (%) | 36.5 -> 82.1 | Helped | One Self-Configurable Model to Solve Many Abstract Visual Reasoning Problems (AAAI 2024 · 6 citations) |
| Use four generated images per view rather than one | CADTalk-Cube, average · Bacc | 79.70 -> 88.75 | Helped | CADTalk: An Algorithm and Benchmark for Semantic Commenting of CAD Programs (CVPR 2024 · 4 citations) |
| Add IIARR to the baseline | MNR · accuracy | 48.7 -> 53.4 | Helped | DARR: A Dual-Branch Arithmetic Regression Reasoning Framework for Solving Machine Number Reasoning (AAAI 2025 · 3 citations) |
| Add the aggregation network to combine multi-scale UNet features | Sketch-photo correspondence, sketch-based segmentation, ZS-SBIR, and sketch recognition · PCK@5, mIoU, mAP@200, and Acc.@1 | 54.65 vs 70.31 PCK@5; 47.52 vs 60.12 mIoU; 0.587 vs 0.761 mAP@200; 66.71 vs 87.02 Acc.@1 | Helped | SketchFusion: Learning Universal Sketch Features through Fusing Foundation Models (CVPR 2025 · 2 citations) |
| Fuse ColorNet with ShapeTexNet | CUB · top-1 accuracy | 80.4 -> 81.8 | Helped | LucidPPN: Unambiguous Prototypical Parts Network for User-centric Interpretable Computer Vision (ICLR 2025 · 1 citation) |
| use the patch backbone with ANP solver rather than monolithic backbone with ANP solver | RAVEN · selection accuracy | 6.0 -> 10.5 | Helped | Beyond Task-Specific Reasoning: A Unified Conditional Generative Framework for Abstract Visual Reasoning (ICML 2025 · 0 citations) |
| use local rather than local-global edit ordering | BDD100K · success rate | 84.8 -> 88.9 | Helped | V-CECE: Visual Counterfactual Explanations via Conceptual Edits (NeurIPS 2025 · 0 citations) |
| Add auxiliary mask loss to single-word prompt optimization | SalientImageNet, single-word prompts · mean activation score | 23.73 ± 32.91 -> 43.93 ± 38.68 | Helped | DEXTER: Diffusion-Guided EXplanations with TExtual Reasoning for Vision Models (NeurIPS 2025 · 0 citations) |
What reviewers will expect
Datasets and baselines named in the 11 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- RAVEN · 3 of 11 papers
- I-RAVEN · 2 of 11 papers
- RAVEN-FAIR · 2 of 11 papers
- PGM · 2 of 11 papers
Baselines
- PrAE · 2 of 11 papers
- GCA · 2 of 11 papers
- ALANS · 2 of 11 papers
- RAISE · 2 of 11 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 38 papers; from OpenAlex and the papers' affiliations).
- Princeton University · 4 papers
- Nanyang Technological University · 3 papers
- University of Nottingham Ningbo China · 3 papers
- Beijing Normal University · 3 papers
- University of California, Los Angeles · 3 papers
- Chinese Academy of Sciences · 2 papers
- Microsoft Research Asia · 2 papers
- University of Edinburgh · 2 papers
- Microsoft · 2 papers
- University of California, Berkeley · 2 papers
Use it
Download BibTeX of 38 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Multimodal and affective interaction modeling (87 papers)
- Human-like visual cognition in VLMs (97 papers)
- Human perception modeling (71 papers)
- Scientific domain vision datasets (47 papers)
Elsewhere in the topic
- Vision-language pretraining (182 papers)
- Medical vision-language models (130 papers)
- Scene graph generation (126 papers)
- Language-grounded scene text and structure (124 papers)
- fMRI visual decoding (106 papers)
- Multimodal retrieval-augmented modeling (85 papers)
- Language-grounded visual concept learning (83 papers)
- Composed and text-based image retrieval (78 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).