Research map / Vision-language and multimodal / Human and embodied modeling
Slot Abstractors: Toward Scalable Abstract Visual Reasoning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Slot Abstractors, combining Slot Attention object-centric encoding with Abstractor relational cross-attention to scale abstract visual reasoning to many-object, multi-relation problems with O(N^2) complexity.
Problem
Abstract visual reasoning from multi-object images requires systematic generalization of rules; prior relational models (OCRA) are limited to a single relation and scale as O(N^4), failing on problems with many objects.
Approach
Replace OCRA's pairwise relational embeddings with Abstractor layers (relational cross-attention with keys/queries from slot feature embeddings and values from position embeddings, plus multi-head self-attention) operating on Slot Attention slots, giving quadratic complexity and multi-relation modeling via multiple heads.
Key results
- On ART same/different, relational-match-to-sample, distribution-of-3, and identity rules, Slot Abstractor reaches 96.36, 91.64, 95.22, 96.41% test accuracy, beating OCRA by up to ~9 points.
- On PGM, Slot Abstractor averages about 5% higher across OOD regimes than prior SOTA, reaching 39.3% on extrapolation vs 29.0% for ARII.
- On V-PROM, Slot Abstractor achieves 67.7% test accuracy vs 51.2% for Relation Net.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace OCRA with Slot-Abstractor | ART · ART tasks | accuracy: average improvement of 3% over all tasks | Helped |
| replace OCRA with Slot-Abstractor | CLEVR-ART · CLEVR-ART tasks | accuracy: by as much as 14% | Helped |
| replace PredRNet with Slot-Abstractor | PGM · PGM OOD regimes | accuracy: as much as 21% | Helped |
| use Slot-Abstractor instead of Relation Net | V-PROM Neutral · V-PROM Neutral | accuracy: 51.2% -> 67.7% | Helped |
| train on 20% rather than 100% of PGM training data | Slot-Abstractor · PGM Neutral | accuracy: 91.5 -> 71.25 | Hurt |
| add Slot Attention instead of using grid locations as objects (ablation) | Slot-Abstractor · ART Relational-match-to-sample | accuracy: 49.85 -> 91.64 | Helped |
| add self-attention and the subsequent feedforward layer (ablation) | Slot-Abstractor · ART Relational-match-to-sample | accuracy: 85.41 -> 91.64 | Helped |
| use factorized feature and position representations (ablation) | Slot-Abstractor · ART Relational-match-to-sample | accuracy: 50.02 -> 91.64 | Helped |
| use relational cross-attention instead of standard cross-attention (ablation) | Slot-Abstractor · ART tasks | accuracy: null | Helped |
| add Slot Attention and use DINO patch/object slots instead of CLS features (ablation) | Slot-Abstractor on V-PROM · V-PROM Neutral | accuracy: 30.01% -> 67.7% | Helped |
Surprising findings
- OCRA could not be trained on PGM even with batch size 1 on an 80GB A100 due to memory, while Slot Abstractor could.
- On PGM, Slot Abstractor trained on 20% of data gets 71.25% vs STSN's 53.35%, but STSN is higher (98.2 vs 91.5) at 100% data in the neutral regime.
Limitations
- Number of slots is fixed, which is problematic when object counts vary widely.
- Performance on PGM extrapolation and some OOD regimes remains well below human-level.
- Quadratic complexity in number of slots still limits scaling.
Experimental setup
Datasets: ART SVRT CLEVR-ART PGM V-PROM
Benchmarks: ART SVRT CLEVR-ART PGM V-PROM
Metrics: test accuracy
Baselines: OCRA GAMR Slot-Transformer STSN PredRNet ARII Slot-RN ResNet Attn-ResNet MRNet
Builds on: Slot Attention Abstractors OCRA DINOSAUR Transformer
Closest papers by meaning
- ARC Is a Vision Problem! (CVPR 2026 · similarity 0.88)
- Beyond Task-Specific Reasoning: A Unified Conditional Generative Framework for Abstract Visual Reasoning (ICML 2025 · similarity 0.88)
- VisuRiddles: Fine-grained Perception is a Primary Bottleneck for Multimodal Large Language Models in Abstract Visual Reasoning (ICLR 2026 · similarity 0.87)
- MARVEL: Multidimensional Abstraction and Reasoning through Visual Evaluation and Learning (NeurIPS 2024 · similarity 0.87)
- Beyond the Doors of Perception: Vision Transformers Represent Relations Between Objects (NeurIPS 2024 · similarity 0.87)
- ATLAS: Agentic or Latent Visual Reasoning? One Word is Enough for Both (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.