Research map / Architectures and efficiency / Attention and transformers
RoSeViT: Role-Separated Vision Transformers for ARC Visual Reasoning
Zhiguang Liu, Yi Shang
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RoSeViT, a role-separated vision transformer for ARC visual reasoning that separates local visual state from global transformation logic.
Problem
ARC tasks require inferring a high-level transformation rule from a few demonstrations and applying it consistently across a grid; dense ViTs entangle local visual state and global transformation logic in shared tokens, which can hinder consistent reasoning.
Approach
Separates token representations by role (local visual state vs. global transformation logic) within a vision transformer rather than using a shared token representation for both.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace dense VARC-ViT backbone with one-step RoSeViT d-1 | VARC protocol · ARC-1 | pass@2 accuracy: 54.5 -> 56.6 | Improved |
| widen dense VARC-ViT to 28M parameters (parameter-matched control) (ablation) | VARC-ViT · ARC-1 | pass@2 accuracy: 54.5 -> 53.6 | Hurt |
| scale dense VARC-ViT to 66M (ablation) | VARC-ViT · ARC-1 | pass@2 accuracy: 54.5 -> 53.0 | Hurt |
| add recurrent refinement (c-2-TTT2) to role-separated backbone (ablation) | RoSeViT · ARC-1 | pass@2 accuracy: 56.6 -> 58.0 | Improved |
| staged training with four recurrent steps and TTT3 (c-d-4-TTT3) (ablation) | RoSeViT · ARC-1 | pass@2 accuracy: 58.0 -> 58.8 | Improved |
| use RoSeViT-Ensemble instead of VARC-Ensemble | full ARC system · ARC-1 | pass@2 accuracy: 60.4 -> 62.6 | Improved |
| stronger test-time adaptation (TTT1 to TTT3) on d-1 (ablation) | RoSeViT d-1 · ARC-1 vs ARC-2 | pass@2 accuracy: ARC-1 56.6 -> 55.5; ARC-2 6.9 -> 8.3 | Mixed |
| replace VARC-ViT with RoSeViT d-1 under default adaptation | VARC protocol · ARC-2 | pass@2 accuracy: 8.3 -> 6.9 | Hurt |
| use pure controller-bottleneck blocks (variants a/b) without dense visual pass (ablation) | RoSeViT · ARC offline training | training/validation curves | Hurt |
| add local 3x3 workspace neighborhood (d over c) (ablation) | RoSeViT · ARC offline training | training/validation curves | Improved |
| zero controller embeddings at inference after TTT (ablation) | RoSeViT d-1 · ARC-1 | pass@2 accuracy: 56.3 / 56.3 -> 43.3 / 43.0 (-13.1) | Hurt |
| zero task token of dense control after TTT (ablation) | Dense 28M · ARC-1 | pass@2 accuracy: 52.9 / 54.9 -> 50.3 / 52.5 (-2.5) | No clear effect |
Experimental setup
Baselines: dense Vision Transformers
Closest papers by meaning
- ARC Is a Vision Problem! (CVPR 2026 · similarity 0.88)
- Vision Transformers with Self-Distilled Registers (NeurIPS 2025 · similarity 0.87)
- Beyond the Doors of Perception: Vision Transformers Represent Relations Between Objects (NeurIPS 2024 · similarity 0.86)
- Transductive Visual Programming: Evolving Tool Libraries from Experience for Spatial Reasoning (ICLR 2026 · similarity 0.85)
- CodeDance: A Dynamic Tool-integrated MLLM for Executable Visual Reasoning (CVPR 2026 · similarity 0.85)
- AdaReasoner: Dynamic Tool Orchestration for Iterative Visual Reasoning (ICLR 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
