Research map / Architectures and efficiency / Attention and transformers
Infusing Lattice Symmetry Priors in Attention Mechanisms for Sample-Efficient Abstract Geometric Reasoning
Mattia Atzeni, Mrinmaya Sachan, Andreas Loukas
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add lattice symmetry attention masks (LatFormer) vs. standard attention/Transformer | ARC tasks requiring geometric priors · ARC Rotate + Translate | fraction of tasks solved: 0.800 vs 0.000 for Transformer | Improved |
| add lattice symmetry attention masks (LatFormer) vs. standard attention/Transformer | ARC tasks requiring geometric priors · ARC Translate | fraction of tasks solved: 0.365 vs 0.038 | Improved |
| precomputed group-action data augmentation for Transformer | Transformer · ARC Rotate + Translate | fraction of tasks solved: 0.200 vs 0.000 | Improved |
| add lattice symmetry attention masks (LatFormer) vs DSL search | ARC tasks requiring geometric priors · ARC Translate | fraction of tasks solved: 0.365 vs 0.673 for DSL search | Hurt |
| add lattice symmetry attention masks (LatFormer) vs DSL search | ARC tasks requiring geometric priors · ARC Rotate + Translate | fraction of tasks solved: 0.800 vs 0.400 | Improved |
| add lattice symmetry attention masks (LatFormer) | LARC geometric-prior tasks · LARC Rotate + Translate | fraction of tasks solved: 1.00 vs 0.00 for LARC (IO + NL pseudo) | Improved |
| add natural language embedding (LatFormer + NL) (ablation) | LatFormer on LARC · LARC Reflect + Translate | fraction of tasks solved: 0.58 vs 0.50 | Improved |
| add lattice symmetry attention masks (LatFormer) | LARC geometric-prior tasks · LARC Translate | fraction of tasks solved: 0.33 vs 0.25 | Improved |
| add lattice symmetry attention masks (LatFormer) | LARC geometric-prior tasks · LARC Scale + Translate | fraction of tasks solved: 1.00 vs 1.00 | No clear effect |
| add noise to one-hot inputs (noise level w=0.2 to 0.6) (ablation) | LatFormer on synthetic tasks · synthetic noise robustness, translation | accuracy: 0.91 at w=0.2 to 0.62 at w=0.6 | Hurt |
| add mask smoothing (heat-diffusion smoothing of masks) (ablation) | LatFormer on synthetic tasks · synthetic geometric transformations | accuracy | Improved |
| add lattice mask experts vs plain attention on synthetic tasks | LatFormer · synthetic geometric transformations sample efficiency | sample efficiency (examples needed): claimed 2 orders of magnitude fewer data | Improved |
Closest papers by meaning
- Thinking with Geometry: Active Geometry Integration for Spatial Reasoning (ICML 2026 · similarity 0.86)
- Beyond 3D VQAs: Injecting 3D Spatial Priors into Vision-Language Models for Enhanced Geometric Reasoning (CVPR 2026 · similarity 0.86)
- Tropical Attention: Neural Algorithmic Reasoning for Combinatorial Algorithms (NeurIPS 2025 · similarity 0.86)
- GeoSense: Internalizing Geometric Necessity Perception for Multimodal Reasoning (ICML 2026 · similarity 0.86)
- GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training (ICLR 2025 · similarity 0.86)
- Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Views (CVPR 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
