AI Paper Atlas

Research map / Architectures and efficiency / Attention and transformers

Infusing Lattice Symmetry Priors in Attention Mechanisms for Sample-Efficient Abstract Geometric Reasoning

Mattia Atzeni, Mrinmaya Sachan, Andreas Loukas

ICML 2023 · Architectures and efficiency › Attention and transformers · approach: Long-context KV and memory attention · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add lattice symmetry attention masks (LatFormer) vs. standard attention/TransformerARC tasks requiring geometric priors · ARC Rotate + Translatefraction of tasks solved: 0.800 vs 0.000 for TransformerImproved
add lattice symmetry attention masks (LatFormer) vs. standard attention/TransformerARC tasks requiring geometric priors · ARC Translatefraction of tasks solved: 0.365 vs 0.038Improved
precomputed group-action data augmentation for TransformerTransformer · ARC Rotate + Translatefraction of tasks solved: 0.200 vs 0.000Improved
add lattice symmetry attention masks (LatFormer) vs DSL searchARC tasks requiring geometric priors · ARC Translatefraction of tasks solved: 0.365 vs 0.673 for DSL searchHurt
add lattice symmetry attention masks (LatFormer) vs DSL searchARC tasks requiring geometric priors · ARC Rotate + Translatefraction of tasks solved: 0.800 vs 0.400Improved
add lattice symmetry attention masks (LatFormer)LARC geometric-prior tasks · LARC Rotate + Translatefraction of tasks solved: 1.00 vs 0.00 for LARC (IO + NL pseudo)Improved
add natural language embedding (LatFormer + NL) (ablation)LatFormer on LARC · LARC Reflect + Translatefraction of tasks solved: 0.58 vs 0.50Improved
add lattice symmetry attention masks (LatFormer)LARC geometric-prior tasks · LARC Translatefraction of tasks solved: 0.33 vs 0.25Improved
add lattice symmetry attention masks (LatFormer)LARC geometric-prior tasks · LARC Scale + Translatefraction of tasks solved: 1.00 vs 1.00No clear effect
add noise to one-hot inputs (noise level w=0.2 to 0.6) (ablation)LatFormer on synthetic tasks · synthetic noise robustness, translationaccuracy: 0.91 at w=0.2 to 0.62 at w=0.6Hurt
add mask smoothing (heat-diffusion smoothing of masks) (ablation)LatFormer on synthetic tasks · synthetic geometric transformationsaccuracyImproved
add lattice mask experts vs plain attention on synthetic tasksLatFormer · synthetic geometric transformations sample efficiencysample efficiency (examples needed): claimed 2 orders of magnitude fewer dataImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy