Research map / Label-efficient and robust learning / Recognition and long-tail
Neural Attentive Circuits
Martin Weiss, Nasim Rahaman, Francesco Locatello, Christopher Pal, Yoshua Bengio, Bernhard Schölkopf, Li Erran Li, Nicolas Ballas
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add ModFC layers to Perceiver IO (ablation) | Perceiver IO · Tiny-ImageNet validation | Acc@1: 58.89 -> 58.91 | No clear effect |
| Add ModFC layers to Perceiver IO (ablation) | Perceiver IO · Tiny-ImageNet-R | Acc@1: 17.57 -> 18.01 | No clear effect |
| Add a sparse graph sampled from a frozen prior (ablation) | Perceiver IO + ModFC · Tiny-ImageNet validation | Acc@1: 58.91 -> 58.31 | Hurt |
| Add a sparse graph sampled from a frozen prior (ablation) | Perceiver IO + ModFC · Tiny-ImageNet-R | Acc@1: 18.01 -> 18.50 | Improved |
| Learn the connectivity graph and module codes jointly (ablation) | Perceiver IO + ModFC + sparse graph · Tiny-ImageNet validation | Acc@1: 58.31 -> 60.76 | Improved |
| Learn the connectivity graph and module codes jointly (ablation) | Perceiver IO + ModFC + sparse graph · Tiny-ImageNet-R | Acc@1: 18.50 -> 19.52 | Improved |
| Use a scale-free rather than ring-of-cliques graph prior (ablation) | NAC · Tiny-ImageNet validation | Acc@1: 60.54 -> 60.76 | Improved |
| Use a ring-of-cliques rather than scale-free graph prior (ablation) | NAC · Tiny-ImageNet-R | Acc@1: 19.52 -> 20.03 | Improved |
| Use NAC rather than Perceiver IO | Point-cloud classification · Point-cloud classification test | test accuracy: 77.4% -> 83.0% | Improved |
| Use NAC rather than Perceiver IO | ListOps · ListOps test | test accuracy: 39.70% -> 41.40% | Improved |
| Use NAC rather than Perceiver IO | ASCII-byte text classification · Text classification from ASCII bytes test | test accuracy: 66.50% -> 68.18% | Improved |
| Drop more than 80% of the least-connected modules at inference (ablation) | NAC · Tiny-ImageNet | inference speedup: ~8x speedup; less than 3% performance loss | Improved |
Closest papers by meaning
- Neural networks are more modular than single neurons suggest (NeurIPS 2026 · similarity 0.85)
- Range, not Independence, Drives Modularity in Biologically Inspired Representations (ICLR 2025 · similarity 0.85)
- Breaking Neural Network Scaling Laws with Modularity (ICLR 2025 · similarity 0.85)
- Provably Neural Active Learning Succeeds via Prioritizing Perplexing Samples (ICML 2024 · similarity 0.85)
- Data-Efficient Learning with Neural Programs (NeurIPS 2024 · similarity 0.85)
- Explainable Neural Networks with Guarantee: A Sparse Estimation Approach (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
