Research map / Architectures and efficiency / RNNs and normalization
Axiomatic Atlas: A Prescriptive Framework for Neural Architecture Design
Minghao Guo, Wojciech Matusik
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Atlas-guided bridges that connect the key node across the bottleneck | GNN with single cross-partition edge · GNN bridge placement (m=8) | accuracy (%): 51.2 -> 97.0 | Improved |
| add random bridges instead of Atlas-placed ones (ablation) | GNN with bottleneck · GNN bridge placement (m=8) | accuracy (%): 51.2 -> 51.8 | No clear effect |
| place bridges near the bottleneck but ignoring the key node (separator) (ablation) | GNN with bottleneck · GNN bridge placement (m=8) | accuracy (%): 51.2 -> 51.1 | No clear effect |
| attach bridges at the signal-carrying key node, endpoints random (ablation) | GNN with bottleneck · GNN bridge placement (m=8) | accuracy (%): 51.2 -> 95.8 | Improved |
| anchor quantization grid to stable percentiles (p25, p75) instead of min-max (Q-Anchor) | MoE gate · MoE INT4 quantization | accuracy (%): 11.9 -> 36.7 | Improved |
| use dynamic-k expert allocation guided by routing uncertainty instead of static k=2 | 6-layer MoE Transformer · WikiText-2 | validation perplexity: 228.85 -> 224.02 | Improved |
| allocate extra experts to random tokens instead of uncertain ones (ablation) | dynamic-k routing · WikiText-2 | gap closed (%): 83% -> 22% | Hurt |
| add orthogonal key projection (OrthoKV) removing shared mean direction | attention at L=2048 · associative recall | accuracy: 0% -> 100% | Improved |
| add OrthoKV at short context | attention at L=32 · associative recall | accuracy: 100% -> 100% | No clear effect |
| apply gentle K-only or K+V OrthoKV projection to pretrained model (ablation) | Mistral-7B-v0.1 · WikiText-103 | perplexity: 4.69 -> 4.75 | No clear effect |
| apply aggressive K+V projection (alpha 0.5) to pretrained model (ablation) | Mistral-7B-v0.1 · WikiText-103 | perplexity: 4.69 -> 40.50 | Hurt |
| S2 structural check passes (no affine bypass) as a zero-cost proxy | DARTS-like cells · 50 CIFAR-10 cells | accuracy (%): 56.4 -> 70.1 | Improved |
Closest papers by meaning
- Neural Architecture Design and Robustness: A Dataset (ICLR 2023 · similarity 0.85)
- InTrain: Intrinsic Trainability for Zero-Cost Neural Architecture Search (CVPR 2026 · similarity 0.85)
- Formal Mechanistic Interpretability: Automated Circuit Discovery with Provable Guarantees (ICLR 2026 · similarity 0.85)
- Structure-Aware Graph Hypernetworks for Neural Program Synthesis (ICLR 2026 · similarity 0.85)
- Structuring Open-Ended NAS: Semi-Automated Design Knowledge Structuring with LLMs for Efficient Neural Architecture Search (NeurIPS 2026 · similarity 0.84)
- LogicMP: A Neuro-symbolic Approach for Encoding First-order Logic Constraints (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
