Research map / Architectures and efficiency / RNNs and normalization
MIMONets: Multiple-Input-Multiple-Output Neural Networks Exploiting Computation in Superposition
Nicolas Menet, Michael Hersche, Geethan Karunaratne, Luca Benini, Abu Sebastian, Abbas Rahimi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use two superposition channels instead of one | MIMOConv · CIFAR10 | inference complexity per sample: 5.335 GMACs for N=1, 2.667 GMACs for N=2 | Improved |
| Increase superposition from two to four inputs per pass | MIMOConv · CIFAR100 | accuracy: 82.30±0.19 -> 78.44±0.30 | Hurt |
| Use dynamic channel partitioning rather than a fixed four-input mode | MIMOConv dynamic (N=1–4) · CIFAR10 | accuracy: 95.58±0.23 -> 97.13±0.11 | Improved |
| Increase superposition from two to four inputs per pass | MIMOFormer (att.) · LRA average | accuracy: 58.61±0.49 -> 49.29±2.91 | Hurt |
| Perform superposition in both attention and the MLP rather than attention only | MIMOFormer (N=2) · LRA average | accuracy: 58.61±0.49 -> 55.98±0.37 | Hurt |
| Use curriculum training that reduces N to N/2 during warmup | MIMOFormer (N=4, att.) · LRA average | accuracy: 49.29±2.91 -> 54.82±0.58 | Improved |
| Add isometry regularization (ablation) | MIMOConv · CIFAR10 and CIFAR100 | accuracy | Improved |
| Train MIMOConv for more epochs (ablation) | MIMOConv · CIFAR10 and CIFAR100 | accuracy | Improved |
| Use 32 or 64 feature maps in the first layer (ablation) | MIMOConv · CIFAR10 and CIFAR100 | training stability | Improved |
| Freeze binding keys during training (ablation) | MIMOConv · CIFAR10 and CIFAR100 | accuracy | No clear effect |
| Use four query/key-value grid channels rather than two (ablation) | MIMOFormer wide models · LRA average | accuracy: 59.86±0.29 -> 56.37±0.35 | Hurt |
| Use DPFP attention features instead of FAVOR+ (ablation) | MIMOFormer (N=2, att.) · Associative recall | accuracy: 96.52±2.79 -> 93.64±12.66 | Hurt |
Closest papers by meaning
- Expand Neurons, Not Parameters (ICML 2026 · similarity 0.83)
- A Convergence Theory for Deep Learning via Over-Parameterization (ICML 2019 · similarity 0.83)
- An Efficient Matrix Multiplication Algorithm for Accelerating Inference in Binary and Ternary Neural Networks (ICML 2025 · similarity 0.83)
- Multilinear Operator Networks (ICLR 2024 · similarity 0.83)
- Harnessing Manycore Processors with Distributed Memory for Accelerated Training of Sparse and Recurrent Models (AAAI 2024 · similarity 0.82)
- Position: A Theory of Deep Learning Must Include Compositional Sparsity (ICML 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
