AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

MIMONets: Multiple-Input-Multiple-Output Neural Networks Exploiting Computation in Superposition

Nicolas Menet, Michael Hersche, Geethan Karunaratne, Luca Benini, Abu Sebastian, Abbas Rahimi

NeurIPS 2023 · Architectures and efficiency › RNNs and normalization · approach: Efficient network compression and training · 8 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use two superposition channels instead of oneMIMOConv · CIFAR10inference complexity per sample: 5.335 GMACs for N=1, 2.667 GMACs for N=2Improved
Increase superposition from two to four inputs per passMIMOConv · CIFAR100accuracy: 82.30±0.19 -> 78.44±0.30Hurt
Use dynamic channel partitioning rather than a fixed four-input modeMIMOConv dynamic (N=1–4) · CIFAR10accuracy: 95.58±0.23 -> 97.13±0.11Improved
Increase superposition from two to four inputs per passMIMOFormer (att.) · LRA averageaccuracy: 58.61±0.49 -> 49.29±2.91Hurt
Perform superposition in both attention and the MLP rather than attention onlyMIMOFormer (N=2) · LRA averageaccuracy: 58.61±0.49 -> 55.98±0.37Hurt
Use curriculum training that reduces N to N/2 during warmupMIMOFormer (N=4, att.) · LRA averageaccuracy: 49.29±2.91 -> 54.82±0.58Improved
Add isometry regularization (ablation)MIMOConv · CIFAR10 and CIFAR100accuracyImproved
Train MIMOConv for more epochs (ablation)MIMOConv · CIFAR10 and CIFAR100accuracyImproved
Use 32 or 64 feature maps in the first layer (ablation)MIMOConv · CIFAR10 and CIFAR100training stabilityImproved
Freeze binding keys during training (ablation)MIMOConv · CIFAR10 and CIFAR100accuracyNo clear effect
Use four query/key-value grid channels rather than two (ablation)MIMOFormer wide models · LRA averageaccuracy: 59.86±0.29 -> 56.37±0.35Hurt
Use DPFP attention features instead of FAVOR+ (ablation)MIMOFormer (N=2, att.) · Associative recallaccuracy: 96.52±2.79 -> 93.64±12.66Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy