Research map / Architectures and efficiency / Attention and transformers
DataMUX: Data Multiplexing for Neural Networks
Vishvak Murahari, Carlos E Jimenez, Runzhe Yang, Karthik R Narasimhan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Hadamard/Ortho multiplexing instead of unstructured mixing for larger N (ablation) | T-MUX · MNLI/NER | accuracy | No clear effect |
| reduce self-attention heads from 12 to 2 (ablation) | T-MUX · MNLI/NER | accuracy | No clear effect |
| use binary masking multiplexing instead of Gaussian Hadamard (ablation) | T-MUX · retrieval/MNLI/NER | accuracy | Hurt |
| unfreeze (learn) Gaussian multiplexing vectors (ablation) | T-MUX · retrieval/MNLI/NER | accuracy | No clear effect |
| use MLP demultiplexing instead of index embeddings (ablation) | T-MUX · NLP tasks | accuracy | Hurt |
| add LowRank multiplexing instead of Ortho (ablation) | MLP multiplexing · MNIST | accuracy: +5% at N=8 | Improved |
| use nonlinear conv multiplexing instead of Ortho for CNN (ablation) | CNN multiplexing · MNIST | accuracy: 56 -> above 80 (N<=4 vs Ortho at N=8 not same N) | Improved |
| increase number of multiplexed instances N from small to 40 | T-MUX · MNLI/NER | accuracy: 10%-15% drop at N=40 | Hurt |
| multiplex 40 instances for throughput | T-MUX · MNLI | throughput: 18x at N=40, 11x at N=20 | Improved |
Closest papers by meaning
- msf-CNN: Patch-based Multi-Stage Fusion with Convolutional Neural Networks for TinyML (NeurIPS 2025 · similarity 0.84)
- RegMix: Data Mixture as Regression for Language Model Pre-training (ICLR 2025 · similarity 0.84)
- An Efficient Matrix Multiplication Algorithm for Accelerating Inference in Binary and Ternary Neural Networks (ICML 2025 · similarity 0.84)
- Data Filtering Networks (ICLR 2024 · similarity 0.83)
- MoLEx: Mixture of Layer Experts for Fine-tuning with Sparse Upcycling (ICLR 2025 · similarity 0.83)
- DEPT: Decoupled Embeddings for Pre-training Language Models (ICLR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
