Research map / Architectures and efficiency / RNNs and normalization
Weight-Space Mixture-of-Experts for Implicit Neural Representation Classification
Stanisław Janik, Michal Byra
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add layer-wise MoE stage before token-wise MoE (ablation) | HMoE-MWT · CIFAR-10 (MoE ablation) | accuracy: 62.57 -> 65.01 | Improved |
| add token-wise MoE stage after layer-wise MoE (ablation) | HMoE-MWT · CIFAR-10 (MoE ablation) | accuracy: 63.21 -> 65.01 | Improved |
| increase active token experts (top-2, top-4) (ablation) | HMoE-MWT token-wise MoE · CIFAR-10 (MoE ablation) | accuracy: 62.57 -> 62.21 | No clear effect |
| increase layer experts from 4 to 8 (ablation) | HMoE-MWT · CIFAR-10 (MoE ablation) | accuracy: 65.01 -> 64.03 | No clear effect |
| increase token experts from 4 to 8 (ablation) | HMoE-MWT · CIFAR-10 (MoE ablation) | accuracy: 65.01 -> 64.53 | No clear effect |
| add classification-loss injection (meta-learning) to HMoE | HMoE · CIFAR-10 (INR classification) | accuracy: 55.44 -> 65.01 | Improved |
| replace dense transformer with hierarchical MoE (no meta-learning) | WT · CIFAR-10 (INR classification) | accuracy: 43.78 -> 55.44 | Improved |
| increase SIREN width from 128 to 256 | HMoE-MWT · CIFAR-10 (INR classification) | accuracy: 65.01 -> 64.40 | No clear effect |
| use HMoE instead of parameter-matched dense MWT at 11M | MWT · Imagenette (parameter-matched) | accuracy: 59.92 -> 62.52 | Improved |
| increase task loss weight from 0.01 to 1 (ablation) | HMoE-MWT · CIFAR-10 (80/20 split ablation) | accuracy: 63.14 -> 55.34 | Hurt |
| increase inner-loop steps from 1 to 6 (ablation) | HMoE-MWT · CIFAR-10 (80/20 split ablation) | accuracy: 57.84 -> 63.14 | Improved |
| reduce coordinate subsampling rate from 1.0 to 0.001 (ablation) | HMoE-MWT · CIFAR-10 (80/20 split ablation) | accuracy: 63.14 -> 11.59 | Hurt |
Closest papers by meaning
- Neural Experts: Mixture of Experts for Implicit Neural Representations (NeurIPS 2024 · similarity 0.88)
- Towards Scalable and Versatile Weight Space Learning (ICML 2024 · similarity 0.86)
- Improved Generalization of Weight Space Networks via Augmentations (ICML 2024 · similarity 0.86)
- Inductive Gradient Adjustment for Spectral Bias in Implicit Neural Representations (ICML 2025 · similarity 0.85)
- MIRE: Matched Implicit Neural Representations (CVPR 2025 · similarity 0.85)
- Implicit Representations via Operator Learning (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
