Research map / 3D vision / Face and shape
Neural Experts: Mixture of Experts for Implicit Neural Representations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Neural Experts, a mixture-of-experts architecture for implicit neural representations with a manager network conditioned on expert features and pretrained to a random uniform expert assignment, evaluated on image, audio, and SDF reconstruction.
Problem
Single-MLP implicit neural representations struggle with local, high-frequency detail and scaling because all parameters affect every coordinate; naive MoE applied to INRs converges poorly to good local minima.
Approach
Apply a mixture of experts to INRs where the manager is conditioned by concatenating its encoder features with the expert encoder features and is pretrained to output a balanced random expert assignment before joint reconstruction training; hard argmax routing lets experts subdivide the domain piecewise.
Key results
- On Kodak with Sine activation, Neural Experts reaches 89.35 PSNR versus 77.50 for the wider SIREN baseline (642.8k params) and 57.23 for the base SIREN (99.8k), with 366k params.
- With FINER activation, Neural Experts reaches 90.84 PSNR, above InstantNGP at 84.56 PSNR (7.7M params).
- On SDF reconstruction, Neural Experts Large achieves mean Trimap IoU(0.001) 0.8180 versus 0.6662 for SIREN Large, with 1.3M vs 1.5M params.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace a single-network Sine INR with Neural Experts | Kodak image reconstruction · Kodak | PSNR: 57.23 -> 89.35 | Helped |
| replace a single-network FINER INR with Neural Experts | Kodak image reconstruction · Kodak | PSNR: 58.08 -> 90.84 | Helped |
| use Neural Experts instead of SIREN | audio reconstruction · Bach, Counting, Two Speakers | MSE (×10^-4): 0.71 -> 0.12; 36.6 -> 1.72; 2.06 -> 0.16 | Helped |
| use Neural Experts instead of SIREN Large | 3D surface reconstruction · 3D shapes (mean) | Trimap IoU (d=0.001): 0.6662 -> 0.8180 | Helped |
| condition the manager by concatenating manager and expert encoder outputs rather than no conditioning (ablation) | Neural Experts · Astronaut image | PSNR: 64.52 -> 81 .17 | Helped |
| use random manager pretraining rather than no pretraining (ablation) | Neural Experts · Astronaut image | PSNR: 56.31 -> 81.17 | Helped |
| add speaker identity segmentation supervision (ablation) | Neural Experts audio reconstruction · Two Speakers | MSE (×10^-4): 0.16 -> 0.15 | Helped |
| use 6 experts instead of 4 (ablation) | Neural Experts image reconstruction · Astronaut image | PSNR: 81.17 -> 82.68 | Helped |
| use 5 experts instead of 4 (ablation) | Neural Experts image reconstruction · Astronaut image | PSNR: 81.17 -> 77.08 | Hurt |
| allocate most parameters to the manager rather than to the experts (ablation) | Neural Experts image reconstruction · Astronaut image | PSNR: 89.35±7.10 -> 83.05±10.1 | Hurt |
Surprising findings
- Manager pretraining to a random uniform expert assignment outperformed semantic (SAM) and k-means segmentation-based pretraining for image reconstruction.
- Segmentation supervision with speaker identity gave slightly better audio MSE (0.15 vs 0.16) but slower convergence.
Limitations
- Training cost grows with the number of experts because all experts are evaluated during training.
- Naive extension to InstantNGP was not competitive, and hash encoding parameters dominate, so fair parameter comparison is hard.
- SoftPlus MoE inherits SoftPlus's difficulty fitting low-frequency signals.
Experimental setup
Datasets: Kodak Stanford 3D scanning repository (Armadillo, Dragon, Lucy, Thai Statue) audio clips (bach, counting, two speakers)
Benchmarks: Kodak Stanford 3D scans (Armadillo, Dragon, Lucy, Thai Statue) audio reconstruction clips
Metrics: PSNR SSIM LPIPS MSE Trimap IoU Chamfer distance IoU
Baselines: SoftPlus MLP SoftPlus+FF SIREN FINER Wider MLP variants Vanilla MoE INR InstantNGP SAM-based manager pretraining
Builds on: Mixture of experts (Jacobs et al.) SIREN FINER BACON
Closest papers by meaning
- Multilinear Mixture of Experts: Scalable Expert Specialization through Factorization (NeurIPS 2024 · similarity 0.87)
- Efficient Data Driven Mixture-of-Expert Extraction from Trained Networks (CVPR 2025 · similarity 0.87)
- VI^3NR: Variance Informed Initialization for Implicit Neural Representations (CVPR 2025 · similarity 0.87)
- Learning Transferable Features for Implicit Neural Representations (NeurIPS 2024 · similarity 0.86)
- Implicit Representations via Operator Learning (ICML 2024 · similarity 0.86)
- Grids Often Outperform Implicit Neural Representation at Compressing Dense Signals (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.