Research map / Architectures and efficiency / Mixture of experts
Expert Gate: Lifelong Learning With a Network of Experts
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use reconstruction-error autoencoder gates to select the expert at test time | Expert Gate · 6 image classification tasks | classification accuracy: 48.7 -> 67.3 | Helped |
| Use separate specialist experts with autoencoder gating instead of joint training | 6-task image classification · 6 image classification tasks (average) | classification accuracy: 66.5 -> 67.3 | Helped |
| Use Expert Gate instead of sequentially fine-tuning one shared model | 3-task image classification · 3 image classification tasks (average) | classification accuracy: 58.2 -> 68.6 | Helped |
| Select transfer source and LwF versus fine-tuning using task relatedness | New-task expert training · High-relatedness task pairs | target-task performance | Helped |
| Apply LwF transfer between low-relatedness tasks | New-task expert training · Low-relatedness task pairs | target-task performance | Hurt |
| Use multiple expert models rather than one sequentially fine-tuned model | Lifelong image classification · Older image classification tasks | classification accuracy | Helped |
| Use a single sequential LwF model rather than exclusive task models | Lifelong image classification · 3 image classification tasks | classification accuracy | Hurt |
| Activate multiple experts when gate confidence exceeds a 0.1 threshold (ablation) | Expert Gate · 6 image classification tasks | classification accuracy: +0.9% | Helped |
| Use 100 hidden neurons in the task autoencoder (ablation) | Expert Gate autoencoder · Image classification tasks | performance | Helped |
| Use LwF instead of fine-tuning for Birds and Scenes | Separate task experts · Birds and Scenes | classification accuracy | Helped |
| Use LwF instead of fine-tuning for Flowers | Separate task experts · Flowers | classification accuracy | Hurt |
Closest papers by meaning
- Mixture of Experts Meets Prompt-Based Continual Learning (NeurIPS 2024 · similarity 0.86)
- Theory on Mixture-of-Experts in Continual Learning (ICLR 2025 · similarity 0.86)
- Orchestrate Latent Expertise: Advancing Online Continual Learning with Multi-Level Supervision and Reverse Self-Distillation (CVPR 2024 · similarity 0.85)
- Divide and not forget: Ensemble of selectively trained experts in Continual Learning (ICLR 2024 · similarity 0.84)
- Learning to Specialize: Joint Gating-Expert Training for Adaptive MoEs in Decentralized Settings (NeurIPS 2025 · similarity 0.84)
- AUTOMEM: Automated Learning of Memory as a Cognitive Skill (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.