Research map / Architectures and efficiency / Mixture of experts
ExPLoRe: Expert Patch-Level Loss Routing for Multi-Objective Masked Image Modeling
Konstantinos Georgiou, Maofeng Tang, Hairong Qi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Let loss gradients flow through dispatch weights (loss-coupling) (ablation) | ExPLoRe 2-expert · ImageNet-1K k-NN (Token+CLS) | k-NN@20 acc: 73.8 -> 75.4 | Improved |
| Add dispatch-weight loss weighting (ablation) | 2-expert MoE · ImageNet-1K k-NN (Token+CLS) | k-NN@20 acc: 74.1 -> 75.4 | Improved |
| Add dispatch-weight loss weighting (ablation) | 64-expert MoE · ImageNet-1K k-NN (Token+CLS) | k-NN@20 acc: 75.3 -> 76.2 | Improved |
| Add dispatch-weight loss weighting (ablation) | 32-expert MoE · ImageNet-1K k-NN (Token+Pixel) | k-NN@20 acc: 73.0 -> 73.7 | Improved |
| Add 2-expert MoE without loss weighting (ablation) | Token+CLS base · ImageNet-1K k-NN | k-NN@20 acc: 75.7 -> 74.1 | Hurt |
| Add 2-expert ExPLoRe to Token+CLS | Token+CLS base · ImageNet-1K linear probe | top-1 acc: 76.2 -> 79.6 | Improved |
| Add 2-expert ExPLoRe to Token+CLS | Token+CLS base · ImageNet-1K k-NN | k-NN@20 acc: 75.7 -> 75.4 | No clear effect |
| Remove entropy regularization or set lambda=0.01 (ablation) | ExPLoRe 2-expert · ImageNet-1K k-NN | k-NN@20 acc: 75.4 -> 66.9 (lambda 0.01); 2.1 (none) | Hurt |
| Use combine-weight loss weighting instead of dispatch weights (ablation) | ExPLoRe 2-expert · ImageNet-1K k-NN | k-NN@20 acc: text: collapses to 2.1; Table 2 combine reg. 75.4 | Mixed |
| Move loss block from 11 to 5/7/9 (ablation) | ExPLoRe 2-expert · ImageNet-1K k-NN | k-NN@20 acc: 75.4 -> 75.2-75.4 | No clear effect |
| Scale Token+Pixel from 32 to 64 experts | ExPLoRe Token+Pixel · ImageNet-1K k-NN | k-NN@20 acc: 73.7 -> 69.3 | Hurt |
| Replace global GradNorm weighting with ExPLoRe | Token+Pixel · ImageNet-1K k-NN | k-NN@20 acc: 32.5 -> 73.7 | Improved |
Closest papers by meaning
- Hard Patches Mining for Masked Image Modeling (CVPR 2023 · similarity 0.88)
- Masked Image Modeling With Local Multi-Scale Reconstruction (CVPR 2023 · similarity 0.88)
- Improving Pixel-based MIM by Reducing Wasted Modeling Capability (ICCV 2023 · similarity 0.88)
- From Prototypes to General Distributions: An Efficient Curriculum for Masked Image Modeling (CVPR 2025 · similarity 0.86)
- Patch-Aware Sample Selection for Efficient Masked Image Modeling (AAAI 2024 · similarity 0.86)
- ChA-MAEViT: Unifying Channel-Aware Masked Autoencoders and Multi-Channel Vision Transformers for Improved Cross-Channel Learning (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
