Research map / Label-efficient and robust learning / Recognition and long-tail
Soft Task-Aware Routing of Experts for Equivariant Representation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes STAR, a soft task-aware routing of projection-head experts (shared expert or MMoE) for joint invariant and equivariant SSL, reducing redundant feature learning and improving transfer on STL10 and ImageNet100.
Problem
Joint invariant and equivariant self-supervised learning uses separate projection heads that independently encode information shared across the two objectives, causing redundant feature learning and inefficient capacity use.
Approach
Model projection heads as experts and route them softly per task (invariant vs equivariant) and per input via MMoE, so experts specialize into shared versus task-specific roles; the MMoE is used only during pretraining and discarded for transfer.
Key results
- STAR-MMoE with ResNet-50 on ImageNet100 achieves 71.23% mean out-of-domain linear accuracy vs 69.72% for EquiMod and best rank in 10 of 11 datasets.
- STAR-MMoE with ResNet-18 on STL10 achieves 53.07% mean out-of-domain accuracy vs 49.54% for EquiMod.
- On VOC07+12 detection with frozen backbone, STAR-MMoE reaches 48.85 AP vs 48.52 for EquiMod.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use soft task-aware expert routing instead of separate task projection heads (ablation) | EquiMod · STL10 out-of-domain classification | mean accuracy: 49.54 -> 53.07 | Helped |
| Add a shared projection head | SimCLR-based invariant-equivariant learning · ImageNet100-pretrained out-of-domain classification (11 datasets) | mean accuracy: 69.72 -> 70.90 | Helped |
| Use soft task-aware expert routing | SimCLR-based invariant-equivariant learning · ImageNet100-pretrained out-of-domain classification (11 datasets) | mean accuracy: 69.72 -> 71.23 | Helped |
| Use soft task-aware expert routing | SimCLR-based invariant-equivariant learning · STL10-pretrained out-of-domain classification (11 datasets) | mean accuracy: 49.54 -> 53.07 | Helped |
| Use soft task-aware expert routing | SimCLR-based invariant-equivariant learning · VOC07+12 object detection | AP: 48.52 -> 48.85 | Helped |
| Add a residual connection (ablation) | EquiMod · STL10 out-of-domain classification | accuracy: 49.54 -> 49.78 | Helped |
| Use a deeper equivariant predictor (ablation) | EquiMod · STL10 out-of-domain classification | accuracy: 49.54 -> 50.85 | Helped |
| Use a deeper equivariant predictor (ablation) | EquiMod · STL10 in-domain classification | accuracy: 87.01 -> 86.03 | Hurt |
| Add a residual connection (ablation) | STAR-MMoE · STL10 out-of-domain classification | accuracy: 51.43 -> 52.07 | Helped |
| Use a deeper equivariant predictor (ablation) | STAR-MMoE · STL10 out-of-domain classification | accuracy: 51.43 -> 52.53 | Helped |
| Use a deeper equivariant predictor (ablation) | STAR-MMoE · STL10 in-domain classification | accuracy: 87.39 -> 86.57 | Hurt |
| Combine MMoE, residual connection, and deeper predictor (ablation) | STAR-MMoE · STL10 out-of-domain classification | accuracy: 51.43 -> 53.07 | Helped |
Surprising findings
- Soft routing was required instead of top-k sparse routing because batch normalization in each expert makes sparse per-sample activation unstable in SSL.
Limitations
- Soft routing activates all experts per sample, reducing computational efficiency and scalability versus sparse top-k MoE.
- Evaluated only on STL10 and ImageNet100 pretraining with ResNet backbones (plus a ViT-S ablation).
- Redundancy is measured by canonical correlation and gradient cosine similarity rather than a direct optimization target.
Experimental setup
Datasets: STL10 ImageNet100 CIFAR10 CIFAR100 Food MIT67 Pets Flowers Caltech101 Cars Aircraft DTD
Benchmarks: CIFAR10 CIFAR100 Food MIT67 Pets Flowers Caltech101 Cars Aircraft DTD SUN397 VOC07+12 object detection
Metrics: linear evaluation accuracy mean accuracy AP few-shot accuracy with 95% CI R-equivariance P-equivariance canonical correlation
Baselines: SimCLR AugSelf EquiMod CARE STL STL + AugMix
Builds on: EquiMod SimCLR MMoE AugSelf CARE
Closest papers by meaning
- RouterInterp: Understanding Superposed Specialisation in Mixture of Experts Routing (ICML 2026 · similarity 0.85)
- Interpreting Equivariant Representations (ICML 2024 · similarity 0.85)
- STAR: Rethinking MoE Routing as Structure-Aware Subspace Learning (ICML 2026 · similarity 0.85)
- A Probabilistic Approach to Learning the Degree of Equivariance in Steerable CNNs (ICML 2024 · similarity 0.84)
- Learning Task-Agnostic Representations through Multi-Teacher Distillation (NeurIPS 2025 · similarity 0.84)
- seq-JEPA: Autoregressive Predictive Learning of Invariant-Equivariant World Models (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.