Research map / Label-efficient and robust learning / Recognition and long-tail
Intrinsic User-Centric Interpretability through Global Mixture of Experts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes InterpretCC, an intrinsically interpretable family (feature gating and group-routing mixture-of-experts) that sparsely activates features or human-defined topical subnetworks per instance, matching black-box accuracy and winning a 56-teacher user study on actionability.
Problem
Intrinsically interpretable models often give nuanced, complex explanations that are hard for domain users (e.g., teachers, clinicians) to act on, while post-hoc explainers are approximate and unfaithful; the paper targets accurate, faithful, sparse, human-actionable local explanations.
Approach
Use instance-dependent Gumbel-Softmax gating to sparsely select input features (Feature Gating) or user-specified feature groups routed to topical expert subnetworks (Group Routing), so the prediction is explicitly a weighted sum over the activated features/groups, making the explanation faithful by construction.
Key results
- ICC Feature Gating is about 9.05% better on average than SENN Features and 3.27% better than NAM on intrinsically interpretable comparisons.
- ICC Group Routing outperforms the non-interpretable base on Geo (81.58 vs 72.96) and Breast Cancer (94.85 vs 89.70) and matches the base elsewhere (matching or exceeding 95% CIs in 15 of 16 comparisons).
- In a 56-teacher Prolific user study, ICC FG ranked highest in global satisfaction and significantly outperformed NAM and SENN on all criteria except completeness; SENN was rated more complete.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Feature Gating instead of the non-interpretable base module | InterpretCC · DSP, HWTS, and OpenXAI | accuracy: 82.81 -> 90.75; 73.93 -> 82.89; 86.67 -> 89.51 | Helped |
| Use Feature Gating instead of the non-interpretable base module | InterpretCC · Breast Cancer | accuracy: 89.70 -> 78.19 | Hurt |
| Use Group Routing instead of the non-interpretable base module | InterpretCC · Geo and Breast Cancer | accuracy: 72.96 -> 81.58; 89.70 -> 94.85 | Helped |
| Use Group Routing instead of the non-interpretable base module | InterpretCC · AG News and SST | accuracy: 89.93 -> 90.35; 91.12 -> 91.75 | Mixed |
| Use Feature Gating instead of SENN Features | InterpretCC · Across reported benchmarks | performance: +9.05% better on average | Helped |
| Use Feature Gating instead of NAM | InterpretCC · Across reported benchmarks | performance: +3.27% better on average | Helped |
| Use Group Routing instead of SENN Concepts | InterpretCC · Across reported benchmarks | performance: +5.63% better on average | Helped |
| Replace L1 mask regularization with annealing L1 regularization (ablation) | InterpretCC Feature Gating · DSP | balanced accuracy: 90.75 +/- 0.01 -> 87.76 +/- 3.12 | Hurt |
| Replace L1 mask regularization with annealing L1 regularization (ablation) | InterpretCC Feature Gating · Geo | balanced accuracy: 71.92 +/- 0.01 -> 81.13 +/- 5.39 | Helped |
| Replace L1 mask regularization with annealing L1 regularization (ablation) | InterpretCC Feature Gating · HWTS | balanced accuracy: 82.89 +/- 0.04 -> 77.58 +/- 0.01 | Hurt |
| Replace L1 mask regularization with annealing L1 regularization (ablation) | InterpretCC Feature Gating · VA | balanced accuracy: 77.80 +/- 0.01 -> 84.81 +/- 0.01 | Helped |
| Change grouping strategy to pattern-based or GPT-4 grouping (ablation) | InterpretCC Group Routing · Geo | performance: +10% | Helped |
Surprising findings
- SENN's explanations were judged more complete than InterpretCC's, a counterexample to the overall preference for InterpretCC.
- Feature Gating underperformed the base model on Breast Cancer (78.19 vs 89.70), suggesting that a higher number of features is needed there.
Limitations
- Discriminator/gating network is a black box, so why features are selected is not explained.
- User-defined groupings may carry little signal and hurt performance; grouping quality depends on expert knowledge.
- User study limited to 56 teachers on one course/task with presentation choices that may bias perception.
Experimental setup
Datasets: EDU MOOC clickstream (DSP, Geo, HWTS, VA) AG News SST Wisconsin Breast Cancer OpenXAI synthetic
Benchmarks: EDU MOOC clickstream AG News SST Wisconsin Breast Cancer OpenXAI synthetic
Metrics: balanced accuracy accuracy ground truth alignment rank agreement feature agreement user study criteria ranking (usefulness, trustworthiness, ac
Baselines: SENN NAM FRESH SHAP LIME Integrated Gradients Top-K routing
Builds on: Switch Transformers Gumbel Softmax Neural Additive Models SENN FRESH
Closest papers by meaning
- Measuring Per-Unit Interpretability at Scale Without Humans (NeurIPS 2024 · similarity 0.86)
- Explainable Neural Networks with Guarantee: A Sparse Estimation Approach (AAAI 2025 · similarity 0.86)
- Mixture of Experts Made Intrinsically Interpretable (ICML 2025 · similarity 0.85)
- Neurons to Words: A Novel Method for Automated Neural Network Interpretability and Alignment (AAAI 2025 · similarity 0.85)
- Local vs. Global Interpretability: A Computational Complexity Perspective (ICML 2024 · similarity 0.85)
- Predictive Concept Decoders: Training Scalable End-to-End Interpretability Assistants (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.