Research map / Label-efficient and robust learning / Recognition and long-tail
Learn from A Rationalist: Distilling Intermediate Interpretable Rationales
Jiayi Dai, Randy Goebel
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| distill teacher rationales and predictions into student (REKD) | ViT Small RE · CIFAR-10 | accuracy: .889 -> .968 | Improved |
| distill teacher rationales and predictions into student (REKD) | ViT Tiny RE · CIFAR-10 | accuracy: .797 -> .936 | Improved |
| distill teacher into student (REKD) | ViT Small RE · CIFAR-100 (20 coarse classes) | accuracy: .779 -> .845 | Improved |
| distill teacher into student (REKD) | ViT Tiny RE · CIFAR-100 (20 coarse classes) | accuracy: .645 -> .777 | Improved |
| distill teacher into student (REKD) | BERT Small RE · IMDB | accuracy: .881 -> .906 | Improved |
| distill teacher into student (REKD) | BERT Mini RE · IMDB | accuracy: .863 -> .892 | Improved |
| use a smaller backbone for RE (less capable base model) (ablation) | RE with ViT · CIFAR-10 | accuracy: CLS-to-RE drop: .020 (Base), .092 (Small) | Hurt |
| lower target rationale ratio ptarget (5% to 75% varied) (ablation) | ViT RE · CIFAR-10 | accuracy: accuracy rises with rationale length | Mixed |
| REKD students versus black-box CLS students | BERT Small and Mini · IMDB | accuracy: .906 vs .889 (Small); .892 vs .877 (Mini) | Improved |
| REKD student versus its Base RE teacher average | ViT Small · CIFAR-10 | accuracy: .968 vs .964 | No clear effect |
| REKD student versus its Base RE teacher average | ViT Small · CIFAR-100 (20 coarse classes) | accuracy: .845 vs .830 | Improved |
Closest papers by meaning
- Explainable Neural Networks with Guarantee: A Sparse Estimation Approach (AAAI 2025 · similarity 0.86)
- Learning Robust Rationales for Model Explainability: A Guidance-Based Approach (AAAI 2024 · similarity 0.86)
- Towards Faithful Explanations: Boosting Rationalization with Shortcuts Discovery (ICLR 2024 · similarity 0.85)
- Breaking Free from MMI: A New Frontier in Rationalization by Probing Input Utilization (ICLR 2025 · similarity 0.84)
- Enhancing Performance of Explainable AI Models with Constrained Concept Refinement (ICML 2025 · similarity 0.84)
- Intrinsic User-Centric Interpretability through Global Mixture of Experts (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
