Research map / Label-efficient and robust learning / Recognition and long-tail
Towards Faithful Explanations: Boosting Rationalization with Shortcuts Discovery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SSR, a semi-supervised selective rationalization method that discovers shortcut tokens by comparing unsupervised rationales with gold rationales, then uses uniform-KL and virtual shortcut representation losses plus random/semantic data augmentation to reduce shortcut reliance.
Problem
Selective rationalization models often exploit spurious shortcuts in data to make predictions and compose rationales, while gold rationale annotations are scarce, limiting supervised and semi-supervised approaches.
Approach
Identify potential shortcut tokens as tokens selected by a trained unsupervised rationalizer but absent from gold rationales, then inject these shortcuts into training via a uniform-prediction KL constraint or virtual shortcut representations, and augment data by replacing shortcut tokens randomly or semantically.
Key results
- SSR_unif with mixed DA reaches 94.5 task F1 on Movies, comparable to AT-BMC's 92.9 and above Vanilla Sup-RAT's 93.6 level.
- SSR_virt with random DA reaches Token-F1 of 36.7 on Movies and 32.4 on Evidence Inference, outperforming the semi-supervised baselines.
- On the out-of-distribution SST-2 dataset, SSR_virt with semantic DA reaches 83.5 F1 versus 78.3 for WSEE with semantic DA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SSR_unif instead of Vanilla Sup-RAT | SSR_unif · Movies | Task F1: 93.6 $\pm$ 0.3 -> 94.3 $\pm$ 0.3 | Helped |
| add random data augmentation (ablation) | SSR_unif · Evidence Inference | Token F1: 26.8 $\pm$ 0.2 -> 33.1 $\pm$ 0.2 | Helped |
| add semantic rather than random data augmentation (ablation) | SSR_unif · Evidence Inference | Token F1: 33.1 $\pm$ 0.2 -> 33.5 $\pm$ 0.4 | Helped |
| add mixed data augmentation (ablation) | SSR_unif · MultiRC | Token F1: 56.2 $\pm$ 0.2 -> 40.3 $\pm$ 0.5 | Hurt |
| add mixed data augmentation (ablation) | SSR_unif · Movies | Task F1: 94.3 $\pm$ 0.3 -> 94.5 $\pm$ 0.2 | Helped |
| use SSR_virt rather than SSR_unif | SSR · Evidence Inference | Task F1: 46.8 $\pm$ 0.3 -> 50.4 $\pm$ 0.3 | Helped |
| add semantic data augmentation (ablation) | SSR_virt · BoolQ | Task F1: 58.2 $\pm$ 0.5 -> 61.1 $\pm$ 0.3 | Helped |
| add mixed data augmentation (ablation) | SSR_virt · Movies | Token F1: 34.6 $\pm$ 0.2 -> 37.4 $\pm$ 0.1 | Helped |
| add shared selector and predictor encoders (ablation) | SSR_unif · Movies | Task F1: 88.3 $\pm$ 0.1 -> 94.3 $\pm$ 0.3 | Helped |
| add shared selector and predictor encoders (ablation) | SSR_virt · Movies | Task F1: 87.9 $\pm$ 0.5 -> 90.0 $\pm$ 0.0 | Helped |
| add shared W_a and W_p parameters (ablation) | SSR_virt · Movies | Task F1: 88.3 $\pm$ 0.3 -> 90.0 $\pm$ 0.0 | Helped |
Surprising findings
- Increasing the proportion of gold rationales beyond 15% on MultiRC degraded SSR performance.
- Mixed data augmentation does not always outperform single augmentation methods, which the authors attribute to duplicated tokens in augmented data.
Limitations
- Shortcut discovery depends on the quality of the unsupervised rationalizer and on requiring some gold rationales.
- Performance on MultiRC degrades with data augmentation and with more gold rationales.
- Reported ST-RAT results on MultiRC are missing because code could not be reproduced.
Experimental setup
Datasets: Movies MultiRC BoolQ Evidence Inference FEVER IMDB SST-2
Benchmarks: Movies MultiRC BoolQ Evidence Inference FEVER IMDB SST-2
Metrics: weighted F1 (task) token-level F1 (rationale)
Baselines: Vanilla Un-RAT IB INVRAT Inter-RAT MCD CAR 3Player Vanilla Sup-RAT Pipeline UNIREX AT-BMC Vanilla Semi-RAT
Builds on: Gumbel-Softmax ERASER benchmark AT-BMC WSEE UNIREX
Closest papers by meaning
- Breaking Free from MMI: A New Frontier in Rationalization by Probing Input Utilization (ICLR 2025 · similarity 0.86)
- Learning Robust Rationales for Model Explainability: A Guidance-Based Approach (AAAI 2024 · similarity 0.86)
- Federated Self-Explaining GNNs with Anti-shortcut Augmentations (ICML 2024 · similarity 0.85)
- Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets (ICML 2025 · similarity 0.84)
- Tailoring Self-Rationalizers with Multi-Reward Distillation (ICLR 2024 · similarity 0.84)
- Navigate Beyond Shortcuts: Debiased Learning Through the Lens of Neural Collapse (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.