Research map / Label-efficient and robust learning / Recognition and long-tail
Learning The Difference That Makes A Difference With Counterfactually-Augmented Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add counterfactually revised reviews to original IMDb training data | Sentiment classifiers · IMDb revised test | accuracy: 55.7 -> 92.0 | Helped |
| Add counterfactually revised reviews to original IMDb training data | Sentiment classifiers · IMDb original test | accuracy: 79.3 -> 81.5 | Helped |
| Add counterfactually revised reviews to original IMDb training data | BERT Base sentiment classifier · IMDb revised test | accuracy: 82.2 -> 95.1 | Helped |
| Add revised IMDb reviews to original reviews | BERT Base sentiment classifier · Amazon Reviews | accuracy: 80.0 -> 85.1 | Helped |
| Add revised IMDb reviews to original reviews | BERT Base sentiment classifier · Semeval 2017 (Twitter) | accuracy: 79.3 -> 82.9 | Helped |
| Add revised IMDb reviews to original reviews | BERT Base sentiment classifier · Yelp Reviews | accuracy: 85.3 -> 89.4 | Helped |
| Train BERT on original SNLI pairs rather than counterfactually revised-premise examples | BERT Base · SNLI Revised Premise (RP) | accuracy: 72.2 -> 39.7 | Hurt |
| Add revised-premise and revised-hypothesis examples to original SNLI training data | BERT Base · SNLI RP & RH | accuracy: 55.4 -> 67.1 | Helped |
| Add revised-premise and revised-hypothesis examples to original SNLI training data | BERT Base · SNLI Original | accuracy: 77.8 -> 73.5 | Hurt |
| Train the hypothesis-only Bi-LSTM on revised-premise and revised-hypothesis data instead of original SNLI | Hypothesis-only Bi-LSTM · SNLI hypothesis-only Original | accuracy: 69.0 -> 32.4 | Hurt |
| Remove edited spans from training reviews (ablation) | IMDb sentiment classifiers · IMDb original test | accuracy: 80.0 -> 57.8 | Hurt |
| Remove edited spans from training reviews (ablation) | IMDb sentiment classifiers · IMDb original test | accuracy: 79.3 -> 60.2 | Hurt |
Closest papers by meaning
- Explaining the Efficacy of Counterfactually Augmented Data (ICLR 2021 · similarity 0.91)
- Robustness to Spurious Correlations in Text Classification via Automatically Generated Counterfactuals (AAAI 2021 · similarity 0.90)
- Gumbel Counterfactual Generation From Language Models (ICLR 2025 · similarity 0.85)
- Towards Characterizing Domain Counterfactuals for Invertible Latent Causal Models (ICLR 2024 · similarity 0.83)
- A General Search-Based Framework for Generating Textual Counterfactual Explanations (AAAI 2024 · similarity 0.83)
- NOMATTERXAI: Generating “No Matter What” Alterfactual Examples for Explaining Black-Box Text Classification Models (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.