Research map / Large language models / Language, safety and interpretability
LIREx: Augmenting Language Inference with Relevant Explanations
Xinyan Zhao, V.G.Vinod Vydiswaran
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add probability-sampled, rationale-enabled explanations to the P-H input | LIREx on SNLI · SNLI test | accuracy: 91.55{ $\pm$.04} -> 91.87{ $\pm$.03} | Improved |
| Use probability-based sampling rather than greedy highest-probability selection (ablation) | LIREx on SNLI · SNLI test | accuracy: 91.73{ $\pm$.03} -> 91.87{ $\pm$.03} | Improved |
| Use explanations alone rather than combining them with the P-H pair (ablation) | LIREx on SNLI · SNLI test | accuracy: 91.87 -> 62.21 | Hurt |
| Remove explanations from the inference input (ablation) | LIREx on SNLI · SNLI test | accuracy: 91.87 -> 91.07 | Hurt |
| Use NILE-generated explanations instead of LIREx-generated explanations (ablation) | LIREx architecture · SNLI test, random explanation selection | accuracy: 91.55 -> 85.58 | Hurt |
| Randomly select an explanation at evaluation instead of using the best explanation (ablation) | LIREx · SNLI test | accuracy: 91.80 -> 91.55 | Hurt |
| Use probability-sampled rationale-enabled explanations with P-H input | LIREx transferred from SNLI to MultiNLI · MultiNLI Dev-matched | accuracy: 80.12 -> 79.85 | Hurt |
| Use probability-sampled rationale-enabled explanations with P-H input | LIREx transferred from SNLI to MultiNLI · MultiNLI Dev-mismatched | accuracy: 79.73 -> 79.79 | Improved |
| Remove label information from the explanation-generation prompt (ablation) | LIREx explanation generator · SNLI | accuracy: 91.59 -> 91.80 | Improved |
| Add three explanations to the P-H input using NILE_all (ablation) | NILE on SNLI · SNLI test | accuracy: 91.49 -> 91.12 | Hurt |
| Add extra negative samples to NILE_all training (ablation) | NILE on SNLI · SNLI test | accuracy: 91.12 -> 90.73 | Hurt |
| Use the selected explanation without the P-H pair (ablation) | LIREx on SNLI · SNLI test | accuracy: 91.87 -> 89.73 | Hurt |
Closest papers by meaning
- e-SNLI: Natural Language Inference with Natural Language Explanations (NeurIPS 2018 · similarity 0.89)
- Towards Interpretable Natural Language Understanding with Explanations as Latent Variables (NeurIPS 2020 · similarity 0.88)
- Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processing (NeurIPS 2021 · similarity 0.88)
- Automated Assessment of Fidelity and Interpretability: An Evaluation Framework for Large Language Models’ Explanations (Student Abstract) (AAAI 2024 · similarity 0.85)
- Faithful Explanations of Black-box NLP Models Using LLM-generated Counterfactuals (ICLR 2024 · similarity 0.85)
- SeqGPT: An Out-of-the-Box Large Language Model for Open Domain Sequence Understanding (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
