Research map / Large language models / Language, safety and interpretability
Probing Natural Language Inference Models through Semantic Fragments
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Evaluate NLI models pre-trained on existing benchmarks on semantic fragments | BERT, ESIM, and Decomp-Attn · Logic and monotonicity fragments | accuracy | Hurt |
| Fine-tune BERT on fragment examples and select hyperparameters by aggregate fragment and original-task scores | BERT · Logic and monotonicity fragments | accuracy: 47.3 -> 98.0; 62.8 -> 97.8 | Helped |
| Fine-tune BERT on fragments | BERT · MNLI Dev (Avg.) | accuracy: ↓ 1.3 | Hurt |
| Fine-tune non-BERT NLI models on monotonicity examples | ESIM and Decomp-Attn · Monotonicity Reasoning | accuracy | Helped |
| Fine-tune non-BERT NLI models on comparative examples | ESIM and Decomp-Attn · Comparatives | accuracy | Mixed |
| Fine-tune Decomp-Attn on fragment examples | Decomp-Attn · Logic Fragments | accuracy | Mixed |
| Train NLI architectures from scratch on semantic fragments rather than benchmark NLI data | NLI architectures · Semantic fragments and benchmark tasks | accuracy | Mixed |
Closest papers by meaning
- metabench - A Sparse Benchmark of Reasoning and Knowledge in Large Language Models (ICLR 2025 · similarity 0.88)
- Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes (ICLR 2026 · similarity 0.88)
- Supervised Knowledge Makes Large Language Models Better In-context Learners (ICLR 2024 · similarity 0.88)
- Making Retrieval-Augmented Language Models Robust to Irrelevant Context (ICLR 2024 · similarity 0.88)
- Finding the Minimal Parameter Budget for Implicit Reasoning: A Data Complexity Driven Scaling Law for Language Models (ICML 2026 · similarity 0.88)
- Linguini: A benchmark for language-agnostic linguistic reasoning (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.