AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

A Positive Case for Faithfulness: Explanations Help Predict Model Behavior

Harry Mayne, Justin Singh Kang, Dewi Sid William Gould, Kannan Ramchandran, Adam Mahdi, Noah Y. Siegel

ICML 2026 · Large language models › Language, safety and interpretability · approach: Faithfulness of LLM explanations

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
give predictors the reference model's self-explanation18 frontier LLMs as reference models · 7,000 counterfactual pairsNSG: NSG 11.0-36.5%; absolute gain 3.8-10.8%Improved
use self-explanations instead of explanations from external models (ablation)NSG evaluation · top 3 models per family, cross-familyNSG: +4.3pp GPT-5, +3.0pp Qwen 3, +1.7pp Gemma 3Improved
scale up reference model size within Qwen 3 family (ablation)Qwen 3 reference models · NSG across datasetsNSG: 11.03 (0.6B) -> 35.61 (32B)Improved
scale up model size among proprietary models (ablation)GPT-5, Claude 4.5, Gemini 3 · NSG across datasetsNSGNo clear effect
raise reasoning strength (none to high) (ablation)Claude Sonnet 4.5 and GPT-5.2 · NSG across datasetsNSGNo clear effect
use user-facing explanation instead of raw chain-of-thought as the explanation (ablation)Qwen 3 models · NSG with 3 predictorsNSGImproved
vary reference prompt role framing (generic, professional, specialist) (ablation)Qwen3-32B · Heart Disease, Income, Breast CancerNSG: 27.4, 29.9, 26.3No clear effect
provide explanation to predictorpredictor models · counterfactuals with unchanged reference answerfalse change rate: 35.4 -> 18.2 (gpt-oss-20b)Improved
use stronger predictors (ablation)NSG evaluation · predictor ensembleabsolute simulatability gain: +11.9 (Qwen3-32B) vs +4.8 (gemini-3-flash)Hurt
longer reference reasoning trace (ablation)proprietary reference models · gpt-oss-20b predictorcorrelation with explanation helpfulness: -0.018 to -0.061No clear effect
move to Moral Machines instead of other datasets (ablation)NSG evaluation · Moral Machines vs Pima DiabetesNSG: 6.0% (Moral Machines) vs 42.8% (Pima Diabetes)Hurt
remove factual (original answer) from baseline: no-information baseline (ablation)Qwen 3 family · 3 predictorssimulatability gainImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy