AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

Evaluating and Explaining Prompt Sensitivity of LLMs Using Interactions

Ruiyang Qin, Qingzhuo Wang, Tian Wang, Zhihua Wei, Wen Shen

ICML 2026 · Large language models › Language, safety and interpretability · approach: Uncertainty quantification for LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
supervised fine-tuning (instruct/chat vs base model) (ablation)LLMs (Llama-2, Qwen, Olmo, Mistral etc.) · ARCIPSImproved
increase model scale within a model series (ablation)LLMs (Llama-2, Qwen2.5, Olmo2) · ARCIPSImproved
use dense architecture instead of MoE (ablation)LLMs (Llama-2, Llama-3, Qwen, Olmo families) · ARCIPSImproved
dense vs MoE with matched active parameters (ablation)LLMs · ARC (controlled for active parameters)IPSImproved
add in-context examples (few-shot vs 0-shot) (ablation)LLMs (instruct/chat models) · ARCIPSImproved
add more shots beyond 1-shot (ablation)LLMs (instruct/chat models) · ARCIPSImproved
four factors (SFT, scale, dense, few-shot) applied to low-order interactions (ablation)LLMs · ARCIPS_lowImproved
four factors (SFT, scale, dense, few-shot) applied to high-order interactions (ablation)LLMs · ARCIPS_highNo clear effect
change prompt template (semantically equivalent) even when output stays the sameLLMs · MCQ samplesfraction of unstable salient interactions: 60%-80% unstableHurt
change prompt template when output changesLLMs · MCQ samplesshare of same-sign and similar-effect interactions: 2.7%-6.8% stableHurt
vary salient-interaction threshold tau from 0.05 to 0.20 (ablation)IPS metric · 50 LLMsIPS ranking correlation: Spearman 0.9905, Pearson 0.9957No clear effect
apply analysis to open-ended Dolly-15k with paraphrases and instruction reordering (ablation)Llama-2 family · Dolly-15kIPS conclusionsNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy