Research map / Large language models / Retrieval and knowledge
Information Flow Reveals When to Trust Language Models
Rui Xu, Yi Chen, Jiujiu Chen, Sihong Xie
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| model progressive layer-wise information flow (simulatability and concentration) instead of single-layer or pooled attention signals | RAG uncertainty quantification · SQuAD2.0, HotpotQA, MS MARCO | AUROC: average AUROC 0.709 vs runner-up 0.676 | Improved |
| use the information-flow method instead of Focus on HotpotQA | LLaMA-3.2-3B-Instruct · HotpotQA test | AUROC: 0.701 (Focus) vs 0.672 (Ours) | Hurt |
| train the calibrator on a different dataset than the test set (ablation) | Information-flow calibrator · Cross-dataset (SQuAD2.0, HotpotQA, MS MARCO) | AUROC: SQuAD2.0 test, LLaMA-3.2-3B: 0.735 / 0.728 / 0.715 across train sets | No clear effect |
| ablate the top-5 context tokens ranked by the flow measures (ablation) | Information-flow rankings · 100 correct samples per dataset | remaining correct predictions: over 50% of correct predictions become incorrect | Hurt |
| ablate the bottom-5 context tokens ranked by the flow measures (ablation) | Information-flow rankings · 100 correct samples per dataset | remaining correct predictions | No clear effect |
| human-verify ranker relevance layouts before computing simulatability (ablation) | Simulatability · Ranker bias check | AUROC and AUPRC: modest increase | Improved |
| apply the same post-hoc calibration to baseline methods (ablation) | Baseline UQ methods · Appendix K | AUROC | No clear effect |
Closest papers by meaning
- Conformal Prediction with Paraphrase-Aware Scoring for LLM Uncertainty Quantification (NeurIPS 2026 · similarity 0.89)
- Distinguishing the Knowable from the Unknowable with Language Models (ICML 2024 · similarity 0.89)
- Conformal Prediction Beyond the Seen: A Missing Mass Perspective for Uncertainty Quantification in Generative Models (NeurIPS 2025 · similarity 0.89)
- Efficient semantic uncertainty quantification in language models via diversity-steered sampling (NeurIPS 2025 · similarity 0.88)
- SPANUQ: Span-Level Uncertainty Quantification for Large Language Model Generation (NeurIPS 2026 · similarity 0.88)
- Forking Paths in Neural Text Generation (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
