Research map / Large language models / Retrieval and knowledge
Feeling of Knowing in Large Language Models
Zichuan Fu, Xian Wu, Jingtong Gao, Wenlin Zhang, Jiaxuan Li, Binhao Wang, Yimin Deng, Guojing Li, Xiaopeng Li, Derong Xu, Yefeng Zheng, Xiangyu Zhao
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Studies the feeling of knowing (FoK), an LLM's self-assessment of whether it knows an answer, and how post-training updates affect its reliability for selective generation, retrieval triggering, and model routing.
Problem
LLMs are frequently updated via post-training, and their self-assessment of whether they know what they know (feeling of knowing) can become miscalibrated, which harms selective generation, retrieval triggering, and model routing.
Approach
Frames post-training-induced changes in the feeling of knowing as a reliability problem for LLM self-assessment; the abstract provides only the motivation, so the specific method is not stated.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply vanilla knowledge SFT (K-SFT) update | Qwen2.5-7B · TriviaQA held-out | FoK AUROC: 0.688 -> 0.519 | Hurt |
| apply vanilla K-SFT update | Qwen2.5 and LLaMA models · 7 models, TriviaQA | FoK AUROC: Qwen2.5-3B -0.220 | Hurt |
| add judgment SFT (J-SFT) after the knowledge update | Qwen2.5-7B · TriviaQA | FoK AUROC: -0.042 -> +0.100 change from Base | Improved |
| use on-policy SFT supervising only corrected answer-relevant suffix instead of vanilla K-SFT (ablation) | Qwen2.5-7B · TriviaQA held-out | FoK AUROC: 0.519 -> 0.603 | Improved |
| add knowledge-delta FoK training (KΔ-FT) after OP-SFT (ablation) | Qwen2.5-7B · TriviaQA held-out | FoK AUROC: 0.603 -> 0.770 | Improved |
| add FGPO after KΔ-FT (ablation) | Qwen2.5-7B · TriviaQA held-out | FoK AUROC: 0.770 -> 0.809 | Improved |
| add FGPO after KΔ-FT (ablation) | Qwen2.5-7B · TriviaQA held-out | answer accuracy: 0.699 -> 0.692 | No clear effect |
| retrain the FoK channel each round instead of freezing after round 1 (ablation) | KΔ-FT · three sequential updates | FoK AUROC: 0.754 -> 0.797 | Improved |
| run three sequential knowledge updates | Qwen2.5-7B · TriviaQA held-out | answer accuracy: 0.672 -> 0.661 | No clear effect |
| apply a MEMIT knowledge edit | Qwen2.5-7B · MEMIT zsRE edit | FoK AUROC on edited facts: 0.594 -> 0.553 | Hurt |
| add FGPO policy training (ablation) | Qwen3-8B (thinking) · coverage-accuracy curves | threshold sensitivity | Mixed |
| use post-hoc estimators on the base model with no knowledge update (ablation) | Base + EliCal · TriviaQA held-out | FoK AUROC: 0.810 vs 0.809 for full method | No clear effect |
Closest papers by meaning
- Large Language Models Must Be Taught to Know What They Don’t Know (NeurIPS 2024 · similarity 0.87)
- How new data permeates LLM knowledge and how to dilute it (ICLR 2025 · similarity 0.86)
- PertEval: Unveiling Real Knowledge Capacity of LLMs with Knowledge-Invariant Perturbations (NeurIPS 2024 · similarity 0.86)
- From Static to Dynamic: Knowledge Metabolism for Large Language Models (AAAI 2024 · similarity 0.86)
- Understanding LLM Behaviors via Compression: Data Generation, Knowledge Acquisition and Scaling Laws (NeurIPS 2025 · similarity 0.86)
- KScope: A Framework for Characterizing the Knowledge Status of Language Models (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
