AI Paper Atlas

Research map / Large language models / Retrieval and knowledge

Do Retrieval Augmented Language Models Know When They Don’t Know?

Youchao Zhou, Heyan Huang, Yicheng Liu, Rui Dai, Xinglin Wang, Xingchen Zhang, Shumin Shi, Yang Deng

AAAI 2026 · Large language models › Retrieval and knowledge · approach: Knowledge-augmented question answering · 6 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add one positive chunk to an all-negative retrieved contextVanilla RALM · RGBen calibrationBrier score (Agree): 0.192 -> 0.026Improved
provide all-negative context instead of no contextVanilla Qwen-2.5-7B · RALMrefusal rate / overall accuracy: OAcc 0.427 -> 0.290Hurt
apply R-tuning refusal trainingQwen-2.5-7B · 0p10n contextover-refusal rate: 0.355 -> 0.678Hurt
apply ICFT(n) in-context fine-tuning with negative contextQwen-2.5-7B · 0p10n contextover-refusal rate: 0.355 -> 0.270Improved
apply ICFT(n)Qwen-2.5-7B · 0p10n contextoverall accuracy: 0.290 -> 0.620Improved
apply ICFT(p) (positive context only)Qwen-2.5-7B · 1p9n contextoverall Brier score: 0.079 -> 0.068Improved
apply refusal tuning (R-tuning, ICFT) with positive context availableQwen-2.5-7B · 1p9n contextoverall accuracy: 0.863 -> 0.830 (R-tuning), 0.787 (ICFT n)Hurt
apply R-tuningQwen-2.5-7B · 0p10n contextdenoising rate: 0.191 -> 0.138Hurt
apply ICFT variantsQwen-2.5-7B · 0p10n contextdenoising rate: 0.191 -> 0.632 to 0.691Improved
add two-stage knowledge-state aware refusal over threshold post refusalVanilla Qwen-2.5-7B · 0p10n contextoverall accuracy / over-refusal: OAcc 0.437 -> 0.523; OR 0.770 -> 0.282Improved
add two-stage refusalICFT(n) Qwen-2.5-7B · 0p10n contextoverall accuracy: 0.673 -> 0.729Improved
increase positive chunks beyond one (5p5n)RALM · RGBen / RGBzh calibrationBrier score (Agree): RGBen 0.026 -> 0.010; RGBzh 0.078 -> 0.150Mixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy