AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

Detecting Errors in AI-Generated Annotations: When and Why Semantic Neighbors Help

Na Di, Ling Li, Zhe Tang, Hao Cheng, Jinlong Pang, Jiaheng Wei, Zhaowei Zhu

ICML 2026 · Large language models › Language, safety and interpretability · approach: Statistical frameworks for LLM evaluation

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add kNN semantic-neighbor references to judge (SAGE vs Direct)LLM judge · HaluEval (LLaMA-3.1)AUROC (%): 66.65 -> 70.47Improved
add kNN semantic-neighbor references to judge (SAGE vs Direct)LLM judge · HaluEval (Ministral)AUROC (%): 57.61 -> 67.26Improved
add kNN references on TruthfulQALLM judge · TruthfulQA (LLaMA-3.1)AUROC (%): 58.57 -> 58.15Hurt
add kNN references on MMLULLM judge · MMLU (Qwen3)AUROC (%): 67.85 -> 68.99 (Random 70.84)Hurt
replace random references with kNN-retrieved referencesLLM judge · AG NewsAUROC (%): 54.33 -> 62.44Improved
add kNN references to VLM captioningVLM judge · Flickr30kAUROC (%): +1.4 to +2.1 points over DirectImproved
add kNN references on COCO captioningVLM judge · COCO (Qwen3-VL)AUROC (%): 55.80 -> 55.66No clear effect
add kNN references on CIFAR-10VLM judge · CIFAR-10 (InternVL3.5)AUROC (%): 72.03 -> 92.99Improved
add kNN references on CIFAR-10 SAIL-VL2VLM judge · CIFAR-10 (SAIL-VL2)AUROC (%): 74.16 -> 66.93Hurt
use unified external encoder for retrieval instead of self-embeddings (ablation)SAGE · ImageNet-1k (Qwen3-VL)AUROC (%): 65.71 -> 75.29Improved
use full-input retrieval instead of question-only embeddings (ablation)SAGE · TruthfulQA (LLaMA-3.1)AUROC (%): 58.15 -> 60.28Improved
keep judge in the pipeline versus bypassing it (SAGE vs DS2) (ablation)SAGE · TruthfulQA (Qwen3)AUROC (%): 67.10 -> 72.61Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy