Research map / Optimization / Implicit bias and generalization
From Flat Facts to Sharp Hallucinations: Detecting Stubborn Errors via Gradient Sensitivity
Liew Yee Zhing, Andrew Huey Ping Tan, Anwar P.P. Abdul Majeed
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use EPGS instead of Eigenscore | hallucination detector · TriviaQA, Llama-3-8B | AUROC: 0.7226 -> 0.8127 | Improved |
| use EPGS instead of Semantic Entropy on stubborn subset | hallucination detector · SQuAD stubborn subset, Llama-2-7B | AUROC: 0.5842 -> 0.7373 | Improved |
| use EPGS with SQuAD-F1 labels instead of best baseline | EPGS · SQuAD, Llama-2-7B (SQuAD-F1 labels) | AUROC: 0.7578 -> 0.7109 | Hurt |
| use EPGS instead of Eigenscore on TruthfulQA | hallucination detector · TruthfulQA | AUROC: 0.34 - 0.39 -> 0.65 - 0.69 | Improved |
| extract gradients from last transformer block instead of final layer norm (ablation) | EPGS · SQuAD, Llama-3-8B | AUROC: 0.5928 -> 0.7742 | Improved |
| extract gradients from last transformer block instead of LLM head (ablation) | EPGS · SQuAD, Llama-3-8B | AUROC: 0.5474 -> 0.7742 | Improved |
| extract gradients from last block instead of middle block (ablation) | EPGS · SVAMP, Llama-2-7B | AUROC: 0.8031 -> 0.8947 | Improved |
| extract gradients from last block instead of middle block (ablation) | EPGS · SQuAD, Llama-2-7B | AUROC: 0.6965 -> 0.6924 | Hurt |
| vary Gaussian noise magnitude from 0.001 to 0.1 (ablation) | EPGS · SQuAD, Llama-2 | AUROC: 0.7847 -> 0.6924 | Mixed |
| use embedding noise instead of paraphrasing (ablation) | EPGS · TriviaQA, Llama-2-7B | AUROC: 0.6985 -> 0.7629 | Improved |
| use embedding noise instead of dropout (ablation) | EPGS · TriviaQA, Llama-3-8B | AUROC: 0.8028 -> 0.8127 | Improved |
| add NER-based entity masking (ablation) | EPGS · NQ, Llama-2-7B | AUROC: as much as 8.2% drop without mask | Improved |
Closest papers by meaning
- Entropy Distribution as a Fingerprint for Hallucinations in Generative Models (NeurIPS 2026 · similarity 0.90)
- In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigation (ICML 2024 · similarity 0.89)
- Scalable Token-Level Hallucination Detection in Large Language Models (NeurIPS 2026 · similarity 0.89)
- Beyond the Global Scores: Fine-Grained Token Grounding as a Robust Detector of LVLM Hallucinations (CVPR 2026 · similarity 0.89)
- Mitigating Hallucinations in Large Vision-Language Models by Adaptively Constraining Information Flow (AAAI 2025 · similarity 0.88)
- LLM-Check: Investigating Detection of Hallucinations in Large Language Models (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
