Research map / Large language models / Retrieval and knowledge

IRCAN: Mitigating Knowledge Conflicts in LLM Generation via Identifying and Reweighting Context-Aware Neurons

NeurIPS 2024 · Large language models › Retrieval and knowledge · approach: Knowledge editing in LLMs · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes IRCAN, which uses integrated-gradient attribution to find FFN neurons that process context and up-weights them to make LLMs more faithful to context under knowledge conflicts.

Problem

LLMs given contextual information that contradicts their parametric knowledge tend to follow memorized knowledge rather than the provided context, producing outdated or hallucinated answers.

Approach

Attribute context-processing importance to individual FFN neurons with a context-aware integrated-gradients score, select neurons shared across instances by co-occurrence, and scale their weights by a factor beta; this is training-free and complementary to contrastive decoding (CAD).

Key results

• On MemoTrap, IRCAN raises LLaMA-2-7B accuracy from 24.52 to 56.08 (129% relative) and LLaMA-3-8B from 20.26 to 47.76 (136% relative).
• IRCAN-CAD achieves the best MemoTrap accuracy across all five models, e.g., 61.83 on LLaMA-2-7B versus 44.56 for CAD.
• General ability on six Open LLM Leaderboard benchmarks changes little under IRCAN (e.g., LLaMA-2-7B average 50.47 to 49.94), though LLaMA-3-8B GSM8K drops from 50.72 to 36.92.

Measured effects reported in the paper

Surprising findings

• ITI with Mass Mean Shift drives Gemma-2B and Amber to 0% accuracy on MemoTrap, failing to produce normal responses.
• IRCAN gives smaller gains on more capable models (LLaMA-3-8B-Instruct, LLaMA-2-13B-Chat) on the multiple-choice tasks, which the authors attribute to noisy ChatGPT-generated contexts.

Limitations

• Evaluated only on a few synthetic datasets; long-context and RAG settings not tested.
• Gains are smaller on more capable models and on the multiple-choice tasks where context is generated by ChatGPT and may be inaccurate.
• No error bars or statistical significance reported.

Experimental setup

Datasets: MemoTrap COSE_KRE ECARE_KRE

Benchmarks: MemoTrap COSE_KRE ECARE_KRE ARC HellaSwag MMLU TruthfulQA Winogrande GSM8K

Metrics: accuracy stubbornness rate

Baselines: Original ITI CAD Opin Based_on prompts

Builds on: integrated gradients knowledge neurons context-aware decoding inference-time intervention

Closest papers by meaning

• Journey to the Center of the Knowledge Neurons: Discoveries of Language-Independent Knowledge Neurons and Degenerate Knowledge Neurons (AAAI 2024 · similarity 0.87)
• Identifying Query-Relevant Neurons in Large Language Models for Long-Form Texts (AAAI 2025 · similarity 0.87)
• Mixture of In-Context Experts Enhance LLMs' Long Context Awareness (NeurIPS 2024 · similarity 0.86)
• Taming Knowledge Conflicts in Language Models (ICML 2025 · similarity 0.86)
• Supervised Knowledge Makes Large Language Models Better In-context Learners (ICLR 2024 · similarity 0.86)
• ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation (NeurIPS 2025 · similarity 0.85)