Research map / Large language models / Retrieval and knowledge

Journey to the Center of the Knowledge Neurons: Discoveries of Language-Independent Knowledge Neurons and Degenerate Knowledge Neurons

AAAI 2024 · Large language models › Retrieval and knowledge · approach: Knowledge editing in LLMs · 9 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Architecture-adapted Multilingual Integrated Gradients (AMIG) to localize factual knowledge neurons in multilingual PLMs, and uses it to find language-independent and degenerate knowledge neurons with cross-lingual editing and fact-checking experiments.

Problem

Localizing where factual knowledge is stored in parameters of multilingual pre-trained language models, which existing gradient-based methods handle poorly across both auto-encoding and auto-regressive architectures and in multiple languages.

Approach

Replaces the zero baseline in integrated gradients with architecture-specific baseline inputs (masking the target word for auto-encoders, eos for auto-regressive models), adds per-language dynamic thresholds, intersects per-language neuron sets to get language-independent neurons, and identifies degenerate neurons as overlapping subsets whose joint suppression removes a fact.

Key results

• On m-BERT Chinese, AMIG's knowledge-neuron success rate (SR) is 10.285 vs. B-KN's 5.579 (+84.34%); on m-GPT Chinese it is 7.859 vs. 5.439 (+44.49%).
• Language-independent knowledge neuron editing gives Chinese SR of 7.175 (m-BERT) and 8.868 (m-GPT), versus 2.292 and 2.350 for monolingual-neuron editing.
• Fact-checking with degenerate knowledge neurons raises F1 on m-GPT English from 0.021 to 0.433 and on m-BERT Chinese from 0.020 to 0.654.

Measured effects reported in the paper

Surprising findings

• Editing neurons from one language's dataset gives lower success than editing language-independent neurons, while sequential editing of two languages' neurons underperforms language-independent editing.
• Original PLMs without degenerate neurons almost always judge facts as correct (recall near 1.0), giving very low F1 scores.

Limitations

• Degenerate neuron detection restricted to pairs of neurons (two-neuron assumption) for tractability.
• Relies on hand-set thresholds (T_low, T_high, tau, lambda) and a custom SR metric.
• Experiments limited to m-BERT, m-GPT, BERT and GPT-2 on mLAMA.

Experimental setup

Datasets: mLAMA LAMA

Benchmarks: mLAMA

Metrics: editing success rate SR precision recall F1-score

Baselines: B-KN

Builds on: knowledge neurons integrated gradients sequential integrated gradients causal tracing

Closest papers by meaning

• What does the Knowledge Neuron Thesis Have to do with Knowledge? (ICLR 2024 · similarity 0.91)
• Knowledge Localization: Mission Not Accomplished? Enter Query Localization! (ICLR 2025 · similarity 0.91)
• Precise Localization of Memories: A Fine-grained Neuron-level Knowledge Editing Technique for LLMs (ICLR 2025 · similarity 0.90)
• Identifying Query-Relevant Neurons in Large Language Models for Long-Form Texts (AAAI 2025 · similarity 0.89)
• Knowledge Circuits in Pretrained Transformers (NeurIPS 2024 · similarity 0.89)
• Logically Consistent Language Models via Neuro-Symbolic Integration (ICLR 2025 · similarity 0.89)