Research map / Graph learning / Knowledge and temporal graphs
Bilinear relational structure fixes reversal curse and enables consistent model editing
Dong-Kyum Kim, Minsung Kim, Jea Kwon, Nakyeong Yang, Meeyoung Cha
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase weight decay to at least 1.0 during training (ablation) | GPT-NeoX 206M trained from scratch on family KG · Unseen father/mother test | test accuracy: low accuracy (weight decay < 1.0) -> near-perfect accuracy for some models | Improved |
| bilinear probe instead of linear or translational probe (ablation) | Not Reversal Cursed model · Layer-wise probe accuracy | probe accuracy: above 95% vs near or below 1/3 baseline | Improved |
| increase linear probe samples n from 10 to 500 (ablation) | Linear relational embedding probe · Probe accuracy | probe accuracy | No clear effect |
| increase bilinear probe samples n (insufficient n) (ablation) | Bilinear probe · Probe accuracy | probe accuracy | Improved |
| model has bilinear structure (Not Reversal Cursed vs Reversal Cursed) | Fine-tuning model edit · Logical generalization after edit | logical generalization | Improved |
| model has bilinear structure | Fine-tuning model edit · Locality | locality | Improved |
| model has bilinear structure | Fine-tuning model edit · Direct edit success | edit success | No clear effect |
| edit early-to-mid layers (1-4) rather than layers 6-9 (ablation) | Not Reversal Cursed models · Logical generalization after edit | logical generalization: layers 1-4 best vs bilinear peak at layers 6-9 | Improved |
Closest papers by meaning
- Relation Also Knows: Rethinking the Recall and Editing of Factual Associations in Auto-Regressive Transformer Language Models (AAAI 2025 · similarity 0.88)
- Representation Shattering in Transformers: A Synthetic Study with Knowledge Editing (ICML 2025 · similarity 0.87)
- AnyEdit: Edit Any Knowledge Encoded in Language Models (ICML 2025 · similarity 0.86)
- Journey to the Center of the Knowledge Neurons: Discoveries of Language-Independent Knowledge Neurons and Degenerate Knowledge Neurons (AAAI 2024 · similarity 0.86)
- Mechanistic Unlearning: Robust Knowledge Unlearning and Editing via Mechanistic Localization (ICML 2025 · similarity 0.85)
- Emergence of Linear Truth Encodings in Language Models (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
