Research map / Large language models / Instruction and fine-tuning
Amplifying Diversity and Quality in Commonsense Knowledge Graph Completion (Student Abstract)
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a lightweight CKGC pipeline that prefix-tunes GPT2-xl on top-k similar ConceptNet tuples, decodes tail entities with penalty-tailored diverse beam search (p-DBS), and filters outputs by beam scores to produce diverse, novel, accurate commonsense tuples.
Problem
Commonsense knowledge graph completion with autoregressive LMs suffers from inefficient full fine-tuning, short input sequences mismatched to pre-training, and low diversity and high similarity among generated tail entities under beam or greedy decoding.
Approach
Prefix-tune a frozen source LM using inputs that concatenate top-k head-entity-similar tuples as natural-language sentences, decode tails with a penalty-tailored diverse beam search that relaxes penalties for sentence-break characters, and filter low-score generations via averaged beam log-probabilities.
Key results
- Ours achieves 0.85 overall accuracy (0.85 for new CK) with 0.89 novelty and 319K tuples after filtering (384K before).
- Ours scores 69.45 on the GPT-3.5 scoring metric versus 52.56 for full fine-tuning and 34.47 for pre-training.
- Full fine-tuning baseline reaches 0.48 accuracy, far below the proposed pipeline's 0.85.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed prefix-tuning, p-DBS, and filtering pipeline instead of Fine-tune | GPT2-xl CKGC · ConceptNet CKGC | accuracy: 0.48 (0.49) -> 0.85 (0.85) | Helped |
| Use the proposed prefix-tuning, p-DBS, and filtering pipeline instead of Fine-tune | GPT2-xl CKGC · ConceptNet CKGC | score: 52.56 -> 69.45 | Helped |
| Use the proposed prefix-tuning, p-DBS, and filtering pipeline instead of Fine-tune | GPT2-xl CKGC · ConceptNet CKGC | novelty: 0.89 -> 0.89 | No clear effect |
Limitations
- Efficiency gains are asserted but not quantified with timing or parameter counts.
- Evaluation relies on automatic metrics and GPT-3.5 scoring, with no human evaluation reported.
- Filter threshold beta and penalty coefficient alpha values are not specified in the extract.
Experimental setup
Datasets: ConceptNet
Benchmarks: ConceptNet
Metrics: accuracy novelty quantity score
Baselines: LAMA COMET BertNet Pre-train Fine-tune
Builds on: COMET prefix-tuning diverse beam search BertNet
Closest papers by meaning
- Modeling Knowledge Graphs with Composite Reasoning (AAAI 2024 · similarity 0.87)
- A Systematic Exploration of Knowledge Graph Alignment with Large Language Models in Retrieval Augmented Generation (AAAI 2025 · similarity 0.86)
- S²DN: Learning to Denoise Unconvincing Knowledge for Inductive Knowledge Graph Completion (AAAI 2025 · similarity 0.86)
- KGGen: Extracting Knowledge Graphs from Plain Text with Language Models (NeurIPS 2025 · similarity 0.86)
- Simple is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation (ICLR 2025 · similarity 0.86)
- Think-on-Graph 2.0: Deep and Faithful Large Language Model Reasoning with Knowledge-guided Retrieval Augmented Generation (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.