Research map / Graph learning / Knowledge and temporal graphs
APKGC: Noise-enhanced Multi-Modal Knowledge Graph Completion with Attention Penalty
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes APKGC, a multimodal knowledge graph completion method that adds adaptive noise sampling to entity text/image inputs and an attention over-trust penalty in a BERT-based encoder, scoring triples with RotatE.
Problem
Multimodal knowledge graph completion (MMKGC) suffers from weak robustness to noisy entity multimodal data and from pre-trained language models over-trusting certain attention tokens (multimodal illusion).
Approach
Adaptively samples Gaussian or average noise into text and visual embeddings during training, and penalizes attention scores above a threshold while redistributing the removed mass to neighboring tokens, with a RotatE scoring function on top of the fused representation.
Key results
- On DB15K, APKGC reaches MRR 0.364 and Hits@1 0.282 versus the prior best NativE (0.362 MRR, 0.273 Hits@1).
- On MKG-W, APKGC reaches MRR 0.374 and Hits@1 0.306, improving Hits@1 over the best prior result (0.296 for NativE per the text's reported gap).
- Ablation on DB15K shows removing the attention penalty drops MRR from 0.374 to 0.360 and removing noise sampling drops it to 0.358.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use APKGC instead of NativE | MMKGC models · DB15K | Hits@1: 0.273 -> 0.282 | Helped |
| use APKGC instead of NativE | MMKGC models · MKG-W | Hits@1: 0.296 -> 0.306 | Helped |
| use all modalities instead of structure only (ablation) | APKGC · DB15K | Hits@1: 0.125 -> 0.282 | Helped |
| use all modalities instead of visual only (ablation) | APKGC · DB15K | Hits@1: 0.261 -> 0.282 | Helped |
| use all modalities instead of textual only (ablation) | APKGC · DB15K | Hits@1: 0.273 -> 0.282 | Helped |
| add textual information to structure and visual information (ablation) | APKGC · DB15K | Hits@1: 0.260 -> 0.282 | Helped |
| add visual information to textual and structure information (ablation) | APKGC · DB15K | Hits@1: 0.279 -> 0.282 | Helped |
| add structure information to visual and textual information (ablation) | APKGC · DB15K | Hits@1: 0.274 -> 0.282 | Helped |
| add adaptive noise sampling (ablation) | APKGC · MKG-W | Hits@1: 0.289 -> 0.306 | Helped |
| add attention penalty (ablation) | APKGC · MKG-W | Hits@1: 0.286 -> 0.306 | Helped |
| add attention increase layer (ablation) | APKGC · MKG-W | Hits@1: 0.297 -> 0.306 | Helped |
| add attention decrease layer (ablation) | APKGC · MKG-W | Hits@1: 0.302 -> 0.306 | Helped |
Limitations
- Gaussian versus average noise sampling must be chosen per dataset rather than being unified.
- Penalty ratio and window hyperparameters need tuning, and overly large values hurt performance.
- Only text and image modalities are used; video and audio are left to future work.
Experimental setup
Datasets: DB15K MKG-W
Benchmarks: DB15K MKG-W
Metrics: MRR Hits@1 Hits@3 Hits@10
Baselines: TransE DistMult ComplEx RotatE PairRE IKRL TransAE RSME VBKGC OTKGE IMF QEB
Builds on: RotatE BERT BEiT MMRNS NativE
Closest papers by meaning
- LAFA: Multimodal Knowledge Graph Completion with Link Aware Fusion and Aggregation (AAAI 2024 · similarity 0.89)
- Tokenization, Fusion, and Augmentation: Towards Fine-grained Multi-modal Entity Representation (AAAI 2025 · similarity 0.87)
- Mixed-Curvature Multi-Modal Knowledge Graph Completion (AAAI 2025 · similarity 0.86)
- GraphVis: Boosting LLMs with Visual Knowledge Graph Integration (NeurIPS 2024 · similarity 0.86)
- Beyond Entities: A Large-Scale Multi-Modal Knowledge Graph with Triplet Fact Grounding (AAAI 2024 · similarity 0.85)
- Multiple Heads are Better than One: Mixture of Modality Knowledge Experts for Entity Representation Learning (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.