Research map / Graph learning / Knowledge and temporal graphs
LAFA: Multimodal Knowledge Graph Completion with Link Aware Fusion and Aggregation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LAFA, a multimodal knowledge graph completion model whose link-aware fusion weighs each entity's images per triple and whose link-aware aggregation attends over neighbors, decoded with ConvE.
Problem
Multimodal knowledge graph completion suffers from visual noise when images are fused into entity embeddings (the importance of an entity's images varies by link) and from ignoring neighborhood structure by treating triples independently.
Approach
Computes link-conditioned attention between an entity and its images for each triple (link aware fusion) and between a central entity and its neighbors (link aware aggregation), fusing visual and structural embeddings by linear combination before a ConvE decoder.
Key results
- On FB15k-237-IMG, LAFA-ConvE reaches Hits@10 0.551 and MR 136, versus 0.504 Hits@10 and MR 221 for MKGformer.
- On WN18-IMG, LAFA-ConvE reaches Hits@10 0.977 and MR 25, improving over CompoundE's 0.972 and MR 36.
- Ablations on FB15k-237-IMG show removing modality interaction attention, link aware fusion, or link aware aggregation each lowers Hits@10 (0.538, 0.540, 0.537 vs 0.551).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LAFA with the ConvE decoder instead of CompoundE | LAFA on FB15k-237-IMG · FB15k-237-IMG | Hits@10: 0.545 -> 0.551 | Helped |
| Use LAFA with the ConvE decoder instead of CompoundE | LAFA on FB15k-237-IMG · FB15k-237-IMG | MR: 151 -> 136 | Helped |
| Use LAFA with the ConvE decoder instead of MKGformer | LAFA on WN18-IMG · WN18-IMG | Hits@10: 0.972 -> 0.977 | Helped |
| Use LAFA with the ConvE decoder instead of MKGformer | LAFA on WN18-IMG · WN18-IMG | MR: 28 -> 25 | Helped |
| Add modality interaction attention instead of traditional vector similarity matching (ablation) | LAFA · FB15k-237-IMG | Hits@10: 0.538 -> 0.551 | Helped |
| Add link-aware fusion using link information (ablation) | LAFA · FB15k-237-IMG | Hits@10: 0.540 -> 0.551 | Helped |
| Add link-aware aggregation of neighbor information (ablation) | LAFA · FB15k-237-IMG | Hits@10: 0.537 -> 0.551 | Helped |
Surprising findings
- DistMult and ComplEx decoders reduce performance relative to ConvE when used with LAFA, which the authors attribute to weakened relation importance in their scoring functions.
Limitations
- Visual noise threshold xi = 0.1 is predefined rather than learned.
- Evaluated only on two relatively small MKGC benchmarks, both with 10 images per entity.
- Gains over the best baselines are modest (roughly 1-2% on FB15k-237-IMG).
Experimental setup
Datasets: FB15k-237-IMG WN18-IMG
Benchmarks: FB15k-237-IMG WN18-IMG
Metrics: Hits@1 Hits@3 Hits@10 MR
Baselines: TransE DistMult ComplEx ConvE LTE-ConvE MRGAT GreenKGC CompoundE IKRL TransAE RSME MKGformer
Builds on: ConvE IKRL MKGformer RSME
Closest papers by meaning
- APKGC: Noise-enhanced Multi-Modal Knowledge Graph Completion with Attention Penalty (AAAI 2025 · similarity 0.89)
- A Dual-Way Enhanced Framework from Text Matching Point of View for Multimodal Entity Linking (AAAI 2024 · similarity 0.88)
- Beyond Entities: A Large-Scale Multi-Modal Knowledge Graph with Triplet Fact Grounding (AAAI 2024 · similarity 0.87)
- Mixed-Curvature Multi-Modal Knowledge Graph Completion (AAAI 2025 · similarity 0.86)
- Tokenization, Fusion, and Augmentation: Towards Fine-grained Multi-modal Entity Representation (AAAI 2025 · similarity 0.86)
- Multiple Heads are Better than One: Mixture of Modality Knowledge Experts for Entity Representation Learning (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.