Research map / Vision-language and multimodal / Cross-modal learning
Multiple Heads are Better than One: Mixture of Modality Knowledge Experts for Entity Representation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MoMoK, a mixture-of-modality-knowledge-experts framework with relation-guided gated fusion, multi-modal joint decision, and CLUB-based expert disentanglement for multi-modal knowledge graph completion.
Problem
Multi-modal knowledge graph completion (MMKGC) fuses entity modality features at the entity level without considering relational context, yielding static entity embeddings that underuse modality information, especially under noisy or missing modalities.
Approach
Relation-guided modality knowledge experts build multiple per-modality embeddings gated by the current relation, a joint decision module fuses modalities into an extra joint modality scored with Tucker, and CLUB mutual-information minimization disentangles the experts within each modality.
Key results
- MoMoK reaches 35.89 MRR / 30.38 Hit@1 on MKG-W and 39.57 MRR / 32.38 Hit@1 on DB15K, outperforming the best baselines.
- On KVC16K, MoMoK achieves 16.87 MRR vs 15.90 for TuckER and 15.26 for AdaMF (the second-best MRR).
- Ablation shows joint training is the most impactful component; removing it drops DB15K MRR from 39.57 to 37.62.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MoMoK instead of MMRNS | MMKGC · MKG-W | MRR: 35.03 -> 35.89 | Helped |
| Use MoMoK instead of AdaMF | MMKGC · MKG-Y | MRR: 38.06 -> 37.91 | Hurt |
| Use MoMoK instead of MMRNS | MMKGC · DB15K | MRR: 32.68 -> 39.57 | Helped |
| Use MoMoK instead of TuckER | MMKGC · KVC16K | MRR: 15.90 -> 16.87 | Helped |
| Add relational temperature (ablation) | MoMoK · MKG-W | MRR: 35.50 -> 35.89 | Helped |
| Add tunable noise (ablation) | MoMoK · MKG-W | MRR: 35.31 -> 35.89 | Helped |
| Add adaptive fusion (ablation) | MoMoK · MKG-W | MRR: 35.34 -> 35.89 | Helped |
| Add joint training (ablation) | MoMoK · MKG-W | MRR: 32.73 -> 35.89 | Helped |
| Add ExID (ablation) | MoMoK · MKG-W | MRR: 34.99 -> 35.89 | Helped |
| Add structure modality to single-modality use (ablation) | MoMoK · MKG-W | MRR: 32.82 -> 35.89 | Helped |
| Add image modality to single-modality use (ablation) | MoMoK · MKG-W | MRR: 32.75 -> 35.89 | Helped |
| Add text modality to single-modality use (ablation) | MoMoK · MKG-W | MRR: 32.62 -> 35.89 | Helped |
Surprising findings
- Per-modality scores alone (structure, image, text) are each weaker than the full model, but the joint modality score alone is close to the full model on MKG-W.
Limitations
- Restricted to embedding-based MMKGC with image and text modalities; not integrated with LLMs.
- Experiments limited to medium-sized benchmarks.
- Gains on some metrics (e.g., MKG-Y MRR) are not best; performance depends on dataset and hyperparameters K and lambda.
Experimental setup
Datasets: MKG-W MKG-Y DB15K KVC16K FB15K-237
Benchmarks: MKG-W MKG-Y DB15K KVC16K FB15K-237
Metrics: MRR Hit@1 Hit@3 Hit@10
Baselines: TransE DistMult ComplEx RotatE PairRE TuckER IKRL TBKGC TransAE RSME MMKRL VBKGC
Builds on: TuckER CLUB MoSE IMF
Closest papers by meaning
- Tokenization, Fusion, and Augmentation: Towards Fine-grained Multi-modal Entity Representation (AAAI 2025 · similarity 0.92)
- Tackling Uncertain Correspondences for Multi-Modal Entity Alignment (NeurIPS 2024 · similarity 0.90)
- Beyond Entities: A Large-Scale Multi-Modal Knowledge Graph with Triplet Fact Grounding (AAAI 2024 · similarity 0.87)
- LAFA: Multimodal Knowledge Graph Completion with Link Aware Fusion and Aggregation (AAAI 2024 · similarity 0.86)
- MESED: A Multi-Modal Entity Set Expansion Dataset with Fine-Grained Semantic Classes and Hard Negative Entities (AAAI 2024 · similarity 0.85)
- Prototype-Guided Multimodal Relation Extraction based on Entity Attributes (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.