Research map / Vision-language and multimodal / Cross-modal learning
DOGE: LLMs-Enhanced Hyper-Knowledge Graph Recommender for Multimodal Recommendation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DOGE, which uses a multimodal LLM to generate image-aware semantic item features and a frozen hyper-knowledge graph (item co-occurrence, hyper-relations, user similarity) on top of LightGCN-style propagation for multimodal recommendation.
Problem
Multimodal recommenders underuse image features, over-rely on text similarity when propagating item-item signals, and neglect hyper-relations among items, users and modalities.
Approach
Prompt a multimodal LLM with item image and text to produce a semantic modality embedding that links text and image, then build a hyper-knowledge graph from user-item co-occurrence, top-K hyperedge subsets, and user similarity, propagating over heterogeneous and homogeneous graphs.
Key results
- DOGE reaches R@20 of 0.1100 on Baby, 0.0677 on Kitchen, and 0.0718 on Electronics, versus DRAGON's 0.1021, 0.0655, 0.0678.
- The authors report an average 7.2% improvement over the strongest baseline DRAGON across datasets.
- Ablation: removing the HKG hurts most on Electronics; removing semantic modality hurts most on Kitchen.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DOGE instead of DRAGON | Multimodal recommendation · Baby | R@20: 0.1021 -> 0.11 | Helped |
| Use DOGE instead of DRAGON | Multimodal recommendation · Kitchen | R@20: 0.0655 -> 0.0677 | Helped |
| Use DOGE instead of DRAGON | Multimodal recommendation · Electronics | R@20: 0.0678 -> 0.0718 | Helped |
| Add LLM-generated semantic modality features (ablation) | DOGE · Baby | R@20 | Helped |
| Add LLM-generated semantic modality features (ablation) | DOGE · Kitchen | R@20 | Helped |
| Add LLM-generated semantic modality features (ablation) | DOGE · Electronics | R@20 | Helped |
| Add the hyper-knowledge graph (HKG) (ablation) | DOGE · Baby | R@20 | Helped |
| Add the hyper-knowledge graph (HKG) (ablation) | DOGE · Kitchen | R@20 | Helped |
| Add the hyper-knowledge graph (HKG) (ablation) | DOGE · Electronics | R@20 | Helped |
| Increase the non-text modality proportion in the item homogeneous graph (ablation) | DOGE · Baby, Kitchen, and Electronics | R@20 and N@20 | Mixed |
Limitations
- Baselines LATTICE and MICRO could not be run on Kitchen and Electronics on a single RTX 4090, leaving those cells empty.
- Multimodal LLM used for semantic features is not named in the text.
- Hyperparameter grid for item-item weights is tuned on test-facing validation trends; reported gains are modest on some metrics.
Experimental setup
Datasets: Amazon Baby Amazon Home and Kitchen Amazon Electronics
Benchmarks: Amazon Baby Amazon Kitchen Amazon Electronics
Metrics: Recall@10 Recall@20 NDCG@10 NDCG@20
Baselines: LightGCN VBPR MMGCN DualGNN GRCN LATTICE SLMRec BM3 MICRO FREEDOM MGCN DRAGON
Builds on: FREEDOM DRAGON LightGCN MMRec BPR
Closest papers by meaning
- LGMRec: Local and Global Graph Learning for Multimodal Recommendation (AAAI 2024 · similarity 0.90)
- Beyond Graph Convolution: Multimodal Recommendation with Topology-aware MLPs (AAAI 2025 · similarity 0.89)
- Harnessing Multimodal Large Language Models for Multimodal Sequential Recommendation (AAAI 2025 · similarity 0.88)
- Bridging the User-side Knowledge Gap in Knowledge-aware Recommendations with Large Language Models (AAAI 2025 · similarity 0.88)
- Seeing Beyond Noise: Joint Graph Structure Evaluation and Denoising for Multimodal Recommendation (AAAI 2025 · similarity 0.88)
- MENTOR: Multi-level Self-supervised Learning for Multimodal Recommendation (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.