Research map / Vision-language and multimodal / Multimodal LLMs
CoMem: Compositional Concept-Graph Memory for Vision–Language Adaptation
Heng Zhou, Jing Tang, Jusheng zhang, Yanshu Li, Canran Xiao, Liwei Hou, Zong Ke, Jiawei Yao
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CoMem, a compositional concept-graph memory for continual vision-language adaptation that avoids storing raw data under privacy and memory budgets.
Problem
Continual vision-language learning from non-stationary streams under privacy and memory constraints, where naive finetuning causes forgetting and harms transfer.
Approach
Uses a compositional concept-graph memory in place of raw-data replay to sustain stable yet plastic vision-language capability; details beyond the truncated abstract are unavailable.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use relation-aware MMD for the feature generator instead of vanilla RBF (ablation) | CoMem · Cross-domain retrieval | Avg mR: 75.7 -> 76.6 | Improved |
| Add entropy gate to replay distillation (ablation) | CoMem · Cross-domain retrieval | Average forgetting (AF): 2.8 -> 1.9 | Improved |
| Add compositional consistency loss (PoE + relation contrast) (ablation) | CoMem · Cross-domain retrieval | Avg mR: 74.9 -> 76.6 | Improved |
| Use stop-gradient so replay loss does not update the generator (ablation) | CoMem · Cross-domain retrieval | Average forgetting (AF): 2.6 -> 1.9 | Improved |
| Sample subgraphs with k-DPP + Steiner + MH instead of uniform (ablation) | CoMem · Cross-domain retrieval | Avg mR: 75.2 -> 76.6 | Improved |
| Add Metropolis-Hastings accept step after k-DPP + Steiner (ablation) | CoMem · Cross-domain retrieval | Avg mR: 76.2 -> 76.6 | Improved |
| Use frozen teacher with shared low-rank verifier instead of student verifier or dense per-concept projections (ablation) | CoMem · Cross-domain retrieval | Average forgetting (AF): 2.5 -> 1.9 | Improved |
| Add edge anchors to node anchors in replay (ablation) | CoMem · SVLC relation families and VQA relation skills | AUROC / accuracy: +1.7 pp AUROC, +1.6 pp Acc | Improved |
| Use fixed relation schema instead of dynamic relation schema (ablation) | CoMem · Cross-domain retrieval | Avg mR: 73.4 -> 76.6 | Improved |
| Use manually curated data instead of automated parsing (ablation) | CoMem · Cross-domain retrieval | Avg mR: 76.6 -> 76.2 | No clear effect |
| Increase anchor budget from 8K to 64K tokens (ablation) | CoMem · Cross-domain retrieval | Avg mR: 75.8 -> 76.7 | Improved |
| Scale backbone from ViT-B/16 to ViT-L/14 (ablation) | CoMem · Cross-domain retrieval | Avg mR: +0.6 to +1.1 pp | Improved |
Closest papers by meaning
- Towards General Continuous Memory for Vision-Language Models (NeurIPS 2025 · similarity 0.89)
- VisMem: Latent Vision Memory Unlocks Potential of Vision-Language Models (CVPR 2026 · similarity 0.88)
- Continual Vision-Language Retrieval via Dynamic Knowledge Rectification (AAAI 2024 · similarity 0.87)
- MemEIC: A Step Toward Continual and Compositional Knowledge Editing (NeurIPS 2025 · similarity 0.87)
- Dynamic Cross-Modal Prompt Generation for Multimodal Continual Instruction Tuning (NeurIPS 2026 · similarity 0.87)
- Too Late to Recall: Explaining the Two-Hop Problem in Multimodal Knowledge Retrieval (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
