Research map / Vision-language and multimodal / Human and embodied modeling
Personal Visual Memory from Explicit and Implicit Evidence
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces a synthetic benchmark for personal visual memory (explicit entities and implicit facts) and VisualMem, a hybrid visual-text memory module plugged into the MemOS text backend, which substantially outperforms prior memory systems on the new benchmark while staying competitive on LOCOMO and PersonaMem.
Problem
Long-term memory for personalized agents is text-centric; existing memory systems reduce images to generic captions, losing identity, ownership, and cross-image evidence needed for personal visual memory.
Approach
A structured visual memory module that jointly interprets images with dialogue context, defers commitment for ambiguous images via a pending state that is revisited as evidence accumulates, and extracts relationship, entity, and fact-level memories while keeping a standard text-memory backend.
Key results
- VisualMem reaches 84.1 overall accuracy on the visual memory benchmark versus 56.0 for the strongest memory baseline MemOS.
- VisualMem scores 58.1 on LOCOMO and 46.3 on PersonaMem, versus 56.8 and 45.5 for MemOS.
- Removing the pending mechanism drops overall accuracy from 84.1 to 81.5 with full context ablation reported in the table (text-only backend ablation drops to 60.1).
Surprising findings
- Visual-only memory without text memory (80.7 overall) outperforms text-plus-visual with a size-2 window (76.9) on several categories, and text-only memory performs far worse than visual-only on person and asset questions.
Limitations
- Benchmark is synthetically generated and may not reflect real user interaction variability.
- Relies on strong multimodal models for generation and interpretation, so quality depends on their biases.
- Evaluated on only 10 personas and limited benchmarks; no study of robustness to noisy or lower-quality images.
Experimental setup
Datasets: LOCOMO PersonaMem PersonaHub
Benchmarks: VisualMem benchmark LOCOMO PersonaMem
Metrics: accuracy tokens
Baselines: Full context Oracle Self-RAG HippoRAG2 LightMem SimpleMem Mem0 MemOS
Builds on: MemOS PersonaHub
Closest papers by meaning
- MemLeak: Diagnosing Information Leaks in Multimodal Agent Memory (NeurIPS 2026 · similarity 0.89)
- Personalize-then-Store: Benchmarking and Learning Personalized Memory for Long-Horizon Agents (NeurIPS 2026 · similarity 0.88)
- Embodied Agents Meet Personalization: Investigating Challenges and Solutions Through the Lens of Memory Utilization (ICLR 2026 · similarity 0.87)
- WorldMemArena: Evaluating Multimodal Agent Memory Through Action–World Interaction (NeurIPS 2026 · similarity 0.87)
- A Framework for Inference Inspired by Human Memory Mechanisms (ICLR 2024 · similarity 0.87)
- Navigating Massive Visual Context in Retrieval-Augmented Generation via Multimodal Memory Graph (ICML 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.