Research map / Video / Video-language understanding
MemoryFusion: Cross-Temporal Memory Learning for Multimodal Video Fusion
Gong Meiqi, Hao Zhang, Jiayi Ma
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MemoryFusion, a two-stage multimodal video fusion framework using cross-temporal memory learning over historical frames at multiple temporal scales to improve robustness and temporal consistency.
Problem
Multimodal video fusion methods underperform because they insufficiently exploit historical frames and use simplified temporal consistency modeling.
Approach
Builds a two-stage framework on cross-temporal memory learning that leverages historical information across multiple temporal scales and progressively refines temporal consistency.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Temporal Smoothing Module (ablation) | MemoryFusion stage 1 · M3SVD | flowD: 4.2172 -> 3.6135 | Improved |
| add Temporal Smoothing Module (ablation) | MemoryFusion stage 1 · M3SVD | VIF: 0.4943 -> 0.4877 | No clear effect |
| add Short-Term Memory Bank (ablation) | MemoryFusion stage 1 · M3SVD | VIF: 0.4817 -> 0.4877 | Improved |
| add Short-Term Memory Bank (ablation) | MemoryFusion stage 1 · M3SVD | flowD: 3.6586 -> 3.6135 | No clear effect |
| use dual-criterion LTMB update instead of random update (ablation) | MemoryFusion stage 1 · M3SVD | VIF: 0.4825 -> 0.4877 | Improved |
| add Long-Term Memory Bank (ablation) | MemoryFusion stage 1 · M3SVD | MI: 2.0031 -> 2.1631 | Improved |
| add Long-Term Memory Bank (ablation) | MemoryFusion stage 1 · M3SVD | CC: 0.4646 -> 0.4872 | Improved |
| add temporal loss in stage 2 (ablation) | MemoryFusion · M3SVD | flowD: 3.2065 -> 2.6759 | Improved |
| use 3D convolution instead of 2D in stage 2 (ablation) | MemoryFusion · M3SVD | flowD: 3.5290 -> 2.6759 | Improved |
| add stage-2 Temporal Refinement Module (ablation) | MemoryFusion · M3SVD | flowD: 3.6135 -> 2.6759 | Improved |
| apply TRM plug-and-play | MMIF-EMMA · M3SVD | flowD: 4.6592 -> 2.9549 | Improved |
| apply TRM plug-and-play | UniVF · M3SVD | flowD: 4.0271 -> 2.8025 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 2 of 2 on most shared results.
- TemCoCo — TemCoCo: Temporally Consistent Multi-modal Video Fusion with Visual-Semantic Collaboration (ICCV 2025) · better on 4 of 4 shared results
- VideoFusion — VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation (CVPR 2023) · better on 2 of 2 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- A Unified Solution to Video Fusion: From Multi-Frame Learning to Benchmarking (NeurIPS 2025 · similarity 0.87)
- TRIP: Temporal Residual Learning with Image Noise Prior for Image-to-Video Diffusion Models (CVPR 2024 · similarity 0.85)
- One-Step Diffusion for Detail-Rich and Temporally Consistent Video Super-Resolution (NeurIPS 2025 · similarity 0.85)
- Learning Temporally Consistent Video Depth from Video Diffusion Priors (CVPR 2025 · similarity 0.85)
- Improving LLM Video Understanding with 16 Frames Per Second (ICML 2025 · similarity 0.85)
- 3D-SPATIAL MULTIMODAL MEMORY (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
