Research map / Video / Video generation
MemLearner: Learning to Query Context Memory for Video World Models
Jiwen Yu, Jianxiong Gao, Jianhong Bai, Yiran Qin, Kaiyi Huang, Quande Liu, Xintao Wang, Pengfei Wan, Kun Gai, Xihui Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Q-to-P attention for query tokens (ablation) | MemLearner · collected dataset GT Comp. | PSNR: 17.27 -> 21.23 | Improved |
| add Q-to-Q attention (ablation) | MemLearner · collected dataset GT Comp. | PSNR: 21.23 -> 21.19 | No clear effect |
| add context tokens as queries in attention (ablation) | MemLearner · collected dataset GT Comp. | PSNR / fps: 21.23 -> 21.34 PSNR; 0.54 -> 0.28 fps | No clear effect |
| increase query layers from 1 to 5 (ablation) | MemLearner · collected dataset GT Comp. | PSNR: 18.16 -> 21.23 | Improved |
| increase query layers from 5 to 20 (ablation) | MemLearner · collected dataset GT Comp. | PSNR / fps: 21.23 -> 21.37 PSNR; 0.54 -> 0.36 fps | No clear effect |
| add camera pose embeddings to C and Q tokens (ablation) | MemLearner · collected dataset GT Comp. | PSNR: 21.17 -> 21.23 | No clear effect |
| init Q tokens with random noise instead of noisy P tokens (ablation) | MemLearner · collected dataset GT Comp. | PSNR: 21.17 -> 21.23 | No clear effect |
| add rendered occlusion dataset (Ours) to CaM training (ablation) | MemLearner · Revisit Comp. | PSNR: 18.35 -> 18.57 | Improved |
| train on real-world data only (ablation) | MemLearner · Revisit Comp. | PSNR: 18.57 -> 15.32 | Hurt |
| add real-world data to rendered training (ablation) | MemLearner · Real Quality | FID: 161.83 -> 115.47 | Improved |
| use query module sharing the video model instead of a separate from-scratch module (ablation) | Separate Module · collected dataset GT Comp. | PSNR: 9.16 -> 21.23 | Improved |
| use separately pre-trained query module with L1 supervision instead of end-to-end (ablation) | MemLearner · collected dataset GT Comp. | PSNR: 21.23 -> 16.56 | Hurt |
Closest papers by meaning
- Learning World Models for Interactive Video Generation (NeurIPS 2025 · similarity 0.89)
- Video World Models with Long-term Spatial Memory (NeurIPS 2025 · similarity 0.89)
- WorldMem: Long-term Consistent World Simulation with Memory (NeurIPS 2025 · similarity 0.87)
- Re-thinking Temporal Search for Long-Form Video Understanding (CVPR 2025 · similarity 0.87)
- Addressable Memory for Video World Models (NeurIPS 2026 · similarity 0.87)
- StateSpaceDiffuser: Bringing Long Context to Diffusion World Models (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
