AI Paper Atlas

Research map / Continual learning / Online and replay-based

Episodic Memory in Lifelong Language Learning

Cyprien de Masson d'Autume, Sebastian Ruder, Lingpeng Kong, Dani Yogatama

NeurIPS 2019 · Continual learning › Online and replay-based · approach: Replay buffer construction and update · 96 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Combine sparse experience replay with nearest-neighbor local adaptationBERT-base lifelong learning · Text classificationaccuracy: 68.8 -> 70.6Improved
Combine sparse experience replay with nearest-neighbor local adaptationBERT-base lifelong learning · Question answeringF1: 60.3 -> 62.4Improved
Add sparse replay to the encoder-decoder learnerBERT-base lifelong learning · Text classificationaccuracy: 18.4 -> 57.8Improved
Add sparse replay to the encoder-decoder learnerBERT-base lifelong learning · Question answeringF1: 53.1 -> 57.9Improved
Use nearest neighbors rather than random memory examples for local adaptation (ablation)MbPA-ARC · Text classificationaccuracy: 58.2 -> 70.6Improved
Use nearest neighbors rather than random memory examples for local adaptation (ablation)MbPA-ARC · Question answeringF1: 59.7 -> 62.4Improved
Reduce memory capacity from 100% to 10% (ablation)MbPA-ARC · Text classificationaccuracy: 70.6 -> 67.6Hurt
Reduce memory capacity from 100% to 10% (ablation)MbPA-ARC · Question answeringF1: 62.0 -> 61.5Hurt
Reduce memory capacity from 100% to 50% (ablation)MbPA-ARC · Text classificationaccuracy: 70.6 -> 70.3Hurt
Reduce memory capacity from 100% to 50% (ablation)MbPA-ARC · Question answeringF1: 62.0 -> 61.6Hurt
Increase the number of local-adaptation neighbors from K=8 to K=128 (ablation)MbPA-ARC · Text classificationaccuracy: 68.4 -> 71.6Improved
Increase the number of local-adaptation neighbors from K=8 to K=32 (ablation)MbPA-ARC · Question answeringF1: 60.2 -> 62.0Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy