Research map / Architectures and efficiency / RNNs and normalization
Sleep-Like Unsupervised Replay Improves Performance When Data Are Limited or Unbalanced (Student Abstract)
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Applies a sleep-like unsupervised replay phase (SRC with local Hebbian plasticity on a spiking mapping) to ANNs trained on small or imbalanced MNIST/Fashion-MNIST subsets, boosting accuracy in the low-data regime.
Problem
ANN performance degrades when training data are limited or class-imbalanced, whereas humans learn quickly from few examples.
Approach
Maps a backpropagation-trained fully-connected ANN to a spiking network, runs an unsupervised sleep phase with Poisson input and local Hebbian-type plasticity (sleep replay consolidation), then maps back to an ANN, applied once after limited-data training rather than across continual tasks.
Key results
- Sleep (SRC) gave a substantial 20-30% accuracy increase for models trained on 0.5-10% of MNIST or Fashion-MNIST data.
- With more than 10% of data, sleep alone reduced accuracy by about 10-15%, which fine-tuning on the original limited data largely recovered.
- On class-imbalanced training sets, most classes improved after sleep; synaptic analysis suggested gains come from increased response sparsity.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply sleep replay consolidation after training | ANNs trained with 0.5-10% of the data · MNIST and Fashion MNIST | accuracy: 20-30% | Helped |
| Apply sleep replay consolidation after training | ANNs trained with more than approximately 10% of the data · MNIST and Fashion MNIST | accuracy: 10-15% | Hurt |
| Fine-tune on the original training data after sleep replay consolidation | ANNs trained with more than approximately 10% of the data · MNIST and Fashion MNIST | accuracy | Helped |
| Apply sleep replay consolidation | ANNs trained with class-imbalanced data · MNIST and Fashion MNIST | class-wise accuracy | Mixed |
Surprising findings
- Sleep alone slightly hurt performance when more than about 10% of the data was used, and this was remedied by fine-tuning.
Limitations
- Results limited to MNIST and Fashion-MNIST with a small fully-connected network.
- Sleep alone reduces accuracy when training data are abundant; requires fine-tuning to recover.
- Applied only once after limited-data training rather than in a continual setting.
Experimental setup
Datasets: MNIST Fashion-MNIST
Benchmarks: MNIST Fashion-MNIST
Metrics: accuracy
Baselines: baseline ANN trained on limited data after sleep + fine-tuning
Builds on: sleep replay consolidation (SRC)
Closest papers by meaning
- When to Learn and When to Stop: Quitting at the Optimal Time (Student Abstract) (AAAI 2025 · similarity 0.85)
- SleepFM: Multi-modal Representation Learning for Sleep Across Brain Activity, ECG and Respiratory Signals (ICML 2024 · similarity 0.83)
- Improving Deep Learning Speed and Performance Through Synaptic Neural Balance (AAAI 2025 · similarity 0.83)
- SPICED: A Synaptic Homeostasis-Inspired Framework for Unsupervised Continual EEG Decoding (NeurIPS 2025 · similarity 0.83)
- Hebbian Learning based Orthogonal Projection for Continual Learning of Spiking Neural Networks (ICLR 2024 · similarity 0.83)
- Do Your Best and Get Enough Rest for Continual Learning (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.