Research map / Time series / Foundation models for time series
Vector Quantization Pretraining for EEG Time Series with Random Projection and Phase Alignment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VQ-MTM, a BERT-style masked time-series model for EEG that uses a parameter-free random-projection quantizer plus a Fourier phase-alignment module to produce semantic pseudo-labels for pretraining.
Problem
Self-supervised pretraining for EEG time series lacks well-defined semantic units (tokens); EEG is noisy/corrupted and periodic, so phase-shifted or noisy variants get different tokens and reconstruction targets are unstable.
Approach
Combines a random-projection quantizer (generalizing Johnson-Lindenstrauss to inner products, cosine similarity on a fixed random codebook) with a phase-alignment step based on the time-phase-shift equivariance of the Fourier transform, so time-shifted variants map to the same pseudo-label for masked prediction.
Key results
- On TUSZ seizure detection (60-s clips), VQ-MTM reaches 0.904 AUROC vs. 0.834 for the second-best baseline PatchTST.
- On TUSZ seizure classification (60-s), VQ-MTM achieves 0.615 weighted F1 vs. 0.585 for MAE (5.13% improvement over the best baseline).
- On TUAB seizure detection, VQ-MTM achieves 0.868 (12-s) and 0.871 (60-s) AUROC, best among compared methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use VQ-MTM instead of PatchTST | EEG seizure detection on TUSZ, 12-S · TUSZ seizure detection 12-S | AUROC: 0.866 -> 0.887 | Helped |
| Use VQ-MTM instead of PatchTST | EEG seizure detection on TUSZ, 60-S · TUSZ seizure detection 60-S | AUROC: 0.834 -> 0.904 | Helped |
| Use VQ-MTM instead of PatchTST | EEG seizure classification on TUSZ, 12-S · TUSZ seizure classification 12-S | weighted F1-score: 0.607 -> 0.620 | Helped |
| Use VQ-MTM instead of MAE | EEG seizure classification on TUSZ, 60-S · TUSZ seizure classification 60-S | weighted F1-score: 0.585 -> 0.615 | Helped |
| Use VQ-MTM instead of TimesNet | EEG seizure detection on TUAB, 12-S · TUAB seizure detection 12-S | AUROC: 0.863 -> 0.868 | Helped |
| Use VQ-MTM instead of BIOT | EEG seizure detection on TUAB, 60-S · TUAB seizure detection 60-S | AUROC: 0.865 -> 0.871 | Helped |
| Add channel separation rather than fusing channel features before pretraining (ablation) | VQ-MTM on TUSZ seizure detection, 12-S · TUSZ seizure detection 12-S | AUROC: 0.838 -> 0.887 | Helped |
| Add channel separation rather than fusing channel features before pretraining (ablation) | VQ-MTM on TUSZ seizure detection, 60-S · TUSZ seizure detection 60-S | AUROC: 0.856 -> 0.904 | Helped |
| Add channel separation rather than fusing channel features before pretraining (ablation) | VQ-MTM on TUSZ seizure classification, 12-S · TUSZ seizure classification 12-S | weighted F1-score: 0.507 -> 0.620 | Helped |
| Add channel separation rather than fusing channel features before pretraining (ablation) | VQ-MTM on TUSZ seizure classification, 60-S · TUSZ seizure classification 60-S | weighted F1-score: 0.594 -> 0.615 | Helped |
| Add the phase-aligning module (ablation) | VQ-MTM on TUSZ seizure detection, 12-S · TUSZ seizure detection 12-S | AUROC: 0.870 -> 0.887 | Helped |
| Add the phase-aligning module (ablation) | VQ-MTM on TUSZ seizure detection, 60-S · TUSZ seizure detection 60-S | AUROC: 0.880 -> 0.904 | Helped |
Surprising findings
- Removing the phase-aligning module drops classification weighted F1 on 60-s clips from 0.615 to 0.507, a 21.3% relative gain from alignment.
- SimMTM performs worst on all tasks, attributed to memory burden from generating many positive-pair copies on large datasets.
Limitations
- Random projection and codebook are fixed after initialization and not learned, which the authors flag as a limitation.
- Evaluation focuses on EEG seizure tasks on TUH corpora; generalization to other time series is only partially checked in appendix experiments on ECG.
Experimental setup
Datasets: TUH EEG Seizure Corpus (TUSZ) TUH EEG Abnormal Corpus (TUAB)
Benchmarks: TUSZ TUAB
Metrics: AUROC weighted F1-score
Baselines: DCRNN TimesNet MAE PatchTST SimMTM BIOT
Builds on: BERT BEST-RQ random-projection quantization MAE Johnson-Lindenstrauss lemma PatchTST
Closest papers by meaning
- DMNet: Self-comparison Driven Model for Subject-independent Seizure Detection (NeurIPS 2024 · similarity 0.83)
- Long-Term EEG Partitioning for Seizure Onset Detection (AAAI 2025 · similarity 0.83)
- Standing on the Shoulders of Giants: Rethinking EEG Foundation Model Pretraining via Multi-Teacher Distillation (NeurIPS 2026 · similarity 0.83)
- Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI (ICLR 2024 · similarity 0.82)
- EEGPT: Pretrained Transformer for Universal and Reliable Representation of EEG Signals (NeurIPS 2024 · similarity 0.82)
- MindAlign: Bridging EEG, Vision, and Language for Zero-Shot Visual Decoding (NeurIPS 2026 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.