Research map / Time series / Foundation models for time series
REBAR: Retrieval-Based Reconstruction for Time-series Contrastive Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes REBAR, a learned retrieval-based reconstruction similarity that selects positive pairs for time-series contrastive learning, achieving state-of-the-art linear-probe results on HAR, PPG, and ECG.
Problem
Positive pair construction for time-series contrastive learning is hard because standard augmentations (shuffling, scaling) can destroy signal semantics and time-series invariances are not well understood.
Approach
Use a learned measure, REBAR error, where one subsequence's masked context is reconstructed only from retrieved regions of a candidate subsequence via dilated-convolution cross-attention; the lowest-error candidate becomes the positive, and other candidates and other series' anchors serve as negatives.
Key results
- REBAR linear probe on HAR: 0.9535 accuracy, 0.9965 AUROC, 0.9891 AUPRC, matching fully supervised accuracy and exceeding all contrastive baselines.
- REBAR on PPG: 0.4138 accuracy and 0.6977 AUROC, beating TS2Vec (0.4023 / 0.6428) and the fully supervised model's AUROC (0.6241).
- REBAR on ECG: 0.8154 accuracy vs 0.7814 for fully supervised, but AUROC 0.9146 and AUPRC 0.8985 trail the supervised model's 0.9329 and 0.9260.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use REBAR instead of the strongest listed baseline | linear-probe classification · HAR | AUROC: 0.9938 -> 0.9965 | Helped |
| use REBAR instead of the strongest listed baseline | linear-probe classification · PPG | AUROC: 0.6456 -> 0.6977 | Helped |
| use REBAR instead of the fully supervised model | linear-probe classification · ECG | Accuracy: 0.7814 -> 0.8154 | Helped |
| use REBAR instead of the fully supervised model | linear-probe classification · ECG | AUROC: 0.9329 -> 0.9146 | Hurt |
| remove dilated convolutions (ablation) | REBAR · HAR | Accuracy: 0.9535 -> 0.8930 | Hurt |
| remove dilated convolutions (ablation) | REBAR · HAR | NMI: 0.7721 -> 0.5094 | Hurt |
| remove RevIN (ablation) | REBAR · HAR | Accuracy: 0.9535 -> 0.9451 | Hurt |
| add positional embedding (ablation) | REBAR · HAR | NMI: 0.7721 -> 0.6522 | Hurt |
| train REBAR cross-attention for 1 epoch (ablation) | REBAR · HAR | NMI: 0.7721 -> 0.5213 | Hurt |
| use intermittent masks for REBAR training instead of extended masks (ablation) | REBAR validation experiment · REBAR validation experiment | Accuracy: 0.4580 -> 0.2888 | Hurt |
| set α=0.5 instead of α=0 (ablation) | REBAR · HAR | NMI: 0.7721 -> 0.6273 | Hurt |
| set α=1 instead of α=0 (ablation) | REBAR · HAR | NMI: 0.7721 -> 0.4430 | Hurt |
Surprising findings
- Lower reconstruction error does not reliably track downstream performance; REBAR trained for more epochs reduces reconstruction error but downstream results are robust across epochs.
- Replacing the dilated convolutions with a vanilla linear layer drops HAR accuracy by 6.4% and NMI by 34.1%, though vanilla cross-attention still beats some baselines.
Limitations
- Reconstruction error is not a valid distance metric (not symmetric, no triangle inequality).
- Alpha weighting between within- and between-series losses must be chosen per modality and trades off accuracy against clusterability.
- Evaluated on only three datasets with a fixed encoder architecture; ECG AUROC/AUPRC trail the fully supervised model.
Experimental setup
Datasets: HAR (raw accelerometer and gyroscope) WESAD PPG MIT-BIH Atrial Fibrillation
Benchmarks: HAR PPG (WESAD) ECG (MIT-BIH AF)
Metrics: Accuracy AUROC AUPRC ARI NMI
Baselines: TS2Vec TNC CPC SimCLR Sliding-MSE Fully Supervised
Builds on: NT-Xent PulseImpute RevIN TS2Vec TNC
Closest papers by meaning
- On the Similarities of Embeddings in Contrastive Learning (ICML 2025 · similarity 0.87)
- TimesURL: Self-Supervised Contrastive Learning for Universal Time Series Representation Learning (AAAI 2024 · similarity 0.86)
- $\mathbb{X}$-Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphs (ICLR 2025 · similarity 0.86)
- Retrieval-Enhanced Contrastive Vision-Text Models (ICLR 2024 · similarity 0.86)
- TRACE: Contrastive learning for multi-trial time series data in neuroscience (NeurIPS 2025 · similarity 0.85)
- Joint‑Embedding vs Reconstruction: Provable Benefits of Latent Space Prediction for Self‑Supervised Learning (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.