Research map / Time series / Foundation models for time series
Tri-Level Navigator: LLM-Empowered Tri-Level Learning for Time Series OOD Generalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TTSO, a tri-level learning framework (sample-level perturbation, group-level reweighting, parameter-level minimization) for time series OOD generalization, solved with a stratified cutting-plane localization algorithm, and fine-tunes GPT2 with it.
Problem
Time series out-of-distribution generalization, where models trained on source domains degrade on unseen target domains (e.g., sensor-based activity recognition across users/devices).
Approach
Unifies sample-level (augmentation/perturbation via a GMM) and group-level (worst-case domain reweighting, GroupDRO-style) uncertainty in a tri-level min-max-max formulation, solved via a stratified localization algorithm that uses cutting planes to approximate the constraint set instead of hypergradients; applied to fine-tuning GPT2 with contrastive pre-training and constrained fine-tuning.
Key results
- TTSO* (GPT-2 with TTSO fine-tuning) reaches 72.8% average accuracy across HHAR, PAMAP, WESAD, versus 68.8% for the best listed baseline RSC by the table's ALL column (TTSO without LLM: 71.4%).
- On HHAR, TTSO improves over the second-best baseline by 2.8%, on PAMAP by 4.8%, and on WESAD by 4.9% per the authors.
- Ablation: pretrained GPT-2 with TTSO performs best; randomly initialized GPT-2 with TTSO improves over randomly initialized without TTSO, though less than the pretrained case.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TTSO with pretrained GPT2 fine-tuning instead of Mixup | Time-series OOD classification · HHAR leave-one-domain-out | accuracy (%): 71.9 -> 73.9 | Helped |
| Use TTSO with pretrained GPT2 fine-tuning instead of Mixup | Time-series OOD classification · PAMAP leave-one-domain-out | accuracy (%): 72.5 -> 76.1 | Helped |
| Use TTSO with pretrained GPT2 fine-tuning instead of Mixup | Time-series OOD classification · WESAD leave-one-domain-out | accuracy (%): 65.2 -> 68.4 | Helped |
| Add TTSO fine-tuning to pretrained GPT2 (ablation) | GPT2 · Time-series OOD classification | accuracy | Helped |
| Add pretraining to a randomly initialized GPT2 fine-tuned with TTSO (ablation) | GPT2 fine-tuned with TTSO · Time-series OOD classification | accuracy | Helped |
| Use pretrained GPT2 without TTSO fine-tuning instead of random initialization without TTSO fine-tuning (ablation) | GPT2 · Time-series OOD classification | accuracy | Helped |
Surprising findings
- Pretrained GPT-2 without TTSO fine-tuning still performs relatively well, which the authors read as intrinsic OOD capacity of pretrained LLMs.
Limitations
- Theoretical guarantees rely on assumptions (Lipschitz gradients, bounded gradients, convexity of the constraint after Taylor approximation) that may not hold in deep networks.
- Reported gains are modest and results are averaged over only 3 runs, with unclear statistical significance.
- Some theoretical statements (e.g., Theorem on convergence rate) are loosely stated and the constraint reformulation is approximate.
Experimental setup
Datasets: HHAR PAMAP WESAD SWELL USC-HAD DSADS
Benchmarks: HHAR PAMAP WESAD SWELL USC-HAD DSADS
Metrics: classification accuracy
Baselines: ERM IRM GroupDRO ANDMask RSC Mixup VERx DIFEX AdaRNN GILE DIVERSIFY DFDG
Builds on: GroupDRO invariant risk minimization contrastive learning for time series Layer Normalization Tuning
Closest papers by meaning
- Time-Series Forecasting for Out-of-Distribution Generalization Using Invariant Learning (ICML 2024 · similarity 0.85)
- OOD-Chameleon: Is Algorithm Selection for OOD Generalization Learnable? (ICML 2025 · similarity 0.84)
- TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Series (ICLR 2024 · similarity 0.84)
- Time-LLM: Time Series Forecasting by Reprogramming Large Language Models (ICLR 2024 · similarity 0.84)
- Towards Neural Scaling Laws for Time Series Foundation Models (ICLR 2025 · similarity 0.84)
- CRoFT: Robust Fine-Tuning with Concurrent Optimization for OOD Generalization and Open-Set OOD Detection (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.