Research map / Time series / Foundation models for time series
From Teacher Pathways to Invariant Manifolds: Consensus Subspace Distillation for TSFMs
Zexing Zhang, Tianyang Lei, Jichao Li, Yang Kewei
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add uncertainty injection (K=4) (ablation) | Consensus subspace distillation · Classification (Table 4) | accuracy: 0.698 -> 0.704 | Improved |
| add uncertainty injection (K=8) (ablation) | Consensus subspace distillation · Classification (Table 4) | accuracy: 0.747 -> 0.752 | Improved |
| add uncertainty injection (K=12) (ablation) | Consensus subspace distillation · Classification (Table 4) | accuracy: 0.757 -> 0.761 | Improved |
| add uncertainty injection (K=4) (ablation) | Consensus subspace distillation · Anomaly detection Adj. Best F1 (Table 4) | Adj. Best F1: 0.645 -> 0.651 | Improved |
| add uncertainty injection (K=16) (ablation) | Consensus subspace distillation · Anomaly detection Adj. Best F1 (Table 4) | Adj. Best F1: 0.711 -> 0.716 | Improved |
| increase copied layers K from 4 to 8 (ablation) | Consensus subspace distillation · Classification (Table 4) | accuracy: 0.704 -> 0.752 | Improved |
| increase copied layers K from 8 to 12 (ablation) | Consensus subspace distillation · Classification (Table 4) | accuracy: 0.752 -> 0.761 | Improved |
| increase copied layers K from 12 to 16 (ablation) | Consensus subspace distillation · Classification (Table 4) | accuracy: 0.761 -> 0.758 | No clear effect |
| consensus subspace distillation instead of CRD | Student at 9.87% parameters · Forecasting horizon 96 | MAE: 0.344 -> 0.287 | Improved |
| consensus subspace distillation instead of CRD | Student at 9.87% parameters · Zero-shot classification | accuracy: 0.610 -> 0.633 | Improved |
| consensus subspace distillation instead of SliceGPT | MOMENT-Large compression · Long-horizon forecasting | MAE: 0.528 -> 0.471 | Improved |
| consensus subspace distillation instead of SliceGPT | MOMENT-Large compression · Zero-shot classification (Table 1) | accuracy: 0.685 -> 0.633 | Hurt |
Closest papers by meaning
- Exploring Representations and Interventions in Time Series Foundation Models (ICML 2025 · similarity 0.85)
- Asymmetric Decision-Making in Online Knowledge Distillation: Unifying Consensus and Divergence (ICML 2025 · similarity 0.85)
- Over-parameterized Student Model via Tensor Decomposition Boosted Knowledge Distillation (NeurIPS 2024 · similarity 0.85)
- Towards Neural Scaling Laws for Time Series Foundation Models (ICLR 2025 · similarity 0.85)
- Specialized Foundation Models Struggle to Beat Supervised Baselines (ICLR 2025 · similarity 0.85)
- Less or More From Teacher: Exploiting Trilateral Geometry For Knowledge Distillation (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
