Research map / Time series / Causal and generative time series
Toward Interpretable Evaluation Measures for Time Series Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes WARI, a distance-to-boundary weighted Adjusted Rand Index, and SMS, a state-matching score that classifies segmentation errors into delay, isolation, transition and missing types, and evaluates six time series state detection methods on five real datasets.
Problem
Existing time series segmentation evaluation measures (F1, covering, ARI/NMI) either focus only on change point accuracy or are point-based, so they ignore error position and type and offer limited interpretability.
Approach
WARI weights each timestamp by its distance to the nearest ground-truth change point so that interior errors are penalized more; SMS maps predicted states to real states via Hungarian matching, then classifies error blocks by atomicity into four types and penalizes each with type-specific weights.
Key results
- SMS and WARI penalize isolated errors more than delay errors, while ARI is insensitive to error position and type on synthetic tests.
- On real data, ClaSP ranks highest on univariate datasets and Time2State outperforms others on multivariate datasets under the proposed measures.
- Under random penalty weights in [0,1] across 100 runs, SMS showed an average standard deviation of 0.03169, indicating robustness to weight choice.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SMS rather than ARI to rank the MoCap segmentation example | E2USD and Time2State · MoCap | segmentation quality ranking | Helped |
| Use SMS to evaluate segmentation algorithms on multivariate datasets | Time series segmentation methods · Multivariate datasets | ranking | Helped |
| Use ClaSP for state detection on univariate datasets | Time series segmentation methods · Univariate datasets | ranking | Helped |
| Vary SMS error-type penalty weights randomly across 100 runs | SMS · All five datasets and six algorithms | score standard deviation: 0.03169 | No clear effect |
| Use SMS instead of ARI when predictions contain a single segment | TICC · UCR-SEG | score | Helped |
| Use SMS rather than ARI for segmentations with temporal misalignment | Time series segmentation methods · Real-world datasets | score | Helped |
Surprising findings
- SMS is not adjusted for chance and may overvalue trivial single-segment predictions.
- ARI assigns near-zero scores to single-segment predictions where SMS still gives partial credit.
Limitations
- WARI does not satisfy the error-type property (P3) and interpretability property (P4), per the authors' own checklist.
- SMS is not adjusted for chance and can overvalue simplistic segmentations.
- SMS and WARI scores depend on user-chosen parameters (alpha, penalty weights).
Experimental setup
Datasets: PAMAP2 USC-HAD UCR-SEG ActRecTut MoCap
Benchmarks: PAMAP2 USC-HAD UCR-SEG ActRecTut MoCap
Metrics: WARI SMS ARI NMI WNMI F1 covering
Baselines: ARI NMI AMI F1 score covering score WARI NMI-based WNMI
Builds on: Adjusted Rand Index covering score ClaSP Time2State E2USD
Closest papers by meaning
- Detect, Explain, Interpret: An End-to-End Benchmark for Time Series Anomaly Detection, Explainability and Interpretability. (NeurIPS 2026 · similarity 0.83)
- InteDisUX: Intepretation-Guided Discriminative User-Centric Explanation for Time Series (AAAI 2025 · similarity 0.83)
- The Elephant in the Room: Towards A Reliable Time-Series Anomaly Detection Benchmark (NeurIPS 2024 · similarity 0.82)
- Segment, Shuffle, and Stitch: A Simple Layer for Improving Time-Series Representations (NeurIPS 2024 · similarity 0.82)
- Large Pre-trained time series models for cross-domain Time series analysis tasks (NeurIPS 2024 · similarity 0.82)
- ShapeX: Shapelet-Driven Post Hoc Explanations for Time Series Classification Models (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.