Research map / Video / Video segmentation and tracking
XTSFormer: Cross-Temporal-Scale Transformer for Irregular-Time Event Prediction in Clinical Applications
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes XTSFormer, a transformer neural temporal point process with a feature-based cycle-aware time positional encoding and cross-temporal-scale attention over a bottom-up clustered time hierarchy, for irregular clinical event prediction.
Problem
Predicting next event type and time in irregularly timed clinical event sequences (EHR medication and provider logs) that have cycles, periodicity, and multi-scale interactions, while keeping attention computationally tractable on long sequences.
Approach
Combines a learnable, event-type-conditioned Fourier-style time encoding (FCPE) with a multi-scale hierarchy built by Ward agglomerative clustering of event times, so each node attends only to keys at its own scale level, reducing attention cost to roughly O(L log L).
Key results
- On Medications, XTSFormer reaches 33.5% accuracy and 1.12 RMSE versus 30.2% accuracy and 1.57 RMSE for the best baseline A-NHP.
- On Providers, XTSFormer reaches 43.9% accuracy and 1.75 NLL versus 38.9% accuracy for A-NHP.
- Ablation on Medications shows FCPE plus multi-scale plus Weibull gives 33.5% accuracy vs 25.2% for base PE without multi-scale and exponential distribution.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace the base positional encoding with FCPE (ablation) | XTSFormer with multi-scale attention and Weibull distribution · Medications and Providers | accuracy: 29.3 -> 33.5; 40.2 -> 43.9 | Helped |
| replace the base positional encoding with FCPE (ablation) | XTSFormer with multi-scale attention and Weibull distribution · Medications and Providers | RMSE: 1.65 -> 1.12; 2.51 -> 2.33 | Helped |
| replace the base positional encoding with FCPE (ablation) | XTSFormer with multi-scale attention and Weibull distribution · Medications and Providers | NLL: 2.31 -> 2.23; 1.89 -> 1.75 | Helped |
| add multi-scale attention (ablation) | XTSFormer with base positional encoding and exponential distribution · Medications and Providers | accuracy: 25.2 -> 28.3; 36.7 -> 37.9 | Helped |
| add multi-scale attention (ablation) | XTSFormer with FCPE and Weibull distribution · Medications and Providers | accuracy: 28.9 -> 33.5; 39.6 -> 43.9 | Helped |
| use Weibull instead of Exponential time distribution (ablation) | XTSFormer with FCPE and multi-scale attention · Medications and Providers | accuracy: 30.9 -> 33.5; 38.1 -> 43.9 | Helped |
| use Weibull instead of Exponential time distribution (ablation) | XTSFormer with FCPE and multi-scale attention · Medications and Providers | RMSE: 1.15 -> 1.12; 2.40 -> 2.33 | Mixed |
| use Weibull instead of Exponential time distribution (ablation) | XTSFormer with FCPE and multi-scale attention · Medications and Providers | NLL: 2.20 -> 2.23; 1.81 -> 1.75 | Mixed |
| use FCPE instead of Mercer encoding (ablation) | XTSFormer · Medications and Providers | accuracy: 31.5 -> 33.5; 41.2 -> 43.9 | Helped |
| use FCPE instead of Mercer encoding (ablation) | XTSFormer · Medications and Providers | RMSE: 1.45 -> 1.12; 2.49 -> 2.33 | Helped |
| set the largest scale to S=1, eliminating multiple scales (ablation) | XTSFormer · Medications and Providers | accuracy | Hurt |
Surprising findings
- SAHP has a worse RMSE than other transformer baselines, which the authors attribute to the absence of self-exciting patterns in the EHR data.
- Main-text and appendix results are inconsistent for XTSFormer (e.g., Medications accuracy 33.5 in main table vs 30.9 for FCPE+Exponential in ablation; MIMIC-IV NLL 4.5 vs others around 8).
Limitations
- Predicts one event at a time; multi-step prediction requires repeating the hierarchical clustering, adding preprocessing cost.
- Multi-scale hierarchy is built by a preprocessing clustering step rather than learned end-to-end.
- Private hospital datasets (Medications, Providers) are not released, limiting reproducibility.
Experimental setup
Datasets: Medications (private university hospital EHR) Providers (private university hospital EHR) MIMIC-IV emar (hosp/emar.csv)
Benchmarks: Medications Providers MIMIC-IV
Metrics: accuracy macro F1-score RMSE NLL
Baselines: Hawkes Processes (HP) RMTPP CT-LSTM NJSDE ODETPP SAHP THP A-NHP
Builds on: Transformer Hawkes Process Self-Attentive Hawkes Process Bochner's theorem time kernel (Mercer/time2vec-style encoding) Weibull distribution intensity
Closest papers by meaning
- One Loss to Rule Them All: Marked Time-to-Event for Structured EHR Foundation Models (NeurIPS 2026 · similarity 0.88)
- MOTOR: A Time-to-Event Foundation Model For Structured Medical Records (ICLR 2024 · similarity 0.86)
- Risk Horizons: Structured Hypothesis Spaces for Longitudinal Clinical Prediction (NeurIPS 2026 · similarity 0.85)
- Context Clues: Evaluating Long Context Models for Clinical Prediction Tasks on EHR Data (ICLR 2025 · similarity 0.85)
- Medformer: A Multi-Granularity Patching Transformer for Medical Time-Series Classification (NeurIPS 2024 · similarity 0.84)
- Time-to-Event Pretraining for 3D Medical Imaging (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.