Research map / Time series / Causal and generative time series
Decomposable Transformer Point Processes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DTPP, which decomposes the marked point process likelihood into a log-normal mixture model for inter-event times (Markov in the last mark) and a Transformer-based categorical model for marks, enabling thinning-free inference.
Problem
Neural marked temporal point processes rely on conditional intensity functions, which require Monte Carlo integration in training and the computationally expensive thinning algorithm for sampling at inference, and thinning appears to hurt predictive accuracy.
Approach
Model the inter-event time density with a mark-conditioned log-normal mixture and the mark PMF with a continuous-time Transformer, learn the two parts separately under the decomposed likelihood, and predict deterministically without thinning.
Key results
- Next-event prediction: DTPP achieves lowest RMSE on all four reported datasets (e.g., Amazon 0.12 vs 0.38 for best baseline VI-DPP; Taxi 0.08 vs 0.11), and lowest error rates on all four.
- Long-horizon prediction on Taxi, Taobao, StackOverflow-V2: DTPP avg OTD 15.00, 28.83, 36.75 vs HYPRO-32's 18.81, 36.96, 38.84 (lower is better) while being orders of magnitude faster (e.g., 8,130x on Taxi).
- Average RMSE for CIF-based baselines is 0.58 vs 0.95 for CPDF-based ones (excluding VI-DPP), which the authors interpret as thinning harming time-prediction accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DTPP instead of the strongest reported baseline | Next-event prediction · Amazon, Taxi, Taobao, StackOverflow-V1, and MIMIC-II | RMSE: Amazon: 0.38 -> 0.12; Taxi: 0.11 -> 0.08; Taobao: 0.07 -> 0.05; StackOverflow-V1: 1.18 -> 1.07; MIMIC-II: 0.74 -> 0.72 | Helped |
| Use DTPP instead of the strongest reported baseline | Next-event prediction · Amazon, Taxi, Taobao, StackOverflow-V1, and MIMIC-II | Error: Amazon: 62.58 -> 59.06; Taxi: 7.67 -> 7.12; Taobao: 41.43 -> 40.12; StackOverflow-V1: 52.17 -> 50.41; MIMIC-II: 14.87 -> | Helped |
| Use DTPP instead of HYPRO-32 | Long-horizon prediction · Taxi and Taobao | avg OTD: Taxi: 18.81 -> 15.00; Taobao: 36.96 -> 28.83 | Helped |
| Use DTPP instead of HYPRO-32 | Long-horizon prediction · StackOverflow-V2 | RMSE*: 1.14 -> 2.02 | Hurt |
| Use DTPP instead of HYPRO-32 | Long-horizon prediction · Taxi, Taobao, and StackOverflow-V2 | Time: Taxi: 81.30 -> 0.01; Taobao: 89.11 -> 0.17; StackOverflow-V2: 83.05 -> 0.03 | Helped |
| Increase HYPRO weighted proposals from 2 to 32 (ablation) | HYPRO · Taxi, Taobao, and StackOverflow-V2 | avg OTD: Taxi: 20.35 -> 18.81; Taobao: 39.73 -> 36.96; StackOverflow-V2: 39.84 -> 38.84 | Helped |
| Increase HYPRO weighted proposals from 2 to 32 (ablation) | HYPRO · Taxi, Taobao, and StackOverflow-V2 | Time: Taxi: 44.81 -> 81.30; Taobao: 43.53 -> 89.11; StackOverflow-V2: 46.32 -> 83.05 | Hurt |
| Use DTPP instead of A-NHP | Synthetic data generated by a 1-d Hawkes process · Hawkes1-Synthetic | Next-time prediction | No clear effect |
| Use DTPP instead of A-NHP | Synthetic data generated by a 1-d Hawkes process · Hawkes1-Synthetic | Log-likelihood | Hurt |
Surprising findings
- The CPDF-based models have higher average RMSE than CIF-based baselines on next-event time prediction, contrary to the authors' expectation, though DTPP itself beats all baselines.
- DTPP was significantly outperformed by A-NHP on a 1-d Hawkes synthetic dataset where the single log-normal mixture is misspecified.
Limitations
- Transformer modeling of the mark distribution is data-hungry and may overfit in data-scarce regimes.
- Log-normal mixture with Markov assumption on inter-event times can be misspecified (e.g., 1-d Hawkes synthetic data where DTPP is outperformed by A-NHP).
Experimental setup
Datasets: MIMIC-II Amazon Taxi Taobao StackOverflow-V1 StackOverflow-V2 Hawkes1-Synthetic Hawkes2-Synthetic SAHP-Synthetic
Benchmarks: MIMIC-II Amazon Taxi Taobao StackOverflow-V1 StackOverflow-V2 Hawkes1-Synthetic Hawkes2-Synthetic SAHP-Synthetic
Metrics: RMSE error rate (%) log-likelihood optimal transport distance (OTD) RMSE* inference time
Baselines: THP SAHP A-NHP IFTPP VI-DPP HYPRO
Builds on: Attentive Neural Hawkes Process Intensity-Free Temporal Point Process VI-DPP HYPRO
Closest papers by meaning
- Addressing Mark Imbalance in Integration-free Marked Temporal Point Processes (NeurIPS 2025 · similarity 0.86)
- Transformers on Markov data: Constant depth suffices (NeurIPS 2024 · similarity 0.86)
- TPP-SD: Accelerating Transformer Point Process Sampling with Speculative Decoding (NeurIPS 2025 · similarity 0.86)
- Deep Continuous-Time State-Space Models for Marked Event Sequences (NeurIPS 2025 · similarity 0.85)
- Hyper Hawkes Processes: Interpretable Models of Marked Temporal Point Processes (NeurIPS 2026 · similarity 0.84)
- Local to Global: Learning Dynamics and Effect of Initialization for Transformers (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.