Research map / Architectures and efficiency / Attention and transformers
TPP-SD: Accelerating Transformer Point Process Sampling with Speculative Decoding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adapts speculative decoding to Transformer temporal point process sampling by drafting candidate events with a small CDF-based TPP and verifying them in parallel with a large target model, yielding exact-distribution sampling with 1.3–5.9x wall-time speedups.
Problem
Sampling from Transformer temporal point processes via the thinning algorithm is sequential and slow because each candidate event requires a full Transformer forward pass.
Approach
Identifies structural similarity between thinning and speculative decoding, then builds a CDF-based Transformer TPP (log-normal mixture interval decoder plus categorical type decoder) so that a small draft TPP proposes several events verified in parallel by a large target TPP, with an adjusted-distribution resampling step (including an acceptance-rejection scheme for the continuous interval) that
Key results
- TPP-SD achieves speedups of 2.1x-5.7x on synthetic datasets with the AttNHP encoder and 1.3-5.9x across synthetic and real datasets (e.g., 5.743x on Hawkes with AttNHP), while matching AR sampling's KS statistics and likelihood discrepancies.
- On real datasets, TPP-SD gives Wasserstein distances close to the AR-vs-AR baseline and speedups of 1.6x-5.9x (e.g., 5.849x on Amazon with AttNHP).
- A 1-head, 1-layer draft model gives the highest speedup (4.68x on Multi-Hawkes) without degrading sample quality, while larger drafts raise acceptance rate but reduce speedup.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace autoregressive target-only sampling with TPP-SD using a draft model | Transformer TPP sampling · Synthetic datasets | speedup ratio: 1.3\text{--}5.7× | Helped |
| replace autoregressive target-only sampling with TPP-SD using a draft model | Transformer TPP sampling · Taobao, Amazon, Taxi, StackOverflow | speedup ratio: 1.6\text{--}5.9× | Helped |
| increase draft length γ (ablation) | TPP-SD with AttNHP · Synthetic and real datasets | speedup ratio | Mixed |
| increase draft length γ (ablation) | TPP-SD with AttNHP · Synthetic and real datasets | likelihood discrepancy | No clear effect |
| increase draft model size from 1-head 1-layer to 4-head 6-layer (ablation) | TPP-SD with AttNHP · Multi-Hawkes | speedup ratio: 4.680 -> 2.676 | Hurt |
| increase draft model size from 1-head 1-layer to 4-head 6-layer (ablation) | TPP-SD with AttNHP · Taobao | speedup ratio: 2.838 -> 1.845 | Hurt |
| increase draft model size (ablation) | TPP-SD · Multi-Hawkes and Taobao | acceptance rate | Helped |
| increase draft model size (ablation) | TPP-SD · Multi-Hawkes and Taobao | sampling quality | No clear effect |
Surprising findings
- Larger draft models increase acceptance rate but reduce overall speedup, so the smallest draft gives the best wall-time gain.
- Speedup is inversely related to event type cardinality; datasets with more event types (Taobao, StackOverflow) show lower speedups.
Limitations
- Speedup depends on alignment between draft and target; more event types increase rejections and reduce speedup.
- Smaller draft models that are fast struggle to model temporal dependencies accurately, creating a quality-speed tradeoff.
- Continuous-interval normalization of the adjusted distribution requires an acceptance-rejection scheme rather than direct normalization.
Experimental setup
Datasets: Poisson (synthetic) Hawkes (synthetic) Multi-Hawkes (synthetic) Taobao Amazon Taxi StackOverflow
Benchmarks: Poisson Hawkes Multi-Hawkes Taobao Amazon Taxi StackOverflow
Metrics: likelihood discrepancy KS statistic Wasserstein distance wall-time speedup ratio acceptance rate
Baselines: autoregressive sampling
Builds on: speculative decoding thinning algorithm THP AttNHP SAHP
Closest papers by meaning
- A Theoretical Perspective for Speculative Decoding Algorithm (NeurIPS 2024 · similarity 0.90)
- SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding (ICML 2026 · similarity 0.89)
- DFlash: Block Diffusion for Flash Speculative Decoding (ICML 2026 · similarity 0.88)
- Dynamic-Width Speculative Beam Decoding for LLM Inference (AAAI 2025 · similarity 0.88)
- Accelerated Speculative Sampling Based on Tree Monte Carlo (ICML 2024 · similarity 0.88)
- Accelerating Diffusion LLMs via Adaptive Parallel Decoding (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.