Research map / Architectures and efficiency / Attention and transformers
A Plug-and-Play Bregman ADMM Module for Inferring Event Branches in Temporal Point Processes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a plug-and-play Bregman ADMM (BADMM) module that imposes sparse and low-rank structure on event transition matrices of TPPs, applied within the Hawkes EM E-step and as a replacement for self-attention in THP/SAHP.
Problem
Inferring interpretable, structured event branches (triggering relations) from event sequences in TPPs; existing responsibility matrices and attention maps are over-dense and unstructured.
Approach
Formulate event-branch inference as a KL-prior-regularized optimization over row-normalized lower-triangular transition matrices with l1 plus nuclear-norm (subspace clustering) or l1,2 (sparse group-lasso) regularization, solved by Bregman ADMM and unrolled into a differentiable module that replaces the EM E-step or the attention layer.
Key results
- BADMM_* on THP improves ACC on Conttime from 0.696 (Softmax) to 0.748 and ELL from -0.390 to -0.385.
- On HP with EM, BADMM_* raises Taobao ELL from -0.262 to 0.223 and Retweet ACC from 0.459 to 0.565.
- BADMM-enhanced SAHP and THP rank 12 Angry Men jurors with top-2 agreeing with GPT-4o (Jurors 8 and 3), and SAHP+BADMM includes Juror 4 in top-5.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add BADMM* responsibility-matrix inference to HP EM | HP · Taobao, Retweet, StackOverflow, Conttime | ELL: -0.262 -> 0.223; -9.022 -> -8.902; -3.042 -> -2.744; -1.076 -> -1.073 | Helped |
| Add BADMM* responsibility-matrix inference to HP EM | HP · Amazon | ELL: -2.561 -> -2.561 | No clear effect |
| Replace SAHP Softmax attention with BADMM* | SAHP · Taobao, Retweet, StackOverflow, Amazon, Conttime | ELL: -9.879 -> -9.594; -7.878 -> -7.686; -3.614 -> -3.374; -3.223 -> -3.127; -1.416 -> -1.235 | Helped |
| Replace THP Softmax attention with BADMM* | THP · Taobao, Retweet, Amazon, Conttime | ELL: 0.170 -> 0.193; -9.373 -> -8.753; 0.542 -> 0.543; -0.390 -> -0.385 | Helped |
| Replace THP Softmax attention with BADMM* | THP · StackOverflow | ELL: -0.698 -> -0.696 | No clear effect |
| Use nuclear-norm regularization instead of sparse group-lasso (ablation) | BADMM-enhanced HP · Taobao, Retweet, StackOverflow | ELL: 0.220 -> 0.223; -8.902 -> -8.902; -2.758 -> -2.744 | Helped |
| Use nuclear-norm regularization instead of sparse group-lasso (ablation) | BADMM-enhanced SAHP · Taobao, Retweet, StackOverflow, Amazon | ELL: -9.616 -> -9.594; -7.802 -> -7.686; -3.625 -> -3.374; -3.133 -> -3.127 | Helped |
| Use nuclear-norm regularization instead of sparse group-lasso (ablation) | BADMM-enhanced THP · Taobao, Retweet, Amazon, Conttime | ELL: 0.190 -> 0.193; -8.961 -> -8.753; 0.542 -> 0.543; -0.390 -> -0.385 | Helped |
| Use nuclear-norm regularization instead of sparse group-lasso (ablation) | BADMM-enhanced THP · StackOverflow | ELL: -0.696 -> -0.696 | No clear effect |
| Increase BADMM iteration count beyond a few iterations (ablation) | BADMM module · Figure [ref] hyperparameter evaluation | ACC | No clear effect |
Surprising findings
- Sinkhorn-based attention can degrade performance on some datasets (e.g., THP Amazon ELL drops from 0.542 to 0.288), since doubly-stochastic normalization conflicts with the TPP branching structure.
Limitations
- Attention-based BADMM module requires solving an optimization problem at each step, making inference time-consuming when predicting future events.
- Gains are often small in magnitude on some datasets and lack formal statistical grounding; theoretical connection to Bayesian priors left open.
- Ranking-based rationality evaluation relies on GPT-4o output as ground truth and a single movie dialogue.
Experimental setup
Datasets: Conttime Retweet StackOverflow Amazon Taobao 12 Angry Men
Benchmarks: Conttime Taobao Retweet StackOverflow Amazon 12 Angry Men
Metrics: ELL (log-likelihood per event) ACC (event type prediction accuracy)
Baselines: Classic EM (HP) Softmax attention (SAHP) Softmax attention (THP) Sinkhorn-based attention HPST GPT-4o ranking
Builds on: Hawkes process EM Transformer Hawkes process self-attentive Hawkes process Sinkhorn attention (Sinkformer) HPST
Closest papers by meaning
- Residual TPP: A Unified Lightweight Approach for Event Stream Data Analysis (ICML 2025 · similarity 0.86)
- EasyTPP: Towards Open Benchmarking Temporal Point Processes (ICLR 2024 · similarity 0.84)
- Deep Continuous-Time State-Space Models for Marked Event Sequences (NeurIPS 2025 · similarity 0.84)
- Interacting Diffusion Processes for Event Sequence Forecasting (ICML 2024 · similarity 0.84)
- MOCHA: Discovering Multi-Order Dynamic Causal Structure in Temporal Point Processes (NeurIPS 2026 · similarity 0.84)
- A Variational Autoencoder for Neural Temporal Point Processes with Dynamic Latent Graphs (AAAI 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.