Research map / Large language models / AI and society
SNAP: Semantic Stories for Next Activity Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SNAP, which uses an LLM to build a story template from selected event-log features, converts each trace prefix into a semantic story, and fine-tunes BERT, DeBERTa, or GPT-3 to predict the next activity.
Problem
Next activity prediction in business process management ignores much of the semantic content (free text, utterances, named attributes) in process event logs, which current sequence-only models cannot exploit.
Approach
Transform each prefix trace into a coherent narrative ("semantic story") generated from a LLM-produced template over XGBoost-selected features plus temporal features, and fine-tune a language foundation model as a classifier on these stories.
Key results
- SNAP-G reaches 0.459 accuracy on MIP versus 0.390 for the prior best (Zeltyn et al.), and 0.696 vs 0.675 on BPI13cp (Rama-Maneiro et al.).
- SNAP models significantly outperform SOTA on weighted F1 across all datasets (Wilcoxon p<0.05), per the paper.
- Replacing semantic stories with lists of feature values lowers SNAP-B accuracy on BPI13in (0.758 vs 0.733), Env Permit, and Sepsis; on MIP, DeBERTa without user utterances drops to 0.373 accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use semantic stories instead of a list of feature values (ablation) | SNAP-B · BPI13cp, BPI13in, Env Permit, Sepsis, and Nasa | accuracy and weighted F1-score: BPI13cp: 0.616 -> 0.679 accuracy, 0.614 -> 0.670 F1; BPI13in: 0.733 -> 0.758 accuracy, 0.731 -> 0.754 F1; Env Permit: 0. | Helped |
| use semantic stories instead of a list of feature values (ablation) | SNAP-B · MIP | accuracy and weighted F1-score: 0.426 -> 0.424 accuracy; 0.421 -> 0.421 F1 | No clear effect |
| use named rather than numbered activities (ablation) | SNAP-B · MIP | accuracy and weighted F1-score: 0.401 -> 0.424 accuracy; 0.395 -> 0.421 F1 | Helped |
| use named rather than numbered activities (ablation) | SNAP-B · BPI13cp and Env Permit | accuracy: BPI13cp: 0.670 -> 0.679; Env Permit: 0.877 -> 0.880 | Helped |
| use named rather than numbered activities (ablation) | SNAP-B · BPI13in, Sepsis, and Nasa | accuracy: BPI13in: 0.758 -> 0.758; Sepsis: 0.648 -> 0.649; Nasa: 0.896 -> 0.896 | No clear effect |
| include user utterances (ablation) | SNAP on MIP · MIP | accuracy: 0.373 -> 0.437 | Helped |
| include user utterances (ablation) | SNAP on MIP · MIP | weighted F1-score: 0.358 -> 0.428 | Helped |
| use SNAP semantic stories and fine-tuned language models rather than the strongest reported baseline | SNAP · BPI13cp, Env Permit, Sepsis, and MIP | accuracy: BPI13cp: 0.675 -> 0.696; Env Permit: 0.877 -> 0.890; Sepsis: 0.642 -> 0.655; MIP: 0.390 -> 0.459 | Helped |
Surprising findings
- On MIP, the list-of-values representation performs comparably to semantic stories (0.426 vs 0.424 accuracy), suggesting the story structure matters less there than activity names and utterances.
- Replacing activity names with integers barely hurts most datasets but drops MIP accuracy from 0.424 to 0.401.
Limitations
- Gains are modest on average (~7%) and may be limited in practice.
- Expected to give little benefit on logs with limited semantic content (e.g., only numerical attributes).
- Using an LLM for story templates may conflict with data security needs; manual template construction is suggested.
Experimental setup
Datasets: BPI13cp BPI13in Env Permit Sepsis Nasa MIP
Benchmarks: BPI13cp BPI13in Env Permit Sepsis Nasa MIP
Metrics: accuracy weighted F1-score
Baselines: Camargo et al. (2019) Evermann et al. Hinnka et al. Khan et al. Mauro et al. Pasquadibisceglie et al. Tax et al. Theis et al. Venugopal et al. Rama-Maneiro et al. Zeltyn et al.
Builds on: BERT DeBERTa XGBoost Llama 2
Closest papers by meaning
- WONDERBREAD: A Benchmark for Evaluating Multimodal Foundation Models on Business Process Management Tasks (NeurIPS 2024 · similarity 0.85)
- DABL: Detecting Semantic Anomalies in Business Processes Using Large Language Models (AAAI 2025 · similarity 0.84)
- WorkflowLLM: Enhancing Workflow Orchestration Capability of Large Language Models (ICLR 2025 · similarity 0.83)
- DanmakuTPPBench: A Multi-modal Benchmark for Temporal Point Process Modeling and Understanding (NeurIPS 2025 · similarity 0.82)
- LLM Processes: Numerical Predictive Distributions Conditioned on Natural Language (NeurIPS 2024 · similarity 0.82)
- Episodic Memories Generation and Evaluation Benchmark for Large Language Models (ICLR 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.