Research map / Large language models / Language, safety and interpretability
DABL: Detecting Semantic Anomalies in Business Processes Using Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Fine-tunes Llama 2-Chat 13B with QLoRA on a log of playout-generated normal traces plus simulated ordering and exclusion anomalies to detect and explain semantic anomalies in business processes.
Problem
Existing semantic anomaly detectors for business process traces split each trace into event pairs, losing long-distance dependencies, and explain anomalies only via anomalous event pairs; statistical and conformance-based methods are unsuitable for multi-process logs.
Approach
Feed the entire trace into an LLM prompt and fine-tune the LLM on playout-generated normal traces and synthetic ordering/exclusion anomalies (from 143,137 real process models), so it classifies traces as normal or anomalous and explains the cause in natural language.
Key results
- On unseen-process dataset D1, DABL reaches 94.06% precision, 91.88% F1 and 92.39% accuracy, beating SENSE-BERT (F1 64.53%) and SENSE-SVM (F1 2.20%).
- On seen-process dataset D2, DABL reaches 96.87% F1 and 97.03% accuracy, improving F1 by 21.79 points over SENSE-BERT.
- Cause interpretation achieves ROUGE-L F1 of 76.02% on D1 and 86.56% on D2.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DABL instead of SENSE-BERT | Semantic anomaly detection · D1 (unseen processes) | F1 (%): 64.53 -> 91.88 | Helped |
| Use DABL instead of SENSE-SVM | Semantic anomaly detection · D1 (unseen processes) | accuracy (%): 52.50 -> 92.39 | Helped |
| Use DABL instead of SENSE-BERT | Semantic anomaly detection · D1 (unseen processes) | recall (%): 97.74 -> 89.79 | Hurt |
| Use DABL instead of SENSE-BERT | Semantic anomaly detection · D2 (seen processes, unseen anomalies) | F1 (%): 75.08 -> 96.87 | Helped |
| Use DABL instead of SENSE-BERT | Semantic anomaly detection · D2 (seen processes, unseen anomalies) | accuracy (%): 80.28 -> 97.03 | Helped |
| Use DABL instead of SENSE-BERT | Anomaly-cause interpretation · D1 (unseen processes) | ROUGE-2 F1 (%) | Helped |
| Include normal process behavior from the target processes in training | Semantic anomaly detection methods · D2 versus D1 | precision (%) | Helped |
Surprising findings
- SENSE-SVM achieves 87.95% precision but only 1.12% recall on D1, showing it misses almost all anomalies.
Limitations
- Collected process models may have precision below 1, so they may allow traces not in the original process, adding noise to training data.
- The model may flag traces that are irregular in common sense but normal within a customized process.
- When multiple anomaly types occur in one trace, only one cause is interpreted at a time.
Experimental setup
Datasets: BPM Academic Initiative models Fundamentals of BPM models SAP signavio academic models BPI 2020 travel permit log road traffic fine management log
Benchmarks: D1 (1,000 unseen process models, 28,081 traces) D2 (1,000 seen process models, 40,925 traces) BPI 2020 travel permit log road traffic fine management log
Metrics: precision recall F1 accuracy ROUGE-2 ROUGE-L
Baselines: SEM SENSE-SVM SENSE-BERT
Builds on: QLoRA Llama 2-Chat SENSE process tree playout
Closest papers by meaning
- AlphaMath Almost Zero: Process Supervision without Process (NeurIPS 2024 · similarity 0.84)
- SNAP: Semantic Stories for Next Activity Prediction (AAAI 2025 · similarity 0.84)
- OpenClawBench: Benchmarking Process-side Anomalies in Real-world Agent Execution Trajectories (NeurIPS 2026 · similarity 0.84)
- Taming Imperfect Process Verifiers: A Sampling Perspective on Backtracking (ICLR 2026 · similarity 0.83)
- DCAD-2000: A Multilingual Dataset across 2000+ Languages with Data Cleaning as Anomaly Detection (NeurIPS 2025 · similarity 0.83)
- WorkflowLLM: Enhancing Workflow Orchestration Capability of Large Language Models (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.