Research map / Large language models / Agents, code and math
Latent Logic Tree Extraction for Event Sequence Explanation from LLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LaTee, an amortized EM framework that uses a GFlowNet fine-tuned LLM (OPT) to sample latent logic trees from an LLM prior, scored by a temporal logic point process likelihood, for explaining and predicting event sequences.
Problem
Explaining streaming discrete event sequences (e.g., EHR, user behavior) with concise, human-readable logic trees while keeping the discrete structure inference tractable and leveraging LLM priors.
Approach
Treat logic trees as latent variables in a temporal logic point process; use a GFlowNet (SubTB objective) fine-tuned from a small LLM as an amortized posterior sampler in the E-step, and use the sampled trees to update rule weights and LLM prior parameters in the M-step.
Key results
- On MIMIC-3, GFN fine-tuning reaches 27.50% error rate versus 36.66% for AttNHP (about 25% relative gain).
- On Epic-100, GFN fine-tuning gets 55.25% error rate versus 67.53% for AttNHP (about 18% relative gain).
- On StackOverflow without semantic information, GFN fine-tuning (33.45%) roughly matches AttNHP (33.33%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LaTee with GFN fine-tuning instead of AttNHP | event next-type prediction · MIMIC-3 | ER (%): 36.66(13.76) -> 27.50(8.66) | Helped |
| Use LaTee with GFN fine-tuning instead of AttNHP | event next-type prediction · EPIC-100 | ER (%): 67.53(0.00) -> 55.25(9.01) | Helped |
| Use LaTee with GFN fine-tuning instead of AttNHP | event next-type prediction · StackOverflow | ER (%): 33.33(0.00) -> 33.45(5.12) | No clear effect |
| Increase tree width from 3 to 7 (ablation) | LaTee · EPIC-100 | ER (%): 69.40(9.21) -> 55.25(9.01) | Helped |
| Increase tree width from 3 to 5 (ablation) | LaTee · StackOverflow | ER (%): 34.50(5.23) -> 33.53(5.10) | Helped |
| Increase tree depth from 3 to 4 (ablation) | LaTee · EPIC-100 | ER (%): 69.40(9.21) -> 68.21(8.13) | Helped |
| Increase tree depth from 3 to 4 (ablation) | LaTee · StackOverflow | ER (%): 34.50(5.23) -> 34.41(5.08) | No clear effect |
| Increase base model size from OPT-350M to OPT-6.7B (ablation) | LaTee · EPIC-100 | ER (%): 69.40(9.21) -> 61.40(8.36) | Helped |
| Increase base model size from OPT-350M to OPT-6.7B (ablation) | LaTee · StackOverflow | ER (%): 72.87(16.35) -> 33.45(5.12) | Helped |
| Alternate E-M updates instead of using E-steps only (ablation) | LaTee · Synthetic@5 | ER (%): 62.5; 70.0; 67.5; 70.0 | Mixed |
| Use Mistral-7B-Instruct instead of OPT-1.3B for M-step generation (ablation) | LaTee with OPT-1.3B E-step · Synthetic@5 | ER (%): 97.5 -> 70.0 | Helped |
| Use Mistral-7B-Instruct instead of OPT-1.3B for M-step generation (ablation) | LaTee with OPT-1.3B E-step · Synthetic@5 | MR: 2.23 -> 2.08 | Helped |
Surprising findings
- Zero-shot and few-shot CoT prompting of the local LLM yields 100% error on MIMIC-3 and StackOverflow, and SFT underperforms attention-based TPP models.
- Increasing tree width helped more than increasing tree depth or model size on semantic event sequences, while model size mattered more on non-semantic ones.
Limitations
- Experiments limited to models up to 6.7B parameters and sequences of at most 40 events due to input token limits.
- Small datasets (e.g., 142 StackOverflow records, 400 Epic-100 histories) and reliance on semantic event names for the strongest gains.
Experimental setup
Datasets: MIMIC-III EPIC-KITCHENS-100 StackOverflow Synthetic (generated via thinning from TL-PP)
Benchmarks: MIMIC-3 Epic-100 StackOverflow Synthetic
Metrics: Error Rate (%) Mean Rank NLL
Baselines: AttNHP Pt-AttNHP k-shot CoT Tree-of-Thoughts SFT PPO fine-tuning
Builds on: GFlowNet temporal logic point process backward chaining QLoRA PPO
Closest papers by meaning
- LLM-DR: A Novel LLM-Aided Diffusion Model for Rule Generation on Temporal Knowledge Graphs (AAAI 2025 · similarity 0.86)
- Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning (ICML 2025 · similarity 0.86)
- RuAG: Learned-rule-augmented Generation for Large Language Models (ICLR 2025 · similarity 0.86)
- A State-Transition Framework for Efficient LLM Reasoning (ICLR 2026 · similarity 0.86)
- Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning (NeurIPS 2025 · similarity 0.86)
- Inductive Learning of Logical Theories with LLMs: A Expressivity-graded Analysis (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.