Research map / Large language models / Retrieval and knowledge
A Semantic Parsing Framework for End-to-End Time Normalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Frames time normalization as code generation into an executable SCATE Python library, uses LLMs to augment data with execution-based filtering, and fine-tunes a 0.5B Qwen model that outperforms its Claude 3.7 teacher on TempEval-2013.
Problem
Traditional ISO-TimeML time normalization systems have limited expressivity for compositional, event-relative, and multi-span temporal expressions, and existing SCATE-based systems are complex multi-stage pipelines with scarce annotated data.
Approach
Implement the full SCATE framework as an executable Python library so that a model can generate SCATE code from text and deterministically execute it to obtain timeline intervals; LLMs generate augmented (text, expression, code) triplets validated by code execution, and a small model is trained on augmented plus original data.
Key results
- Fine-tuned Qwen2.5-0.5B on CC-News + Training Set reaches 0.59 accuracy and F1 on the test set, versus 0.49 for Claude 3.7 + SCATE Prompt and 0.51 for GPT 4.1 + SCATE Prompt.
- SCATE code generation with Claude 3.7 outperforms interval few-shot prompting by more than 10 points in accuracy and F1 (0.49 vs 0.38 on test).
- Qwen2.5-0.5B trained only on the 557 TempEval training examples achieves near-zero accuracy (0.01).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the original training set to CC-News augmented training data (ablation) | Qwen2.5-0.5B fine-tuning · TempEval-2013 test | accuracy: 0.37 -> 0.59 | Helped |
| Add CC-News augmented training data to the original training set (ablation) | Qwen2.5-0.5B fine-tuning · TempEval-2013 test | accuracy: 0.01 -> 0.59 | Helped |
| Train on CC-News augmented data instead of relying only on the original training set (ablation) | Qwen2.5-0.5B fine-tuning · TempEval-2013 test | F1: 0.01 -> 0.41 | Helped |
| Generate SCATE code rather than directly generating time intervals | Claude 3.7 prompting · TempEval-2013 test | accuracy: 0.38 -> 0.49 | Helped |
| Generate SCATE code rather than directly generating time intervals | Claude 3.7 prompting · TempEval-2013 test | F1: 0.38 -> 0.52 | Helped |
| Fine-tune on CC-News augmented data and the original training set instead of using Claude 3.7 with the SCATE prompt | Time normalization on TempEval-2013 · TempEval-2013 test | accuracy: 0.49 -> 0.59 | Helped |
| Add the original training set to CC-News augmented training data (ablation) | Qwen2.5-0.5B fine-tuning · TempEval-2013 test | F1: 0.41 -> 0.59 | Helped |
Surprising findings
- The 0.5B model trained on augmented plus original data outperforms its LLM teacher (Claude 3.7 + SCATE Prompt) on the test set.
- Fine-tuning on the small human-annotated TempEval training set alone yields near-zero performance, showing small models need augmented data.
Limitations
- Evaluation limited to TempEval-2013, the only public SCATE-annotated dataset, which lacks open-domain and multilingual diversity.
- Only one small model (Qwen2.5-0.5B) fine-tuned; no larger or alternative architectures explored.
- Error analysis shows span recognition (missed expressions, 70% of 20 errors) is the dominant failure mode.
Experimental setup
Datasets: TempEval-2013 CC-News
Benchmarks: TempEval-2013 (merged dev and test, 313 SCATE code blocks)
Metrics: accuracy precision recall F1 bootstrap 95% confidence intervals
Baselines: Claude 3.7 + Interval Few-shot Prompt Qwen2.5-0.5B + Training Set Qwen2.5-0.5B + CC-News GPT-4.1 + SCATE Prompt Claude 3.7 + SCATE Prompt
Builds on: SCATE ISO-TimeML Code4Struct CodeIE KnowCoder
Closest papers by meaning
- ITFormer: Bridging Time Series and Natural Language for Multi-Modal QA with Large-Scale Multitask Dataset (ICML 2025 · similarity 0.84)
- TimE: A Multi-level Benchmark for Temporal Reasoning of LLMs in Real-World Scenarios (NeurIPS 2025 · similarity 0.84)
- LAST SToP for Modeling Asynchronous Time Series (ICML 2025 · similarity 0.83)
- Filling Memory Gaps: Enhancing Continual Semantic Parsing via SQL Syntax Variance-Guided LLMs Without Real Data Replay (AAAI 2025 · similarity 0.83)
- Diversifying Query: Region-Guided Transformer for Temporal Sentence Grounding (AAAI 2025 · similarity 0.82)
- NeSTR: A Neuro-Symbolic Abductive Framework for Temporal Reasoning in Large Language Models (AAAI 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.