Research map / Time series / Foundation models for time series
TimeSeriesExamAgent: Creating TimeSeries Reasoning Benchmarks at Scale
Malgorzata Gwiazda, Yifu Cai, Mononito Goswami, Arjun Choudhry, Artur Dubrawski
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune on TimeSeriesExamAgent ECG exams | Qwen2.5-VL-3B-Instruct · ECG-QA (MIMIC-IV test, 12000 samples) | accuracy (General): 21.8% -> 47.0% | Improved |
| fine-tune on domain-irrelevant finance and weather exams (ablation) | Qwen2.5-VL-3B-Instruct · ECG-QA (MIMIC-IV test, 12000 samples) | accuracy (General): 21.8% -> 39.7% | Improved |
| fine-tune on ECG exams rather than domain-irrelevant exams (ablation) | Qwen2.5-VL-3B-Instruct · ECG-QA (MIMIC-IV test, 12000 samples) | accuracy (General): 39.7% -> 47.0% | Improved |
| fine-tune on ECG exams | Qwen2.5-VL-3B-Instruct · ECG-QA (MIMIC-IV test, 12000 samples) | accuracy (Parsable): 34.6% -> 49.7% | Improved |
| GPT-5 vs GPT-4o | VLM test taker · TimeSeriesExamAgent exams (mean) | accuracy: 0.440 -> 0.515 | Improved |
| agent-generated questions vs ECG-QA questions | question set · Question diversity (embedding distance) | embedding distance: 0.207 -> 0.301 | Improved |
| agent-generated questions vs ECG-QA questions | question set · Question diversity (normalized Levenshtein) | distance: 0.519 -> 0.542 | No clear effect |
| agent-generated ECG questions vs ECG-QA | question set · LLM-jury specificity (1-10) | score: 5.60 -> 8.43 | Improved |
| agent-generated finance questions vs FinMME | question set · LLM-jury unambiguity (1-10) | score: 7.59 -> 7.24 | Hurt |
| generator DeepSeek V3.2 instead of Claude 4 (ablation) | TimeSeriesExamAgent · MIT-BIH jury scores | jury score | No clear effect |
Closest papers by meaning
- TimeCAP: Learning to Contextualize, Augment, and Predict Time Series Events with Large Language Model Agents (AAAI 2025 · similarity 0.91)
- Time-LLM: Time Series Forecasting by Reprogramming Large Language Models (ICLR 2024 · similarity 0.91)
- Can LLMs Understand Time Series Anomalies? (ICLR 2025 · similarity 0.91)
- Position: What Can Large Language Models Tell Us about Time Series Analysis (ICML 2024 · similarity 0.90)
- TimeXL: Explainable Multi-modal Time Series Prediction with LLM-in-the-Loop (NeurIPS 2025 · similarity 0.90)
- STEM-LTS: Integrating Semantic-Temporal Dynamics in LLM-driven Time Series Analysis (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
