Research map / Time series / Foundation models for time series
Can LLMs Understand Time Series Anomalies?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Tests six-plus hypotheses about whether LLMs and multimodal LLMs understand time series anomalies, using synthetic datasets, text vs. visual inputs, CoT, and arithmetic impairment, finding visual inputs help, CoT does not, and model behavior varies.
Problem
Whether LLMs and multimodal LLMs truly understand time series, tested via zero-shot and few-shot anomaly detection, given conjectures from forecasting work about reasoning, repetition bias, arithmetic, and visual processing.
Approach
Converts anomaly detection into a controlled hypothesis-testing framework using synthetic anomaly datasets, comparing text and visual input representations, CoT prompting, arithmetic impairment (DysCalc), noise injection, and subsampling, evaluated with affinity F1.
Key results
- Visual (image) inputs give much better affinity F1 than text for most anomaly types across M-LLMs, with frequency anomalies the exception for proprietary models.
- Chain-of-thought prompting steadily reduces anomaly detection performance across all models and anomaly types.
- Impairing arithmetic (DysCalc, integer addition accuracy 100% to 12%) leaves anomaly detection performance mostly unchanged, rejecting the arithmetic hypothesis.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| prompt for step-by-step chain-of-thought reasoning (ablation) | LLMs for time series anomaly detection · Synthetic anomaly types | affinity F1 | Hurt |
| inject minimal noise into periodic time series (ablation) | LLMs for frequency anomaly detection · Frequency anomalies | affinity F1 | No clear effect |
| impair arithmetic ability with DysCalc examples (ablation) | Gemini-1.5-Flash · Trend anomalies | affinity F1 | No clear effect |
| impair arithmetic ability with DysCalc examples (ablation) | Gemini-1.5-Flash · Five-digit integer addition | accuracy: 100% -> 12% | Hurt |
| present time series as images rather than text (ablation) | Multimodal LLMs · Synthetic anomaly types | affinity F1 | Mixed |
| use visual input for frequency anomalies (ablation) | Proprietary multimodal LLMs · Frequency anomalies | affinity F1 | Hurt |
| use flat-trend anomalies that are subtle to human vision (ablation) | LLMs for anomaly detection · Flat Trend Anomalies versus Trend Anomalies | affinity F1 | No clear effect |
| subsample time series from 1000 steps to 300 steps (ablation) | LLMs for time series anomaly detection · Synthetic time series | affinity F1 | Helped |
| use Token-per-Digit representation (ablation) | OpenAI GPT models · Text time series inputs | affinity F1 | Mixed |
| use Token-per-Digit rather than another text representation (ablation) | Non-GPT LLMs · Text time series inputs | affinity F1 | No clear effect |
| change the text representation among CSV, Prompt-as-Prefix, Token-per-Digit, and default (ablation) | LLMs for time series anomaly detection · Text time series inputs | affinity F1 | Mixed |
| select different model architectures and training approaches | LLMs for time series anomaly detection · Synthetic anomaly types | affinity F1 | Mixed |
Surprising findings
- Chain-of-thought prompting consistently hurts time series anomaly detection performance.
- Arithmetic impairment via in-context learning does not reduce anomaly detection performance.
Limitations
- Relies mainly on synthetic datasets, with limited evidence on subtle real-world anomalies
- Only four M-LLMs evaluated, with substantial variation across models
- Proprietary model behavior cannot be fully controlled or inspected
Experimental setup
Datasets: Yahoo S5 synthetic point, range, frequency, trend anomaly datasets
Benchmarks: Yahoo S5 synthetic anomaly datasets (point, range, frequency, trend, noisy variants, flat trend)
Metrics: affinity F1 precision recall affinity precision affinity recall F1
Baselines: Isolation Forest Thresholding
Builds on: zero-shot chain-of-thought affiliation metrics
Closest papers by meaning
- Time-LLM: Time Series Forecasting by Reprogramming Large Language Models (ICLR 2024 · similarity 0.91)
- Position: What Can Large Language Models Tell Us about Time Series Analysis (ICML 2024 · similarity 0.90)
- Are Language Models Actually Useful for Time Series Forecasting? (NeurIPS 2024 · similarity 0.90)
- AutoTimes: Autoregressive Time Series Forecasters via Large Language Models (NeurIPS 2024 · similarity 0.89)
- STEM-LTS: Integrating Semantic-Temporal Dynamics in LLM-driven Time Series Analysis (AAAI 2025 · similarity 0.89)
- The Elephant in the Room: Towards A Reliable Time-Series Anomaly Detection Benchmark (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.