Research map / Time series / Foundation models for time series

Revitalizing Medical Time Series with Vision-Informed Retrieval: A Vision-Language Perspective

NeurIPS 2026 · Time series › Foundation models for time series · approach: Time-series language reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ViRe, which renders medical time series as waveform images, encodes them with a frozen CLIP vision encoder into a global Vision Query, and uses cross-attention to retrieve temporal and channel evidence from numerical MedTS tokens for classification.

Problem

Medical time series classifiers (EEG, ECG) treat signals purely as numerical sequences and ignore the waveform morphology that clinicians use for diagnosis.

Approach

Use a frozen CLIP vision encoder applied to deterministic waveform renderings as a morphology-aware Query in asymmetric cross-attention over numerical temporal and channel tokens, so vision selects what numerical evidence to retrieve without being a separate diagnostic pathway.

Key results

• ViRe achieves mean Avg 82.90 across six datasets versus 77.90 for Medformer, a 6.42% relative improvement.
• On APAVA, ViRe Avg reaches 92.79 vs 79.74 for Medformer (16.37% relative gain); ViRe ranks first on five of six datasets, second on PTB-XL (69.47 vs PatchTST 69.90).
• Replacing the Vision Query with a random ViT initialization drops the average gain by 11.81%, while CLIP-Vision yields a 7.72% average gain in the ablation.

Surprising findings

• A randomly initialized ViT vision query degrades performance (-11.81% average), so generic vision encoders can hurt rather than help.
• Zero (all-zero) and Gaussian content-free queries still give 1.92% and 0.76% gains, suggesting part of the improvement comes from the retrieval architecture itself.

Limitations

• PTB-XL is the one dataset where ViRe does not rank first (second to PatchTST), and the gains vary widely across datasets.
• Benefit from the vision prior is largest on small EEG cohorts; the paper's own content-free ablations show a non-trivial gain without vision content, so attribution is partly confounded.
• Results are reported on a single 1-second window setup with preprocessing specific to the Medformer protocol and a single RTX 4090.

Experimental setup

Datasets: APAVA ADFTD TDBrain PTB PTB-XL MIMIC

Benchmarks: APAVA ADFTD TDBrain PTB PTB-XL MIMIC

Metrics: Accuracy Precision Recall F1-Score AUROC AUPRC

Baselines: Autoformer FEDformer Informer Reformer Transformer MTST Nonformer iTransformer PatchTST Medformer

Builds on: Medformer CLIP PatchTST iTransformer

Closest papers by meaning

• MedSIGHT: Towards Grounded Visual Comprehension in Medical Large Vision-Language Models (ICML 2026 · similarity 0.87)
• Unlocking the Power of Spatial and Temporal Information in Medical Multimodal Pre-training (ICML 2024 · similarity 0.86)
• How Do Medical MLLMs Fail? A Study on Visual Grounding in Medical Images (ICLR 2026 · similarity 0.86)
• Integrating Sequence and Image Modeling in Irregular Medical Time Series Through Self-Supervised Learning (AAAI 2025 · similarity 0.86)
• MedAtlas: Evaluating LLMs for Multi-Round, Multi-Task Medical Reasoning Across Diverse Imaging Modalities and Clinical Text (AAAI 2026 · similarity 0.85)
• VILA-M3: Enhancing Vision-Language Models with Medical Expert Knowledge (CVPR 2025 · similarity 0.85)