Research map / Time series / Foundation models for time series
Time Series, Vision, and Language: Exploring the Limits of Alignment in Contrastive Representation Spaces
Pratham Yashwante, Rose Yu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| scale up total encoder size (ablation) | trimodal contrastive alignment · CaTS | TS-TXT cosine margin vs size (Pearson r): TS-IMG r=0.28, TS-TXT r=0.67, IMG-TXT r=0.53 | Improved |
| train on captions with more than double the information density (ablation) | trimodal contrastive alignment · CaTS | cosine margin, Procrustes, mutual kNN: TS-TXT cosine margin 0.57 -> 0.56 | No clear effect |
| increase caption information density from sparse to original (ablation) | trimodal contrastive alignment · CaTS | Procrustes, mutual kNN: siglip + t5 + timesfm TS-TXT Procrustes 1.6072 -> 0.5827 (1-phrase to dense) | Improved |
| add image modality to TS-TXT contrastive training (trimodal vs bimodal) (ablation) | TS-TXT alignment · CaTS | alignment metrics | Improved |
| add third modality to TS-IMG training (ablation) | TS-IMG alignment · CaTS | alignment metrics | Hurt |
| use jointly pretrained VL model with a time series encoder (ablation) | VL-TS configurations · CaTS | IMG-TXT Procrustes | Improved |
| scramble temporal order of line plots (ablation) | TS-IMG alignment · CaTS (DINOv2 configs) | cosine similarity / Procrustes: cosine -79.5%, Procrustes +210.7% | Hurt |
| richer annotated plots instead of generic plots (ablation) | TS-IMG alignment · TRUCE | cosine similarity | Improved |
| use clinical reports (indirect text) instead of direct captions (ablation) | TS-TXT alignment · MIMIC vs CaTS | cosine similarity, Procrustes | Hurt |
| switch report language from English to German (ablation) | trimodal alignment · MIMIC vs PTB-XL | cosine margin, Procrustes | Hurt |
| use code instead of natural language as text for time series (ablation) | TS-TXT alignment · CaTS | alignment metrics | Mixed |
| larger batch size and higher-capacity projection head (ablation) | trimodal alignment · CaTS | alignment metrics | Improved |
Closest papers by meaning
- Displacement Geometry Captures Platonic Shared Reality Across Models and Modalities (NeurIPS 2026 · similarity 0.89)
- The Alignment Illusion in Multimodal Large Language Models (NeurIPS 2026 · similarity 0.89)
- Seeing the Image: Prioritizing Visual Correlation by Contrastive Alignment (NeurIPS 2024 · similarity 0.88)
- Position: The Platonic Representation Hypothesis (ICML 2024 · similarity 0.88)
- Language Models Represent Space and Time (ICLR 2024 · similarity 0.88)
- Escaping Plato's Cave: Towards the Alignment of 3D and Text Latent Spaces (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
