Research map / Time series / Foundation models for time series

LLM Flow Processes for Text-Conditioned Regression

NeurIPS 2026 · Time series › Foundation models for time series · approach: LLM-based time series forecasting · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Combines a flow-matching neural diffusion process with LLM bin probabilities via a gradient-free, closed-form-convolution product-of-experts, yielding text-conditioned, coherent regression trajectories.

Problem

Autoregressive LLM regression suffers error cascades on long prediction horizons and is costly and hard to parallelise, while marginal LLM predictions are over-broad and incoherent; neural diffusion processes produce coherent trajectories but cannot use textual metadata.

Approach

Form a product of experts between a diffusion/flow neural process prior and LLM marginal predictive densities, sampled with a gradient-free score correction exploiting closed-form Gaussian convolution of the binned LLM distribution, so LLM is queried only at context-matched points while the NDP supplies joint structure.

Key results

• On Periodic + linear, LLM-FP achieves MAE 0.26, CRPS 0.18, NLPD 0.20, best among methods (NDP CRPS 0.29, LLM-Direct CRPS 0.69).
• On Mauna Loa CO2, LLM-FP achieves CRPS 0.56 and NLPD 1.43, best on both, versus LLM-Independent CRPS 0.62 and LLM-Direct CRPS 1.52.
• On Change point, LLM-Direct is best (MAE 0.02) while LLM-FP (MAE 0.09) improves on LLM-Independent (MAE 0.29).

Surprising findings

• Autoregressive LLM-Direct shows divergent error-cascade trajectories already at around 96 prediction points.
• On the text-dominated change-point task, LLM-Direct outperforms the combined LLM-FP, which the authors attribute to the NDP prior lacking support for the discontinuity.

Limitations

• Product-of-experts assumes independence between NDP and LLM factors, risking double-counting evidence when correlated or when LLM is miscalibrated.
• LLM conditioning can only reweight the NDP prior and cannot create trajectory structure outside its support, causing shortfall on change point.
• Only a single 27B open-weight LLM without fine-tuning and a simple synthetic GP training prior were studied.

Experimental setup

Datasets: synthetic Gaussian process samples Mauna Loa atmospheric CO2 measurements synthetic San Diego rainfall

Benchmarks: Change point Periodic + linear Mauna Loa CO2

Metrics: MAE CRPS NLPD

Baselines: NDP LLM-Direct LLM-Independent

Builds on: Neural Diffusion Processes LLM Processes Direct Prompt flow matching product of experts

Closest papers by meaning

• LLM Processes: Numerical Predictive Distributions Conditioned on Natural Language (NeurIPS 2024 · similarity 0.89)
• Drag-and-Drop LLMs: Zero-Shot Prompt-to-Weights (NeurIPS 2025 · similarity 0.87)
• TEST-TIME SCALING IN DIFFUSION LLMS VIA HIDDEN SEMI-AUTOREGRESSIVE EXPERTS (ICLR 2026 · similarity 0.86)
• Consolidating Rewarded Perturbations for LLM Post-Training (NeurIPS 2026 · similarity 0.86)
• Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMs (ICLR 2025 · similarity 0.85)
• Beyond Next Token Prediction: Patch-Level Training for Large Language Models (ICLR 2025 · similarity 0.85)