Research map / Architectures and efficiency / RNNs and normalization

A Scalable Multi-Task Model for Virtual Sensors

NeurIPS 2026 · Architectures and efficiency › RNNs and normalization · approach: Dataset distillation and data-efficient training · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a single causal transformer that jointly predicts many virtual sensors autoregressively, learns per-sensor input signal sets via attention-bias relevance vectors, and supports sparsified, selective inference.

Problem

Virtual sensors are built as isolated, application-specific models with expert-chosen inputs, which do not scale, cannot share knowledge across tasks, and lack consistent benchmarks; time series foundation models are costly and only forecast their own inputs.

Approach

A unified decoder-only transformer inserts empty prototype tokens tagged with variate embeddings to predict selected virtual sensors autoregressively, and learns per-sensor input relevance via a shared attention bias from outer products of relevance vectors, which is sparsified to skip irrelevant signals at inference.

Key results

• On CAN bus, sparse input sets and sensor selection cut inference time by up to 415x and memory by 951x with MSE 0.121 vs 0.119 for the unified model with variate embedding.
• Learned signal relevance achieves MSE 0.282 on Traffic and 0.121 on CAN bus, beating random (0.319, 0.601) and correlation-based (0.492, 0.267) input selection at similar sparsity.
• Transfer learning to add a new virtual sensor matches from-scratch MSE using 10% of data and 3.6% of computation.

Surprising findings

• Backpropagation through time diverged despite extensive hyperparameter tuning, so teacher forcing was used for main training.
• Training all virtual sensors simultaneously yields identical relevance vectors, so only a random subset is trained per iteration.

Limitations

• No extensive hyperparameter search for all parameters due to training cost.
• Teacher forcing limits learning of error propagation; BPTT diverged in main training.
• Efficiency gains depend strongly on chosen virtual sensors and dataset.

Experimental setup

Datasets: Traffic Electricity Solar CAN bus (Audi e-tron, 1713 variates)

Benchmarks: Traffic Electricity Solar CAN bus

Metrics: MSE inference time dynamic CUDA peak memory trainable parameters sparsity

Baselines: LSTM (literature) Transformer individual models Unified model without learned relevance Random signal relevance Correlation signal relevance

Builds on: LSTM Transformer decoder time series foundation models (patch-based) HEBO Adam

Closest papers by meaning

• Graph-based Virtual Sensing from Sparse and Partial Multivariate Observations (ICLR 2024 · similarity 0.86)
• Common Task Framework For a Critical Evaluation of Scientific Machine Learning Algorithms (NeurIPS 2025 · similarity 0.81)
• Bridging Remote Sensors with Multisensor Geospatial Foundation Models (CVPR 2024 · similarity 0.81)
• Hierarchical State Space Models for Continuous Sequence-to-Sequence Modeling (ICML 2024 · similarity 0.81)
• Real-Time Calibration Model for Low-Cost Sensor in Fine-Grained Time Series (AAAI 2025 · similarity 0.81)
• Scalable Hierarchical Deep Neural Network for Time Series Analysis in Wearable Sensor-based Human Activity Recognition (AAAI 2025 · similarity 0.81)