Research map / Adversarial robustness and security / Certified and empirical robustness

Benchmarking Sensor-Fault Robustness in Forecasting

NeurIPS 2026 · Adversarial robustness and security › Certified and empirical robustness · approach: Theory of adversarial risk · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces SensorFault-Bench, a CPS sensor-fault stress-test protocol for forecasting that scores eight value/timing/availability fault scenarios across four datasets and reports worst-scenario degradation, clean MSE and worst-scenario fault-time MSE, plus paired evaluation of robustness-improvement methods.

Problem

Standard forecasting evaluation selects models by nominal clean error and does not show whether they remain robust to structured CPS sensor faults (noise, drift, stuck readings, missing data, timing misalignment).

Approach

A target-fixed, input-side sensor-fault benchmark with a shared severity model, worst-scenario degradation as the primary relative robustness score, a disjoint training-only transfer fault family set for fault augmentation, and paired method-baseline deltas.

Key results

• Chronos-2 matches or trails the last-value naive forecaster in clean MSE on Beijing Air Tiantan and Penmanshiel WT08, and has the largest worst-scenario degradation on ETTh1 (1.448) and Traffic (1.911).
• On Penmanshiel WT08, GRU has the lowest D_w (1.064) despite ModernTCN and DLinear having slightly better clean MSE, showing clean-MSE rankings can disagree with fault-time rankings.
• PGD adversarial training reduces D_w by -0.136 on Beijing Air Tiantan and -0.059 on Penmanshiel; fault augmentation gives -0.025 on Traffic; RevIN raises D_w on ETTh1 (+0.124) and Traffic (+0.267).

Surprising findings

• Chronos-2 has the largest worst-scenario degradation on ETTh1 and Traffic despite acceptable clean MSE on Traffic.
• RevIN lowers clean MSE on ETTh1 and Traffic but increases degradation in the same paired comparisons.

Limitations

• Evaluates simulated input-side sensor faults rather than recorded field fault episodes.
• Only one standardized severity model and one active scenario at a time; composed faults and closed-loop effects are out of scope.
• Four datasets and a small robustness-improvement method panel with bounded tuning.

Experimental setup

Datasets: Beijing Multi-Site Air Quality (Tiantan curated) Penmanshiel WT08 SCADA (curated) ETTh1 Traffic

Benchmarks: SensorFault-Bench (eight scenarios over four datasets)

Metrics: worst-scenario degradation D_w clean MSE worst-scenario fault-time MSE paired method-baseline deltas

Baselines: DLinear GRU ModernTCN PatchTST TSMixer SeasonalNaive Chronos-2

Builds on: RevIN PatchTST Chronos-2 DLinear ModernTCN

Closest papers by meaning

• Training ML Models with Predictable Failures (NeurIPS 2026 · similarity 0.83)
• Exactness Matters for Physical Rule Enforcement (NeurIPS 2026 · similarity 0.81)
• Truthfulness of Calibration Measures (NeurIPS 2024 · similarity 0.81)
• Position: AI Safety Must Embrace an Antifragile Perspective (ICML 2025 · similarity 0.80)
• Smooth ECE: Principled Reliability Diagrams via Kernel Smoothing (ICLR 2024 · similarity 0.80)
• fev-bench: A Realistic Benchmark for Time Series Forecasting (NeurIPS 2026 · similarity 0.80)