Research map / Label-efficient and robust learning / Recognition and long-tail
Feature-aware Modulation for Learning from Temporal Tabular Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes feature-aware temporal modulation, which conditions per-feature Yeo-Johnson-based scale, bias and skewness transforms on a temporal embedding, to handle concept drift in temporal tabular data without injecting time as an input feature.
Problem
Temporal distribution shifts in tabular data, where feature-label relationships evolve over time, break static models, while adaptive models overfit to transient patterns.
Approach
Modulate each raw feature (and optionally hidden layers and logits) with time-conditioned scale, bias and Yeo-Johnson skewness parameters predicted from a shared temporal embedding, so feature semantics are aligned across time rather than concatenating time as an input.
Key results
- TabM with temporal modulation achieves the best average rank (3.50) across the TabReD datasets, versus 7.25 for static TabM and 5.125 for TabM with temporal embedding (main table, temporal split).
- MLP with full three-stage modulation improves average performance over baseline MLP by 2.09% on TabReD (ablation).
- Input-only modulation achieves 87.4% of the full-modulation gain for MLP, and modulation at the raw feature level is the most important position.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add temporal modulation on input, intermediate representations, and output logits (ablation) | MLP · TabReD | improvement: +2.09% | Helped |
| add input-level modulation only (ablation) | MLP · TabReD | improvement: 1.83% | Helped |
| add output-logit modulation only (ablation) | MLP · TabReD | improvement: 1.02% | Helped |
| add intermediate-representation modulation only (ablation) | MLP · TabReD | improvement: 0.26% | Helped |
| remove input-level modulation while retaining representation and output modulation (ablation) | MLP temporal modulation · TabReD | performance gain relative to full modulation: 87.4% -> 56.8% | Hurt |
| increase temporal embedding dimension from 8 to 512 (ablation) | MLP with conventional temporal embedding · TabReD | improvement: +0.21% -> +0.40% | Mixed |
| increase temporal embedding dimension from 8 to 512 (ablation) | MLP with temporal modulation · TabReD | improvement: +0.65% -> +1.48% | Helped |
| replace static TabM with TabM using temporal modulation | TabReD · TabReD | average rank: 7.250 -> 3.500 | Helped |
| replace temporal-embedding TabM with temporal-modulation TabM | TabReD · TabReD | average rank: 5.125 -> 3.500 | Helped |
| add temporal modulation to TabM instead of using a temporal embedding | TabM · TabReD random split | average rank: 3.750 -> 3.500 | No clear effect |
| add temporal modulation to TabM instead of using a temporal embedding | TabM · TabReD original temporal splits | average rank: 4.250 -> 3.625 | Helped |
| apply temporal modulation in a no-shift setting | temporal modulation model · No temporal shift | decision-boundary stability | No clear effect |
Surprising findings
- The paper claims this is the first instance where a deep tabular method consistently outperforms GBDT models under temporal distribution shifts.
- Simply concatenating temporal embeddings to inputs does not yield statistically significant gains, per the Nemenyi tests.
Limitations
- Incompatible with PLR embeddings for full-stage modulation, since PLR produces arcsine-like distributions; only raw-feature modulation is used for MLP-PLR and TabM.
- Gains are modest for most models (e.g., around 2% for MLP), and some metrics show no improvement or slight regression.
- Evaluated only on the TabReD benchmark, with a restricted hyperparameter search for some models (25 trials for FT-T and TabR).
Experimental setup
Datasets: TabReD
Benchmarks: TabReD
Metrics: AUC RMSE average rank Nemenyi post-hoc significance
Baselines: Linear XGBoost CatBoost LightGBM RandomForest MLP MLP-PLR FT-Transformer TabR ModernNCA TabM
Builds on: FiLM temporal embedding for tabular data TabM PLR embeddings Yeo-Johnson transformation
Closest papers by meaning
- Understanding the Limits of Deep Tabular Methods with Temporal Shift (ICML 2025 · similarity 0.88)
- TabFSBench: Tabular Benchmark for Feature Shifts in Open Environments (ICML 2025 · similarity 0.88)
- TabPrep: Closing the Feature Engineering Gap in Tabular Benchmarks (NeurIPS 2026 · similarity 0.87)
- Enhancing Tabular Learners with Context-Aware Semantic Embeddings (NeurIPS 2026 · similarity 0.86)
- ConTextTab: A Semantics-Aware Tabular In-Context Learner (NeurIPS 2025 · similarity 0.86)
- TabLog: Test-Time Adaptation for Tabular Data Using Logic Rules (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.