Research map / Adversarial robustness and security / Certified and empirical robustness
TabularBench: Benchmarking Adversarial Robustness for Tabular Deep Learning in Real-world Use-cases
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces TabularBench, a benchmark and model zoo measuring adversarial (constrained) robustness of tabular deep learning classifiers across five real-world use cases, with and without synthetic data augmentation and adversarial training.
Problem
Adversarial evasion robustness of deep tabular models under realistic feature constraints has no standardized benchmark, and it is unclear how architectures, adversarial training, and data augmentation affect robustness.
Approach
Evaluates tabular DL classifiers with the Constrained Adaptive Attack (CAA) on five constrained binary tasks (CTU, LCLD, Malware, URL, WiDS), combining adversarial training with six synthetic-data augmentations (CTGAN, TVAE, WGAN, TableGAN, GOGGLE, Cutmix) and releasing a model zoo, dataset zoo, and leaderboard.
Key results
- Without augmentation, adversarial training raises constrained robust accuracy on URL for TabTransformer from 8.9% to 56.7% while the ID accuracy stays similar (93.6 to 93.9).
- Pearson correlation between ID and robust performance rises from 0.15 to 0.76 on LCLD after adversarial training with augmentation.
- Cutmix is the best augmentation for robustness in 7 of 20 scenarios, while CTGAN is best for ID performance across use cases.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add adversarial training | Tabular deep learning architectures · CTU, LCLD, MALWARE, URL, and WIDS | robust accuracy | Helped |
| Add data augmentation to standard training | Tabular deep learning architectures · CTU and URL | robust accuracy | Mixed |
| Use CTGAN data augmentation | Tabular deep learning models · All use cases | ID performance | Helped |
| Use Cutmix data augmentation | Tabular deep learning models with adversarial training · Across 20 architecture and task scenarios | robust accuracy: 7/20 scenarios | Helped |
| Add data augmentation to adversarial training | Tabular deep learning models · URL | robust accuracy | Helped |
| Increase perturbation size to ε=5 with AT plus augmentation rather than AT alone | TabNet · URL | robust accuracy: above 40%; 0% | Helped |
| Use data augmentation with adversarial training rather than adversarial training alone | Tabular deep learning models · URL, LCLD, and WIDS | robust accuracy: 4/20 scenarios | Mixed |
| Evaluate attacks without enforcing domain constraints | Robustness evaluation of tabular models · All use cases | robust accuracy | Hurt |
Surprising findings
- Data augmentation that improves ID performance can hurt robust performance, so test accuracy is misleading for robustness.
- Evaluation with unconstrained attacks is misleading: under standard training there is no relation between unconstrained and constrained robustness.
Limitations
- Only L2 distance and binary classification are considered
- Only data-augmentation-based adversarial training defenses are evaluated
- Only five datasets qualify under the authors' criteria
Experimental setup
Datasets: LCLD CTU-13 URL (ISCX-URL2016) WiDS Malware (PE files)
Benchmarks: LCLD CTU-13 URL WiDS Malware
Metrics: robust accuracy on constrained examples AUC accuracy MCC precision recall
Baselines: XGBoost TabTransformer TabNet RLN STG VIME
Builds on: CAA Constrained Adaptive Attack (CAPGD + MOEVA) RobustBench CTGAN Tableshift
Closest papers by meaning
- Constrained Adaptive Attack: Effective Adversarial Attack Against Deep Neural Networks for Tabular Data (NeurIPS 2024 · similarity 0.91)
- TabArena: A Living Benchmark for Machine Learning on Tabular Data (NeurIPS 2025 · similarity 0.88)
- TabReD: Analyzing Pitfalls and Filling the Gaps in Tabular Deep Learning Benchmarks (ICLR 2025 · similarity 0.88)
- OODRobustBench: a Benchmark and Large-Scale Analysis of Adversarial Robustness under Distribution Shift (ICML 2024 · similarity 0.87)
- TabM: Advancing tabular deep learning with parameter-efficient ensembling (ICLR 2025 · similarity 0.87)
- VT-Bench: A Unified Benchmark for Visual-Tabular Multi-Modal Learning (ICML 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.