Research map / Architectures and efficiency / Attention and transformers
End-to-End Compression for Tabular Foundation Models
Guri Zabërgja, Rafiq Kamel, Arlind Kadra, Christian Frey, Josif Grabocka
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| compress training context with learned TACO compressor (1% rate) instead of full context | POT predictor transformer · Synthetic 15k x 500, no KV cache | subsequent predict time: 28.67s -> 306ms | Improved |
| compress context to 1% with TACO | POT predictor transformer · Synthetic 15k x 500, no KV cache | predict memory: 22.45GB -> 549MB (-97.6%) | Improved |
| compress context to 1% with TACO | POT predictor transformer · TabArena classification (26 datasets) | mean ROC-AUC: 0.862 -> 0.855 | No clear effect |
| add TACO compression to enable KV caching | POT with KV cache · Synthetic grid N 5k-50k, M 100-1000 | feasibility (OOM): POT OOM on 32 of 100 grid datasets | Improved |
| TACO compression raises fit-time memory (no KV cache) | TACO · Synthetic 15k x 500 | fit memory: 36MB -> 92MB | Hurt |
| train compressor and predictor jointly instead of freezing predictor at TabPFN v2.0 weights (ablation) | TACO · TabArena classification | ROC-AUC | Improved |
| train one model with compression rate sampled from {1,2,4,8,16}% instead of rate-specific models (ablation) | TACO · TabArena classification | ROC-AUC | No clear effect |
| use learned TACO compressor instead of random or kNN subsampling of context (ablation) | POT · TabArena classification, 1%-16% rates | ROC-AUC | Improved |
| chunk-and-stitch compression with TACO instead of random/kNN subsampling | POT / TabPFNv2 · MetroPT-3 | AUPRC: 0.7293 -> 0.8955 (vs POT Random) | Improved |
| higher compression rate (more retained rows) (ablation) | TACO vs baselines · TabArena classification | ROC-AUC gap | Mixed |
| higher compression rate 1% -> 16% (ablation) | TACO · Synthetic 15k x 500, no KV cache | subsequent predict speedup: ×93.6 -> ×15 | Hurt |
Closest papers by meaning
- TabDPT: Scaling Tabular Foundation Models on Real Data (NeurIPS 2025 · similarity 0.90)
- Retrieval & Fine-Tuning for In-Context Tabular Models (NeurIPS 2024 · similarity 0.89)
- Tabular Foundation Model for Generative Modelling (NeurIPS 2026 · similarity 0.89)
- TabICL: A Tabular Foundation Model for In-Context Learning on Large Data (ICML 2025 · similarity 0.88)
- TabSTAR: A Tabular Foundation Model for Tabular Data with Text Fields (NeurIPS 2025 · similarity 0.88)
- TabFlex: Scaling Tabular Learning to Millions with Linear Attention (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
