AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

Is One Layer Enough? Understanding Inference Dynamics in Tabular Foundation Models

Amir Rezaei Balef, Mykhailo Koshil, Katharina Eggensperger

ICML 2026 · Architectures and efficiency › RNNs and normalization · approach: Layer-wise analysis of language models · 0 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train individual decoders per layer (tabular logit lens) instead of using the original decoder at intermediate layers6 tabular foundation models (TabPFN v1/v2/2.5, TabICL, LimiX-2M/16M) · TabArena (15 binary tasks) and PMLBmini (34 tasks)normalized ROC-AUCImproved
skip an early layer (ablation)TFMs · binary classification, layer ablationnormalized ROC-AUCHurt
skip a middle or late layer (ablation)TFMs · binary classification, layer ablationnormalized ROC-AUCNo clear effect
repeat a single layer at inference (ablation)LimiX-16M and TabPFN(v1) · binary classification, layer ablationnormalized ROC-AUCMixed
swap adjacent layers (ablation)TFMs, especially TabPFN(v2) · binary classification, layer ablationnormalized ROC-AUCHurt
skip the first layer (self-repair check) (ablation)TFMs · binary classification, tabular logit lensintermediate ROC-AUCHurt
skip a middle or late layer (self-repair check) (ablation)TFMs, especially TabPFN(v2) · binary classification, tabular logit lensintermediate ROC-AUCNo clear effect
use a strong input encoder (TabICL compression, LimiX-2M RBF preprocessing)TabICL and LimiX-2M · layer skippingnormalized ROC-AUCImproved
use a single transformer layer instead of six (nanoTabPFN) (ablation)nanoTabPFN · PMLBmini and TabArenanormalized performanceHurt
loop one shared layer up to six times instead of six distinct layers (ablation)nanoTabPFN (single layer, looped) · PMLBmini and TabArenanormalized performance: uses 20% of the parameters, comparable performanceImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy