Research map / Probabilistic and scientific modeling / Causal discovery

Learning Causal Orderings for In-Context Tabular Prediction

NeurIPS 2026 · Probabilistic and scientific modeling › Causal discovery · approach: Interventional causal discovery · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TabOrder, a tabular foundation model whose attention is masked by a causal order learned end-to-end via likelihood, enabling order-constrained imputation and prediction without causal supervision.

Problem

Tabular foundation models rely on correlational structure, which becomes unreliable under distribution shift or intervention, while causal discovery methods are dataset-specific and decoupled from predictive architectures; missing data further complicates causal order identification.

Approach

A dual-transformer TFM infers a soft variable ordering from unconstrained attention scores and uses a sigmoid-relaxed causal mask so each cell is predicted only from earlier variables; the order is trained jointly by a Gaussian log-likelihood on masked entries, with a missingness-aware variance increment that theoretically amplifies the likelihood asymmetry between causal directions under additive

Key results

• On the Sachs cd3cd28 reference condition, TabOrder reaches topological divergence about 0.21 and is the only method reported to achieve this, with 0.12-0.29 across most interventional conditions (PKC inhibition excluded).
• TabOrder is the most accurate imputation method on CTR23 for missing rates of at least 30% and on CC18 for at least 40%, while HyperImpute is best at lower missingness.
• In the mechanism-shift chain experiment, TabOrder remains reasonably robust under intervention while TabPFN and XGBoost degrade.

Surprising findings

• Missingness can facilitate causal direction identification: the theory shows that a missingness-aware variance strictly increases likelihood and can amplify the forward-backward likelihood gap under ANMs.

Limitations

• Learns topological orders rather than full DAGs; sparse graph refinement is left to pruning.
• Theoretical identifiability justification is limited to additive noise / causal additive model classes under standard assumptions.
• Pretraining priors and the effect of interventional training data are not studied; pretraining used only synthetic SCMs.

Experimental setup

Datasets: synthetic nonlinear ANM/SCM datasets Sachs protein signaling single-cell data OpenML CC18 CTR23

Benchmarks: Sachs protein signaling causal ordering CC18 imputation and prediction CTR23 imputation and prediction synthetic nonlinear ANM causal order recovery three-variable chain intervention prediction

Metrics: topological divergence negative log-likelihood average rank MSE

Baselines: TabPFN XGBoost CAM SCORE DAS NoGAM NOTEARS TOPIC AVICI R2Srt KNN ICE

Builds on: TabPFN AVICI causal additive models HyperImpute

Closest papers by meaning

• Learning Structural Causal Models from Ordering: Identifiable Flow Models (AAAI 2025 · similarity 0.88)
• CausalTab: Pretraining Across Causal Environments for Tabular Causal Discovery (NeurIPS 2026 · similarity 0.88)
• Causal Order: The Key to Leveraging Imperfect Experts in Causal Inference (ICLR 2025 · similarity 0.87)
• Deriving Causal Order from Single-Variable Interventions: Guarantees & Algorithm (ICLR 2025 · similarity 0.87)
• Ordering-Based Causal Discovery for Linear and Nonlinear Relations (NeurIPS 2024 · similarity 0.87)
• A Fixed-Point Approach for Causal Generative Modeling (ICML 2024 · similarity 0.86)