Research map / Probabilistic and scientific modeling / Molecules and proteins
Embedding Foundation Model Predictions in Discrete-Choice Models with Structural Guarantees
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a two-stage adapter that embeds tabular foundation model (Mitra, TabPFN) choice probabilities as a fixed feature inside a sign-constrained multinomial logit, preserving MNL economic structure (monotonicity, value-of-time, zero leak) while recovering most of the FM accuracy gain.
Problem
Tabular foundation models achieve strong choice-prediction accuracy but violate economic logic (monotone demand in price, plausible willingness-to-pay, zero probability on unavailable alternatives).
Approach
Treat the foundation model's predicted probability vector as a precomputed feature inside an MNL utility; fit MNL structural coefficients first with the correction held at zero, then freeze them and fit a small zero-initialized neural correction, so the MRS/value-of-time equals the MNL ratio by construction under a fixed-q protocol.
Key results
- Adapter gains 6.4 pp test accuracy on average over MNL and up to 12.8 pp, with 100% cost monotonicity across cells.
- Raw TabPFN is monotone in cost on only 28.9% of LPMC test rows and 40.8% of IoT-Wearables rows; adapter is 100% monotone on all cells.
- Feature-augmented MNL drifts Swissmetro VOT to 1508 and 678 CHF/hr versus MNL 84.4; adapter keeps 84.4 exactly.
- Joint training collapses the structural cost coefficient by 3-17x on transportation datasets (ablation A3).
Surprising findings
- Joint training collapses the structural cost coefficient by 3-17x on transportation datasets, while accuracy stays comparable or slightly higher.
- Adapter's Swissmetro calibration is worse than MNL (ECE 8.4% Mitra, 17.0% TabPFN vs 4.0%), not fixed by temperature scaling, isotonic regression, or bootstrap ensembling.
Limitations
- Adapter trade-off ratios are inherited from the Stage 1 MNL specification and cannot be corrected by the foundation-model term if the MNL is misspecified.
- Row-level splits on Swissmetro stated-preference data may leak respondent preferences across splits.
- Adapter ECE on Swissmetro remains substantially worse than MNL, a bias-driven limitation.
Experimental setup
Datasets: Swissmetro LPMC IoT-Wearables
Benchmarks: Swissmetro LPMC IoT-Wearables
Metrics: test accuracy monotonicity rate value-of-time / willingness-to-pay trade-off ratio availability leak expected calibration error NLL Brier score aggregate share change under counterfactual cost increase
Baselines: multinomial logit raw foundation model monotonic neural network feature-augmented multinomial logit convex ensemble knowledge distillation
Builds on: multinomial logit TabPFN Mitra knowledge distillation monotonic neural networks
Closest papers by meaning
- Logic-Logit: A Logic-Based Approach to Choice Modeling (ICLR 2025 · similarity 0.84)
- Risk Aware Benchmarking of Large Language Models (ICML 2024 · similarity 0.83)
- Decongestion by Representation: Learning to Improve Economic Welfare in Marketplaces (ICLR 2024 · similarity 0.83)
- 3D-Properties: Identifying Challenges in DPO and Charting a Path Forward (ICLR 2025 · similarity 0.82)
- Preference Learning of Latent Decision Utilities with a Human-like Model of Preferential Choice (NeurIPS 2024 · similarity 0.82)
- Multivariate Stochastic Dominance via Optimal Transport and Applications to Models Benchmarking (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.