Research map / Probabilistic and scientific modeling / Molecules and proteins
DeepHalo: A Neural Choice Model with Controllable Context Effects
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DeepHalo, a neural choice model that decomposes utility into order-controlled context (Halo) interaction terms via a residual architecture with feature inputs, and is a universal approximator in the featureless setting.
Problem
Modeling context-dependent choice (Halo/context effects among alternatives) with features while keeping interaction order explicit and interpretable, since existing models either ignore features, restrict to first-order effects, or entangle all orders.
Approach
Decompose utility via inclusion-exclusion into order-wise permutation-equivariant interaction terms, and implement them as a residual network where layer l adds interactions of order up to l using shared context summaries (aggregation over alternatives) modulated by head-specific transforms of base embeddings.
Key results
- On LPMC test NLL, DeepHalo achieves 0.6430 vs. 0.6487 for TCNet (best baseline).
- On Expedia test NLL, DeepHalo achieves 2.5263 vs. 2.5340 for TCNet.
- On SFOshop test NLL, DeepHalo achieves 1.5263 vs. CMNL 1.5686; on SFOwork test NLL 0.8066 vs. MLP 0.8120.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DeepHalo instead of MLP | Featureless choice model · Hotel test | NLL: 0.7523 -> 0.7483 | Helped |
| Use DeepHalo instead of MLP | Featureless choice model · SFOshop test | NLL: 1.5523 -> 1.5263 | Helped |
| Use DeepHalo instead of MLP | Featureless choice model · SFOwork test | NLL: 0.8120 -> 0.8066 | Helped |
| Use DeepHalo instead of TCNet | Feature-based choice model · LPMC test | NLL: 0.6487 -> 0.6430 | Helped |
| Use DeepHalo instead of TCNet | Feature-based choice model · Expedia test | NLL: 2.5340 -> 2.5263 | Helped |
| Increase model depth from 3 to 7 at a fixed parameter budget (ablation) | DeepHalo · Synthetic choice data, 200k parameters | Training RMSE: 0.0434 -> 0.0183 | Helped |
| Increase model depth from 3 to 7 at a fixed parameter budget (ablation) | DeepHalo · Synthetic choice data, 500k parameters | Training RMSE: 0.0419 -> 0.0130 | Helped |
| Constrain interaction order to 2 rather than using full-order DeepHalo (ablation) | DeepHalo · SFOshop | NLL: 1.5339 vs. 1.5331 | No clear effect |
| Constrain interaction order to 2 rather than post hoc truncating an MLP at order 2 (ablation) | Choice model · Synthetic (true order ≤2) | NLL: 0.7643 -> 0.0922 | Helped |
| Constrain interaction order to 2 rather than post hoc truncating an MLP at order 2 (ablation) | Choice model · SFOshop | NLL: 1.6065 -> 1.5339 | Helped |
Surprising findings
- On the Expedia dataset, TCNet attains a lower train NLL (2.4865) than DeepHalo (2.5059), while DeepHalo wins on test.
- On SFOshop, DeepHalo restricted to second-order interactions (1.5339) nearly matches full-order DeepHalo (1.5331).
Limitations
- Full-rank featureless case requires width J and deeper networks for high-order interactions; width expansion is heuristic.
- Context effect coefficients are not identifiable without constraints; only relative context effects are interpretable.
- Interpretability and efficiency claims are largely qualitative; experiments are on modest-size datasets.
Experimental setup
Datasets: Hotel SFOshop SFOwork LPMC Expedia synthetic beverage market share synthetic high-order choice data
Benchmarks: Hotel SFOshop SFOwork LPMC Expedia
Metrics: negative log-likelihood RMSE
Baselines: MNL MLP CMNL TasteNet RUMnet DLCL ResLogit FateNet TCNet
Builds on: universal logit model FATE contextual MNL low-rank Halo MNL
Closest papers by meaning
- Improving Context-Aware Preference Modeling for Language Models (NeurIPS 2024 · similarity 0.88)
- Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHF (ICLR 2024 · similarity 0.88)
- Preference Learning of Latent Decision Utilities with a Human-like Model of Preferential Choice (NeurIPS 2024 · similarity 0.87)
- Deep Value Benchmark: Measuring Whether Models Generalize Deep values or Shallow Preferences (NeurIPS 2025 · similarity 0.86)
- On-the-fly Preference Alignment via Principle-Guided Decoding (ICLR 2025 · similarity 0.86)
- Eliciting Human Preferences with Language Models (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.