Research map / Label-efficient and robust learning / Domain adaptation and generalization

Language-Induced Priors for Domain Adaptation

NeurIPS 2026 · Label-efficient and robust learning › Domain adaptation and generalization · approach: Conditional distribution alignment · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Language-Induced Priors (LIP): an LLM answers subgroup multiple-choice relevance queries about source domains given a target text description, a conditional-logit model fits the prior, and an EM algorithm uses it to weight source domains in cold-start domain adaptation.

Problem

Domain adaptation in the cold-start regime (scarce target data) cannot reliably distinguish relevant from irrelevant source domains, causing negative transfer.

Approach

Use expert textual descriptions of only the target domain, elicited via LLM multiple-choice preferences over source subgroups, to fit a prior over source relevance (LIP), which is then integrated into a Bayesian hierarchical EM that refines relevance weights as target data accumulates.

Key results

• On C-MAPSS at 90% RUL, LIP-C achieves 14.3 RMSE vs 21.4 for Uniform EM and 43.7 for Target Only.
• On MuJoCo hopper with N0=128, LIP-C reaches 2670 reward vs 1283 for Uniform EM and 19 for Target Only.
• LIP-G (a misspecified prior preferring g=7) still outperforms uniform prior at N0=128 but recovers more slowly at N0=512 (1760 vs 2491).

Surprising findings

• A misspecified LLM-generated prior (LIP-G preferring g=7 over closer gravities) helped over uniform prior in the cold-start regime but slowed recovery at larger N0.
• On C-MAPSS at 30% RUL, LIP-C (20.1) underperforms Uniform EM (15.2), showing LIP is not uniformly beneficial.

Limitations

• Theoretical finite-sample guarantee requires a basin-entry hypothesis that depends on LIP correctness.
• Gaussian/quadratic approximations (small tau, expected Fisher information) are used for efficiency; the exact guarantees are for tau=0 and specific settings.
• Only LLM-produced priors on a few tasks; LIP quality is crucial and not robustly characterized across LLMs.

Experimental setup

Datasets: C-MAPSS FD001 MuJoCo hopper replay data

Benchmarks: C-MAPSS FD001 MuJoCo hopper

Metrics: RMSE episodic reward

Baselines: Uniform EM Pooled Target Only

Builds on: expectation-maximization conditional logit IQL soft actor-critic AutoElicit

Closest papers by meaning

• BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Models (NeurIPS 2024 · similarity 0.87)
• BayesPrompt: Prompting Large-Scale Pre-Trained Language Models on Few-shot Inference via Debiased Domain Abstraction (ICLR 2024 · similarity 0.86)
• Boosting Multi-Domain Fine-Tuning of Large Language Models through Evolving Interactions between Samples (ICML 2025 · similarity 0.86)
• Large Language Models to Enhance Bayesian Optimization (ICLR 2024 · similarity 0.86)
• Improving Context-Aware Preference Modeling for Language Models (NeurIPS 2024 · similarity 0.86)
• Revisiting Source-Free Domain Adaptation: Insights into Representativeness, Generalization, and Variety (CVPR 2025 · similarity 0.86)