Research map / Large language models / Alignment and preferences

Prompts to Proxies: Emulating Human Preferences via a Compact LLM Ensemble

NeurIPS 2026 · Large language models › Alignment and preferences · approach: Reward model design and preference models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes P2P, a two-stage inference-only system that builds a diverse pool of persona-prompted LLM agents via entropy-guided adaptive sampling and then selects a compact ensemble with constrained lasso to reproduce American Trends Panel and WVS survey distributions.

Problem

Aligning LLM synthetic agents to the aggregate response distribution of a target human population using only public aggregate survey data, without individual demographic profiles or finetuning.

Approach

Frames population preference alignment as representation learning with a functional basis of proxy agents, generated by attribute-based prompting with bandit-style entropy-driven sampling, and recovers population weights via L1-regularized (constrained lasso) regression over survey questions.

Key results

• P2P achieves average test MSE 0.014 across 14 ATP waves versus 0.026 (Vanilla) and 0.029 (PERSONA), a 43% improvement over the best baseline, at about $0.8 per survey.
• On the SFT stress test, P2P reaches 1-JSD of 0.71-0.75 vs 0.78 for the SFT-calibrated model while using under 3% of its training data.
• Cross-locale WVS results give test MSE of about 0.015 (US), 0.014 (GB) and 0.024 (HK).

Surprising findings

• Transferring US-generated endowments to Hong Kong yields lower test MSE than HK-native endowments, though HK-native does better on culturally divergent questions.
• Replacing the regression stage with simple averaging of agent responses roughly doubles test MSE (0.0104 to 0.0254).

Limitations

• Stress test on SFT baseline is limited to one protocol with topic shift and Gemini-based pipelines.
• Performance degrades for culturally distinct locales (HK) and with weaker backends such as smaller models.
• Only categorical survey responses are handled; freeform output is left to future work.

Experimental setup

Datasets: American Trends Panel (via OpinionsQA) World Values Survey Wave 7 PERSONA

Benchmarks: American Trends Panel (14 waves) World Values Survey Wave 7

Metrics: test MSE entropy cost (USD) 1-JSD EMD MCPA

Baselines: Vanilla endowment prompting PERSONA SFT distribution-calibrated model

Builds on: OpinionsQA pluralistic alignment demographic conditioning PERSONA

Closest papers by meaning

• Latent Preference Coding: Aligning Large Language Models via Discrete Latent Codes (ICML 2025 · similarity 0.87)
• Questioning the Survey Responses of Large Language Models (NeurIPS 2024 · similarity 0.87)
• Understanding the Learning Dynamics of Alignment with Human Feedback (ICML 2024 · similarity 0.87)
• No Preference Left Behind: Group Distributional Preference Optimization (ICLR 2025 · similarity 0.86)
• Group Preference Optimization: Few-Shot Alignment of Large Language Models (ICLR 2024 · similarity 0.86)
• When Can Proxies Improve the Sample Complexity of Preference Learning? (ICML 2025 · similarity 0.86)