Research map / Large language models / Alignment and preferences
What Do Large Language Models Know About Opinions?
Erfan Jahanparast, Zhiqing Hong, Serina Chang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| probe residual stream activations instead of reading next-token probabilities | Llama-3.1-8B · OpinionQA/SubPOP 2-option | KL divergence: 0.223 -> 0.073 | Improved |
| probe residual stream activations instead of reading next-token probabilities | Llama-3.1-8B · OpinionQA/SubPOP 3-option | KL divergence: 0.241 -> 0.113 | Improved |
| use a nonlinear MLP probe instead of logistic regression (ablation) | linear probe · OpinionQA/SubPOP 2-option | KL divergence: 0.073 -> 0.068 | No clear effect |
| use a single shared probe for all groups instead of per-group probes (ablation) | Llama-3.1-8B probe · OpinionQA/SubPOP 2-option | KL divergence: 0.071 -> 0.074 | No clear effect |
| switch prompt format from QA to BIO (ablation) | Llama-3.1-8B probe · OpinionQA/SubPOP 2-option | KL divergence: 0.143 -> 0.067 | Improved |
| switch prompt format from QA to PORTRAY (ablation) | Llama-3.1-8B probe · OpinionQA/SubPOP 3-option | KL divergence: 0.219 -> 0.119 | Improved |
| fine-tune only the unembeddings (ablation) | Llama-3.1-8B · OpinionQA/SubPOP 2-option | KL divergence: 0.213 -> 0.082 | Improved |
| fine-tune only the unembeddings (ablation) | Llama-3.1-8B · OpinionQA/SubPOP 3-option | KL divergence: 0.234 -> 0.126 | Improved |
| LoRA fine-tune the full model rather than probing (ablation) | Llama-3.1-8B · OpinionQA/SubPOP 2-option | KL divergence: 0.213 -> 0.046 | Improved |
| probe on GlobalOpinionQA country opinions | Llama-3.1-8B · GlobalOpinionQA 2-option | KL divergence: 0.150 -> 0.092 | Improved |
| probe on GlobalOpinionQA country opinions | Llama-3.1-8B · GlobalOpinionQA 3-option | KL divergence: 0.193 -> 0.142 | Improved |
| intervene on all layers 2-30 instead of only layers 9-18 (ablation) | SAE steering · Steering avg improvement | improvement ratio: 0.440 vs. 0.463 | No clear effect |
Closest papers by meaning
- Bridging Human and LLM Judgments: Understanding and Narrowing the Gap (NeurIPS 2025 · similarity 0.89)
- Potemkin Understanding in Large Language Models (ICML 2025 · similarity 0.89)
- More Is Not More: What Matters for Diversity in LLM Opinions? (NeurIPS 2026 · similarity 0.88)
- Do Large Language Models Perform the Way People Expect? Measuring the Human Generalization Function (ICML 2024 · similarity 0.88)
- Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference (ICML 2024 · similarity 0.88)
- Characterizing Truthfulness in Large Language Model Generations with Local Intrinsic Dimension (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
