Research map / Large language models / AI and society
MindVote: When AI Meets the Wild West of Social Media Opinion
Xutao Mao, Ezra Xuanru Tao, Leyao Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune Llama-2-13B on survey data (SubPop) instead of the general instruction model | Llama-2-13B · MindVote | 1-Wasserstein: 0.807 -> 0.774 | Hurt |
| fine-tune Llama-3-70B on survey data (SubPop) | Llama-3-70B · MindVote | 1-Wasserstein: 0.844 -> 0.805 | Hurt |
| fine-tune Mistral-7B on survey data (SubPop) | Mistral-7B · MindVote | 1-Wasserstein: 0.808 -> 0.782 | Hurt |
| fine-tune Llama-2-13B on survey data (SubPop) | Llama-2-13B · MindVote | One-hot accuracy: 0.369 -> 0.378 | Mixed |
| add platform context (ablation) | LLM prompts (average of 15 models) · MindVote | 1-Wasserstein drop (%): -5.12 | Improved |
| add topical context (ablation) | LLM prompts (average of 15 models) · MindVote | 1-Wasserstein drop (%): -4.52 | Improved |
| add all social context metadata (ablation) | LLM prompts (average of 15 models) · MindVote | 1-Wasserstein drop (%): -5.91 | Improved |
| add all context to survey-specialized models (ablation) | SubPop-Mistral-7B · MindVote | 1-Wasserstein drop (%): -7.54 | Improved |
| context priming versus no context priming in zero-shot (ablation) | all evaluated LLMs · MindVote | 1-Wasserstein | Improved |
| increase few-shot examples from 3 to 4 (ablation) | open-source LLMs · MindVote | 1-Wasserstein: 4-shot worse than 3-shot | No clear effect |
| increase context length, informality or niche topic (ablation) | LLMs, especially survey-specialized models · MindVote | 1-Wasserstein | Hurt |
| translate polls into another language (EN vs ZH) (ablation) | LLMs · MindVote | 1-Wasserstein | No clear effect |
Closest papers by meaning
- DEBATE: A Large-Scale Benchmark for Evaluating Opinion Dynamics in Role-Playing LLM Agents (NeurIPS 2026 · similarity 0.87)
- AI-Mediated Communication Can Steer Collective Opinion (NeurIPS 2026 · similarity 0.87)
- Generative Social Choice: The Next Generation (ICML 2025 · similarity 0.87)
- Measuring And Improving Persuasiveness Of Large Language Models (ICLR 2025 · similarity 0.87)
- Representative Ranking for Deliberation in the Public Sphere (ICML 2025 · similarity 0.86)
- SocialStigmaQA: A Benchmark to Uncover Stigma Amplification in Generative Language Models (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
