Research map / Large language models / AI and society

Political Neutrality as Balanced Approval: A Large-Scale Human Evaluation of AI Responses

NeurIPS 2026 · Large language models › AI and society · approach: Data commons and human baselines · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a definition of AI political neutrality as maximum equal approval across opposing issue-side groups and releases the PARETO dataset from a large U.S. human study of AI responses to 20 controversial issues.

Problem

Defining and measuring AI political neutrality in a testable, issue-specific way, since existing neutrality evaluations use only a U.S. liberal-conservative axis and lack human validation.

Approach

Neutrality is defined as the response on the Pareto frontier of approval across opposing issue-side groups that minimizes imbalance (maximum equal approval), measured by surveying participants' own issue positions and approval of AI responses; balanced two-side responses most often reach this point.

Key results

• Every one of the 20 issues has an AI response with approval above 0.60 from both sides.
• The balanced GPT response is the empirical maximum equal approval response in 60% of issues (second place 15%) and sits on the Pareto frontier in 85% of issues.
• Balanced responses lose less than 10% approval on average relative to single-side responses, and default responses (except Grok) lean liberal, with GPT showing the largest gap (0.72 liberal vs 0.60 conservative).

Surprising findings

• Fairness ratings barely drop (<=2%) from single-side to balanced responses, smaller losses than the approval question.
• Grok's default responses lean conservative on some issues and rarely reach the Pareto frontier (5% of issues).

Limitations

• Only 20 U.S. issues and two sides per issue, with sides mapped from one canonical survey question each.
• Responses generated by a single prompting pipeline; only eight response types per prompt, so the empirical MEA is limited to tested responses.
• Approval measures perceived fairness and agreement, not factual accuracy or persuasion quality.

Experimental setup

Datasets: ELI5 One Million Reddit Questions Roper Center iPoll surveys

Benchmarks: PARETO

Metrics: approval score empirical maximum equal approval rate Pareto frontier membership win rate regression coefficients

Baselines: Grok default Gemini default Claude default Llama default GPT single-side

Builds on: pluralistic alignment Overton pluralism political compass tests

Closest papers by meaning

• Position: Political Neutrality in AI Is Impossible — But Here Is How to Approximate It (ICML 2025 · similarity 0.93)
• AI-Mediated Communication Can Steer Collective Opinion (NeurIPS 2026 · similarity 0.86)
• Inverse Constitutional AI: Compressing Preferences into Principles (ICLR 2025 · similarity 0.86)
• When Neutral Summaries Are Not That Neutral: Quantifying Political Neutrality in LLM-Generated News Summaries (Student Abstract) (AAAI 2025 · similarity 0.86)
• Examining Alignment of Large Language Models through Representative Heuristics: the case of political stereotypes (ICLR 2025 · similarity 0.85)
• Collaborative Disagreement Resolution for Scalable Oversight (ICML 2026 · similarity 0.85)