Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Political Actor Agent: Simulating Legislative System for Roll Call Votes Prediction with Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the Political Actor Agent (PAA), an LLM role-playing agent framework that predicts U.S. House roll call votes using legislator profiles, multi-view planning, and a leader-follower influence mechanism, without training.
Problem
Roll call vote prediction for legislators is limited by embedding-based methods that need predefined features, large training data, and offer poor interpretability; the paper targets interpretable, data-efficient vote prediction.
Approach
Model each legislator as an LLM role-playing agent whose prompt profile includes personal info, constituency, sponsorship and sampled voting records; agents reason from trustee, delegate, and follower views, and leader agents vote first to influence followers, with no model training.
Key results
- PAA with GPT-4o-mini reached 91.8% accuracy and 92.2 F1 on split_244, versus 86.5% accuracy for the best baseline UPPAM on that split.
- On split_622, PAA-G achieved 92.1 accuracy and 93.0 F1, above the best baseline UPPAM (91.7 accuracy, 86.3 F1).
- Ablation on split_244 showed removing the profile module lowered PAA-G accuracy from 91.8 to 78.6.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GPT-4o-mini as the PAA base model instead of the strongest baseline | PAA · split_244 | accuracy: 86.5 ± 0.10 -> 91.8 ± 0.15 | Helped |
| Use GPT-4o-mini as the PAA base model instead of the strongest baseline | PAA · split_433 | accuracy: 88.5 ± 0.03 -> 91.3 ± 0.20 | Helped |
| Use GPT-4o-mini as the PAA base model instead of the strongest baseline | PAA · split_622 | accuracy: 91.7 ± 0.08 -> 92.1 ± 0.10 | Helped |
| Add the profile module (ablation) | PAA · split_244 | accuracy: 78.6 ± 0.10 -> 91.8 ± 0.15 | Helped |
| Add the planning module (ablation) | PAA · split_244 | accuracy: 80.1 ± 0.12 -> 91.8 ± 0.15 | Helped |
| Add the acting module (ablation) | PAA · split_244 | accuracy: 80.7 ± 0.19 -> 91.8 ± 0.15 | Helped |
| Use profiles with anonymized legislator names and bill numbers (ablation) | PAA · split_244 | accuracy: 91.8 ± 0.15 -> 90.8 ± 0.10 | Hurt |
| Swap names of legislators with differing stances in the profile (ablation) | PAA · split_244 | accuracy: 91.8 ± 0.15 -> 90.1 ± 0.08 | Hurt |
| Add legislator personal information to the profile (ablation) | PAA · split_244 | accuracy: 82.9 ± 0.17 -> 91.8 ± 0.15 | Helped |
| Add constituency information to the profile (ablation) | PAA · split_244 | accuracy: 90.9 ± 0.06 -> 91.8 ± 0.15 | No clear effect |
| Add sponsorship activity to the profile (ablation) | PAA · split_244 | accuracy: 83.6 ± 0.09 -> 91.8 ± 0.15 | Helped |
| Add voting records to the profile (ablation) | PAA · split_244 | accuracy: 83.4 ± 0.13 -> 91.8 ± 0.15 | Helped |
Surprising findings
- Using the entire training set of voting records in the profile degraded performance as the training split grew, suggesting excess records confuse the LLM.
- Anonymizing legislator names and bill numbers only slightly reduced performance, which the authors interpret as PAA relying mainly on profile information rather than memorized pretraining knowledge.
Limitations
- Current architecture does not integrate diverse data sources such as social media and news.
- Supports only roll call vote prediction as a downstream task.
- LLM hallucination only partly addressed; consistency is a limited measure, and GPT-4o-mini was less consistent than Llama-3-70B.
Experimental setup
Datasets: 117th-118th U.S. House voting records Wikipedia pages of legislators and constituencies bill sponsorship and cosponsorship data legislators' Twitter posts
Benchmarks: 117th-118th U.S. House roll call votes (split_244, split_433, split_622)
Metrics: accuracy macro-averaged F1
Baselines: ideal-vector LSTM+GCN Vote+MTL PAR UPPAM
Builds on: ideal point model role-playing agents
Closest papers by meaning
- Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models? (NeurIPS 2025 · similarity 0.83)
- Examining Alignment of Large Language Models through Representative Heuristics: the case of political stereotypes (ICLR 2025 · similarity 0.82)
- When Debate Helps: Proposal Supply and Verification-Aware Readout in Multi-Agent Reasoning (NeurIPS 2026 · similarity 0.80)
- Multi-Agent Debate for LLM Judges with Adaptive Stability Detection (NeurIPS 2025 · similarity 0.80)
- A Vision for Reinventing Credible Elections with Artificial Intelligence (AAAI 2025 · similarity 0.80)
- Enhancing Decision-Making of Large Language Models via Actor-Critic (ICML 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.