Research map / Large language models / AI and society
Persuasion for Social Good: How to Build and Break AI
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
An AAAI-25 invited talk abstract summarizing the speaker's work on persuasive AI: the Cicero Diplomacy negotiation agent, persuasion-based jailbreaking of GPT-4, and a study of chatbot identity disclosure effects on donation persuasion.
Problem
Studying how AI systems can persuade humans for social good, how persuasion can be used to probe AI safety through jailbreaking, and how chatbot identity disclosure affects people's behavior.
Approach
Decouples strategic planning (an RL-based planning engine on symbolic game states) from language generation (a language-model dialogue model) for negotiation in Diplomacy; and uses a taxonomy of 40 persuasion strategies to paraphrase harmful queries into persuasive prompts for jailbreaking.
Key results
- Cicero achieved 2nd place in an online Diplomacy tournament (published in Science).
- Persuasive prompts raised attack success rate on GPT-4 from 0% to 92%.
- Online factorial experiment found people are more likely to donate when they believe they are talking to humans rather than chatbots.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use persuasive prompts instead of traditional attacks | Harmful-query attacks on GPT-4 · GPT-4 | attack success rate: 0% to 92% | Helped |
| Make chatbot identity hidden rather than disclosed | Donation persuasion task · Donation persuasion task | donation likelihood | Helped |
| Increase chatbot competence | Chatbot in donation persuasion task when users know it is a chatbot · Donation persuasion task | donation likelihood | Helped |
Surprising findings
- Persuasive paraphrasing of harmful queries is more human-readable and harder to defend than gradient-based attacks.
Limitations
- Abstract-level talk summary; few methodological details given.
- Defense mechanisms (e.g., summarization) described only as promising.
Experimental setup
Metrics: attack success rate tournament placement
Builds on: Cicero
Closest papers by meaning
- Measuring And Improving Persuasiveness Of Large Language Models (ICLR 2025 · similarity 0.84)
- Undermining Mental Proof: How AI Can Make Cooperation Harder by Making Thinking Easier (AAAI 2025 · similarity 0.81)
- Position: Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedback (ICML 2024 · similarity 0.81)
- Theory of Mind and Persuasion Beyond Conversation: Assessing the Capacity of LLMs to Induce Belief States via Planning and Action (NeurIPS 2026 · similarity 0.81)
- Regulating AI: Applying Insights from Behavioural Economics and Psychology to the Application of Article 5 of the EU AI Act (AAAI 2024 · similarity 0.80)
- Towards Trustworthy Autonomous Systems via Conversations and Explanations (AAAI 2024 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.