AI Paper Atlas

Research map / Large language models / Instruction and fine-tuning

Democratizing Diplomacy: A Harness for Evaluating Any Large Language Model on Full-Press Diplomacy

Alexander Duffy, Samuel J Paech, Ishana Shastri, Elizabeth Karpinski, Baptiste Alloui-Cros, Tyler Marques, Matthew Lyle Olson

AAAI 2026 · Large language models › Instruction and fine-tuning · approach: Fine-tuning for reasoning data · 3 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
progressively more aggressive prompts (V1 to V3) for the game agent (ablation)Mistral-Small-24B · Diplomacy hold ratehold rate: 58.9% -> 24.1%Improved
use V3 offensive prompts instead of baseline (ablation)Devstral-Small · Diplomacy as Francewin rate: 3/10 -> 9/10Improved
use V3 prompts (ablation)Mistral-Small-24B · Diplomacy support orderssupport order success: +18%Improved
remove negotiation (no-press) when comparing o3 with Kimi-K2 (ablation)o3 · Diplomacy game scoregame score gap vs Kimi-K2: +0.65 (p=0.79) -> +3.1 (p=0.021)Improved
use jailbreak command persuasion instead of reasoned argument (ablation)persuader model vs Mistral-Small · Diplomacy persuasionallegiance shift success: jailbreak 31%, lies 11%Improved
appeal to reason, empathy or fairness for persuasion (ablation)persuader model vs Mistral-Small · Diplomacy persuasionsuccess rateNo clear effect
increase model sizeLLM players · Diplomacy game scoregame scoreImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy