Research map / Large language models / Alignment and preferences
Quantifying and Optimizing Global Faithfulness in Persona-driven Role-playing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the Active-Passive-Constraint (APC) score, a fine-grained NLI- and relevance-based faithfulness metric for persona-driven role-playing, distilled into DeBERTa-V3 discriminators, and uses it as a reward for DPO to improve persona faithfulness.
Problem
Persona-driven role-playing (PRP) agents must stay faithful to all persona statements, but existing faithfulness evaluation relies on coarse-grained, unclear LLM-based scoring.
Approach
Treat PRP faithfulness as a MAX-CSP over persona statements: each statement is an active constraint (response must be entailed) when relevant to the query, or a passive constraint (response must not be contradicted) when irrelevant. The APC score sums per-statement NLI probabilities weighted by relevance probabilities, and a regularized ΔAPC subtracts a neutral baseline. APC is then used as a rewa
Key results
- On original characters, ΔAPC from the DeBERTa evaluator correlates closely with human scores; RAG+APC-DPO reaches 5.8 ΔAPC (DeBERTa) on Eve (30 statements) vs 4.8 for RAG without DPO.
- Distilled DeBERTa discriminators reach about 90% accuracy relative to GPT-4 labels; DeBERTa-large matches Gemma-2B-level performance at about 6x faster inference.
- On famous figures with 77–599 statements, RAG+APC-DPO achieves the best average ΔAPC of 6.9 versus 5.7 for RAG alone and 3.5 for Gemma-7B.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add APC-based DPO to RAG (ablation) | RAG with Gemma-1.1-7B-it · Famous figures (average) | ΔAPC: 5.7 -> 6.9 | Helped |
| add APC-based DPO to RAG (ablation) | RAG with Gemma-1.1-7B-it · Alice, Bob, and Eve | ΔAPC: 2.8 -> 2.9; 4.0 -> 5.2; 4.8 -> 5.8 | Helped |
| add APC-based DPO to EU (ablation) | EU with Gemma-1.1-7B-it · Famous figures (average) | ΔAPC: -0.7 -> 0.2 | Helped |
| add APC-based DPO to EU (ablation) | EU with Gemma-1.1-7B-it · Alice, Bob, and Eve | ΔAPC: 2.6 -> 2.7; 3.4 -> 3.8; 3.6 -> 3.9 | Mixed |
| add APC-based DPO to LCM (ablation) | LCM with Gemma-1.1-7B-it · Alice, Bob, and Eve | ΔAPC: 2.6 -> 2.8; 4.5 -> 5.3; 3.9 -> 5.1 | Helped |
| use experience upload instead of the persona-agnostic Gemma baseline | Gemma-1.1-7B-it · Famous figures (average) | ΔAPC: 3.5 -> -0.7 | Hurt |
| use RAG instead of LCM | in-context PRP methods · Alice | ΔAPC: 2.6 -> 2.8 | Helped |
| increase the number of in-context persona statements beyond the effective range (ablation) | LCM for Eve · Eve | ΔAPC | Hurt |
| add APC-based DPO to the model (ablation) | PRP methods for Voldemort · Voldemort | active-constraint reward | Helped |
| add APC-based DPO to the model (ablation) | PRP methods for Voldemort · Voldemort | passive-constraint penalty | Helped |
| use APC score rather than coarse-grained scoring | faithfulness evaluator · Alice, Bob, and Eve | Spearman correlation with human evaluation: 97.18 vs 92.42; 99.10 vs 86.27; 99.10 vs 81.40 | Helped |
Surprising findings
- Experience upload (EU) degrades faithfulness on famous figures, averaging -0.7 ΔAPC, below the vanilla model.
- Coarse-grained GPT-4 scoring correlates less with human judgment than APC and degrades as persona size grows (e.g., 81.40 vs 99.10 Spearman-style consistency on Eve).
Limitations
- APC scoring traverses all persona statements, which is inefficient as statement count grows.
- Summing satisfaction probabilities treats all statements as equally important and ignores semantic redundancy.
- Evaluation relies on GPT-4-distilled discriminators and GPT-4 labels, introducing GPT-4 preference bias.
Experimental setup
Datasets: Character-LLM persona statements (Beethoven, Newton, Socrates) Wikipedia-derived persona statements for famous figures GPT-4-generated relevance and NLI data
Benchmarks: Character-LLM interview questions on handcrafted original characters (Alice, Bob, Eve) Interview questions for famous figures (Spartacus, Hermione, Voldemort, Cleopatra, Caesar, MLK)
Metrics: APC score ΔAPC score human score (0-10 scale) Spearman correlation with human evaluation
Baselines: Gemma-1.1-7b-it prompting ChatGPT GPT-4 Experience Upload Long-context Memory Retrieval-augmented generation GPT-4 coarse-grained score
Builds on: direct preference optimization Character-LLM retrieval-augmented generation DeBERTa-V3
Closest papers by meaning
- Consistently Simulating Human Personas with Multi-Turn Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
- Codifying Character Logic in Role-Playing (NeurIPS 2025 · similarity 0.87)
- Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs (ICLR 2024 · similarity 0.85)
- Persona Vectors: Monitoring and Controlling Character Traits in Language Models (NeurIPS 2026 · similarity 0.84)
- CoSER: Coordinating LLM-Based Persona Simulation of Established Roles (ICML 2025 · similarity 0.84)
- LLM Generated Persona is a Promise with a Catch (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.