Research map / Large language models / AI and society

Feedback Forensics: A Toolkit to Measure AI Personality

NeurIPS 2026 · Large language models › AI and society · approach: Data commons and human baselines

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Feedback Forensics, an open-source toolkit that uses AI annotators to measure personality traits encouraged by pairwise human feedback datasets and exhibited by AI models.

Problem

Model personality traits (tone, style, sycophancy) are hard to specify upfront and are only implicitly captured by human-feedback leaderboards like Chatbot Arena, leaving their concrete effects opaque.

Approach

Annotate pairwise response data with relative AI-judge trait selections (plus human preference and target-model annotations) and compute relevance, Cohen's kappa, and a combined strength metric to quantify how much each trait is encouraged by feedback or exhibited by a model.

Key results

• Gemini-2.5-Flash single-vote annotator chosen; GPT-5-mini achieves mean 92% choice agreement with human annotations, Gemini-2.5-Flash close second.
• Chatbot Arena analysis finds well-formatted, verbose, factually correct and confident responses are encouraged; PRISM prefers more polite and less casual language.
• GPT-5 stands out for less verbose, less formatted responses; Llama-4-Maverick arena version is more verbose, enthusiastic and engaging than public version.

Surprising findings

• Expert human annotations encourage the same traits as non-expert ones but with less strength, while AI annotations show the most strength, suggesting AI annotators follow simpler heuristics.

Limitations

• Results are relative to the prompt/response data distribution and reference models (GPT-4o).
• Relies on LLM annotators that may carry their own biases; correlation does not imply causation.
• Relevance agreement with humans is lower than choice agreement; human validation based on a single author's annotations.

Experimental setup

Datasets: Chatbot Arena (arena-human-preference-100k subset, 10k) MultiPref (10k) PRISM (~8k) Llama-4-Maverick-03-26-Experimental battles 500 English prompts from arena-human-preference-100k

Benchmarks: Chatbot Arena MultiPref PRISM

Metrics: strength relevance Cohen's kappa agreement human agreement (relevance and choice)

Baselines: GPT-4o-mini GPT-4.1-mini GPT-5-mini Gemini-2.5-Flash

Builds on: Inverse Constitutional AI VibeCheck Chatbot Arena

Closest papers by meaning

• Open Models, Closed Minds? On Agents Capabilities in Mimicking Human Personalities through Open Large Language Models (AAAI 2025 · similarity 0.87)
• RealityTest: How People Probe AI Identity and Whether Models Disclose It (NeurIPS 2026 · similarity 0.85)
• Persona Vectors: Monitoring and Controlling Character Traits in Language Models (NeurIPS 2026 · similarity 0.85)
• OdysSim: Building Foundation Models for Human Behavior Simulation (NeurIPS 2026 · similarity 0.85)
• Position: AI Evaluation Should Learn from How We Test Humans (ICML 2025 · similarity 0.85)
• Instrumental Choices: Measuring the Propensity of LLM Agents to Pursue Instrumental Behaviors (NeurIPS 2026 · similarity 0.84)