Research map / Multi-agent systems and games / LLM agents
Measuring and Mitigating Rapport Bias of Large Language Models under Multi-Agent Social Interactions
Maojia Song, Tej Deep Pala, Ruiwen Zhou, Weisheng Jin, Amir Zadeh, Chuan Li, Dorien Herremans, Soujanya Poria
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Studies how LLMs build rapport from prior multi-agent interactions, resist misinformation, and integrate peer input, and proposes ways to measure and mitigate the resulting rapport bias.
Problem
LLMs deployed in multi-agent systems may be shaped by peer interactions in ways beyond conformity bias, such as building rapport from previous interactions, which can affect resistance to misinformation and integration of peer input.
Approach
Broadens the scope of social bias in multi-agent LLM settings from conformity to rapport built across interactions, and measures and mitigates this rapport bias. Details beyond the abstract are not available.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add empowered prompting | small LLMs (<=32B) · KAIROS | Original accuracy / O-K delta: Original 57.36% -> 61.73%; KAIROS 53.87% -> 54.82%; O-K -5.65% -> -11.25% | Mixed |
| add empowered prompting | large LLMs (>32B) · KAIROS | KAIROS accuracy: 77.46% -> 80.56%; O-K -3.64% -> +0.12% | Improved |
| add reflective prompting | small LLMs (<=32B) · KAIROS | KAIROS accuracy: 53.87% -> 51.04%; O-K -5.65% -> -11.30% | Hurt |
| replace SFT with GRPO | Qwen2.5 and Llama small models · KAIROS | accuracy: +12.3% Original, +16.4% KAIROS on average | Improved |
| include multi-agent context in GRPO training (ablation) | Qwen2.5-7B · KAIROS | KAIROS accuracy (%): 57.7 -> 65.5; O-K -20.7 -> -6.8 | Improved |
| include multi-agent context in GRPO training (ablation) | Qwen2.5-3B · KAIROS | KAIROS accuracy (%): 53.8 -> 57.9 accuracy; O-K -14.2 -> -6.1 | Improved |
| use debating system prompt and debating reward instead of normal prompt and outcome reward (ablation) | GRPO MAS training · KAIROS | accuracy and robustness: Qwen2.5-7B KAIROS 65.5 -> 62.0 | No clear effect |
| filter GRPO data to low-confidence samples (ablation) | GRPO-MAS-DS-DR · KAIROS | KAIROS accuracy (%): Qwen2.5-7B 62.0 -> 54.3; O-K -6.9 -> -14.4 | Hurt |
| filter GRPO data to low-correctness samples (ablation) | GRPO-MAS-DS-DR · KAIROS | KAIROS accuracy (%): Qwen2.5-7B 62.0 -> 49.9; O-K -6.9 -> -21.2 | Hurt |
| fine-tune with SFT on templated gold responses | Qwen2.5-7B · KAIROS | KAIROS accuracy (%): 52.3 -> 44.0; O-K -10.6 -> -22.4 | Hurt |
| higher rapport with peers | LLMs · KAIROS | resistance: avg 31.7-point resistance gap between SUPPORT and OPPOSE-HARD | Mixed |
Closest papers by meaning
- Do as We Do, Not as You Think: the Conformity of Large Language Models (ICLR 2025 · similarity 0.89)
- ALSO: Adversarial Online Strategy Optimization for Social Agents (ICML 2026 · similarity 0.88)
- Systematic Failures in Collective Reasoning under Distributed Information in Multi-Agent LLMs (ICML 2026 · similarity 0.88)
- Incentivizing Truthful Language Models via Peer Elicitation Games (NeurIPS 2025 · similarity 0.88)
- Bias Amplification in Language Model Evolution: An Iterated Learning Perspective (NeurIPS 2024 · similarity 0.88)
- More Capable, Less Cooperative? When LLMs Fail at Zero-Cost Collaboration (ICML 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
