Research map / Multi-agent systems and games / Games and equilibria
Joint Scoring Rules: Competition Between Agents Avoids Performative Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes zero-sum joint scoring rules over multiple predictor agents so a principal can elicit honest conditional predictions and deterministically pick their most preferred action, avoiding performative prediction; validated in a toy environment.
Problem
Conditional predictions from an expert agent can be manipulated to steer the principal's action toward more predictable outcomes (performative prediction), and with a single agent and symmetric scoring rule the principal cannot always pick their most preferred action.
Approach
Evaluate multiple agents with a linearly zero-sum, strictly proper, symmetric (LSS) scoring rule combined with an ISA-max decision rule, so agents have no incentive to influence the chosen action and honest conditional predictions become an equilibrium that identifies the principal's best action.
Key results
- Zero-sum (LSS) training increases predictive accuracy and principal utility, while no-intervention training becomes performative and its non-performative accuracy sharply drops after an initial increase.
- An LSS objective using two dropout-generated predictions untrains previously learned performativity faster and to a larger degree than exact LSS or non-performative training (statistically significant).
- Theory: ISA-max decision rule with LSS scoring is jointly quasi-strictly proper for n>=2; LSS is shown to be the unique scoring form achieving this; binary search identifies the best action in O(log|A|) comparisons.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train with an LSS zero-sum objective instead of no intervention | Predictive model in the toy environment · Toy environment, predictions across all actions | predictive accuracy | Helped |
| Train with an LSS zero-sum objective instead of no intervention | Predictive model in the toy environment · Toy environment | principal utility | Helped |
| Train without an intervention rather than with an LSS objective | Predictive model in the toy environment · Toy environment, chosen actions | predictive accuracy | Helped |
| Use two distinct predictions from the same model in the LSS objective instead of identical detached predictions | LSS objective · Toy environment, previously performative model | performativity | Helped |
| Use two distinct predictions from the same model in the LSS objective instead of identical detached predictions | LSS objective · Toy environment, previously performative model | predictive accuracy | Helped |
| Use two distinct predictions from the same model in the LSS objective instead of identical detached predictions | LSS objective · Toy environment, previously performative model | principal utility | Helped |
| Use an exact LSS objective with identical detached predictions rather than two distinct predictions | LSS objective · Toy environment, previously performative model | performativity | No clear effect |
| Remove the gradient through the principal's action choice (ablation) | Training of a previously performative model · Toy environment, previously performative model | performativity | Hurt |
Surprising findings
- Training without intervention gains the largest performative accuracy but its non-performative accuracy later sharply drops, showing performativity compounds during training.
- Two slightly different dropout-generated predictions from the same model untrain performativity better than exact self-agreement, suggesting disagreement helps escape stuck gradients.
Limitations
- Experiments only in a toy environment with eight actions and outcomes; not tested on real-world data or complex environments.
- Assumes all agents share the same information about ground truth; the disagreement-seeking rule is fragile to small errors or noise.
- Mean-max decision rule proof only holds for n=2 agents; for n>=3 the guarantee requires coalition assumptions.
Experimental setup
Datasets: synthetic toy environment with randomly initialized neural net ground truth
Benchmarks: toy environment with eight actions and eight outcomes
Metrics: predictive accuracy performative predictive accuracy principal utility performativity
Baselines: no intervention training non-performative training single-agent symmetric scoring rule
Builds on: Othman and Sandholm 2010 impossibility result Oesterheld and Conitzer 2020 Chen et al. 2011 Perdomo et al. 2020 performative prediction
Closest papers by meaning
- Advantage Alignment Algorithms (ICLR 2025 · similarity 0.85)
- Opponent Modeling based on Subgoal Inference (NeurIPS 2024 · similarity 0.85)
- Maximizing utility in multi-agent environments by anticipating the behavior of other learners (NeurIPS 2024 · similarity 0.85)
- Look-Ahead Reasoning on Learning Platforms (NeurIPS 2025 · similarity 0.85)
- Impartial Selection with Predictions (NeurIPS 2025 · similarity 0.85)
- Configurable Mirror Descent: Towards a Unification of Decision Making (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.