Research map / Multi-agent systems and games / Games and equilibria
Re-evaluating evaluation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add a 2-dimensional cyclic interaction component to Elo | Elo · Go win-loss prediction (8 algorithms) | Frobenius norm: 0.85 -> 0.35 | Helped |
| Add a 2-dimensional cyclic interaction component to Elo | Elo · Go win-loss prediction (8 algorithms) | logistic loss: 1.41 -> 1.27 | Helped |
| Use Nash averaging over environments instead of uniform averaging | Agent evaluation on Arcade Learning Environment · Arcade Learning Environment | top-agent ranking | Mixed |
Closest papers by meaning
- Conservative classifiers do consistently well with improving agents: characterizing statistical and online learning (NeurIPS 2025 · similarity 0.83)
- AI Evaluation Should Require Standardized Item-Level Data Releases (NeurIPS 2026 · similarity 0.83)
- MixEval-X: Any-to-any Evaluations from Real-world Data Mixture (ICLR 2025 · similarity 0.83)
- SureMap: Simultaneous mean estimation for single-task and multi-task disaggregated evaluation (NeurIPS 2024 · similarity 0.82)
- MLR-Bench: Evaluating AI Agents on Open-Ended Machine Learning Research (NeurIPS 2025 · similarity 0.82)
- Re-evaluating Open-ended Evaluation of Large Language Models (ICLR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.