Research map / Large language models / AI and society
Unlocking the Potential of Global Human Expertise
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RHEA, an evolutionary AI framework that distills diverse expert-submitted prescriptors into neural networks and recombines/refines them via population-based search, validated on a synthetic domain and the XPRIZE Pandemic Response Challenge.
Problem
Bringing together diverse human expertise, in the form of many heterogeneous models, is hard as the knowledge base grows; the paper targets identifying, combining and refining complementary expert solutions for global policy problems such as COVID-19 non-pharmaceutical interventions.
Approach
Distill each expert prescriptor (rules, epidemiological models, gradient-based methods, etc.) into a canonical neural network by behavioral cloning, inject these into the initial population of an evolutionary multi-objective search (ESP with NSGA-II), so evolution recombines and refines expert building blocks beyond what MoE or weighted ensembles can reach.
Key results
- RHEA's REM is nearly 100%, meaning decision-makers with empirical cost preferences would most often pick RHEA prescriptors.
- RHEA improves about 75% of the distilled Pareto front solutions (domination rate w.r.t. Distilled).
- In the synthetic domain, RHEA reliably discovers the full optimal Pareto front, whereas evolution alone does not, even with 10 interventions.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Recombine and refine distilled expert prescriptors through evolutionary search | RHEA · XPRIZE COVID-19 policies | Pareto-front performance | Helped |
| Initialize evolution with distilled expert prescriptors rather than random models | ESP · XPRIZE COVID-19 policies | REM | Helped |
| Distill submitted expert prescriptors into neural networks | XPRIZE prescriptors · XPRIZE COVID-19 policies | rank correlation with submitted models: ≈0.7 | Helped |
| Use cases per 100K residents as input instead of case growth rate | Distilled prescriptors · XPRIZE COVID-19 policies | training-data fit | Helped |
| Initialize search with diverse expert prescriptors rather than random solutions | Illustrative synthetic domain · Illustrative domain | optimal Pareto-front discovery | Helped |
| Use evolution from scratch without expert knowledge | Illustrative synthetic domain · Illustrative domain | optimal Pareto-front discovery | Hurt |
| Use GPI-LS instead of RHEA | Illustrative synthetic domain · Illustrative domain | Pareto-front discovery | Hurt |
Surprising findings
- Some expert teams with unimpressive quantitative performance make outsized contributions to the final evolved front.
- Evolution from scratch sometimes finds better solutions than distilled expert models, though RHEA still improves most distilled solutions.
Limitations
- Evaluation relies on a surrogate predictor rather than real-world or user studies.
- Uniform IP cost measure; geo-specific cost calibration and temporal discounting not included.
- Approximation errors from distillation and surrogate may compound when applied to the real world.
Experimental setup
Datasets: Oxford COVID-19 government response tracker XPRIZE Pandemic Response Challenge submissions (169 prescriptors)
Benchmarks: XPRIZE Pandemic Response Challenge evaluation (reference predictor) Synthetic Pareto-front domain
Metrics: hypervolume improvement (HVI) domination rate (DR) max case reduction (MCR) R1 uniform (RUN) R1 empirical (REM)
Baselines: Distilled expert models Evolution from scratch (random init) Random initial models GPI-LS Mixture-of-Experts Weighted Ensembling GPI-PD Envelope Q-Learning
Builds on: Evolutionary Surrogate-assisted Prescription (ESP) NSGA-II knowledge distillation Mixture-of-Experts Weighted Ensembling
Closest papers by meaning
- Confounding-Robust Deferral Policy Learning (AAAI 2025 · similarity 0.83)
- Human Expertise in Algorithmic Prediction (NeurIPS 2024 · similarity 0.82)
- Decoding AI’s Nudge: A Unified Framework to Predict Human Behavior in AI-Assisted Decision Making (AAAI 2024 · similarity 0.81)
- An Evolutionary Perspective on AI Alignment (Student Abstract) (AAAI 2025 · similarity 0.81)
- Position: Stop Making Unscientific AGI Performance Claims (ICML 2024 · similarity 0.81)
- When Models Know More Than They Can Explain: Quantifying Knowledge Transfer in Human-AI Collaboration (NeurIPS 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.