Research map / Multi-agent systems and games / Games and equilibria
Automating Data Annotation under Strategic Human Agents: Risks and Potential Solutions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Studies how retraining a classifier with model-annotated samples (plus few human labels) interacts with strategic agents who best respond, showing acceptance rises while qualification may fall, proposes a probabilistic-sampler "refined retraining" to stabilize dynamics, and analyzes demographic-parity fairness effects.
Problem
Long-term dynamics when ML models are periodically retrained on model-annotated samples while strategic human agents adapt features in response, affecting acceptance, qualification, classifier bias and group fairness.
Approach
Formalizes sequential strategic classification where the training set accumulates model-annotated (from post-best-response agents) and few human-annotated samples; proves acceptance rate increases while qualification may decrease, and proposes labeling model-annotated samples via a probabilistic sampler h_t to stabilize dynamics.
Key results
- Acceptance rate a_t increases monotonically over retraining rounds in Gaussian and German Credit experiments, while qualification rate q_t decreases starting from t=1 under the stated conditions.
- When K/N=0.05 (N=2000, K=100), acceptance rate tends toward 1 in numerous datasets, implying the original retraining accepts all agents in the long run.
- Applying refined retraining to advantaged group only and early stopping can reach near-perfect demographic parity at intermediate rounds (minimum unfairness at t=5 in the reported experiment).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Retrain with model-annotated samples from strategic agents | Original retraining process · Strategic-agent population over retraining rounds | acceptance rate | Helped |
| Retrain with model-annotated samples from strategic agents | Original retraining process · Strategic-agent population under the theorem's stated conditions | qualification rate | Hurt |
| Use a probabilistic sampler instead of hard model labels for model-annotated samples (ablation) | Refined retraining process · Strategic-agent population | acceptance rate | No clear effect |
| Use a probabilistic sampler instead of hard model labels for model-annotated samples (ablation) | Refined retraining process · Strategic-agent population | classifier bias | No clear effect |
| Use a probabilistic sampler instead of hard model labels for model-annotated samples (ablation) | Refined retraining process · Two groups with systematic bias and no innate difference | demographic parity unfairness | No clear effect |
| Use original retraining for both groups rather than refined retraining (ablation) | Two groups with systematic bias and no innate difference · Demographic parity | unfairness | Helped |
| Apply refined retraining to group i and original retraining to group j (ablation) | Two groups with systematic bias and no innate difference · Demographic parity | unfairness | Mixed |
| Impose a fairness constraint at every retraining round (ablation) | Refined retraining process · Two groups with potentially different feature distributions and costs | disadvantaged-group status | Hurt |
| Add fairness intervention when the disadvantaged group's noisy response satisfies the stated bound (ablation) | Refined retraining process · Two groups under noisy best response | disadvantaged-group status | No clear effect |
| Allow agents to respond strategically rather than remain non-strategic (ablation) | Original retraining process · Synthetic and semi-synthetic datasets | dynamics of acceptance rate, qualification rate, and classifier bias | Mixed |
| Add noise to agents' estimates of decision outcomes (ablation) | Original retraining process · Gaussian and German Credit datasets | acceptance rate, qualification rate, and classifier bias | No clear effect |
| Use only the most recent round's samples instead of accumulated training data (ablation) | Original retraining process · Gaussian and Uniform-linear datasets | acceptance rate and qualification rate | No clear effect |
Surprising findings
- Enforcing group fairness constraints at every retraining round may fail to benefit the disadvantaged group in the long run, since it can prevent the group from becoming advantaged.
- Systematic bias that underestimates a group's qualification can reduce classifier bias, so the bias may be beneficial for decision-maker welfare.
Limitations
- Theoretical results rely on assumptions (perfect hypothesis class learnability, monotone likelihood ratio) that may fail in practice.
- Experiments on real data violate several assumptions, and dynamics are less smooth.
- Only linear (logistic) classifiers and quadratic-cost best responses analyzed.
Experimental setup
Datasets: Uniform synthetic Gaussian synthetic German Credit Credit Approval
Benchmarks: Uniform synthetic Gaussian synthetic German Credit Credit Approval
Metrics: acceptance rate qualification rate classifier bias demographic parity unfairness
Baselines: original retraining process early stopping fairness intervention
Builds on: strategic classification performative prediction noisy best response
Closest papers by meaning
- Look-Ahead Reasoning on Learning Platforms (NeurIPS 2025 · similarity 0.85)
- Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHF (ICLR 2024 · similarity 0.84)
- NestRL: A Nested Training Regime for Mutual Adaptation in Human–AI Teaming (NeurIPS 2026 · similarity 0.84)
- Position: Challenges and Future Directions of Data-Centric AI Alignment (ICML 2025 · similarity 0.84)
- SKILL-MIX: a Flexible and Expandable Family of Evaluations for AI Models (ICLR 2024 · similarity 0.84)
- Learning Human-like Representations to Enable Learning Human Values (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.