Research map / Multi-agent systems and games / Games and equilibria
Contracting with a Learning Agent
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Studies optimal dynamic contracts against no-regret (mean-based) learning agents; shows a free-fall contract (linear contract then zero contract) is optimal for linear/success-failure settings and extends to single-dimensional scaling, with a time-horizon uncertainty analysis.
Problem
Repeated principal-agent contracting where the agent responds with no-regret learning rather than best-responding; the principal wants to optimize dynamic contracts against such a learner, which is open in general games.
Approach
Shows that against a mean-based learner, the optimal dynamic contract is a free-fall: offer a fixed linear contract for a fraction of time, then switch to the zero contract so the agent descends through its action space while the principal collects reward at zero cost. Extends to p-scaled non-linear contracts and characterizes the effect of uncertainty about the time horizon.
Key results
- For linear (success/failure) contract settings, an optimal dynamic contract against a mean-based agent is a free-fall contract (linear contract then zero contract), computable in polynomial time.
- Example: principal utility rises from 1/3 T (best static) to 5/12 T (dynamic), a factor of 5/4, in a three-action instance.
- Dynamic contracts can be win-win: welfare improvement up to Theta(n) and agent utility improvement Theta(log n) on a positive-measure set of instances.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a free-fall contract instead of the optimal static contract | Repeated contract against a mean-based learner · Illustrative 3-action contract | principal utility: 1/3 T -> ≈5/12 T; factor of 5/4 | Helped |
| Use a free-fall contract instead of the optimal static contract | Repeated contract against a mean-based learner · Welfare improvement example | welfare: factor of 5/4 | Helped |
| Use an optimal dynamic contract instead of the optimal static contract | Repeated contract settings in the constructed positive-measure family · Contract games with n actions | agent utility: Θ(log(n)) | Helped |
| Use an optimal dynamic contract instead of the optimal static contract | Repeated contract settings in the constructed positive-measure family · Contract games with n actions | principal utility: Θ(n) | Helped |
| Use a dynamic contract against a no-swap-regret learner rather than a static contract | Repeated principal-agent problem · General contract setting | principal utility | No clear effect |
| Use free-fall contracts rather than unrestricted dynamic linear contracts | Repeated linear contract against a mean-based learner · Linear contract setting | principal utility | No clear effect |
| Restrict dynamic contracts to single-dimensional scaling | Repeated general contract against a mean-based learner · Arbitrary contract setting | principal utility | No clear effect |
| Give the principal uncertainty about the time horizon | Repeated contract against a mean-based learner · Contract setting with uncertain T | principal utility advantage over the optimal static contract | Hurt |
| Allow dynamic contracts without single-dimensional scaling | Repeated general contract · Arbitrary contract setting | principal utility | Mixed |
Surprising findings
- Against a no-swap-regret agent the best static contract is optimal, so a less sophisticated mean-based learner can leave both players better off.
- Even mild uncertainty about the time horizon substantially limits the principal's ability to beat the best static contract.
Limitations
- Main optimality result is for linear contracts / success-failure outcomes; general contracts only with single-dimensional scaling.
- Assumes the principal knows the time horizon T; uncertainty about T degrades the advantage.
Experimental setup
Metrics: principal utility agent utility welfare
Baselines: optimal static contract no-swap-regret agent
Builds on: Deng et al. 2019 strategizing against no-regret learners linear contracts
Closest papers by meaning
- Learning Optimal Contracts: How to Exploit Small Action Spaces (ICLR 2024 · similarity 0.92)
- Contract Design Under Approximate Best Responses (ICML 2025 · similarity 0.90)
- Generalized Principal-Agent Problem with a Learning Agent (ICLR 2025 · similarity 0.88)
- Learning in Online Principal-Agent Interactions: The Power of Menus (AAAI 2024 · similarity 0.88)
- Online Information Acquisition: Hiring Multiple Agents (ICLR 2024 · similarity 0.86)
- Incentivized Learning in Principal-Agent Bandit Games (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.