Research map / Large language models / AI and society
Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a "two-ticket" hiring system in which the hirer applies its own LLM manipulation to each submitted resume and scores the better version, with theory on fairness and accuracy under strategic LLM edits plus a small resume-screening case study.
Problem
Unequal access to generative AI tools lets some job applicants strategically improve resumes, which harms hiring accuracy and fairness under score-based screening.
Approach
Models LLM resume edits as stochastic strategic manipulations in a Stackelberg classification game, and has the hirer apply its own LLM to each submission and score the better of original and hirer-edited versions, which provably reduces outcome disparity and raises TPR under no-false-positive thresholds.
Key results
- Two-Ticket SVM raises TPR from 0.024 to 0.108 relative to traditional SVM in the case study (BS difference 0.084, 95% range [0.05, 0.163]).
- Two-Ticket improves accuracy for PM role (e.g., 0.548 to 0.563 with no LLM for the unqualified-candidate condition) and UX role (0.633 to 0.689).
- TPR disparity reduction is not statistically significant: the 95% bootstrap interval of disparity differences [-0.0619, 0.139] includes zero.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply Two-Ticket hiring instead of Traditional SVM | Resume Matcher screening with applicant manipulations · Djinni resume screening test set | TPR: 0.024 ± 0.09 -> 0.108 ± 0.08; BS difference: 0.084 ± 0.04 | Helped |
| Apply Two-Ticket hiring instead of Traditional SVM | Resume Matcher screening with applicant manipulations · Djinni resume screening test set | TPR Disparity: 0.00409 ± 0.12 -> 0.04269 ± 0.06; BS difference: 0.0386 ± 0.051; 95% BS Diffs Range: [-0.0619, 0.139] | No clear effect |
| Apply Two-Ticket hiring instead of Traditional | Resume Matcher classifier · PM Role; U: No LLMs, P: GPT-4o | Accuracy: $0.548 \pm 0.003$ -> $0.563 \pm 0.004$ | Helped |
| Apply Two-Ticket hiring instead of Traditional | Resume Matcher classifier · UX Designer Role; U: No LLMs, P: GPT-4o | Accuracy: $0.633 \pm 0.005$ -> $ 0.689 \pm 0.005$ | Helped |
| Apply Two-Ticket hiring instead of Traditional | Resume Matcher classifier · PM Role; U: GPT-3.5, P: GPT-4o | Accuracy: $0.543 \pm 0.003$ -> $0.551 \pm 0.003$ | Helped |
| Apply Two-Ticket hiring instead of Traditional | Resume Matcher classifier · UX Designer Role; U: GPT-3.5, P: GPT-4o | Accuracy: $ 0.629 \pm 0.006$ -> $ 0.650 \pm 0.006$ | Helped |
| Apply Two-Ticket hiring instead of Traditional | Resume Matcher classifier · PM Role; U: GPT-4o-mini, P: GPT-4o | Accuracy: $0.554 \pm 0.003$ -> $0.561 \pm 0.003$ | Helped |
| Apply Two-Ticket hiring instead of Traditional | Resume Matcher classifier · UX Designer Role; U: GPT-4o-mini, P: GPT-4o | Accuracy: $0.662 \pm 0.005$ -> $0.677 \pm 0.005$ | Helped |
| Use ChatGPT-4o rather than Mixtral-8x7B for resume manipulation | Resume score screening · Djinni resumes | fraction of resumes with increased scores: 75% -> 50% | Helped |
| Apply a second round of the same LLM manipulation | Resume scoring · Djinni resumes | resume score | No clear effect |
Surprising findings
- Sequentially applying the same LLM yields diminishing changes, supporting the idea that a hirer-side LLM pass changes resumes less than the first applicant pass.
- Prompts mentioning a specific job title caused LLMs to insert the title into unqualified resumes, inflating scores and degrading the classifier.
Limitations
- Fairness improvement in the case study is not statistically significant, and the threshold-consistency condition is assumed rather than generally guaranteed.
- Small single-scorer case study (Resume-Matcher, 520 resumes, one job type pair) with approximated no-false-positive thresholds.
- Theory assumes mild manipulations that preserve qualified/unqualified separation, and does not separate LLM-alterable from unalterable features in practice.
Experimental setup
Datasets: Djinni Recruiting dataset (520 sampled resumes)
Benchmarks: Djinni resume sample (PM and UI/UX roles)
Metrics: true positive rate TPR disparity between groups accuracy false positive rate
Baselines: Traditional SVM hiring (one-ticket)
Builds on: strategic classification multivariate stochastic dominance
Closest papers by meaning
- Monoculture in Matching Markets (NeurIPS 2024 · similarity 0.82)
- Putting Gale & Shapley to Work: Guaranteeing Stability Through Learning (NeurIPS 2024 · similarity 0.82)
- On Mitigating Affinity Bias through Bandits with Evolving Biased Feedback (ICML 2025 · similarity 0.82)
- Dissecting Submission Limit in Desk-Rejections: A Mathematical Analysis of Fairness in AI Conference Policies (ICML 2025 · similarity 0.82)
- FairJob: A Real-World Dataset for Fairness in Online Systems (NeurIPS 2024 · similarity 0.82)
- Matchings Under Biased and Correlated Evaluations (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.