Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations | Lee Cohen et al. | ResearchPod

Transcript: Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations

Alex: Automated resume screeners might be rewarding the best chatbot, not the best candidate. A new proposal says the fix is for employers to run every resume through a chatbot themselves.

Sam: That sounds backwards. Before we get to why, whose work is this and what kind of paper is it?

Alex: It's from Lee Cohen, Jack Hsieh, Connie Hong and Judy Hanwen Shen at Stanford, posted to arXiv in February 2025. It's a method paper with proofs, plus an experiment on real resumes.

Sam: Plenty of people use AI to edit resumes now. Why should a researcher outside hiring care?

Alex: Because the paper cites a survey where 57 percent of respondents admitted using AI to write their resume. If paid models polish better than free ones, the screener quietly filters for who could afford the better tool. The authors propose a fix that helps fairness and accuracy at once, without knowing who used what. So we'll cover the problem, the one idea behind the fix, and how far the evidence goes.

Sam: Start with the problem. What did they actually observe?

Alex: They took tech-sector resumes and asked a range of models to improve them with a generic prompt. Then they scored them against job postings with an open-source tool that mimics an applicant tracking system, the software that automatically scores and filters resumes. They saw three things. First, edits usually raise scores, but not always; some resumes score lower afterward. Second, premium models raise scores more.

Sam: How much more? Is this a small edge or something that changes who gets through?

Alex: For a product manager posting at DoorDash, GPT-4o, Claude 3.5 Sonnet and DeepSeek-V3 pushed unqualified resumes, from a different occupation, to look indistinguishable from unedited qualified ones. Cheaper models like GPT-3.5 Turbo didn't move scores much on average. The third observation matters most later: running the same model a second time barely changes anything. The first pass standardizes language and structure, and the second pass stays close to it.

Sam: So this is a gaming problem, which my field would call strategic classification. What's different from the classic version?

Alex: Right, strategic classification studies decision rules when people adjust their inputs to get a better outcome. Classically, gaming is costly and deterministic, so people only do it when it guarantees success. Here prompting is nearly free, so almost everyone does it. The outcome is random, because model outputs vary. And the employer can't tell whether a resume was edited, or which model did it. Earlier fairness work in hiring usually assumed you knew group membership. This paper assumes you don't.

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations

Lee Cohen, Jack Hsieh, Connie Hong, Judy Hanwen Shen

Feb 18, 2025·8:05·arxiv·Original Paper

Strategic ClassificationTwo-ticket SchemeGenerative AI ManipulationGroup-independent DecisionAlgorithmic HiringLarge Language Model (LLM) Manipulation

Listen

8 min

Summary

Research Question and Approach

As job seekers increasingly use generative AI to optimize their resumes, candidates with access to premium LLMs gain an unfair advantage, leading to hiring disparities and reduced decision accuracy for employers. This paper models this challenge as a strategic classification game where hiring algorithms must screen candidates who manipulate their resumes stochastically. The authors propose a two-ticket scheme where the hiring algorithm automatically applies an additional LLM manipulation to every incoming resume and evaluates both the submitted and the newly manipulated version to make a final decision.

Main Findings

The researchers demonstrate that traditional hiring processes are inherently biased when candidates have unequal access to LLM tools. By introducing the two-ticket scheme—which treats the original resume and the hirer-manipulated version as two separate 'tickets' to entry—the authors prove that hiring disparities decrease. Furthermore, they extend this to an n-ticket scheme, proving that as the number of manipulations increases, hiring outcomes converge to a group-independent decision, effectively neutralizing the advantage provided by superior individual LLM access. Empirical validation using an open-source resume screening tool confirms that the two-ticket scheme significantly improves true positive rates and reduces fairness disparities between candidate groups.

Why It Matters

This work provides a foundational theoretical framework for understanding how generative AI disrupts traditional applicant screening. It offers a practical, low-cost intervention for employers who want to maintain fairness without needing to detect whether an applicant has used AI or which specific model they accessed. The approach allows organizations to leverage AI to level the playing field, ensuring that hiring decisions are based on candidate qualifications rather than their ability to access expensive prompt engineering tools.

Key Terms and Definitions

Strategic Classification — A game-theoretic framework where applicants strategically modify their features (e.g., resumes) to influence a classifier's decision to their advantage.
Two-Ticket Scheme — A hiring intervention where the system considers both the original candidate submission and a version of that submission manipulated by an employer-side LLM.
No False Positives Objective — A decision criterion for hiring algorithms that prioritizes maximizing the true positive rate while strictly prohibiting the selection of any unqualified candidates.
Stochastic Manipulation — The process by which an LLM modifies a resume in a non-deterministic way, meaning the same input might result in different output scores across multiple attempts.
Fundamental Features — The essential, factual attributes of a candidate’s resume (e.g., years of experience, specific technical skills) that the hiring model aims to evaluate.

Abstract

In an era of increasingly capable foundation models, job seekers are turning to generative AI tools to enhance their application materials. However, unequal access to and knowledge about generative AI tools can harm both employers and candidates by reducing the accuracy of hiring decisions and giving some candidates an unfair advantage. To address these challenges, we introduce a new variant of the strategic classification framework tailored to manipulations performed using large language models, accommodating varying levels of manipulations and stochastic outcomes. We propose a ``two-ticket'' scheme, where the hiring algorithm applies an additional manipulation to each submitted resume and considers this manipulated version together with the original submitted resume. We establish theoretical guarantees for this scheme, showing improvements for both the fairness and accuracy of hiring decisions when the true positive rate is maximized subject to a no false positives constraint. We further generalize this approach to an $n$-ticket scheme and prove that hiring outcomes converge to a fixed, group-independent decision, eliminating disparities arising from differential LLM access. Finally, we empirically validate our framework and the performance of our two-ticket scheme on real resumes using an open-source resume screening tool.

Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations: Research Paper Podcast