AI resume screeners, audited: bias, accuracy, and the EU rules

AI resume screeners are now the default first reader of your application — and one of the most studied sources of algorithmic bias in production AI. Over the past two years, independent audits have run identical candidate pools through screening models, changing only the name at the top of the resume, and measured who survives. This review pulls those audits together, adds what a 33,000-job field audit found about the filters that run before the AI, and lays out what the EU AI Act now requires of anyone using these tools.

What the identical-resume audits found#

The anchor study is Kyra Wilson and Aylin Caliskan's "Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval," presented at AIES 2024 with NIST funding. The method is exactly the kind of audit every vendor should be running: 554 real resumes, each paired with 80 first names — 20 per demographic group (Black women, Black men, white women, white men), with the last name "Williams" held constant — ranked against 571 job descriptions across nine occupations, using three open-weight embedding models of the family that powers production screening tools (E5-Mistral-7B-Instruct, GritLM-7B, SFR-Embedding-Mistral). Roughly 40,000 paired comparisons per model.

The headline numbers, replicated across all three models:

• Race: white-associated names ranked higher in 85.1% of paired comparisons. Black-associated names in only 8.6% (ties in 6.3%).
• Gender: male-associated names preferred in 51.9% of comparisons, female-associated names in 11.1%, equal outcomes in 37%.
• Intersectionality: in head-to-head Black-male-names versus white-male-names comparisons, Black male names were preferred in 0% of tests across all three models. Functional exclusion.

To put that in regulatory terms: the EEOC's Uniform Guidelines on Employee Selection Procedures (29 CFR Part 1607) define adverse impact as any group's selection rate falling below four-fifths (80%) of the highest group's rate. Black names' 8.6% rate against white names' 85.1% gives a ratio of roughly 10% — about eight times worse than the 80% threshold.

A 2025 follow-up paper, "Fairness Is Not Enough," replicated the 85.1% white-preference figure across more than 3 million comparisons and identified an "Illusion of Neutrality" effect: some models that look racially flat do so because they match on superficial keywords rather than evaluating substance — apparent fairness masking the absence of screening quality. And a 2025 paper by Xu, Li, and Jiang documented an AI self-preferencing bias: evaluator LLMs were 26–98% more likely (in odds-ratio terms) to select a resume they themselves had generated than an equivalent human-written one — even when human judges rated the human-written versions clearer and more coherent.

The bias humans can't unsee#

Audits of the model are only half the story. A November 2025 University of Washington study asked 528 people to pick candidates for 16 different jobs while working with simulated LLM recommendations carrying varying levels of racial bias. Presented at the AAAI/ACM Conference on AI, Ethics, and Society in Madrid, the finding was stark: with neutral AI, participants selected white and non-white applicants at equal rates. With a moderately biased AI, participants mirrored the AI's preferences. In severe-bias scenarios, human reviewers made only slightly less biased decisions than the AI itself — following the AI's preferred candidates roughly 90% of the time.

This undercuts the standard industry defense. About 80% of organizations using AI hiring tools say they don't reject applicants without human review. But the research suggests "human in the loop" is a weak corrective: reviewers tend to accept the AI's judgment unless the bias is obvious. Meanwhile, roughly 21% of companies automatically reject candidates at some stage with no human review at all.

The filters before the AI: a 33,000-job field audit#

One caveat to the lab studies: they measure what happens once a resume reaches the model. Pin's 2026 audit of 33,000+ jobs and 37,000+ recruiter sourcing searches (January 2024–May 2026) measures what happens one step earlier — which resumes ever get seen:

• 70.7% of sourcing searches apply an employer-prestige filter: only candidates who worked at a named set of companies are allowed through.
• 45.7% set a years-of-experience floor; of those, 51.9% require 5+ years and 12.4% require 10+.
• 96% of all minimum-tenure filters are set at exactly 12 months at the current employer — the platform default, not recruiter judgment.
• SHRM's 2026 State of AI in HR survey (1,908 HR professionals) found 19% of organizations using hiring automation report their tools have screened out qualified applicants.

The point for buyers: much of the public debate treats the AI model as the source of unfairness, but the pool it scores has already been narrowed by human-designed filter defaults. A proper audit covers both halves of the funnel.

How to run your own audit#

You don't need a research lab. The paired-comparison protocol from the published studies is reproducible:

1. Build the pool. Take 20–50 real resumes for a role, redact names, and create name-swapped copies (white-coded, Black-coded, male-coded, female-coded names, same surname).
2. Run the screener blind. Feed each version through your screening tool and record rankings and scores. Vary only the name.
3. Measure the gap. Compare selection rates across name groups. Apply the four-fifths rule: any group selected at less than 80% of the highest group's rate signals adverse impact.
4. Test the humans too. Have recruiters review a sample with and without the AI's recommendation, and check whether reviewers are simply rubber-stamping the tool.
5. Audit the defaults. Log which filters your team applies at sourcing stage — tenure floors, prestige filters, experience minimums — and ask whether the default or the recruiter made that decision.

Apart Research's 2025 interpretability study showed bias favoring male-associated names specifically in ambiguous cases — so run your tests on borderline candidates, not just clear accepts and rejects. That's where the model exercises the most discretion and where bias concentrates.

What the EU AI Act now demands#

Under the EU AI Act (Regulation (EU) 2024/1689), AI systems used for recruitment and screening are explicitly high-risk — Annex III, section 4 covers systems "for recruitment or selection," including analyzing and filtering job applications and evaluating candidates. That classification triggers a full compliance burden:

Fines for the most serious violations reach €35 million or 7% of global turnover. Employers deploying these tools must also inform workers' representatives and affected workers before use (Art. 26(7)), and a deployer-side fundamental rights impact assessment applies. Broader deployer obligations for high-risk systems took effect on 2 August 2026.

The enforcement landscape around it is tightening: NYC's Local Law 144 on automated employment decision tools, the EEOC's iTutorGroup settlement, and the Mobley v. Workday age-discrimination certification all sit in the path of teams screening resumes with AI. As one SaaS compliance audit summarized: an AI CV-screening product is high-risk, full stop — but an AI that merely drafts job descriptions without evaluating applicants is not.

The takeaway#

The audit evidence is consistent: screening AI favors white-coded names by wide margins, excludes Black male candidates in head-to-head tests, is rubber-stamped by human reviewers who don't spot moderate bias, and inherits pools already narrowed by blunt filter defaults. The accuracy problem runs deeper than bias — self-preferencing and the "illusion of neutrality" show some models match keywords instead of evaluating substance, meaning a "fair-looking" tool can still be a bad screening tool.

For buyers, the checklist is now short: demand vendor bias-test results using paired comparisons, run your own periodic audits including the human review step, and document everything the EU AI Act requires of high-risk systems. The era of treating resume screeners as neutral infrastructure is over — by the evidence, and now by law.