The one lesson to read twice
Everything else in this course is about saving time. This lesson is about not causing harm you cannot see and cannot undo.
Hiring is one of the most heavily regulated things an organisation does. Bringing AI into it does not create a new legal universe, it drops your process into the existing one at speed and at scale. A biased human reviewer affects the applications on their desk. A biased screening rule affects everyone who applies, quietly, consistently, and without anyone finding out why they were rejected.
This lesson is not legal advice, and it deliberately does not tell you what the law says where you are. Requirements differ by country, state, city and sector, and they are changing quickly. Before you automate any part of screening, assessment or selection, take advice from your own legal or compliance function.
In plain English
- Training data:
- The past examples a system learned from. In hiring, often the CVs and outcomes of people you hired before.
- Proxy variable:
- Something innocent that stands in for a protected characteristic, like a postcode standing in for ethnicity.
- Bias audit:
- A structured check of whether a selection tool passes or rejects some groups at noticeably different rates.
- Protected characteristic:
- A category the law protects from discrimination, such as sex, race, age, disability or religion. The exact list varies by place.
- Right to human review:
- A candidate's ability to ask that a person, not software, looks at a decision affecting them.
Why yesterday's data is the problem
A system trained on your past hiring learns what your past hiring looked like. If your engineering team has been overwhelmingly one gender for a decade, a tool trained on that history learns that those CVs are the successful ones. Nobody wrote a rule saying so. It emerged, and it will now be applied to every applicant, evenly and invisibly.
The word people reach for is bias, which makes it sound like an attitude. It is not an attitude. It is a statistical echo of your own history, delivered back to you as a recommendation, with confidence.
Removing names and photographs does not fix this. Models pick up proxies. University, postcode, career gaps, phrasing that correlates with a first language, the year someone left school. You can strip the obvious identifiers and still reproduce the same pattern through a dozen small signals nobody thought about.
The kinds of obligation that commonly exist
The specifics vary. The categories recur almost everywhere, and knowing the categories tells you what to ask your legal team.
Restrictions on automated decisions. Many places limit decisions made about people with no meaningful human involvement, particularly where the effect is significant. Being rejected for a job is significant.
Disclosure to candidates. It is increasingly common to be required to tell applicants that automated tools are being used in the process, sometimes before they apply.
Bias testing. Some jurisdictions require selection tools to be tested for differential impact across groups, sometimes by an independent party, sometimes published.
A route to human review. Candidates may have the right to ask that a person reconsider, and to be given some explanation of the basis for the decision.
Record keeping. You may need to show, later, why a specific person was rejected. If the honest answer is that a tool scored them low and nobody knows why, you are in a difficult position.
The pattern behind all of these is the same: they exist because rejection at scale is invisible to the rejected. A candidate who never hears back cannot tell whether they were unlucky, unqualified, or filtered by a rule that would not survive daylight. The obligations restore the possibility of asking.
โ Weak prompt
Prompt
Screen these applications and reject anyone who is not a strong fit. Give me the shortlist only.
Output
A shortlist of eight names, with the rest discarded and no record of why.
You have made 92 decisions about real people and cannot explain a single one. There is no audit trail, no human involvement, and nothing to hand a candidate who asks.
โ Good prompt
Prompt
Read this application against the six essential criteria below. For each criterion, quote the evidence in the application that speaks to it, or write No evidence found. Do not give an overall score, a recommendation or a rating. I am making the decision.
Output
Six criteria, each with quoted evidence or an explicit note that nothing in the application addressed it.
This is extraction, not judgement. It speeds up your reading, the quoted evidence is checkable, and the decision and the explanation both remain yours.
Read the application below against these essential criteria:
- [criterion]
- [criterion]
- [criterion]
For each criterion, quote the exact words in the application that address it.
If nothing addresses it, write: No evidence found.
Rules:
- Do not rate, score, rank or recommend
- Do not infer anything not stated in the text
- Do not comment on writing style, personality or fit
- Quote only, do not paraphrase
Application:
[paste it]
Checkpoint
AI trained on past hiring reproduces past hiring. Before automating any screening step, ask your legal or compliance function what applies to you, and keep records that let you explain any individual rejection.
Questions to take to your legal team
Do not walk in asking whether you can use AI. Walk in with the specifics, because that is the only way to get a usable answer.
I am considering using AI in hiring in this specific way:
[describe exactly what the tool would do and at which stage]
Write me a list of questions to put to our legal and compliance team.
Cover: whether this counts as an automated decision, what we may need
to tell candidates, whether testing for differential impact is expected,
what records we should keep, and what a candidate could ask us for later.
Do not answer the questions. Do not tell me what the law says.
Produce the questions only, in plain language, ready to send.
If you take one operational habit from this lesson, make it this: keep the human decision and the human reasons. Whatever the tool produced, the record should show a named person, what they considered, and why. That record is what you will want in twelve months, and it is also, quietly, what makes the process fair rather than merely fast.
Here is our current shortlisting process, step by step:
[paste it]
Identify anything in it that could act as a proxy for age, sex, race,
disability, nationality, socioeconomic background or caring responsibilities,
even indirectly. Explain the mechanism in each case.
Then list which steps produce a decision with no meaningful human judgement.
Do not tell me whether any of this is lawful. I will ask our legal team.
๐ Quiz
Question 1 of 4Why can a tool trained on past hiring data end up discriminating?