AI Bias Audit Memo: Turn a Job Description + Screening Criteria Into a Structured Liability Risk Assessment

The task

HR leads and People Ops managers who own the ATS or a resume-screening tool need a defensible paper trail before rolling anything AI-driven into requisitions. This workflow turns a job description plus the screening rubric your vendor (or your team) is using into a structured bias-and-liability memo you can hand to Legal, your CHRO, or an external auditor. Do it before every new role family goes live, and re-run it when a vendor pushes a model update.

Before AI

Today this looks like: pulling the JD, exporting the screening criteria from the ATS, cross-referencing EEOC guidance and any state-specific rules (NYC 144, Illinois AIVI, Colorado AI Act), then drafting a memo. A careful pass runs 3-4 hours per role family, and most teams skip it or copy last quarter's memo with the title swapped. New research makes that shortcut riskier: a recent study found that AI is more likely than humans to form biases when hiring, which changes the risk math for any team layering LLM-based screening on top of a JD.

The workflow

Step 1 — Extract the screening signals actually being used.

Paste the JD and the screening rubric. Get back a clean list of what the model or rule set is really filtering on, separated from the polish.

You are an HR compliance analyst. I will paste a job description and a screening rubric below. Do three things and label them clearly:

(A) EXTRACTED SIGNALS — every attribute, keyword, credential, years-of-experience threshold, school/employer name, or behavioral trait the rubric filters on. Bullet list. Be exhaustive; include soft signals like "culture fit" or "communication style."

(B) INFERRED PROXIES — for each extracted signal, note any known proxy relationship with a protected class under US federal employment law (race, color, religion, sex, national origin, age 40+, disability, genetic info, pregnancy). If no known proxy relationship, write "none identified." Do not speculate wildly; cite the mechanism briefly (e.g., "ZIP-code adjacent → race proxy per redlining literature").

(C) JD/RUBRIC MISMATCH — signals present in the rubric but not justified by the JD's actual duties. These are the highest-risk items.

Output as three labeled sections. No preamble.

JOB DESCRIPTION — Senior Customer Success Manager, Northstar Retail Analytics (fictional company)

We're hiring a Senior CSM to own our top 20 enterprise accounts. You'll run quarterly business reviews, drive expansion, and partner with Product on the roadmap.

Responsibilities:
- Own renewal and expansion for a book of ~20 accounts ($8M ARR)
- Lead QBRs with VP/C-level stakeholders
- Partner with Solutions Engineering on technical escalations
- Maintain Gainsight health scores and forecast accuracy

Requirements:
- 5+ years in enterprise SaaS customer success
- Bachelor's degree required, MBA preferred
- Experience at a top-tier SaaS company (Salesforce, HubSpot, Gainsight, or similar)
- Strong executive presence and polished communication
- Willing to travel 30% (client HQs primarily east coast US)

SCREENING RUBRIC (used by our AI resume screener, "TalentSift v3.2"):
- Auto-reject: <5 years CSM experience
- Auto-reject: no bachelor's degree listed
- +10 points: MBA from top-25 program
- +8 points: prior employer in "elite SaaS" list (25 companies)
- +5 points: LinkedIn photo present and "professional"
- +5 points: no employment gaps >6 months in last 10 years
- +3 points: keywords "executive presence," "polished," "articulate"
- -5 points: resume submitted from ZIP code outside top-50 US metros
- Sort: composite score descending, top 40 forwarded to recruiter

Step 2 — Score the disparate-impact risk and map to legal exposure.

Now turn the extracted signals into a risk-rated table tied to specific compliance frameworks.

Using the three sections you just produced, build a RISK REGISTER as a markdown table with these columns:

| Signal | Risk Level (High/Med/Low) | Protected Class(es) at Risk | Framework Triggered | Recommended Action |

Frameworks to reference where applicable: Title VII disparate impact, ADEA (age), ADA, EEOC 2023 guidance on algorithmic decision-making tools, the EEOC's four-fifths rule for adverse impact, NYC Local Law 144 (bias audit + candidate notice), Illinois AI Video Interview Act, Colorado AI Act (high-risk system obligations). If a signal doesn't map to any framework, say "no framework triggered — internal policy only."

Recommended Action must be concrete: "remove signal," "replace with duty-based proxy," "require four-fifths audit before use," "add candidate disclosure per NYC 144," etc. Do not recommend "consult legal" as a standalone action.

Below the table, list the top 3 signals by risk in a "Priority Fixes" section with a one-sentence justification each.

Step 3 — Draft the memo.

Now draft a two-page bias audit memo suitable for the CHRO and outside employment counsel. Use this structure exactly:

1. Header — TO / FROM / DATE / RE (role and screening tool name pulled from the input)
2. Executive Summary — 4 sentences max. State the overall risk posture (Low/Medium/High), the count of High-risk signals, and whether the tool is safe to deploy as-is, deploy with fixes, or pause.
3. Scope — what was reviewed (the JD, the rubric, the tool version).
4. Findings — the risk register table from Step 2, verbatim.
5. Priority Fixes — the top 3 from Step 2, expanded to 2-3 sentences each with the specific rubric edit proposed.
6. Open Questions for Vendor — 5 bullet questions the HR team should send to the screening tool vendor before go-live (e.g., adverse impact ratio by protected class, training data provenance, model update cadence, ability to disable specific signals, audit log access).
7. Recommended Next Steps — numbered, with owners (HR Ops, Legal, Vendor, Hiring Manager) and a suggested deadline in business days.

Tone: neutral, factual, no hedging language like "might potentially." If evidence for a claim is thin, say "requires vendor confirmation."

Gotchas

• The model will over-index on obvious signals (ZIP code, photo) and miss subtle ones like "elite employer" lists that function as a school/network proxy. If Step 1's INFERRED PROXIES section is short, push back and re-run with "be more aggressive on second-order proxies."
• It will hallucinate specific statutory citations. Treat every framework reference as a pointer for Legal to verify, not a legal conclusion. The EEOC's four-fifths rule is real and stable; state AI laws change quickly.
• The memo is not a bias audit. A real audit under NYC Local Law 144 requires an independent auditor and actual selection-rate math on real candidate data. This workflow gets you to the point where you know whether you need one and what to test.
• Vendor black boxes break Step 1. If your vendor won't disclose the rubric, the memo's finding writes itself: pause deployment until they will. The MIT Technology Review coverage of the recent bias study is a useful attachment when you send that email.
• Don't paste real candidate data or real employee names into the prompt. JD and rubric only. If your rubric contains named comparator employees, redact first.

Time saved

A careful manual pass — reading the JD, extracting the rubric, cross-referencing four to six compliance frameworks, and drafting the memo — runs 3-4 hours for someone who does this regularly, longer for someone who doesn't. This workflow gets you to a reviewable draft in roughly 20-30 minutes, leaving the saved time for the part that actually needs a human: pushing back on the hiring manager about the "elite SaaS" list. Estimate: 2-3 hours saved per screening tool reviewed, assuming you review 4-8 role families per quarter.

What we got when we ran it

We executed every prompt above, in order, against the sample input (model: claude-sonnet-4-6). Outputs below are real and unedited apart from length truncation.

Step 1 output

(A) EXTRACTED SIGNALS - Minimum 5 years of enterprise SaaS customer success experience (hard cutoff) - Bachelor's degree (hard auto-reject if absent) - MBA preferred; +10 points if MBA from top-25 ranked program - Prior employer on internal "elite SaaS" list of 25 companies (+8 points) - Experience specifically at named employers: Salesforce, HubSpot, Gainsight, or "similar" - "Executive presence" as a keyword on resume (+3 points) - "Polished" as a keyword on resume (+3 points) - "Articulate" as a keyword on resume (+3 points) - LinkedIn photo present and assessed as "professional" (+5 points) - No employment gaps longer than 6 months within the last 10 years (+5 points) - Resume submitted from a ZIP code within the top-50 US metros (−5 points if outside) - Willingness to travel 30% (stated in JD, implicit screen) - East coast client travel orientation (geographic preference implied) - Gainsight platform proficiency (mentioned in JD duties) - Ability to lead QBRs with VP/C-level stakeholders - Forecast accuracy / Gainsight health score management - "Culture fit" (implicit in "polished communication" and photo assessment) - Communication style judgment (subjective, applied at resume stage before any interview) --- ## (B) INFERRED PROXIES | Extracted Signal | Proxy Risk & Mechanism | |---|---| | Bachelor's degree auto-reject | **Race, national origin, socioeconomic status proxy.** Black, Hispanic/Latino, and first-generation immigrant candidates hold bachelor's degrees at lower rates due to documented structural inequities in higher education access. EEOC guidance and *Griggs v. Duke Power Co.* (1971) require degree

…output truncated at 250 words.

Step 2 output

Risk Register — TalentSift v3.2 Screening Rubric | Signal | Risk Level | Protected Class(es) at Risk | Framework Triggered | Recommended Action | |---|---|---|---|---| | LinkedIn photo assessed as "professional" (+5 pts) | **HIGH** | Race, color, sex, religion, national origin, disability, age 40+ | Title VII disparate impact; ADA (visible disability); ADEA; EEOC 2023 algorithmic guidance (automated appearance assessment explicitly flagged); NYC Local Law 144 (bias audit required for automated employment decisions) | **Remove signal entirely.** No duty-based justification exists. If photo hosting platform (LinkedIn) is used at all in screening, configure tool to suppress photo rendering before scoring. Document removal in audit log. | | ZIP code penalty (−5 pts for non-top-50 metro) | **HIGH** | Race, color, national origin, disability | Title VII disparate impact (ZIP-to-race proxy via redlining literature; *Robinson v. Nationstar*, 2019 FHA precedent applicable by analogy); EEOC 2023 algorithmic guidance; NYC Local Law 144; Colorado AI Act (high-risk system — geographic filters must be documented and audited) | **Remove signal.** Replace with a single duty-based screen: confirmed willingness to travel 30%, applied uniformly. Run a retrospective four-fifths audit on any prior cohorts screened with this filter before retiring it. | | "Articulate" as scoreable keyword (+3 pts) | **HIGH** | Race (primarily Black professionals) | Title VII disparate impact; EEOC 2023 algorithmic guidance (keyword screens that encode racialized language explicitly mentioned); NYC Local Law 144 | **Remove signal.** Do not substitute synonyms with equivalent proxy risk ("well-spoken," "eloquent" carry the same documented

…output truncated at 250 words.

Step 3 output

--- MEMORANDUM — PRIVILEGED AND CONFIDENTIAL Attorney-Client Communication / Attorney Work Product --- TO: Chief Human Resources Officer; Outside Employment Counsel FROM: HR Compliance Analysis Function DATE: 2025-07-14 RE: Bias Audit — Senior Customer Success Manager Requisition; TalentSift v3.2 Automated Screening Tool --- ## 1. Executive Summary This audit assessed the automated resume screening rubric deployed via TalentSift v3.2 for the Senior Customer Success Manager requisition at Northstar Retail Analytics and identified an overall risk posture of HIGH. The rubric contains four signals rated High-risk and three additional signals rated Medium-High, each carrying independent disparate impact exposure under Title VII, the ADEA, and the ADA, as well as compliance obligations under the EEOC's 2023 algorithmic decision-making guidance and NYC Local Law 144. The tool is not safe to deploy as-is; the recommended disposition is pause deployment, implement Priority Fixes, and complete a bias audit of the composite scoring model before resuming use on any live requisition. --- ## 2. Scope The following materials were reviewed in their entirety: - Job Description: Senior Customer Success Manager, Northstar Retail Analytics (synthetic sample, version reviewed 2025-07-14) - Screening Rubric: TalentSift v3.2 automated resume scoring logic, including all point-addition rules, point-deduction rules, auto-reject thresholds, and candidate forwarding parameters (top-40 composite score sort) - Frameworks Applied: Title VII of the Civil Rights Act of 1964 (disparate impact doctrine, Griggs v. Duke Power Co., 401 U.S. 424 (1971)); Age Discrimination in Employment Act (ADEA); Americans with Disabilities Act (ADA); Pregnancy Discrimination Act; EEOC Guidance on the

…output truncated at 250 words.