NYC Local Law 144 requires employers using automated employment decision tools (AEDTs) to conduct an annual independent bias audit and publish summary results before using the tool. The law applies to any employer or employment agency in New York City that uses AI or algorithmic tools to screen, evaluate, or rank job candidates or employees for hiring or promotion decisions. Non-compliance carries daily fines of up to $1,500 per violation.
Key Takeaways
- Any AI or algorithmic tool that substantially assists or replaces human decision-making in hiring or promotion qualifies as an AEDT under LL144.
- An independent bias audit analyzing impact ratios across sex, race/ethnicity, and intersectional categories must be completed within one year before the tool is used.
- Employers must publish the audit summary on their website and provide at least 10 business days' notice to candidates before using an AEDT.
- Penalties range from $500 for a first violation to $500–$1,500 for each subsequent violation, assessed per day the violation continues.
- The auditor must be independent—no financial interest in the employer or vendor that developed the tool.
What is NYC Local Law 144?
NYC Local Law 144, officially titled "A Local Law to amend the administrative code of the city of New York, in relation to automated employment decision tools," is a first-of-its-kind regulation targeting the use of artificial intelligence in employment decisions. The law was signed by Mayor Eric Adams in December 2021, with enforcement beginning on July 5, 2023.
The law does not ban AI in hiring. Instead, it creates a transparency and accountability framework: if you use an automated tool to make or substantially assist with employment decisions in New York City, you must audit it for bias, publish the results, and notify candidates.
LL144 was enacted by the New York City Council to address growing concerns that algorithmic hiring tools could perpetuate or amplify discrimination against protected groups. The law is enforced by the NYC Department of Consumer and Worker Protection (DCWP), which has authority to investigate complaints, conduct audits, and issue civil penalties.
Why it matters: LL144 is the first municipal law in the United States to mandate bias audits for AI hiring tools. It has become the baseline standard that other jurisdictions reference when drafting their own AI employment regulations. If you're wondering whether you need an AI audit, LL144 compliance is the starting point.
Who does Local Law 144 apply to?
The law applies to two categories of entities operating in New York City:
- Employers that use an AEDT to screen candidates for employment or employees for promotion within New York City.
- Employment agencies that use an AEDT to screen candidates on behalf of employers for positions located in New York City.
The geographic scope is specific: the law covers positions based in New York City. If you are a remote company hiring for a role that will be performed in NYC, the law applies. If you are a NYC-headquartered company hiring for a role in another jurisdiction, LL144 does not apply to that specific position—though other state or federal regulations may.
What about vendors?
The legal obligation falls on the employer or employment agency, not the technology vendor. However, in practice, most HR technology vendors proactively conduct bias audits and publish summary results to support their clients' compliance. If your vendor has not conducted an audit, the responsibility to obtain one remains yours.
This is critical for companies that rely on third-party applicant tracking systems, resume screening platforms, or AI-powered assessment tools. Purchasing a tool does not transfer compliance risk. You must verify that a valid, current bias audit exists for any AEDT you deploy for NYC roles.
What is an automated employment decision tool (AEDT)?
The DCWP final rules define an AEDT as any computational process, derived from machine learning, statistical modeling, data analytics, or artificial intelligence, that issues a simplified output—including a score, classification, or recommendation—that is used to substantially assist or replace discretionary decision-making for employment decisions.
Two conditions must be met for a tool to qualify as an AEDT:
- Computational process: The tool uses machine learning, statistical modeling, data analytics, or AI to generate its output.
- Substantially assists or replaces human judgment: The tool's output is used to materially influence hiring or promotion decisions. The DCWP clarified that a tool "substantially assists" a decision when it is relied upon to overrule or weight human decision-making, or when human review of candidates is informed primarily by the tool's output.
Examples of tools that typically qualify as AEDTs
- Resume screening or parsing tools that rank, score, or filter applicants
- Video interview analysis platforms that assess candidate responses
- Chatbot-based pre-screening tools that determine which candidates advance
- AI-driven assessment platforms that score cognitive abilities, personality traits, or job fit
- Internal promotion recommendation systems that rank employees
What does NOT qualify as an AEDT
- A tool that does not generate a score, classification, or ranking (for example, a search engine that retrieves resumes without ranking them)
- A tool whose output is not used to substantially assist or replace discretionary decision-making (for example, a scheduling tool used to book interviews)
- Junk email or spam filters applied to incoming applications
What does the bias audit require?
The bias audit is the technical core of LL144 compliance. It must be conducted by an independent auditor and must analyze the AEDT's impact on candidates across protected categories. The audit must use historical data from the tool's actual use—or, where historical data is unavailable, test data that is representative of the applicant population.
For a deeper explanation of how bias testing metrics work, see our dedicated guide.
Impact ratio analysis
The audit must calculate impact ratios (also called selection rates or adverse impact ratios) for each protected category. The impact ratio compares the selection rate of a given group to the selection rate of the most-selected group.
For a scoring-based AEDT (one that produces numerical scores), the audit must calculate the average score for each group and compare it to the highest-scoring group. For a classification-based AEDT (one that classifies candidates into categories such as "advance" or "reject"), the audit must calculate the proportion of each group that receives a favorable classification.
The four-fifths rule: While LL144 does not explicitly mandate the four-fifths (80%) threshold from the EEOC's Uniform Guidelines on Employee Selection Procedures, it remains the primary benchmark for evaluating disparate impact. An impact ratio below 0.80 (meaning a protected group is selected at less than 80% the rate of the most-selected group) is generally considered evidence of adverse impact and warrants further investigation or remediation.
Required demographic categories
The audit must calculate impact ratios across three dimensions:
| Category | Required Groups |
|---|---|
| Sex | Male, Female |
| Race/Ethnicity | Hispanic or Latino, White, Black or African American, Native Hawaiian or Other Pacific Islander, Asian, American Indian or Alaska Native, Two or More Races |
| Intersectional | All combinations of sex and race/ethnicity categories (e.g., Hispanic/Latino Female, White Male, Black/African American Female) |
The intersectional analysis is significant. It is not sufficient to show that your tool performs fairly across sex alone or race alone. You must examine combined categories. An AEDT could pass the bias audit on sex and race independently but still show adverse impact against, for example, Black women or Latino men when the categories are combined.
Data requirements
The final rules specify that the bias audit should use data from the AEDT's actual deployment. Where the employer has at least one year of historical usage data, that data should be used. For newly deployed tools or tools with insufficient historical data, the auditor may use test data or data provided by the vendor, provided the data is representative of the applicant pool.
If any demographic category contains fewer than a statistically meaningful number of data points, the auditor must note this limitation. The DCWP rules do not specify a minimum sample size, but standard statistical practice generally requires at least 30 observations per group for reliable analysis, and larger samples for intersectional categories.
What are the notice requirements?
Beyond the audit itself, LL144 imposes two separate notice obligations:
1. Notice to candidates
Employers must notify candidates at least 10 business days before the AEDT is used. The notice must include:
- That an AEDT will be used in connection with the assessment or evaluation of the candidate
- The job qualifications and characteristics the AEDT will evaluate
- Information about the candidate's right to request an alternative selection process or accommodation
- Instructions for requesting the data source, type, and retention policy for personal data collected by the AEDT
This notice can be provided on the careers page, in the job posting, or via direct communication (such as email) to the candidate.
2. Publication of bias audit results
The summary of results from the most recent bias audit must be publicly available on the employer's website. The published summary must include:
- The date of the most recent audit
- The source and explanation of the data used
- The number of individuals assessed by the AEDT, broken down by category
- The selection or scoring rates and impact ratios for each category
The summary must remain on the website for at least six months after the AEDT was most recently used for an employment decision.
What are the penalties for non-compliance?
Penalty structure: Violations of LL144 carry civil penalties of $500 for the first violation and $500 to $1,500 for each subsequent violation. Each day that a violation continues constitutes a separate violation. Each individual use of an AEDT in violation of the law can also be treated as a separate violation.
The financial exposure can accumulate rapidly. Consider a company that uses an AEDT to screen 50 candidates per day without a valid bias audit. Under a strict interpretation, each screening could constitute a separate violation. Over a 30-day period, this could result in penalties ranging from $750,000 to over $2.2 million.
Beyond DCWP penalties, employers face additional legal exposure:
- EEOC enforcement: If a bias audit reveals significant disparate impact and the employer continues using the tool, the results could be used as evidence in a Title VII discrimination claim.
- Private litigation: Candidates who are adversely affected by a biased AEDT may bring claims under the New York City Human Rights Law (NYCHRL) or the New York State Human Rights Law (NYSHRL), which provide broader protections than federal law.
- Reputational risk: Published audit results showing significant disparities can attract media scrutiny and damage employer brand.
The cost of a proper bias audit is a fraction of the potential penalties. For a breakdown of what audits cost and how to evaluate providers, see our AI audit cost comparison.
How do I choose a bias auditor?
LL144 requires that the bias audit be conducted by an independent auditor. The law defines independence as having no financial interest in, or being employed by, the employer or vendor that developed or distributed the AEDT.
Beyond meeting the independence requirement, here is what to evaluate when selecting an auditor:
Technical qualifications
- Statistical expertise: The auditor should have demonstrated experience in disparate impact analysis, selection rate calculations, and intersectional analysis. Ideally, the team includes industrial-organizational (I/O) psychologists, data scientists, or statisticians with employment testing experience.
- Legal knowledge: The auditor should understand LL144's specific requirements, EEOC guidelines, and the interplay between the bias audit and broader anti-discrimination law.
- AI/ML understanding: For complex AEDTs, the auditor needs the ability to evaluate model outputs and understand how algorithmic decisions are generated.
Process and deliverables
- Clear methodology: The auditor should document their analytical approach, data handling procedures, and statistical methods before beginning the audit.
- Compliant report format: The deliverable should include both a detailed audit report (for internal use) and a publication-ready summary that meets the DCWP's disclosure requirements.
- Remediation guidance: The best auditors don't just flag problems—they provide actionable recommendations for reducing disparate impact when it is found.
Red flags to watch for
- Auditors who guarantee a "passing" result before reviewing the data
- Firms with financial relationships with the AEDT vendor
- Providers that do not conduct intersectional analysis
- Reports that lack documentation of methodology or data limitations
What should an LL144 bias audit report include?
While the law specifies what must be included in the published summary, a thorough bias audit report should contain significantly more detail for internal compliance and risk management purposes:
Minimum required elements (for publication)
- Date of the audit
- Source and explanation of data used in the audit
- Number of individuals assessed, by category
- Selection rates or scoring rates for each category
- Impact ratios for each category
Recommended additional elements (for internal use)
- Methodology documentation: Detailed description of analytical methods, statistical tests applied, and thresholds used
- Data quality assessment: Evaluation of missing data, sample size adequacy, and demographic response rates
- Four-fifths rule analysis: Explicit comparison of each impact ratio against the 0.80 threshold with flagging of groups falling below it
- Statistical significance testing: Beyond the four-fifths rule, tests such as the Fisher exact test or chi-square test to determine whether observed disparities are statistically significant
- Intersectional analysis detail: Full breakdown of selection rates for all sex-by-race/ethnicity combinations, including sample sizes for each cell
- Limitations and caveats: Honest documentation of data gaps, small sample sizes, and any factors that may affect the reliability of results
- Remediation recommendations: Where adverse impact is found, specific suggestions for recalibrating the tool, adjusting scoring weights, or modifying candidate evaluation criteria
- Comparison to prior audits: If the AEDT has been previously audited, trend analysis showing whether disparities are improving, stable, or worsening
LL144 timeline: key dates
NYC Local Law 144 signed into law by Mayor Eric Adams.
DCWP publishes proposed rules for implementing LL144; public comment period opens.
Second public hearing held. Revised rules address scope of AEDT definition and data requirements.
DCWP publishes final rules. Enforcement date confirmed as July 5, 2023.
Enforcement begins. Employers using AEDTs for NYC roles must have a valid bias audit, published summary, and candidate notice in place.
Bias audits must be renewed at least annually. Employers must ensure continuous compliance for all AEDTs in use.
How does RunAIAudit help with LL144 compliance?
RunAIAudit provides end-to-end LL144 bias audit services designed to move employers from uncertainty to compliance efficiently. Our process covers every requirement of the law:
- Independent bias audit: Our auditors meet the independence requirements of LL144. We have no financial relationships with AEDT vendors.
- Full statistical analysis: We calculate selection rates, impact ratios, and intersectional breakdowns across all required demographic categories, using the four-fifths rule and additional statistical significance testing.
- Publication-ready summary: We deliver a formatted summary that meets DCWP disclosure requirements, ready to publish on your website.
- Detailed internal report: Beyond the public summary, we provide a comprehensive internal report with methodology documentation, data quality assessment, and remediation recommendations.
- Candidate notice templates: We provide compliant notice language for job postings, career pages, and candidate communications.
- Ongoing compliance support: LL144 requires annual re-auditing. We offer scheduled audit renewals to keep you continuously compliant.
Get your LL144 bias audit
Don't wait for a DCWP investigation. Our team can complete your bias audit and deliver publication-ready results within two weeks.
Start your auditFrequently asked questions
When did NYC Local Law 144 go into effect?
NYC Local Law 144 was signed into law in December 2021. Enforcement by the NYC Department of Consumer and Worker Protection (DCWP) began on July 5, 2023, after a delayed effective date and a public comment period on the final rules.
How often must a bias audit be conducted under LL144?
A bias audit must be conducted no more than one year prior to the use of an AEDT. This means the audit must be repeated at least annually for any tool that remains in active use for hiring or promotion decisions.
Does Local Law 144 apply to vendors or only employers?
The law places compliance obligations on employers and employment agencies that use AEDTs in New York City. However, many vendors proactively conduct bias audits and publish results to help their employer clients meet compliance requirements. Ultimately, the employer bears the legal responsibility.
What data is needed for an LL144 bias audit?
A bias audit requires historical data on the AEDT's outputs, including selection or scoring rates broken down by sex category (male, female), race/ethnicity category (Hispanic/Latino, White, Black/African American, Native Hawaiian/Pacific Islander, Asian, Native American/Alaska Native, two or more races), and intersectional combinations of sex and race/ethnicity. Where historical data is unavailable, representative test data may be used.
Can I use my own employees to conduct the bias audit?
No. The law requires that the bias audit be conducted by an independent auditor. The auditor cannot be employed by, or have a financial interest in, the employer or vendor that developed the AEDT. Independence is a core requirement of the statute.
What happens if my AEDT fails the bias audit?
The law does not prohibit the use of an AEDT that shows disparate impact. It requires that the audit results be published and that candidates be notified. However, using a tool with known significant disparities creates legal risk under existing anti-discrimination laws such as Title VII, NYSHRL, and NYCHRL. Most employers use audit findings to recalibrate or replace tools showing unacceptable bias.