Automated Candidate Screening: How to Set Up Each Stage
Which screening steps to automate, what each one should decide, and where a recruiter reviews before anyone is rejected

Automated candidate screening software applies your criteria to every applicant, using knockout questions, resume ranking, skills assessments, and similar steps, before a recruiter reviews the results. It earns its place when a role draws more applications than your team can read in full.
The steps involved aren't interchangeable, though. Some reject candidates, some only change the order in which a recruiter sees people, and some book the next conversation. A rejected candidate may never reach a person, so the first setup decision is which steps can reject at all.
Key takeaways
- Give every automated step one job (reject, sort, or schedule) and limit automatic rejection to objective must-haves.
- Put a recruiter review point before any rejection that rests on judgment, and read a sample of low-ranked candidates.
- Test the screen on a role you recently filled and compare its picks with the candidates your team advanced.
- Track completion, drop-off, and candidate feedback alongside recruiter time, and check the rules in each location where you hire.
How the stages of automated candidate screening fit together
An automated screen runs as a sequence, with each stage applying its criteria before passing candidates to the next stage or to a recruiter. Screening is one part of wider recruitment automation.
Not every stage involves artificial intelligence (AI). Knockout questions and scheduling run on fixed rules, while matching, ranking, and some chatbot screens may rely on AI models. In SHRM's 2024 survey, about a third of U.S. HR professionals who recruit with AI used it to screen resumes, communicate with applicants, or automate candidate searches. SHRM reported those three uses as one combined figure.
The table sets out six common stages and where a recruiter's review fits in each.
| Stage | What it checks | What it can decide | Where a recruiter reviews |
|---|---|---|---|
| Knockout questions | Yes or no answers on must-haves | Reject or sort | Before rejecting on any judgment call |
| Resume parsing | Text pulled into structured fields | Nothing on its own | Spot checks for misread fields |
| Matching and ranking | Fit against role criteria | Sort | Samples from the bottom of the list |
| Skills assessments | Job-related tasks or tests | Sort or reject below a cutoff | Cutoff scores and accommodation requests |
| Chatbot or one-way video screens | Answers to set questions | Sort or schedule | Transcripts before any rejection |
| Interview scheduling | Calendar availability | Schedule | Reschedules and missed calls |
What each stage decides and what to evaluate
A knockout question and a ranking tool go wrong in different ways, so each stage needs its own checks. The questions below apply whether you use the features built into your applicant tracking system (ATS) or add candidate screening software.
Knockout questions
Knockout questions work well for objective yes-or-no requirements, such as holding a required license or being available for a fixed shift. They're often built into the ATS already, so check what yours offers before paying for a separate tool.
The risk lies in questions that sound objective but describe a preference rather than a requirement. Consider a role that rejects anyone who answers no to holding a bachelor's degree. A candidate with equivalent experience drops out before anyone reads the resume. Turning the question into a sorting criterion keeps the preference but moves those candidates down the list instead of out of it, where a recruiter will see them.
Resume parsing
Parsing pulls work history, skills, and education from a resume into structured ATS fields. It doesn't decide anything on its own, but every later stage reads those fields. A resume-parsing error can make a strong candidate look weak. To catch it, compare a sample of parsed profiles against the original resumes, including some with multi-column layouts and nontraditional job titles.
Matching and ranking
Matching and ranking tools score each candidate against the role's criteria and sort the list. It's the stage most associated with automated resume screening. Some ATS platforms include basic matching, and dedicated AI resume screening tools add matching on related skills and titles. Because ranking only sorts, its effect depends on how far down the list your team reads.
When you evaluate either kind of tool, ask whether it explains each ranking and lets you adjust which criteria it weights for each role. An explanation helps a recruiter spot a ranking built on the wrong signal, such as a keyword match on a skill the candidate mentioned only in passing. Adjustable weights matter because the same title can carry different priorities for different hiring managers or clients.
Skills assessments
Skills assessments ask candidates to complete a job-related task, such as a coding exercise or a writing sample. They can sort by score or reject below a cutoff, so a cutoff deserves the same scrutiny as a knockout question. Evaluate whether the task reflects the work, how long it takes, and whether it works with screen readers. A longer task may show more, but it asks a lot of candidates who haven't spoken to anyone yet, so keep early assessments short.
Offer an accommodation route, such as extra time or a different format, with a person answering requests. In agency work, the client may supply the test, so agree on cutoffs and accommodation handling before the role opens.
Chatbot and one-way video screens
Chatbots and one-way video tools ask every candidate the same set of questions and pass the answers on for review. Because the answers require judgment, these screens work best for sorting or scheduling, with a recruiter reviewing transcripts before anyone is rejected. Asking everyone the same questions makes answers easier to compare, but consistency alone doesn't show that the screen is fair.
A recruiting chatbot can also answer candidate questions about pay, location, or schedule. Tell candidates what you'll record, who reviews it, and how to ask for a phone call instead.
Interview scheduling
Scheduling tools book the recruiter screen once a candidate clears earlier stages, so they decide the timing rather than the outcome. Evaluate calendar sync, time zone handling, reminders, and self-service rescheduling. The pre-screening interview that follows can then cover what automation couldn't, such as the context behind a career change.
How to set up automated screening from the intake
Every stage of automated candidate screening draws its criteria from the intake conversation with the hiring manager or client, so setup starts there.
- Write criteria from the intake: List the agreed requirements before you open any screening settings.
- Separate must-haves from preferences: A must-have is something the person can't do the job without, such as a required license. Put preferences, such as a degree or specific industry background, in sorting rather than rejection.
- Give each stage one decision: Record which stages can reject, which only sort, and which schedule.
- Write the candidate messages: Draft the confirmation, the next-step invitation, and the rejection message before launch. Explain what happens next without promising a timeline you may not keep.
- Add accommodation routes: Assign a named contact for accommodation requests at every stage, so a candidate who can't complete a timed test or video screen is still considered.
Agency recruiters keep criteria per role and per client, because two clients hiring for similar titles may define must-haves differently. Either client may also ask how candidates were screened. In-house teams own the process and its records directly. Either way, a clear record of each stage's decisions helps when you compare automated recruiting software, because you'll know what a tool needs to support.
Check low-ranked candidates before anyone is rejected
The safest place for a recruiter review point in automated candidate screening is just before any judgment-based rejection. Judgment-based criteria include the relevance of past experience, answers to open questions, and video responses.
It's also worth reading below the cutoff on a ranked list, even when the top candidates look strong. A 2025 University of Washington study of 528 online U.S. participants using simulated AI found that people mirrored the AI's recommendations, including biased ones. It wasn't a study of working recruiters, but it suggests a ranking can shape the judgment of whoever reviews it.
When you sample, pull candidates from the bottom of the list and read their resumes in full. Career changers, people with contract histories, candidates with employment gaps, and those with nontraditional titles are profiles a fixed set of criteria can misread. If you'd have advanced someone the system ranked low, find the criterion that pushed them down and adjust it.
See why candidates rank low before you reject them
Book a demoHow do you know the screen is working?
A screen is working when its picks line up with what experienced recruiters would choose, and candidates move through it without unexpected drop-off. Before launch, run automated candidate screening on applicants from a role you recently filled. Compare its results with the candidates your team advanced and hired, and where they disagree, look at the criterion behind the difference.
After launch, track candidate outcomes alongside recruiter time, because a setup can reduce workload while losing qualified people.
- Completion and drop-off by stage: A sharp drop at one stage can point to an overlong assessment or unclear instructions.
- Candidate feedback: A short post-process survey shows whether people understood the steps and knew how to ask for help.
- Recruiter time: Comparing time per role before and after shows whether candidate screening automation saves effort or moves it elsewhere.
Recheck the setup after you change criteria, edit a posting, or the vendor updates the tool, because each change can shift who passes.
Which rules apply to automated screening?
The rules depend on where the role is based, so check each hiring location's requirements with counsel before launch. New York City's Local Law 144, in effect since July 5, 2023, covers employers and employment agencies that use automated employment decision tools for NYC jobs. It requires a bias audit within one year of use, public audit information, and notices to candidates or employees. Agencies are named alongside employers, so agency recruiters filling NYC roles should agree with clients on who handles each requirement.
Illinois Public Act 103-0804, effective January 1, 2026, bars Illinois employers from using AI that has a discriminatory effect on protected classes. It also bars employers from using zip codes as a proxy for those classes and requires notice when they use AI. California's Civil Rights Council rules, effective October 1, 2025, require covered employers to keep automated-decision data with employment records for at least four years.
Federal EEOC record-keeping rules require covered U.S. private employers to keep application forms and hiring records for one year. The year runs from the record date or the personnel action, whichever is later. A candidate notice that meets one rule may not satisfy another. The practices in this guide support a careful process, but they don't prove compliance.
Check who your screen rejects before you widen it
Once the screen holds up on a recently filled role, roll automated candidate screening out one role at a time rather than across every opening at once. Each new role has its own must-haves, so reviewing a sample of rejected resumes early shows whether the rules still fit.
Check the reasoning before you widen your screen
Book a demoFrequently asked questions
Should candidates be able to opt out of AI screening?
Offering a person-reviewed alternative, such as a phone call with a recruiter, keeps candidates who can't or won't use the tool in your pool. Whether an opt-out is legally required depends on where the role is based, so confirm that with counsel.
How many stages does an automated screen need?
Use only as many stages as the role's volume and requirements call for. A role with a small, specialized applicant pool may need only knockout questions and scheduling, while a high-volume role may justify ranking and an assessment.
About the author

Founder & CEO of Rotto, building tools that help tech recruiters source better candidates, faster.





