CRM Data Deduplication: How Recruiters Find and Merge Duplicate Records

Match rules, merge rules and prevention steps that keep opt-outs, notes and history intact when one person shows up two or three times

Written by Mihailo Bozic
A recruiter compares two candidate records on a laptop screen while making notes in a co-working space

In a recruiting database, CRM data deduplication takes more care than tidying sales leads. One person can be a candidate, a client contact and a referral source at once. When their records split, the notes, opt-outs and history split with them.

Deduplication means finding the records in your customer relationship management (CRM) system or applicant tracking system (ATS) that describe the same person or company. You combine them into one accurate record and stop new copies from being created. Because each merge is a decision about a real person, the goal is one record that keeps every opt-out, consent record and note from both versions.

Key takeaways

  • Match records on strong identifiers like personal email, phone number or profile link, never on name alone.
  • Sort possible matches into three groups: merge, review by hand, or mark as different people.
  • When two records disagree on an opt-out or consent, keep the more restrictive one.
  • Put a match check in front of record creation and imports so new duplicates stop at entry.

What is CRM data deduplication?

CRM data deduplication is the process of identifying records that refer to the same person or organization and merging them into one record you can trust. It also covers the checks that keep new duplicates out. In recruiting, the CRM or ATS holds candidates and the client contacts who hire them, so duplicates can appear on either side or across the two.

The broader term "data deduplication" also names a storage technique for removing repeated copies of files, which this guide doesn't cover. Here the focus is on records about people and companies, where the hard part is deciding whether two records describe the same human being.

Why duplicates pile up in a recruiting database

Recruiting databases can collect duplicates quickly because records arrive from so many directions. Resume parsing and job board imports create a record each time someone applies. A candidate who applies once with a personal email and again with a new address can land as two people. Spreadsheet and sourcing tool uploads add more when nobody checks the file first.

People also change in ways that defeat simple matching. A candidate you placed a few years ago may now be a hiring manager at a client, with a candidate record and a separate client contact record. That split can happen when a recruitment CRM and an ATS each hold half the picture. Name changes, nicknames and different spellings add near-matches that a strict rule misses.

Team habits contribute too. Two recruiters on similar searches may each add the same person from different sources. Integrations that sync in both directions can also recreate a record that someone just merged on the other side.

What duplicate records cost a recruiting desk

The most visible cost falls on candidates. Two recruiters can contact the same person about the same role without knowing it. In agency work, a candidate can be submitted twice to one client, which raises awkward questions about who represents them.

Other costs come from information that sits on the wrong record. An opt-out on one version doesn't stop an email sent from the other, so someone who asked to stop hearing from you may still get outreach. Notes split across records also mean nobody sees the full history before a call, which weakens the context that candidate relationship management depends on.

Reporting suffers as well, since duplicates inflate pipeline counts, source-of-hire figures and talent pool sizes. Decisions about where to spend sourcing time then rest on overcounted numbers. Deduplication is one part of keeping the database usable, and broader CRM hygiene handles outdated details and the rest of the cleanup.

How to deduplicate CRM data step by step

CRM data deduplication works best as a repeatable process that moves from a small, backed-up scope through matching, review and merging to prevention. The flow chart shows how each possible match moves through that decision, and the numbered steps add the setup and checking around it.

  1. Back up and pick a scope: Export the records you plan to touch so you can restore a bad merge. Start with one segment, such as candidates added in the last year.
  2. Set match rules: Decide which fields count as strong evidence of a match and which only support one, using the field table below.
  3. Sort matches by confidence: Merge high-confidence matches and send uncertain ones to a person for review. Mark confirmed different people so the pair stops resurfacing.
  4. Choose the surviving record and merge rules: Keep every opt-out and consent record and the most recently confirmed contact details. Keep notes, activity, submissions and attachments from both records.
  5. Merge and spot-check a sample: Open a sample of merged records and confirm the history, opt-outs and contact details came through.
  6. Prevent new duplicates: Add a match check before a record is created and require the fields your match rules use. Set import rules and name an owner for the review queue.

Which fields make a reliable match

Strong identifiers belong to one person and rarely change hands, so a match on one of them carries weight. Weak identifiers like name and employer help confirm a match but shouldn't trigger a merge alone.

FieldMatch strengthWatch out for
Personal emailStrongShared family addresses and typos
Work emailMediumChanges when the person changes jobs
Phone numberStrongOffice lines and recycled numbers
Public profile linkStrongOld links and different link formats
Full nameWeakCommon names and nicknames and name changes
Current employerWeakOut of date after a job change
CityWeakRelocation and remote work

Matching works best when you combine fields, so a shared personal email plus a similar name is stronger evidence than a matching name and city. Match only on contact and work details. Never use protected characteristics such as age, gender or ethnicity to decide whether two records belong to the same person.

Merge rules to check before you combine records

The merge rules decide what the surviving record keeps, and in recruiting the safest default is to keep information rather than overwrite it. Two rules apply to every field. When two records disagree on an opt-out or consent, the more restrictive one wins, and when you're unsure two records describe the same person, don't merge them.

  • Contact details: Keep every email and phone number and mark the most recently confirmed one as preferred. An older address may still be the one the person answers.
  • Opt-outs and consent: Carry over every opt-out, unsubscribe and consent record with its date, and apply the most restrictive status.
  • Status and stage: Keep the most advanced active stage, and check with both owners if the person appears in two searches.
  • Notes and activity: Keep all notes, calls, emails and meetings from both records in date order.
  • Documents: Keep every resume and attachment, and label versions by date instead of deleting older ones.
  • Ownership: Agree on one owner before the merge, especially in agency teams where ownership affects who represents the candidate.
  • Source: Keep the original source and the date the person first entered the database, since source reporting depends on it.

Consider a candidate who applied through a job board with a personal email and was later added from a referral with a work email. The job board record holds an email opt-out. After the merge, the opt-out stays, both emails stay with the newer one marked as preferred, and both recruiters' notes remain visible.

Find the right people when a role opens

Book a demo

Privacy and record-keeping when you merge records

Privacy and record-keeping rules are part of why opt-outs and history deserve the most care in a merge. For commercial email, the FTC's CAN-SPAM guide (August 2023) says you must honor an opt-out request within 10 business days. An opt-out on either record has to survive the merge and apply to every address you keep.

For businesses covered by the California Consumer Privacy Act (CCPA), the CPPA's FAQ says the law covers California residents who are job applicants and contacts for business customers. The same FAQ lists a right to correct inaccurate personal information. Deduplication supports that right by giving each person one record to correct instead of several.

Record-keeping rules are a reason to combine history rather than delete it, since EEOC regulations require covered employers to keep personnel or employment records for one year. Once a charge is filed, the records must be kept until the charge or any related lawsuit is resolved. Because that rule is written for employers, check with counsel how it applies to your candidate records before purging anything. Other states have their own privacy laws too.

How often to deduplicate CRM data

Deduplication works as an ongoing habit at entry, with larger cleanups at set points. The match check at record creation runs all the time, so a recruiter adding a referral sees the existing record before creating a new one.

The review queue for uncertain matches needs a set rhythm, such as weekly, and a named owner. Otherwise unreviewed pairs pile up and get merged in a rush later. A larger pass makes sense before a system migration, an integration change or a big import, since those are the moments when duplicates arrive in bulk.

Run deduplication before you fill in missing fields. Enriching CRM data on two copies of the same person doubles the work and can attach new details to the wrong record. The same order applies to client contacts, where checking that B2B contact data is current is simpler once each contact exists only once.

Start with one segment before the next import

Strong match rules, a person reviewing uncertain pairs and merge rules that keep every opt-out and note turn CRM data deduplication into a set of careful decisions. Prevention at entry keeps that work from repeating, because each duplicate stopped at creation is one you never review. Pick one segment this week, back it up and run your match rules. Then work through the review queue before your next import adds more records.

Start your next search with the right shortlist

Book a demo

Frequently asked questions

Should deduplication be automated or done by hand?

Both have a place. Automated merging suits matches on strong identifiers like a shared personal email. Records that match only on name, employer or city need a person to review them first.

Can a merge be undone?

That depends on your system, and some can't split a merged record cleanly. A backup taken before the merge and a log of combined records let you rebuild the originals if two different people were merged.

What should happen when a candidate becomes a client contact?

Decide as a team whether your system holds one record with both roles or two linked records, and apply that choice consistently. Either way, the person's opt-outs and consent should apply across both roles.

How do recruiters keep outreach lists from creating duplicates?

Check outreach lists against existing records before you import them, matching on email and phone. A clean email outreach setup also keeps unsubscribes on the existing record instead of a new copy.

About the author

Mihailo Bozic
Mihailo Bozic

Founder & CEO @ Rotto

Founder & CEO of Rotto, building tools that help tech recruiters source better candidates, faster.

Related articles

B2B Technographic Data: A Practical Guide for Recruiters
Recruiter Playbooks

B2B Technographic Data: A Practical Guide for Recruiters

Learn what B2B technographic data shows, how providers collect it, and how recruiters can use it for business development, sourcing, and vetting providers.

By Mihailo Bozic
AI Recruiting: A Practical Guide for Recruiters
Hiring TrendsRecruiter Playbooks

AI Recruiting: A Practical Guide for Recruiters

AI recruiting explained for working recruiters: what AI does at each hiring stage, the risks, the U.S. rules to know, and how to start safely.

By Mihailo Bozic
Candidate Data: What It Includes and How Recruiters Use It
Recruiter Playbooks

Candidate Data: What It Includes and How Recruiters Use It

Learn what candidate data includes, six practical ways to use it in recruiting, and how to keep your candidate records accurate, fair, and lawful.

By Mihailo Bozic
How to Use AI in Your ATS
Recruiter Playbooks

How to Use AI in Your ATS

Use AI in your ATS without missing good candidates: start with features a recruiter reviews, then test ranking on a role you have already filled.

By Mihailo Bozic
Campus Recruiting: How to Plan a Season That Ends in Hires
Recruiter Playbooks

Campus Recruiting: How to Plan a Season That Ends in Hires

Plan campus recruiting from school choice to start date: pick schools, set offer deadlines schools accept, track intern conversion, and know agency rules.

By Mihailo Bozic
9-Box Grid: What It Is, How to Run One, and What It Means for Hiring
Recruiter Playbooks

9-Box Grid: What It Is, How to Run One, and What It Means for Hiring

Learn what a 9-box grid is, what each box means, how to run a fair 9-box talent review, and how to turn the gaps it shows into a hiring plan.

By Mihailo Bozic
Rotto

Built for recruiters who work from a brief, not a boolean.

rotto.com