Deduplication is the process of detecting and merging duplicate leads before they are processed further. The goal is to ensure that one and the same prospect enters distribution only once. Without consistent deduplication, buyers pay for the same contact multiple times and risk approaching it in parallel. Deduplication is therefore a core element of data validation and sound lead hygiene.
Hard and soft duplicates
There are two types of duplicates, detected with different levels of strictness.
- Hard duplicates – an exact match on stable attributes such as phone number or email address. These can be identified and blocked reliably by machine.
- Soft duplicates – a probable match based on a combination of name, postal code and a time window. Detection here works with thresholds, because the same person may enter details slightly differently.
Ideally, both methods apply the moment a lead is accepted, not only after the fact.
Example
A prospect requests a quote within two days through two different comparison portals. Both records carry the same mobile number. Deduplication recognises the hard duplicate, discards or merges the second record with the first, and thereby prevents two buyers from paying for the same contact.
Relation to Leadnodes
Leadnodes automatically checks every incoming lead for duplicates the moment it is captured via API, webhook, email, CSV, Zapier or Make. Hard duplicates are matched on phone number and email, soft duplicates on the combination of name, postal code and time window. Detected duplicates are removed before rule-based distribution begins, so a prospect is delivered only once to a suitable buyer. This lowers complaints and keeps data quality high. Processing is GDPR-compliant and hosted in Germany.
Frequently asked questions
What is the difference between a hard and a soft duplicate?
A hard duplicate exists when stable identifiers such as phone number or email match exactly. A soft duplicate is a probable match detected through name, postal code and a time window. Hard duplicates are blocked, while soft duplicates are scored against thresholds.
Why is deduplication so important in lead trading?
Without deduplication, buyers purchase the same prospect several times and may contact it in parallel. This causes duplicate costs, annoys the prospect and leads to complaints. Clean deduplication protects both budget and reputation.
When should deduplication happen?
Ideally right when the lead is accepted, before distribution. That way duplicates are caught before a buyer pays for them. A pre-ping additionally helps to intercept duplicates before the actual sale takes place.
Would you like incoming leads deduplicated and cleanly distributed automatically? Book a demo
