When records in CRM, ordering and service systems have similar names, do they represent one customer? That depends on whether customer means a legal entity, branch, account or corporate group. Establish reversible identity mappings before deciding which attributes to aggregate. Similarity supplies evidence; it cannot determine identity alone. The following engineering recommendations draw on public sources and do not describe an IDENIFE deployment or measured results.
Define the entity before normalizing strings
Entity resolution determines whether records refer to the same real-world object; deduplication may additionally remove redundant records. Do not collapse these into an irreversible deletion. Two legal entities in one group, or two sales accounts belonging to one legal entity, can have similar names while carrying different business and financial responsibilities. Define the target entity, permitted one-to-many relationships and boundaries that automatic merging must not cross.
Preserve the source system, original record identifier and raw fields, then maintain a separate mapping from source records to target entities. Represent group membership, branch relationships and shared contacts as relationships, rather than giving them a common entity identifier. A mistaken mapping can then be reversed without dismantling the original orders, contracts and contacts.
Normalization may standardize whitespace, punctuation and character forms, but should not indiscriminately remove branch, location or organization-type terms that distinguish entities. Retain raw and normalized values with the rule version. Identical identifiers are strong evidence only when their meaning, scope and source reliability agree. Account numbers stored in similarly named fields are not necessarily cross-system identity evidence.
Candidate recall and final matching are separate gates
Comparing every record pair quickly becomes expensive. Splink separates candidate generation from subsequent match scoring: blocking rules reduce the comparison space before candidate pairs receive scores.[1] Blocking here means bringing potentially matching records into a comparison set, not denying access.
Use the union of several targeted rules, such as agreement on a trusted identifier, normalized name plus location, or name fragments plus verified contact details. Document the error pattern each rule covers. A phone number can be shared by a group; an address can belong to a business park or registration agent. Neither agreement alone establishes a common legal identity. Missing values should not count as agreement.
Comparing only exact-name matches and reporting accuracy within that set hides misses caused by name changes and data-entry differences. Build an independently confirmed set of same-entity pairs and measure how many enter the candidate set: candidate recall. Its denominator must be established outside the candidate set, or records missed at the first gate will never appear in acceptance testing.
When candidate recall is low, classify the missed cases and add targeted rules. Unrestricted broadening increases computation and review costs while introducing more similar but distinct entities. Very large blocks may need extra discriminating fields or separate processing; record exclusions and provide a review path for them.
A score measures evidence, not automatically a real probability
Deterministic rules are explainable but may miss cases; probabilistic matching combines evidence across fields and uses thresholds to trade false links against missed links.[2] For enterprise implementation, ask how much independent information a field provides. Agreement on a common name should not carry the same interpretation as agreement on a rare, trusted identifier.
Store field comparisons separately from the final score. Name edit distance, address overlap, matching contact details and missingness should remain traceable. Name similarity and an embedding similarity calculated from that same name are strongly related; adding both can count the same evidence twice. When a model assumes independence, check for overconfidence caused by these correlations.
Do not label an uncalibrated similarity score as the probability of a shared customer identity. Even a model that outputs probabilities needs independent evaluation: within score bands, compare predicted values with observed correctness under the actual candidate distribution. A sample hand-picked from obvious duplicates does not establish performance across the business database. Changes in source systems, languages or missingness require renewed checks.
Use three decision bands to preserve human judgment
Separate outcomes into automatic linking, human review and no link for now. Automatic linking requires evidence consistent with the business's tolerated false-merge risk and no hard conflict. Intermediate cases need reviewers with business context; records lacking evidence remain separate. No link for now does not prove that two records are different.
Different verified entity identifiers with the same scope can block an automatic merge. This should trigger investigation of identifier errors, organizational change or the target-entity definition, rather than simply erasing all other evidence. A review interface should show supporting and opposing evidence, not just a high score and a confirmation button.
Set thresholds using false-merge losses, missed-link losses and review capacity. Financial consolidation and duplicate marketing contacts have different cost structures. There is no universal enterprise score threshold. If the review queue exceeds capacity, allow a backlog or narrow the automated scope; do not lower evidence requirements just to clear the queue.
Correct-looking pairs do not guarantee a correct customer cluster
Consider this constructed scenario. A has verified entity identifier Alpha, while C has a different verified identifier, Beta. B has no identifier, uses a group-wide phone number and has a name similar to both A and C. If A–B and B–C pass similarity rules, grouping by connectivity alone puts all three in one entity. The two local relationships do not resolve the conflict between Alpha and Beta.
Before adding a link, or after forming a cluster, check constraints across the whole cluster. Does it contain mutually exclusive identifiers, cross a prohibited organizational boundary, or use an information-poor record to bridge clearly distinct groups? Here B can remain under review, while the common phone number provides a clue to group membership instead of forcing a choice of legal entity.
These checks add computational and review costs, particularly where customer identities are propagated to many downstream systems. Reviewing only high-scoring pairs can miss large clusters created by chains of links. Additionally inspect large clusters, rapidly growing clusters and clusters joined by a single weak-evidence bridge.
Test error denominators and the ability to reverse a mapping
The UK Office for National Statistics distinguishes match rate from linkage quality and uses precision and recall to describe different errors.[3] This article adopts that quality perspective as engineering guidance, not as an enterprise compliance obligation. Report candidate recall, the proportion of linked pairs that are correct, and the proportion of independently confirmed same-entity pairs recovered by the final process.
Also track cluster conflicts, review backlog and downstream effects. Samples should cover source systems, missingness patterns and score bands. If risky cases are deliberately oversampled, weight aggregate estimates to the actual population rather than presenting a raw sample average as production accuracy. Where ground truth is limited, report coverage and unresolved cases instead of an excessively precise headline.
Production mappings will encounter name changes, newly supplied identifiers and historical corrections. Incremental processing can draw on versioning and replay in CDC data products, but customer identity requires its own version history. Record input versions, rule versions, evidence and effective scope for every link, split and reviewer rejection. A replay must explicitly choose the historical mapping or the corrected mapping; it must not silently change historical definitions.
Finally, rehearse reversal: introduce a constructed false merge, remove the link and recompute affected customer metrics. Confirm that original records remain, downstream dependencies can be located and basic facts such as monetary amounts have not disappeared through deduplication. A changed customer count may be legitimate; missing fact rows cannot be explained away as successful deduplication. Validate shadow mappings before gradually enabling downstream use—a more costly approach with clearer boundaries.