Gartner estimates that poor data quality costs the average organization $12.9 million per year. The largest single contributor to that cost is duplicate records: the same customer appearing as two, three, or ten different entries across your systems.
This is not a minor data hygiene issue. Duplicate records mean your churn model counts one churning customer as two retained ones. Your CLV model underestimates high-value customers whose spend is fragmented across records. Your marketing team sends three copies of the same campaign to the same person. Your compliance team fails KYC reviews because the full relationship picture is spread across unlinked records.
The reason this problem persists is not that nobody has tried to solve it. It is that the standard approaches hit a ceiling, and the remaining 15-20% of unresolved duplicates cause outsized damage.
customer_records (suspected duplicates)
| record_id | name | address | phone | |
|---|---|---|---|---|
| R-001 | Robert J. Smith | rsmith@gmail.com | 123 Main St, Apt 4B | 555-0142 |
| R-002 | Bob Smith | bob.smith@work.com | 123 Main Street #4B | 555-0142 |
| R-003 | R. Smith | — | P.O. Box 881, 10001 | — |
| R-004 | Robert Smith Jr. | rjsmith@outlook.com | 456 Oak Ave | 555-0199 |
| R-005 | Roberto Smith | rsmith@gmail.com | 123 Main St, 4B | — |
R-001, R-002, R-003, and R-005 are likely the same person. R-004 is a different person (son). Deterministic rules struggle here: email matches R-001/R-005 but not R-002/R-003. Address matches partially. Name varies wildly.
transaction_patterns (last 6 months)
| record_id | top_merchants | avg_txn | preferred_time | payment_method |
|---|---|---|---|---|
| R-001 | Whole Foods, Shell, Amazon | $67.40 | Evenings | Visa ending 4821 |
| R-002 | Whole Foods, Shell, Amazon | $72.10 | Evenings | Debit ending 9033 |
| R-003 | Whole Foods, Shell Gas, AMZN | $68.90 | Evenings | Visa ending 4821 |
| R-004 | GameStop, Chipotle, Spotify | $23.50 | Afternoons | Apple Pay |
| R-005 | Whole Foods, Shell, Amazon | $65.20 | Evenings | Visa ending 4821 |
Highlighted: R-001, R-002, R-003, and R-005 share near-identical behavioral fingerprints. R-004 has a completely different pattern. The graph confirms what attributes alone cannot.
How entities become duplicated
Duplicates enter databases through six primary channels, and understanding these channels explains why simple matching rules fail.
Data entry variation
"Robert Smith" and "Bob Smith" are the same person. "123 Main Street, Apt 4B" and "123 Main St #4B" are the same address. "ABC Corporation" and "ABC Corp." and "A.B.C. Corporation" are the same company. Every field that accepts free text generates variation. At enterprise scale, with millions of records entered by thousands of people over years, the variation is enormous.
System mergers and acquisitions
When companies merge, their databases merge. Customer A in System 1 may be the same as Customer B in System 2, but the identifiers are different, the schemas are different, and the field formats are different. A large bank that has completed three acquisitions in the last decade may have the same customer in four separate systems with four different customer IDs.
Channel proliferation
A customer who shops in-store, online, and through a mobile app may be three separate records: one with a loyalty card number, one with an email address, and one with a phone number. None of the identifiers overlap. The only way to connect them is through behavioral and relational signals.
Life events
Name changes (marriage, legal changes), address changes (moves), phone number changes, and email changes all create potential duplicates. A customer who moved and changed their name will not match their previous record on any deterministic attribute.
Why deterministic matching hits a ceiling
The standard approach to entity resolution is deterministic rule matching: if email matches exactly, merge. If first name, last name, and zip code match, merge. If phone number matches, merge. These rules are fast, interpretable, and they handle the easy 80%.
The problem is the other 20%.
A rule that merges on exact email match misses customers who use different email addresses for different channels (work email for B2B, personal email for consumer). A rule that merges on name plus zip code generates false positives for common names in dense zip codes (there are 347 "John Smith" records in the 10001 zip code). Tightening the rules reduces false positives but increases false negatives. Loosening the rules does the opposite.
This is the fundamental limitation: deterministic rules optimize along a single dimension (attribute similarity) and cannot break through the precision-recall tradeoff without additional signal.
Probabilistic matching is better, but still limited
Probabilistic matching (Fellegi-Sunter models) assigns weights to each matching field based on how discriminating it is. An exact match on Social Security number is weighted higher than an exact match on first name. The total weight determines the match probability.
This is a meaningful improvement over deterministic rules because it handles partial matches and field-level confidence. But it still only looks at attributes on the record itself. Two records with completely different attributes but identical behavioral patterns are invisible to probabilistic matching.
Attribute-only matching
- Compares name, address, email, phone fields
- Rules handle the easy 80% of duplicates
- False positives on common names in dense areas
- Misses customers with different identifiers per channel
- Cannot resolve records after name/address changes
Relational + behavioral matching
- Compares transaction patterns, product preferences, timing
- Identifies shared connections and network overlap
- Resolves cross-channel identities through behavioral similarity
- Handles name/address changes by matching on behavior continuity
- Catches the remaining 15-20% that rules miss
How graph ML transforms entity resolution
The breakthrough in entity resolution comes from looking beyond the record to the relational context around it. Two records that share no attributes may still be clearly the same entity when you look at what they are connected to.
Behavioral fingerprinting
Every customer has a behavioral fingerprint: the products they buy, the times they shop, the payment methods they use, the stores they visit, the categories they browse. Two records with different names and email addresses but near-identical behavioral fingerprints are almost certainly the same person. Graph ML learns these fingerprints from the transaction-product-store graph, not from manually engineered features.
Shared network analysis
If Record A and Record B both transact with the same set of merchants, live in the same building (different apartment numbers), and share a phone number with the same emergency contact, the probability that they are the same person is high. These shared connections are edges in the graph. The model measures overlap in the local neighborhood of each record to assess match probability.
Temporal pattern matching
When Record A stops transacting and Record B starts transacting in the same geographic area with similar patterns, this temporal handoff suggests a single customer who changed identifiers. The model learns these temporal discontinuities as evidence of entity continuity.
temporal_handoff (suspected identity change)
| date | R-001 transactions | R-003 transactions | location |
|---|---|---|---|
| Jan 2025 | Whole Foods $72, Shell $45 | — | Manhattan |
| Feb 2025 | Amazon $134, Shell $51 | — | Manhattan |
| Mar 2025 | Whole Foods $68 | — | Manhattan |
| Apr 2025 | — | Whole Foods $71, Shell $48 | Manhattan |
| May 2025 | — | Amazon $142, Shell $52 | Manhattan |
Highlighted: R-001 stops transacting in March. R-003 starts in April with near-identical patterns at the same merchants, in the same area. This temporal handoff strongly suggests the same person with a new record (perhaps a replaced card or new account).
Transitivity and graph propagation
If Record A matches Record B with 95% confidence, and Record B matches Record C with 90% confidence, what is the probability that A and C are the same entity? Deterministic systems cannot propagate matches through chains. Graph models handle transitivity natively because the information flows along edges. This resolves clusters of duplicates that pairwise matching misses.
transitive_match_chain
| pair | direct_attribute_sim | direct_match_prob | transitive_evidence |
|---|---|---|---|
| R-001 / R-002 | 0.61 | 0.72 | Shared phone: 555-0142 |
| R-002 / R-005 | 0.38 | 0.68 | Same work email domain + Visa 4821 |
| R-001 / R-005 | 0.72 | 0.81 | Shared email: rsmith@gmail.com |
| R-001 / R-003 | 0.24 | 0.41 | No direct link strong enough |
Pairwise rules set at 0.70 threshold would match R-001/R-002 and R-001/R-005, but miss R-001/R-003 (score 0.41). Graph propagation chains: R-001 matches R-002 (phone), R-002 matches R-005 (email domain), R-005 matches R-003 (Visa ending 4821 + merchant overlap). The full cluster resolves through transitivity.
Real-world impact of better entity resolution
When a large retailer resolved its duplicate customer records using relational matching, three things happened.
Marketing efficiency improved by 22%. The marketing team had been sending multiple copies of campaigns to the same customers through different records. Resolving duplicates reduced campaign volume while maintaining reach, cutting direct marketing costs by 22% with no drop in response rates.
CLV models became 35% more accurate. High-value customers whose spend was fragmented across multiple records had been undervalued. After resolution, the CLV model correctly identified the top decile, improving retention program targeting and increasing retained revenue by $18 million annually.
Compliance risk dropped. Financial institutions are required to maintain a single customer view for KYC and AML compliance. Unresolved duplicates create fragmented profiles that regulators flag as deficiencies. Better entity resolution reduces the risk of regulatory fines that can reach hundreds of millions of dollars.
resolution_method_comparison
| method | precision | recall | F1 score | false_merge_rate |
|---|---|---|---|---|
| Exact email match | 99.1% | 42.3% | 59.3% | 0.9% |
| Name + zip code | 84.7% | 68.9% | 75.9% | 15.3% |
| Probabilistic (Fellegi-Sunter) | 91.2% | 78.4% | 84.3% | 8.8% |
| Behavioral + relational ML | 96.8% | 93.1% | 94.9% | 3.2% |
Highlighted: behavioral and relational matching achieves 94.9% F1 by combining attribute signals with transaction patterns, shared networks, and temporal continuity.
PQL Query
PREDICT match_probability FOR EACH customer_pairs.pair_id WHERE customer_pairs.attribute_similarity > 0.3
Score all candidate record pairs for match probability. The model considers not just name/address similarity but behavioral fingerprints, shared merchant networks, payment method overlap, and temporal transaction patterns.
Output
| pair | attribute_sim | behavioral_sim | match_prob | verdict |
|---|---|---|---|---|
| R-001 / R-002 | 0.61 | 0.94 | 0.97 | Match |
| R-001 / R-003 | 0.38 | 0.91 | 0.93 | Match |
| R-001 / R-005 | 0.72 | 0.96 | 0.98 | Match |
| R-001 / R-004 | 0.54 | 0.12 | 0.08 | No match |
The foundation model approach
Traditional ML entity resolution requires training a custom model: generating labeled pairs (match/non-match), engineering features from both records and their relational context, training a classifier, and calibrating thresholds. This works well but requires 3-6 months of data science effort and ongoing maintenance.
KumoRFM applies a foundation model approach to entity resolution. The model has been pre-trained on relational patterns across thousands of databases. It already understands the universal patterns that indicate entity matches: behavioral similarity, network overlap, temporal continuity, and attribute fuzzy matching.
You connect your database, specify which records to resolve, and the model produces match probabilities based on the full relational context. No feature engineering, no labeled training data, no custom model. The pre-trained understanding of relational patterns transfers to your specific data.
The implication is significant. Entity resolution has always been treated as a data engineering project: months of work to build matching rules, ongoing maintenance as data changes, and a ceiling on accuracy that attribute-only matching cannot break through. Graph ML on relational data raises that ceiling. Foundation models eliminate the engineering cost to reach it.
If your database has more than a million customer records and you have not performed entity resolution in the last 12 months, you are almost certainly making decisions based on fragmented data. The question is not whether you have duplicates. It is how much they are costing you.