Detect duplicate records after changes

Purpose

Staff receive duplicate suggestions as records change, while saving remains independent of the detection work.

How It Works

Rhythm processes committed record changes in the background. It selects changes to the names, identity information and contact details used for matching. Other edits do not trigger another comparison. Records are found through shared, normalized values rather than a comparison with the entire workspace. A common switchboard number can be disregarded while a rarer value still connects two records. The pass writes all keys for a batch before looking for matches within it. It refreshes evidence on existing pairs and removes detected suggestions that no longer qualify. Human decisions remain separate from the changing resemblance. Linked employer details do not count as a person's own matching information. A failed pass returns errors for the batch so processing can be attempted again. The product should complete the import-completion connection before import changes begin carrying a job marker.

Relationships

Scoring supplies the reasons for each suggestion. Duplicate settings determine the action associated with a match. Re-checks use the same detection process, and review decisions constrain which suggestions may disappear.

Show evidence (sources, provenance, origins, pinned commits)

The Formal Contract

The narrative above explains; the sections below are authoritative — the rules, scenarios and tests this document binds.

Outcomes: Background changes produce review suggestions without another staff action.Detection failures do not block saves, imports or integration writes.Overly common values do not create an unbounded queue of comparisons.
Surfaces: job
Preconditions: A workspace contact or organization has a committed create, edit or delete.
Postconditions: Detection has refreshed the applicable keys and inserted, updated or removed the affected candidates.
Intent sources: services-crm:docs/merging/README.md#Status (architecture and plan approved by Andrew 2026-10-11)services-crm:docs/merging/decisions.md#Settled by inspection or engineering practice (LLM in detection: none)

Controls

18 controls apply, from the corpus control catalogue.

TC-CRY-004 TC-DAT-001 TC-DAT-002 TC-DAT-003 TC-IAM-001 TC-IAM-005 TC-IAM-006 TC-LOG-005 TC-MON-001 TC-MON-002 TC-PI-001 TC-PI-002 TC-PI-004 TC-PRV-001 TC-PRV-002 TC-TEN-001 TC-TEN-002 TC-TEN-003

Business Rules

12 rules, 11 to be proved before approval. A rule's product intent is derived from its origin: 12 confirmed by a product decision, the rest inheriting this capability's confirmed.

Id	Rule	Kind	Enforced at	Provenance
`01`must prove Confirmed	Committed change events trigger background detection; saving neither waits for that work nor fails because detection fails. origin: services-crm:docs/merging/README.md#Status (architecture and plan approved by Andrew 2026-10-11) proved by `ATS-CRM-DQ-001-01`, `ATS-CRM-DQ-001-11`, `ATS-CRM-DQ-001-20`, `TST-CRM-DQ-176`	policy	service	observed services-crm:modules/duplicates/services/detection/duplicate-change-events.ts:DUPLICATE_DETECTION_HANDLER_NAME
`02`must prove Confirmed	Detection refreshes keys and scores only for changes to matching names, birth date, spouse, affiliations, organization identity and hierarchy, origin identifiers or contact details, excluding job title, notes, status and region. ✓ For example, a coordinator corrects a member's last name; the member's pairs are re-scored.✗ By comparison, a coordinator changes the member's job title; nothing is re-scored. proved by `ATS-CRM-DQ-001-02`, `ATS-CRM-DQ-001-11`, `ATS-CRM-DQ-001-20`, `TST-CRM-DQ-176`, `TST-CRM-DQ-180`	policy	service	test services-crm:modules/duplicates/tests/integration/services/detection/duplicate-change-events.integration.spec.js#wrote no key, a job title changing no key and no evidence
`03`must prove Confirmed	Comparison requires a shared key: contacts use e-mail, phone, phonetic name, phonetic surname with postal code, surname stem with birth date or origin; organizations use normalized name, phonetic name with postal code, non-free-mail domain, phone or origin. origin: services-crm:docs/merging/README.md#Status (architecture and plan approved by Andrew 2026-10-11) proved by `ATS-CRM-DQ-001-03`, `ATS-CRM-DQ-001-20`	calculation	service	test services-crm:modules/duplicates/tests/unit/services/matching/match-keys.spec.js#names the six contact key types of §3
`04`must prove Confirmed	Lookup skips a workspace key value with more than 200 holders, still uses rarer keys and adds a zero-point reason identifying the skipped key type. proved by `ATS-CRM-DQ-001-04`, `ATS-CRM-DQ-001-20`, `TST-CRM-DQ-180`	calculation	service	test services-crm:modules/duplicates/tests/integration/services/detection/duplicate-detection.integration.spec.js#still found the two records a rarer key joins, which is the telephone number only they hold
`05`must prove Confirmed	When a key becomes too common, detection removes open detected pairs supported only by it, refreshes pairs with independent evidence and preserves dismissed, confirmed and manual pairs. origin: D-47 (cited in services-crm:docs/merging/duplicate-detection.md#3. Match keys; no milestone record holds it, FND-CRM-DQ-018) proved by `ATS-CRM-DQ-001-05`, `ATS-CRM-DQ-001-20`, `TST-CRM-DQ-180`	lifecycle	service	test services-crm:modules/duplicates/tests/integration/services/detection/duplicate-detection.integration.spec.js#retired the pair the number alone joined, its only evidence being disqualified
`06`must prove Confirmed	Within a detection batch, all records receive their keys before comparison, allowing records created together to match each other. proved by `ATS-CRM-DQ-001-06`, `ATS-CRM-DQ-001-12`, `ATS-CRM-DQ-001-20`, `TST-CRM-DQ-180`	calculation	service	test services-crm:modules/duplicates/tests/integration/services/detection/duplicate-detection.integration.spec.js#found the two records of the batch in each other, which no lookup outside the batch could have done
`07`must prove Confirmed	An open detected pair disappears when it loses every shared key or scores below 35; dismissed, confirmed, manual and portal-sign-up pairs remain stored. proved by `ATS-CRM-DQ-001-07`, `ATS-CRM-DQ-001-11`, `ATS-CRM-DQ-001-14`, `ATS-CRM-DQ-001-20`, `TST-CRM-DQ-176`, `TST-CRM-DQ-180`	lifecycle	service	test services-crm:modules/duplicates/tests/integration/services/persistence/duplicate-candidates.integration.spec.js#removes exactly the pairs whose other record no longer scores
`08`must prove Confirmed	Rescoring updates an existing pair's score, band, evidence and evaluation time without replacing its identity, status, origin, dismissal or first detection time. proved by `ATS-CRM-DQ-001-08`, `ATS-CRM-DQ-001-13`, `ATS-CRM-DQ-001-20`, `TST-CRM-DQ-180`	invariant	service, database	test services-crm:modules/duplicates/tests/integration/services/persistence/duplicate-candidates.integration.spec.js#keeps the moment the pair was first detected, which no re-score owns
`09`must prove Confirmed	Contact matching excludes addresses and phones linked from an organization, so shared employer details do not establish a duplicate pair. proved by `ATS-CRM-DQ-001-09`, `ATS-CRM-DQ-001-20`, `TST-CRM-DQ-180`	calculation	service	test services-crm:modules/duplicates/tests/integration/services/detection/duplicate-detection.integration.spec.js#pairs the two people on nothing their organization supplies
`10`must prove Confirmed	A failed detection pass returns failures for every event in its batch, allowing the entire batch to be delivered again. proved by `ATS-CRM-DQ-001-01`, `ATS-CRM-DQ-001-20`	lifecycle	service	test services-crm:modules/duplicates/tests/integration/services/detection/duplicate-change-events.integration.spec.js#fails every event of the group, so the queue delivers the whole group again
`11` Confirmed	Import-marked changes are reserved for a later completion pass, but current captured events lack that marker and receive ordinary detection.	policy	service	test services-crm:modules/duplicates/tests/unit/services/detection/duplicate-change-events.spec.js#answers no job id for the events the capture writes today, so every one of them is processed
`12`must prove Confirmed	A detection pass reads and creates pairs only within the workspace of its execution. proved by `ATS-CRM-DQ-001-10`, `ATS-CRM-DQ-001-20`, `TST-CRM-DQ-180`	invariant	service, database	test services-crm:modules/duplicates/tests/integration/services/detection/duplicate-detection.integration.spec.js#answers a pass over a record of ANOTHER tenant with nothing, the tenant being the execution’s

Scenarios

Id	Given	When	Then	Provenance
`01`	Two records for the same person arrive together.	The background handler processes their change events.	One pair records their shared names, birth date and address.	test services-crm:modules/duplicates/tests/integration/services/detection/duplicate-change-events.integration.spec.js#found the two records in each other, which only a batch whose keys are written first can do
`02`	A previously rare number supports two pairs, one of which also shares an e-mail address.	Another holder pushes the number above the frequency ceiling.	The phone-only pair disappears and the e-mail-supported pair remains.	test services-crm:modules/duplicates/tests/integration/services/detection/duplicate-detection.integration.spec.js#kept the pair that also shares an e-mail address, by its id

Edge Cases

Id	Given	When	Then	Provenance
`03`	Two employees link their work phone and address from their employer.	Detection evaluates both contacts.	Those linked details supply neither keys nor duplicate evidence.	test services-crm:modules/duplicates/tests/integration/services/detection/duplicate-detection.integration.spec.js#keys neither the linked number nor the linked address’s postal code, which are the organization’s

Planning

Implementation

Detect duplicate records after changes ​

Purpose ​

How It Works ​

Relationships ​