--- id: DOM-CRM-DQ-001 title: Possible Duplicate realises: CON-CRM-DQ-001 product_area: crm module: data-quality revision: 1 status: draft design_intent: proposed implementation_evidence: observed source_sha: services-crm: a8d1364f1dd44b723bb5f9acba12f983e9ec8d69 react-console: baf6dae382d6f41dce976c7f848667ccc37e076f owned_by: SVC-CRM-DQ-duplicates lifecycle: STATE-CRM-DQ-001 custom_fields: none readable_by: - SVC-CRM-DQ-merges - SVC-CRM-ENT-entities identity: >- The workspace and record type scope a canonically ordered pair with a service-assigned identity. Groups retain the prior identity contributing the most records; rules have configuration identities, exceptions are unique per created record, and each re-check has its own run identity. attributes: - name: record type kind: enum required: true computed: false notes: >- Only contact and organization comparisons are supported; each uses its own tables. - name: the two records kind: reference required: true computed: false notes: >- The pair names two different records in byte order. Removing either record cascades its pairs. - name: score kind: number required: true computed: true notes: >- Detection stores whole-number scores from thirty-five to one hundred. Confirmation initially assigns one hundred, but later rescoring can replace it. - name: confidence band kind: enum required: true computed: true notes: 'The stored score determines low, medium, high or very high confidence.' - name: reasons kind: collection required: true computed: true notes: >- The row keeps scored evidence and applied ceilings. Readers render and mask it; manual confirmation initially writes a reviewer reason that later detection can replace. - name: scoring version kind: number required: true computed: true notes: The version identifies the matching keys and scoring weights used. - name: evidence fingerprint kind: text required: false computed: true notes: >- A computed digest records strong evidence and supports comparison with the evidence held at dismissal. - name: review standing kind: enum required: true computed: false notes: >- Open, dismissed and confirmed are stored standings. Merging removes pairs through record deletion rather than storing a merged standing. - name: origin kind: enum required: true computed: false notes: >- Detected, manual and portal registration identify how the pair originated; rescoring preserves that value. - name: dismissal kind: collection required: false computed: false notes: >- Dismissal keeps its author, instant, evidence and optional note of at most one thousand characters. Restoration clears these fields. - name: group kind: reference required: false computed: true notes: >- Grouping follows connected pairs of one standing; confirmed pairs are excluded. - name: first detected and last evaluated kind: datetime required: true computed: true notes: >- Rescoring preserves the first detection instant and advances the last evaluation instant. - name: match keys kind: collection required: true computed: true notes: >- Detection replaces each record’s normalized comparison keys, including contact communication and name signals or organization name, domain, phone and origin signals. - name: group · counts and strength kind: computed required: true computed: true notes: >- Each stored component carries its counts, strongest score and band, detection instants and an indication that traversal stopped at the record limit. - name: duplicate rule kind: collection required: true computed: false notes: >- A rule carries its label, record type, evidence pattern, response, confidence floor, unique position, activity and portal switches, and version. - name: creation exception kind: collection required: false computed: false notes: >- An exception records the authorized creator, instant, explanation, rules bypassed and matched records. - name: re-check run kind: collection required: false computed: false notes: >- A run retains initial totals, current progress, scoring version, outcome, failure and timing; batch rows store inclusive record windows. relationships: - to: DOM-CRM-ENT-001 cardinality: >- A contact pair references two contacts; each contact can participate in many pairs. ownership: >- Entities owns contacts. Data Quality owns pair and key links that cascade on deletion and group state repaired by regrouping. cross_service: false - to: DOM-CRM-ENT-002 cardinality: >- An organization pair references two organizations; each organization can participate in many pairs. ownership: >- Entities owns organizations. Data Quality owns their comparisons and repairs grouping when they disappear. cross_service: false - to: DOM-CRM-DQ-003 cardinality: >- A group or record selection can start multiple plans without a persisted association. ownership: >- Plans retain selected record identities without storing the originating review group. cross_service: false - to: DOM-CRM-DQ-004 cardinality: A merge can move outside dismissals and remove other participant pairs. ownership: >- Duplicate persistence owns pair updates even when called inside a merge transaction. cross_service: false invariants: - id: INV-DOM-CRM-DQ-001-01 statement: >- Canonical record ordering and pair uniqueness permit only one row for each comparison, with a band consistent with its score. enforced_at: - database provenance: observed source: >- services-crm:modules/duplicates/db/duplicate-candidates.table.ts:candidateConstraints - id: INV-DOM-CRM-DQ-001-02 statement: >- Stored scores start at thirty-five; confidence floors are thirty-five, fifty, seventy and ninety. enforced_at: - service - database provenance: test source: >- services-crm:modules/duplicates/tests/unit/services/matching/bands.spec.js#gives 34 no band at all, so the pair is not stored - id: INV-DOM-CRM-DQ-001-03 statement: >- Dismissed standing requires both a dismissal instant and author; restoration removes both. enforced_at: - database provenance: observed source: >- services-crm:modules/duplicates/db/duplicate-candidates.table.ts:candidateConstraints - id: INV-DOM-CRM-DQ-001-04 statement: >- Rescoring retains pair identity, origin and first detection, and preserves standing and dismissal unless a separate reopening changes them. enforced_at: - service provenance: observed source: >- services-crm:modules/duplicates/services/persistence/duplicate-candidates.ts:upsertDuplicateCandidates - id: INV-DOM-CRM-DQ-001-05 statement: >- Detection removes only open detected pairs that no longer qualify, leaving reviewed and non-detected origins stored. enforced_at: - service provenance: test source: >- services-crm:modules/duplicates/tests/integration/services/persistence/duplicate-candidates.integration.spec.js#removes exactly the pairs whose other record no longer scores - id: INV-DOM-CRM-DQ-001-06 statement: >- Database record links prevent pairs and keys from surviving record deletion; regrouping repairs the remaining membership. enforced_at: - database provenance: test source: >- services-crm:modules/merges/tests/integration/services/merges/duplicate-groups-after-merge.integration.spec.js#leaves the group holding the two records that are left, counted as it now stands - id: INV-DOM-CRM-DQ-001-07 statement: >- Pair changes and their affected group recomputation share the same transaction. enforced_at: - service - database provenance: test source: >- services-crm:modules/duplicates/tests/integration/services/detection/duplicate-detection.integration.spec.js#grouped every pair it stored, the groups of a batch being written in the pass’s own transaction (§14) - id: INV-DOM-CRM-DQ-001-08 statement: >- Groups contain two to ten records connected at one standing, excluding confirmed pairs. enforced_at: - service - database provenance: test source: >- services-crm:modules/duplicates/tests/unit/services/groups/group-components.spec.js#says the group it cut short is bounded, so a reader knows there may be more of it - id: INV-DOM-CRM-DQ-001-09 statement: 'Draft matching does not create records, keys or pairs.' enforced_at: - service provenance: test source: >- services-crm:modules/duplicates/tests/e2e/duplicates.e2e.spec.js#writes nothing: the draft is not a record, so the queue is as it was - id: INV-DOM-CRM-DQ-001-10 statement: >- An authorized creation exception is written atomically with its new record, is unique for that record and cannot be updated by the application. enforced_at: - service - database provenance: test source: >- services-crm:modules/duplicates/tests/integration/services/persistence/duplicate-block-overrides.integration.spec.js#leaves no contact behind, the record and its audit sharing one transaction - id: INV-DOM-CRM-DQ-001-11 statement: >- Rule replacement requires the current version, a nonblank trimmed label and a unique integer position between one and one hundred. enforced_at: - api - service - database provenance: test source: >- services-crm:modules/duplicates/tests/integration/services/persistence/duplicate-rules.integration.spec.js#refuses the place another rule already holds - id: INV-DOM-CRM-DQ-001-12 statement: >- Guarded run creation permits only one queued or running re-check for each workspace record type. enforced_at: - service provenance: test source: >- services-crm:modules/duplicates/tests/integration/services/persistence/duplicate-rescans.integration.spec.js#refuses a second rescan of that record type, so two passes never rescore the same records at once - id: INV-DOM-CRM-DQ-001-13 statement: Conditional batch completion adds that batch to progress at most once. enforced_at: - service - database provenance: test source: >- services-crm:modules/duplicates/tests/integration/services/queue/rescan-queue.integration.spec.js#counts the batch once, so a redelivery never moves the progress past the end - id: INV-DOM-CRM-DQ-001-14 statement: >- Comparison operates within the execution workspace for both reads and writes. enforced_at: - service - database provenance: test source: >- services-crm:modules/duplicates/tests/integration/services/detection/duplicate-detection.integration.spec.js#answers a pass over a record of ANOTHER tenant with nothing, the tenant being the execution’s - id: INV-DOM-CRM-DQ-001-15 statement: >- No duplicate-processing record declares custom-field capability or custom-value storage. enforced_at: - service provenance: inferred source: >- services-crm:modules/duplicates (no metadata object registration; db/*.table.ts declares no custom-value table) physical: - contact_match_keys - organization_match_keys - contact_duplicate_candidates - organization_duplicate_candidates - contact_duplicate_groups - organization_duplicate_groups - duplicate_rules - duplicate_block_overrides - duplicate_rescans - duplicate_rescan_batches evidence: - >- services-crm at a8d1364f: modules/duplicates/db/*.table.ts, duplicate-bands.ts; services/persistence/*.ts; services/detection/*.ts; services/groups/group-components.ts; services/candidates/*.ts; docs/merging/duplicate-detection.md, architecture.md §C.1 to §C.3 and §C.10. - >- rhythm-v2-docs: specs/crm/data-quality/planning/concepts/CON-CRM-DQ-001 (facets) and the rules of CAP-CRM-DQ-001 to CAP-CRM-DQ-011 cited as invariant sources. --- # Possible Duplicate ## What It Models A possible duplicate holds evidence that two records may represent the same identity, together with the team’s current decision. Comparison keys, review groups, rules, creation exceptions and re-checks are facets of this model rather than additional business-record owners. ## Identity and Attributes Pair identity survives rescoring. Scores and reasons can change independently of a retained review decision, including a manual confirmation. Groups keep counted, bounded components and select surviving identities by contribution when membership changes. ## Relationships Entities owns the records being compared. A selected group can seed a merge plan without creating a stored link, and merging calls duplicate persistence to carry outside dismissals before absorbed records disappear. ## Invariants The structured invariants distinguish database shape constraints from service operations. Keeping pair decisions does not freeze their evidence or score. ## How It Is Stored and Shared The duplicates module owns these tables in CRM. Entities consults its creation guard and writes exceptions through duplicate persistence; merging invokes the same owner functions within its transaction. These records have no registered custom-field capability. ## Open Design Questions Attention remains held until group metadata exists. Import completion and the portal configuration flag are not connected to their intended callers, and re-check recovery still has the recorded failure and concurrency gaps.