--- id: SVC-CRM-DQ-duplicates title: Duplicates Service Module product_area: crm module: data-quality repository: services-crm source_module: modules/duplicates revision: 1 status: draft design_intent: proposed implementation_evidence: observed inference: [] source_sha: services-crm: a8d1364f1dd44b723bb5f9acba12f983e9ec8d69 owns: - DOM-CRM-DQ-001 does_not_own: - >- Entities owns the compared records and their ordinary creation and deletion; duplicate services supply the guard and exception persistence. - >- Merges owns plans, attempts and aliases; the duplicate record endpoint uses its absorbed-identity response wrapper. - >- Platform owns event transport, attention delivery, job monitoring and portal account linking. operations: - operation_id: getDuplicatesHealth intent: Report module health and deployed build information. access: public permission: 'none: public health route' realises: [] published_to_submissions: 'not published: reports deployment health and captures no business request.' - operation_id: listDuplicateGroups intent: >- Page review groups by record type, standing and supported filters, in the selected strength, detection or size order. access: console permission: crm.duplicates.read realises: - CAP-CRM-DQ-007 published_to_submissions: >- not published: reads or reports existing review state and accepts no captured business request. - operation_id: getDuplicateGroup intent: >- Read a group’s member summaries, comparison evidence and constituent pairs. access: console permission: crm.duplicates.read realises: - CAP-CRM-DQ-007 - CAP-CRM-DQ-002 published_to_submissions: >- not published: reads or reports existing review state and accepts no captured business request. - operation_id: dismissDuplicateGroup intent: >- Reject the open comparisons of a whole group atomically and return the resulting dismissed component with prior decision attribution. access: console permission: crm.duplicates.review realises: - CAP-CRM-DQ-008 published_to_submissions: >- not published: a staff review decision made in the console about existing records, with no intake process or capture binding. - operation_id: restoreDuplicateGroupDismissal intent: >- Reopen the comparisons attributed to the selected group dismissal and return the resulting component. access: console permission: crm.duplicates.review realises: - CAP-CRM-DQ-008 published_to_submissions: >- not published: a staff review decision made in the console about existing records, with no intake process or capture binding. - operation_id: getDuplicateSummary intent: >- Report bounded duplicate workload totals, latest detection, re-check activity and recent merging for both record types. access: console permission: crm.duplicates.read realises: - CAP-CRM-DQ-011 published_to_submissions: >- not published: reads or reports existing review state and accepts no captured business request. - operation_id: listDuplicateCandidates intent: >- Page pairs by type, standing, confidence floor or participating record, ordered by descending score. access: console permission: crm.duplicates.read realises: - CAP-CRM-DQ-007 published_to_submissions: >- not published: reads or reports existing review state and accepts no captured business request. - operation_id: addManualDuplicateCandidate intent: >- Create or update a human-confirmed pair with full score and its reviewer explanation. access: console permission: crm.duplicates.review realises: - CAP-CRM-DQ-009 published_to_submissions: >- not published: a staff review decision made in the console about existing records, with no intake process or capture binding. - operation_id: getDuplicateCandidate intent: 'Read one comparison’s score, confidence and masked explanation.' access: console permission: crm.duplicates.read realises: - CAP-CRM-DQ-007 - CAP-CRM-DQ-002 published_to_submissions: >- not published: reads or reports existing review state and accepts no captured business request. - operation_id: listRecordDuplicateCandidates intent: >- Read at most one hundred comparisons for a record and include its merge-launch group identities; reject an absorbed identity as merged. access: console permission: crm.duplicates.read realises: - CAP-CRM-DQ-010 published_to_submissions: >- not published: reads or reports existing review state and accepts no captured business request. - operation_id: dismissDuplicateCandidate intent: >- Record a pair rejection and optional note, returning existing attribution if already rejected. access: console permission: crm.duplicates.review realises: - CAP-CRM-DQ-008 published_to_submissions: >- not published: a staff review decision made in the console about existing records, with no intake process or capture binding. - operation_id: restoreDuplicateCandidate intent: Clear a pair’s dismissal and return its current open evidence. access: console permission: crm.duplicates.review realises: - CAP-CRM-DQ-008 published_to_submissions: >- not published: a staff review decision made in the console about existing records, with no intake process or capture binding. - operation_id: checkForDuplicates intent: >- Return up to five draft matches and blocking context without creating any matching state. access: console permission: crm.duplicates.read realises: - CAP-CRM-DQ-004 published_to_submissions: >- not published: a read-only advisory check; any future intake would call it inside its own capture contract rather than as a capture itself. - operation_id: startDuplicateRescan intent: >- Create a type-wide re-check or refuse with the workspace’s existing queued or running run. access: console permission: crm.duplicates.configure realises: - CAP-CRM-DQ-005 published_to_submissions: >- not published: workspace configuration maintained by administrators, not an intake capture. - operation_id: listDuplicateRescans intent: 'List up to fifty recent workspace re-checks, optionally filtered by type.' access: console permission: crm.duplicates.configure realises: - CAP-CRM-DQ-005 published_to_submissions: >- not published: reads or reports existing review state and accepts no captured business request. - operation_id: getDuplicateRescan intent: >- Read the workspace run by identity after validating the path type, without verifying that type matches the run. access: console permission: crm.duplicates.configure realises: - CAP-CRM-DQ-005 published_to_submissions: >- not published: reads or reports existing review state and accepts no captured business request. - operation_id: listDuplicateRules intent: Read the configured rules in evaluation order. access: console permission: crm.duplicates.configure realises: - CAP-CRM-DQ-003 published_to_submissions: >- not published: reads or reports existing review state and accepts no captured business request. - operation_id: updateDuplicateRule intent: >- Replace a current-version rule’s matching settings, action, confidence floor, position and switches. access: console permission: crm.duplicates.configure realises: - CAP-CRM-DQ-003 published_to_submissions: >- not published: workspace configuration maintained by administrators, not an intake capture. events: - name: 'contact change event (stream contact, handler duplicate-detection)' direction: consumed when: Dispatch delivers committed contact changes selected by matching fields. payload_summary: >- Record and workspace context with changed fields, attribution and correlation. - name: >- organization change event (stream organization, handler duplicate-detection) direction: consumed when: >- Dispatch delivers committed organization changes selected by matching fields. payload_summary: >- Record and workspace context with changed fields, attribution and correlation. - name: crm.open_duplicate_groups attention item direction: produced when: >- The attention definition is held, so opening or changing groups currently emits no item. payload_summary: 'The declared payload describes the group, type, counts and confidence.' jobs: - >- Minute-scheduled and nudged re-check dispatch claims eligible windows for queue publication. - >- A re-check invocation receives one message, with four concurrent consumers and three receives before dead-letter holding. - >- The declared daily attention sweep skips publication while metadata registration remains held. integrations: - system: Platform portal sign-up direction: inbound contract: >- Registration calls CRM-owned candidate and create functions through a grant restricted to the Platform application role. - system: Entities contact and organization create direction: inbound contract: >- Record creation uses an in-process preflight block check and writes its authorized exception within the creation transaction. consistency: - rule: Pair decisions and their affected grouping are written together. kind: transactional compensation: Rollback retains both prior comparison and group state. - rule: Detection processes committed changes asynchronously. kind: eventual compensation: >- Queue recovery repeats failed delivery; exhausted work needs redrive or a later pass. - rule: Creation and its authorized exception share a commit. kind: transactional compensation: Exception failure rolls back the new record. - rule: Re-check evidence refreshes in independent detection passes and windows. kind: eventual compensation: >- Committed work remains after failure, and leftover failed-run windows can still execute. - rule: The record-specific duplicate route checks absorbed identities. kind: transactional compensation: It reads aliases committed by merging and reports the destination. evidence: - >- services-crm at a8d1364f: modules/duplicates/aws/api/duplicates.routes.ts (route declarations), duplicates.schemas.ts; aws/duplicates.module.ts; aws/logic/*.lambda.ts; services/**; attention/open-duplicate-groups.attention.ts; aws/crm.service.ts (DuplicatesPermissions); db/migrations/0077_portal_sign_up_contacts.sql. controls: [TC-AVL-004, TC-AZ-001, TC-AZ-002, TC-AZ-003, TC-AZ-004, TC-AZ-005, TC-AZ-006, TC-CRY-004, TC-DAT-001, TC-DAT-002, TC-DAT-003, TC-DAT-004, TC-DAT-005, TC-IAM-001, TC-IAM-002, TC-IAM-004, TC-IAM-005, TC-IAM-006, TC-LOG-001, TC-LOG-002, TC-LOG-003, TC-LOG-004, TC-LOG-005, TC-MON-001, TC-MON-002, TC-PI-001, TC-PI-002, TC-PI-003, TC-PI-004, TC-PI-005, TC-PRV-001, TC-PRV-002, TC-TEN-001, TC-TEN-002, TC-TEN-003] --- # Duplicates Service Module ## What This Service Is For The duplicates module provides matching evidence, staff review and creation-response settings. Its background detector follows record changes, while a dedicated dispatcher and consumer perform requested re-checks. ## Boundary Comparison keys, pairs, groups, rules, exceptions and re-check work belong here. Entities owns contacts and organizations, and merging owns execution; both call duplicate persistence when their transactions need its records. ## Operations in Words Review reads and decisions use separate permissions. Configuration authority controls rules and re-checks, and draft matching needs duplicate-read authority. The structured operations carry exact access rules and capability links. ## Events and Jobs Contact and organization change handlers refresh evidence. Re-check dispatch has its own queue and schedule. The declared group attention item remains held, and CRM-owned database functions provide the portal registration seam. ## Consistency Guarantees Review and grouping commit together; record creation and exceptions do too. Detection and re-check progress are eventual, with committed earlier work retained on later failure. All operations explicitly declare their unpublished Submissions status.