Rhythm 2.0 Specifications

Appearance

CAP-PFM-DJE-012Import records using their producer identities

Markdown

CAP-PFM-DJE-012

Import records using their producer identities

Markdown
  • DocumentDraft
  • Product intentConfirmed
  • ImplementationPartially observed

Planning

Feature
FEAT-PFM-DJE-04
Exercised by
FAS-PFM-DJE-04-01, FAS-PFM-DJE-04-02, FAS-PFM-DJE-04-03, FAS-PFM-DJE-04-04, FAS-PFM-DJE-04-05
Key concepts
CON-PFM-DJE-004, CON-PFM-DJE-001
Actors
ACT-PFM-036, ACT-PFM-035, ACT-PFM-037
Journeys
JNY-PFM-DJE-001, JNY-PFM-DJE-004, JNY-PFM-DJE-005
Related
CAP-PFM-DJE-006, CAP-PFM-DJE-013, CAP-PFM-DJE-014, CAP-PFM-DJE-015
Open questions
OQ-PFM-DJE-005
Proved by
ATS-PFM-DJE-012

Implementation

Executable suites
TST-PFM-DJE-029, TST-PFM-DJE-042, TST-PFM-DJE-046, TST-PFM-DJE-047, TST-PFM-DJE-049, TST-PFM-DJE-069, TST-PFM-DJE-122, TST-PFM-DJE-132

KindCapabilityProduct areaPlatformModuleData Job EngineRevision1

Read atmacroservice-libraries 54536a44services-crm a8d1364freact-console baf6dae3

Import records using their producer identities ​

Purpose ​

Staff and integrations deliver keyed input that can be reconciled, corrected and completed when its dependencies arrive.

How It Works ​

A producer names each input record with its own key and supplies the values for the selected import operation. References can name other producer keys. Rhythm stores the link from each successful key to its destination record. Insert, upsert and patch have distinct repeat-delivery behavior. Owned details are written with their record so an invalid detail does not leave partial creation behind. Missing required references produce blocked outcomes. Staff can retry those outcomes after loading the dependencies. Initial imports use the loading workspace context and declared suppression choices. File preparation takes place before record writes. The automatic-role option remains a recorded product requirement, while duplicate-detection behavior still needs Product’s decision.

Relationships ​

  • Conflict protection governs later staff edits and revision order.
  • Spreadsheet mapping prepares typed input for file processing.
  • Producer-key lookup provides reconciliation after import.

The Formal Contract

The narrative above explains; the sections below are authoritative — the rules, scenarios and tests this document binds.

Outcomes
Producer identities support repeatable import behavior.Records and their owned details commit as one unit.Missing references produce recoverable blocked outcomes.
Surfaces
consoleapijob
Intent sources
services-platform:docs/milestones/PLAT-M19-data-job-engine-foundations.md#D1services-platform:docs/milestones/PLAT-M19-data-job-engine-foundations.md#D5services-platform:docs/milestones/plans/PLAT-M19-plan.md#S12services-platform:docs/milestones/plans/PLAT-M19-plan.md#S13services-platform:docs/milestones/plans/PLAT-M19-plan.md#S14services-crm:docs/milestones/CRM-M7-data-operations.md#D-38services-crm:docs/milestones/CRM-M7-data-operations.md#E-11services-crm:docs/milestones/CRM-M7-data-operations.md#M2-D13

Controls

43 controls apply, from the corpus control catalogue.

TC-AVL-001 TC-AVL-004 TC-AZ-001 TC-AZ-002 TC-AZ-003 TC-AZ-004 TC-CHG-001 TC-CHG-002 TC-CHG-003 TC-CHG-004 TC-CHG-005 TC-CHG-006 TC-CHG-007 TC-CRY-001 TC-CRY-002 TC-CRY-003 TC-CRY-004 TC-DAT-001 TC-DAT-002 TC-DAT-003 TC-DAT-004 TC-IAM-001 TC-IAM-002 TC-IAM-004 TC-IAM-005 TC-LOG-004 TC-LOG-005 TC-MON-001 TC-PI-001 TC-PI-002 TC-PI-003 TC-PI-004 TC-PI-005 TC-PRV-001 TC-PRV-002 TC-PRV-004 TC-TEN-001 TC-TEN-002 TC-TEN-003 TC-TEN-004 TC-VEN-001 TC-VEN-002 TC-VEN-003

Permissions

KeyChecked atScope
crm.contacts.bulkapitenant
crm.contacts.updateapitenant
crm.organizations.bulkapitenant
crm.organizations.updateapitenant
crm.activities.bulkapitenant
crm.regions.bulkapitenant
crm.activity-types.bulkapitenant

Business Rules

13 rules, 10 to be proved before approval. A rule's product intent is derived from its origin: 13 confirmed by a product decision, the rest inheriting this capability's confirmed.

IdRuleKindEnforced atProvenance
01must prove
Confirmed
Insert creates an unmapped key and skips a mapped key; upsert creates or replaces a whole record; update patches supplied fields and skips satisfied contact changes.
origin: services-platform:docs/milestones/plans/PLAT-M19-plan.md#S13
proved by ATS-PFM-DJE-006-10, ATS-PFM-DJE-012-01, ATS-PFM-DJE-012-13, ATS-PFM-DJE-012-14, ATS-PFM-DJE-013-06, TST-PFM-DJE-042, TST-PFM-DJE-132
policyservicetest
services-crm:modules/data-jobs/tests/integration/services/operations/record-imports.integration.spec.js#skips every record of the same file delivered a second time, rather than making a second organization
02must prove
Confirmed
Keyed inserts skip existing mappings; upserts skip lower revisions as stale and equal revisions as duplicates, applying higher or unspecified revisions; patches use their separate revision rules.
origin: services-platform:docs/milestones/plans/PLAT-M19-plan.md#S13
proved by ATS-PFM-DJE-012-02, ATS-PFM-DJE-012-15, ATS-PFM-DJE-014-11, TST-PFM-DJE-029
policyservicetest
macroservice-libraries:packages/data-jobs/src/tests/integration/keyed-data-jobs.integration.spec.ts#skips a delivery a newer one has overtaken, so a late file never undoes the current record
03must prove
Confirmed
A successful record write and its producer mapping commit together; a refused creation leaves no mapping and corrected input can create it.
proved by ATS-PFM-DJE-003-14, ATS-PFM-DJE-006-10, ATS-PFM-DJE-012-03, ATS-PFM-DJE-012-11, ATS-PFM-DJE-012-13, ATS-PFM-DJE-012-15, ATS-PFM-DJE-012-16, ATS-PFM-DJE-014-11, ATS-PFM-DJE-015-06, TST-PFM-DJE-029
invariantservice, databasetest
macroservice-libraries:packages/data-jobs/src/tests/integration/keyed-data-jobs.integration.spec.ts#writes no crosswalk row for an item it refused, so the corrected delivery of that key is a create
04must prove
Confirmed
An unresolved required producer reference blocks its input record, allowing retry after the dependency is loaded without uploading the original file again.
origin: services-platform:docs/milestones/plans/PLAT-M19-plan.md#S12
proved by ATS-PFM-DJE-004-10, ATS-PFM-DJE-012-04, ATS-PFM-DJE-012-12, ATS-PFM-DJE-012-17, ATS-PFM-DJE-016-15, TST-PFM-DJE-029, TST-PFM-DJE-046, TST-PFM-DJE-047
policyservicetest
macroservice-libraries:packages/data-jobs/src/tests/integration/keyed-data-jobs.integration.spec.ts#blocks the child, and a retry after the parent is loaded applies it and finishes the job
05must prove
Confirmed
A record and its owned details commit together; an invalid owned address prevents both record creation and mapping.
proved by ATS-PFM-DJE-003-14, ATS-PFM-DJE-012-05, ATS-PFM-DJE-012-11, ATS-PFM-DJE-012-14, ATS-PFM-DJE-013-06, TST-PFM-DJE-132
invariantservice, databasetest
services-crm:modules/data-jobs/tests/integration/services/operations/record-imports.integration.spec.js#writes nothing at all of the rejected organization: not the row, not its addresses, not its mapping
06must prove
Confirmed
Imported contacts and organizations receive workspace-assigned numbers, ignoring supplied numbers and defaulting external identity to the producer key; configuration items retain their supplied identifiers.
proved by ATS-PFM-DJE-012-06, TST-PFM-DJE-049, TST-PFM-DJE-132
policyservicetest
services-crm:modules/data-jobs/tests/integration/services/operations/record-imports.integration.spec.js#keeps each producer key as its organization’s external id, the record sending none of its own
07must prove
Confirmed
Initial import requires a sandbox workspace in loading state, suppresses only declared side effects and preserves validation and derived values while allowing archived options as historical input.
origin: services-platform:docs/milestones/PLAT-M19-data-job-engine-foundations.md#D5
proved by ATS-PFM-DJE-012-07, ATS-PFM-DJE-012-17, ATS-PFM-DJE-016-15, TST-PFM-DJE-122
policyservicetest
services-crm:modules/data-jobs/tests/integration/services/jobs/initial-import-guarantees.integration.spec.js#keeps the archived option an initial import carries, as history
08must prove
Confirmed
Hard deletion removes associated producer mappings so later input can create the record again; merge redirects mappings to the survivor.
proved by ATS-PFM-DJE-012-08, TST-PFM-DJE-132
policyservice, databasetest
services-crm:modules/data-jobs/tests/integration/services/operations/record-imports.integration.spec.js#takes the keys of the deleted records with them
09must prove
Confirmed
Record files are prepared before the write transaction; an unsavable file rejects its record while storage outages receive transient handling.
proved by ATS-PFM-DJE-012-09, TST-PFM-DJE-069, TST-PFM-DJE-122
policyservicetest
services-crm:modules/data-jobs/tests/integration/services/jobs/initial-import-guarantees.integration.spec.js#applies every organization whose logo could be saved, and rejects the two that could not be written
10must prove
Confirmed
The product should offer an automatic-role import option that defaults off; the recorded decision is not implemented in the scoped request contracts.
origin: services-crm:docs/milestones/CRM-M7-data-operations.md#M2-D13
proved by ATS-PFM-DJE-012-10
policydocumented
services-crm:docs/milestones/CRM-M7-data-operations.md#M2-D13
11
Confirmed
A correcting file job can identify the earlier job and expose that relationship in both job details.
policyapidocumented
services-crm:docs/data-jobs.md#The bulk lane
12
Confirmed
Imports resolve identity by producer key; a new key can create another person without raising a duplicate candidate.
policyservicedocumented
services-crm:docs/data-jobs.md#What is not built
13
Confirmed
Initial activity import preserves historical creation, change and owner information and marks migration; other imported records use the current submitter’s stamps.
policyservicedocumented
services-crm:docs/data-jobs.md#Historical import

Scenarios

IdGivenWhenThenProvenance
01A producer supplies an organization file through insert, repeated insert, upsert and update jobs.The engine processes each operation.The passes respectively create, skip mapped identities, replace full records and patch supplied values.test
react-console:acceptance/crm/playwright/journeys/data-operations/CRM-ACC-0643.spec.ts#CRM-ACC-0643
02A file places branches before their head office.The administrator loads the file and then retries blocked input.Branches initially wait for the head office, then attach after it arrives.test
react-console:acceptance/crm/playwright/journeys/data-operations/CRM-ACC-0647.spec.ts#CRM-ACC-0647

The Markdown files under specs/ in rhythmsoftware/rhythm-v2-docs are the canonical source of truth. This reader is a generated, read-only projection of them.

Rhythm Software