--- id: TST-PFM-DJE-132 title: Record imports kind: integration product_area: platform module: data-job-engine revision: 1 status: draft source_sha: services-platform: ad3f6a6582843008ed0f5a17b131878d15829af1 macroservice-libraries: 54536a441935c6e09fdb7f96f0bec71c52ba4e03 services-crm: a8d1364f1dd44b723bb5f9acba12f983e9ec8d69 services-membership: 5994c270e8a37a30f848e99ba5b5ee6b87c08893 react-console: baf6dae382d6f41dce976c7f848667ccc37e076f repository: services-crm files: - modules/data-jobs/tests/integration/services/operations/record-imports.integration.spec.js implements: - ATS-PFM-DJE-012-01 - ATS-PFM-DJE-012-05 - ATS-PFM-DJE-012-06 - ATS-PFM-DJE-012-08 environment: local implementation_state: implemented latest_result: not-run latest_run_at: null latest_run_ref: null conformance: not-verified estimated_minutes: 5 isolation: >- Suite-owned fixture records, job identifiers and workspace contexts; teardown removes the fixture data. evidence: - >- services-crm:modules/data-jobs/tests/integration/services/operations/record-imports.integration.spec.js assertion_inventory: - 'an import of organizations: the first delivery and the second' - creates one organization per record of the file - 'writes the organization’s own collections with it, so one item is the whole aggregate' - 'remembers each producer key as the record it made, which is what a later delivery is matched by' - 'keeps each producer key as its organization’s external id, the record sending none of its own' - >- skips every record of the same file delivered a second time, rather than making a second organization - 'leaves one organization per key after both deliveries, which is what the skips were for' - >- writes no mapping of this producer in another tenant, because a crosswalk row is the tenant’s own - 'answers nothing for the producer’s key in another tenant, so no job can reach across' - 'an import of a contact and an organization: a record deleted by hand, delivered again' - remembered both keys after the first delivery (premise) - takes the keys of the deleted records with them - >- creates the contact again from its key delivered after the delete, rather than skipping it as a duplicate - >- creates the organization again from its key delivered after the delete, rather than skipping it as a duplicate - 'points each key at the record the second delivery made, which the tenant has' - 'an import of organizations: an upsert, and a record the database refuses' - changes the organization the key already names rather than making a second one - writes the upserted values to the record the key was loaded as - >- rejects the organization naming a type the tenant does not have and the one with two addresses of a type, and applies the two beside them - 'names the field the organization’s own rule refused, which the rule names only in its sentence' - names the rule that refused it and no value of the record - >- names the address and the phone number the tenant has no type for, each at its field, among the organization’s own - 'says of that address what was wrong, and not the type it named' - >- writes nothing at all of the rejected organization: not the row, not its addresses, not its mapping - 'an import of contacts: an address naming a type the tenant does not have' - 'rejects the contact whose address names the type, and applies the one beside it' - 'names that address at its field, as an invalid value' - 'writes no contact for it, so the crosswalk knows only the one applied' - >- an import of contacts: the rules of CRM-M1 (organizations-only types, field limits, unknown countries) - applies the good contact and rejects the three that break a rule - names the organizations-only address type at its field (M1-D8) - names the unknown country at its field (M1-D3) - names the overlong first name at its field (M1-D3) - 'writes no contact for the three, so the crosswalk knows only the one applied' - 'an import of organizations: a head office delivered in one group with the branches that name it' - 'rejects and blocks none of the six, the branches having been listed before their head office' - applies all six in one pass - attaches every branch to the head office it named - still blocks a branch whose head office is outside its group and nothing has loaded - 'applies that branch on retry once its head office has been loaded, attached to it' - 'an import of contacts: a reference the file names before it is loaded' - 'blocks the contact whose organization nothing has loaded, rather than rejecting it' - >- offers Retry on the finished job that holds the blocked row, to the holders of the keys its route requires (D-39) - 'writes no record for the blocked contact, so a job that waits has applied nothing' - 'says which key it was waiting for, by the property the file named it with' - writes no contact while it is blocked - applies the contact once the organization it named has been loaded and the job is retried - affiliates the contact with the organization the key named - >- makes that affiliation the contact’s primary organization, which is what the file said with is_primary - 'an import of contacts: the aggregate, the number, and the spouses of a second pass' - 'writes the whole person in one item: the contact, its e-mail, its phone number and its address' - 'keeps the identity the contact had in the system it came from, in the column that exists for it' - keeps the producer’s key as the external id of the contact that sent none of its own - >- gives the contact this tenant’s next record number, which is the one thing an import cannot carry over - >- keeps the id the producer asked for, because a contact’s id is the client’s to choose before the create - >- generates an id for the contact that asked for none, so a file may supply some ids and not others - 'names the record the producer’s key became in the crosswalk, whichever way the id was decided' - >- applies the first half of the second pass and finds the second already applied by it, the spouse link being reciprocal - 'marries each contact to the one the other key names, which is the cycle one pass could not write' - 'an import of configuration: the types a tenant’s records are described by' - creates one configuration record per key of the file - 'keeps the textual id the producer chose, so every record that names it reads the same' - changes the record the key already names when the same key is delivered again - >- an import job: what the job row remembers of it, and which tenant may say it is an initial import - records the producer the job’s keys belong to - 'takes an ordinary import to be operational, so nothing of it is held back' - >- refuses an initial import of a tenant that is live, because a live tenant’s notices are nobody’s to hold back - accepts it for a tenant being loaded - >- records on the job the one effect an imported contact holds back, which is what the reaction reads - 'freezes one item per record of the file, each with the producer’s key' - 'names the mapping’s job, so the row says which delivery wrote it' - >- an import of records holding an archived option: history under an initial import, refused otherwise - applies both organizations of the initial import - keeps the archived option on the organization the initial import created - applies the initial import’s upsert that moves a loaded organization to the archived option - keeps the archived option on the organization the upsert updated - applies the contact of the initial import - keeps the archived option on the contact the initial import created - 'rejects the organization of an ordinary import, naming the archived option' - 'rejects the contact of an ordinary import, naming the archived option' - maps only the records that were written - 'an import of contacts: a re-delivery by key after a staff member edited the contact' - >- refuses the keyed update as a version conflict, because somebody edited the contact since the producer wrote it - 'says who changed the contact since and when, and no value' - counts the keyed update a conflict and nothing applied - keeps the staff member’s edit through the refused keyed update - >- leaves the crosswalk at the revision and version the producer last wrote, a conflict having written nothing - 'applies the keyed update under the overwrite policy, the producer winning' - advances the crosswalk to the revision and version the overwriting update wrote - >- refuses the keyed upsert as a version conflict under the default policy, keeping the staff member’s edit - 'an import of contacts: a patch delivered by key after a newer revision was applied (D-38)' - 'skips the late patch as stale, writing nothing' - 'leaves the contact as the newer revision wrote it, at the version it had' - keeps the crosswalk at revision 3 through the stale patch - >- applies a patch of a newer revision, and advances the crosswalk to that revision and the version it wrote - 'blocks a patch of a key nothing has loaded, rather than creating or ignoring it' registration_note: Registration describes existing source; the suite is not executed by this documentation job. --- # Record imports ## Verification Boundary The suite checks the production service or shared engine against its database fixture. The structured assertion inventory records the checks present at the pinned revision. ## What This Suite Proves The existing checks exercise record imports. The specification bindings identify the narrower outcomes supported by those assertions. ## What It Does Not Prove Local execution does not establish deployed queue, storage or service behavior. Independent test cases do not establish a complete cross-feature process. ## Real Dependencies and Test Doubles The owning service exercises production persistence and domain behavior against real Postgres. The fixture supplies caller contexts; queue delivery, storage and external calls are doubled where the file installs a test port. ## Must Not Be Doubled The database effects, security context and production operation under examination remain real. Their outcomes cannot be replaced with canned success responses.