--- id: ATS-PFM-DJE-003 scope: CAP-PFM-DJE-003 product_area: platform module: data-job-engine revision: 2 status: proposed approved_by: null approved_at: null source_sha: macroservice-libraries: 54536a441935c6e09fdb7f96f0bec71c52ba4e03 services-crm: a8d1364f1dd44b723bb5f9acba12f983e9ec8d69 services-platform: ad3f6a6582843008ed0f5a17b131878d15829af1 scenarios: - id: ATS-PFM-DJE-003-01 title: Redelivery does not repeat committed changes level: api proves: - BR-CAP-PFM-DJE-003-01 preconditions: Prepare a queued batch of two contacts. steps: - do: Process it twice and also deliver two copies concurrently. expect: Each contact receives one change and one receipt. provenance: test source: >- services-crm:modules/data-jobs/tests/integration/services/queue/data-job-worker.integration.spec.js#applies nothing the second time expected: Each contact receives one change and one receipt. negative: false data_needs: No additional fixture is needed. bound_to: - TST-PFM-DJE-139 - TST-PFM-DJE-117 provenance: test source: >- services-crm:modules/data-jobs/tests/integration/services/queue/data-job-worker.integration.spec.js#applies nothing the second time category: concurrency binding_evidence: - >- services-crm:modules/data-jobs/tests/integration/services/queue/data-job-worker.integration.spec.js#applies nothing the second time - >- services-crm:modules/data-jobs/tests/integration/services/queue/data-job-worker.integration.spec.js#changes each contact once between them, whichever delivery got there first - >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#applies nothing a second time when the queue delivers a committed batch again - id: ATS-PFM-DJE-003-02 title: Individual refusals preserve successful work level: api proves: - BR-CAP-PFM-DJE-003-02 preconditions: 'Prepare 100 contacts with five inputs that are invalid, missing or locked.' steps: - do: Run the contact update. expect: The job reports 95 applied and five coded refusals; refused records are left intact. provenance: test source: >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#refuses the other five, counting each as what refused it expected: The job reports 95 applied and five coded refusals; refused records are left intact. negative: true data_needs: A prepared collection of contacts. bound_to: - TST-PFM-DJE-117 - TST-PFM-DJE-100 provenance: test source: >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#refuses the other five, counting each as what refused it category: rule binding_evidence: - >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#refuses the other five, counting each as what refused it - >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#finishes the job as completed with rejections, which is what a job with both outcomes is - >- react-console:acceptance/data-jobs/playwright/journeys/foundations/DJE-ACC-0004.spec.ts#DJE-ACC-0004 - id: ATS-PFM-DJE-003-03 title: Scoped access conceals an inaccessible record level: api proves: - BR-CAP-PFM-DJE-003-03 preconditions: Grant the submitter access to one of two contacts. steps: - do: Process an update covering both. expect: The accessible contact changes; the other is reported not found. provenance: test source: >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#rejects that contact as not found, which tells the person nothing about a record they may not see expected: The accessible contact changes; the other is reported not found. negative: true data_needs: Record-scoped grants. bound_to: - TST-PFM-DJE-117 provenance: test source: >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#rejects that contact as not found, which tells the person nothing about a record they may not see category: permission binding_evidence: - >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#rejects that contact as not found, which tells the person nothing about a record they may not see - id: ATS-PFM-DJE-003-04 title: Refusals omit invalid values level: api proves: - BR-CAP-PFM-DJE-003-04 preconditions: Submit a value exceeding its field limit. steps: - do: Inspect the refusal information. expect: The code and affected field are present; the supplied value is absent. provenance: test source: >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#names the rule a refused value broke, and quotes no value of the record expected: The code and affected field are present; the supplied value is absent. negative: true data_needs: No additional fixture is needed. bound_to: - TST-PFM-DJE-117 - TST-PFM-DJE-100 provenance: test source: >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#names the rule a refused value broke, and quotes no value of the record category: invariant binding_evidence: - >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#names the rule a refused value broke, and quotes no value of the record - >- react-console:acceptance/data-jobs/playwright/journeys/foundations/DJE-ACC-0004.spec.ts#DJE-ACC-0004 - id: ATS-PFM-DJE-003-05 title: Commit splitting isolates the failing object level: unit proves: - BR-CAP-PFM-DJE-003-05 preconditions: Prepare 25 objects including one rejected by the database. steps: - do: Execute the commit unit. expect: >- Only the failing object is isolated and refused, within the bounded number of transactions. provenance: test source: >- macroservice-libraries:packages/data-jobs/src/commit-split.spec.ts#halves the unit until the failing object stands alone, and isolates only it expected: 'Only the failing object is isolated and refused, within the bounded number of transactions.' negative: true data_needs: No additional fixture is needed. bound_to: - TST-PFM-DJE-011 provenance: test source: >- macroservice-libraries:packages/data-jobs/src/commit-split.spec.ts#halves the unit until the failing object stands alone, and isolates only it category: failure binding_evidence: - >- macroservice-libraries:packages/data-jobs/src/commit-split.spec.ts#halves the unit until the failing object stands alone, and isolates only it - >- macroservice-libraries:packages/data-jobs/src/commit-split.spec.ts#stays inside the transaction bound for every number of failing objects in a batch of 25 - id: ATS-PFM-DJE-003-06 title: Group refusal leaves every member intact level: api proves: - BR-CAP-PFM-DJE-003-06 preconditions: Prepare three grouped items with one invalid member and separate ungrouped items. steps: - do: Apply the request and separately validate it. expect: >- The group is unwritten, its other members receive group-member refusals, and the ungrouped items proceed; validation reports equivalent refusals. provenance: test source: >- macroservice-libraries:packages/data-jobs/src/tests/integration/group-refusals.integration.spec.ts#writes none of the group, rejects the other two as group_member_refused, and applies the items of no group expected: >- The group is unwritten, its other members receive group-member refusals, and the ungrouped items proceed; validation reports equivalent refusals. negative: true data_needs: Objects supplied by the group fixture. bound_to: - TST-PFM-DJE-027 provenance: test source: >- macroservice-libraries:packages/data-jobs/src/tests/integration/group-refusals.integration.spec.ts#writes none of the group, rejects the other two as group_member_refused, and applies the items of no group category: invariant binding_evidence: - >- macroservice-libraries:packages/data-jobs/src/tests/integration/group-refusals.integration.spec.ts#writes none of the group, rejects the other two as group_member_refused, and applies the items of no group - >- macroservice-libraries:packages/data-jobs/src/tests/integration/group-refusals.integration.spec.ts#reports the same rejections when the job only validates, and writes nothing - id: ATS-PFM-DJE-003-07 title: The rejection guard uses cumulative thresholds level: unit proves: - BR-CAP-PFM-DJE-003-07 preconditions: Supply representative outcome totals. steps: - do: Evaluate early and late guard boundaries. expect: >- Rates strictly above 50% pause at 100–999 attempts; rates strictly above 20% pause from 1,000 attempts. provenance: test source: >- macroservice-libraries:packages/data-jobs/src/rejection-guard.spec.ts#pauses a job whose first hundred items are mostly refused expected: >- Rates strictly above 50% pause at 100–999 attempts; rates strictly above 20% pause from 1,000 attempts. negative: true data_needs: No additional fixture is needed. bound_to: - TST-PFM-DJE-019 provenance: test source: >- macroservice-libraries:packages/data-jobs/src/rejection-guard.spec.ts#pauses a job whose first hundred items are mostly refused category: boundary binding_evidence: - >- macroservice-libraries:packages/data-jobs/src/rejection-guard.spec.ts#pauses a job whose first hundred items are mostly refused - >- macroservice-libraries:packages/data-jobs/src/rejection-guard.spec.ts#leaves the early rate alone once past it, and holds the job to the lower rate instead - >- macroservice-libraries:packages/data-jobs/src/rejection-guard.spec.ts#counts a block and a conflict as refusals, because each says the input was wrong - id: ATS-PFM-DJE-003-08 title: Persistent transient errors and systemic errors settle differently level: api proves: - BR-CAP-PFM-DJE-003-08 preconditions: Inject repeated transient errors and a separate systemic error. steps: - do: 'Process the job, then resume after the transient condition clears.' expect: >- The transient case pauses and can finish after recovery; the systemic case fails with a safe summary. provenance: proposed source: 'services-crm:docs/data-jobs.md#What each contact''s outcome can be' expected: >- The transient case pauses and can finish after recovery; the systemic case fails with a safe summary. negative: true data_needs: Controllable failure injection. bound_to: [] provenance: proposed source: 'services-crm:docs/data-jobs.md#What each contact''s outcome can be' category: failure binding_evidence: [] - id: ATS-PFM-DJE-003-09 title: Revoked access prevents later batches level: api proves: - BR-CAP-PFM-DJE-003-09 preconditions: Remove bulk access from the submitter of a running job. steps: - do: Deliver another batch. expect: 'The job fails with access revoked, retains commits and marks remaining work unattempted.' provenance: test source: >- services-crm:modules/data-jobs/tests/integration/services/queue/data-job-worker.integration.spec.js#fails the whole job with the code, not just the batch expected: 'The job fails with access revoked, retains commits and marks remaining work unattempted.' negative: true data_needs: Mutable permission grants. bound_to: - TST-PFM-DJE-139 - TST-PFM-DJE-062 provenance: test source: >- services-crm:modules/data-jobs/tests/integration/services/queue/data-job-worker.integration.spec.js#fails the whole job with the code, not just the batch category: permission binding_evidence: - >- services-crm:modules/data-jobs/tests/integration/services/queue/data-job-worker.integration.spec.js#fails the whole job with the code, not just the batch - >- services-crm:modules/data-jobs/tests/integration/services/queue/data-job-worker.integration.spec.js#counts the batches that will never run as not attempted, so none is left looking like work in flight - >- react-console:acceptance/crm/playwright/journeys/data-operations/CRM-ACC-0663.spec.ts#CRM-ACC-0663 - id: ATS-PFM-DJE-003-10 title: A repeated contact value produces no extra revision level: api proves: - BR-CAP-PFM-DJE-003-11 preconditions: Prepare a contact already holding the requested value. steps: - do: Process that same change. expect: >- The receipt reports applied and unchanged at the existing version; no new history entry is written. provenance: test source: >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#answers the item applied and unchanged, at the version the contact still stands at expected: >- The receipt reports applied and unchanged at the existing version; no new history entry is written. negative: false data_needs: No additional fixture is needed. bound_to: - TST-PFM-DJE-117 provenance: test source: >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#answers the item applied and unchanged, at the version the contact still stands at category: edge binding_evidence: - >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#answers the item applied and unchanged, at the version the contact still stands at - >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#writes no revision, so the history names only the job that changed the contact - id: ATS-PFM-DJE-003-11 title: Submission commits before processing can account for it category: interaction level: api exercises: - CAP-PFM-DJE-001 - CAP-PFM-DJE-003 proves: - BR-CAP-PFM-DJE-001-10 - BR-CAP-PFM-DJE-003-01 preconditions: Prepare a submission and a transaction failure immediately before commit. steps: - do: >- Refuse the first transaction, inspect job storage, then submit successfully and process the saved batches twice. expect: >- The refused submission leaves no job or queued work; the successful job has one committed outcome per position. provenance: proposed source: >- macroservice-libraries:packages/data-jobs/README.md#A worked example: updating contacts in bulk; services-crm:modules/data-jobs/tests/integration/services/queue/data-job-worker.integration.spec.js#changes each contact once between them, whichever delivery got there first expected: >- The refused submission leaves no job or queued work; the successful job has one committed outcome per position. negative: true data_needs: Prepare a submission and a transaction failure immediately before commit. bound_to: [] provenance: proposed source: >- macroservice-libraries:packages/data-jobs/README.md#A worked example: updating contacts in bulk; services-crm:modules/data-jobs/tests/integration/services/queue/data-job-worker.integration.spec.js#changes each contact once between them, whichever delivery got there first - id: ATS-PFM-DJE-003-12 title: Cancellation accounts for settled and abandoned work category: interaction level: api exercises: - CAP-PFM-DJE-003 - CAP-PFM-DJE-004 proves: - BR-CAP-PFM-DJE-003-01 - BR-CAP-PFM-DJE-004-02 preconditions: Prepare a multi-batch job and hold one batch in flight. steps: - do: >- Cancel while that batch is active, let it settle, and inspect receipts and unattempted positions. expect: >- The active batch may commit once; later batches do not start and the job settles cancelled with unattempted work counted. provenance: proposed source: >- services-crm:modules/data-jobs/tests/integration/services/queue/data-job-worker.integration.spec.js#changes each contact once between them, whichever delivery got there first; services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#ends the job cancelled, not failed, once the batch it waited for has finished expected: >- The active batch may commit once; later batches do not start and the job settles cancelled with unattempted work counted. negative: false data_needs: Prepare a multi-batch job and hold one batch in flight. bound_to: [] provenance: proposed source: >- services-crm:modules/data-jobs/tests/integration/services/queue/data-job-worker.integration.spec.js#changes each contact once between them, whichever delivery got there first; services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#ends the job cancelled, not failed, once the batch it waited for has finished - id: ATS-PFM-DJE-003-13 title: Inspection reconciles partial processing category: interaction level: api exercises: - CAP-PFM-DJE-003 - CAP-PFM-DJE-005 proves: - BR-CAP-PFM-DJE-003-02 - BR-CAP-PFM-DJE-003-04 - BR-CAP-PFM-DJE-005-03 - BR-CAP-PFM-DJE-005-05 preconditions: 'Prepare valid, invalid and inaccessible contacts in one job.' steps: - do: >- Process the job and read all result and refusal pages alongside the changed records' history. expect: >- The pages account for each position; eligible changes have attribution and refusals disclose no record values. provenance: proposed source: >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#refuses the other five, counting each as what refused it; services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#names the rule a refused value broke, and quotes no value of the record; services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#answers the next page from the cursor the last one gave, so no item is read twice; services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#stamps the job on the revision, so every record one job changed can be listed expected: >- The pages account for each position; eligible changes have attribution and refusals disclose no record values. negative: true data_needs: 'Prepare valid, invalid and inaccessible contacts in one job.' bound_to: [] provenance: proposed source: >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#refuses the other five, counting each as what refused it; services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#names the rule a refused value broke, and quotes no value of the record; services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#answers the next page from the cursor the last one gave, so no item is read twice; services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#stamps the job on the revision, so every record one job changed can be listed - id: ATS-PFM-DJE-003-14 title: An invalid owned detail reserves no producer identity category: interaction level: api exercises: - CAP-PFM-DJE-003 - CAP-PFM-DJE-012 proves: - BR-CAP-PFM-DJE-003-02 - BR-CAP-PFM-DJE-012-03 - BR-CAP-PFM-DJE-012-05 preconditions: Prepare a keyed contact with an invalid address and a valid neighboring contact. steps: - do: >- Import both, inspect records and mappings, then submit corrected input under the refused key. expect: >- The valid neighbor commits, the invalid record has no partial detail or mapping, and correction can create it. provenance: proposed source: >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#refuses the other five, counting each as what refused it; macroservice-libraries:packages/data-jobs/src/tests/integration/keyed-data-jobs.integration.spec.ts#writes no crosswalk row for an item it refused, so the corrected delivery of that key is a create; services-crm:modules/data-jobs/tests/integration/services/operations/record-imports.integration.spec.js#writes nothing at all of the rejected organization: not the row, not its addresses, not its mapping expected: >- The valid neighbor commits, the invalid record has no partial detail or mapping, and correction can create it. negative: false data_needs: Prepare a keyed contact with an invalid address and a valid neighboring contact. bound_to: [] provenance: proposed source: >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#refuses the other five, counting each as what refused it; macroservice-libraries:packages/data-jobs/src/tests/integration/keyed-data-jobs.integration.spec.ts#writes no crosswalk row for an item it refused, so the corrected delivery of that key is a create; services-crm:modules/data-jobs/tests/integration/services/operations/record-imports.integration.spec.js#writes nothing at all of the rejected organization: not the row, not its addresses, not its mapping evidence: - >- Reviewed evidence: macroservice-libraries (54536a44): README.md (What the engine is; Groups succeed or fail together); src/run-data-job-batch.ts, commit-split.ts, rejection-guard.ts, data-job-config.ts; tests commit-split.spec.ts, rejection-guard.spec.ts, group-refusals.integration.spec.ts, batch-skips.integration.spec.ts, data-job-engine.integration.spec.ts. - >- Reviewed evidence: services-crm (a8d1364f): docs/data-jobs.md (What each contact's outcome can be; How a job runs); services/queue/data-job-worker.ts, sqs-redelivery.ts; services/operations/error-classification.ts; tests data-job-worker.integration.spec.js, contact-data-jobs.integration.spec.js. - 'Reviewed evidence: services-platform (ad3f6a65): docs/data-jobs/platform-data-operations.md §3.' - >- Testing back-fill: related capability rules, domain invariants, state transitions and crossing workflows at the pinned source revisions. coverage: rule: covered transition: 'gap: FND-PFM-DJE-063' validation: 'gap: FND-PFM-DJE-063' permission: covered edge: covered failure: covered boundary: covered concurrency: covered invariant: covered interaction: covered coverage_notes: transition: >- Selected lifecycle examples do not cover every applicable object and batch transition. The linked finding retains the remaining matrix. execution: >- Covered means a specification scenario exists. Proposed scenarios and registered suites with no run result do not establish executed conformance. related: - CAP-PFM-DJE-004 - CAP-PFM-DJE-005 - CAP-PFM-DJE-012 workflows: - WF-PFM-DJE-002 coverage_evidence: - rule: BR-CAP-PFM-DJE-003-01 scenarios: - ATS-PFM-DJE-003-01 - ATS-PFM-DJE-003-11 - ATS-PFM-DJE-003-12 provenance: proposed source: >- services-crm:modules/data-jobs/tests/integration/services/queue/data-job-worker.integration.spec.js#changes each contact once between them, whichever delivery got there first - rule: BR-CAP-PFM-DJE-003-02 scenarios: - ATS-PFM-DJE-003-02 - ATS-PFM-DJE-003-13 - ATS-PFM-DJE-003-14 provenance: proposed source: >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#refuses the other five, counting each as what refused it - rule: BR-CAP-PFM-DJE-003-03 scenarios: - ATS-PFM-DJE-003-03 provenance: proposed source: >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#rejects that contact as not found, which tells the person nothing about a record they may not see - rule: BR-CAP-PFM-DJE-003-04 scenarios: - ATS-PFM-DJE-003-04 - ATS-PFM-DJE-003-13 provenance: proposed source: >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#names the rule a refused value broke, and quotes no value of the record - rule: BR-CAP-PFM-DJE-003-05 scenarios: - ATS-PFM-DJE-003-05 provenance: proposed source: >- macroservice-libraries:packages/data-jobs/src/commit-split.spec.ts#halves the unit until the failing object stands alone, and isolates only it - rule: BR-CAP-PFM-DJE-003-06 scenarios: - ATS-PFM-DJE-003-06 provenance: proposed source: >- macroservice-libraries:packages/data-jobs/src/tests/integration/group-refusals.integration.spec.ts#writes none of the group, rejects the other two as group_member_refused, and applies the items of no group - rule: BR-CAP-PFM-DJE-003-07 scenarios: - ATS-PFM-DJE-003-07 provenance: proposed source: 'macroservice-libraries:packages/data-jobs/src/rejection-guard.ts:rejectionGuardVerdict' - rule: BR-CAP-PFM-DJE-003-08 scenarios: - ATS-PFM-DJE-003-08 provenance: proposed source: 'services-crm:docs/data-jobs.md#What each contact''s outcome can be' - rule: BR-CAP-PFM-DJE-003-09 scenarios: - ATS-PFM-DJE-003-09 provenance: proposed source: >- services-crm:modules/data-jobs/tests/integration/services/queue/data-job-worker.integration.spec.js#fails the whole job with the code, not just the batch - rule: BR-CAP-PFM-DJE-003-11 scenarios: - ATS-PFM-DJE-003-10 provenance: proposed source: >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#writes no revision, so the history names only the job that changed the contact --- # Process valid records and report individual refusals: Capability Test Specification ## Coverage The structured coverage rows link required rules to their scenarios. A covered category has a defined check; executable coverage and successful execution require separate evidence. The category map keeps incomplete lifecycle matrices and other missing checks visible as findings. ## Interactions With Related Capabilities Submission commits before processing can account for it. Cancellation accounts for settled and abandoned work. Inspection reconciles partial processing. An invalid owned detail reserves no producer identity. The interaction scenarios use the related capabilities' rule references and identify the shared fixture and observable outcome.