--- id: CAP-PFM-DJE-001 title: Submit an accountable change across many records feature: FEAT-PFM-DJE-01 product_area: platform module: data-job-engine revision: 1 status: draft product_intent: confirmed implementation_evidence: observed supersedes: - CAP-DJE-ENG-001 - RUL-DJE-ENG-001 - RUL-DJE-ENG-002 - RUL-DJE-ENG-003 - RUL-DJE-ENG-005 source_sha: macroservice-libraries: 54536a441935c6e09fdb7f96f0bec71c52ba4e03 services-crm: a8d1364f1dd44b723bb5f9acba12f983e9ec8d69 services-platform: ad3f6a6582843008ed0f5a17b131878d15829af1 react-console: baf6dae382d6f41dce976c7f848667ccc37e076f intent_sources: - 'services-platform:docs/milestones/PLAT-M19-data-job-engine-foundations.md#D-37' - 'services-platform:docs/milestones/plans/PLAT-M19-plan.md#A1' - 'services-platform:docs/milestones/plans/PLAT-M19-plan.md#A8' - 'services-platform:docs/milestones/plans/PLAT-M19-plan.md#S01' - 'services-platform:docs/milestones/PLAT-M19-data-job-engine-foundations.md#Status' purpose: >- Staff and connected systems submit a single authorized job to make a change across a collection of records. outcomes: - A job retains the submitted records and changes for later processing. - Retrying a keyed submission returns the existing job. - Bulk work uses the authority of its submitter. concepts: - CON-PFM-DJE-001 actors: - ACT-PFM-035 - ACT-PFM-036 surfaces: - console - api preconditions: [] postconditions: [] permissions: - key: crm.contacts.bulk where_checked: api scope: tenant - key: crm.organizations.bulk where_checked: api scope: tenant - key: crm.contact-types.bulk where_checked: api scope: tenant - key: crm.contact-roles.bulk where_checked: api scope: tenant - key: crm.organization-types.bulk where_checked: api scope: tenant - key: crm.address-types.bulk where_checked: api scope: tenant - key: crm.phone-number-types.bulk where_checked: api scope: tenant - key: crm.regions.bulk where_checked: api scope: tenant - key: crm.activity-types.bulk where_checked: api scope: tenant - key: crm.activities.bulk where_checked: api scope: tenant - key: crm.contacts.update where_checked: api scope: tenant - key: crm.contacts.delete where_checked: api scope: tenant - key: crm.organizations.update where_checked: api scope: tenant - key: crm.organizations.delete where_checked: api scope: tenant - key: platform.custom-fields.bulk where_checked: api scope: tenant rules: - id: BR-CAP-PFM-DJE-001-01 proved_by: [ATS-PFM-DJE-001-01, TST-PFM-DJE-108, TST-PFM-DJE-109] statement: >- A caller submitting or controlling a job needs both the bulk permission for its record kind and the operation permission, such as update or delete; holding either alone is insufficient. kind: permission enforced_at: - api - service must_prove: true provenance: test source: >- services-crm:modules/data-jobs/tests/e2e/data-jobs.e2e.spec.js#refuses a caller who may update one contact but holds no bulk key with 403 origin: 'services-platform:docs/milestones/plans/PLAT-M19-plan.md#A8' - id: BR-CAP-PFM-DJE-001-02 proved_by: [ATS-PFM-DJE-001-02, TST-PFM-DJE-109, TST-PFM-DJE-110] statement: >- An interactive request contains either at most 500 selected record identifiers with a common change or at most 500 prepared records; larger inputs use a file. kind: validation enforced_at: - api - service must_prove: true provenance: test source: >- services-crm:modules/data-jobs/tests/e2e/organization-update.e2e.spec.js#refuses a selection past the interactive lane’s 500 with 400 origin: 'services-platform:docs/milestones/plans/PLAT-M19-plan.md#A1' - id: BR-CAP-PFM-DJE-001-03 proved_by: [ATS-PFM-DJE-001-03, TST-PFM-DJE-110, TST-PFM-DJE-117] statement: >- A common change can address only declared editable fields, including permitted custom fields; system fields, collections and undeclared fields are refused by name before processing. kind: validation enforced_at: - api - service must_prove: true provenance: test source: >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#refuses a field the bulk update may not set, naming it, before any contact is touched origin: 'services-platform:docs/milestones/plans/PLAT-M20-plan.md#A2' - id: BR-CAP-PFM-DJE-001-04 proved_by: [ATS-PFM-DJE-001-04, ATS-PFM-DJE-001-11, ATS-PFM-DJE-001-14, ATS-PFM-DJE-002-06, ATS-PFM-DJE-009-11, TST-PFM-DJE-117] statement: >- The job stores its selected records and submitted changes so later processing uses that saved input. kind: invariant enforced_at: - service - database must_prove: true provenance: test source: >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#freezes the patch on every item, so a later apply writes what was submitted and not what is asked for then origin: none - id: BR-CAP-PFM-DJE-001-05 proved_by: [ATS-PFM-DJE-001-05, ATS-PFM-DJE-001-13, ATS-PFM-DJE-005-08, TST-PFM-DJE-053, TST-PFM-DJE-117] statement: >- For 30 days, the same submitter can repeat an idempotency key with the same request to retrieve its original job; reusing the key for a different request is refused. kind: policy enforced_at: - service - database must_prove: true provenance: test source: >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#answers the second submission with the first job, having written nothing origin: 'services-platform:docs/milestones/plans/PLAT-M19-plan.md#S01' - id: BR-CAP-PFM-DJE-001-06 proved_by: [ATS-PFM-DJE-001-06, ATS-PFM-DJE-001-15, ATS-PFM-DJE-018-09, TST-PFM-DJE-117] statement: >- A submission must name a declared record kind and operation; an unsupported choice is refused with the supported choices. kind: validation enforced_at: - api - service must_prove: true provenance: test source: >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#refuses an object type CRM performs no bulk operation on, naming what it does perform origin: none - id: BR-CAP-PFM-DJE-001-07 proved_by: [ATS-PFM-DJE-001-07, TST-PFM-DJE-052] statement: >- CRM interactive submissions accept updates and deletes by Rhythm identifier; creating records or matching producer keys requires the file lane. kind: validation enforced_at: - api must_prove: true provenance: observed source: 'services-crm:modules/data-jobs/aws/api/data-jobs.schemas.ts:dataJobRequestSchema' origin: none - id: BR-CAP-PFM-DJE-001-08 proved_by: [ATS-PFM-DJE-001-08, TST-PFM-DJE-108, TST-PFM-DJE-116] statement: >- A collection exposes only jobs for its own record kind; a mismatched job is not found, as is a job in another workspace. kind: invariant enforced_at: - service must_prove: true provenance: test source: >- services-crm:modules/data-jobs/tests/integration/services/jobs/configuration-data-jobs.integration.spec.js#is not found by the apply, so a validated job of contacts cannot be started from here origin: none - id: BR-CAP-PFM-DJE-001-09 proved_by: [ATS-PFM-DJE-001-09, TST-PFM-DJE-035, TST-PFM-DJE-117] statement: >- Jobs, their input and their outcomes remain private to their workspace, including when another workspace attempts to run them. kind: invariant enforced_at: - service - database must_prove: true provenance: test source: >- macroservice-libraries:packages/data-jobs/src/tests/integration/tenant-isolation.integration.spec.ts#does not let another tenant see a job, or learn that it exists origin: 'services-platform:docs/milestones/plans/PLAT-M19-plan.md#S11' - id: BR-CAP-PFM-DJE-001-10 proved_by: [ATS-PFM-DJE-001-10, ATS-PFM-DJE-001-12, ATS-PFM-DJE-003-11] statement: >- Submission records the job, input, batches, durable queue entries and any idempotency entry in one transaction; failure rolls back those writes. kind: invariant enforced_at: - service - database must_prove: false provenance: documented source: 'macroservice-libraries:packages/data-jobs/README.md#A worked example: updating contacts in bulk' origin: none scenarios: - id: SC-CAP-PFM-DJE-001-01 given: An administrator with contact bulk and update access selects 120 contacts. when: They request the same job-title change for that selection. then: 'Rhythm records one job, its 120 input records and five batches of 25.' provenance: documented source: 'macroservice-libraries:packages/data-jobs/README.md#A worked example: updating contacts in bulk' - id: SC-CAP-PFM-DJE-001-02 given: A network timeout causes a caller to resend the same keyed submission. when: Rhythm receives the repeated request. then: The response identifies the original job without creating more work. provenance: test source: >- services-crm:modules/data-jobs/tests/integration/services/jobs/contact-data-jobs.integration.spec.js#answers the second submission with the first job, having written nothing edge_cases: [] constraints: [] depends_on: [] related: - CAP-PFM-DJE-002 - CAP-PFM-DJE-003 - CAP-PFM-DJE-005 journeys: - JNY-PFM-DJE-001 - JNY-PFM-DJE-002 - JNY-PFM-DJE-005 open_questions: [] evidence: - >- Reviewed evidence: macroservice-libraries (54536a44): README.md (The operation contract; A worked example); src/validate-data-job-request.ts, submit-data-job.ts, row-level-security.ts; tests tenant-isolation.integration.spec.ts, data-job-engine.integration.spec.ts. - >- Reviewed evidence: services-crm (a8d1364f): aws/api/data-jobs.routes.ts, organization-data-jobs.routes.ts, bulk-data-jobs.routes.ts, data-jobs.schemas.ts; services/operations/data-operations.ts; tests contact-data-jobs.integration.spec.js, configuration-data-jobs.integration.spec.js, data-jobs.e2e.spec.js, organization-update.e2e.spec.js. - >- Reviewed evidence: services-platform (ad3f6a65): aws/api/custom-field-data-jobs.routes.ts, custom-object-record-data-jobs.routes.ts; services/jobs/data-job-surfaces.ts. - >- Reviewed evidence: react-console (baf6dae3): lib/api/data-jobs/capabilities.ts, submit.ts; acceptance/crm/journeys/data-operations/CRM-ACC-0653, 0654. controls: [TC-AVL-001, TC-AVL-004, TC-AZ-001, TC-AZ-002, TC-AZ-003, TC-AZ-004, TC-DAT-001, TC-DAT-002, TC-DAT-003, TC-DAT-004, TC-IAM-001, TC-IAM-006, TC-LOG-004, TC-LOG-005, TC-MON-001, TC-PI-001, TC-PI-002, TC-PI-003, TC-PI-004, TC-PI-005, TC-PRV-001, TC-PRV-002, TC-TEN-001, TC-TEN-002, TC-TEN-003, TC-TEN-004, TC-VEN-001, TC-VEN-002, TC-VEN-003] exercised_by: [FAS-PFM-DJE-01-01, FAS-PFM-DJE-01-02, FAS-PFM-DJE-01-03, FAS-PFM-DJE-01-04, FAS-PFM-DJE-01-05] suites: [TST-PFM-DJE-035, TST-PFM-DJE-052, TST-PFM-DJE-053, TST-PFM-DJE-108, TST-PFM-DJE-109, TST-PFM-DJE-110, TST-PFM-DJE-116, TST-PFM-DJE-117] --- # Submit an accountable change across many records ## Purpose Staff and connected systems submit a single authorized job to make a change across a collection of records. ## How It Works A data administrator selects the records to change and supplies the intended operation. A connected system can submit equivalent work through the API. Rhythm checks the permissions and input shape before recording the job. The saved input gives subsequent processing a stable request to follow. Interactive changes travel with the request; imports and larger changes use uploaded files. A repeated keyed request retrieves the earlier job, while an unkeyed request creates new work. Staff follow the resulting job to review progress and outcomes. The product should preserve this authorization model as additional product areas adopt the engine. ## Relationships - Previewing a change uses the submitted job as its starting point. - Batch processing consumes the saved input. - Each product area declares the operations its records support.