--- product_area: platform module: data-job-engine revision: 1 status: draft owned_by: MOD-PFM-DJE synonyms: - bulk job - import job - Data Import (console navigation) supersedes: - CON-PFM-DJE-002 id: CON-PFM-DJE-001 name: Data Job feature: FEAT-PFM-DJE-01 product_intent: confirmed implementation_evidence: observed source_sha: macroservice-libraries: 54536a441935c6e09fdb7f96f0bec71c52ba4e03 services-crm: a8d1364f1dd44b723bb5f9acba12f983e9ec8d69 services-platform: ad3f6a6582843008ed0f5a17b131878d15829af1 react-console: baf6dae382d6f41dce976c7f848667ccc37e076f definition: >- A data job records a requested operation across records of one kind, retaining its input context, progress and individual outcomes for inspection and recovery. not_to_be_confused_with: - >- Durable Platform work coordinates background execution; the data job records the requested record operation. - Data exports retrieve information and belong to Queries and Exports. - A migration run coordinates several dataset jobs rather than replacing their individual records. facets: - name: Record kind and operation meaning: The target record category and requested operation determine the applicable domain contract. used_for: Selecting supported work and its required authority. - name: Processing lane meaning: 'Interactive input travels with the request, while bulk input is read from a completed file.' used_for: Choosing an appropriate input and execution path. - name: Review mode meaning: Validation records a preview; application performs eligible record writes. used_for: Separating inspection from approval where preview is supported. - name: Progress state meaning: 'State describes whether processing is waiting, active, held or settled.' used_for: Explaining progress and available controls. - name: Batch organization meaning: A batch contains a portion of the retained job input or one planned file chunk. used_for: Scheduling processing and handling interruption. - name: Impact fingerprint meaning: The preview totals and their fingerprint identify the report used for approval. used_for: Connecting a reviewed impact to its application. - name: Submission identity meaning: A caller-supplied idempotency key links repeated requests to the original job. used_for: Distinguishing request retry from new work. - name: Available controls meaning: Backend action descriptions identify supported recovery controls and required permissions. used_for: Presenting controls appropriate to the job’s current condition. - name: Related work meaning: 'Correction, revalidation and restoration links retain the relationship between separate jobs.' used_for: Following work that repairs or compensates earlier input. - name: Record outcomes meaning: >- Receipts, refusals, conflicts, blocks, skips and unattempted counts account for the job’s individual input records. used_for: Investigating results and choosing correction or recovery work. relationships: - to: CON-PFM-DJE-003 statement: A file-processing job references a completed upload and its recorded version. provenance: documented source: 'services-crm:docs/data-jobs.md#The bulk lane' - to: CON-PFM-DJE-004 statement: Keyed processing consults producer identity mappings alongside record writes. provenance: documented source: 'macroservice-libraries:packages/data-jobs/README.md#Keyed writes and the ID crosswalk' - to: CON-PFM-DJE-005 statement: >- A migration run coordinates child jobs for its datasets, with Platform datasets using bounded interactive processing. provenance: documented source: 'services-platform:docs/data-jobs/data-runs.md#1. The one idea' - to: CON-PFM-DJE-006 statement: Capacity factors and workspace budgets constrain admission of bulk batches. provenance: documented source: 'macroservice-libraries:packages/data-jobs/README.md#The bulk lane' lifecycle: - state: accepted meaning: Recorded input awaiting its processing phase. how_it_changes: Submission selects validation or application and records the initial state. - state: 'validating, validated' meaning: 'Checks are in progress, or their report awaits approval.' how_it_changes: Approval queues application; acting on an old preview can expire it. - state: 'queued, running' meaning: Batches are waiting or actively being processed. how_it_changes: 'Controls, failures and completion determine the next state.' - state: paused meaning: Later work is held while earlier commits remain. how_it_changes: 'Supported resume, retry or cancellation determines recovery.' - state: 'succeeded, completed with rejections, cancelled, failed, expired' meaning: Processing has settled with its recorded outcome. how_it_changes: Supported blocked-item retry can reopen eligible completed work; other final states remain settled. acted_on_by: - CAP-PFM-DJE-001 - CAP-PFM-DJE-002 - CAP-PFM-DJE-003 - CAP-PFM-DJE-004 - CAP-PFM-DJE-005 - CAP-PFM-DJE-006 - CAP-PFM-DJE-007 - CAP-PFM-DJE-008 - CAP-PFM-DJE-009 - CAP-PFM-DJE-010 - CAP-PFM-DJE-014 - CAP-PFM-DJE-018 - CAP-PFM-DJE-012 visible_to: - ACT-PFM-035 - ACT-PFM-036 - ACT-PFM-037 - ACT-PFM-012 evidence: - >- Reviewed evidence: macroservice-libraries (54536a44): packages/data-jobs/README.md; src/data-job-states.ts, data-job-config.ts, data-job-actions.ts, submit-data-job.ts, job-control.ts. - >- Reviewed evidence: services-crm (a8d1364f): docs/data-jobs.md; modules/data-jobs/db/data-jobs.table.ts; services/jobs/contact-data-jobs.ts. - >- Reviewed evidence: services-platform (ad3f6a65): docs/data-jobs/platform-data-operations.md; docs/milestones/PLAT-M19-data-job-engine-foundations.md (D3 to D6). - >- Reviewed evidence: react-console (baf6dae3): app/(console)/console/[tenantId]/dataManagement/dataJobs pages; lib/api/data-jobs/rules.ts. --- # Data Job ## What It Is A data job is the continuing account of work requested across a set of records. It connects the request to its processing and results, so staff can inspect a bulk change after the screen that launched it has closed. A contact update, a file import and a supported catalogue change can all produce this kind of account. The records affected by the request still belong to their own product areas. Submission gives the job an identity and retains its input context. Some input is stored with the request, while file processing starts from a completed upload. Staff can therefore distinguish the requested work from later edits to the source material or destination records. Progress and record outcomes remain attached to the job, including when only part of the input is eligible. ## What It Is Not A job does not establish a new identity for every contact it changes. Nor does it replace those contacts’ history. An export serves a different purpose: taking information out. A migration run coordinates several datasets and relies on the individual jobs beneath it. Background execution machinery drives processing but is not the account staff inspect. ## How It Is Used Administrators launch supported changes from list dialogs or file-import tools and follow their jobs in Data Imports. Integrations submit work through the API and inspect its outcome. Implementation staff encounter jobs as the dataset work beneath migration runs. Good use includes reviewing available impact information, checking refusals and understanding which recovery action fits the problem. A malformed input value needs correction, while a missing dependency may need another dataset first. A job’s overall state is useful context, but its individual outcomes explain what actually happened. ## Its Facets **Record kind and operation.** These identify the domain contract responsible for the target records and the requested change. **Processing lane and review mode.** Together they describe how input is supplied and whether the chosen surface supports a preview before application. A file is not automatically eligible for every review mode. **Progress state and batch organization.** They let staff distinguish waiting from active processing and understand why interruption can leave earlier changes intact. **Impact fingerprint and submission identity.** One connects approval to a retained preview; the other lets a caller distinguish a repeated request from new work. **Available controls and related work.** These explain how inspection, correction and restoration remain separate operations linked to the original request. **Record outcomes.** These are a facet of the job rather than a separate broad concept. They retain the evidence needed to inspect individual input, including successful work and reasons that other work did not proceed. ## How It Relates Uploads supply file input, producer identities support reconciliation, and migration runs coordinate related datasets. Capacity protection influences admission of bulk work. Record-owning modules provide their own validation and write behavior. The job connects these participants into an inspectable operation without taking ownership of their domain records. ## What It Could Become Product still needs to review large-file previews, retention enforcement and the planned tools for investigating very large result sets. A richer account of excluded restoration records would help staff understand recovery. Those extensions should continue to distinguish a requested operation, its individual evidence and the destination records it affects.