---
id: SVC-PFM-DJE-engine
title: Data Job Engine Library
product_area: platform
module: data-job-engine
repository: macroservice-libraries
source_module: packages/data-jobs
revision: 1
status: draft
design_intent: confirmed
origin: >-
  services-platform:docs/proposals/rhythm-data-job-engine-plan.md §5.3 "Why a
  library and per-service tables, not a central service" and §7.2 the operation
  contract (approved by Andrew 2026-10-12); PLAT-M19 Product Accepted 2026-10-22
  (Platform runs the same engine version as CRM)
implementation_evidence: observed
inference: []
source_sha:
  macroservice-libraries: 54536a441935c6e09fdb7f96f0bec71c52ba4e03
owns:
  - DOM-PFM-DJE-001
  - DOM-PFM-DJE-004
does_not_own:
  - >-
    Physical engine rows belong to adopting services, which construct their
    tables from the shared contracts.
  - >-
    Record owners supply domain validation, asset preparation, persistence and
    any compensation plan.
  - >-
    Adopters provide message transport, storage, state-machine hosting, capacity
    and caller context.
operations: []
events: []
jobs:
  - name: >-
      Shared bulk task functions: chunkSource, planBatches, admitBatches,
      settleIfDone, failBulkDataJob
    kind: 'state machine tasks, hosted by each adopting service'
    does: >-
      Provide file-reading, planning, admission, settlement and failure tasks
      for adopter-hosted orchestration.
  - name: batch dispatcher (dispatchDataJobBatches)
    kind: called by the service's scheduled or in-request dispatcher
    does: >-
      Lease queued batches, call the adopter’s publisher and remove accepted
      publications.
  - name: submission purge (purgeDataJobSubmissions)
    kind: called by the service's daily schedule
    does: Remove workspace submission keys beyond their configured 30-day retention.
integrations: []
consistency:
  - rule: >-
      Interactive submission commits its job, preserved input, batches, queued
      work and optional ledger key together.
    kind: transactional
    compensation: Transaction rollback removes incomplete submission.
  - rule: >-
      Domain effects share their commit unit with receipts and any producer
      mappings.
    kind: transactional
    compensation: Refused units roll back; splitting isolates eligible survivors.
  - rule: >-
      Execution and summary progress settle later through adopter transport or
      request execution.
    kind: eventual
    compensation: >-
      Receipts protect replay; exhausted transient work pauses, permanent
      failure retains earlier commits.
  - rule: >-
      Domain receipts and history produce a separately reviewed compensation
      job.
    kind: eventual
    compensation: Versioned compensation preserves later edits and reports excluded records.
evidence:
  - >-
    macroservice-libraries at 54536a44: packages/data-jobs/README.md,
    CHANGELOG.md; src/define-data-operation.ts, data-job-states.ts,
    data-job-config.ts, submit-data-job.ts, validate-data-job-request.ts,
    run-data-job-batch.ts, job-control.ts, plan-data-job-rollback.ts,
    revalidate-data-job-conflicts.ts, version-capture.ts, apply-keyed-item.ts,
    keyed-item-decision.ts, bulk-data-job-tasks.ts, admission-budget.ts,
    admission-planner.ts, batch-queue-dispatcher.ts, submission-purge.ts,
    row-level-security.ts, data-job-columns.ts, data-job-actions.ts.
  - >-
    The package declares no HTTP routes; adopting services publish the
    operations (SVC-PFM-DJE-crm-data-jobs, SVC-PFM-DJE-data-jobs).
controls: [TC-AVL-004, TC-AVL-005, TC-DAT-001, TC-DAT-002, TC-DAT-003, TC-DAT-004, TC-IAM-001, TC-IAM-002, TC-IAM-004, TC-IAM-005, TC-IAM-006, TC-LOG-002, TC-LOG-003, TC-LOG-004, TC-LOG-005, TC-MON-001, TC-PI-001, TC-PI-002, TC-PI-003, TC-PI-004, TC-PI-005, TC-TEN-001, TC-TEN-002, TC-TEN-003, TC-VEN-001, TC-VEN-002, TC-VEN-003]
---

# Data Job Engine Library

## What This Service Is For

The library supplies the common processing model for bulk record changes. An adopter declares input versions, authority, editable fields, matching identities, dependencies, grouping, domain checks and persistence. Optional preparation and rollback declarations keep record-specific work with its owner.

## Boundary

The package owns the job and producer-mapping designs but hosts no persistent instances or HTTP service. CRM and Platform provide tables, execution contexts and external adapters. Library tasks run inside those adopters.

## Operations in Words

Adopting services expose submission, inspection, approval, controls and reconciliation through their own routes. The package offers callable functions rather than a route catalogue and publishes no Submissions binding.

## Events and Jobs

Adopters host file orchestration tasks, batch dispatch and submission-key purge. The library produces no business event; record owners determine their own history and publication.

## Consistency Guarantees

Submission and item effects have database transaction boundaries. Transport, progress and control propagation settle later. Explicit rollback is a reviewed versioned job, with no automatic reversal of earlier commits.