---
id: DOM-PFM-DJE-005
title: Data Run
realises: CON-PFM-DJE-005
product_area: platform
module: data-job-engine
revision: 1
status: draft
design_intent: proposed
origin: >-
  services-platform:docs/proposals/rhythm-data-job-engine-plan.md §12
  (new-workspace loading) and §13.3 (reset and reload), approved 2026-10-12;
  PLAT-M19 D5 (loading state and reset only into sandbox workspaces). The run
  design itself (services-platform:docs/data-jobs/data-runs.md; PLAT-M22) is not
  yet Product Accepted.
implementation_evidence: observed
source_sha:
  services-platform: ad3f6a6582843008ed0f5a17b131878d15829af1
  services-crm: a8d1364f1dd44b723bb5f9acba12f983e9ec8d69
  services-membership: 5994c270e8a37a30f848e99ba5b5ee6b87c08893
owned_by: SVC-PFM-DJE-data-jobs
readable_by: []
lifecycle: STATE-PFM-DJE-004
custom_fields: none
identity: >-
  The service assigns each run an identity within its workspace. Registration
  checks for an unfinished run of the same package but does not uniquely
  serialize concurrent first registrations. Dataset names are unique within a
  run, and steps also have server-assigned identities.
attributes:
  - name: package identity and label
    kind: text
    required: true
    computed: false
    notes: The manifest supplies its package identity and display label.
  - name: purpose and mode
    kind: enum
    required: true
    computed: false
    notes: >-
      Runs implement initial import; their chosen validation or application mode
      is passed to dataset jobs.
  - name: manifest upload and format version
    kind: reference
    required: true
    computed: false
    notes: >-
      The completed manifest upload and supported package version identify the
      registered declaration.
  - name: producer
    kind: text
    required: true
    computed: false
    notes: The package names its producing system and source workspace.
  - name: state and failure
    kind: enum
    required: true
    computed: true
    notes: >-
      The service records registered, running, paused, succeeded, failed or
      cancelled progress and safe failure detail.
  - name: started and finished
    kind: date
    required: false
    computed: true
    notes: >-
      Optional start and finish instants distinguish unstarted work from
      completed execution.
  - name: steps
    kind: collection
    required: true
    computed: false
    notes: >-
      The manifest supplies at most 100 dataset steps with their type,
      operation, input version, upload, integrity claims, count and
      dependencies.
  - name: step progress
    kind: computed
    required: true
    computed: true
    notes: >-
      A step retains its child job, state, at most two attempts, lifecycle
      times, safe failure detail and copied outcome counts.
  - name: summary
    kind: computed
    required: true
    computed: true
    notes: Reads total the current step counts into a run summary.
  - name: readiness verdict
    kind: computed
    required: true
    computed: true
    notes: >-
      Reads derive readiness blockers from completion, mode, failures,
      unresolved outcomes and count shortfalls; activation does not consult this
      answer.
relationships:
  - to: DOM-PFM-DJE-003
    cardinality: 'many-to-many: runs can reuse uploads and each run can name several'
    ownership: Platform retains upload objects independently of run reset.
    cross_service: false
  - to: DOM-PFM-DJE-001
    cardinality: 'one-to-many: each dataset attempt identifies its submitted job'
    ownership: >-
      The destination service owns the job; the run copies its identity and
      progress counts.
    cross_service: true
  - to: DOM-PFM-DJE-004
    cardinality: 'one-to-many: reconciliation can consult multiple producer mappings'
    ownership: Each destination owns the mappings read through its lookup operation.
    cross_service: true
invariants:
  - id: INV-DOM-PFM-DJE-005-01
    statement: >-
      Registration requires the manifest’s destination workspace to be in
      implementation status and loading state.
    enforced_at:
      - service
    provenance: test
    source: >-
      services-platform:modules/data-jobs/tests/integration/services/runs/register-run.integration.spec.js#refuses
      an initial import into a tenant that is not loading
  - id: INV-DOM-PFM-DJE-005-02
    statement: >-
      Registration writes nothing unless every dataset upload is completed and
      its stored size and available digest agree with the manifest.
    enforced_at:
      - service
    provenance: observed
    source: >-
      services-platform:modules/data-jobs/services/runs/register-run.ts:requireFileAsClaimed
  - id: INV-DOM-PFM-DJE-005-03
    statement: >-
      Dataset ordering rejects unknown dependencies and cycles before persisting
      the run.
    enforced_at:
      - service
    provenance: test
    source: >-
      services-platform:modules/data-jobs/tests/integration/services/runs/register-run.integration.spec.js#refuses
      a package whose datasets cannot be ordered, before any row of it is
      written
  - id: INV-DOM-PFM-DJE-005-04
    statement: >-
      Step rows link submitted states to a child job, and the driver permits at
      most two submissions with the second reserved for blocked work after
      dependencies finish.
    enforced_at:
      - database
      - service
    provenance: observed
    source: >-
      services-platform:modules/data-jobs/db/data-run-steps.table.ts:data_run_steps_job_check
  - id: INV-DOM-PFM-DJE-005-05
    statement: >-
      The persistence transition table prevents a final run from returning to
      active execution.
    enforced_at:
      - service
    provenance: observed
    source: >-
      services-platform:modules/data-jobs/services/persistence/data-runs.ts:DATA_RUN_TRANSITIONS
  - id: INV-DOM-PFM-DJE-005-06
    statement: >-
      Dataset calls use the requesting staff member’s authority and forward the
      current token without retaining it.
    enforced_at:
      - service
    provenance: test
    source: >-
      services-platform:modules/data-jobs/tests/integration/services/runs/run-driver.integration.spec.js#forwards
      the staff caller’s own token with every call, and invents no identity of
      its own
  - id: INV-DOM-PFM-DJE-005-07
    statement: >-
      Failure and cancellation preserve committed records; the run performs no
      automatic compensation and workspace reset is the removal path.
    enforced_at:
      - service
    provenance: documented
    source: >-
      services-platform:modules/data-jobs/services/runs/run-driver.ts (module
      comment, "It stops on failure and keeps what it did")
physical:
  - data_runs
  - data_run_steps
evidence:
  - >-
    services-platform at ad3f6a65: modules/data-jobs/db/data-runs.table.ts,
    data-run-steps.table.ts; services/runs/package-manifest.ts, register-run.ts,
    run-driver.ts, run-summary.ts, data-job-port.ts, data-run-ports.ts,
    data-run-services.ts; services/persistence/data-runs.ts;
    aws/api/data-runs.routes.ts; docs/data-jobs/data-runs.md, tenant-reset.md.
  - >-
    services-crm at a8d1364f: services/jobs/records-by-source-key.ts;
    services/jobs/initial-import-gate.ts.
  - >-
    services-membership at 5994c270:
    services/operations/structure-data-operations.ts (declared operations only).
---

# Data Run

## What It Models

A data run coordinates one initial migration package across the applications that own its datasets. It records dependency order, child jobs and copied progress. A separate readiness answer helps staff assess the resulting load. The run delegates all business writes to those child jobs.

## Identity and Attributes

The package identity relates attempts to the producer’s delivery, while the run identity distinguishes each registration. A duplicate unfinished package is refused when visible to the registration transaction; concurrent absence checks remain a gap. Manifest claims and system-computed progress remain separate.

## Relationships

Manifest and dataset uploads can be reused by later runs. Child jobs belong to CRM or Platform and their counts are copied onto steps when polled. Producer mappings support reconciliation without becoming run-owned records.

## Invariants

Registration gates the destination, available upload evidence and dependency graph. The driver forwards staff authority and limits blocked work to one additional attempt. Final runs remain final, and no parent outcome removes committed business records.

## How It Is Stored and Shared

Platform owns runs and steps under workspace row security and exposes them through staff-only system routes. Remote datasets use the current staff token; Platform datasets execute locally in a workspace context. Reset preserves migration evidence, while deletion removes it. Runs enable no custom fields.

## Open Design Questions

Readiness neither blocks activation nor detects excess counts. Submission failures and concurrent passes can leave untracked child work; cancellation can finish while a child still writes. Unexpected progress responses can remain unsettled indefinitely. Membership participation is still incomplete.