---
id: WF-PFM-DJE-012
title: Drive a data run
trigger: 'Staff start, resume or read a running migration run.'
product_area: platform
module: data-job-engine
revision: 1
status: draft
design_intent: proposed
implementation_evidence: observed
inference: []
source_sha:
  services-platform: ad3f6a6582843008ed0f5a17b131878d15829af1
  services-crm: a8d1364f1dd44b723bb5f9acba12f983e9ec8d69
implements:
  - CAP-PFM-DJE-016
  - CAP-PFM-DJE-015
moves:
  - STATE-PFM-DJE-004
  - STATE-PFM-DJE-005
  - STATE-PFM-DJE-001
crosses:
  - SVC-PFM-DJE-data-jobs
  - SVC-PFM-DJE-crm-data-jobs
hops:
  - 'n': 1
    actor_or_service: Run driver
    action: Authorize remaining record categories through each service port.
    anchor: >-
      services-platform:modules/data-jobs/services/runs/run-driver.ts:authorizeSteps
    seam: sync
    on_failure: Stop on access denial before this authorization phase writes progress.
    left_inconsistent: Earlier passes retain their existing progress.
    observer: The request response exposes the result.
    provenance: test
    source: >-
      services-platform:modules/data-jobs/tests/integration/services/runs/run-driver.integration.spec.js#forwards
      the staff caller’s own token with every call, and invents no identity of
      its own
  - 'n': 2
    actor_or_service: Run driver
    action: >-
      Set registered or paused work running; a running start proceeds to
      advancement.
    anchor: >-
      services-platform:modules/data-jobs/services/runs/run-driver.ts:startDataRun
    seam: sync
    on_failure: Refuse terminal parent states.
    left_inconsistent: Failed state changes preserve prior state.
    observer: The request response exposes the result.
    provenance: observed
    source: >-
      services-platform:modules/data-jobs/services/runs/run-driver.ts:startDataRun
  - 'n': 3
    actor_or_service: Run driver and the owning services
    action: Poll active child jobs and save recognized progress and counts.
    anchor: 'services-platform:modules/data-jobs/services/runs/run-driver.ts:pollStep'
    seam: sync
    on_failure: >-
      Use transient-read grace; permanent errors fail a step and access denial
      escapes the pass.
    left_inconsistent: >-
      Earlier progress writes in this pass may remain; unexpected successful
      payloads can stay running.
    observer: The run read; step failure detail.
    provenance: observed
    source: >-
      services-platform:modules/data-jobs/services/runs/data-job-port.ts:readProgress
  - 'n': 4
    actor_or_service: Run driver
    action: Fail the parent when a recorded step fails and skip pending steps.
    anchor: >-
      services-platform:modules/data-jobs/services/runs/run-driver.ts:advanceDataRun
    seam: sync
    on_failure: Return persistence failures for later caller handling.
    left_inconsistent: Other active child jobs can keep writing.
    observer: The run read and summary.
    provenance: test
    source: >-
      services-platform:modules/data-jobs/tests/integration/lifecycle/data-run-guarantees.integration.spec.js#leaves
      the run where it stood while nobody reads it, though a dataset finished in
      its service
  - 'n': 5
    actor_or_service: Run driver and the owning services
    action: >-
      Submit dependency-ready or eligible blocked datasets and save each
      returned child identity.
    anchor: 'services-platform:modules/data-jobs/services/runs/run-driver.ts:submitStep'
    seam: sync
    on_failure: Submission errors escape before step settlement.
    left_inconsistent: A remote child can exist without the parent recording it.
    observer: The request error can be the only indication of failed submission.
    provenance: observed
    source: 'services-platform:modules/data-jobs/services/runs/run-driver.ts:submitStep'
  - 'n': 6
    actor_or_service: Run driver
    action: >-
      Settle when all datasets succeeded or finally blocked, then compute
      summary and readiness.
    anchor: >-
      services-platform:modules/data-jobs/services/runs/run-summary.ts:summariseDataRun
    seam: sync
    on_failure: Return any persistence or read failure.
    left_inconsistent: >-
      Parent success can coexist with readiness blockers, and activation does
      not consume the verdict.
    observer: The staff reading the summary.
    provenance: test
    source: >-
      services-platform:modules/data-jobs/tests/unit/run-summary.spec.ts#is not
      ready when a dataset’s counts do not add up to what the producer claimed
idempotency: >-
  Starting running work advances a pass. Child submission is not given a
  run-derived idempotency key, so a successful but unrecorded remote call can be
  submitted again.
concurrency: >-
  Multiple readers can drive the same parent. State writes lock rows, but remote
  calls and child bookkeeping are separate and no pass-wide lock prevents
  duplicate submissions.
branches:
  - >-
    Platform datasets execute in process; CRM datasets use remote collection
    routes.
  - Pause holds new submissions while existing children continue.
waits:
  - >-
    Staff or automation must keep issuing driving reads; unwatched parents do
    not advance.
evidence:
  - >-
    services-platform at ad3f6a65: services/runs/run-driver.ts,
    data-job-port.ts, data-run-ports.ts, data-run-services.ts, run-summary.ts;
    aws/api/data-runs.routes.ts; docs/data-jobs/data-runs.md.
  - >-
    services-crm at a8d1364f: aws/api/bulk-data-jobs.routes.ts (the routes a
    step is submitted to).
controls: [TC-AVL-001, TC-AVL-004, TC-AVL-005, TC-AZ-001, TC-AZ-002, TC-AZ-003, TC-AZ-004, TC-AZ-005, TC-CHG-001, TC-CHG-002, TC-CHG-003, TC-CHG-004, TC-CHG-005, TC-CHG-006, TC-CHG-007, TC-CRY-001, TC-CRY-002, TC-CRY-003, TC-CRY-004, TC-DAT-001, TC-DAT-002, TC-DAT-003, TC-DAT-006, TC-IAM-001, TC-IAM-002, TC-IAM-003, TC-IAM-004, TC-IAM-005, TC-IAM-006, TC-LOG-001, TC-LOG-004, TC-LOG-005, TC-MON-001, TC-PI-001, TC-PI-002, TC-PI-003, TC-PI-004, TC-PI-005, TC-PRV-001, TC-PRV-002, TC-TEN-001, TC-TEN-002, TC-TEN-003, TC-VEN-001, TC-VEN-002, TC-VEN-003]
---

# Drive A Data Run

## What It Does

A caller-driven pass authorizes remaining datasets, polls active children and submits eligible work. It settles the parent from recorded step outcomes and returns a separate readiness assessment.

## The Flow

```mermaid
flowchart TD
  H1["Run driver"] --> H2
  H2["Run driver"] --> H3
  H3["Run driver and the owning services"] --> H4
  H4["Run driver"] --> H5
  H5["Run driver and the owning services"] --> H6
  H6["Run driver"]
```

## When Something Fails

Transient reads have a 30-minute grace based on saved progress. Submission errors and unexpected payloads can leave unfinished or untracked work. Parent failure does not stop every other active child or remove the load.

## Open Design Questions

Child submission tracking, cancellation and readiness need stronger recovery boundaries.