---
id: STATE-PFM-DJE-002
title: Data job batch
object: DOM-PFM-DJE-001
product_area: platform
module: data-job-engine
revision: 1
status: draft
design_intent: confirmed
origin: >-
  services-platform:docs/proposals/rhythm-data-job-engine-plan.md §9.3 batch
  algorithm, §9.6 crash and redelivery (approved 2026-10-12);
  services-platform:docs/queued-work.md (the queued-work standard)
implementation_evidence: observed
source_sha:
  macroservice-libraries: 54536a441935c6e09fdb7f96f0bec71c52ba4e03
  services-crm: a8d1364f1dd44b723bb5f9acba12f983e9ec8d69
states:
  - name: pending
    meaning: 'A batch exists and awaits work, with or without a queue entry.'
  - name: running
    meaning: A delivery has begun and its attempt is recorded.
  - name: done
    meaning: Batch processing has settled and contributed once to finished progress.
  - name: failed_transient
    meaning: >-
      Transient processing exhausted its allowed attempts and awaits operator
      recovery.
  - name: not_attempted
    meaning: >-
      Processing abandoned this batch; unfinished items contribute to
      not-attempted totals.
transitions:
  - from: pending
    to: running
    trigger: Take a batch delivery
    guard: 'The job is queued, running or validating'
    side_effects:
      - increment its delivery attempt
    who: Worker or request
    provenance: documented
    source: >-
      macroservice-libraries:packages/data-jobs/src/run-data-job-batch.ts
      (module comment, steps 1 and 2)
  - from: running
    to: done
    trigger: Settle batch items
    guard: Conditional completion finds the batch pending or running
    side_effects:
      - increment finished progress once; settle the parent when no work remains
    who: Batch runner
    provenance: documented
    source: >-
      macroservice-libraries:packages/data-jobs/README.md#The counters on the
      job row
  - from: running
    to: failed_transient
    trigger: Exhaust transient attempts
    guard: The default five attempts have been used
    side_effects:
      - pause the parent with a transient-failure reason
    who: Batch runner
    provenance: observed
    source: >-
      macroservice-libraries:packages/data-jobs/src/run-data-job-batch.ts:DATA_JOB_MAX_BATCH_ATTEMPTS
  - from: pending
    to: not_attempted
    trigger: Abandon unfinished work
    guard: Pending work is cancelled or failed
    side_effects:
      - count its items as not attempted
    who: Cancellation or failure handling
    provenance: observed
    source: 'macroservice-libraries:packages/data-jobs/src/job-control.ts:cancelDataJob'
  - from: not_attempted
    to: pending
    trigger: Resume held work
    guard: The parent is paused
    side_effects:
      - restore a queue entry
    who: Operator
    provenance: observed
    source: 'macroservice-libraries:packages/data-jobs/src/job-control.ts:resumeDataJob'
  - from: failed_transient
    to: pending
    trigger: Recover exhausted work
    guard: The parent is paused and recovery is permitted
    side_effects:
      - restore queued work; use receipts to omit committed items
    who: Operator
    provenance: observed
    source: 'macroservice-libraries:packages/data-jobs/src/job-control.ts:resumeDataJob'
  - from: done
    to: pending
    trigger: Requeue an approved preview or dependency retry
    guard: The preview is approved or this completed batch contains blocked work
    side_effects:
      - recompute the parent’s finished-batch count
    who: Approver or operator
    provenance: documented
    source: >-
      macroservice-libraries:packages/data-jobs/README.md#The counters on the
      job row
invalid:
  - from: done
    to: running
    why: >-
      Direct redelivery of a completed batch does no work; reopening requires
      explicit requeue.
  - from: not_attempted
    to: running
    why: >-
      An abandoned batch needs an explicit requeue before normal processing can
      continue.
concurrency: >-
  Queue claims use skip-locked selection. Duplicate deliveries can still
  overlap, but receipt insertion determines which transaction may write each
  item. Conditional completion increments finished progress once. CRM relies on
  queue redelivery; Platform makes transient attempts within the calling
  request.
evidence:
  - >-
    macroservice-libraries at 54536a44:
    packages/data-jobs/src/data-job-states.ts (DataJobBatchState),
    run-data-job-batch.ts, batch-queue-dispatcher.ts, job-control.ts, README.md.
  - >-
    services-crm at a8d1364f: services/queue/data-job-dispatch.ts,
    data-job-worker.ts, sqs-redelivery.ts.
---

# Data Job Batch

## The Lifecycle

```mermaid
stateDiagram-v2
  [*] --> pending
  pending --> running: take a batch delivery
  running --> done: settle batch items
  running --> failed_transient: exhaust transient attempts
  pending --> not_attempted: abandon unfinished work
  not_attempted --> pending: resume held work
  failed_transient --> pending: recover exhausted work
  done --> pending: requeue an approved preview or dependency retry
```

## Transitions Explained

Every planned batch begins pending. Delivery records an attempt before item processing; conditional completion makes finished progress resistant to duplicate deliveries. Exhausted transient work waits for operator recovery, while abandoned work has a separate not-attempted state. Approval and dependency retry can reopen completed batches.

## What Cannot Happen

A finished delivery does not reopen itself. Explicit recovery determines which batches become pending again.

## Concurrency and Timing

Receipts protect concurrent delivery, while queue claims reduce ordinary duplication. CRM retries through transport; Platform retries immediately in process and can lose claimed work if the request stops.