---
title: Data Job Engine Design Examination
examined_at: 2026-10-23T00:00:00.000Z
examiner: codex
source_sha:
  rhythm-v2-docs: 1070cb57a7637fe65287eee492a7bd0275f77d80
  services-platform: ad3f6a6582843008ed0f5a17b131878d15829af1
  macroservice-libraries: 54536a441935c6e09fdb7f96f0bec71c52ba4e03
  services-crm: a8d1364f1dd44b723bb5f9acba12f983e9ec8d69
  services-membership: 5994c270e8a37a30f848e99ba5b5ee6b87c08893
  react-console: baf6dae382d6f41dce976c7f848667ccc37e076f
branch: agent/platform-data-job-engine-desi-exam-1
examination_status: complete; examined design remains subject to Andrew’s approval
counts:
  artifacts: 65
  domain_objects: 5
  state_models: 6
  workflows: 20
  service_contracts: 6
  screen_contracts: 8
  decision_records: 20
  operations: 164
  invariants: 38
  transitions: 54
  workflow_hops: 94
  source_anchor_paths: 112
  new_findings: 7
  inference_operations_required: 0
provenance: observed
source:
  - >-
    rhythm-v2-docs:specs/CONVENTIONS.md §§6,9; specs/templates;
    specs/platform/data-job-engine/MODULE.md; planning; FINDINGS.md;
    ID_REGISTRY.md; specs/platform/RESPONSIBILITIES.md
  - 'macroservice-libraries:packages/data-jobs/src; packages/data-jobs/README.md'
  - >-
    services-platform:modules/data-jobs; modules/capacity; modules/partitions;
    cited proposal, standard and milestone records
  - 'services-crm:modules/data-jobs; cited CRM-M7 records'
  - 'services-membership:modules/data-jobs'
  - >-
    react-console:app/(console)/console/[tenantId]/dataManagement; cited
    data-management components and data-job clients
validation:
  dependency_install: npm ci passed
  conformance: 'npm run check: zero errors'
  tests: 'npm test: 4 tests passed, 0 failed'
  runtime_services: not executed; examination reads pinned implementation and test sources
findings_added:
  - FND-PFM-DJE-50
  - FND-PFM-DJE-51
  - FND-PFM-DJE-52
  - FND-PFM-DJE-53
  - FND-PFM-DJE-54
  - FND-PFM-DJE-55
  - FND-PFM-DJE-56
---

# Data Job Engine Design Examination

## Scope and Method

All 65 design artifacts and the module’s Platform responsibility row and prose section were examined and rewritten. The examination compared object declarations with persistence and registration, state rows with transition tables and direct writes, workflow hops with their execution paths, service operations with route declarations, and screen behavior with components and clients. Planning artifacts were comparison evidence and were left unchanged.

The source-and-anchor audit opened 112 distinct paths named by invariant, transition, hop and operation evidence. The apparent missing upload test title exists with an escaped apostrophe in its source; composite console mapping evidence was split into individual symbols. This is a source examination, with corpus conformance and checker tests; it does not establish deployed behavior or execute the service integration and browser suites.

The requested design brief is absent, and the planning artifacts still carry draft status and proposed product intent. Approved proposal decisions support particular choices, while approval of the entire planning layer remains unestablished. Those limits are recorded as a finding rather than silently treating the planning layer as approved.

## Counts and Coverage

| Examined kind | Count | Coverage |
|---|---:|---|
| Domain objects | 5 | All six planning concept numbers accounted for; the second concept is superseded, so its number is not reused |
| State models | 6 | 54 transition rows compared with declared tables and direct execution writes |
| Workflows | 20 | 94 hops; interactive and bulk execution, upload, runs, capacity, maintenance and console handoff |
| Service contracts | 6 | 164 HTTP operations: 124 CRM, 36 Platform data-job and 4 capacity operations |
| Screen contracts | 8 | Four routed Data Management surfaces and four hosted bulk-action/dialog surfaces |
| Decision records | 20 | Origins retained; alternatives and consequences rewritten with implementation qualifications |
| Object invariants | 38 | Sources and stated enforcement boundaries examined |
| Inference declarations | 34 | Every service, workflow and screen declares an empty inference list |
| Model tasks | 0 | No model call found in the scoped module execution or its cited console paths; no inference artifact is required |

All 23 physical module tables have an object home: fourteen job/evidence tables across CRM and Platform plus three CRM bulk-execution tables belong to the job object; two producer-mapping tables belong to the source-key object; one upload table, two run tables and one capacity-pin table belong to their corresponding objects. Domain target tables, shared cache and parameter storage, and maintained history tables are dependencies rather than additional module-owned objects. Membership contributes no engine table or HTTP route at its pinned revision.

| Execution surface | Workflow coverage |
|---|---|
| Interactive dispatch, queue delivery and batch processing | Submission, batch execution and control workflows |
| Bulk state machine, admission and bulk worker queue | File-job workflow with shared batch execution |
| Platform request-based runner | Platform small-job workflow |
| Upload storage protocol | Multipart upload workflow |
| Staff-driven run coordination | Registration, driving and cancellation workflows |
| Capacity minute schedule and emergency switch | Sentinel publication and pin workflows |
| Daily partition maintenance and deployment creation pass | Partition workflow |
| Daily CRM submission-key purge | Submission-ledger purge workflow |
| Workspace reset and purge participation | Evidence-removal workflow |

The two CRM processing queues and their dead-letter boundaries are part of the interactive and bulk execution contracts. Scheduled capacity delivery also has a dead-letter resource; this examination does not invent an automatic dead-letter recovery consumer. There is one bulk execution state-machine definition; the other six state models describe object lifecycles, not six deployed orchestration machines.

## Changes by Artifact

Every row below includes a final wording pass over its prose and human-readable statements. Verification commits precede the individual artifact rewrite commits; supplemental wording commits cover relationship, observer and pattern text. Identifiers, field names, planning links, approval values, origins and repository pins retain their identity.

| Artifact | Verification and final text |
|---|---|
| ADR-PFM-DJE-001 | Retained the cited shared-engine decision; rewrote local adoption and Membership limits. |
| ADR-PFM-DJE-002 | Retained transactional receipts; limited duplicate protection to the same job item. |
| ADR-PFM-DJE-003 | Corrected fingerprint approval to the stored preview rather than aggregate impact recomputation. |
| ADR-PFM-DJE-004 | Retained individual refusals and commit splitting; rewrote partial success and guard thresholds. |
| ADR-PFM-DJE-005 | Retained guarded compensation; rewrote whole-record exclusions and unsupported recovery. |
| ADR-PFM-DJE-006 | Qualified protection with the explicit overwrite-policy exception. |
| ADR-PFM-DJE-007 | Corrected retained first/last job identifiers and the concurrent absent-mapping race. |
| ADR-PFM-DJE-008 | Retained shared batch execution while distinguishing input persistence and preview limits. |
| ADR-PFM-DJE-009 | Added the usable one-slot share exception and separated capacity hold from stalled-work failure. |
| ADR-PFM-DJE-010 | Removed the invented prohibition on service identities; described the unimplemented identity alternative. |
| ADR-PFM-DJE-011 | Retained committed loads and reset recovery; rewrote partial reset consequences. |
| ADR-PFM-DJE-012 | Qualified measured versus declared digests and storage-first crash recovery. |
| ADR-PFM-DJE-013 | Corrected direct escalation, active-pin precedence and metric-read versus publication failure. |
| ADR-PFM-DJE-014 | Preserved the open partition-ownership question and rewrote the maintenance alternatives. |
| ADR-PFM-DJE-015 | Retained the opt-in custom-field standard without inventing enabling registrations. |
| ADR-PFM-DJE-016 | Retained explicit unpublished declarations as proposed classification; rewrote alternatives and consequences. |
| ADR-PFM-DJE-017 | Qualified engine keys versus upload/run checks and corrected application grant descriptions. |
| ADR-PFM-DJE-018 | Retained domain-owned history and events; rewrote progress-polling consequences. |
| ADR-PFM-DJE-019 | Retained request execution as observed design and qualified unrecovered claims. |
| ADR-PFM-DJE-020 | Separated decided retention from the only implemented age purge. |
| DOM-PFM-DJE-001 | Qualified receipt totals, group boundaries and state enforcement; corrected relationships and rewrote object descriptions. |
| DOM-PFM-DJE-003 | Distinguished file bytes from completion metadata; corrected digest measurement, state enforcement, abort cleanup and upload reuse. |
| DOM-PFM-DJE-004 | Removed the absent-key locking guarantee and qualified mapping ownership without job foreign keys. |
| DOM-PFM-DJE-005 | Qualified package registration concurrency and corrected reusable-upload relationships. |
| DOM-PFM-DJE-006 | Corrected pin precedence and the one-slot service-share exception. |
| SCR-PFM-DJE-001 | Verified the server area-home redirect; removed the unsupported approval claim and rewrote the entry contract. |
| SCR-PFM-DJE-002 | Added exact-job lookup; preserved the broad page alias versus contacts-route refusal; rewrote list behavior. |
| SCR-PFM-DJE-003 | Corrected target discovery and unfiltered registry offerings; rewrote wizard behavior and patterns. |
| SCR-PFM-DJE-004 | Corrected ordinary Apply confirmation and disabled rejection filtering; rewrote detail and category-routed controls. |
| SCR-PFM-DJE-005 | Corrected visibility to the bulk key alone; rewrote selection-size routing and unsupported actions. |
| SCR-PFM-DJE-006 | Separated host visibility from service update authorization; rewrote validation, conflict revalidation and results. |
| SCR-PFM-DJE-007 | Separated host visibility from service delete authorization; rewrote automatic preview and counted approval. |
| SCR-PFM-DJE-008 | Removed the single-instant query-result promise; rewrote bounded collection and retry. |
| STATE-PFM-DJE-001 | Separated declared intermediate vocabulary from observed writes and added automatic validating-to-paused settlement. |
| STATE-PFM-DJE-002 | Verified pending, claimed, completed, abandoned and transient-exhaustion paths; rewrote guards and effects. |
| STATE-PFM-DJE-003 | Corrected expiry and abort descriptions, including retained assembled objects after verification failure. |
| STATE-PFM-DJE-004 | Qualified driving-request concurrency, terminal behavior and readiness separate from success. |
| STATE-PFM-DJE-005 | Verified step dependency, polling and terminal branches; rewrote meanings and transitions. |
| STATE-PFM-DJE-006 | Added direct normal-to-critical escalation and failed-metric fallback transitions; removed the invented forbidden jump. |
| SVC-PFM-DJE-capacity | Changed staff access to system; removed storage publication from business events and added outbound storage integration. |
| SVC-PFM-DJE-crm-data-jobs | Reconciled 124 factory-expanded operations, permissions and capabilities; rewrote intents and domain-event ownership. |
| SVC-PFM-DJE-data-jobs | Changed staff run access to system; reconciled 36 operations and qualified upload and request-runner behavior. |
| SVC-PFM-DJE-engine | Verified shared execution ports and absence of HTTP routes; rewrote adoption and consistency boundaries. |
| SVC-PFM-DJE-membership-data-jobs | Retained operation declarations without claiming runnable storage, routes or reconciliation; rewrote the limitation. |
| SVC-PFM-DJE-partitions | Verified the daily and deployment passes with no HTTP routes; rewrote schedule and consistency descriptions. |
| WF-PFM-DJE-001 | Corrected simultaneous idempotent submission to conflict rather than an ordinary reuse response. |
| WF-PFM-DJE-002 | Qualified group-boundary and refusal-guard behavior; rewrote all hops and failure descriptions. |
| WF-PFM-DJE-003 | Corrected the lazy-expiry refusal and described stored-fingerprint approval without a fresh aggregate impact. |
| WF-PFM-DJE-004 | Verified state control, abandon and retry paths; rewrote failure and partial-write limits. |
| WF-PFM-DJE-005 | Verified receipt-based compensation and exclusion boundaries; rewrote planning and application behavior. |
| WF-PFM-DJE-006 | Verified conflict-only child validation and unsupported-mode limits; rewrote the workflow. |
| WF-PFM-DJE-007 | Qualified concurrent first producer deliveries and surviving duplicate-domain records. |
| WF-PFM-DJE-008 | Verified chunking, admission, settlement and failure routing; rewrote waits, retry and budget behavior. |
| WF-PFM-DJE-009 | Verified workspace-wide in-request draining and the missing recovery runner; rewrote the execution boundary. |
| WF-PFM-DJE-010 | Qualified upload digest and storage-first failure behavior; rewrote transfer and completion steps. |
| WF-PFM-DJE-011 | Qualified registration concurrency and package-upload reuse; rewrote manifest checks. |
| WF-PFM-DJE-012 | Verified caller-driven polling and dataset submission without cross-service serialization; rewrote the run flow. |
| WF-PFM-DJE-013 | Verified cancellation calls, authorization refusal and logged non-access failures; rewrote retained-load behavior. |
| WF-PFM-DJE-014 | Qualified application grants and owner-authorized evidence removal; rewrote reset and purge boundaries. |
| WF-PFM-DJE-015 | Corrected capacity jumps, pin precedence and separate throttle restoration; rewrote independent publication failures. |
| WF-PFM-DJE-016 | Verified transactional pin audit and eventual sentinel consumption; rewrote expiry and release behavior. |
| WF-PFM-DJE-017 | Verified schedule and deployment passes, independent table failures and retained detached storage; rewrote maintenance. |
| WF-PFM-DJE-018 | Verified CRM submission-ledger age purge and missing Platform/evidence purge; rewrote scope. |
| WF-PFM-DJE-019 | Removed the common-snapshot promise from paged selection collection; rewrote dialog handoff. |
| WF-PFM-DJE-020 | Changed mapping evidence from backend target tests to console metadata derivation; rewrote wizard hops. |

Platform Responsibilities now lists the exact 164 exposed operation identifiers, the five owned objects and an empty business-event list. Its module text delegates record saves, lifecycle coordination, identity and orchestration without promising universal overwrite protection or recovery. The module registry status is design examined; no artifact is approved by this job.

## Changes to Evidence and Technical Values

Verification changed the producer-lock invariant’s provenance and source to observed mapping-write and keyed-apply implementation, qualified job-count and transition-table claims with their actual enforcement paths, and replaced the wizard mapping test source with its console metadata derivation. The upload lifecycle invariant no longer claims database enforcement of service transitions. Capacity transitions gained direct critical escalation and second-failed-read fallback, while the incorrect forbidden escalation was removed; validating-to-paused settlement was added to the job model.

Staff run and capacity-pin operations now declare system access. Upload and run relationships describe reuse, mapping relationships retain job identifiers without foreign keys, ordinary detail-page Apply does not unconditionally require confirmation, and menu/dialog visibility no longer claims update or delete permission gates absent from the console. Capacity cache/parameter writes are outbound integrations rather than business events. Exact-job search was added to the list actions, and rejection-code filtering is identified as unavailable.

No identifier was renumbered, no source revision was advanced, and no approved intent was inferred from code. Existing evidence references were retained except for the verified corrections above; the responsibility artifact gained missing pins for the repositories supplying its module row. Decision origins remain the existing records, including answered planning choices and open questions.

## Open Disagreements and Preparation Gate

Seven new findings cover concurrent first producer delivery, capacity behavior differing from the planning narrative, simultaneous idempotent submission conflict, the absent brief and planning-approval gate, direct state writes outside the declared transition table, non-snapshot query collection, and weaker console permission gates. The existing findings remain open, including activation without readiness checks, request-runner recovery, upload completion recovery, unimplemented evidence retention, incomplete Membership adoption and disabled result tooling.

Before implementation preparation, Andrew needs to establish the planning approval and missing brief, choose the intended first-producer concurrency guarantee and capacity transitions, and decide the required recovery and lifecycle-readiness gates. Source observation supports the corrected account but does not resolve those product disagreements. No planning rule was edited to erase a disagreement.

## Acceptance

Dependency installation and corpus checking passed with zero conformance errors. All four repository checker tests passed with no failures. No service migration, deployment or shared development-data change was made.

## Verdict for Andrew

The examined design is a faithful, qualified account of the pinned module implementation, with unresolved planning differences kept visible as findings. Its weakest areas are concurrent first producer writes and recovery across request, storage and remote-service boundaries, where the intended guarantees exceed the demonstrated implementation. Before implementation preparation starts, approve the planning boundary and decide the required identity, recovery, capacity and activation-readiness guarantees.