Skip to content

CAP-PFM-DJE-016Load and reconcile a migration package across its owning applications

Markdown

CAP-PFM-DJE-016

Load and reconcile a migration package across its owning applications

Markdown
  • DocumentDraft
  • Product intentProposed
  • ImplementationObserved

KindCapabilityProduct areaPlatformModuleData Job EngineRevision2

Read atservices-platform ad3f6a65services-crm a8d1364f

The Formal Contract

The narrative above explains; the sections below are authoritative — the rules, scenarios and tests this document binds.

Outcomes
Invalid package declarations are caught before dataset jobs begin.Dependent datasets wait for the records they reference.Staff can distinguish a finished load from a load ready for customer use.
Surfaces
api
Intent sources
services-platform:docs/milestones/PLAT-M19-data-job-engine-foundations.md#D1services-platform:docs/milestones/PLAT-M19-data-job-engine-foundations.md#D2services-platform:docs/milestones/PLAT-M19-data-job-engine-foundations.md#D5services-platform:docs/milestones/plans/PLAT-M19-plan.md#S22services-platform:docs/milestones/plans/PLAT-M22-plan.md#A2services-platform:docs/milestones/plans/PLAT-M22-plan.md#A4services-platform:docs/milestones/PLAT-M22-migration-proof-and-close-out.md#D3

Controls

38 controls apply, from the corpus control catalogue.

TC-AVL-004 TC-AZ-001 TC-AZ-002 TC-AZ-003 TC-AZ-004 TC-AZ-005 TC-CHG-001 TC-CHG-002 TC-CHG-003 TC-CHG-004 TC-CHG-005 TC-CHG-006 TC-CHG-007 TC-DAT-001 TC-DAT-003 TC-DAT-006 TC-IAM-001 TC-IAM-002 TC-IAM-003 TC-IAM-004 TC-IAM-005 TC-LOG-001 TC-LOG-004 TC-LOG-005 TC-MON-001 TC-PI-001 TC-PI-002 TC-PI-003 TC-PI-004 TC-PI-005 TC-PRV-001 TC-PRV-002 TC-TEN-001 TC-TEN-002 TC-TEN-003 TC-VEN-001 TC-VEN-002 TC-VEN-003

Permissions

KeyChecked atScope
hq.data-runs.manageapiglobal
platform.custom-fields.bulkservicetenant

Business Rules

14 rules, 13 to be proved before approval. A rule's product intent is derived from its origin: 0 confirmed by a product decision, the rest inheriting this capability's proposed.

IdRuleKindEnforced atProvenance
01must prove
Proposed
Data Run routes require the staff import-management permission and remain available during workspace loading.
permissionapiobserved
02must prove
Proposed
Registering an initial migration requires both implementation status and loading state; other workspace states produce a named refusal.
validationservicetest
03must prove
Proposed
Before saving a run, registration checks completed uploads against declared dataset sizes and available digests, requires an upload for every known dataset, and checks the destination workspace and any configured environment.
validationserviceobserved
04must prove
Proposed
A package contains at most one hundred datasets, arranged into dependency waves that allow independent datasets together; missing dependencies and cycles prevent registration.
calculationservicetest
services-platform:modules/data-jobs/tests/integration/services/runs/register-run.integration.spec.js#refuses a package whose datasets cannot be ordered, before any row of it is written
05must prove
Proposed
Registration rejects a package when it finds an unfinished run for that package in the same workspace, and allows registration again after that run reaches a final state.
validationservicetest
services-platform:modules/data-jobs/tests/integration/services/runs/register-run.integration.spec.js#refuses a second registration of a package whose run has not finished, naming the run that holds it
06must prove
Proposed
Starting, resuming or reading a running run drives progress by copying job results, stopping at the first failed dataset, skipping pending work on failure and submitting datasets whose dependencies have finished.
lifecycleservicetest
services-platform:modules/data-jobs/tests/integration/lifecycle/data-run-guarantees.integration.spec.js#leaves the run where it stood while nobody reads it, though a dataset finished in its service
07must prove
Proposed
Each dataset becomes a job in its owning application under the requesting staff member’s authority; Platform datasets run locally with the same access checks, and an access refusal preserves the run state.
permissionservicetest
services-platform:modules/data-jobs/tests/integration/services/runs/run-driver.integration.spec.js#forwards the staff caller’s own token with every call, and invents no identity of its own
08must prove
Proposed
A dataset ending with blocked records receives one further submission after its dependencies finish, then remains blocked if unresolved while dependent datasets may continue.
policyservicetest
09must prove
Proposed
Pause prevents further dataset submissions, Resume requires a paused run and Cancel attempts to stop running jobs and skips pending work without undoing committed records; final run states permit no further transition.
lifecycleservicetest
services-platform:modules/data-jobs/tests/integration/services/runs/run-driver.integration.spec.js#cancels the jobs a cancelled run had already submitted, in their own services
10must prove
Proposed
The run copies each job’s counts and adds them into totals, while its separate readiness verdict lists unfinished or validation-only work, failed or cancelled runs, failed datasets, unresolved rejections, blocked records and shortfalls against declared counts.
calculationservicetest
services-platform:modules/data-jobs/tests/unit/run-summary.spec.ts#is not ready when a dataset’s counts do not add up to what the producer claimed
11must prove
Proposed
A permanent failure to read a running job fails its dataset and then the run, whereas a submission exception reaches the caller and can leave the run unfinished.
policyserviceobserved
12
Proposed
Data Runs use configured application connections for Platform and CRM.
policyserviceobserved
13must prove
Proposed
A downstream access-denied exception interrupts cancellation, but other cancellation exceptions are logged and ignored, allowing the run to become cancelled while a child job continues.
lifecycleserviceobserved
14must prove
Proposed
A transient job-read failure preserves the step during the thirty-minute grace period measured from its last successful progress write; a failed read beyond that period fails the step.
lifecycleserviceobserved

Scenarios

IdGivenWhenThenProvenance
01A test package contains organizations, contacts and a further dataset with declared dependencies.Implementation staff start the run and continue polling its progress.The datasets finish in dependency order and the fully accounted test load has no readiness blockers.test
services-platform:modules/data-jobs/tests/integration/lifecycle/data-run-guarantees.integration.spec.js#accounts for every record the package claimed, and finds the tenant ready

The Markdown files under specs/ in rhythmsoftware/rhythm-v2-docs are the canonical source of truth. This reader is a generated, read-only projection of them.