--- id: ATS-PFM-DJE-014 scope: CAP-PFM-DJE-014 product_area: platform module: data-job-engine revision: 2 status: proposed approved_by: null approved_at: null source_sha: macroservice-libraries: 54536a441935c6e09fdb7f96f0bec71c52ba4e03 services-crm: a8d1364f1dd44b723bb5f9acba12f983e9ec8d69 scenarios: - id: ATS-PFM-DJE-014-01 title: File submission requires an available completed upload level: api proves: - BR-CAP-PFM-DJE-014-01 preconditions: 'Prepare foreign, absent and still-open upload identifiers.' steps: - do: Submit jobs referencing each. expect: >- Foreign and absent uploads have indistinguishable refusals; unfinished input identifies its state. provenance: test source: >- services-crm:modules/data-jobs/tests/e2e/bulk-data-jobs.e2e.spec.js#refuses an upload of another tenant with 400, saying only that it was not found expected: >- Foreign and absent uploads have indistinguishable refusals; unfinished input identifies its state. negative: true data_needs: Upload fixtures. bound_to: - TST-PFM-DJE-107 provenance: test source: >- services-crm:modules/data-jobs/tests/e2e/bulk-data-jobs.e2e.spec.js#refuses an upload of another tenant with 400, saying only that it was not found category: validation binding_evidence: - >- services-crm:modules/data-jobs/tests/e2e/bulk-data-jobs.e2e.spec.js#refuses an upload of another tenant with 400, saying only that it was not found - >- services-crm:modules/data-jobs/tests/e2e/bulk-data-jobs.e2e.spec.js#refuses an upload id of no upload at all in the same words, so neither answer tells a caller whose it is - >- services-crm:modules/data-jobs/tests/e2e/bulk-data-jobs.e2e.spec.js#refuses an upload whose file has not arrived with 400, naming the state it is in - id: ATS-PFM-DJE-014-02 title: Supported file formats are explicit level: api proves: - BR-CAP-PFM-DJE-014-02 preconditions: 'Prepare gzipped CSV, plain JSON document and zip input.' steps: - do: Attempt file submission. expect: CSV is read through its header and unsupported formats are refused. provenance: test source: >- macroservice-libraries:packages/data-jobs/src/tests/integration/bulk-data-jobs.integration.spec.ts#reads a gzipped CSV through its header, which is the other shape a producer exports expected: CSV is read through its header and unsupported formats are refused. negative: true data_needs: Files in the specified formats. bound_to: - TST-PFM-DJE-024 - TST-PFM-DJE-065 provenance: test source: >- macroservice-libraries:packages/data-jobs/src/tests/integration/bulk-data-jobs.integration.spec.ts#reads a gzipped CSV through its header, which is the other shape a producer exports category: boundary binding_evidence: - >- macroservice-libraries:packages/data-jobs/src/tests/integration/bulk-data-jobs.integration.spec.ts#reads a gzipped CSV through its header, which is the other shape a producer exports - >- react-console:acceptance/crm/playwright/journeys/data-operations/CRM-ACC-0666.spec.ts#CRM-ACC-0666 - id: ATS-PFM-DJE-014-03 title: Chunk targets yield to atomic groups level: unit proves: - BR-CAP-PFM-DJE-014-03 preconditions: Supply grouped and ungrouped lines of different sizes. steps: - do: Plan chunk boundaries. expect: >- Limits close eligible chunks, but oversized first records and groups already present can exceed targets. provenance: test source: >- macroservice-libraries:packages/data-jobs/src/chunk-plan.spec.ts#closes the chunk when one more item would pass the item limit expected: >- Limits close eligible chunks, but oversized first records and groups already present can exceed targets. negative: false data_needs: No additional fixture is needed. bound_to: - TST-PFM-DJE-008 provenance: test source: >- macroservice-libraries:packages/data-jobs/src/chunk-plan.spec.ts#closes the chunk when one more item would pass the item limit category: edge binding_evidence: - >- macroservice-libraries:packages/data-jobs/src/chunk-plan.spec.ts#closes the chunk when one more item would pass the item limit - >- macroservice-libraries:packages/data-jobs/src/chunk-plan.spec.ts#keeps a group whole, even past both limits, because a group divided could never commit as one unit - >- macroservice-libraries:packages/data-jobs/src/chunk-plan.spec.ts#never closes an empty chunk, so an item larger than the whole limit is written alone rather than nowhere - >- macroservice-libraries:packages/data-jobs/src/chunk-plan.spec.ts#holds a group it has seen anywhere in the chunk, not only on its last line - id: ATS-PFM-DJE-014-04 title: Malformed input is line-specific until the file guard trips level: api proves: - BR-CAP-PFM-DJE-014-04 preconditions: Prepare a thousand-line file with three bad lines and a mostly malformed file. steps: - do: Process both. expect: The first reports 997 applications and three numbered refusals; the second fails. provenance: test source: >- macroservice-libraries:packages/data-jobs/src/tests/integration/bulk-data-jobs.integration.spec.ts#reads a thousand-line file into chunks, applies 997 records and rejects three lines by their line number expected: The first reports 997 applications and three numbered refusals; the second fails. negative: true data_needs: The two input files. bound_to: - TST-PFM-DJE-024 provenance: test source: >- macroservice-libraries:packages/data-jobs/src/tests/integration/bulk-data-jobs.integration.spec.ts#reads a thousand-line file into chunks, applies 997 records and rejects three lines by their line number category: failure binding_evidence: - >- macroservice-libraries:packages/data-jobs/src/tests/integration/bulk-data-jobs.integration.spec.ts#reads a thousand-line file into chunks, applies 997 records and rejects three lines by their line number - >- macroservice-libraries:packages/data-jobs/src/tests/integration/bulk-data-jobs.integration.spec.ts#fails the job when most of the file cannot be read, rather than reading nine million lines of it - id: ATS-PFM-DJE-014-05 title: Admission observes all remaining budgets level: api proves: - BR-CAP-PFM-DJE-014-05 preconditions: Prepare competing jobs in two workspaces. steps: - do: Request admission. expect: Total admitted work stays within the budget and the fixture shares room across workspaces. provenance: test source: >- macroservice-libraries:packages/data-jobs/src/tests/integration/bulk-data-jobs.integration.spec.ts#hands over no more batches at once than the budget allows expected: Total admitted work stays within the budget and the fixture shares room across workspaces. negative: false data_needs: Competing-job fixtures. bound_to: - TST-PFM-DJE-024 - TST-PFM-DJE-112 provenance: test source: >- macroservice-libraries:packages/data-jobs/src/tests/integration/bulk-data-jobs.integration.spec.ts#hands over no more batches at once than the budget allows category: rule binding_evidence: - >- macroservice-libraries:packages/data-jobs/src/tests/integration/bulk-data-jobs.integration.spec.ts#hands over no more batches at once than the budget allows - >- services-crm:modules/data-jobs/tests/integration/services/jobs/bulk-admission.integration.spec.js#admits nothing more to the first tenant once the two tenants hold the whole ceiling - >- services-crm:modules/data-jobs/tests/integration/services/jobs/bulk-admission.integration.spec.js#leaves each tenant holding two batches of the ceiling’s four - id: ATS-PFM-DJE-014-06 title: Admission priority and rotation prevent unfair grants level: unit proves: - BR-CAP-PFM-DJE-014-06 preconditions: Prepare waiting jobs across workspaces with varied ages. steps: - do: Calculate admission grants. expect: Grants rotate by workspace rather than number of jobs and prioritize the starved job. provenance: test source: >- macroservice-libraries:packages/data-jobs/src/admission-planner.spec.ts#gives the slots in turn to each tenant with batches waiting expected: Grants rotate by workspace rather than number of jobs and prioritize the starved job. negative: false data_needs: No additional fixture is needed. bound_to: - TST-PFM-DJE-002 provenance: test source: >- macroservice-libraries:packages/data-jobs/src/admission-planner.spec.ts#gives the slots in turn to each tenant with batches waiting category: concurrency binding_evidence: - >- macroservice-libraries:packages/data-jobs/src/admission-planner.spec.ts#gives the slots in turn to each tenant with batches waiting - >- macroservice-libraries:packages/data-jobs/src/admission-planner.spec.ts#gives a tenant with five jobs no more than a tenant with one (scenario 13) - >- macroservice-libraries:packages/data-jobs/src/admission-planner.spec.ts#puts a starved job ahead of the round-robin - id: ATS-PFM-DJE-014-07 title: Stalled blocking work produces a failure level: api proves: - BR-CAP-PFM-DJE-014-07 preconditions: >- Hold the admission ceiling with batches that never advance and configure a ten-minute stall interval. steps: - do: Continue admission attempts beyond the configured ten-minute interval. expect: The job fails as stalled and identifies the blocking ceiling. provenance: test source: >- macroservice-libraries:packages/data-jobs/src/tests/integration/bulk-data-jobs.integration.spec.ts#fails a job whose ceiling has been held by work that is not moving, rather than waiting for room for ever expected: The job fails as stalled and identifies the blocking ceiling. negative: true data_needs: A fixture-controlled clock. bound_to: - TST-PFM-DJE-024 provenance: test source: >- macroservice-libraries:packages/data-jobs/src/tests/integration/bulk-data-jobs.integration.spec.ts#fails a job whose ceiling has been held by work that is not moving, rather than waiting for room for ever category: failure binding_evidence: - >- macroservice-libraries:packages/data-jobs/src/tests/integration/bulk-data-jobs.integration.spec.ts#fails a job whose ceiling has been held by work that is not moving, rather than waiting for room for ever - id: ATS-PFM-DJE-014-08 title: Pausing releases admission room level: api proves: - BR-CAP-PFM-DJE-014-08 preconditions: Pause a job occupying its workspace ceiling. steps: - do: Request admission for another job. expect: The waiting job receives available room. provenance: test source: >- macroservice-libraries:packages/data-jobs/src/tests/integration/bulk-data-jobs.integration.spec.ts#admits the next job when the one holding the tenant’s ceiling has been paused, because a paused batch is not in flight expected: The waiting job receives available room. negative: false data_needs: No additional fixture is needed. bound_to: - TST-PFM-DJE-024 provenance: test source: >- macroservice-libraries:packages/data-jobs/src/tests/integration/bulk-data-jobs.integration.spec.ts#admits the next job when the one holding the tenant’s ceiling has been paused, because a paused batch is not in flight category: transition binding_evidence: - >- macroservice-libraries:packages/data-jobs/src/tests/integration/bulk-data-jobs.integration.spec.ts#admits the next job when the one holding the tenant’s ceiling has been paused, because a paused batch is not in flight - id: ATS-PFM-DJE-014-09 title: Distant and resumed group repeats expose the current grouping limit category: invariant level: api proves: - BR-CAP-PFM-DJE-014-11 preconditions: >- Prepare a file with one group repeated immediately, after two closed chunks, and after a source-reader checkpoint. steps: - do: Read the file through each placement and inspect refused lines and planned groups. expect: >- The immediate repeat is refused; distant and resumed repeats can enter separate chunks, exposing the bounded detection limit. provenance: proposed source: 'macroservice-libraries:packages/data-jobs/src/chunk-source.ts:closedGroupKeys' expected: >- The immediate repeat is refused; distant and resumed repeats can enter separate chunks, exposing the bounded detection limit. negative: true data_needs: >- Prepare a file with one group repeated immediately, after two closed chunks, and after a source-reader checkpoint. bound_to: [] provenance: proposed source: 'macroservice-libraries:packages/data-jobs/src/chunk-source.ts:closedGroupKeys' - id: ATS-PFM-DJE-014-10 title: Only a completed workspace upload supplies file work category: interaction level: api exercises: - CAP-PFM-DJE-011 - CAP-PFM-DJE-014 proves: - BR-CAP-PFM-DJE-011-07 - BR-CAP-PFM-DJE-011-09 - BR-CAP-PFM-DJE-014-01 preconditions: 'Create open, completed and foreign-workspace upload sessions.' steps: - do: Submit equivalent file jobs against each session and process the accepted job. expect: >- Only the completed local session supplies input; unavailable sessions create no processing work and foreign input is concealed. provenance: proposed source: >- services-platform:modules/data-jobs/tests/integration/services/uploads/upload-sessions.integration.spec.js#refuses to abort a completed session, whose object a job may already be reading; services-platform:modules/data-jobs/tests/integration/services/uploads/upload-sessions.integration.spec.js#refuses to complete another tenant’s session, and makes no S3 call for it; services-platform:modules/data-jobs/services/uploads/upload-sessions.ts:presignUploadParts; services-crm:modules/data-jobs/tests/e2e/bulk-data-jobs.e2e.spec.js#refuses an upload id of no upload at all in the same words, so neither answer tells a caller whose it is expected: >- Only the completed local session supplies input; unavailable sessions create no processing work and foreign input is concealed. negative: true data_needs: 'Create open, completed and foreign-workspace upload sessions.' bound_to: [] provenance: proposed source: >- services-platform:modules/data-jobs/tests/integration/services/uploads/upload-sessions.integration.spec.js#refuses to abort a completed session, whose object a job may already be reading; services-platform:modules/data-jobs/tests/integration/services/uploads/upload-sessions.integration.spec.js#refuses to complete another tenant’s session, and makes no S3 call for it; services-platform:modules/data-jobs/services/uploads/upload-sessions.ts:presignUploadParts; services-crm:modules/data-jobs/tests/e2e/bulk-data-jobs.e2e.spec.js#refuses an upload id of no upload at all in the same words, so neither answer tells a caller whose it is - id: ATS-PFM-DJE-014-11 title: File processing preserves keyed replay outcomes category: interaction level: api exercises: - CAP-PFM-DJE-012 - CAP-PFM-DJE-014 proves: - BR-CAP-PFM-DJE-012-02 - BR-CAP-PFM-DJE-012-03 - BR-CAP-PFM-DJE-014-03 - BR-CAP-PFM-DJE-014-10 preconditions: Prepare a multi-chunk revisioned JSON Lines file. steps: - do: >- Process it, redeliver a committed chunk, then submit the same file as another keyed import. expect: >- Chunk redelivery creates no repeated writes and the second import reports duplicate outcomes for applied revisions. provenance: proposed source: >- macroservice-libraries:packages/data-jobs/src/tests/integration/keyed-data-jobs.integration.spec.ts#skips a delivery a newer one has overtaken, so a late file never undoes the current record; macroservice-libraries:packages/data-jobs/src/tests/integration/keyed-data-jobs.integration.spec.ts#writes no crosswalk row for an item it refused, so the corrected delivery of that key is a create; macroservice-libraries:packages/data-jobs/src/chunk-plan.ts:shouldCloseChunk; macroservice-libraries:packages/data-jobs/src/chunk-source.ts:ChunkRun; services-crm:docs/data-jobs.md#The bulk lane expected: >- Chunk redelivery creates no repeated writes and the second import reports duplicate outcomes for applied revisions. negative: false data_needs: Prepare a multi-chunk revisioned JSON Lines file. bound_to: [] provenance: proposed source: >- macroservice-libraries:packages/data-jobs/src/tests/integration/keyed-data-jobs.integration.spec.ts#skips a delivery a newer one has overtaken, so a late file never undoes the current record; macroservice-libraries:packages/data-jobs/src/tests/integration/keyed-data-jobs.integration.spec.ts#writes no crosswalk row for an item it refused, so the corrected delivery of that key is a create; macroservice-libraries:packages/data-jobs/src/chunk-plan.ts:shouldCloseChunk; macroservice-libraries:packages/data-jobs/src/chunk-source.ts:ChunkRun; services-crm:docs/data-jobs.md#The bulk lane - id: ATS-PFM-DJE-014-12 title: A critical protection factor holds admission without stalled failure category: interaction level: api exercises: - CAP-PFM-DJE-014 - CAP-PFM-DJE-019 proves: - BR-CAP-PFM-DJE-014-05 - BR-CAP-PFM-DJE-014-07 - BR-CAP-PFM-DJE-019-03 preconditions: Prepare a file job with waiting batches and a readable capacity factor of zero. steps: - do: >- Attempt admission beyond fifteen minutes, then publish normal capacity and attempt it again. expect: >- Zero capacity holds the job without a stalled failure; restored capacity admits work within the remaining ceilings. provenance: proposed source: >- macroservice-libraries:packages/data-jobs/src/tests/integration/bulk-data-jobs.integration.spec.ts#hands over no more batches at once than the budget allows; macroservice-libraries:packages/data-jobs/src/admission-budget.ts:tenantBulkCeiling; macroservice-libraries:packages/data-jobs/src/tests/integration/bulk-data-jobs.integration.spec.ts#fails a job whose ceiling has been held by work that is not moving, rather than waiting for room for ever; services-platform:modules/capacity/tests/unit/services/capacity/decide-state.spec.ts#stops bulk and quarters heavy in critical, keeping writes and reads whole expected: >- Zero capacity holds the job without a stalled failure; restored capacity admits work within the remaining ceilings. negative: true data_needs: Prepare a file job with waiting batches and a readable capacity factor of zero. bound_to: [] provenance: proposed source: >- macroservice-libraries:packages/data-jobs/src/tests/integration/bulk-data-jobs.integration.spec.ts#hands over no more batches at once than the budget allows; macroservice-libraries:packages/data-jobs/src/admission-budget.ts:tenantBulkCeiling; macroservice-libraries:packages/data-jobs/src/tests/integration/bulk-data-jobs.integration.spec.ts#fails a job whose ceiling has been held by work that is not moving, rather than waiting for room for ever; services-platform:modules/capacity/tests/unit/services/capacity/decide-state.spec.ts#stops bulk and quarters heavy in critical, keeping writes and reads whole evidence: - >- Reviewed evidence: macroservice-libraries (54536a44): README.md (The bulk lane); src/chunk-source.ts, chunk-plan.ts, bulk-data-job-tasks.ts, admission-budget.ts, admission-planner.ts, bulk-batch-budget.ts, bulk-data-job-workflow.ts; tests chunk-plan.spec.ts, admission-budget.spec.ts, admission-planner.spec.ts, admit-batches.spec.ts, bulk-data-jobs.integration.spec.ts. - >- Reviewed evidence: services-crm (a8d1364f): docs/data-jobs.md (The bulk lane); services/jobs/bulk-data-job-submission.ts, bulk-admission.ts, bulk-data-job.ts; services/persistence/tenant-workload-policies.ts; db/bulk-data-jobs.table.ts; tests bulk-admission.integration.spec.js, bulk-data-job-lane.integration.spec.js, bulk-data-jobs.integration.spec.js, bulk-data-jobs.e2e.spec.js. - >- Testing back-fill: related capability rules, domain invariants, state transitions and crossing workflows at the pinned source revisions. coverage: rule: covered transition: 'gap: FND-PFM-DJE-074' validation: covered permission: 'gap: FND-PFM-DJE-074' edge: covered failure: covered boundary: covered concurrency: covered invariant: covered interaction: covered coverage_notes: transition: >- Selected lifecycle examples do not cover every applicable object and batch transition. The linked finding retains the remaining matrix. execution: >- Covered means a specification scenario exists. Proposed scenarios and registered suites with no run result do not establish executed conformance. related: - CAP-PFM-DJE-011 - CAP-PFM-DJE-012 - CAP-PFM-DJE-019 workflows: - WF-PFM-DJE-008 - WF-PFM-DJE-015 coverage_evidence: - rule: BR-CAP-PFM-DJE-014-01 scenarios: - ATS-PFM-DJE-014-01 - ATS-PFM-DJE-014-10 provenance: proposed source: >- services-crm:modules/data-jobs/tests/e2e/bulk-data-jobs.e2e.spec.js#refuses an upload id of no upload at all in the same words, so neither answer tells a caller whose it is - rule: BR-CAP-PFM-DJE-014-02 scenarios: - ATS-PFM-DJE-014-02 provenance: proposed source: 'services-crm:docs/data-jobs.md#The bulk lane' - rule: BR-CAP-PFM-DJE-014-03 scenarios: - ATS-PFM-DJE-014-03 - ATS-PFM-DJE-014-11 provenance: proposed source: >- macroservice-libraries:packages/data-jobs/src/chunk-plan.ts:shouldCloseChunk; macroservice-libraries:packages/data-jobs/src/chunk-source.ts:ChunkRun - rule: BR-CAP-PFM-DJE-014-04 scenarios: - ATS-PFM-DJE-014-04 provenance: proposed source: >- macroservice-libraries:packages/data-jobs/src/tests/integration/bulk-data-jobs.integration.spec.ts#fails the job when most of the file cannot be read, rather than reading nine million lines of it - rule: BR-CAP-PFM-DJE-014-05 scenarios: - ATS-PFM-DJE-014-05 - ATS-PFM-DJE-014-12 provenance: proposed source: >- macroservice-libraries:packages/data-jobs/src/tests/integration/bulk-data-jobs.integration.spec.ts#hands over no more batches at once than the budget allows; macroservice-libraries:packages/data-jobs/src/admission-budget.ts:tenantBulkCeiling - rule: BR-CAP-PFM-DJE-014-06 scenarios: - ATS-PFM-DJE-014-06 provenance: proposed source: >- macroservice-libraries:packages/data-jobs/src/admission-planner.spec.ts#puts a starved job ahead of the round-robin - rule: BR-CAP-PFM-DJE-014-07 scenarios: - ATS-PFM-DJE-014-07 - ATS-PFM-DJE-014-12 provenance: proposed source: >- macroservice-libraries:packages/data-jobs/src/tests/integration/bulk-data-jobs.integration.spec.ts#fails a job whose ceiling has been held by work that is not moving, rather than waiting for room for ever - rule: BR-CAP-PFM-DJE-014-08 scenarios: - ATS-PFM-DJE-014-08 provenance: proposed source: >- macroservice-libraries:packages/data-jobs/src/tests/integration/bulk-data-jobs.integration.spec.ts#admits the next job when the one holding the tenant’s ceiling has been paused, because a paused batch is not in flight - rule: BR-CAP-PFM-DJE-014-11 scenarios: - ATS-PFM-DJE-014-09 provenance: proposed source: 'macroservice-libraries:packages/data-jobs/src/chunk-source.ts:closedGroupKeys' --- # Process file jobs within shared workload limits: Capability Test Specification ## Coverage The structured coverage rows link required rules to their scenarios. A covered category has a defined check; executable coverage and successful execution require separate evidence. The category map keeps incomplete lifecycle matrices and other missing checks visible as findings. ## Interactions With Related Capabilities Only a completed workspace upload supplies file work. File processing preserves keyed replay outcomes. A critical protection factor holds admission without stalled failure. The interaction scenarios use the related capabilities' rule references and identify the shared fixture and observable outcome.