--- id: DOM-PFM-DJE-006 title: Workload Capacity realises: CON-PFM-DJE-006 product_area: platform module: data-job-engine revision: 1 status: draft design_intent: confirmed origin: >- services-platform:docs/proposals/api-workload-governance/README.md (decisions D1 to D10 approved by Andrew 2026-10-17; brief 3.2, the capacity sentinel; brief 3.1, bulk admission); services-platform:docs/workload-governance.md (the standard). Every threshold and budget is a proposed starting value until measured. implementation_evidence: observed source_sha: services-platform: ad3f6a6582843008ed0f5a17b131878d15829af1 macroservice-libraries: 54536a441935c6e09fdb7f96f0bec71c52ba4e03 services-crm: a8d1364f1dd44b723bb5f9acba12f983e9ec8d69 owned_by: SVC-PFM-DJE-capacity readable_by: - SVC-PFM-DJE-crm-data-jobs - SVC-PFM-DJE-engine lifecycle: STATE-PFM-DJE-006 custom_fields: none identity: >- One published protection state describes the shared database environment. A singleton staff pin can override it; setting another pin replaces the existing value. attributes: - name: protection state kind: enum required: true computed: true notes: 'The minute decision selects normal, constrained, critical or emergency.' - name: workload factors kind: computed required: true computed: true notes: >- State factors reduce bulk and heavy work first; emergency also halves write and read factors. - name: sentinel memory kind: collection required: true computed: true notes: >- Persisted sentinel memory retains the previous state, timing, throttle status and failed-read count. - name: latency baseline kind: computed required: true computed: true notes: >- Hourly baseline calculations provide comparison values for each API’s observed latency. - name: staff pin kind: collection required: false computed: false notes: >- An optional staff override records state, reason, expiry within 24 hours and actor. - name: emergency stage throttles kind: enum required: true computed: false notes: >- A separate switch controls emergency API stage reductions and is disabled by default. - name: bulk batch budget kind: number required: true computed: false notes: >- Configured cluster and service budgets bound file batches; workspace profiles are capped below shares greater than one. A share of one remains usable. relationships: - to: DOM-PFM-DJE-001 cardinality: 'one-to-many: the published bulk factor is consumed by multiple admissions' ownership: >- The sentinel publishes capacity; adopting services enforce admission locally. cross_service: true invariants: - id: INV-DOM-PFM-DJE-006-01 statement: >- Automatic escalation uses three supporting readings, recovery uses five calmer readings, and automatic changes observe five-minute spacing. enforced_at: - service provenance: test source: >- services-platform:modules/capacity/tests/unit/services/capacity/decide-state.spec.ts#makes no second change within five minutes of the last - id: INV-DOM-PFM-DJE-006-02 statement: >- Automatic emergency requires critical first, although pins can select any state; scaling alone never classifies a reading above constrained. enforced_at: - service provenance: observed source: >- services-platform:modules/capacity/services/capacity/decide-state.ts:assessReading - id: INV-DOM-PFM-DJE-006-03 statement: >- Without an active pin, one failed signal read holds the prior state and the next restores normal; failed pin reads are treated as no override. enforced_at: - service provenance: observed source: >- services-platform:modules/capacity/services/capacity/decide-state.ts:decideCapacityState - id: INV-DOM-PFM-DJE-006-04 statement: >- A pin must name a valid state, nonblank reason and future expiry within 24 hours, and invalid input leaves the existing value intact. enforced_at: - api - service - database provenance: test source: >- services-platform:modules/capacity/tests/integration/services/persistence/capacity-state-pins.integration.spec.js#refuses an expiry more than a day ahead, naming expires_at - id: INV-DOM-PFM-DJE-006-05 statement: >- Pin replacement and removal append audit evidence of their before and after values. enforced_at: - service provenance: test source: >- services-platform:modules/capacity/tests/integration/services/pins/capacity-pins.integration.spec.js#records the pin and its removal, each with what it was and what it became - id: INV-DOM-PFM-DJE-006-06 statement: >- Admission caps each workspace below a service share greater than one, preserves an indivisible share of one, and waits without failure when the capacity factor is zero. enforced_at: - service provenance: observed source: >- macroservice-libraries:packages/data-jobs/src/admission-budget.ts:tenantBulkCeiling - id: INV-DOM-PFM-DJE-006-07 statement: >- Cache and parameter publications are independent, and the sentinel invocation fails only when both channels fail. enforced_at: - service provenance: documented source: >- services-platform:modules/capacity/services/capacity/run-sentinel.ts (module comment, "Failure handling") physical: - capacity_state_pins - the capacity cache key and parameter (published state) - 'tenant_workload_policies (workspace bulk profile, read only)' evidence: - >- services-platform at ad3f6a65: modules/capacity/services/capacity/decide-state.ts, run-sentinel.ts, publish-state.ts, apply-emergency-throttles.ts, capacity-signals.ts, capacity-baseline.ts; services/pins/capacity-pins.ts; db/capacity-state-pins.table.ts; aws/api/capacity.routes.ts; aws/logic/capacity-sentinel.lambda.ts; docs/workload-governance.md. - >- macroservice-libraries at 54536a44: packages/data-jobs/src/admission-budget.ts, admission-planner.ts, bulk-batch-budget.ts; README.md#The bulk lane. - 'services-crm at a8d1364f: modules/data-jobs/services/jobs/bulk-admission.ts.' --- # Workload Capacity ## What It Models Workload capacity represents the database protection decision used to limit background processing. A scheduled sentinel derives the state from pressure readings, while staff can temporarily override it during an incident. Job admission turns the published factors into batch budgets. ## Identity and Attributes The environment shares one state and one optional staff pin. Sentinel memory gives readings a timing context, and latency baselines distinguish ordinary API performance from pressure. Configured budgets and an emergency-stage switch accompany the decision; the switch starts disabled. ## Relationships CRM file admission consumes the published bulk factor and workspace profile limits. This object describes protection decisions and budgets; consuming services remain responsible for applying them. Gateway limits and licensing retain their own owners. ## Invariants Timing filters reduce rapid oscillation. Automatic escalation can jump directly to critical, while emergency requires prior critical state. An active pin takes precedence over signal failure; absent a pin, repeated failed reads restore normal. Admission preserves a share of one as a specific exception to the below-share cap. ## How It Is Stored and Shared Platform owns the pin table and sentinel publication. Staff manage pins through system routes; the scheduled sentinel has reserved concurrency of one. Consumers read cache or parameter values through shared adapters. The decision and pin records declare no custom-field capability. ## Open Design Questions Thresholds and budgets remain starting values requiring measurement. Product must decide whether to enable emergency stage changes. Capacity wait presentation and the long-term home of partition maintenance also remain open.