--- id: CAP-PFM-DJE-020 title: Maintain and retire monthly history partitions feature: FEAT-PFM-DJE-06 product_area: platform module: data-job-engine revision: 1 status: draft product_intent: proposed implementation_evidence: observed supersedes: [] source_sha: services-platform: ad3f6a6582843008ed0f5a17b131878d15829af1 macroservice-libraries: 54536a441935c6e09fdb7f96f0bec71c52ba4e03 intent_sources: [] purpose: >- Scheduled maintenance prepares monthly storage for registered history tables and retires expired partitions through their archive contracts. outcomes: - Registered tables have current and future monthly partitions. - Expired partitions follow archive-before-drop retirement. - Per-table failures are visible without stopping other table passes. concepts: - CON-PFM-DJE-006 actors: - ACT-PFM-012 surfaces: - job preconditions: [] postconditions: [] permissions: [] rules: - id: BR-CAP-PFM-DJE-020-01 proved_by: [ATS-PFM-DJE-019-11, ATS-PFM-DJE-020-01, ATS-PFM-DJE-020-05, TST-PFM-DJE-194] statement: >- The daily 03:17 UTC schedule ensures the current month and next two months for registered tables, leaving existing partitions intact. kind: lifecycle enforced_at: - service - database must_prove: true provenance: observed source: >- services-platform:modules/partitions/tests/integration/services/maintenance/maintain-partitions.integration.spec.js#creates the current month and the two after it on the first run, and drops nothing; services-platform:modules/partitions/aws/logic/partitions-maintenance.schedule.ts:createPartitionsMaintenanceSchedule origin: none - id: BR-CAP-PFM-DJE-020-02 proved_by: [ATS-PFM-DJE-020-02, TST-PFM-DJE-194] statement: >- Expired partitions follow detach, archive and drop ordering; the helper contract retains detached secured data after archive failure for a later attempt. kind: lifecycle enforced_at: - service - database must_prove: true provenance: documented source: 'services-platform:docs/partitions.md#The helpers' origin: none - id: BR-CAP-PFM-DJE-020-03 proved_by: [ATS-PFM-DJE-020-03] statement: >- The partition helper contract creates a child with its parent’s row security, policies and privileges in the same transaction. kind: invariant enforced_at: - database must_prove: true provenance: documented source: 'services-platform:docs/partitions.md#A partition is secured like its parent' origin: none - id: BR-CAP-PFM-DJE-020-04 proved_by: [ATS-PFM-DJE-019-11, ATS-PFM-DJE-020-04, ATS-PFM-DJE-020-05, TST-PFM-DJE-194] statement: >- A table failure is reported in an overall failed run while other table passes continue; failure metrics and full-run heartbeat metrics support separate alarms. kind: policy enforced_at: - service must_prove: true provenance: test source: >- services-platform:modules/partitions/tests/integration/services/maintenance/maintain-partitions.integration.spec.js#fails the run as a whole when one table’s pass failed, naming that table origin: none - id: BR-CAP-PFM-DJE-020-05 proved_by: [] statement: Partition month boundaries follow UTC rather than workspace-local dates. kind: policy enforced_at: - database must_prove: false provenance: documented source: 'services-platform:docs/partitions.md#The helpers' origin: none - id: BR-CAP-PFM-DJE-020-06 proved_by: [] statement: >- Partition maintenance serves registered history from other modules; job receipts and refusals remain unpartitioned. kind: policy enforced_at: - database must_prove: false provenance: documented source: 'macroservice-libraries:packages/data-jobs/README.md#Partitioning is deferred' origin: none scenarios: - id: SC-CAP-PFM-DJE-020-01 given: One of two registered table passes fails. when: Scheduled maintenance runs. then: >- The overall run names its failed table while the other table prepares future months and retires its two expired months. provenance: test source: >- services-platform:modules/partitions/tests/integration/services/maintenance/maintain-partitions.integration.spec.js#still keeps the other table: the months ahead created, the two expired months archived and dropped edge_cases: [] constraints: [] depends_on: [] related: - CAP-PFM-DJE-019 journeys: [] open_questions: - OQ-PFM-DJE-004 evidence: - >- Reviewed evidence: services-platform (ad3f6a65): docs/partitions.md; modules/partitions/** (settings, scheduled maintenance, maintain-partitions, partitions-maintenance.work); tests maintain-partitions.integration.spec.js, partitions-maintenance.work.spec.ts, scheduled-maintenance.spec.ts, partitions-maintenance-alarms.spec.ts. - >- Reviewed evidence: macroservice-libraries (54536a44): packages/data-jobs/README.md (Partitioning is deferred). controls: [TC-AZ-001, TC-AZ-002, TC-AZ-003, TC-AZ-004, TC-CHG-007, TC-DAT-004, TC-IAM-001, TC-IAM-006, TC-LOG-002, TC-LOG-003, TC-LOG-004, TC-LOG-005, TC-MON-001, TC-MON-002, TC-PI-001, TC-PI-005, TC-TEN-001, TC-TEN-002, TC-TEN-003, TC-VEN-001, TC-VEN-002, TC-VEN-003] exercised_by: [FAS-PFM-DJE-06-01, FAS-PFM-DJE-06-02, FAS-PFM-DJE-06-03, FAS-PFM-DJE-06-04, FAS-PFM-DJE-06-05] suites: [TST-PFM-DJE-194] --- # Maintain and retire monthly history partitions ## Purpose Scheduled maintenance prepares monthly storage for registered history tables and retires expired partitions through their archive contracts. ## How It Works Platform maintains selected history tables in monthly partitions. The scheduled pass ensures the current month and the configured months ahead. Existing partitions remain intact. Retirement uses the table’s archive contract before removing expired storage. The helper contract governs child-table security and UTC boundaries. One table’s failure is recorded while the remaining table passes continue. A failed pass produces the failure metric rather than a success heartbeat. Separate alarms identify failures and prolonged absence of successful runs. This registered scope serves history outside job processing. Product still needs to decide its long-term module home. ## Relationships - Workload protection groups this scheduled storage maintenance with capacity operations.