--- id: STATE-PFM-QRY-014 title: Monthly snapshot run object: DOM-PFM-QRY-006 product_area: platform module: queries revision: 1 status: draft design_intent: confirmed implementation_evidence: observed source_sha: services-platform: ad3f6a6582843008ed0f5a17b131878d15829af1 states: - name: planned meaning: The period version is recorded for its owning dispatcher to consider. - name: queued meaning: Admission reserved capacity and is arranging the durable attempt job. - name: running meaning: 'Generation is executing or resuming, using the selected single-transaction or chunked mode.' - name: validating meaning: The final transaction is checking generated results before publication. - name: published meaning: >- Validation and publication succeeded; later versions can replace this version as current without removing it. - name: failed meaning: 'The attempt failed, either awaiting its configured retry time or settled as a final failure.' - name: skipped meaning: Planning recorded an ineligible period and its loading or policy reason. - name: cancelled meaning: >- Cancellation was recorded. Further cursor updates and publication are refused, though earlier committed chunks may remain. transitions: - from: planned to: queued trigger: The owner dispatcher claims the planned run. guard: Owner locking and total and backfill caps permit admission. side_effects: - the row is queued; the dispatcher separately submits its attempt job who: system (snapshot dispatcher) provenance: observed source: >- services-platform:node_modules/@rhythmsoftware/macroservice-snapshots/dist/snapshot-repository.js:claimRuns - from: queued to: running trigger: The job worker starts the queued run. guard: The row remains queued. side_effects: - >- the attempt counter and start time are recorded; execution selects its retained generation mode who: system (snapshot run worker) provenance: observed source: >- services-platform:node_modules/@rhythmsoftware/macroservice-snapshots/dist/snapshot-repository.js:startRun - from: running to: validating trigger: Generation enters final validation. guard: The locked run remains running or validating. side_effects: - validation occurs inside the final transaction who: system (snapshot run worker) provenance: observed source: >- services-platform:node_modules/@rhythmsoftware/macroservice-snapshots/dist/snapshot-repository.js:beginValidation - from: validating to: published trigger: Final validation passes. guard: 'Final generation, validation and publication share the transaction.' side_effects: - the version becomes current and the previous version is retained who: system (snapshot run worker) provenance: observed source: >- services-platform:node_modules/@rhythmsoftware/macroservice-snapshots/dist/snapshot-repository.js:publishRun - from: running to: failed trigger: Generation or exhausted job processing fails. guard: The row remains in an admitted status. side_effects: - >- the failing transaction rolls back; retryable attempt settlement schedules the configured delay, with earlier chunks retained who: system (job failure consequence or reconciler) provenance: observed source: 'services-platform:modules/snapshots/services/runs/snapshot-run-workflow.ts' - from: validating to: failed trigger: The final transaction fails validation. guard: The row can settle as an admitted attempt. side_effects: - >- final publication rolls back; prior chunks remain unpublished and the previous current version is preserved who: system provenance: observed source: >- services-platform:node_modules/@rhythmsoftware/macroservice-snapshots/dist/snapshot-run-workflow.js:advanceSnapshotRun - from: failed to: queued trigger: The failed attempt reaches its next-attempt time. guard: A retry is scheduled and owner capacity permits admission. side_effects: - another attempt job is submitted who: system (snapshot dispatcher) provenance: observed source: >- services-platform:node_modules/@rhythmsoftware/macroservice-snapshots/dist/snapshot-states.js (SNAPSHOT_RETRY_DELAYS_MINUTES) - from: planned to: cancelled trigger: 'Cancellation records workspace removal, policy or operator reason.' guard: The run has not settled. side_effects: - the reason is saved and further publication is prevented who: system or operator provenance: observed source: >- services-platform:node_modules/@rhythmsoftware/macroservice-snapshots/dist/snapshot-repository.js:cancelRun - from: running to: cancelled trigger: Cancellation occurs during generation. guard: The run has not settled. side_effects: - a later cursor update or publication is refused; earlier committed chunks are not removed who: system or operator provenance: observed source: >- services-platform:node_modules/@rhythmsoftware/macroservice-snapshots/dist/snapshot-repository.js:cancelRun - from: queued to: cancelled trigger: Cancellation occurs before the attempt starts. guard: The run remains queued. side_effects: - the reason is saved and publication is prevented who: system or operator provenance: observed source: >- services-platform:node_modules/@rhythmsoftware/macroservice-snapshots/dist/snapshot-repository.js:cancelRun - from: validating to: cancelled trigger: Cancellation occurs during validation. guard: The run has not settled. side_effects: - the reason is saved and publication is prevented who: system or operator provenance: observed source: >- services-platform:node_modules/@rhythmsoftware/macroservice-snapshots/dist/snapshot-repository.js:cancelRun - from: failed to: cancelled trigger: Cancellation replaces a pending retry. guard: The failed run still has a next-attempt time. side_effects: - the retry time is cleared and cancellation is settled who: system or operator provenance: observed source: >- services-platform:node_modules/@rhythmsoftware/macroservice-snapshots/dist/snapshot-repository.js:cancelRun - from: running to: running trigger: The worker resumes existing generation. guard: The run remains running. side_effects: - the cursor is retained without incrementing the attempt count who: system (snapshot run worker) provenance: observed source: >- services-platform:node_modules/@rhythmsoftware/macroservice-snapshots/dist/snapshot-repository.js:startRun - from: validating to: validating trigger: The worker resumes existing validation. guard: The run remains validating. side_effects: - the attempt count is retained who: system (snapshot run worker) provenance: observed source: >- services-platform:node_modules/@rhythmsoftware/macroservice-snapshots/dist/snapshot-repository.js:startRun - from: queued to: failed trigger: The job fails before generation starts. guard: The row remains queued. side_effects: - failure settlement records the outcome and any eligible retry time who: system (job failure consequence or reconciler) provenance: observed source: >- services-platform:node_modules/@rhythmsoftware/macroservice-snapshots/dist/snapshot-repository.js:failRun invalid: - from: published to: running why: Rerunning a published period creates a new version rather than restarting the released one. - from: skipped to: queued why: >- Revisiting a skipped period requires a new requested version rather than admission of the skipped row. concurrency: >- Unique indexes protect active and current versions per workspace, type and period. Owner-scoped claim locking enforces admission caps. Queued rows without an attached job can be reoffered after five minutes; active periods refuse overlapping manual reruns. invariants: - INV-DOM-PFM-QRY-006-01 - INV-DOM-PFM-QRY-006-02 - INV-DOM-PFM-QRY-006-05 - INV-DOM-PFM-QRY-006-06 evidence: - >- services-platform at ad3f6a6: modules/snapshots/**; @rhythmsoftware/macroservice-snapshots (snapshot-states, snapshot-repository, snapshot-run-workflow); docs/proposals/snapshot-architecture.md (approved 2026-10-15) --- # Monthly snapshot run ## The Lifecycle ```mermaid stateDiagram-v2 state "planned" as s0 state "queued" as s1 state "running" as s2 state "validating" as s3 state "published" as s4 state "failed" as s5 state "skipped" as s6 state "cancelled" as s7 s0 --> s1 s1 --> s2 s2 --> s3 s3 --> s4 s2 --> s5 s3 --> s5 s5 --> s1 s0 --> s7 s2 --> s7 s1 --> s7 s3 --> s7 s5 --> s7 s2 --> s2 s3 --> s3 s1 --> s5 ``` Monthly runs distinguish eligibility planning, admission, generation and final publication. Each attempt has a durable job, while the ledger preserves period versions. ## Transitions Explained Small runs generate, validate and publish in one transaction. Large runs commit chunks with their resume cursor, then finalize and publish transactionally. ## What Cannot Happen A failed final validation leaves the previous current version intact but may retain staged chunks. Cancellation also prevents publication rather than proving removal of earlier committed chunks. ## Concurrency and Timing Retryable failures use widening delays; the fourth failed attempt settles finally. Publication lateness and final failures have separate operational alarms.