--- id: STATE-CRM-REG-004 title: Recalculation run object: DOM-CRM-REG-004 product_area: crm module: regions revision: 2 status: draft design_intent: confirmed origin: >- Andrew, 2026-10-10 (the failure reason is redacted), cited in services-crm:docs/migrations/crm-regions/README.md#Assignment and recalculation triggers in Rolodex 2.0; the queued-work standard §3 (the queue owns retries), cited in services-crm:modules/regions/services/region-assignment/region-recalculation.ts implementation_evidence: observed source_sha: services-crm: a8d1364f1dd44b723bb5f9acba12f983e9ec8d69 react-console: baf6dae382d6f41dce976c7f848667ccc37e076f states: - name: requested meaning: The start operation has been entered; this boundary condition is not a persisted run status. - name: running meaning: Traversal is unfinished and can retain a redacted explanation from the latest failed batch. - name: completed meaning: 'Both phases have finished, with final counts, completion time and no retained batch error.' - name: failed meaning: 'Failure settlement succeeded after a final failed delivery, leaving counts and a redacted explanation.' transitions: - from: requested to: running trigger: An authorized caller requests recalculation. guard: >- The caller needs recalculation authority; an existing running run is selected rather than inserting another. side_effects: - Insertion records the current generation and captures the initial continuation event. - 'The request executes batches for its ten-second budget, which can finish a small workspace.' - Successful request processing then nudges dispatch. who: An administrator provenance: observed source: >- services-crm:modules/regions/tests/integration/services/persistence/region-recalculations.integration.spec.js#region recalculations table (integration) › refuses a second running run for the tenant; services-crm:modules/regions/services/region-assignment/region-recalculation.ts:startRegionRecalculation; services-crm:modules/regions/aws/api/regions.routes.ts:recalculateRegionAssignments - from: running to: running trigger: 'A batch advances, a worker requests continuation, or a nonterminal delivery fails.' guard: The run must still have running status. side_effects: - A batch commits placements and progress together and clears any prior error. - Worker budget exhaustion increments continuation to request another delivery. - Batch failure rolls back and attempts redacted failure settlement before rethrowing. who: 'The recalculation worker, the start request, or a repeated start request' provenance: test source: >- services-crm:modules/regions/tests/integration/services/region-assignment/region-recalculation-errors.integration.spec.js#a region recalculation that fails (integration) › the reason it records › records the redacted reason on a run it leaves running, so the next delivery can get past the cause - from: running to: completed trigger: Processing finishes the organization phase. guard: The final organization traversal returns less than a full batch. side_effects: - Persistence sets completion time and clears the retained error. - The outstanding-policy flag clears only if a completed generation covers current policy. who: The recalculation worker or the start request provenance: test source: >- services-crm:modules/regions/tests/integration/services/region-assignment/region-recalculation-errors.integration.spec.js#a region recalculation that fails (integration) › a run that gets past what stopped it › answers the whole record of the completed run, with no reason left on it - from: running to: failed trigger: A batch fails on the queue’s final delivery. guard: The consumer must identify this receive as final. side_effects: - >- Settlement attempts to record failed status and the reason; settlement failure can leave running status with only a log. - >- Queue exhaustion dead-letters the message, while successful failed status frees the workspace for a new run. who: The recalculation worker provenance: observed source: >- services-crm:modules/regions/tests/integration/services/region-assignment/region-recalculation-errors.integration.spec.js#a region recalculation that fails (integration) › the last delivery of a run’s event › fails the run, because nothing will deliver its event again; services-crm:modules/regions/services/region-assignment/region-recalculation.ts:recordFailure; services-crm:docs/change-events.md#7. Failures and recovery invalid: - from: failed to: running why: Regional start creates a new run after terminal failure; it does not reopen the failed record. - from: completed to: running why: A completed run remains historical; later starts create another run. - from: running to: failed why: Retryable batch failure does not intentionally settle terminal status. - from: running to: completed why: Only finishing both traversal phases completes a run; there is no cancellation route. concurrency: >- The database allows one running run per workspace. Batch workers serialize on the run row and lock traversal records before regional processing. Repeated starts also process the selected run. Later policy changes affect later reads and can leave another recalculation outstanding. If final failure settlement fails, a running row can remain after queue exhaustion. invariants: - INV-DOM-CRM-REG-004-01 - INV-DOM-CRM-REG-004-02 - INV-DOM-CRM-REG-004-04 - INV-DOM-CRM-REG-004-05 - INV-DOM-CRM-REG-004-06 evidence: - >- services-crm at a8d1364f: modules/regions/services/region-assignment/region-recalculation.ts; services/persistence/region-recalculations.ts; db/region-recalculations.table.ts; aws/api/regions.routes.ts (recalculateRegionAssignments); tests/integration/services/region-assignment/region-recalculation-errors.integration.spec.js. - >- react-console at baf6dae3: components/regions/recalculation.tsx (useRecalculation, polling every 1.5 seconds). --- # Recalculation run ## The Lifecycle ```mermaid stateDiagram-v2 [*] --> requested requested --> running: insert or select running run running --> running: progress or retryable failure running --> completed: finish organizations running --> failed: final failure settlement succeeds completed --> [*] failed --> [*] ``` ## Transitions Explained Only running, completed and failed are persisted statuses. The start request records or selects a run and executes a short series of batches. Remaining work reaches a worker through captured events. The worker advances transactional progress and requests another delivery when its budget expires. Retryable failures preserve committed progress and attempt to store a redacted reason. ## What Cannot Happen Terminal runs do not reopen through regional operations. There is no cancellation action. Final delivery failure does not guarantee failed status: settlement is best effort, and an error during settlement can leave a run stuck in running. ## Concurrency and Timing Run locks prevent committed batches from repeating. Record locks coordinate saves during processing. The console polls an adopted running run every one and a half seconds, but it does not display monitoring errors or restore failed historical runs after reload.