--- id: WF-CRM-REG-003 title: Recalculate every record under the current policy trigger: Staff with recalculation authority request a run through the saved-policy view or regional API. product_area: crm module: regions revision: 2 status: draft design_intent: confirmed origin: >- Andrew, 2026-10-08 (saving rules changes no record; staff press Recalculate) and 2026-10-10 (the failure reason is redacted), cited in services-crm:docs/migrations/crm-regions/README.md; the queued-work standard §3 (the queue owns retries; nothing in the handler counts failures), cited in services-crm:modules/regions/services/region-assignment/region-recalculation.ts implementation_evidence: observed inference: [] source_sha: services-crm: a8d1364f1dd44b723bb5f9acba12f983e9ec8d69 react-console: baf6dae382d6f41dce976c7f848667ccc37e076f implements: - CAP-CRM-REG-011 - CAP-CRM-REG-012 - CAP-CRM-REG-010 moves: - STATE-CRM-REG-004 - STATE-CRM-REG-003 - STATE-CRM-REG-002 crosses: - SVC-CRM-REG-regions - SVC-CRM-ENT-entities idempotency: >- Another start selects an existing running run and can advance it. Batch progress and placement updates commit together under the run lock, so redelivery continues beyond committed work. Terminal runs perform no more traversal, and unchanged regional decisions require no new placement write. concurrency: >- One running run is permitted per workspace. Workers serialize each batch on that run, then lock traversal records. Policy is reread during processing without the available policy read lock; policy changes may affect later batches and leave a further recalculation outstanding. hops: - 'n': 1 actor_or_service: Console Assignment Rules tab action: >- The saved-policy view places Recalculate in its banner, toolbar or action menu. Disabled assignment hides it; missing run authority disables it. anchor: 'react-console:components/regions/recalculation.tsx:recalculatePlacement' seam: sync on_failure: A caller without console run authority cannot use the control. left_inconsistent: The absence of a permitted action changes no record. observer: The displayed control and its permission explanation inform staff. provenance: test source: 'react-console:components/regions/__tests__/recalculation.test.tsx' - 'n': 2 actor_or_service: Regions API (recalculateRegionAssignments) action: >- The API checks recalculation authority and transactionally selects the running run or inserts one with the current policy generation. Insertion captures its initial worker event. anchor: 'services-crm:modules/regions/services/region-assignment/region-recalculation.ts:startRegionRecalculation' seam: sync on_failure: Authority failure records nothing; the unique running-run race normally selects the winning run. left_inconsistent: A refused start leaves regional records untouched. observer: Run persistence and the start response identify the selected execution. provenance: observed source: >- services-crm:modules/regions/tests/integration/services/persistence/region-recalculations.integration.spec.js#region recalculations table (integration) › refuses a second running run for the tenant; services-crm:modules/regions/services/region-assignment/region-recalculation.ts:startRegionRecalculation; services-crm:modules/regions/aws/api/regions.routes.ts:recalculateRegionAssignments - 'n': 3 actor_or_service: 'Regions API, within the request' action: >- The request executes batches of 500 until its ten-second budget expires or traversal finishes, then returns the run and nudges dispatch. anchor: 'services-crm:modules/regions/aws/api/regions.routes.ts:recalculateRegionAssignments' seam: sync on_failure: >- A batch failure rolls back and attempts to store a redacted reason before the request returns an error; that error bypasses the nudge. left_inconsistent: >- The recorded run can remain running while the console reports a failed start. Scheduled dispatch can publish the waiting initial event. observer: 'API logs, persisted run reads and the start-error toast expose different parts of the outcome.' provenance: observed source: 'services-crm:modules/regions/services/region-assignment/region-recalculation.ts:runBatch' - 'n': 4 actor_or_service: Change-events dispatcher and queue action: Shared dispatch publishes insert and continuation events for the registered recalculation runner. anchor: 'services-crm:aws/crm.change-events.ts (region_recalculation stream)' seam: event on_failure: A failed publication leaves captured work for a later dispatcher attempt. left_inconsistent: Unpublished work waits with its last committed cursor. observer: Dispatch logs and run modification time indicate stalled progress. provenance: documented source: 'services-crm:docs/change-events.md#The handlers CRM registers' - 'n': 5 actor_or_service: Recalculation worker (region-recalculation-runner) action: >- The worker uses a forty-second budget to lock each run batch, traverse the next records, force common reconciliation and commit counts and cursor. Short batches advance the phase; the last organization phase completes the run. anchor: 'services-crm:modules/regions/services/region-assignment/region-recalculation.ts:recalculateBatch' seam: queue on_failure: >- Batch failure rolls back and attempts separate redacted settlement before rethrowing. Final delivery attempts failed status, but settlement failure can leave running status after dead-lettering. left_inconsistent: >- Earlier placements remain committed. A terminal failure leaves partial traversal; failed settlement can also prevent creating a replacement run. observer: >- Run reads expose status and counts. The console displays reasons only for failed status and does not surface monitoring-read errors; batch logs provide counts. provenance: observed source: >- services-crm:modules/regions/tests/integration/services/region-assignment/region-recalculation-errors.integration.spec.js#a region recalculation that fails (integration) › the last delivery of a run’s event › fails the run, because nothing will deliver its event again; services-crm:modules/regions/services/region-assignment/region-recalculation.ts:recordFailure - 'n': 6 actor_or_service: Recalculation worker action: 'Unfinished work at budget expiry increments the continuation counter, capturing another worker event.' anchor: >- services-crm:modules/regions/services/persistence/region-recalculations.ts:recordRecalculationContinuation seam: event on_failure: >- A continuation-write error fails the current delivery, allowing queue redelivery from committed progress. left_inconsistent: The run remains at its committed position until delivery succeeds or operator recovery occurs. observer: The stored continuation and run modification time expose the handoff. provenance: observed source: 'services-crm:modules/regions/services/region-assignment/region-recalculation.ts:runRecalculationToBudget' - 'n': 7 actor_or_service: Settings read (getRegionAssignmentSettings) action: >- Settings reads compare current generation with the highest generation of a completed run to compute outstanding recalculation. anchor: 'services-crm:modules/regions/services/persistence/region-assignment-settings.ts:recalculationNeeded' seam: sync on_failure: A failed settings read gives the policy view a load failure. left_inconsistent: The read failure makes the flag unavailable but changes no placement. observer: The saved-policy view uses the returned flag for its needed banner. provenance: test source: >- services-crm:modules/regions/tests/integration/services/region-assignment/region-assignment-rules.integration.spec.js#region assignment rules and slots (integration) › whether a recalculation is needed › a rule saved while a recalculation is under way › says one is still needed, because the run was recorded under the rules as they were before branches: - >- With assignment disabled, traversal still advances and counts records while reconciliation changes no region. - A repeated start works on the selected running run during its own request budget. - >- If the request completes traversal, subsequent initial-event delivery sees the terminal run and performs no further batch. waits: - The console polls the selected run and refreshes policy when that run settles. - Continuation publication and delivery determine the delay between worker budgets. evidence: - >- services-crm at a8d1364f: modules/regions/services/region-assignment/region-recalculation.ts; services/persistence/region-recalculations.ts; aws/api/regions.routes.ts; aws/crm.change-events.ts; docs/change-events.md; tests/integration/services/region-assignment/region-recalculation-errors.integration.spec.js; tests/integration/services/persistence/region-recalculations.integration.spec.js. - >- react-console at baf6dae3: components/regions/recalculation.tsx; components/regions/assignment-rules-view.tsx. controls: [TC-AVL-004, TC-AZ-001, TC-AZ-002, TC-AZ-003, TC-AZ-004, TC-AZ-005, TC-AZ-006, TC-DAT-001, TC-DAT-002, TC-DAT-003, TC-IAM-001, TC-IAM-005, TC-IAM-006, TC-LOG-004, TC-LOG-005, TC-MON-001, TC-MON-002, TC-PI-001, TC-PI-002, TC-PI-003, TC-PI-004, TC-PI-005, TC-PRV-001, TC-PRV-002, TC-TEN-001, TC-TEN-002, TC-TEN-003] --- # Recalculate every record under the current policy ## What It Does Recalculation applies saved coverage across existing records when staff request it. Traversal visits contacts before organizations and forces the shared decision routine to reconsider eligible input. Overrides, update locks and records with neither current nor remembered input remain protected. Progress is durable between batches. ## The Flow ```mermaid sequenceDiagram participant C as Staff participant A as Regional API participant R as Run persistence participant E as Entities participant Q as Change queue participant W as Runner C->>A: Request recalculation A->>R: Insert or select running run loop Request budget A->>E: Reconcile traversal batch A->>R: Commit progress with placements end A-->>C: Run result A->>Q: Nudge captured event dispatch Q->>W: Initial or continuation delivery loop Worker budget W->>R: Lock run W->>E: Reconcile next batch W->>R: Advance committed progress end opt Work remains W->>R: Increment continuation end C->>A: Poll adopted run ``` ## When Something Fails A failed batch rolls back without reversing earlier batches. Failure settlement attempts to retain a redacted reason, and queue retries continue from the committed cursor. Successful progress clears that reason. Final failed delivery attempts terminal failed status. If that settlement also fails, the queue can exhaust while the run remains running, requiring recovery outside the regional routes. An in-request batch failure reports an error after insertion has committed and skips the dispatcher nudge. Scheduled dispatch can still pick up the captured run event. The console does not automatically discover the newly recorded run from the failed response. ## Open Design Questions Recovery for failed settlement and an ambiguous start response needs an explicit implementation plan. Cancellation, examined-count wording and monitoring failures remain unresolved. Policy reads also need consistency protection before completion can support a stronger placement guarantee.