Reprocess compliance across the workspace
What It Does
A reprocess reevaluates existing organizations after administration changes their role expectations. An association with many chapters needs the scan to survive bounded workers and partial failure. The route records or reuses a durable run and returns promptly, while later workers retain progress and finish or fail explicitly.
Our reading is that a durable cursor is useful only when progress and business changes share a commit. Otherwise a retry could skip an organization that was never recalculated or keep repeating one that already changed. Here each batch owns that boundary; the whole workspace is not one rollback unit.
What Starts It
A staff member with organization update authority requests another compliance scan. The service records a durable run or returns the workspace’s existing running run. That response begins or identifies queued work; it does not report that every organization has already been assessed. Operators inspect worker progress and the terminal result to determine how much work completed.
The Flow
The diagram shows the sequence and service seams. Each synchronous stage waits for its result, while captured-event, queue and scheduled stages can run after the originating request has returned. The hop descriptions below identify the write and failure boundary, so the last arrow must not be mistaken for one atomic transaction across every participant.
Hop by Hop
1. Entities route
The route requires organization update authority and starts a run or returns the workspace’s running one. It records the scan request and asks for dispatch. The immediate response’s organization count is request information, not a report that all those organizations have already been refreshed.
2. CRM change-events dispatcher
The dispatcher publishes the insertion event to the reprocess handler. Queue acceptance is the point at which work is handed off; until then the saved run can remain waiting. Failed publication retains the event so later dispatch can make another attempt.
3. Compliance reprocess runner
The worker locks the run and takes the next ordered organization batch. Up to twenty-five subjects are recalculated per transaction; cursor and counts commit with their assessments. A failed batch rolls back together, while earlier successful batches remain a valid partial result.
4. Compliance reprocess runner
When the worker reaches its time budget before finishing, it increments the run’s continuation counter. That captured change requests another worker delivery. A failure to record continuation fails the current delivery, leaving the still-running run available for normal queue recovery.
5. Compliance reprocess runner
The exhausted scan records completion and finish time. A batch failure on the final permitted receive records terminal failure and a redacted reason, then throws into queue dead-letter handling. A later delivery of either terminal run skips it rather than opening it for more work.
When Something Fails
Hop 1 — Entities route
The failure boundary is: Refuse persistence failure without recording a partial run. The remaining work or data is: No partial business write remains. The caller receives the result.
Hop 2 — CRM change-events dispatcher
The failure boundary is: Retain the event until queue acceptance succeeds. The remaining work or data is: The run can remain recorded but not yet started. Dispatcher logs identify delayed starts. A later attempt can be required; the scheduling or publication boundary does not establish an immediate user-visible completion.
Hop 3 — Compliance reprocess runner
The failure boundary is: Roll back a failed batch, record a redacted reason and throw for ordinary SQS retry. The remaining work or data is: Earlier batches remain committed while later organizations await successful continuation. Run counts, reason and consumer logs show partial progress. This failure is downstream of committed input; operators investigate its own delivery rather than treating the caller’s original save as failed.
Hop 4 — Compliance reprocess runner
The failure boundary is: Fail the delivery if continuation cannot be recorded so the queue can retry. The remaining work or data is: The still-running run may await another successful delivery. Run progress and continuation records show the wait. A later attempt can be required; the scheduling or publication boundary does not establish an immediate user-visible completion.
Hop 5 — Compliance reprocess runner
The failure boundary is: The final failure is dead-lettered; requesting a new run is the recovery path. Redrive skips the terminal run. The remaining work or data is: A failed run leaves a partially reprocessed workspace. Run status, redacted reason and the dead-letter queue identify terminal failure. This failure is downstream of committed input; operators investigate its own delivery rather than treating the caller’s original save as failed.
Repeats and Races
Starting while a run is already running returns that run. Database enforcement resolves simultaneous starts the same way. Worker redelivery uses the committed cursor and locks the run, so it resumes unfinished batches rather than repeating committed progress or letting two workers independently scan the same position.
A completed or failed run is terminal. Redrive does not reopen it; recovery requests another run. Type policy can change between batches because there is no pinned policy snapshot. Nested evaluations can produce ordinary notice submissions, whose later dispatch and delivery remain distinct from successful scan completion.
What Is Observable
The caller receives the result. Dispatcher logs identify delayed starts. Run counts, reason and consumer logs show partial progress. Run progress and continuation records show the wait. Run status, redacted reason and the dead-letter queue identify terminal failure.
The requester sees the accepted scan message, while operators use stored run status, cursor, counts and redacted reason to distinguish waiting, progress, completion and failure. The route does not provide a live progress page, and notice delivery is established separately through Platform outcomes.
Open Design Questions
A failed run can leave a partially refreshed workspace, and a completed run can have used different policy revisions across its batches. The existing findings preserve those boundaries and the limited operator surface for inspecting progress. The immediate request message must not be read as a synchronous completion report.