---
id: WF-PFM-INT-013
title: Settle unknown and overdue calls
trigger: The minute schedule starts execution investigation.
product_area: platform
module: integrations
revision: 1
status: draft
design_intent: confirmed
implementation_evidence: observed
origin: >-
  docs/milestones/INTG-M2-calls-through-the-gateway.md#Decisions M2-D2, M2-D6, M2-D8 (Andrew);
  docs/integrations/README.md#The pipeline (13 §5) (Andrew 2026-10-19 decision 3)
inference: []
source_sha:
  services-platform: ad3f6a6582843008ed0f5a17b131878d15829af1
implements:
  - CAP-PFM-INT-010
  - CAP-PFM-INT-011
moves:
  - STATE-PFM-INT-007
crosses: []
idempotency: 'Repeated settlement writes use status comparison, but provider lookup invocations can still repeat.'
concurrency: >-
  The scheduled function has shared concurrency capacity rather than a reconciler mutex. Overlapping lookups
  can occur; outcome updates compare status.
hops:
  - 'n': 1
    actor_or_service: Scheduler
    action: Invoke the execution investigation pass on the minute schedule.
    anchor: 'services-platform:modules/integrations/aws/logic/integrations-gateway.lambda.ts'
    seam: schedule
    on_failure: The next pass revisits due work; backlog and silence alarms expose delay.
    left_inconsistent: Due investigations can remain waiting.
    observer: Reconciler alarms
    provenance: documented
    source: 'services-platform:docs/integrations/README.md#The service functions and their parameters'
  - 'n': 2
    actor_or_service: Reconciler
    action: >-
      Look up due uncertain executions using their connection’s current pinned release and active deployment
      and original execution identity.
    anchor: 'services-platform:modules/integrations/services/executions/reconciler.ts:reconcileExecutions'
    seam: sync
    on_failure: Count a failed lookup as inconclusive.
    left_inconsistent: The provider’s outcome remains uncertain.
    observer: 'Activity “Settling: next check at”'
    provenance: observed
    source: 'services-platform:modules/integrations/services/executions/reconciler.ts:reconcileExecutions'
  - 'n': 3
    actor_or_service: Reconciler
    action: >-
      Settle conclusive lookups, back off unresolved results and at the fifth lookup either fail non-sensitive
      not-found work or require a person.
    anchor: 'services-platform:modules/integrations/services/executions/reconciler.ts:reconcileExecution'
    seam: sync
    on_failure: A status comparison loses harmlessly to another settlement.
    left_inconsistent: The saved outcome remains the winner’s; provider lookups can still repeat.
    observer: Activity and Needs Attention
    provenance: observed
    source: 'services-platform:modules/integrations/services/executions/reconciler.ts:reconcileExecution'
  - 'n': 4
    actor_or_service: Reconciler
    action: Request administrator mail for work needing a person and emit attention telemetry.
    anchor: 'services-platform:modules/integrations/services/notices/connection-notices.ts'
    seam: sync
    on_failure: Log failed mail delivery.
    left_inconsistent: The attention record exists even if no email arrives.
    observer: Needs Attention and the Rhythm HQ list
    provenance: observed
    source: 'services-platform:modules/integrations/services/notices/connection-notices.ts'
  - 'n': 5
    actor_or_service: Reconciler
    action: >-
      Inspect overdue asynchronous calls, lost attempts and lost continuation work and make their
      investigation due immediately.
    anchor: 'services-platform:modules/integrations/services/executions/reconciler.ts:timeOutExecution'
    seam: sync
    on_failure: A later pass repeats a failed timeout write.
    left_inconsistent: The old in-flight state can remain until recovery.
    observer: Execution timed-out alarm
    provenance: observed
    source: 'services-platform:modules/integrations/services/executions/reconciler.ts:timeOutExecution'
  - 'n': 6
    actor_or_service: Executions operator
    action: >-
      Record a person’s verified outcome and note for an execution awaiting attention, without another
      provider operation.
    anchor: 'services-platform:modules/integrations/services/executions/execution-operations.ts:resolveExecution'
    seam: sync
    on_failure: Refuse an execution outside the attention state.
    left_inconsistent: An incorrect recorded result is not automatically corrected.
    observer: Integration history
    provenance: observed
    source: 'services-platform:modules/integrations/services/executions/execution-operations.ts:resolveExecution'
  - 'n': 7
    actor_or_service: Gateway
    action: >-
      Create completion events atomically for asynchronous work and for synchronous callers left without a
      final result.
    anchor: 'services-platform:modules/integrations/services/executions/settle-execution.ts:settleExecution'
    seam: sync
    on_failure: A competing outcome emits no duplicate event.
    left_inconsistent: Event delivery can still wait on its separate routing workflow.
    observer: The invoking application
    provenance: observed
    source: 'services-platform:modules/integrations/services/executions/settle-execution.ts:settleExecution'
waits:
  - 'Lookup waits of thirty seconds, two minutes, ten minutes, an hour and six hours'
evidence:
  - >-
    services-platform at ad3f6a65: modules/integrations/services/executions/reconciler.ts, timeouts.ts,
    execution-operations.ts, settle-execution.ts; docs/integrations/README.md (The reconciler; Settling is one
    path)
  - >-
    services-platform:modules/integrations/tests/integration/services/executions/reconciler.integration.spec.js#looks
    the execution up on the same connection with the contracts’ lookup shape, never repeating the authorize
  - >-
    services-platform:modules/integrations/tests/integration/services/executions/reconciler.integration.spec.js#keeps
    looking through the fourth lookup, never failing a payment the provider may still show
  - >-
    services-platform:modules/integrations/tests/integration/services/executions/reconciler.integration.spec.js#backs
    off 2 min, 10 min, 1 h and 6 h while the lookup cannot tell
  - >-
    services-platform:modules/integrations/tests/integration/services/executions/reconciler.integration.spec.js#lets
    an operator resolve a needs_attention execution by hand, audited, with no call to the provider
  - >-
    services-platform:modules/integrations/tests/integration/services/executions/execution-settlement.integration.spec.js#is
    written once for two settlements that both read the execution running, the loser moving nothing
controls: [TC-AVL-001, TC-AVL-004, TC-AVL-005, TC-AZ-001, TC-AZ-002, TC-AZ-003, TC-AZ-004, TC-AZ-005, TC-CHG-001, TC-CHG-002, TC-CHG-003, TC-CHG-004, TC-CHG-005, TC-CHG-006, TC-DAT-001, TC-DAT-002, TC-DAT-003, TC-IAM-001, TC-IAM-002, TC-IAM-003, TC-IAM-004, TC-IAM-005, TC-IAM-006, TC-LOG-001, TC-LOG-002, TC-LOG-003, TC-LOG-004, TC-LOG-005, TC-MON-001, TC-MON-002, TC-PI-001, TC-PI-002, TC-PI-003, TC-PI-004, TC-PI-005, TC-PRV-001, TC-PRV-002, TC-TEN-001, TC-TEN-002, TC-TEN-003, TC-VEN-001, TC-VEN-002, TC-VEN-003]
---

# Settle unknown and overdue calls

## What It Does

Investigation asks what the provider did with an uncertain execution instead of repeating its operation. The same scheduled pass detects work that has become overdue or lost. Exhausted uncertainty becomes a person’s task, except that final non-sensitive not-found results can fail automatically.

## The Flow

```mermaid
sequenceDiagram
  participant P0 as Scheduler
  participant P1 as Reconciler
  participant P2 as Executions operator
  participant P3 as Gateway
  P0->>P0: 1. Invoke the execution investigation pass on the minute schedule.
  P1->>P1: 2. Look up due uncertain executions using their connection’s current pinned release and active deployment and ori
  P1->>P1: 3. Settle conclusive lookups
  P1->>P1: 4. Request administrator mail for work needing a person and emit attention telemetry.
  P1->>P1: 5. Inspect overdue asynchronous calls
  P2->>P2: 6. Record a person’s verified outcome and note for an execution awaiting attention
  P3->>P3: 7. Create completion events atomically for asynchronous work and for synchronous callers left without a final res
```

## When Something Fails

The next pass revisits due work; backlog and silence alarms expose delay. Due investigations can remain waiting. Count a failed lookup as inconclusive. The provider’s outcome remains uncertain. A status comparison loses harmlessly to another settlement. The saved outcome remains the winner’s; provider lookups can still repeat. Log failed mail delivery. The attention record exists even if no email arrives. A later pass repeats a failed timeout write. The old in-flight state can remain until recovery. Refuse an execution outside the attention state. An incorrect recorded result is not automatically corrected. A competing outcome emits no duplicate event. Event delivery can still wait on its separate routing workflow.

## Open Design Questions

No declared reconciler mutex establishes exclusion between runs. Investigation uses the current account pin and active deployment, and archive expiry remains undecided.