---
id: STATE-PFM-INT-008
title: Provider circuit
object: DOM-PFM-INT-002
product_area: platform
module: integrations
revision: 1
status: draft
design_intent: confirmed
implementation_evidence: partially-observed
origin: >-
  docs/integrations/README.md#The pipeline (13 §5) (Andrew 2026-10-19 decision 2);
  docs/milestones/INTG-M2-calls-through-the-gateway.md#Decisions M2-D4
source_sha:
  services-platform: ad3f6a6582843008ed0f5a17b131878d15829af1
states:
  - name: closed
    meaning: Admission allows calls; the absence of a circuit record also means closed.
  - name: open
    meaning: Admission refuses calls until the probe becomes due.
  - name: half_open
    meaning: Admission is testing recovery with the probe call.
transitions:
  - from: closed
    to: open
    trigger: Reach the unavailable threshold
    guard: The process-local consecutive-failure count reaches five in its breaker window
    side_effects:
      - Delay the next probe by sixty seconds.
    who: Gateway runtime breaker
    provenance: documented
    source: 'services-platform:docs/integrations/README.md#Circuits (13 §10)'
  - from: open
    to: half_open
    trigger: Admit the recovery probe
    guard: The open wait elapsed and this admission claims the probe
    side_effects:
      - Allow the probing request.
    who: Gateway admission
    provenance: documented
    source: 'services-platform:docs/integrations/README.md#Circuits (13 §10)'
  - from: half_open
    to: closed
    trigger: Observe a recovered probe
    guard: The probe does not report unavailable failure
    side_effects:
      - Clear the breaker’s failure count.
    who: Gateway runtime breaker
    provenance: documented
    source: 'services-platform:docs/integrations/README.md#Circuits (13 §10)'
  - from: half_open
    to: open
    trigger: Observe another unavailable probe
    guard: Probe failure is unavailable or timeout
    side_effects:
      - Double the delay without exceeding fifteen minutes.
    who: Gateway runtime breaker
    provenance: documented
    source: 'services-platform:docs/integrations/README.md#Circuits (13 §10)'
  - from: open
    to: closed
    trigger: Reset through staff operations
    guard: The staff caller has circuit-operation authority
    side_effects:
      - Record the reset in workspace history.
    who: Rhythm staff
    provenance: observed
    source: 'services-platform:modules/integrations/services/executions/execution-operations.ts'
  - from: half_open
    to: closed
    trigger: Reset while a probe is pending
    guard: The staff caller has circuit-operation authority
    side_effects:
      - Close and audit the circuit independent of its former status.
    who: Rhythm staff
    provenance: inferred
    source: 'services-platform:modules/integrations/services/executions/execution-operations.ts:resetCircuit'
    note: The reset operation accepts the circuit’s existing status.
invalid:
  - from: closed
    to: half_open
    why: 'Recovery probing begins only after an open wait, while closed admission has no probe.'
concurrency: >-
  Persistent circuit writes compare versions, but reads can use a five-second process cache. Opening counts
  are process-local, so distributed failures need not reach the threshold in one process. The complete
  distributed probe and threshold behavior remains a proof gap.
invariants:
  - INV-DOM-PFM-INT-002-09
  - INV-DOM-PFM-INT-003-08
evidence:
  - >-
    services-platform at ad3f6a65: modules/integrations/db/integration-circuit-states.table.ts;
    services/executions/circuit-store.ts; docs/integrations/README.md (Circuits);
    docs/milestones/INTG-M2-calls-through-the-gateway.plan.md#12. Risks
---

# Provider circuit

## The Lifecycle

```mermaid
stateDiagram-v2
  state "closed" as s0
  state "open" as s1
  state "half_open" as s2
  s0 --> s1: Reach the unavailable threshold
  s1 --> s2: Admit the recovery probe
  s2 --> s0: Observe a recovered probe
  s2 --> s1: Observe another unavailable probe
  s1 --> s0: Reset through staff operations
  s2 --> s0: Reset while a probe is pending
```

## Transitions Explained

The breaker protects the selected provider account without changing routing. Unavailable outcomes can open it; expiry permits a recovery probe and its result closes the circuit or lengthens the wait. Staff reset provides an explicit operational override.

## What Cannot Happen

Recovery probing begins only after an open wait, while closed admission has no probe.

## Concurrency and Timing

Persistent circuit writes compare versions, but reads can use a five-second process cache. Opening counts are process-local, so distributed failures need not reach the threshold in one process. The complete distributed probe and threshold behavior remains a proof gap.