---
id: ATS-PFM-LLM-009
scope: CAP-PFM-LLM-009
product_area: platform
module: llm
revision: 2
status: proposed
source_sha:
  services-platform: ad3f6a6582843008ed0f5a17b131878d15829af1
scenarios:
  - id: ATS-PFM-LLM-009-01
    title: Old nonterminal requests receive estimated settlement
    level: api
    proves:
      - BR-CAP-PFM-LLM-009-01
    preconditions: An unsettled request is older than one day.
    steps: Run eligible recovery.
    expected: >-
      The request records estimated credits and no cents, its recorded hold is released and the period
      receives the estimate.
    negative: false
    data_needs: Fixtures representing an unsettled request is older than one day.
    bound_to:
      - TST-PFM-LLM-046
    provenance: test
    source: >-
      services-platform:modules/llm/tests/integration/services/reconciliation/llm-settlement-sweep.integration.spec.js#settles
      it at its estimate, because its real cost was never learned
    category: transition
    test_sources:
      - >-
        services-platform:modules/llm/tests/integration/services/reconciliation/llm-settlement-sweep.integration.spec.js#settles
        it at its estimate, because its real cost was never learned
      - >-
        services-platform:modules/llm/tests/integration/services/reconciliation/llm-settlement-sweep.integration.spec.js#records
        no money figure, which would be a guess about a call nobody measured
      - >-
        services-platform:modules/llm/tests/integration/services/reconciliation/llm-settlement-sweep.integration.spec.js#releases
        the hold, which is the whole point of the sweep
      - >-
        services-platform:modules/llm/tests/integration/services/reconciliation/llm-settlement-sweep.integration.spec.js#charges
        the invocation’s estimate, not the hold
  - id: ATS-PFM-LLM-009-02
    title: Recent work remains untouched
    level: api
    proves:
      - BR-CAP-PFM-LLM-009-02
    preconditions: A request is only a minute old.
    steps: Invoke recovery.
    expected: It examines none of that work and preserves its hold without a past-window alert.
    negative: false
    data_needs: Fixtures representing a request is only a minute old.
    bound_to:
      - TST-PFM-LLM-046
    provenance: test
    source: >-
      services-platform:modules/llm/tests/integration/services/reconciliation/llm-settlement-sweep.integration.spec.js#leaves
      the hold in place, so a live call keeps the credits it reserved
    category: boundary
    test_sources:
      - >-
        services-platform:modules/llm/tests/integration/services/reconciliation/llm-settlement-sweep.integration.spec.js#leaves
        the hold in place, so a live call keeps the credits it reserved
      - >-
        services-platform:modules/llm/tests/integration/services/reconciliation/llm-settlement-sweep.integration.spec.js#does
        not alert on it
  - id: ATS-PFM-LLM-009-03
    title: Successful recovery is not charged twice
    level: api
    proves:
      - BR-CAP-PFM-LLM-009-03
    preconditions: A request is eligible for recovery.
    steps: Run reconciliation twice.
    expected: The first settles and alerts once; the second finds no unsettled request and adds no charge.
    negative: false
    data_needs: Fixtures representing a request is eligible for recovery.
    bound_to:
      - TST-PFM-LLM-046
    provenance: test
    source: >-
      services-platform:modules/llm/tests/integration/services/reconciliation/llm-settlement-sweep.integration.spec.js#finds
      nothing to examine on the second, because a settled call is not unsettled
    category: invariant
    test_sources:
      - >-
        services-platform:modules/llm/tests/integration/services/reconciliation/llm-settlement-sweep.integration.spec.js#finds
        nothing to examine on the second, because a settled call is not unsettled
      - >-
        services-platform:modules/llm/tests/integration/services/reconciliation/llm-settlement-sweep.integration.spec.js#charges
        the period once
      - >-
        services-platform:modules/llm/tests/integration/services/reconciliation/llm-settlement-sweep.integration.spec.js#alerts
        once across both runs, so a standing problem is not an hourly flood
  - id: ATS-PFM-LLM-009-04
    title: Stale tasks become abandoned
    level: api
    proves:
      - BR-CAP-PFM-LLM-009-05
    preconditions: One execution is older than the window and another remains recent.
    steps: Run task recovery.
    expected: >-
      The stale record becomes abandoned at detection with no duration and unchanged counters, while the
      recent task remains running.
    negative: false
    data_needs: Fixtures representing one execution is older than the window and another remains recent.
    bound_to:
      - TST-PFM-LLM-045
    provenance: test
    source: >-
      services-platform:modules/llm/tests/integration/services/reconciliation/llm-execution-sweep.integration.spec.js#abandons
      the stale running execution
    category: transition
    test_sources:
      - >-
        services-platform:modules/llm/tests/integration/services/reconciliation/llm-execution-sweep.integration.spec.js#abandons
        the stale running execution
      - >-
        services-platform:modules/llm/tests/integration/services/reconciliation/llm-execution-sweep.integration.spec.js#marks
        it abandoned, with when the sweep found it and no duration
      - >-
        services-platform:modules/llm/tests/integration/services/reconciliation/llm-execution-sweep.integration.spec.js#leaves
        the totals as the settlements put them
      - >-
        services-platform:modules/llm/tests/integration/services/reconciliation/llm-execution-sweep.integration.spec.js#leaves
        a recent running execution running
  - id: ATS-PFM-LLM-009-05
    title: Counter drift is reported without repair
    level: api
    proves:
      - BR-CAP-PFM-LLM-009-06
    preconditions: One of two closed tasks disagrees with its settled requests.
    steps: Run the counter check.
    expected: 'One discrepancy identifies both totals and the execution, leaving its stored counters unchanged.'
    negative: false
    data_needs: Fixtures representing one of two closed tasks disagrees with its settled requests.
    bound_to:
      - TST-PFM-LLM-045
    provenance: test
    source: >-
      services-platform:modules/llm/tests/integration/services/reconciliation/llm-execution-sweep.integration.spec.js#counts
      the one execution that differs
    category: rule
    test_sources:
      - >-
        services-platform:modules/llm/tests/integration/services/reconciliation/llm-execution-sweep.integration.spec.js#counts
        the one execution that differs
      - >-
        services-platform:modules/llm/tests/integration/services/reconciliation/llm-execution-sweep.integration.spec.js#logs
        both sides of the disagreement
      - >-
        services-platform:modules/llm/tests/integration/services/reconciliation/llm-execution-sweep.integration.spec.js#leaves
        the counters as they were, never correcting them
  - id: ATS-PFM-LLM-009-06
    title: Recovered requests enter reporting once
    category: interaction
    level: api
    proves:
      - BR-CAP-PFM-LLM-009-01
      - BR-CAP-PFM-LLM-010-08
    exercises:
      - CAP-PFM-LLM-009
      - CAP-PFM-LLM-010
    preconditions: A stale nonterminal request has no prior daily cell.
    steps: 'Recover it, rebuild reporting, then repeat both passes.'
    expected: >-
      The request contributes one estimated-credit settlement with unknown cents and repeated rebuilding
      preserves the same totals.
    negative: false
    data_needs: 'An isolated workspace with the records, clock instants and caller authority stated in the preconditions.'
    bound_to: []
    provenance: proposed
    source:
      - 'services-platform:modules/llm/aws/lambda/RECONCILIATION.handler.ts'
      - 'services-platform:modules/llm/services/reconciliation/rollup-settled-invocations.ts'
evidence:
  - >-
    services-platform at ad3f6a6582843008ed0f5a17b131878d15829af1: bound or cited
    modules/llm/tests/integration/services/reconciliation/llm-execution-sweep.integration.spec.js;
    modules/llm/tests/integration/services/reconciliation/llm-settlement-sweep.integration.spec.js.
coverage:
  rule: 'gap: FND-PFM-LLM-068'
  transition: 'gap: FND-PFM-LLM-068'
  validation: 'gap: FND-PFM-LLM-068'
  permission: not-applicable
  edge: 'gap: FND-PFM-LLM-068'
  failure: 'gap: FND-PFM-LLM-068'
  boundary: covered
  concurrency: 'gap: FND-PFM-LLM-068'
  invariant: 'gap: FND-PFM-LLM-068'
  interaction: covered
coverage_notes:
  transition: >-
    Existing transition scenarios are retained; the model transition register below records the remaining
    exhaustive mapping obligation.
  permission: >-
    This capability declares no permission key; caller authorization belongs to its consuming operation or a
    related capability.
  interaction: >-
    The related interaction register cites a concrete experiment for every dependency and related capability,
    including cross-capability workflow seams.
  invariant: >-
    The domain-invariant register maps each relevant invariant to a scenario or to the capability coverage
    finding; an example does not establish every invariant.
  rule: >-
    All must-prove rules have scenario references, but the coverage finding retains untested clauses and
    combined outcomes; scenario presence alone is not exhaustive rule proof.
related_interactions:
  - with: CAP-PFM-LLM-003
    scenarios:
      - ATS-PFM-LLM-003-08
    source: 'services-platform:modules/llm/services/execution/with-llm-execution.ts'
  - with: CAP-PFM-LLM-007
    scenarios:
      - ATS-PFM-LLM-007-12
    source: 'services-platform:modules/llm/services/persistence/allowance-periods.ts'
  - with: CAP-PFM-LLM-008
    scenarios:
      - ATS-PFM-LLM-008-10
    source: 'services-platform:modules/llm/services/persistence/allowance-periods.ts'
  - with: CAP-PFM-LLM-010
    scenarios:
      - ATS-PFM-LLM-009-06
    source: >-
      services-platform:modules/llm/tests/e2e/llm-usage.e2e.spec.js#refuses a customer’s administrator,
      whatever they hold at their customer’s scope
state_models:
  - STATE-PFM-LLM-001
  - STATE-PFM-LLM-002
  - STATE-PFM-LLM-006
coverage_obligations:
  must_prove:
    - rule: BR-CAP-PFM-LLM-009-01
      scenarios:
        - ATS-PFM-LLM-009-01
        - ATS-PFM-LLM-009-06
      source: 'services-platform:modules/llm/aws/lambda/RECONCILIATION.handler.ts'
      coverage: partial
      gap: FND-PFM-LLM-068
    - rule: BR-CAP-PFM-LLM-009-02
      scenarios:
        - ATS-PFM-LLM-009-02
      source: >-
        services-platform:modules/llm/tests/integration/services/reconciliation/llm-settlement-sweep.integration.spec.js#leaves
        the hold in place, so a live call keeps the credits it reserved
      coverage: specified
    - rule: BR-CAP-PFM-LLM-009-03
      scenarios:
        - ATS-PFM-LLM-009-03
      source: 'services-platform:modules/llm/services/reconciliation/llm-settlement-sweep.ts'
      coverage: partial
      gap: FND-PFM-LLM-068
    - rule: BR-CAP-PFM-LLM-009-05
      scenarios:
        - ATS-PFM-LLM-009-04
      source: 'services-platform:modules/llm/services/reconciliation/llm-settlement-sweep.ts'
      coverage: specified
    - rule: BR-CAP-PFM-LLM-009-06
      scenarios:
        - ATS-PFM-LLM-009-05
      source: 'services-platform:modules/llm/services/reconciliation/llm-settlement-sweep.ts'
      coverage: specified
  state_transitions:
    - model: STATE-PFM-LLM-001
      transition: 1
      from: estimated
      to: refused
      trigger: The access check rejects the AI or operation switch.
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: 'services-platform:modules/llm/services/execution/run-operation.ts:runOperation'
    - model: STATE-PFM-LLM-001
      transition: 2
      from: estimated
      to: refused
      trigger: Admission cannot fit the reservation or allocate a running slot.
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: 'services-platform:modules/llm/services/execution/run-operation.ts:runOperation'
    - model: STATE-PFM-LLM-001
      transition: 3
      from: estimated
      to: reserved
      trigger: Admission successfully reserves credits and a slot.
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: 'services-platform:modules/llm/services/persistence/invocations.ts:markInvocationReserved'
    - model: STATE-PFM-LLM-001
      transition: 4
      from: reserved
      to: dispatched
      trigger: Request capture has returned and its metadata was written.
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: 'services-platform:modules/llm/services/persistence/invocations.ts:markInvocationDispatched'
    - model: STATE-PFM-LLM-001
      transition: 5
      from: dispatched
      to: completed
      trigger: The loop returns an outcome other than failed.
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: 'services-platform:modules/llm/services/persistence/invocations.ts:markInvocationCompleted'
    - model: STATE-PFM-LLM-001
      transition: 6
      from: dispatched
      to: failed
      trigger: The loop reports failure after provider execution or attempt recording.
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: 'services-platform:modules/llm/services/persistence/invocations.ts:markInvocationFailed'
    - model: STATE-PFM-LLM-001
      transition: 7
      from: completed
      to: pending
      trigger: A completed loop result contains no reported usage.
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: 'services-platform:modules/llm/services/persistence/invocations.ts:markInvocationPending'
    - model: STATE-PFM-LLM-001
      transition: 8
      from: completed
      to: settled
      trigger: Runtime accounting closes a completed result.
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: 'services-platform:modules/llm/services/execution/run-operation.ts:settleInvocation'
    - model: STATE-PFM-LLM-001
      transition: 9
      from: pending
      to: settled
      trigger: Runtime or stale-request recovery closes an unmeasured result.
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: 'services-platform:modules/llm/services/execution/run-operation.ts:settleInvocation'
    - model: STATE-PFM-LLM-001
      transition: 10
      from: failed
      to: settled
      trigger: Runtime accounting closes a failed loop result.
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: 'services-platform:modules/llm/services/execution/run-operation.ts:settleInvocation'
    - model: STATE-PFM-LLM-001
      transition: 11
      from: dispatched
      to: settled
      trigger: An exception escapes the protected execution block.
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: 'services-platform:modules/llm/services/execution/run-operation.ts:runOperation'
    - model: STATE-PFM-LLM-001
      transition: 12
      from: estimated
      to: settled
      trigger: Recovery selects a stale estimated request.
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: >-
        services-platform:modules/llm/services/reconciliation/llm-settlement-sweep.ts:sweepUnsettledInvocations
    - model: STATE-PFM-LLM-001
      transition: 13
      from: reserved
      to: settled
      trigger: Recovery selects a stale reserved request.
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: >-
        services-platform:modules/llm/services/reconciliation/llm-settlement-sweep.ts:sweepUnsettledInvocations
    - model: STATE-PFM-LLM-001
      transition: 14
      from: dispatched
      to: settled
      trigger: Recovery selects a stale dispatched request.
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: >-
        services-platform:modules/llm/tests/integration/services/reconciliation/llm-settlement-sweep.integration.spec.js#settles
        it at its estimate, because its real cost was never learned
    - model: STATE-PFM-LLM-002
      transition: 1
      from: absent
      to: running
      trigger: A consumer opens a declared task.
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: 'services-platform:modules/llm/services/execution/with-llm-execution.ts:withLlmExecution'
    - model: STATE-PFM-LLM-002
      transition: 2
      from: running
      to: completed
      trigger: The task body returns successfully.
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: 'services-platform:modules/llm/services/persistence/executions.ts:closeExecution'
    - model: STATE-PFM-LLM-002
      transition: 3
      from: running
      to: failed
      trigger: The task body throws an error.
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: 'services-platform:modules/llm/services/execution/with-llm-execution.ts:withLlmExecution'
    - model: STATE-PFM-LLM-002
      transition: 4
      from: running
      to: abandoned
      trigger: Recovery selects a task still running after a day.
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: 'services-platform:modules/llm/services/persistence/executions.ts:markExecutionAbandoned'
    - model: STATE-PFM-LLM-002
      transition: 5
      from: completed
      to: completed
      trigger: A request settles after its enclosing task completed.
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: 'services-platform:modules/llm/services/persistence/executions.ts:addInvocationToExecution'
    - model: STATE-PFM-LLM-006
      transition: 1
      from: absent
      to: owed
      trigger: A settlement reaches a charged-usage threshold.
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: 'services-platform:modules/llm/services/metering/llm-credit-ledger.ts:evaluateCrossings'
    - model: STATE-PFM-LLM-006
      transition: 2
      from: owed
      to: leased
      trigger: A sender tries to claim the recorded crossing.
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: 'services-platform:modules/llm/services/persistence/allowance-periods.ts:claimThresholdNotice'
    - model: STATE-PFM-LLM-006
      transition: 3
      from: leased
      to: delivered
      trigger: At least one recipient is handled successfully or suppressed by delivery policy.
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: 'services-platform:modules/llm/services/persistence/allowance-periods.ts:stampThresholdNotified'
    - model: STATE-PFM-LLM-006
      transition: 4
      from: leased
      to: nobody_told
      trigger: The recipient resolver returns no eligible administrators.
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: 'services-platform:modules/llm/services/notices/threshold-notice.ts:sendThresholdNotice'
    - model: STATE-PFM-LLM-006
      transition: 5
      from: leased
      to: owed
      trigger: Nobody is reached or the sender stops and the lease expires.
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: 'services-platform:modules/llm/services/notices/retry-threshold-notices.ts:retryThresholdNotices'
    - model: STATE-PFM-LLM-006
      transition: 6
      from: leased
      to: given_up
      trigger: The last allowed claim ends without completion.
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: 'services-platform:modules/llm/services/persistence/allowance-periods.ts:claimThresholdNotice'
    - model: STATE-PFM-LLM-006
      transition: 7
      from: leased
      to: leased
      trigger: Delivery occurred but completion was not recorded before lease expiry.
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: 'services-platform:modules/llm/services/notices/threshold-notice.ts:sendThresholdNotice'
  domain_invariants:
    - invariant: INV-DOM-PFM-LLM-001-03
      scenarios: []
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: 'services-platform:modules/llm/services/persistence/invocations.ts:markInvocationSettled'
    - invariant: INV-DOM-PFM-LLM-001-04
      scenarios: []
      gap: FND-PFM-LLM-068
      provenance: proposed
      source: 'services-platform:modules/llm/services/persistence/invocations.ts:markInvocationSettled'
coverage_limits:
  finding: FND-PFM-LLM-068
  statement: >-
    Every early request state, terminal-request accounting drift and failure between recovery settlement and
    period movement need fault injection. Sweeping and rollup follow-up failures need combined verification.
  provenance: proposed
  source:
    - 'services-platform:modules/llm/aws/lambda/RECONCILIATION.handler.ts'
    - >-
      services-platform:modules/llm/tests/integration/services/reconciliation/llm-settlement-sweep.integration.spec.js#leaves
      the hold in place, so a live call keeps the credits it reserved
    - 'services-platform:modules/llm/services/reconciliation/llm-settlement-sweep.ts'
    - 'services-platform:modules/llm/services/reconciliation/llm-settlement-sweep.ts'
    - 'services-platform:modules/llm/services/reconciliation/llm-settlement-sweep.ts'
    - 'services-platform:modules/llm/services/persistence/invocations.ts:markInvocationSettled'
    - 'services-platform:modules/llm/services/persistence/invocations.ts:markInvocationSettled'
  invariants_without_scenarios:
    - INV-DOM-PFM-LLM-001-03
    - INV-DOM-PFM-LLM-001-04
---

# Recover interrupted requests and identify accounting drift: Capability Test Specification

## Coverage

| Rule | Scenario |
|---|---|
| `BR-CAP-PFM-LLM-009-01` | `ATS-PFM-LLM-009-01` |
| `BR-CAP-PFM-LLM-009-02` | `ATS-PFM-LLM-009-02` |
| `BR-CAP-PFM-LLM-009-03` | `ATS-PFM-LLM-009-03` |
| `BR-CAP-PFM-LLM-009-05` | `ATS-PFM-LLM-009-04` |
| `BR-CAP-PFM-LLM-009-06` | `ATS-PFM-LLM-009-05` |

## Interactions With Related Capabilities

The interaction register links concrete experiments with keep one account for a multi-request ai task; admit ai work against the organization’s monthly allowance; notify administrators at allowance thresholds; monitor ai usage across organizations. Each experiment checks the outcome at the shared boundary; an unbound experiment remains proposed.

Every early request state, terminal-request accounting drift and failure between recovery settlement and period movement need fault injection. Sweeping and rollup follow-up failures need combined verification.