Rhythm 2.0 Specifications

Appearance

CAP-PFM-LLM-002Bound AI conversations and handle recoverable failures

Markdown

CAP-PFM-LLM-002

Bound AI conversations and handle recoverable failures

Markdown
  • DocumentDraft
  • Product intentProposed
  • ImplementationObserved

Planning

Feature
FEAT-PFM-LLM-01
Exercised by
FAS-PFM-LLM-01-01, FAS-PFM-LLM-01-02, FAS-PFM-LLM-01-05
Key concepts
CON-PFM-LLM-001, CON-PFM-LLM-002
Actors
ACT-PFM-032, ACT-PFM-034
Depends on
CAP-PFM-LLM-001, CAP-PFM-LLM-005
Related
CAP-PFM-LLM-012
Proved by
ATS-PFM-LLM-002

Implementation

Executable suites
TST-PFM-LLM-018, TST-PFM-LLM-020, TST-PFM-LLM-064, TST-PFM-LLM-065, TST-PFM-LLM-067, TST-PFM-LLM-068

KindCapabilityProduct areaPlatformModuleAI Requests, Usage and the Jazz AI TeammateRevision1

Read atservices-platform ad3f6a65

Bound AI conversations and handle recoverable failures ​

Purpose ​

Keep an AI conversation accountable while giving recoverable failures a controlled route forward.

How It Works ​

Platform runs the model conversation using the limits published for its operation. The model can ask for tools or offer a final answer. The consuming feature can accept that answer or return a validation message for another attempt. Ordinary tool errors become information the model can use to repair its request. Recoverable provider faults take the retry path, while permanent faults end it. Attempts retain operational details without retaining the conversation body. Budget checks occur at specific points, so an accepted answer can take precedence over later checks. The product should settle the meaning of these limits before presenting them as absolute cost or time guarantees.

Relationships ​

  • Run AI requests with consistent attribution and accounting supplies the prerequisite behavior for this outcome.
  • Publish AI operation definitions and historical prices supplies the prerequisite behavior for this outcome.

The Formal Contract

The narrative above explains; the sections below are authoritative — the rules, scenarios and tests this document binds.

Outcomes
Conversation guards restrict ordinary turns, tool batches, time and credits.Retryable provider faults receive bounded retries.Attempt records retain provider usage and pricing information.
Surfaces
apijob
Preconditions
Admission has reserved capacity for the request.
Postconditions
The conversation records its outcome and provider attempts.

Controls

37 controls apply, from the corpus control catalogue.

TC-AI-001 TC-AI-002 TC-AI-003 TC-AI-004 TC-AI-005 TC-AI-006 TC-AI-007 TC-AVL-001 TC-AVL-004 TC-AZ-001 TC-AZ-002 TC-AZ-003 TC-AZ-004 TC-AZ-006 TC-DAT-001 TC-DAT-002 TC-DAT-003 TC-IAM-001 TC-IAM-002 TC-IAM-004 TC-IAM-005 TC-LOG-004 TC-LOG-005 TC-MON-001 TC-PI-001 TC-PI-002 TC-PI-003 TC-PI-004 TC-PI-005 TC-PRV-001 TC-PRV-002 TC-TEN-001 TC-TEN-002 TC-TEN-003 TC-VEN-001 TC-VEN-002 TC-VEN-003

Business Rules

15 rules, 11 to be proved before approval. A rule's product intent is derived from its origin: 0 confirmed by a product decision, the rest inheriting this capability's proposed.

IdRuleKindEnforced atProvenance
01must prove
Proposed
A conversation without an accepted answer ends when it has used the permitted turns, after sending that many model requests.
proved by ATS-PFM-LLM-001-11, ATS-PFM-LLM-002-01, ATS-PFM-LLM-002-12, TST-PFM-LLM-020, TST-PFM-LLM-068
policyservicetest
services-platform:modules/llm/tests/unit/runtime/run-tool-loop.spec.ts#stops a conversation that will not converge
02must prove
Proposed
Without an accepted final answer, an oversized tool batch ends the conversation before any handler in that batch runs.
proved by ATS-PFM-LLM-002-02, TST-PFM-LLM-068
policyservicetest
services-platform:modules/llm/tests/unit/runtime/run-tool-loop.spec.ts#runs no handler for the turn it refused, rather than half of it
03must prove
Proposed
After checking final answers and terminal responses, a tool-requesting turn stops further work when accumulated rounded turn costs are greater than the hold; spending exactly the hold continues and the crossing turn remains billable.
proved by ATS-PFM-LLM-002-03, TST-PFM-LLM-068
calculationserviceobserved
services-platform:modules/llm/runtime/run-tool-loop.ts:runToolLoop
04must prove
Proposed
Before starting another turn, the loop refuses to send when no total time remains and caps the provider attempt timeout at the remaining conversation time.
proved by ATS-PFM-LLM-002-04, TST-PFM-LLM-068
policyserviceobserved
services-platform:modules/llm/runtime/run-tool-loop.ts
05must prove
Proposed
Timeouts, ordinary rate limits, overload, server faults, broken connections and aborts can retry with increasing waits and a default of three attempts; malformed or unauthorized provider requests do not retry.
proved by ATS-PFM-LLM-002-05, ATS-PFM-LLM-002-14, TST-PFM-LLM-065, TST-PFM-LLM-067
policyserviceobserved
services-platform:modules/llm/runtime/execute-with-llm-retry.ts
06must prove
Proposed
A provider rate limit that identifies a depleted Rhythm balance is classified as credit exhaustion and does not retry.
origin: Tribal TRI-1899 (services-platform:docs/llm.md#The runtime)
proved by ATS-PFM-LLM-002-06, TST-PFM-LLM-067
policyservicetest
services-platform:modules/llm/tests/unit/runtime/retry-policy.spec.ts#is not retried, because retrying it reads a purchase problem as load
07must prove
Proposed
Attempts record their turn, retry, usage and timing and receive individual prices; after 64 retained attempts, additional attempts increase the count and mark the record incomplete without adding rows.
proved by ATS-PFM-LLM-002-07, ATS-PFM-LLM-002-13, TST-PFM-LLM-018, TST-PFM-LLM-020
calculationserviceobserved
services-platform:modules/llm/services/persistence/invocations.ts:recordInvocationAttempt
08
Proposed
For a nonterminal turn within budget, the model receives one combined tool-results message and every answered call consumes tool budget.
policyserviceobserved
services-platform:modules/llm/runtime/run-tool-loop.ts
09
Proposed
A thrown tool-handler error becomes an error result for the model, so handler messages must be suitable to disclose to that model.
policyservicetest
services-platform:modules/llm/tests/unit/runtime/run-tool-loop.spec.ts#passes the failure back to the model as a tool result marked in error
10must prove
Proposed
Rejected final answers return the feature’s feedback as tool errors and consume tool budget; a failing answer judge also becomes a rejection and further turns use the existing budgets.
proved by ATS-PFM-LLM-002-08, TST-PFM-LLM-068
policyserviceobserved
services-platform:modules/llm/runtime/run-tool-loop.ts
11
Proposed
A provider refusal ends without retry, while an output-token cutoff produces an incomplete outcome rather than an accepted answer.
policyserviceobserved
services-platform:modules/llm/runtime/run-tool-loop.ts
12must prove
Proposed
Stable instructions, tool definitions and explicitly stable content can receive cache markers, with excess markers dropped so the provider receives at most four.
proved by ATS-PFM-LLM-002-09, TST-PFM-LLM-064
calculationservicetest
services-platform:modules/llm/tests/unit/runtime/anthropic-provider.spec.ts#never sends more than the four Anthropic takes, however many blocks asked to be cached
13
Proposed
Failures do not trigger a substitute model or provider, and consuming features cannot replace published routing or limits.
policyservicedocumented
services-platform:docs/llm/llm-foundation-and-generate-query.md#4.4 Accounting Rules
14must prove
Proposed
An accepted final answer completes before the later tool and credit checks and does not increase the answered-tool count.
proved by ATS-PFM-LLM-002-10
policyserviceobserved
services-platform:modules/llm/runtime/run-tool-loop.ts:runToolLoop
15must prove
Proposed
A requested tool without a supplied handler produces a recoverable error result, allowing later turns within the remaining budgets.
proved by ATS-PFM-LLM-002-11
policyserviceobserved
services-platform:modules/llm/runtime/run-tool-loop.ts:runToolLoop

Scenarios

IdGivenWhenThenProvenance
01The first provider attempt is temporarily overloaded.A permitted retry succeeds.Both attempts remain visible and the successful usage determines the charge.test
services-platform:modules/llm/tests/integration/lifecycle/run-operation.integration.spec.js#charges only what the successful attempt reported
02The consumer rejects the first final answer.The model receives repair feedback.A later accepted answer completes the conversation within the remaining controls.test
services-platform:modules/llm/tests/unit/runtime/run-tool-loop.spec.ts#ends on the answer the consumer accepted, which is the repaired one

The Markdown files under specs/ in rhythmsoftware/rhythm-v2-docs are the canonical source of truth. This reader is a generated, read-only projection of them.

Rhythm Software