Rhythm 2.0 Specifications

Appearance

ATS-PFM-LLM-004undefined

Markdown

ATS-PFM-LLM-004

undefined

Markdown
  • DocumentProposed

Planning

Proves capability
CAP-PFM-LLM-004

Design

Executed by
TST-PFM-LLM-014, TST-PFM-LLM-020, TST-PFM-LLM-027, TST-PFM-LLM-034, TST-PFM-LLM-072

KindAcceptance test setProduct areaPlatformModuleAI Requests, Usage and the Jazz AI TeammateRevision2

Read atservices-platform ad3f6a65

undefined ​

Coverage ​

The scenarios identify the rules or journey assertions they are intended to prove. A test binding names existing evidence; an unbound proposal remains work to verify.

ScenarioTitleLevelProvenance
ATS-PFM-LLM-004-01Request and response capture have recordsapitest
ATS-PFM-LLM-004-02Redaction preserves ordinary text and replaces credentialsunittest
ATS-PFM-LLM-004-03Attempt records contain operational fields onlyapitest
ATS-PFM-LLM-004-04Organization deletion preserves its neighborsapitest
ATS-PFM-LLM-004-05Evidence capture discloses skipped and failed outcomesapiproposed

The interaction register links concrete experiments with run ai requests with consistent attribution and accounting; inspect captured ai content with a live access grant. Each experiment checks the outcome at the shared boundary; an unbound experiment remains proposed.

Capture skip and serialization failures, metadata-write interruption, retention expiry and purge while traffic continues still need explicit outcomes. Purge quiescence is an external precondition.

Coverage

  • Rule gap: FND-PFM-LLM-063
  • Transition gap: FND-PFM-LLM-063
  • Validation covered
  • Permission not-applicable
  • Edge gap: FND-PFM-LLM-063
  • Failure covered
  • Boundary covered
  • Concurrency gap: FND-PFM-LLM-063
  • Invariant gap: FND-PFM-LLM-063
  • Interaction covered

Test Scenarios

7 scenarios proving CAP-PFM-LLM-004; 5 bound to an existing test.

01 Request and response capture have recordsruleapitest
Proves
BR-CAP-PFM-LLM-004-01
Preconditions
A scripted provider produces a normal result.
Steps
Execute the request and inspect its evidence objects and metadata.
Expected
Capture records exist for request and response, with request capture preceding dispatch and response capture following the loop result.
Data
Fixtures representing a scripted provider produces a normal result.
Bound to
Source
services-platform:modules/llm/tests/integration/lifecycle/run-operation.integration.spec.js#captured the request before it was sent
02 Redaction preserves ordinary text and replaces credentialsvalidationunittest
Proves
BR-CAP-PFM-LLM-004-02
Preconditions
Payload examples contain secret fields and bearer, model-provider, cloud and web keys and connection strings.
Steps
Redact every example.
Expected
Recognized credentials disappear while ordinary question text remains.
Data
Fixtures representing payload examples contain secret fields and bearer, model-provider, cloud and web keys and connection strings.
Bound to
Source
services-platform:modules/llm/tests/unit/services/evidence-redaction.spec.ts#replaces an apiKey whole, whatever it holds
03 Attempt records contain operational fields onlyinvariantapitest
Proves
BR-CAP-PFM-LLM-004-03
Preconditions
A conversation creates several attempt records.
Steps
Inspect the retained attempts.
Expected
Their columns contain no prompt or tool input or output bodies.
Data
Fixtures representing a conversation creates several attempt records.
Bound to
Source
services-platform:modules/llm/tests/integration/services/persistence/invocations.integration.spec.js#records no prompt, tool input or tool output on any of them
04 Organization deletion preserves its neighborsboundaryapitest
Proves
BR-CAP-PFM-LLM-004-05 BR-CAP-PFM-LLM-004-06
Preconditions
Two organizations hold AI data and evidence, including a similar-prefix neighbor.
Steps
Delete one organization’s AI resources.
Expected
Its records and evidence versions are removed in dependency order, with other organizations unchanged.
Data
Fixtures representing two organizations hold AI data and evidence, including a similar-prefix neighbor.
Bound to
Source
services-platform:modules/llm/tests/integration/services/maintenance/tenant-participant.integration.spec.js#leaves the tenant nothing at all in any table it declares, or in the evidence bucket
05 Evidence capture discloses skipped and failed outcomesfailureapinegativeproposed
Proves
BR-CAP-PFM-LLM-004-09
Preconditions
Missing and failing evidence stores and a deployed-memory configuration are available.
Steps
Capture with no store, fail serialization or storage, and construct a memory store in deployed configuration.
Expected
Capture returns skipped or failed as applicable; deployed memory-store construction refuses.
Data
Fault-injection fixtures for the described failure boundary.
Source
services-platform:modules/llm/services/evidence/evidence-store.ts
06 Capture status controls privileged readsinteractionapinegativeproposed
Proves
BR-CAP-PFM-LLM-004-01 BR-CAP-PFM-LLM-016-04 INV-DOM-PFM-LLM-001-09
Preconditions
A request has one captured redacted body and one skipped body; the reader holds an eligible grant.
Steps
Read both body kinds, remove the captured object and read again.
Expected
The first body is redacted, skipped avoids storage access, and missing captured content returns purged with audited status.
Data
An isolated workspace with the records, clock instants and caller authority stated in the preconditions.
Source
services-platform:modules/llm/services/execution/run-operation.ts:recordCapture,services-platform:modules/llm/services/evidence/evidence-access.ts
07 Captured content requires a retained referenceinvariantapitest
Proves
BR-CAP-PFM-LLM-004-04
Preconditions
The suite owns isolated fixtures representing the valid control and the contrasting case described by its steps.
Steps
Write captured request and response states without an evidence reference.
Expected
Both writes are refused by the storage constraint.
Data
Suite-owned fixtures, with a fixed clock where dates or grant windows matter.
Bound to
Source
services-platform:modules/llm/tests/integration/db/llm-invocations.table.integration.spec.js#refuses a captured request with nowhere to find it

The Markdown files under specs/ in rhythmsoftware/rhythm-v2-docs are the canonical source of truth. This reader is a generated, read-only projection of them.

Rhythm Software