ATS-PFM-LLM-005 Publish AI operation definitions and historical prices: Capability Test Specification
Markdown Design / Platform / AI Requests, Usage and the Jazz AI Teammate / AI Catalog and Pricing / Capability Test Specification Coverage Rule gap: FND-PFM-LLM-064 Transition gap: FND-PFM-LLM-064 Validation covered Permission covered Edge covered Failure gap: FND-PFM-LLM-064 Boundary covered Concurrency covered Invariant gap: FND-PFM-LLM-064 Interaction covered Test Scenarios 17 scenarios proving CAP-PFM-LLM-005
01 Reservation obeys the declared ceiling rule unit test
Proves BR-CAP-PFM-LLM-005-01INV-DOM-PFM-LLM-002-05 Preconditions A projected estimate exceeds its request ceiling, alongside an ordinary operation. Steps Calculate both holds. Expected The large projection is capped and the ordinary estimate stays below its ceiling. Data Fixtures representing a projected estimate exceeds its request ceiling, alongside an ordinary operation. Bound to Source services-platform:modules/llm/tests/unit/catalog/pricing.spec.ts #reserves the per-turn estimate across every turn the operation may take 04 Historical versions resist mutation invariant api negative test
Proves BR-CAP-PFM-LLM-005-04INV-DOM-PFM-LLM-002-04 Preconditions Published routing and price versions exist. Steps Attempt updates and deletions under application and publishing roles. Expected Every mutation is refused and historical contents remain intact. Data Fixtures representing published routing and price versions exist. Bound to Source services-platform:modules/llm/tests/integration/db/llm-catalog-tables.integration.spec.js #refuses an update by the owner, who publishes the versions, so no deploy can rewrite one 06 Credit and cent rounding serve different purposes boundary unit test
Proves BR-CAP-PFM-LLM-005-06INV-DOM-PFM-LLM-002-06 Preconditions Reported quantities produce fractional costs. Steps Calculate credit and cent prices over the summed dimensions. Expected Credits round upward and cents to nearest after a single final division. Data Fixtures representing reported quantities produce fractional costs. Bound to Source services-platform:modules/llm/tests/unit/catalog/pricing.spec.ts #rounds a fractional credit up, so small calls cannot slip past an allowance 08 HQ catalog access requires the staff usage key permission api negative test
Proves BR-CAP-PFM-LLM-005-10Preconditions A keyed staff reader, a customer administrator and an unkeyed staff caller are available. Steps Read catalog and rates as each caller. Expected Only keyed staff receive the catalog and rates; denied callers receive no data. Data Published catalog and three distinct security contexts. Bound to Source services-platform:modules/llm/tests/e2e/llm-usage.e2e.spec.js #answers the catalog and the rates,services-platform:modules/llm/tests/e2e/llm-usage.e2e.spec.js#refuses a staff member who holds only another HQ key,services-platform:modules/llm/tests/e2e/llm-usage.e2e.spec.js#refuses a customer’s administrator, whatever they hold at their customer’s scope 09 Stub guards protect catalog-priced checks interaction api negative proposed
Proves BR-CAP-PFM-LLM-005-06BR-CAP-PFM-LLM-006-02Preconditions A declared operation is priced and stub mode has a prepared script. Steps Run its scripted request and attempt to construct the live adapter in the same mode. Expected Scripted usage is priced normally while live-adapter construction refuses before reaching a model. Data An isolated workspace with the records, clock instants and caller authority stated in the preconditions. Source services-platform:modules/llm/catalog/pricing.ts,services-platform:modules/llm/tests/unit/runtime/anthropic-provider.spec.ts#refuses, rather than reaching a provider a test never meant to call 10 A changed estimate changes affordability interaction api negative proposed
Proves BR-CAP-PFM-LLM-005-01BR-CAP-PFM-LLM-007-02Preconditions Remaining capacity admits the old reservation but cannot admit the larger reservation in a new routing version. Steps Publish the changed estimate, resolve afresh and request admission. Expected The new estimate drives refusal without repricing or changing earlier settled requests. Data An isolated workspace with the records, clock instants and caller authority stated in the preconditions. Source services-platform:modules/llm/catalog/pricing.ts,services-platform:modules/llm/services/persistence/allowance-periods.ts 11 Reporting retains historical price identity interaction api proposed
Proves BR-CAP-PFM-LLM-005-05BR-CAP-PFM-LLM-010-08Preconditions Requests under two price versions settle on the same reporting day. Steps Rebuild the day and read model and operation breakdowns. Expected Both historical charges contribute at their recorded prices rather than the currently published rate. Data An isolated workspace with the records, clock instants and caller authority stated in the preconditions. Source services-platform:modules/llm/tests/integration/services/persistence/rate-versions.integration.spec.js #leaves the old version at the old price, because an invocation settled under it still names it,services-platform:modules/llm/services/reconciliation/rollup-settled-invocations.ts 12 Limit validation follows declared operation identity interaction api negative proposed
Proves BR-CAP-PFM-LLM-005-02BR-CAP-PFM-LLM-013-03Preconditions An organization has stored limits and a catalog with a known operation. Steps Preview disabling the known operation, then try a name absent from the catalog. Expected The declared operation is accepted for preview without writes; the unknown name is refused without changing limits or audit. Data An isolated workspace with the records, clock instants and caller authority stated in the preconditions. Source services-platform:modules/llm/services/catalog/materialize-catalog.ts,services-platform:modules/llm/services/limits/tenant-limits.ts 13 A historical price remains readable unchanged invariant api test
Proves BR-CAP-PFM-LLM-005-05Preconditions The suite owns isolated fixtures representing the valid control and the contrasting case described by its steps. Steps Publish a newer rate and read the earlier version by its identity. Expected The earlier schedule remains unchanged and does not become the current price. Data Suite-owned fixtures, with a fixed clock where dates or grant windows matter. Bound to Source services-platform:modules/llm/tests/integration/services/persistence/rate-versions.queries.integration.spec.js #answers the first by its id, unchanged, which is how a past settlement stays explicable 14 Concurrent identical publication shares one version concurrency api test
Proves BR-CAP-PFM-LLM-005-03Preconditions The suite owns isolated fixtures representing the valid control and the contrasting case described by its steps. Steps Race publication of the same routing declaration. Expected At most one append occurs and all callers receive the same version. Data Suite-owned fixtures, with a fixed clock where dates or grant windows matter. Bound to Source services-platform:modules/llm/tests/integration/services/persistence/routing-versions.integration.spec.js #answers them all the same version, so a loser agrees with the winner rather than retrying 15 Historical routing remains beside its successor invariant api test
Proves BR-CAP-PFM-LLM-005-04Preconditions The suite owns isolated fixtures representing the valid control and the contrasting case described by its steps. Steps Publish a changed routing and inspect both versions by number and history. Expected The earlier routing is unchanged and history orders the newer version first. Data Suite-owned fixtures, with a fixed clock where dates or grant windows matter. Bound to Source services-platform:modules/llm/tests/integration/services/persistence/routing-versions.queries.integration.spec.js #answers the first by its number, unchanged, which is what makes a past call attributable 16 Routing identity ignores object property order invariant unit test
Proves BR-CAP-PFM-LLM-005-03Preconditions The suite owns isolated fixtures representing the valid control and the contrasting case described by its steps. Steps Digest equal routing declarations whose object properties have different insertion order. Expected Both declarations receive the same content digest. Data Suite-owned fixtures, with a fixed clock where dates or grant windows matter. Bound to Source services-platform:modules/llm/tests/unit/catalog/digest-declarations.spec.ts #is the same for two declarations differing only in property order 17 Usage spelling preserves declared pricing edge unit test
Proves BR-CAP-PFM-LLM-005-08Preconditions The suite owns isolated fixtures representing the valid control and the contrasting case described by its steps. Steps Resolve authored and stored spellings of one priced dimension, then an unknown dimension. Expected Known spellings share the declared numerator while the unknown dimension resolves no price. Data Suite-owned fixtures, with a fixed clock where dates or grant windows matter. Bound to Source services-platform:modules/llm/tests/unit/catalog/rate-conversions.spec.ts #resolves the same dimension reported in the stored spelling ← Previous Capability Test Specification ATS-PFM-LLM-004 · undefined Next Capability Test Specification → ATS-PFM-LLM-006 · Check AI behavior with scripted models and deliberate live runs: Capability Test Specification