--- id: ATS-PFM-LLM-010 scope: CAP-PFM-LLM-010 product_area: platform module: llm revision: 2 status: proposed source_sha: services-platform: ad3f6a6582843008ed0f5a17b131878d15829af1 react-console: baf6dae382d6f41dce976c7f848667ccc37e076f scenarios: - id: ATS-PFM-LLM-010-01 title: Unqualified callers cannot inspect system usage level: api proves: - BR-CAP-PFM-LLM-010-01 preconditions: 'Customer authority, an unrelated staff key and absent security context are represented.' steps: Request system usage as each caller. expected: Every caller is refused. negative: true data_needs: 'Authorized and unauthorized fixtures for the scenario, with controlled request and usage records.' bound_to: - TST-PFM-LLM-009 - TST-PFM-LLM-054 provenance: test source: >- services-platform:modules/llm/tests/e2e/llm-usage.e2e.spec.js#refuses a customer’s administrator, whatever they hold at their customer’s scope category: permission test_sources: - >- services-platform:modules/llm/tests/e2e/llm-usage.e2e.spec.js#refuses a customer’s administrator, whatever they hold at their customer’s scope - >- services-platform:modules/llm/tests/e2e/llm-usage.e2e.spec.js#refuses a staff member who holds only another HQ key - >- services-platform:modules/llm/tests/e2e/llm-usage.e2e.spec.js#refuses a caller with no security context - >- services-platform:modules/llm/tests/unit/aws/api/llm-usage-permissions.spec.ts#serves every Rhythm HQ route as a system route - id: ATS-PFM-LLM-010-02 title: Windows have preceding comparisons level: api proves: - BR-CAP-PFM-LLM-010-02 preconditions: Usage spans current and preceding reporting periods. steps: Read supported windows and submit an unsupported window. expected: Supported reports include their preceding comparison and invalid windows return a named input error. negative: false data_needs: Fixtures representing usage spans current and preceding reporting periods. bound_to: - TST-PFM-LLM-049 - TST-PFM-LLM-009 - TST-PFM-LLM-082 provenance: test source: >- services-platform:modules/llm/tests/integration/services/usage-reports/platform-usage.integration.spec.js#is available, with the window it covers and the window before it category: validation test_sources: - >- services-platform:modules/llm/tests/integration/services/usage-reports/platform-usage.integration.spec.js#is available, with the window it covers and the window before it - >- services-platform:modules/llm/tests/integration/services/usage-reports/platform-usage.integration.spec.js#answers the settled credits and cents with the window before for the delta - >- services-platform:modules/llm/tests/e2e/llm-usage.e2e.spec.js#answers 400 naming the window it does not know - >- services-platform:modules/llm/tests/unit/services/usage-reports/usage-windows.spec.ts#starts twenty-nine days before today, which makes thirty days with today - id: ATS-PFM-LLM-010-03 title: Live status follows capacity and access level: unit proves: - BR-CAP-PFM-LLM-010-03 - INV-DOM-PFM-LLM-004-08 preconditions: >- Balances represent below-warning, warning, full, excess and zero allowance, disabled AI and organization, and newly added capacity. steps: Derive each status. expected: States follow committed capacity and access; new capacity can restore a previously constrained position. negative: false data_needs: >- Fixtures representing balances represent below-warning, warning, full, excess and zero allowance, disabled AI and organization, and newly added capacity. bound_to: - TST-PFM-LLM-081 provenance: test source: >- services-platform:modules/llm/tests/unit/services/usage-reports/usage-status.spec.ts#is warned at exactly eighty percent category: transition test_sources: - >- services-platform:modules/llm/tests/unit/services/usage-reports/usage-status.spec.ts#is warned at exactly eighty percent - >- services-platform:modules/llm/tests/unit/services/usage-reports/usage-status.spec.ts#is hard stopped when all of the allowance is committed - >- services-platform:modules/llm/tests/unit/services/usage-reports/usage-status.spec.ts#is disabled when the limits row switches AI off, whatever has been used - >- services-platform:modules/llm/tests/unit/services/usage-reports/usage-status.spec.ts#is within again when an override raises the allowance past a crossing already recorded - id: ATS-PFM-LLM-010-04 title: Utilisation handles excess and zero capacity level: unit proves: - BR-CAP-PFM-LLM-010-04 preconditions: 'Normal, over-allowance and zero-allowance positions exist.' steps: Calculate the percentages. expected: 'Values round to one decimal, exceed 100% when applicable and remain empty for zero allowance.' negative: false data_needs: 'Fixtures representing normal, over-allowance and zero-allowance positions exist.' bound_to: - TST-PFM-LLM-081 provenance: test source: >- services-platform:modules/llm/tests/unit/services/usage-reports/usage-status.spec.ts#is the committed share of the allowance to one decimal category: boundary test_sources: - >- services-platform:modules/llm/tests/unit/services/usage-reports/usage-status.spec.ts#is the committed share of the allowance to one decimal - >- services-platform:modules/llm/tests/unit/services/usage-reports/usage-status.spec.ts#is more than a hundred when the period is over its allowance - >- services-platform:modules/llm/tests/unit/services/usage-reports/usage-status.spec.ts#is null when the allowance is zero, which has no share - id: ATS-PFM-LLM-010-05 title: Trend folding preserves all small series level: api proves: - BR-CAP-PFM-LLM-010-05 preconditions: More than ten dimensions have reported usage. steps: Read the grouped trend. expected: 'The ten largest remain separate, Other exactly sums the rest and empty dates are absent.' negative: false data_needs: Fixtures representing more than ten dimensions have reported usage. bound_to: - TST-PFM-LLM-049 provenance: test source: >- services-platform:modules/llm/tests/integration/services/usage-reports/platform-usage.integration.spec.js#keeps the ten largest series and folds the rest into one called other category: rule test_sources: - >- services-platform:modules/llm/tests/integration/services/usage-reports/platform-usage.integration.spec.js#keeps the ten largest series and folds the rest into one called other - >- services-platform:modules/llm/tests/integration/services/usage-reports/platform-usage.integration.spec.js#totals the folded series from exactly the series it folded - >- services-platform:modules/llm/tests/integration/services/usage-reports/platform-usage.integration.spec.js#answers a day’s point as money and a request count, days without usage absent - id: ATS-PFM-LLM-010-06 title: Daily rebuilding keeps classifications and organization boundaries level: api proves: - BR-CAP-PFM-LLM-010-08 - INV-DOM-PFM-LLM-004-01 - INV-DOM-PFM-LLM-004-02 preconditions: >- Closed, refused, failed, released and running work spans two organizations and UTC dates, with a late settlement. steps: Rebuild before and after the late settlement. expected: >- Closed classifications and applicable costs remain in their start-date cells, running work is excluded and only the late settlement’s pair rebuilds. negative: false data_needs: >- Fixtures representing closed, refused, failed, released and running work spans two organizations and UTC dates, with a late settlement. bound_to: - TST-PFM-LLM-047 provenance: test source: >- services-platform:modules/llm/tests/integration/services/reconciliation/rollup-settled-invocations.integration.spec.js#counts a call on the day it started even when it settled on the next category: rule test_sources: - >- services-platform:modules/llm/tests/integration/services/reconciliation/rollup-settled-invocations.integration.spec.js#counts a call on the day it started even when it settled on the next - >- services-platform:modules/llm/tests/integration/services/reconciliation/rollup-settled-invocations.integration.spec.js#gives a failed call the credits and tokens its attempts cost - >- services-platform:modules/llm/tests/integration/services/reconciliation/rollup-settled-invocations.integration.spec.js#counts a refused call with zero credits, no rate version and no attempts - >- services-platform:modules/llm/tests/integration/services/reconciliation/rollup-settled-invocations.integration.spec.js#leaves a call still in flight out of the rollup - >- services-platform:modules/llm/tests/integration/services/reconciliation/rollup-settled-invocations.integration.spec.js#rebuilds only the pair the late settlement belongs to - id: ATS-PFM-LLM-010-07 title: Progress resumes from unfinished changes level: api proves: - BR-CAP-PFM-LLM-010-09 - INV-DOM-PFM-LLM-004-03 preconditions: A rebuild has insufficient time for all changed pairs. steps: Run a bounded pass followed by a completing pass. expected: The watermark stops at the earliest unfinished change then advances on completion without regression. negative: false data_needs: Fixtures representing a rebuild has insufficient time for all changed pairs. bound_to: - TST-PFM-LLM-047 provenance: test source: >- services-platform:modules/llm/tests/integration/services/reconciliation/rollup-settled-invocations.integration.spec.js#advances the watermark to the run start less the settlement margin category: invariant test_sources: - >- services-platform:modules/llm/tests/integration/services/reconciliation/rollup-settled-invocations.integration.spec.js#advances the watermark to the run start less the settlement margin - >- services-platform:modules/llm/tests/integration/services/reconciliation/rollup-settled-invocations.integration.spec.js#stops the watermark at the earliest change it did not finish, not at the run start - >- services-platform:modules/llm/tests/integration/services/reconciliation/rollup-settled-invocations.integration.spec.js#is taken up by the next run, which finishes the rest and completes - id: ATS-PFM-LLM-010-08 title: HQ cost formatting pairs dollars and credits level: browser proves: - BR-CAP-PFM-LLM-010-10 preconditions: A credit-and-cent cost pair is available. steps: Format it for HQ. expected: The result presents dollars together with credits. negative: false data_needs: Fixtures representing a credit-and-cent cost pair is available. bound_to: - TST-PFM-LLM-003 provenance: test source: 'react-console:lib/hq/ai-usage-format.test.ts#shows cents as dollars beside the credits' category: rule test_sources: - 'react-console:lib/hq/ai-usage-format.test.ts#shows cents as dollars beside the credits' - id: ATS-PFM-LLM-010-09 title: HQ navigation preserves organization selection category: interaction level: browser proves: - BR-CAP-PFM-LLM-010-01 - BR-CAP-PFM-LLM-011-02 exercises: - CAP-PFM-LLM-010 - CAP-PFM-LLM-011 preconditions: HQ reporting lists an organization with request history. steps: Open its organization card and use View Requests. expected: The directory opens with that organization already selected and subsequent details remain scoped to it. negative: false data_needs: 'An isolated workspace with the records, clock instants and caller authority stated in the preconditions.' bound_to: [] provenance: proposed source: - >- services-platform:modules/llm/tests/e2e/llm-usage.e2e.spec.js#refuses a customer’s administrator, whatever they hold at their customer’s scope - >- services-platform:modules/llm/tests/e2e/llm-usage.e2e.spec.js#answers 404 for a request of another tenant, the tenant policy binding the read - id: ATS-PFM-LLM-010-10 title: Daily cells have a unique reporting identity category: invariant level: api proves: - BR-CAP-PFM-LLM-010-08 preconditions: >- The suite owns isolated fixtures representing the valid control and the contrasting case described by its steps. steps: 'Insert two cells with the same day, workspace, dimensions and rate version.' expected: The second cell is refused rather than adding another copy of the charge. negative: false data_needs: 'Suite-owned fixtures, with a fixed clock where dates or grant windows matter.' bound_to: - TST-PFM-LLM-017 provenance: test source: >- services-platform:modules/llm/tests/integration/db/llm-usage-daily.table.integration.spec.js#refuses a second cell with the same day, organization, dimensions and rate version test_sources: - >- services-platform:modules/llm/tests/integration/db/llm-usage-daily.table.integration.spec.js#refuses a second cell with the same day, organization, dimensions and rate version - id: ATS-PFM-LLM-010-11 title: Reporting progress cannot move backward category: invariant level: api proves: - BR-CAP-PFM-LLM-010-09 preconditions: >- The suite owns isolated fixtures representing the valid control and the contrasting case described by its steps. steps: Write a progress instant and then attempt to write an earlier instant. expected: The retained watermark remains at the later instant. negative: false data_needs: 'Suite-owned fixtures, with a fixed clock where dates or grant windows matter.' bound_to: - TST-PFM-LLM-044 provenance: test source: >- services-platform:modules/llm/tests/integration/services/persistence/usage-rollups.queries.integration.spec.js#never moves backwards test_sources: - >- services-platform:modules/llm/tests/integration/services/persistence/usage-rollups.queries.integration.spec.js#never moves backwards - id: ATS-PFM-LLM-010-12 title: Execution leaders disclose unreadable workspaces category: edge level: api proves: - BR-CAP-PFM-LLM-010-07 preconditions: >- The suite owns isolated fixtures representing the valid control and the contrasting case described by its steps. steps: Collect closed task leaders with one workspace read made to fail. expected: Available leaders remain usable and the response separately identifies the unreadable workspace. negative: false data_needs: 'Suite-owned fixtures, with a fixed clock where dates or grant windows matter.' bound_to: - TST-PFM-LLM-048 provenance: test source: >- services-platform:modules/llm/tests/integration/services/usage-reports/execution-leaders.integration.spec.js#names a tenant whose read failed and still answers the others test_sources: - >- services-platform:modules/llm/tests/integration/services/usage-reports/execution-leaders.integration.spec.js#names a tenant whose read failed and still answers the others evidence: - >- services-platform at ad3f6a6582843008ed0f5a17b131878d15829af1: bound or cited modules/llm/tests/e2e/llm-usage.e2e.spec.js; modules/llm/tests/integration/services/reconciliation/rollup-settled-invocations.integration.spec.js; modules/llm/tests/integration/services/usage-reports/platform-usage.integration.spec.js; modules/llm/tests/unit/aws/api/llm-usage-permissions.spec.ts; modules/llm/tests/unit/services/usage-reports/usage-status.spec.ts; modules/llm/tests/unit/services/usage-reports/usage-windows.spec.ts. - 'react-console at baf6dae382d6f41dce976c7f848667ccc37e076f: bound or cited lib/hq/ai-usage-format.test.ts.' coverage: rule: 'gap: FND-PFM-LLM-069' transition: 'gap: FND-PFM-LLM-069' validation: covered permission: covered edge: covered failure: 'gap: FND-PFM-LLM-069' boundary: covered concurrency: 'gap: FND-PFM-LLM-069' invariant: 'gap: FND-PFM-LLM-069' interaction: covered coverage_notes: transition: >- Existing transition scenarios are retained; the model transition register below records the remaining exhaustive mapping obligation. interaction: >- The related interaction register cites a concrete experiment for every dependency and related capability, including cross-capability workflow seams. invariant: >- The domain-invariant register maps each relevant invariant to a scenario or to the capability coverage finding; an example does not establish every invariant. rule: >- All must-prove rules have scenario references, but the coverage finding retains untested clauses and combined outcomes; scenario presence alone is not exhaustive rule proof. related_interactions: - with: CAP-PFM-LLM-005 scenarios: - ATS-PFM-LLM-005-11 source: 'services-platform:modules/llm/catalog/pricing.ts' - with: CAP-PFM-LLM-007 scenarios: - ATS-PFM-LLM-007-13 source: 'services-platform:modules/llm/services/persistence/allowance-periods.ts' - with: CAP-PFM-LLM-009 scenarios: - ATS-PFM-LLM-009-06 source: 'services-platform:modules/llm/aws/lambda/RECONCILIATION.handler.ts' - with: CAP-PFM-LLM-011 scenarios: - ATS-PFM-LLM-010-09 source: >- services-platform:modules/llm/tests/e2e/llm-usage.e2e.spec.js#refuses a customer’s administrator a tenant’s usage even in their own tenant state_models: - STATE-PFM-LLM-005 coverage_obligations: must_prove: - rule: BR-CAP-PFM-LLM-010-01 scenarios: - ATS-PFM-LLM-010-01 - ATS-PFM-LLM-010-09 source: >- services-platform:modules/llm/tests/e2e/llm-usage.e2e.spec.js#refuses a customer’s administrator, whatever they hold at their customer’s scope coverage: specified - rule: BR-CAP-PFM-LLM-010-02 scenarios: - ATS-PFM-LLM-010-02 source: 'services-platform:modules/llm/services/usage-reports/platform-usage.ts' coverage: specified - rule: BR-CAP-PFM-LLM-010-03 scenarios: - ATS-PFM-LLM-010-03 source: 'services-platform:modules/llm/services/usage-reports/usage-status.ts' coverage: partial gap: FND-PFM-LLM-069 - rule: BR-CAP-PFM-LLM-010-04 scenarios: - ATS-PFM-LLM-010-04 source: 'services-platform:modules/llm/services/usage-reports/usage-status.ts' coverage: specified - rule: BR-CAP-PFM-LLM-010-05 scenarios: - ATS-PFM-LLM-010-05 source: 'services-platform:modules/llm/services/usage-reports/platform-usage.ts' coverage: specified - rule: BR-CAP-PFM-LLM-010-08 scenarios: - ATS-PFM-LLM-010-06 - ATS-PFM-LLM-010-10 source: 'services-platform:modules/llm/services/reconciliation/rollup-settled-invocations.ts' coverage: partial gap: FND-PFM-LLM-069 - rule: BR-CAP-PFM-LLM-010-09 scenarios: - ATS-PFM-LLM-010-07 - ATS-PFM-LLM-010-11 source: 'services-platform:modules/llm/services/reconciliation/rollup-settled-invocations.ts' coverage: partial gap: FND-PFM-LLM-069 - rule: BR-CAP-PFM-LLM-010-10 scenarios: - ATS-PFM-LLM-010-08 source: 'react-console:lib/hq/ai-usage-format.test.ts#shows cents as dollars beside the credits' coverage: specified state_transitions: - model: STATE-PFM-LLM-005 transition: 1 from: within to: warned trigger: Commitments increase or effective capacity decreases into warning range. gap: FND-PFM-LLM-069 provenance: proposed source: 'services-platform:modules/llm/services/usage-reports/usage-status.ts:deriveUsageStatus' - model: STATE-PFM-LLM-005 transition: 2 from: warned to: hard_stopped trigger: Committed usage reaches full effective capacity. gap: FND-PFM-LLM-069 provenance: proposed source: 'services-platform:modules/llm/services/usage-reports/usage-status.ts:deriveUsageStatus' - model: STATE-PFM-LLM-005 transition: 3 from: within to: hard_stopped trigger: A large movement skips directly from below warning to full usage. gap: FND-PFM-LLM-069 provenance: proposed source: 'services-platform:modules/llm/services/usage-reports/usage-status.ts:deriveUsageStatus' - model: STATE-PFM-LLM-005 transition: 4 from: hard_stopped to: within trigger: 'Capacity rises, commitments fall or the month changes.' gap: FND-PFM-LLM-069 provenance: proposed source: 'services-platform:modules/llm/services/overrides/allowance-overrides.ts:grantOverride' - model: STATE-PFM-LLM-005 transition: 5 from: hard_stopped to: warned trigger: Capacity returns without removing the warning. gap: FND-PFM-LLM-069 provenance: proposed source: 'services-platform:modules/llm/services/usage-reports/usage-status.ts:deriveUsageStatus' - model: STATE-PFM-LLM-005 transition: 6 from: warned to: within trigger: Capacity increases or commitments decrease below warning. gap: FND-PFM-LLM-069 provenance: proposed source: 'services-platform:modules/llm/services/usage-reports/usage-status.ts:deriveUsageStatus' - model: STATE-PFM-LLM-005 transition: 7 from: within to: disabled trigger: The AI switch is disabled or organization availability changes. gap: FND-PFM-LLM-069 provenance: proposed source: 'services-platform:modules/llm/services/limits/tenant-limits.ts:checkOperationAccess' - model: STATE-PFM-LLM-005 transition: 8 from: disabled to: within trigger: AI and organization availability return. gap: FND-PFM-LLM-069 provenance: proposed source: 'services-platform:modules/llm/services/limits/tenant-limits.ts:upsertTenantLimits' domain_invariants: - invariant: INV-DOM-PFM-LLM-004-01 scenarios: - ATS-PFM-LLM-010-06 provenance: documented source: 'services-platform:modules/llm/services/persistence/usage-rollups.ts:rebuildUsageDay' - invariant: INV-DOM-PFM-LLM-004-02 scenarios: - ATS-PFM-LLM-010-06 provenance: documented source: 'services-platform:docs/llm.md#The daily rollup' - invariant: INV-DOM-PFM-LLM-004-03 scenarios: - ATS-PFM-LLM-010-07 provenance: documented source: >- services-platform:modules/llm/services/reconciliation/rollup-settled-invocations.ts:rollUpSettledInvocations - invariant: INV-DOM-PFM-LLM-004-04 scenarios: [] gap: FND-PFM-LLM-069 provenance: proposed source: 'services-platform:docs/llm.md#Reporting' - invariant: INV-DOM-PFM-LLM-004-07 scenarios: [] gap: FND-PFM-LLM-069 provenance: proposed source: 'services-platform:modules/llm/services/usage-reports/section.ts:readSection' - invariant: INV-DOM-PFM-LLM-004-08 scenarios: - ATS-PFM-LLM-010-03 provenance: documented source: 'services-platform:modules/llm/services/usage-reports/usage-status.ts:deriveUsageStatus' coverage_limits: finding: FND-PFM-LLM-069 statement: >- Unavailable-section rendering, independent live position reads during changes and unreadable-workspace reporting progress need feature-level proof. The browser probe verifies loading rather than accounting agreement. provenance: proposed source: - >- services-platform:modules/llm/tests/e2e/llm-usage.e2e.spec.js#refuses a customer’s administrator, whatever they hold at their customer’s scope - 'services-platform:modules/llm/services/usage-reports/platform-usage.ts' - 'services-platform:modules/llm/services/usage-reports/usage-status.ts' - 'services-platform:modules/llm/services/usage-reports/usage-status.ts' - 'services-platform:modules/llm/services/usage-reports/platform-usage.ts' - 'services-platform:modules/llm/services/reconciliation/rollup-settled-invocations.ts' - 'services-platform:modules/llm/services/reconciliation/rollup-settled-invocations.ts' - 'react-console:lib/hq/ai-usage-format.test.ts#shows cents as dollars beside the credits' - 'services-platform:docs/llm.md#Reporting' - 'services-platform:modules/llm/services/usage-reports/section.ts:readSection' invariants_without_scenarios: - INV-DOM-PFM-LLM-004-04 - INV-DOM-PFM-LLM-004-07 --- # undefined ## Coverage The scenarios identify the rules or journey assertions they are intended to prove. A test binding names existing evidence; an unbound proposal remains work to verify. | Scenario | Title | Level | Provenance | |---|---|---|---| | `ATS-PFM-LLM-010-01` | Unqualified callers cannot inspect system usage | api | test | | `ATS-PFM-LLM-010-02` | Windows have preceding comparisons | api | test | | `ATS-PFM-LLM-010-03` | Live status follows capacity and access | unit | test | | `ATS-PFM-LLM-010-04` | Utilisation handles excess and zero capacity | unit | test | | `ATS-PFM-LLM-010-05` | Trend folding preserves all small series | api | test | | `ATS-PFM-LLM-010-06` | Daily rebuilding keeps classifications and organization boundaries | api | test | | `ATS-PFM-LLM-010-07` | Progress resumes from unfinished changes | api | test | | `ATS-PFM-LLM-010-08` | HQ cost formatting pairs dollars and credits | browser | test | ## Interactions With Related Capabilities The interaction register links concrete experiments with publish ai operation definitions and historical prices; admit ai work against the organization’s monthly allowance; recover interrupted requests and identify accounting drift; inspect an organization’s ai account and request history. Each experiment checks the outcome at the shared boundary; an unbound experiment remains proposed. Unavailable-section rendering, independent live position reads during changes and unreadable-workspace reporting progress need feature-level proof. The browser probe verifies loading rather than accounting agreement.