--- id: ATS-PFM-LLM-012 scope: CAP-PFM-LLM-012 product_area: platform module: llm revision: 2 status: proposed source_sha: services-platform: ad3f6a6582843008ed0f5a17b131878d15829af1 react-console: baf6dae382d6f41dce976c7f848667ccc37e076f scenarios: - id: ATS-PFM-LLM-012-01 title: Customer authority cannot open HQ explanations level: api proves: - BR-CAP-PFM-LLM-012-01 preconditions: A customer administrator is authenticated. steps: Request a problem explanation. expected: The HQ route refuses access. negative: true data_needs: Fixtures representing a customer administrator is authenticated. bound_to: - TST-PFM-LLM-006 provenance: test source: >- services-platform:modules/llm/tests/e2e/llm-adjustments-and-organization.e2e.spec.js#is refused 403 for a customer’s administrator, because the route is system category: permission test_sources: - >- services-platform:modules/llm/tests/e2e/llm-adjustments-and-organization.e2e.spec.js#is refused 403 for a customer’s administrator, because the route is system - id: ATS-PFM-LLM-012-02 title: Original refusal arithmetic does not change level: api proves: - BR-CAP-PFM-LLM-012-02 preconditions: A refusal is followed by an additional capacity grant. steps: Explain the same refusal before and after the grant. expected: Policy and decision-time arithmetic remain unchanged. negative: false data_needs: Fixtures representing a refusal is followed by an additional capacity grant. bound_to: - TST-PFM-LLM-025 provenance: test source: >- services-platform:modules/llm/tests/integration/services/explain/explain-invocation.integration.spec.js#reproduces the snapshot’s state as block 2, with the arithmetic as a sentence category: rule test_sources: - >- services-platform:modules/llm/tests/integration/services/explain/explain-invocation.integration.spec.js#reproduces the snapshot’s state as block 2, with the arithmetic as a sentence - >- services-platform:modules/llm/tests/integration/services/explain/explain-invocation.integration.spec.js#reproduces the snapshot’s policy as block 1 - >- services-platform:modules/llm/tests/integration/services/explain/explain-invocation.integration.spec.js#leaves blocks 1 and 2 as the snapshot had them, whatever has changed since - >- services-platform:modules/llm/tests/integration/services/explain/explain-invocation.integration.spec.js#states the in-flight count against the cap as the arithmetic - id: ATS-PFM-LLM-012-03 title: Missing grant references remain explicit level: api proves: - BR-CAP-PFM-LLM-012-03 preconditions: A snapshot references an existing grant and an unreadable grant. steps: Open its explanation. expected: The readable grant includes its window and the other remains listed as not found. negative: false data_needs: Fixtures representing a snapshot references an existing grant and an unreadable grant. bound_to: - TST-PFM-LLM-025 provenance: test source: >- services-platform:modules/llm/tests/integration/services/explain/explain-invocation.integration.spec.js#reads an override then in force back by id, with its window category: edge test_sources: - >- services-platform:modules/llm/tests/integration/services/explain/explain-invocation.integration.spec.js#reads an override then in force back by id, with its window - >- services-platform:modules/llm/tests/integration/services/explain/explain-invocation.integration.spec.js#says so when an override the snapshot named cannot be read back, rather than dropping it - id: ATS-PFM-LLM-012-04 title: Current admission check observes later capacity level: api proves: - BR-CAP-PFM-LLM-012-04 preconditions: A refused estimate becomes affordable after a grant. steps: Explain it before and after that grant. expected: 'The present check changes from refusal to admission, lists later grants and takes no hold.' negative: false data_needs: Fixtures representing a refused estimate becomes affordable after a grant. bound_to: - TST-PFM-LLM-025 provenance: test source: >- services-platform:modules/llm/tests/integration/services/explain/explain-invocation.integration.spec.js#says the same request would still be refused now, before the grant category: rule test_sources: - >- services-platform:modules/llm/tests/integration/services/explain/explain-invocation.integration.spec.js#says the same request would still be refused now, before the grant - >- services-platform:modules/llm/tests/integration/services/explain/explain-invocation.integration.spec.js#says the same request would be admitted now, after an override granted since is in force - >- services-platform:modules/llm/tests/integration/services/explain/explain-invocation.integration.spec.js#holds no credits by explaining - id: ATS-PFM-LLM-012-05 title: Failure histories disclose retries and truncation level: api proves: - BR-CAP-PFM-LLM-012-06 preconditions: Failed work has several attempts and an incomplete retained history. steps: Open its failure explanation. expected: 'Classification, outcome and attempt verdicts appear with an incomplete-history indication.' negative: false data_needs: Fixtures representing failed work has several attempts and an incomplete retained history. bound_to: - TST-PFM-LLM-025 provenance: test source: >- services-platform:modules/llm/tests/integration/services/explain/explain-invocation.integration.spec.js#lists the attempts in order with their status codes and the verdict on each category: failure test_sources: - >- services-platform:modules/llm/tests/integration/services/explain/explain-invocation.integration.spec.js#lists the attempts in order with their status codes and the verdict on each - >- services-platform:modules/llm/tests/integration/services/explain/explain-invocation.integration.spec.js#carries each attempt’s error name and latency - >- services-platform:modules/llm/tests/integration/services/explain/explain-invocation.integration.spec.js#says the account of a long conversation is incomplete when attempts were truncated - id: ATS-PFM-LLM-012-06 title: Healthy or foreign requests do not explain level: api proves: - BR-CAP-PFM-LLM-012-08 preconditions: 'Healthy settled, nonexistent and foreign problem requests are represented.' steps: Request explanations for all three. expected: Healthy work returns nothing-to-explain while missing and foreign work returns not found. negative: true data_needs: 'Fixtures representing healthy settled, nonexistent and foreign problem requests are represented.' bound_to: - TST-PFM-LLM-006 - TST-PFM-LLM-025 provenance: test source: >- services-platform:modules/llm/tests/e2e/llm-adjustments-and-organization.e2e.spec.js#says 409 IllegalOperation for a request that settled without error category: validation test_sources: - >- services-platform:modules/llm/tests/e2e/llm-adjustments-and-organization.e2e.spec.js#says 409 IllegalOperation for a request that settled without error - >- services-platform:modules/llm/tests/e2e/llm-adjustments-and-organization.e2e.spec.js#says 404 for a request the tenant does not have - >- services-platform:modules/llm/tests/integration/services/explain/explain-invocation.integration.spec.js#refuses another organization’s request as not found, though it is refused and would explain - id: ATS-PFM-LLM-012-07 title: Support follows a refusal from overview to explanation level: browser proves: - AS-JNY-PFM-LLM-002-01 - AS-JNY-PFM-LLM-002-02 - AS-JNY-PFM-LLM-002-03 preconditions: An organization has a refused request. steps: 'Open its overview, request directory, request detail and explanation.' expected: >- Displayed account figures agree and the explanation distinguishes the original refusal from present admission without showing captured content. negative: false data_needs: Fixtures representing an organization has a refused request. bound_to: [] provenance: proposed source: FND-PFM-LLM-027 category: interaction test_sources: [] - id: ATS-PFM-LLM-012-08 title: Later capacity changes only present admission category: interaction level: api proves: - BR-CAP-PFM-LLM-012-04 - BR-CAP-PFM-LLM-014-07 exercises: - CAP-PFM-LLM-012 - CAP-PFM-LLM-014 preconditions: A refusal snapshot precedes a capacity grant. steps: 'Explain before granting, after granting and after revocation.' expected: >- The present affordability check follows the grant while retained policy and decision-time arithmetic remain unchanged. negative: false data_needs: 'An isolated workspace with the records, clock instants and caller authority stated in the preconditions.' bound_to: [] provenance: proposed source: - 'services-platform:modules/llm/services/explain/explain-invocation.ts' - 'services-platform:modules/llm/services/overrides/allowance-overrides.ts' - id: ATS-PFM-LLM-012-09 title: Explanation keeps retry verdicts readable category: failure level: unit proves: - BR-CAP-PFM-LLM-012-06 preconditions: >- The suite owns isolated fixtures representing the valid control and the contrasting case described by its steps. steps: Derive explanation verdicts for retryable and terminal attempt classifications. expected: The verdict distinguishes temporary retry guidance from outcomes that another attempt cannot repair. negative: true data_needs: 'Suite-owned fixtures, with a fixed clock where dates or grant windows matter.' bound_to: - TST-PFM-LLM-074 provenance: test source: >- services-platform:modules/llm/tests/unit/services/explain/explain.spec.ts#is retries_spent for the last failure of a retryable status test_sources: - >- services-platform:modules/llm/tests/unit/services/explain/explain.spec.ts#is retries_spent for the last failure of a retryable status evidence: - >- services-platform at ad3f6a6582843008ed0f5a17b131878d15829af1: bound or cited modules/llm/tests/e2e/llm-adjustments-and-organization.e2e.spec.js; modules/llm/tests/integration/services/explain/explain-invocation.integration.spec.js. - >- react-console at baf6dae382d6f41dce976c7f848667ccc37e076f: bound or cited e2e/dev/hq-ai-usage.dev.spec.ts; e2e/mock/hq-ai-usage.mock.spec.ts. coverage: rule: 'gap: FND-PFM-LLM-071' transition: not-applicable validation: covered permission: covered edge: covered failure: covered boundary: 'gap: FND-PFM-LLM-071' concurrency: 'gap: FND-PFM-LLM-071' invariant: 'gap: FND-PFM-LLM-071' interaction: covered coverage_notes: transition: This capability defines read-only presentation or validation and introduces no owned lifecycle transition. interaction: >- The related interaction register cites a concrete experiment for every dependency and related capability, including cross-capability workflow seams. invariant: >- The domain-invariant register maps each relevant invariant to a scenario or to the capability coverage finding; an example does not establish every invariant. rule: >- All must-prove rules have scenario references, but the coverage finding retains untested clauses and combined outcomes; scenario presence alone is not exhaustive rule proof. related_interactions: - with: CAP-PFM-LLM-002 scenarios: - ATS-PFM-LLM-002-13 source: >- services-platform:modules/llm/tests/unit/runtime/run-tool-loop.spec.ts#stops a conversation that will not converge - with: CAP-PFM-LLM-007 scenarios: - ATS-PFM-LLM-007-15 source: 'services-platform:modules/llm/services/persistence/allowance-periods.ts' - with: CAP-PFM-LLM-011 scenarios: - ATS-PFM-LLM-011-09 source: >- services-platform:modules/llm/tests/e2e/llm-usage.e2e.spec.js#refuses a customer’s administrator a tenant’s usage even in their own tenant - with: CAP-PFM-LLM-014 scenarios: - ATS-PFM-LLM-012-08 source: >- services-platform:modules/llm/tests/e2e/llm-overrides.e2e.spec.js#is refused the grant before a handler runs, whatever the customer’s scope grants state_models: [] coverage_obligations: must_prove: - rule: BR-CAP-PFM-LLM-012-01 scenarios: - ATS-PFM-LLM-012-01 source: >- services-platform:modules/llm/tests/e2e/llm-adjustments-and-organization.e2e.spec.js#is refused 403 for a customer’s administrator, because the route is system coverage: specified - rule: BR-CAP-PFM-LLM-012-02 scenarios: - ATS-PFM-LLM-012-02 source: 'services-platform:modules/llm/services/explain/explain-invocation.ts' coverage: specified - rule: BR-CAP-PFM-LLM-012-03 scenarios: - ATS-PFM-LLM-012-03 source: >- services-platform:modules/llm/tests/integration/services/explain/explain-invocation.integration.spec.js#says so when an override the snapshot named cannot be read back, rather than dropping it coverage: partial gap: FND-PFM-LLM-071 - rule: BR-CAP-PFM-LLM-012-04 scenarios: - ATS-PFM-LLM-012-04 - ATS-PFM-LLM-012-08 source: 'services-platform:modules/llm/services/explain/explain-invocation.ts' coverage: partial gap: FND-PFM-LLM-071 - rule: BR-CAP-PFM-LLM-012-06 scenarios: - ATS-PFM-LLM-012-05 - ATS-PFM-LLM-012-09 source: 'services-platform:modules/llm/services/explain/explain-invocation.ts' coverage: partial gap: FND-PFM-LLM-071 - rule: BR-CAP-PFM-LLM-012-08 scenarios: - ATS-PFM-LLM-012-06 source: 'services-platform:modules/llm/services/explain/explain-invocation.ts' coverage: specified state_transitions: [] domain_invariants: - invariant: INV-DOM-PFM-LLM-004-05 scenarios: [] gap: FND-PFM-LLM-071 provenance: proposed source: 'services-platform:modules/llm/aws/api/llm-usage-api-mapper.ts:toInvocationDetailResponse' coverage_limits: finding: FND-PFM-LLM-071 statement: >- Concurrent changes while the present-position blocks are assembled and snapshot-free or truncated explanations need combined boundary proof. A complete support journey remains unexecuted. provenance: proposed source: - >- services-platform:modules/llm/tests/e2e/llm-adjustments-and-organization.e2e.spec.js#is refused 403 for a customer’s administrator, because the route is system - 'services-platform:modules/llm/services/explain/explain-invocation.ts' - >- services-platform:modules/llm/tests/integration/services/explain/explain-invocation.integration.spec.js#says so when an override the snapshot named cannot be read back, rather than dropping it - 'services-platform:modules/llm/services/explain/explain-invocation.ts' - 'services-platform:modules/llm/services/explain/explain-invocation.ts' - 'services-platform:modules/llm/services/explain/explain-invocation.ts' - 'services-platform:modules/llm/aws/api/llm-usage-api-mapper.ts:toInvocationDetailResponse' invariants_without_scenarios: - INV-DOM-PFM-LLM-004-05 --- # Explain an AI refusal or unsuccessful conversation: Capability Test Specification ## Coverage | Rule | Scenario | |---|---| | `BR-CAP-PFM-LLM-012-01` | `ATS-PFM-LLM-012-01` | | `BR-CAP-PFM-LLM-012-02` | `ATS-PFM-LLM-012-02` | | `BR-CAP-PFM-LLM-012-03` | `ATS-PFM-LLM-012-03` | | `BR-CAP-PFM-LLM-012-04` | `ATS-PFM-LLM-012-04` | | `BR-CAP-PFM-LLM-012-06` | `ATS-PFM-LLM-012-05` | | `BR-CAP-PFM-LLM-012-08` | `ATS-PFM-LLM-012-06` | ## Interactions With Related Capabilities The interaction register links concrete experiments with bound ai conversations and handle recoverable failures; admit ai work against the organization’s monthly allowance; inspect an organization’s ai account and request history; provide temporary ai capacity and revoke it. Each experiment checks the outcome at the shared boundary; an unbound experiment remains proposed. Concurrent changes while the present-position blocks are assembled and snapshot-free or truncated explanations need combined boundary proof. A complete support journey remains unexecuted.