--- title: Platform LLM testing back-fill examined_at: '2026-10-23' examiner: codex source_sha: services-platform: ad3f6a6582843008ed0f5a17b131878d15829af1 react-console: baf6dae382d6f41dce976c7f848667ccc37e076f rhythm-v2-docs: 9b95399181c8ec6670ce3d12d700e59643668692 branch: agent/platform-llm-testing-1 counts: specifications_extended: 20 existing_scenarios: 148 scenarios_added: 86 scenarios_added_by_category: rule: 6 transition: 2 validation: 8 permission: 2 edge: 3 failure: 4 boundary: 6 concurrency: 3 invariant: 7 interaction: 45 feature_suites_written: 7 feature_scenarios_written: 35 executable_suites_registered: 87 executable_suites_by_kind: unit: 38 integration: 43 api-lifecycle: 4 browser-lifecycle: 2 bindings_made: 190 bound_capability_scenarios: 166 bound_feature_scenarios: 1 gap_findings: 21 contradiction_findings: 1 category_gaps: 98 unbound_capability_scenarios: 68 unbound_feature_scenarios: 34 scope: source_tests: - 'react-console:e2e/dev/hq-ai-usage.dev.spec.ts' - 'react-console:e2e/mock/hq-ai-usage.mock.spec.ts' - 'react-console:lib/hq/ai-usage-format.test.ts' - 'react-console:lib/hq/ai-usage.test.ts' - 'services-platform:modules/llm/tests/e2e/allowance-warning.e2e.spec.js' - 'services-platform:modules/llm/tests/e2e/llm-adjustments-and-organization.e2e.spec.js' - 'services-platform:modules/llm/tests/e2e/llm-limits.e2e.spec.js' - 'services-platform:modules/llm/tests/e2e/llm-overrides.e2e.spec.js' - 'services-platform:modules/llm/tests/e2e/llm-usage.e2e.spec.js' - 'services-platform:modules/llm/tests/integration/db/llm-allowance-overrides.table.integration.spec.js' - 'services-platform:modules/llm/tests/integration/db/llm-catalog-tables.integration.spec.js' - 'services-platform:modules/llm/tests/integration/db/llm-executions.table.integration.spec.js' - 'services-platform:modules/llm/tests/integration/db/llm-invocations-query.view.integration.spec.js' - 'services-platform:modules/llm/tests/integration/db/llm-invocations.table.integration.spec.js' - 'services-platform:modules/llm/tests/integration/db/llm-overrides-migration.integration.spec.js' - 'services-platform:modules/llm/tests/integration/db/llm-tenant-limits.table.integration.spec.js' - 'services-platform:modules/llm/tests/integration/db/llm-usage-daily.table.integration.spec.js' - 'services-platform:modules/llm/tests/integration/lifecycle/executions.integration.spec.js' - 'services-platform:modules/llm/tests/integration/lifecycle/llm-ledger-concurrency.integration.spec.js' - 'services-platform:modules/llm/tests/integration/lifecycle/run-operation.integration.spec.js' - 'services-platform:modules/llm/tests/integration/lifecycle/tenant-limits.integration.spec.js' - >- services-platform:modules/llm/tests/integration/services/adjustments/adjust-allowance.integration.spec.js - 'services-platform:modules/llm/tests/integration/services/catalog/materialize-catalog.integration.spec.js' - 'services-platform:modules/llm/tests/integration/services/evidence/evidence-access.integration.spec.js' - 'services-platform:modules/llm/tests/integration/services/explain/explain-invocation.integration.spec.js' - 'services-platform:modules/llm/tests/integration/services/limits/tenant-limits.integration.spec.js' - >- services-platform:modules/llm/tests/integration/services/maintenance/tenant-participant.integration.spec.js - 'services-platform:modules/llm/tests/integration/services/notices/threshold-notice.integration.spec.js' - >- services-platform:modules/llm/tests/integration/services/overrides/allowance-overrides.integration.spec.js - >- services-platform:modules/llm/tests/integration/services/persistence/allowance-periods.integration.spec.js - >- services-platform:modules/llm/tests/integration/services/persistence/allowance-periods.queries.integration.spec.js - 'services-platform:modules/llm/tests/integration/services/persistence/executions.integration.spec.js' - >- services-platform:modules/llm/tests/integration/services/persistence/executions.queries.integration.spec.js - 'services-platform:modules/llm/tests/integration/services/persistence/invocations.integration.spec.js' - >- services-platform:modules/llm/tests/integration/services/persistence/invocations.queries.integration.spec.js - 'services-platform:modules/llm/tests/integration/services/persistence/limits.integration.spec.js' - 'services-platform:modules/llm/tests/integration/services/persistence/limits.queries.integration.spec.js' - 'services-platform:modules/llm/tests/integration/services/persistence/overrides.integration.spec.js' - >- services-platform:modules/llm/tests/integration/services/persistence/overrides.queries.integration.spec.js - 'services-platform:modules/llm/tests/integration/services/persistence/rate-versions.integration.spec.js' - >- services-platform:modules/llm/tests/integration/services/persistence/rate-versions.queries.integration.spec.js - >- services-platform:modules/llm/tests/integration/services/persistence/routing-versions.integration.spec.js - >- services-platform:modules/llm/tests/integration/services/persistence/routing-versions.queries.integration.spec.js - >- services-platform:modules/llm/tests/integration/services/persistence/usage-rollups.queries.integration.spec.js - >- services-platform:modules/llm/tests/integration/services/reconciliation/llm-execution-sweep.integration.spec.js - >- services-platform:modules/llm/tests/integration/services/reconciliation/llm-settlement-sweep.integration.spec.js - >- services-platform:modules/llm/tests/integration/services/reconciliation/rollup-settled-invocations.integration.spec.js - >- services-platform:modules/llm/tests/integration/services/usage-reports/execution-leaders.integration.spec.js - >- services-platform:modules/llm/tests/integration/services/usage-reports/platform-usage.integration.spec.js - 'services-platform:modules/llm/tests/integration/services/usage-reports/tenant-usage.integration.spec.js' - 'services-platform:modules/llm/tests/integration/services/usage/organization-usage.integration.spec.js' - 'services-platform:modules/llm/tests/unit/aws/api/llm-usage-api-mapper.spec.ts' - 'services-platform:modules/llm/tests/unit/aws/api/llm-usage-limits-api.spec.ts' - 'services-platform:modules/llm/tests/unit/aws/api/llm-usage-permissions.spec.ts' - 'services-platform:modules/llm/tests/unit/catalog/classify-usage-dimensions.spec.ts' - 'services-platform:modules/llm/tests/unit/catalog/digest-declarations.spec.ts' - 'services-platform:modules/llm/tests/unit/catalog/llm-executions.spec.ts' - 'services-platform:modules/llm/tests/unit/catalog/pricing.spec.ts' - 'services-platform:modules/llm/tests/unit/catalog/rate-conversions.spec.ts' - 'services-platform:modules/llm/tests/unit/catalog/validate-catalog-declarations.spec.ts' - 'services-platform:modules/llm/tests/unit/db/invocation-transitions.spec.ts' - 'services-platform:modules/llm/tests/unit/llm-settings.spec.ts' - 'services-platform:modules/llm/tests/unit/runtime/anthropic-credentials.spec.ts' - 'services-platform:modules/llm/tests/unit/runtime/anthropic-provider.spec.ts' - 'services-platform:modules/llm/tests/unit/runtime/execute-with-llm-retry.spec.ts' - 'services-platform:modules/llm/tests/unit/runtime/llm-mode.spec.ts' - 'services-platform:modules/llm/tests/unit/runtime/retry-policy.spec.ts' - 'services-platform:modules/llm/tests/unit/runtime/run-tool-loop.spec.ts' - 'services-platform:modules/llm/tests/unit/runtime/stub-provider.spec.ts' - 'services-platform:modules/llm/tests/unit/services/adjustments/adjustment-request.spec.ts' - 'services-platform:modules/llm/tests/unit/services/allowance-overrides.spec.ts' - 'services-platform:modules/llm/tests/unit/services/evidence-redaction.spec.ts' - 'services-platform:modules/llm/tests/unit/services/evidence/evidence-access.spec.ts' - 'services-platform:modules/llm/tests/unit/services/explain/explain.spec.ts' - 'services-platform:modules/llm/tests/unit/services/llm-metering-errors.spec.ts' - 'services-platform:modules/llm/tests/unit/services/llm-metrics.spec.ts' - 'services-platform:modules/llm/tests/unit/services/llm-operation-disabled.spec.ts' - 'services-platform:modules/llm/tests/unit/services/refusal-snapshot.spec.ts' - 'services-platform:modules/llm/tests/unit/services/tenant-limits.spec.ts' - 'services-platform:modules/llm/tests/unit/services/threshold-notice.spec.ts' - 'services-platform:modules/llm/tests/unit/services/usage-reports/usage-status.spec.ts' - 'services-platform:modules/llm/tests/unit/services/usage-reports/usage-windows.spec.ts' - 'services-platform:modules/llm/tests/unit/services/usage/organization-usage.spec.ts' - 'services-platform:modules/llm/tests/unit/services/usage/usage-features.spec.ts' - 'services-platform:modules/llm/tests/unit/services/usage/usage-sentences.spec.ts' - 'services-platform:tests/unit/llm-provider-boundary.spec.ts' - 'services-platform:tests/unit/llm-stub-mode-config.spec.ts' unregistered_acceptance_suites: No executable LLM acceptance suite is identified in the pinned module records or registry scope. excluded_synthesis_suite: >- The module guide cites the repository-wide synthesis suite outside the allowed registry read paths; infrastructure proof remains a coverage gap. corrections: - ATS-PFM-LLM-013-09 overstates a single error-mapping test validation: install: passed corpus_check: passed; zero errors checker_tests: passed; seven tests source_suites: not-run verdict: >- The module now has a traceable test hierarchy linking its capability specifications, cumulative feature suites and existing executable files. The register preserves proposed experiments and explicit lifecycle, interruption and customer-acceptance gaps without advancing product approval. Documentation checks validate the corpus only; the source suites remain unexecuted and their conformance remains unverified. merge: reason: >- The default branch contains the report and feature-scenario checker fixes required for the requested test hierarchy. retained_control_findings: FND-PFM-LLM-051 through FND-PFM-LLM-059 new_testing_findings: FND-PFM-LLM-060 through FND-PFM-LLM-081 product_changes: >- The merged control applicability is retained; this job authors only testing artifacts, their register, finding records and the module registry marker. verification: reciprocal_suite_bindings: passed pinned_test_title_resolution: passed feature_capability_scenario_references: passed new_category_and_identifier_reservations: passed source_repositories_unchanged: true --- # Platform LLM Testing Back-Fill ## Counts and Boundaries The structured counts distinguish newly authored experiments, retained scenarios, executable bindings and remaining gaps. Capability categories describe specification coverage, while suite results describe execution evidence; these are separate measures. ## Verdict The module now has a traceable test hierarchy linking its capability specifications, cumulative feature suites and existing executable files. The register preserves proposed experiments and explicit lifecycle, interruption and customer-acceptance gaps without advancing product approval. Documentation checks validate the corpus only; the source suites remain unexecuted and their conformance remains unverified.