---
id: CAP-PFM-LLM-010
title: Monitor AI usage across organizations
feature: FEAT-PFM-LLM-04
product_area: platform
module: llm
revision: 1
status: draft
product_intent: proposed
implementation_evidence: observed
supersedes: []
source_sha:
  services-platform: ad3f6a6582843008ed0f5a17b131878d15829af1
  react-console: baf6dae382d6f41dce976c7f848667ccc37e076f
intent_sources:
  - 'services-platform:docs/proposals/llm-observability/11-decisions.md#D-7'
  - 'services-platform:docs/milestones/plans/PLAT-M1-plan.md#4. Assumptions'
purpose: 'Help Rhythm staff compare cost, activity and allowance pressure across customers.'
outcomes:
  - Usage reports compare activity and charges across reporting windows.
  - Organization status highlights capacity concerns.
  - Daily aggregates retain organization and usage dimensions.
concepts:
  - CON-PFM-LLM-004
  - CON-PFM-LLM-003
actors:
  - ACT-PFM-033
  - ACT-PFM-004
  - ACT-PFM-012
surfaces:
  - console
  - api
  - job
preconditions: []
postconditions: []
permissions:
  - key: hq.llm-usage.read
    where_checked: api
    scope: global
  - key: hq.llm-usage.read
    where_checked: console
    scope: global
rules:
  - id: BR-CAP-PFM-LLM-010-01
    proved_by: [ATS-PFM-LLM-010-01, ATS-PFM-LLM-010-09, TST-PFM-LLM-009, TST-PFM-LLM-054]
    statement: >-
      Only staff with the HQ AI usage read key can read system-wide usage; customer-scoped authority and
      unrelated HQ keys cannot authorize it.
    kind: permission
    enforced_at:
      - api
    must_prove: true
    provenance: test
    source: >-
      services-platform:modules/llm/tests/e2e/llm-usage.e2e.spec.js#refuses a customer’s administrator,
      whatever they hold at their customer’s scope
    origin: none
  - id: BR-CAP-PFM-LLM-010-02
    proved_by: [ATS-PFM-LLM-010-02, TST-PFM-LLM-009, TST-PFM-LLM-049, TST-PFM-LLM-082]
    statement: >-
      Reporting accepts month to date or thirty or ninety UTC days including today and compares with the
      immediately preceding equal-length period; unknown windows are rejected.
    kind: calculation
    enforced_at:
      - api
      - service
    must_prove: true
    provenance: observed
    source: 'services-platform:modules/llm/services/usage-reports/platform-usage.ts'
    origin: none
  - id: BR-CAP-PFM-LLM-010-03
    proved_by: [ATS-PFM-LLM-007-13, ATS-PFM-LLM-010-03, TST-PFM-LLM-081]
    statement: >-
      Live status is disabled for an inactive organization or disabled AI, otherwise stopped when committed
      usage reaches allowance, warned from 80%, and within below that; later capacity can improve status.
    kind: calculation
    enforced_at:
      - service
    must_prove: true
    provenance: observed
    source: 'services-platform:modules/llm/services/usage-reports/usage-status.ts'
    origin: none
  - id: BR-CAP-PFM-LLM-010-04
    proved_by: [ATS-PFM-LLM-010-04, TST-PFM-LLM-081]
    statement: >-
      Utilisation divides charged plus held credits by base plus active capacity, rounds to one decimal, may
      exceed 100% and has no percentage for a zero allowance.
    kind: calculation
    enforced_at:
      - service
    must_prove: true
    provenance: observed
    source: 'services-platform:modules/llm/services/usage-reports/usage-status.ts'
    origin: none
  - id: BR-CAP-PFM-LLM-010-05
    proved_by: [ATS-PFM-LLM-010-05, TST-PFM-LLM-049]
    statement: >-
      Trends can group by organization, application, model, provider or operation, retain ten largest series
      and aggregate the remainder exactly as Other, omitting empty dates.
    kind: calculation
    enforced_at:
      - service
    must_prove: true
    provenance: observed
    source: 'services-platform:modules/llm/services/usage-reports/platform-usage.ts'
    origin: none
  - id: BR-CAP-PFM-LLM-010-06
    proved_by: []
    statement: >-
      Organization rows combine spend, requests, refusals, failures, utilisation, allowance, capacity and
      status, with cost, utilisation or refusal sorting and stable pagination.
    kind: policy
    enforced_at:
      - api
      - service
    must_prove: false
    provenance: observed
    source: 'services-platform:modules/llm/services/usage-reports/platform-usage.ts'
    origin: none
  - id: BR-CAP-PFM-LLM-010-07
    proved_by: [ATS-PFM-LLM-010-12, TST-PFM-LLM-048]
    statement: >-
      Breakdowns provide group cost, activity, average latency and failure rate; cross-organization execution
      leaders identify expensive closed tasks and organizations whose reads failed.
    kind: calculation
    enforced_at:
      - service
    must_prove: false
    provenance: observed
    source: 'services-platform:modules/llm/services/usage-reports/platform-usage.ts'
    origin: none
  - id: BR-CAP-PFM-LLM-010-08
    proved_by: [ATS-PFM-LLM-005-11, ATS-PFM-LLM-009-06, ATS-PFM-LLM-010-06, ATS-PFM-LLM-010-10, TST-PFM-LLM-017, TST-PFM-LLM-047]
    statement: >-
      Hourly rebuilding replaces each affected organization’s UTC-day totals, assigns requests to their start
      day, excludes running work, and retains refused, failed, released and settled classifications with their
      applicable charges.
    kind: calculation
    enforced_at:
      - service
      - database
    must_prove: true
    provenance: observed
    source: 'services-platform:modules/llm/services/reconciliation/rollup-settled-invocations.ts'
    origin: none
  - id: BR-CAP-PFM-LLM-010-09
    proved_by: [ATS-PFM-LLM-010-07, ATS-PFM-LLM-010-11, TST-PFM-LLM-044, TST-PFM-LLM-047]
    statement: >-
      Rollup progress trails the run start by five minutes and stops at the earliest unfinished change on
      error or exhausted time without moving backward; delay beyond two hours alarms.
    kind: invariant
    enforced_at:
      - service
      - database
    must_prove: true
    provenance: observed
    source: 'services-platform:modules/llm/services/reconciliation/rollup-settled-invocations.ts'
    origin: none
  - id: BR-CAP-PFM-LLM-010-10
    proved_by: [ATS-PFM-LLM-010-08, TST-PFM-LLM-003]
    statement: HQ presents list-price dollars with the equivalent credits alongside.
    kind: policy
    enforced_at:
      - console
    must_prove: true
    provenance: test
    source: 'react-console:lib/hq/ai-usage-format.test.ts#shows cents as dollars beside the credits'
    origin: 'services-platform:docs/proposals/llm-observability/11-decisions.md#D-7'
  - id: BR-CAP-PFM-LLM-010-11
    proved_by: []
    statement: An unavailable report section shows its failure state independently of successfully read sections.
    kind: policy
    enforced_at:
      - service
      - console
    must_prove: false
    provenance: observed
    source: 'react-console:components/console/hq/ai-usage/ai-usage-tiles.tsx'
    origin: none
  - id: BR-CAP-PFM-LLM-010-12
    proved_by: []
    statement: >-
      HQ query recipes cover organizational, daily and model spend and organizational refusals; the HQ
      reporting reader can query daily rollups without gaining access to organization request records.
    kind: permission
    enforced_at:
      - database
    must_prove: false
    provenance: documented
    source: 'services-platform:docs/llm.md#Reporting'
    origin: none
scenarios:
  - id: SC-CAP-PFM-LLM-010-01
    given: Two organizations have requests across a UTC day boundary.
    when: The aggregate rebuild runs.
    then: Each request contributes to its own organization and start day.
    provenance: test
    source: >-
      services-platform:modules/llm/tests/integration/services/reconciliation/rollup-settled-invocations.integration.spec.js#keeps
      the other organization's calls in its own cell, with the four token sums as reported
  - id: SC-CAP-PFM-LLM-010-02
    given: A support reader opens the HQ usage overview.
    when: Its mocked reads complete.
    then: 'The page renders metrics, trends, organization states and breakdowns.'
    provenance: test
    source: >-
      react-console:e2e/mock/hq-ai-usage.mock.spec.ts#shows the tiles, the trend, who needs attention and the
      organizations with their states
edge_cases: []
constraints:
  - Reporting uses UTC days while allowance periods follow the organization’s calendar.
depends_on:
  - CAP-PFM-LLM-007
  - CAP-PFM-LLM-009
related:
  - CAP-PFM-LLM-005
  - CAP-PFM-LLM-011
journeys:
  - JNY-PFM-LLM-002
open_questions: []
evidence:
  - >-
    services-platform at ad3f6a6582843008ed0f5a17b131878d15829af1: docs/llm.md (The HQ surface, Platform-wide;
    The daily rollup; Reporting); modules/llm/README.md; aws/api/llm-usage.routes.ts (getHqLlmUsageSummary,
    getHqLlmUsageTrend, listHqLlmUsageTenants, getHqLlmUsageBreakdown, listHqLlmUsageExecutions);
    services/usage-reports/platform-usage.ts, usage-status.ts, usage-windows.ts, execution-leaders.ts,
    tenant-positions.ts, section.ts; services/reconciliation/rollup-settled-invocations.ts;
    db/llm-usage-daily.table.ts, llm-usage-daily-query.view.ts, llm-rollup-watermarks.table.ts; tests
    e2e/llm-usage.e2e.spec.js, integration/services/usage-reports/*,
    integration/services/reconciliation/rollup-settled-invocations.integration.spec.js,
    integration/db/llm-usage-daily.table.integration.spec.js, unit/services/usage-reports/*.
  - >-
    react-console at baf6dae382d6f41dce976c7f848667ccc37e076f: app/(console)/hq/aiUsage/page.tsx;
    components/console/hq/ai-usage/ai-usage-dashboard.tsx, ai-usage-tiles.tsx, ai-usage-trend-card.tsx,
    breakdowns-card.tsx, organizations-card.tsx, needs-attention-card.tsx, top-operations-card.tsx,
    catalog-card.tsx; lib/hq/ai-usage.ts, ai-usage-format.ts; e2e/mock/hq-ai-usage.mock.spec.ts,
    e2e/dev/hq-ai-usage.dev.spec.ts.
controls: [TC-AVL-001, TC-AZ-001, TC-AZ-002, TC-AZ-003, TC-AZ-004, TC-AZ-005, TC-IAM-003, TC-LOG-001, TC-LOG-004, TC-LOG-005, TC-MON-001, TC-MON-002, TC-PI-001, TC-PI-003, TC-PI-005, TC-TEN-001, TC-TEN-002, TC-TEN-003, TC-VEN-001, TC-VEN-002, TC-VEN-003]
exercised_by: [FAS-PFM-LLM-04-01, FAS-PFM-LLM-04-02, FAS-PFM-LLM-04-03, FAS-PFM-LLM-04-04, FAS-PFM-LLM-04-05]
suites: [TST-PFM-LLM-003, TST-PFM-LLM-009, TST-PFM-LLM-017, TST-PFM-LLM-044, TST-PFM-LLM-047, TST-PFM-LLM-048, TST-PFM-LLM-049, TST-PFM-LLM-054, TST-PFM-LLM-081, TST-PFM-LLM-082]
---

# Monitor AI usage across organizations

## Purpose

Help Rhythm staff compare cost, activity and allowance pressure across customers.

## How It Works

Authorized staff open the system-wide AI Usage view and choose a reporting window. Settled daily figures support its totals, trends and breakdowns. Live allowance positions supply the organization statuses beside those historical figures. Staff can sort organizations and follow a row into a particular customer’s usage. Smaller trend series fold into an Other series without losing their totals. Costly closed executions provide another way to identify expensive work. A failed section reports its unavailable state while independent sections can continue. The hourly rebuild maintains a progress watermark, and operators receive an alarm when that work falls behind.

## Relationships

- Admit AI work against the organization’s monthly allowance supplies the prerequisite behavior for this outcome.
- Recover interrupted requests and identify accounting drift supplies the prerequisite behavior for this outcome.