--- id: DOM-PFM-QRY-001 title: Dataset realises: CON-PFM-QRY-001 product_area: platform module: queries revision: 1 status: draft design_intent: proposed implementation_evidence: observed source_sha: services-platform: ad3f6a6582843008ed0f5a17b131878d15829af1 react-console: baf6dae382d6f41dce976c7f848667ccc37e076f owned_by: SVC-PFM-QRY-queries readable_by: - SVC-PFM-QRY-explorer - SVC-PFM-QRY-quick-search lifecycle: STATE-PFM-QRY-002 custom_fields: none identity: >- A dataset has a stable name used by every query consumer. Application registrations are product-wide; custom-object datasets are workspace-scoped. Staff reserve permanent custom-table names. A resolved description belongs to the requesting person and workspace rather than being a separate stored object. attributes: - name: dataset id kind: text required: true computed: false notes: 'The stable dataset name used by questions, recipes, list views and search projections.' - name: owning application kind: reference required: true computed: false notes: The registering application supplies the dataset and its query access requirement. - name: scope kind: enum required: true computed: false notes: >- Workspace datasets read customer records; system datasets support cross-customer staff work. References remain within their declared scope. - name: source kind: enum required: true computed: false notes: >- The dataset comes from an application manifest, an active custom object or a published custom query table. - name: query permission kind: reference required: true computed: false notes: >- Description and execution require the declared dataset query grant, together with any declared record-read and application-access checks. - name: fields kind: collection required: true computed: true notes: >- Available fields combine registered, related, computed, configured and eligible custom fields. Metadata supplies labels, types and supported operations; hidden fields remain unavailable in the ordinary field chooser. - name: relationships and collections kind: collection required: false computed: true notes: >- Reference paths, registered joins and collection relationships define the permitted ways to reach related records. - name: scopes and discovery rule kind: collection required: false computed: false notes: >- Registered scopes select populations such as all records or recently viewed; discovery metadata identifies records omitted by ordinary selection. - name: defaults kind: collection required: false computed: false notes: >- Column and sort suggestions support authoring. Suggested criteria are not silently applied to submitted execution requests. - name: quick search projection kind: collection required: false computed: false notes: >- An optional source projection defines search display values and fields that indexing must omit. - name: retired at kind: date required: false computed: true notes: >- Registration marks omitted datasets retired while retaining their declarations for dependent work. - name: custom query table · definition kind: collection required: false computed: false notes: >- Staff maintain the custom table’s label, description, category, lifecycle status and write version. - name: custom query table · versions kind: collection required: false computed: false notes: >- A table keeps one editable draft and immutable published history, including reviewed SQL, canonical SQL, its fingerprint and output metadata. - name: custom query table · assignments kind: collection required: false computed: false notes: >- Inclusion can cover all workspaces, a customer or one workspace; exclusions and published-version pins refine that access. - name: custom query table · history kind: collection required: false computed: true notes: 'History retains the actor and action with bounded codes, excluding SQL and record contents.' - name: resolved description kind: computed required: false computed: true notes: >- Resolution combines workspace metadata and person access under stamp-based cache keys and reports located problems. relationships: - to: DOM-PFM-QRY-002 cardinality: one-to-many ownership: >- Queries retain their base and joined dataset references; datasets do not maintain a reverse query collection. cross_service: false - to: DOM-PFM-QRY-005 cardinality: one-to-zero-or-one ownership: >- Quick Search owns the optional record-type index and obtains its projection vocabulary from the dataset. cross_service: true - to: DOM-PFM-QRY-004 cardinality: one-to-many ownership: >- Query files retain their source dataset; workspace export declarations use a separate coverage registry. cross_service: false invariants: - id: INV-DOM-PFM-QRY-001-01 statement: >- Dataset description, validation and execution check surface access and the dataset grant, including any declared record-read or application requirement. enforced_at: - api - service provenance: observed source: >- services-platform:modules/queries/services/describe/resolve-query-model.ts:resolveQueryModel; modules/queries/services/describe/dataset-grants.ts - id: INV-DOM-PFM-QRY-001-02 statement: >- Compilation restricts base records to the execution workspace and relates joined or picked records within that workspace, alongside database row security. enforced_at: - service - database provenance: documented source: 'services-platform:docs/compliance/control-coverage.md#Component: queries (TC-TEN-001)' - id: INV-DOM-PFM-QRY-001-03 statement: >- The registration service checks the manifest against metadata and the database catalog before persisting its declarations. enforced_at: - service provenance: observed source: >- services-platform:modules/queries/services/registration/register-query-application.ts:registerQueryApplication - id: INV-DOM-PFM-QRY-001-04 statement: Registration retains omitted datasets with a retirement marker instead of deleting them. enforced_at: - service provenance: observed source: >- services-platform:modules/queries/services/persistence/registered-query-objects.ts:retireRegisteredQueryObjectsExcept - id: INV-DOM-PFM-QRY-001-05 statement: >- Description caches use the stamps read at resolution start. Trigger updates commit with metadata writes, but the stamp and metadata reads do not share a guaranteed database snapshot. enforced_at: - database - service provenance: observed source: >- services-platform:modules/queries/services/describe/resolve-query-model.ts:withCachedReads; modules/queries/services/persistence/query-description-stamps.ts:readQueryDescriptionStamps - id: INV-DOM-PFM-QRY-001-06 statement: >- Custom fields appear in a dataset description only when the source object registration enables that capability. enforced_at: - service provenance: documented source: 'services-platform:docs/standards/custom-field-capability.md#3.3 What Platform enforces' - id: INV-DOM-PFM-QRY-001-07 statement: A reserved custom-table name cannot identify a different definition later. enforced_at: - service - database provenance: observed source: 'services-platform:modules/queries/db/custom-query-tables.table.ts' - id: INV-DOM-PFM-QRY-001-08 statement: >- Only one draft may exist for a custom table; published versions allow supersession but reject content changes and deletion. enforced_at: - database provenance: observed source: >- services-platform:db/migrations/0065_custom_query_tables.sql:custom_query_table_versions_immutable - id: INV-DOM-PFM-QRY-001-09 statement: >- Execution verifies the published canonical SQL fingerprint and refuses a mismatch without returning the SQL. enforced_at: - service provenance: observed source: >- services-platform:modules/queries/db/custom-query-table-versions.table.ts (sql_hash); modules/queries/services/describe/resolve-query-model.ts - id: INV-DOM-PFM-QRY-001-10 statement: >- Custom-table discovery requires an applicable inclusion without a workspace exclusion; other callers receive not found. enforced_at: - service provenance: observed source: >- services-platform:modules/queries/db/custom-query-table-assignments.table.ts; modules/queries/services/custom-tables/assignment-visibility.ts - id: INV-DOM-PFM-QRY-001-11 statement: >- Custom-table changes commit their history event with the change, without SQL or record values in the event. enforced_at: - service provenance: observed source: 'services-platform:modules/queries/services/custom-tables/custom-table-library.ts' - id: INV-DOM-PFM-QRY-001-12 statement: >- Only system datasets are available through staff system-query operations; workspace datasets appear missing there. enforced_at: - api - service provenance: test source: >- services-platform:modules/queries/tests/e2e/system-queries.e2e.spec.js#refuses a tenant dataset as one that does not exist, with 404 and never 403 physical: - platform.registered_query_applications - platform.registered_query_objects - platform.registered_query_picks - platform.registered_query_collections - platform.registered_query_computed_fields - platform.registered_query_configured_relationships - platform.query_description_stamps - platform.custom_query_tables - platform.custom_query_table_versions - platform.custom_query_table_columns - platform.custom_query_table_assignments - platform.custom_query_table_events - >- platform.registered_objects, platform.custom_objects and the field and label tables (read only; owned by Metadata) - 'the query description cache (ElastiCache for Valkey, no table)' evidence: - >- services-platform at ad3f6a6: modules/queries/db/registered-query-*.table.ts, custom-query-*.table.ts, query-description-stamps.table.ts; services/describe/*.ts; services/registration/register-query-application.ts; services/custom-tables/*.ts; platform-datasets.ts; tenant-query-families.ts - >- services-platform at ad3f6a6: docs/queries/semantic-query-layer.md; docs/queries/custom-query-tables/rhythm-custom-query-tables-architecture.md; docs/standards/custom-field-capability.md; docs/compliance/control-coverage.md --- # Dataset ## What It Models A dataset is the vocabulary through which applications make records queryable. It combines the owner’s record shape with the workspace’s configuration and the current person’s permitted operations. ## Identity and Attributes Registered datasets, workspace custom objects and staff-authored custom tables share the query interface. Their declarations remain distinguishable so retirement, publication and access failures can be explained. ## Relationships Queries and derived consumers retain references to datasets. Business-record ownership remains with the application that supplies the source. ## Invariants The invariant rows locate registration, execution, custom-table history and access protections at their enforcement boundaries. ## How It Is Stored and Shared Application declarations and custom-table definitions are durable; resolved descriptions are cached projections. The query service owns resolution and SQL compilation. ## Open Design Questions The production validation-workspace limitation and the description-read consistency gap remain recorded findings.