release: spec-kit dev to main

.github/workflows/ci.yml

@@ -3,12 +3,12 @@ name: CI
 # A draft pull request runs no jobs. `ready_for_review` is in the types so that
 # marking it ready starts the run: without it the skipped jobs would stand as
 # passing required checks on the commit and nothing would have tested it. A push
-# to main still runs, as the only test of the merged tree.
+# to dev or main still runs, as the only test of the merged tree.
 on:
   push:
-    branches: [main]
+    branches: [dev, main]
   pull_request:
-    branches: [main]
+    branches: [dev, main]
     types: [opened, synchronize, reopened, ready_for_review]
 
 jobs:
@@ -29,6 +29,8 @@ jobs:
             module: dfo_provenance_docs
           - package: python/descriptions
             module: dfo_descriptions
+          - package: python/client-contract
+            module: dfo_client_contract
     defaults:
       run:
         working-directory: ${{ matrix.package }}
@@ -74,6 +76,7 @@ jobs:
             python/dfo-spec-kit-components,
             python/provenance-docs,
             python/descriptions,
+            python/client-contract,
           ]
         python-version: ["3.12", "3.13"]
     defaults:
@@ -141,3 +144,6 @@ jobs:
 
       - name: Validate step and description vectors
         run: uv run tools/validate_descriptions_vectors.py
+
+      - name: Validate the client contract
+        run: uv run tools/validate_client_contract.py

README.md

@@ -1,6 +1,6 @@
 # dfo-spec-kit
 
-The executable form of the platform's shared contracts. It exists so the consumers of these contracts, the Search API in `dfo-apis`, the harness and Analysis MCP in `dfo-ai`, the website, and the docs, share one source of truth instead of hand-maintained mirrors. This repo is not the spec registry nor the op-spec schema.
+The executable form of the platform's shared contracts. It exists so the consumers of these contracts, the Search API in `apis`, the harness and Analysis MCP in `ai`, the website, and the docs, share one source of truth instead of hand-maintained mirrors. This repo is not the spec registry nor the op-spec schema.
 
 The target design (repository shape, artifact families, versioning, governance, and delivery sequence) lives in the kit design, maintained outside this repo. This README only orients. That document is authoritative.
 
@@ -14,36 +14,37 @@ The **component-spec family** (the presentation catalogue, composition schema, o
 
 ## Status
 
-Five Python distributions exist so far. `python/spec-kit/` (`dfo-spec-kit`, import `dfo_spec_kit`): `identity/` implements the spec1 identity scheme (role-slot projection, JCS canonicalisation, SHA-256 `spec_id` minting), conformance-tested against the vector suite. `units/` reads a unit ref into a typed value (`Money`, `UnpricedMoney`, `Percent`, `Count` or `Other`) from `registries/units.json`, conformance-tested against `vectors/units/`. Money whose ref leaves prices unstated cannot be compared until `complete_prices` applies the StatVar's measurement qualifier, and constant-price money with no base year cannot be compared at all. `models/` has no code yet; model extraction comes next. `python/frame-grammar/` (`dfo-spec-kit-frame-grammar`, import `dfo_frame_grammar`): the frame grammar's executable form -- canonical frame bytes and `content_version` minting for a materialised observation table -- conformance-tested against its own vector suite, and depending on `dfo-spec-kit` for the identity canonicalisation primitives it reuses. `python/dfo-spec-kit-components/` (`dfo-spec-kit-components`, import `dfo_spec_kit_components`): the presentation contract bundle as package data, holding the envelope schema, the per-member component schemas, the write-side intent grammar, and their conformance fixtures. Generated Pydantic models for the catalogue land in this same package once the catalogue freezes. `python/provenance-docs/` (`dfo-spec-kit-provenance-docs`, import `dfo_provenance_docs`): the provenance documentation grammar, which decodes the escaped Markdown documents a Provenance node carries and parses the transformations, changelog, methodology and notes templates, conformance-tested against its own vector suite. `python/descriptions/` (`dfo-spec-kit-descriptions`, import `dfo_descriptions`): the step records and descriptions contract, holding the step schema, the operation list, the description schema and a check of the structure rules a step list must meet beyond its schema, conformance-tested against its own vector suite.
+Six Python distributions exist so far. `python/spec-kit/` (`dfo-spec-kit`, import `dfo_spec_kit`): `identity/` implements the spec1 identity scheme (role-slot projection, JCS canonicalisation, SHA-256 `spec_id` minting), conformance-tested against the vector suite. `units/` reads a unit ref into a typed value (`Money`, `UnpricedMoney`, `Percent`, `Count` or `Other`) from `registries/units.json`, conformance-tested against `vectors/units/`. Money whose ref leaves prices unstated cannot be compared until `complete_prices` applies the StatVar's measurement qualifier, and constant-price money with no base year cannot be compared at all. `models/` has no code yet; model extraction comes next. `python/frame-grammar/` (`dfo-spec-kit-frame-grammar`, import `dfo_frame_grammar`): the frame grammar's executable form -- canonical frame bytes and `content_version` minting for a materialised observation table -- conformance-tested against its own vector suite, and depending on `dfo-spec-kit` for the identity canonicalisation primitives it reuses. `python/dfo-spec-kit-components/` (`dfo-spec-kit-components`, import `dfo_spec_kit_components`): the presentation contract bundle as package data, holding the envelope schema, the per-member component schemas, the write-side intent grammar, and their conformance fixtures. Generated Pydantic models for the catalogue land in this same package once the catalogue freezes. `python/provenance-docs/` (`dfo-spec-kit-provenance-docs`, import `dfo_provenance_docs`): the provenance documentation grammar, which decodes the escaped Markdown documents a Provenance node carries and parses the transformations, changelog, methodology and notes templates, conformance-tested against its own vector suite. `python/descriptions/` (`dfo-spec-kit-descriptions`, import `dfo_descriptions`): the step records and descriptions contract, holding the step schema, the operation list, the description schema and a check of the structure rules a step list must meet beyond its schema, conformance-tested against its own vector suite. `python/client-contract/` (`dfo-spec-kit-client-contract`, import `dfo_client_contract`): the contract the DFO Python and TypeScript clients are built to, as package data: the served OpenAPI document of the composed dfo-apis app, the response members the clients act on, the caller's next step for each refusal code, the code of each error a client raises itself, the scenario and `DatasetRef` formats with their records, and the client rules with their vectors. dfo-apis CI checks its app against the same files.
 
-The Python distributions live at `python/spec-kit/`, `python/frame-grammar/`, `python/dfo-spec-kit-components/`, `python/provenance-docs/`, and `python/descriptions/`, each its own installable unit with its own version and dependency surface. A TypeScript package (`ts/`) opens only when the component catalogue freeze lands.
+The Python distributions live at `python/spec-kit/`, `python/frame-grammar/`, `python/dfo-spec-kit-components/`, `python/provenance-docs/`, `python/descriptions/`, and `python/client-contract/`, each its own installable unit with its own version and dependency surface. A TypeScript package (`ts/`) opens only when the component catalogue freeze lands.
 
-`dfo-spec-kit-frame-grammar` depends on `dfo-spec-kit` as an intra-repo path source rather than a git dependency, so the two packages develop together in this repo: a change to identity canonicalisation and a change here that depends on it land in the same commit. `dfo-spec-kit-components` depends on `dfo-spec-kit-descriptions` as an intra-repo path source, because the envelope's `description` references `description.schema.json` by its `$id`. A consumer pins both distributions at one revision. Its catalogue bundle is static data plus one accessor function, `bundle_dir()`. `dfo-spec-kit-provenance-docs` also stands alone, depending on `markdown-it-py` only. `dfo-spec-kit-descriptions` depends on `rfc8785` only.
+`dfo-spec-kit-frame-grammar` depends on `dfo-spec-kit` as an intra-repo path source rather than a git dependency, so the two packages develop together in this repo: a change to identity canonicalisation and a change here that depends on it land in the same commit. `dfo-spec-kit-components` depends on `dfo-spec-kit-descriptions` as an intra-repo path source, because the envelope's `description` references `description.schema.json` by its `$id`. A consumer pins both distributions at one revision. Its catalogue bundle is static data plus one accessor function, `bundle_dir()`. `dfo-spec-kit-provenance-docs` also stands alone, depending on `markdown-it-py` only. `dfo-spec-kit-descriptions` depends on `rfc8785` only. `dfo-spec-kit-client-contract` depends on the standard library only.
 
 Publishing is undecided: no registry workflow exists for any package. Until that's resolved, install a Python package as a git dependency pinned to one commit, e.g.
 
 ```bash
-uv add "git+https://github.com/ONEcampaign/dfo-spec-kit@<rev>#subdirectory=python/spec-kit"
-uv add "git+https://github.com/ONEcampaign/dfo-spec-kit@<rev>#subdirectory=python/frame-grammar"
-uv add "git+https://github.com/ONEcampaign/dfo-spec-kit@<rev>#subdirectory=python/dfo-spec-kit-components"
-uv add "git+https://github.com/ONEcampaign/dfo-spec-kit@<rev>#subdirectory=python/provenance-docs"
-uv add "git+https://github.com/ONEcampaign/dfo-spec-kit@<rev>#subdirectory=python/descriptions"
+uv add "git+https://github.com/DevelopmentFinanceObservatory/spec-kit@<rev>#subdirectory=python/spec-kit"
+uv add "git+https://github.com/DevelopmentFinanceObservatory/spec-kit@<rev>#subdirectory=python/frame-grammar"
+uv add "git+https://github.com/DevelopmentFinanceObservatory/spec-kit@<rev>#subdirectory=python/dfo-spec-kit-components"
+uv add "git+https://github.com/DevelopmentFinanceObservatory/spec-kit@<rev>#subdirectory=python/provenance-docs"
+uv add "git+https://github.com/DevelopmentFinanceObservatory/spec-kit@<rev>#subdirectory=python/descriptions"
+uv add "git+https://github.com/DevelopmentFinanceObservatory/spec-kit@<rev>#subdirectory=python/client-contract"
 ```
 
 A package's metadata names its intra-repo dependency by distribution name alone, and no index serves these distributions. uv reads the path source in `[tool.uv.sources]` and resolves the dependency from the same checkout, so `uv add` of `frame-grammar` or `dfo-spec-kit-components` alone installs `dfo-spec-kit` or `dfo-spec-kit-descriptions` at that commit. Every other installer reads the bare name and fails, so name both distributions at the same commit:
 
 ```bash
 pip install \
-  "dfo-spec-kit-components @ git+https://github.com/ONEcampaign/dfo-spec-kit@<rev>#subdirectory=python/dfo-spec-kit-components" \
-  "dfo-spec-kit-descriptions @ git+https://github.com/ONEcampaign/dfo-spec-kit@<rev>#subdirectory=python/descriptions"
+  "dfo-spec-kit-components @ git+https://github.com/DevelopmentFinanceObservatory/spec-kit@<rev>#subdirectory=python/dfo-spec-kit-components" \
+  "dfo-spec-kit-descriptions @ git+https://github.com/DevelopmentFinanceObservatory/spec-kit@<rev>#subdirectory=python/descriptions"
 ```
 
-A uv project pins each kit distribution it imports in its own `[tool.uv.sources]`, every entry at the same `rev`, as `dfo-apis` pins `dfo-spec-kit-components` and `dfo-spec-kit-frame-grammar`:
+A uv project pins each kit distribution it imports in its own `[tool.uv.sources]`, every entry at the same `rev`, as `apis` pins `dfo-spec-kit-components` and `dfo-spec-kit-frame-grammar`:
 
 ```toml
 [tool.uv.sources]
-dfo-spec-kit-components = { git = "https://github.com/ONEcampaign/dfo-spec-kit.git", rev = "<rev>", subdirectory = "python/dfo-spec-kit-components" }
-dfo-spec-kit-descriptions = { git = "https://github.com/ONEcampaign/dfo-spec-kit.git", rev = "<rev>", subdirectory = "python/descriptions" }
+dfo-spec-kit-components = { git = "https://github.com/DevelopmentFinanceObservatory/spec-kit.git", rev = "<rev>", subdirectory = "python/dfo-spec-kit-components" }
+dfo-spec-kit-descriptions = { git = "https://github.com/DevelopmentFinanceObservatory/spec-kit.git", rev = "<rev>", subdirectory = "python/descriptions" }
 ```
 
 ## Layout
@@ -68,6 +69,8 @@ dfo-spec-kit/
 │                                      # the provenance documentation grammar and its vector suite as package data
 ├── python/descriptions/              # the dfo-spec-kit-descriptions package (import dfo_descriptions),
 │                                      # the step and description schemas, the operation list and their vector suite as package data
+├── python/client-contract/           # the dfo-spec-kit-client-contract package (import dfo_client_contract),
+│                                      # the served OpenAPI document, strict fields, caller actions, scenarios, ref format and client rules as package data
 ├── policies/                         # compatibility policy per artifact family
 └── tools/                            # codegen + drift-check scripts, including the registry and vector validators
 ```

python/client-contract/.pre-commit-config.yaml

@@ -0,0 +1,121 @@
+# Pre-commit hooks for Python + data repos
+# Install dev deps:  uv sync --group dev
+# Enable hooks:      pre-commit install
+# Run all files:     pre-commit run --all-files
+# Update hooks:      pre-commit autoupdate
+# Skip once:         git commit --no-verify
+
+repos:
+  # ---------------------------------------------------------------------------
+  # Ruff: fast linter + formatter
+  # ---------------------------------------------------------------------------
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: v0.15.21
+    hooks:
+      - id: ruff-check
+        args: [--fix]
+        files: ^python/client-contract/
+      - id: ruff-format
+        files: ^python/client-contract/
+
+  # ---------------------------------------------------------------------------
+  # Core hygiene, JSON/YAML/TOML, security
+  # ---------------------------------------------------------------------------
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v6.0.0
+    hooks:
+      # Whitespace & endings
+      - id: trailing-whitespace
+        args: [--markdown-linebreak-ext=md]
+      - id: end-of-file-fixer
+      - id: mixed-line-ending
+        args: [--fix=lf]
+
+      # Conflicts & platform quirks
+      - id: check-case-conflict
+      - id: check-merge-conflict
+
+      # Python sanity
+      - id: debug-statements
+      - id: check-ast
+      - id: check-shebang-scripts-are-executable
+      - id: check-builtin-literals
+      - id: check-docstring-first
+
+      # Config/data formats
+      - id: check-json
+      - id: pretty-format-json
+        args: [--autofix, --indent=2, --no-sort-keys, --no-ensure-ascii]
+        exclude: |
+          (?x)^(
+            \.idea/.*|
+            \.vscode/.*
+          )$
+      - id: check-yaml
+        args: [--allow-multiple-documents]
+      - id: check-toml
+
+      # Basic secret hygiene
+      - id: detect-private-key
+
+      # Large files (1MB) with sensible excludes
+      - id: check-added-large-files
+        args: [--maxkb=5000]
+        exclude: |
+          (?x)^(
+            .*\.parquet|
+            .*\.feather|
+            uv\.lock
+          )$
+
+  # ---------------------------------------------------------------------------
+  # Type checking (uses settings from pyproject.toml)
+  # ---------------------------------------------------------------------------
+  - repo: local
+    hooks:
+      - id: ty
+        name: ty type check
+        language: system
+        entry: bash -c 'cd python/client-contract && uv run ty check'
+        types: [python]
+        files: ^python/client-contract/
+        pass_filenames: false
+
+  # ---------------------------------------------------------------------------
+  # Spelling (docs + code comments)
+  # ---------------------------------------------------------------------------
+  - repo: https://github.com/codespell-project/codespell
+    rev: v2.4.3
+    hooks:
+      - id: codespell
+        args: [--write-changes]
+        # The contract files are data: served bodies, keys and codes read as
+        # misspellings, and --write-changes would rewrite them.
+        exclude: |
+          (?x)^(
+            .*\.lock|
+            .*\.parquet|
+            .*\.feather|
+            .*/dfo_client_contract/.*\.json
+          )$
+
+  # ---------------------------------------------------------------------------
+  # GitHub Actions linting
+  # ---------------------------------------------------------------------------
+  - repo: https://github.com/rhysd/actionlint
+    rev: v1.7.12
+    hooks:
+      - id: actionlint
+
+  # ---------------------------------------------------------------------------
+  # Markdown formatting (GFM + front matter)
+  # ---------------------------------------------------------------------------
+  - repo: https://github.com/hukkin/mdformat
+    rev: 1.0.0
+    hooks:
+      - id: mdformat
+        additional_dependencies:
+          - mdformat-gfm>=1.0.0
+          - mdformat-frontmatter>=2.1.0
+        # Example: limit to docs if desired
+        # files: ^(README\.md|docs/|.*/README\.md)$