Skip to content
Merged
16 changes: 16 additions & 0 deletions docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,22 @@ just check # lint, type check, and the full suite, as CI runs them

`just check` and CI run the normative identity vector tests (`tests/engine/test_identity_vectors.py`) from the `spec-kit` repository. `just check` and `just test` find a clone at `../dfo-spec-kit`, or at the path in `DFO_SPEC_KIT_DIR`. A git worktree needs `DFO_SPEC_KIT_DIR` set explicitly, because the sibling fallback resolves relative to the worktree's own directory, not the main checkout's. `just check` fails when no clone resolves.

### Changing what the clients read

`tests/contract/test_client_contract.py` checks the app against the client contract spec-kit
ships as `dfo-spec-kit-client-contract`. Each recorded scenario's requests and responses must
validate against the app's OpenAPI document, every required member `strict-fields.json` lists
must stay required and non-nullable, every enum value it lists must stay admitted, and every
problem code a response carries must stay registered on its route at its status. When the test
fails for a wire change you intend, commit the change, record the contract against your branch's
local app with spec-kit's `tools/capture_client_contract.py`
(`openapi --dfo-apis <checkout>`, then `record --base-url http://127.0.0.1:8083/api`), open the
spec-kit PR, and cut a spec-kit release. Move all six spec-kit pins to that release's commit in
the same dfo-apis PR as the change: `dfo-spec-kit` and `dfo-spec-kit-client-contract` in the root
`pyproject.toml`, and the four in `packages/dfo-data-tools/pyproject.toml`. A change that keeps
the shape and alters behaviour passes this test, and the sdk's scheduled end-to-end run catches
it after merge.

### SDMX live tests

`tests/dc_sdmx/` runs offline against recorded wire fixtures. `test_live.py` calls a real Data Commons deployment and is skipped unless `DFO_SDMX_LIVE_BASE` is set.
Expand Down
8 changes: 4 additions & 4 deletions packages/dfo-data-tools/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -86,10 +86,10 @@ graph-release = ["google-cloud-spanner>=3.57", "dfo-data-tools[coverage]"]
[tool.uv.sources]
# See the root pyproject.toml's dfo-spec-kit source comment for the
# pin-consistency invariant this rev must satisfy.
dfo-spec-kit-frame-grammar = { git = "https://github.com/DevelopmentFinanceObservatory/spec-kit.git", rev = "95a15cf0af7f4d34c113a1b811712a93e1baa2cb", subdirectory = "python/frame-grammar" }
dfo-spec-kit-components = { git = "https://github.com/DevelopmentFinanceObservatory/spec-kit.git", rev = "95a15cf0af7f4d34c113a1b811712a93e1baa2cb", subdirectory = "python/dfo-spec-kit-components" }
dfo-spec-kit-provenance-docs = { git = "https://github.com/DevelopmentFinanceObservatory/spec-kit.git", rev = "95a15cf0af7f4d34c113a1b811712a93e1baa2cb", subdirectory = "python/provenance-docs" }
dfo-spec-kit-descriptions = { git = "https://github.com/DevelopmentFinanceObservatory/spec-kit.git", rev = "95a15cf0af7f4d34c113a1b811712a93e1baa2cb", subdirectory = "python/descriptions" }
dfo-spec-kit-frame-grammar = { git = "https://github.com/DevelopmentFinanceObservatory/spec-kit.git", rev = "4ef22b70d0f11af87ed9a78afda841ab66f6230c", subdirectory = "python/frame-grammar" }
dfo-spec-kit-components = { git = "https://github.com/DevelopmentFinanceObservatory/spec-kit.git", rev = "4ef22b70d0f11af87ed9a78afda841ab66f6230c", subdirectory = "python/dfo-spec-kit-components" }
dfo-spec-kit-provenance-docs = { git = "https://github.com/DevelopmentFinanceObservatory/spec-kit.git", rev = "4ef22b70d0f11af87ed9a78afda841ab66f6230c", subdirectory = "python/provenance-docs" }
dfo-spec-kit-descriptions = { git = "https://github.com/DevelopmentFinanceObservatory/spec-kit.git", rev = "4ef22b70d0f11af87ed9a78afda841ab66f6230c", subdirectory = "python/descriptions" }

[build-system]
requires = ["uv_build>=0.11,<0.12"]
Expand Down
24 changes: 15 additions & 9 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,8 @@ test = [
# A bare `just test` selects the tests affected by a change; CI never
# passes --testmon, so it stays inactive there.
"pytest-testmon>=2.2.0",
# tests/contract/test_client_contract.py reads the recorded client scenarios.
"dfo-spec-kit-client-contract",
]

[build-system]
Expand Down Expand Up @@ -163,7 +165,10 @@ dfo-data-tools = { workspace = true }
# uv resolves relative to frame-grammar's own pinned commit -- so this
# project's own `dfo-spec-kit` source MUST resolve to that identical commit,
# or `uv lock` refuses with "conflicting URLs for package dfo-spec-kit"
# (confirmed live). All five pins move together as a single rev. This rev's
# (confirmed live). All six pins move together as a single rev: these four,
# `dfo-spec-kit`, and `dfo-spec-kit-client-contract`. uv does not force the
# contract package onto this commit; tests/contract/test_client_contract.py
# checks that its rev equals `dfo-spec-kit`'s. This rev's
# registry schema has no `spec_resubmit` door: a code with a raise site in
# this repo maps to `dataset_resubmit` instead. `entity_set_too_large` is
# mapped on `search.resolve` for the `raw_text` and `parsed` doors, which serve
Expand Down Expand Up @@ -236,12 +241,14 @@ dfo-data-tools = { workspace = true }
# The registry refuses a part's `source_code` on search.resolve with
# `source_code_not_held`, `source_code_ambiguous`, and
# `source_code_shares_variable`.
# The pin is a commit of spec-kit's main. It registers
# slot_default_not_applied and the three source-code refusals, and retires
# starved_axis_binding_restored.
# Follow-up: move all five to a real spec-kit tag once one is cut, so a
# future force-push or GC on spec-kit's main can't silently break this lock.
dfo-spec-kit = { git = "https://github.com/DevelopmentFinanceObservatory/spec-kit.git", rev = "95a15cf0af7f4d34c113a1b811712a93e1baa2cb", subdirectory = "python/spec-kit" }
# The pin is 4ef22b7 on spec-kit's dev branch. It registers
# slot_default_not_applied, the three source-code refusals and
# whole_total_available, retires starved_axis_binding_restored, and holds the
# client contract `dfo-spec-kit-client-contract` reads.
# Follow-up: move all six to a real spec-kit tag once one is cut, so a future
# force-push or GC on spec-kit's dev can't silently break this lock.
dfo-spec-kit = { git = "https://github.com/DevelopmentFinanceObservatory/spec-kit.git", rev = "4ef22b70d0f11af87ed9a78afda841ab66f6230c", subdirectory = "python/spec-kit" }
dfo-spec-kit-client-contract = { git = "https://github.com/DevelopmentFinanceObservatory/spec-kit.git", rev = "4ef22b70d0f11af87ed9a78afda841ab66f6230c", subdirectory = "python/client-contract" }

[tool.pytest.ini_options]
testpaths = ["tests"]
Expand Down Expand Up @@ -285,9 +292,8 @@ python = ".venv"
# literally in a docstring about averaging three ratios. "dependant" is
# FastAPI's own attribute name (`APIRoute.dependant`).
# heading_words_specs.json holds graph labels verbatim ("Assitance" in a UNICEF series).
# CHANEL Foundation is an OECD private donor (code 1645) named in code_index.declarations.
skip = "tests/render/heading_words_specs.json"
ignore-words-list = "caf,nam,som,tha,fro,zar,nd,retuned,unparseable,uncomplete,re-use,pre-empt,pre-empting,fo,convertor,convertors,disjointness,meaned,dependant,chanel"
ignore-words-list = "caf,nam,som,tha,fro,zar,nd,retuned,unparseable,uncomplete,re-use,pre-empt,pre-empting,fo,convertor,convertors,disjointness,meaned,dependant"

[tool.ruff]
target-version = "py312"
Expand Down
4 changes: 4 additions & 0 deletions src/search/engine/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -1235,6 +1235,10 @@ async def lifespan(app: FastAPI) -> AsyncIterator[None]:
UpstreamVersionChangedError("exemplar rate table changed version during its load"),
),
lifespan=lifespan,
registry_endpoints={
"/api/search/resolve": "search.resolve",
"/api/search/options": "search.options",
},
request_scope=lambda: _request_scope(catalogue, release, conversions, variable_breadth),
)

Expand Down
Empty file added tests/contract/__init__.py
Empty file.
Loading
Loading