Repository navigation
test(contract): check the app against the client contract, and bind Search's registry endpoints - #72
Merged
Merged
Conversation
tests/contract/test_client_contract.py checks the composed app against the scenarios, strict fields and actions that dfo-spec-kit-client-contract ships. Each recorded exchange must match a route and validate against its request and response schemas. Every strict member must stay required and non-nullable, and every strict enum value must stay admitted. Every action must name an active registry code, and every problem code must carry the status the server would pick for its route. The six spec-kit pins move to 98d78be together, and the test group gains dfo-spec-kit-client-contract. CONTRIBUTING.md describes how to change what the clients read.
The Search surface declared no registry_endpoints, so registered_status read a Search code's status across every endpoint, and found two for facet_binding_unsupported: 501 on Search and 422 elsewhere. /api/search/resolve binds search.resolve and /api/search/options binds search.options, as the Graph and Datasets routes bind theirs. Search's classifier sets each status itself, so no response changes. tests/engine/test_refusal_registry.py checks that every search.* endpoint the registry maps is bound to a route, and that each Search exception exemplar goes out at the status the registry gives its code on each Search endpoint that maps it.
jm-rivera
changed the title
Keep the public contribution guide and move the contract update guidance into the maintainer notes. Describe the test as schema and registry validation of recorded exchanges.
Move all six spec-kit pins to 95a15cf, where spec-kit#11 landed on dev, so the gate checks the 118 scenarios the SDK clients replay.
# Conflicts: # packages/dfo-data-tools/pyproject.toml # pyproject.toml # uv.lock
jm-rivera
marked this pull request as ready for review
October 23, 2026 11:57
jm-rivera
marked this pull request as draft
October 23, 2026 11:59
jm-rivera
marked this pull request as ready for review
October 23, 2026 11:59
The pin comment names the spec-kit dev commit and what it registers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The API checks recorded client exchanges against the composed app's current schemas and code registry. It builds OpenAPI with Search, Datasets, Graph, and Presentation enabled, then validates recorded request bodies, response bodies, and problem statuses. Strict required fields must remain required and non-nullable, strict enum values must remain admitted, and every contract action must name an active registry code. Empty scenario, exchange, or problem coverage fails the gate. This establishes schema and registry compatibility; SDK replay and live journeys check client handling and deployed behavior.
Search binds
/api/search/resolveand/api/search/optionsto their registry endpoints. This disambiguates codes whose status differs across surfaces, with refusal exemplars verifying each Search mapping.Wire-change guidance is in
docs/development.md.CONTRIBUTING.mdcovers public contributions.Merge after #66, #67, and #68. Their wire changes are required by the recorded scenarios. All six spec-kit pins currently identify
98d78beand must move together to the final release commit containing DevelopmentFinanceObservatory/spec-kit#10 and DevelopmentFinanceObservatory/spec-kit#11 before merge. The test group includesdfo-spec-kit-client-contract, and a check keeps its pin aligned with the kit.Testing: a current-dev composition with #65, #66, #67, and #68 passed all 101 contract cases plus 99 registry, source-choice, source-scope, and assumption tests, serially. The standalone branch depends on the earlier PRs for the recorded wire behavior.