diff --git a/python/client-contract/CHANGELOG.md b/python/client-contract/CHANGELOG.md index 87cfe85..5cf3edf 100644 --- a/python/client-contract/CHANGELOG.md +++ b/python/client-contract/CHANGELOG.md @@ -13,7 +13,8 @@ All notable changes to the `dfo-spec-kit-client-contract` Python package will be - The status fallback for a refusal whose code has no row: 400 and 422 to `InvalidRequestError`, 401 to `AuthenticationError`, 403 to `PermissionDeniedError`, 404 and 410 to `NotFoundError`, 408 to `DfoTimeoutError`, 429 to `RateLimitError`, and any other 4xx or 5xx to `InvalidRequestError` or `ServerError`. - `scenario.schema.json`, the scenario record, and 118 scenarios: 31 recorded against the deployed dfo-apis dev app at `035d29cd0`, and 87 synthetic, each derived from a recorded one by a one-line edit, with 19 binary bodies totalling 1,124,502 bytes and records and bodies together 13,080,568 bytes. The scenarios cover the several-part choice entries and the assumptions each identity entry echoes, a picked reading whose assumption Search states in its answer, the place choice through a no_data part's options, the choice refusals, a part without variable_text, an unknown status without a restatement, a shared label, a one-part option whose data_query holds two parts, the separate and operations echo, an unsupported part's placeholder, the timed_out, unsupported and unknown gaps, an unknown answer kind, the `SearchWarning` rule, the stacked dataset's `part` column, the frame layout, a strict fetch's `DataWarning`, a spec fetch that rebinds one of two entity roles and keeps the other, one that drops a role bound to null, a spec whose destination is a named group fetched with a window or with its origin rebound, whose sentence lookup posts the groups of the roles the build left bound, a twice cut-off build, a cut-off build retried after its Retry-After, a build refused 504 then 503 and one refused 503 then 504, a refused build, a build refused 501 `capability_unavailable`, a digest mismatch, a signed URL refused on both rounds, a relative files-route Location, an artifact url on a foreign origin, on the storage host and absolute on the API's origin, a manifest definition without identity or schema_version, a Search spec without identity or resolved, nested specs no rule reads, an empty parts list, a Search body that is not an object, a codeless 503 problem document that is not retryable, a 401, an identity resubmit answering another spec, a part whose status is not a string, a Search spec whose definition names another spec_id, a Search response with no JSON body, an option with no data_query, empty entities and facets, a Sunset header issued once, `x-dfo-warning` messages and a Deprecation and Sunset pair, an unreadable versions page, a mint and a spec fetch whose build answers 202 and is polled to done, a resumed operation, a rate-limited poll, 503 polls retried within the budget an answered poll restores, a 404 poll that raises, a failed operation posted again, a failed operation with no error, and a ref whose stored file expired, read through the file selector, rebuilt at the same version, at a newer one, or refused with the newest version read from the versions route: the first version it lists with no `base_content_version`, the newest plain build, so a derived version listed before it is skipped. - `client-codes.json` and its schema: the 23 codes a client sets on the errors it raises itself, each with its error class, retryable flag and condition, disjoint from `actions.json` and `registries/codes.json`, and the `client_codes()` and `client_codes_schema()` accessors. Every scenario error the client raises itself carries its code, and `refusal_without_code` names a refusal whose body carries no code. -- `proxy-codes.json` and its schema, the website proxy's `rate_limited` and `transient_unavailable` codes with their statuses, and `warning-codes.json` and its schema, the 5 codes a client sets on the warnings it issues itself, with the `proxy_codes()` and `warning_codes()` accessors. +- `proxy-codes.json` and its schema, the website proxy's `rate_limited` and `transient_unavailable` codes with their statuses, and `warning-codes.json` and its schema, the 6 codes a client sets on the warnings it issues itself, with the `proxy_codes()` and `warning_codes()` accessors. +- `ref_digest_changed` in `warning-codes.json`, a `DfoWarning` for a ref member whose `content_version` serves a file with another digest than the ref's, because the server rendered the same data again. The client returns the data, with no error under strict. - Error outcomes give the error's `trace_id`, the body's, else the `x-dfo-trace-id` header's, else null. Scenario outcomes give `retry_waits`, the waits before each retry, and `warnings`, the warning codes a step's responses issued. Scenario steps take no `timeout`. - `dataset-ref.schema.json`, the frozen `DatasetRef` format, with accepted and rejected examples. - `without_party_totals` in the ref's artifact rendering, `true` for the version rendered with party totals left out and absent for a plain rendering, with `ref-without-party-totals` and `ref-without-party-totals-false-rejected` as its examples. diff --git a/python/client-contract/README.md b/python/client-contract/README.md index 1a175ec..3e323d6 100644 --- a/python/client-contract/README.md +++ b/python/client-contract/README.md @@ -118,7 +118,7 @@ An answer's gaps follow from its parts. A candidates part is a `choice` gap, its ## Client codes -`client-codes.json` lists, sorted by code, each code a client sets on an error it raises when no problem document carries one: the `code`, the error `class` it raises, `retryable`, and a one-line `condition`. `retryable` is the flag both clients generate onto the error: `deadline_exceeded`, `download_failed` and `network_error` are retryable, `digest_mismatch` and the rest are not, and `refusal_without_code` is retryable as its status is, which its row gives as `by_status`. Both clients set an error's `code` from this list, so every error a client raises carries a code. A refusal keeps the server's code: the problem document's `code`, or the website proxy's `error`. A refusal whose body carries neither raises `refusal_without_code` with the class its HTTP status maps to, which its row gives as `by_status`. No other row does. A problem document without a code still gives the error its `retryable` and `trace_id`, so a 503 whose body says `retryable: false` raises at once, which `search-refused-without-code` pins. Every refusal's trace id is the response's trace id, as the trace-id rule under Scenarios gives it. `search-refused-without-code` pins the body's winning over the header, and `search-unauthorized` the header's for a refusal with no body. No client code is a code in `actions.json` or `registries/codes.json`, and each class is one the scenario schema names. Each reason the key-lookup rule raises, `key_empty`, `profile_not_found`, `profile_without_key` and `credentials_file_invalid`, is a row of class `AuthenticationError`, as are `key_invalid`, a key outside visible ASCII, and `credentials_file_unreadable`, a credentials file that exists but cannot be read. `header_invalid`, a header value with a control character, `insecure_transport`, a key over plain http to a non-loopback host, and `base_url_invalid`, a base URL that is empty, not absolute http or https, or carries userinfo, are `InvalidRequestError` rows. `response_too_large`, a decoded body past its cap, is a `ServerError` row. None of these six is retryable. The list holds no warning. `warning-codes.json` lists, sorted by code, each code a client sets on a warning it issues itself, with its class and condition: `route_deprecated`, a `DfoDeprecationWarning`, for a `Deprecation` or `Sunset` header; `server_warning`, a `DfoWarning`, for an `x-dfo-warning` header; and `key_checksum_mismatch`, `key_not_base62` and `key_unknown_prefix`, each a `DfoWarning`, for a key that fails the key-format rule with that reason. Both clients generate their warning codes from it and issue no other. No warning code is a code in another table or in `registries/codes.json`. A `SearchWarning` carries the code of the one way the readings differ from the question, a departure's or a warning's code or the verb of an operation Search did not run, and a null code when they differ in several ways. A `DataWarning` carries the manifest warning's code. +`client-codes.json` lists, sorted by code, each code a client sets on an error it raises when no problem document carries one: the `code`, the error `class` it raises, `retryable`, and a one-line `condition`. `retryable` is the flag both clients generate onto the error: `deadline_exceeded`, `download_failed` and `network_error` are retryable, `digest_mismatch` and the rest are not, and `refusal_without_code` is retryable as its status is, which its row gives as `by_status`. Both clients set an error's `code` from this list, so every error a client raises carries a code. A refusal keeps the server's code: the problem document's `code`, or the website proxy's `error`. A refusal whose body carries neither raises `refusal_without_code` with the class its HTTP status maps to, which its row gives as `by_status`. No other row does. A problem document without a code still gives the error its `retryable` and `trace_id`, so a 503 whose body says `retryable: false` raises at once, which `search-refused-without-code` pins. Every refusal's trace id is the response's trace id, as the trace-id rule under Scenarios gives it. `search-refused-without-code` pins the body's winning over the header, and `search-unauthorized` the header's for a refusal with no body. No client code is a code in `actions.json` or `registries/codes.json`, and each class is one the scenario schema names. Each reason the key-lookup rule raises, `key_empty`, `profile_not_found`, `profile_without_key` and `credentials_file_invalid`, is a row of class `AuthenticationError`, as are `key_invalid`, a key outside visible ASCII, and `credentials_file_unreadable`, a credentials file that exists but cannot be read. `header_invalid`, a header value with a control character, `insecure_transport`, a key over plain http to a non-loopback host, and `base_url_invalid`, a base URL that is empty, not absolute http or https, or carries userinfo, are `InvalidRequestError` rows. `response_too_large`, a decoded body past its cap, is a `ServerError` row. None of these six is retryable. The list holds no warning. `warning-codes.json` lists, sorted by code, each code a client sets on a warning it issues itself, with its class and condition: `ref_digest_changed`, a `DfoWarning`, for a ref member whose `content_version` serves a file with another digest than the ref's, because the server rendered the same data again, where the client returns the data; `route_deprecated`, a `DfoDeprecationWarning`, for a `Deprecation` or `Sunset` header; `server_warning`, a `DfoWarning`, for an `x-dfo-warning` header; and `key_checksum_mismatch`, `key_not_base62` and `key_unknown_prefix`, each a `DfoWarning`, for a key that fails the key-format rule with that reason. Both clients generate their warning codes from it and issue no other. No warning code is a code in another table or in `registries/codes.json`. A `SearchWarning` carries the code of the one way the readings differ from the question, a departure's or a warning's code or the verb of an operation Search did not run, and a null code when they differ in several ways. A `DataWarning` carries the manifest warning's code. ## Scenarios diff --git a/python/client-contract/src/dfo_client_contract/warning-codes.json b/python/client-contract/src/dfo_client_contract/warning-codes.json index 337e2a7..5d55edf 100644 --- a/python/client-contract/src/dfo_client_contract/warning-codes.json +++ b/python/client-contract/src/dfo_client_contract/warning-codes.json @@ -16,6 +16,11 @@ "class": "DfoWarning", "condition": "The resolved key starts with neither dfo_k_ nor dfo_p_, by the key-format rule; the client sends it anyway." }, + { + "code": "ref_digest_changed", + "class": "DfoWarning", + "condition": "A ref member's content_version serves a file whose digest differs from the ref's, because the server rendered the same data again; the client returns the data." + }, { "code": "route_deprecated", "class": "DfoDeprecationWarning", diff --git a/tools/tests/test_validate_client_contract.py b/tools/tests/test_validate_client_contract.py index 58fa9c3..7ce8bde 100644 --- a/tools/tests/test_validate_client_contract.py +++ b/tools/tests/test_validate_client_contract.py @@ -1184,6 +1184,11 @@ def change(doc: Any) -> None: _add_action({"code": "spec_not_found", "status": 404, "condition": "x"}), "'spec_not_found' is also a code in registries/codes.json", ), + ( + "warning-codes.json", + _drop_action("ref_digest_changed"), + "no row for 'ref_digest_changed'", + ), ( "warning-codes.json", _drop_action("route_deprecated"), @@ -1742,6 +1747,7 @@ def change(doc: Any) -> None: "proxy-code-unlisted", "proxy-code-wrong-status", "proxy-code-is-a-registry-code", + "warning-code-ref-digest-changed-missing", "warning-code-missing", "warning-code-no-rule-issues", "warning-code-is-a-client-code", diff --git a/tools/validate_client_contract.py b/tools/validate_client_contract.py index 3ead4d7..44ac99f 100644 --- a/tools/validate_client_contract.py +++ b/tools/validate_client_contract.py @@ -34,8 +34,8 @@ row of class `AuthenticationError`. - `proxy-codes.json` and `warning-codes.json` list codes in order, each once, none of them a code another table or `registries/codes.json` holds, and the - warning codes are those the client rules issue: `route_deprecated`, - `server_warning`, and one `key_` per key-format reason. + warning codes are those the client rules issue: `ref_digest_changed`, + `route_deprecated`, `server_warning`, and one `key_` per key-format reason. - Each scenario's file name matches its `name`, names are unique, `expect` holds one outcome per step, a step that acts on an earlier result follows one, a synthetic scenario derives from a recorded one, and every body file @@ -490,11 +490,16 @@ def _check_warning_codes( key_reasons: Sequence[str], ) -> list[str]: """As `_check_other_codes`, and the codes are those a client's rules - issue: each header warning, and one `key_` warning per - key-format reason.""" + issue: `ref_digest_changed`, each header warning, and one `key_` + warning per key-format reason.""" codes = [row["code"] for row in document["codes"]] problems = _check_other_codes("warning-codes.json", codes, taken) - wanted = {ROUTE_DEPRECATED, SERVER_WARNING, *(f"key_{r}" for r in key_reasons)} + wanted = { + REF_DIGEST_CHANGED, + ROUTE_DEPRECATED, + SERVER_WARNING, + *(f"key_{r}" for r in key_reasons), + } problems.extend( f"warning-codes.json: no row for {code!r}, a warning a client issues" for code in sorted(wanted - set(codes)) @@ -836,6 +841,9 @@ def retry_wait(headers: Mapping[str, str], attempt: int) -> float: return min(RETRY_BACKOFF_FIRST * 2**attempt, RETRY_BACKOFF_CAP) +# The warning a client issues when a ref's file digest has changed. +REF_DIGEST_CHANGED = "ref_digest_changed" + # The headers a client warns on, and the warning-codes.json code of each. DEPRECATION_HEADERS = ("deprecation", "sunset") ROUTE_DEPRECATED = "route_deprecated"