feat(client-contract): add entity_choice and per-row entity_refs, re-record at apis 99e62038c

python/client-contract/CHANGELOG.md

@@ -7,11 +7,11 @@ All notable changes to the `dfo-spec-kit-client-contract` Python package will be
 ### Added
 
 - Package scaffolding (uv, ruff, ty, pytest).
-- `served/openapi.json`, the composed dfo-apis app's OpenAPI document, and `served/SOURCE.json`, naming dfo-apis commit `0ba344270784ae9929a369da8acd00b399dfced8`. The document admits `timed_out` as a no_data reason and the optional `entity_refs` on each answer part, a list of `EntityRefs`, the graph nodes each restatement entity text binds.
+- `served/openapi.json`, the composed dfo-apis app's OpenAPI document, and `served/SOURCE.json`, naming dfo-apis commit `99e62038ceb4b459c2ac699549076ff123c166aa`. The document admits `timed_out` as a no_data reason and the optional `entity_refs` on each answer part, a list of `EntityRefs` with one entry per restatement entity row, the graph nodes that row binds. `NoDataOption` and `SentenceToken` carry `entity_choice`, the index into `NoData.ambiguous_entities` of the group an entity_ambiguous option settles or a slot token asks, so a client groups a place menu by it when two wordings share one text.
 - `strict-fields.json` and its schema: 29 members that stay required and non-nullable, each one a client reads, and the answer kinds, part statuses and operation statuses the clients branch on.
 - `actions.json` and its schema: the caller's next step for each of the 102 active refusal codes on `problem_details` or `no_data_reason`, with the gap kind of each no_data reason.
 - The status fallback for a refusal whose code has no row: 400 and 422 to `InvalidRequestError`, 401 to `AuthenticationError`, 403 to `PermissionDeniedError`, 404 and 410 to `NotFoundError`, 408 to `DfoTimeoutError`, 429 to `RateLimitError`, and any other 4xx or 5xx to `InvalidRequestError` or `ServerError`.
-- `scenario.schema.json`, the scenario record, and 120 scenarios: 31 recorded against the deployed dfo-apis dev app at `0ba344270`, and 89 synthetic, each derived from a recorded one by a one-line edit, with 19 binary bodies totalling 1,124,502 bytes and records and bodies together 13,161,129 bytes. The scenarios cover the several-part choice entries and the assumptions each identity entry echoes, a picked reading whose assumption Search states in its answer, the place choice through a no_data part's options, the choice refusals, a part without variable_text, an unknown status without a restatement, a shared label, a one-part option whose data_query holds two parts, the separate and operations echo, an unsupported part's placeholder, the timed_out, unsupported and unknown gaps, an unknown answer kind, the `SearchWarning` rule, the stacked dataset's `part` column, the frame layout, a strict fetch's `DataWarning`, a spec fetch that rebinds one of two entity roles and keeps the other, one that drops a role bound to null, a spec whose destination is a named group fetched with a window or with its origin rebound, whose sentence lookup posts the groups of the roles the build left bound, a twice cut-off build, a cut-off build retried after its Retry-After, a build refused 504 then 503 and one refused 503 then 504, a refused build, a build refused 501 `capability_unavailable`, a digest mismatch, a signed URL refused on both rounds, a relative files-route Location, an artifact url on a foreign origin, on the storage host and absolute on the API's origin, a manifest definition without identity or schema_version, a Search spec without identity or resolved, nested specs no rule reads, an empty parts list, a Search body that is not an object, a codeless 503 problem document that is not retryable, a 401, an identity resubmit answering another spec, a part whose status is not a string, a Search spec whose definition names another spec_id, a Search response with no JSON body, an option with no data_query, empty entities and facets, a Sunset header issued once, `x-dfo-warning` messages and a Deprecation and Sunset pair, an unreadable versions page, a mint and a spec fetch whose build answers 202 and is polled to done, a resumed operation, a rate-limited poll, 503 polls retried within the budget an answered poll restores, a 202 that reads running, polls whose `retry_after_ms` changes from one to the next, a 404 poll that raises, a failed operation posted again, a failed operation with no error, and a ref whose stored file expired, read through the file selector, rebuilt at the same version, at a newer one, or refused with the newest version read from the versions route: the first version it lists with no `base_content_version`, the newest plain build, so a derived version listed before it is skipped.
+- `scenario.schema.json`, the scenario record, and 121 scenarios: 32 recorded against the deployed dfo-apis dev app at `99e62038c`, and 89 synthetic, each derived from a recorded one by a one-line edit, with 19 binary bodies totalling 1,124,808 bytes and records and bodies together 13,186,279 bytes. `search-place-ambiguous-twice` pins two entity_ambiguous groups of one text, each option and slot token naming its group by `entity_choice`. The scenarios cover the several-part choice entries and the assumptions each identity entry echoes, a picked reading whose assumption Search states in its answer, the place choice through a no_data part's options, the choice refusals, a part without variable_text, an unknown status without a restatement, a shared label, a one-part option whose data_query holds two parts, the separate and operations echo, an unsupported part's placeholder, the timed_out, unsupported and unknown gaps, an unknown answer kind, the `SearchWarning` rule, the stacked dataset's `part` column, the frame layout, a strict fetch's `DataWarning`, a spec fetch that rebinds one of two entity roles and keeps the other, one that drops a role bound to null, a spec whose destination is a named group fetched with a window or with its origin rebound, whose sentence lookup posts the groups of the roles the build left bound, a twice cut-off build, a cut-off build retried after its Retry-After, a build refused 504 then 503 and one refused 503 then 504, a refused build, a build refused 501 `capability_unavailable`, a digest mismatch, a signed URL refused on both rounds, a relative files-route Location, an artifact url on a foreign origin, on the storage host and absolute on the API's origin, a manifest definition without identity or schema_version, a Search spec without identity or resolved, nested specs no rule reads, an empty parts list, a Search body that is not an object, a codeless 503 problem document that is not retryable, a 401, an identity resubmit answering another spec, a part whose status is not a string, a Search spec whose definition names another spec_id, a Search response with no JSON body, an option with no data_query, empty entities and facets, a Sunset header issued once, `x-dfo-warning` messages and a Deprecation and Sunset pair, an unreadable versions page, a mint and a spec fetch whose build answers 202 and is polled to done, a resumed operation, a rate-limited poll, 503 polls retried within the budget an answered poll restores, a 202 that reads running, polls whose `retry_after_ms` changes from one to the next, a 404 poll that raises, a failed operation posted again, a failed operation with no error, and a ref whose stored file expired, read through the file selector, rebuilt at the same version, at a newer one, or refused with the newest version read from the versions route: the first version it lists with no `base_content_version`, the newest plain build, so a derived version listed before it is skipped.
 - `client-codes.json` and its schema: the 23 codes a client sets on the errors it raises itself, each with its error class, retryable flag and condition, disjoint from `actions.json` and `registries/codes.json`, and the `client_codes()` and `client_codes_schema()` accessors. Every scenario error the client raises itself carries its code, and `refusal_without_code` names a refusal whose body carries no code.
 - `proxy-codes.json` and its schema, the website proxy's `rate_limited` and `transient_unavailable` codes with their statuses, which the proxy sends in problem documents of the shape every refusal has, both retryable, with `Retry-After` on the 429, so a client reads one refusal shape, and `warning-codes.json` and its schema, the 6 codes a client sets on the warnings it issues itself, with the `proxy_codes()` and `warning_codes()` accessors.
 - `ref_digest_changed` in `warning-codes.json`, a `DfoWarning` for a ref member whose `content_version` serves a file with another digest than the ref's, because the server rendered the same data again. The client returns the data, with no error under strict.

python/client-contract/README.md

@@ -207,7 +207,7 @@ Each record names the commit in the responses' `x-dfo-commit` header, and the ru
 uv run tools/capture_client_contract.py record --base-url http://127.0.0.1:8083/api
 ```
 
-A record's `origin.base_url` is the base URL the run was given, a local app's included. A local app's signed URL, which its launcher serves at `/__e2e__/blob/{bucket}/{object}` on the API's origin, is written as `https://storage.googleapis.com/{bucket}/{object}` with its query redacted, as a deployment signs it, because a production download is cross-origin and the client sends no key there; the run, like the validator, refuses a record holding any absolute URL other than such a storage URL, or a `location` under the base URL, so a local app's host under another name or port is refused too. A server answers a version it has built inline, so `mint-async-build` and `operation-resume` each need a spec just above the 100,000-row sync budget that the server has not built: ODA disbursements from the 32 DAC countries to the 133 G77 countries from 1963 to 2024 for the mint, and from the 38 OECD countries to the G77 countries from 1966 to 2024 for the resumed operation, each one variable over some 3,700 donor and recipient pairs, so the done manifest stays small. A local recording therefore starts on a fresh datasets database with `DATASETS_SYNC_BUDGET_ROWS=100000`. `--scenarios-dir` writes records outside the package, unvalidated.
+A record's `origin.base_url` is the base URL the run was given, a local app's included. A local app's signed URL, which its launcher serves at `/__e2e__/blob/{bucket}/{object}` on the API's origin, is written as `https://storage.googleapis.com/{bucket}/{object}` with its query redacted, as a deployment signs it, because a production download is cross-origin and the client sends no key there; the run, like the validator, refuses a record holding any absolute URL other than such a storage URL, or a `location` under the base URL, so a local app's host under another name or port is refused too. A server answers a version it has built inline, so `mint-async-build` and `operation-resume` each need a spec just above the 100,000-row sync budget that the server has not built: ODA disbursements from the 32 DAC countries to the 133 G77 countries from 1964 to 2024 for the mint, and from the 38 OECD countries to the G77 countries from 1965 to 2024 for the resumed operation, each one variable over some 3,700 donor and recipient pairs, so the done manifest stays small. A local recording therefore starts on a fresh datasets database with `DATASETS_SYNC_BUDGET_ROWS=100000`. `--scenarios-dir` writes records outside the package, unvalidated.
 
 ## DatasetRef
 

python/client-contract/src/dfo_client_contract/scenarios/answer-candidate-specs-not-a-list.json

@@ -30,7 +30,7 @@
         "status": 200,
         "headers": {
           "content-type": "application/json",
-          "x-dfo-trace-id": "21bb19c92c824a70b2eb378e85a9cfd0"
+          "x-dfo-trace-id": "962188ce8826466e8510fb819d5dc26d"
         },
         "json": {
           "schema_version": 1,
@@ -41,21 +41,21 @@
             "extract_skipped": false
           },
           "diagnostics": {
-            "engine_build": "0ba344270784ae9929a369da8acd00b399dfced8",
-            "trace_id": "21bb19c92c824a70b2eb378e85a9cfd0",
+            "engine_build": "99e62038ceb4b459c2ac699549076ff123c166aa",
+            "trace_id": "962188ce8826466e8510fb819d5dc26d",
             "warnings": [],
             "timing_ms": {
-              "total": 8287,
+              "total": 1710,
               "by_step": {
                 "coverage": 0,
-                "extract": 1330,
-                "cache_lookup": 2
+                "extract": 0,
+                "cache_lookup": 0
               }
             },
             "llm_usage": {
-              "input_tokens": 2237,
-              "output_tokens": 143,
-              "cached_tokens": 842,
+              "input_tokens": 0,
+              "output_tokens": 0,
+              "cached_tokens": 0,
               "jev_input_tokens": 13358,
               "jev_output_tokens": 2855
             },
@@ -67,28 +67,28 @@
               "variable_text": "mortality",
               "warnings": [],
               "timing_ms": {
-                "total": 6951,
+                "total": 1705,
                 "by_step": {
-                  "recall": 261,
+                  "recall": 1,
                   "phase_resolve_places": 0,
                   "phase_check_entities": 0,
-                  "shape_arcs": 7,
-                  "shape_probe": 122,
+                  "shape_arcs": 4,
+                  "shape_probe": 2,
                   "shape_groups": 6,
                   "shape_labels": 1,
                   "shape_admission": 1,
                   "phase_derive_shapes": 0,
-                  "shape": 241,
+                  "shape": 119,
                   "phase_prune_and_rank": 0,
-                  "bind": 142,
-                  "phase_bind": 17,
-                  "phase_settle": 6,
-                  "phase_place_bindings": 490,
+                  "bind": 146,
+                  "phase_bind": 19,
+                  "phase_settle": 2,
+                  "phase_place_bindings": 30,
                   "phase_facet_requests": 0,
-                  "materialise": 115,
-                  "phase_materialise": 1,
-                  "answer": 243,
-                  "phase_outcome": 5435
+                  "materialise": 21,
+                  "phase_materialise": 25,
+                  "answer": 30,
+                  "phase_outcome": 1312
                 }
               },
               "rendered_sentence": "2 readings. Kenya's deaths rate per capita. World Development Indicators · 2013 to 2022. Kenya's number of deaths. Kenya Census · 2018 to 2022.",
@@ -810,7 +810,7 @@
                           "start_year": 2013,
                           "end_year": 2022
                         },
-                        "capability_version": "cap_9deef72763b45f40",
+                        "capability_version": "cap_88defe77cc79e0c0",
                         "default_reading": {
                           "applied": true,
                           "removed_stat_vars": []
@@ -2344,7 +2344,7 @@
                           "start_year": 2013,
                           "end_year": 2022
                         },
-                        "capability_version": "cap_9deef72763b45f40",
+                        "capability_version": "cap_88defe77cc79e0c0",
                         "default_reading": {
                           "applied": true,
                           "removed_stat_vars": []
@@ -2579,7 +2579,7 @@
       "error": {
         "class": "ContractError",
         "code": "contract_violation",
-        "trace_id": "21bb19c92c824a70b2eb378e85a9cfd0"
+        "trace_id": "962188ce8826466e8510fb819d5dc26d"
       }
     }
   ]