Repository navigation
feat(client-contract): identify client requests and present datasets in reading order - #25
Merged
Merged
Conversation
jm-rivera
force-pushed
the
feat/reading-defaults
branch
from
October 26, 2026 09:31
41c47f8 to
ca207f0
Compare
jm-rivera
force-pushed
the
feat/client-identity
branch
from
October 26, 2026 09:39
342707e to
b24cd9e
Compare
…he base URL and role keys The client-identity rule (rules/client-identity.json, 18 vectors) has every API request carry X-DFO-Client: <client>/<version> (<runtime>), such as dfo-python/0.1.0 (CPython 3.14) or dfo-ts/0.1.0 (browser). A caller's headers or header provider naming it raise header_invalid, and no request to another origin, a signed storage URL included, carries it. Scenario records keep no request headers, so a client's replay checks that each path and origin_path request carries the header and each url request does not. The column-order rule becomes reading-order (rules/reading-order.json, 10 vectors with rows). Columns run: the filled reading slots, then the other columns whose values differ between rows, then the constant columns, with stat_var last. A stack puts part first and judges the groups over the stacked table. Each member's rows are sorted by the filled slots without value and the time period, then by the varying columns (stat_var among them when it varies), then by the time period, so each series' years read together. Keys sort ascending with nulls last, ties keep served order, and members keep member order. The validator reads served Parquet once per file through served_table and checks every scenario's columns and every vector's rows. 28 scenarios change their columns only; no body file changes, so digests stay the server's. A base URL is valid when it is an absolute http or https URL with a host and no userinfo (an empty one included), query, fragment, dot segment, character outside visible ASCII, or IPv4 host in a form other than four decimal numbers. A relative URL is valid only in a TypeScript client inside a browser page, which resolves it against the page's address, and a valid value is used less its trailing slashes. Seven new vectors in rules/base-url.json pin these cases. The User-Agent sentence now says TypeScript sends dfo-ts/<version> outside a browser page or browser worker, in Node, Deno, Bun or a server runtime such as Cloudflare Workers. The role-keys rule (rules/role-keys.json, 13 vectors) names the keys of entities=: place for the subject, posted as observationAbout, and origin and destination for flows, posted as themselves. Spec.roles and specRoles(spec) list a spec's roles under those keys, a flow spec takes both directions, and a key outside the spec's roles raises invalid_argument naming the typed key and the spec's roles, so observationAbout and column names such as financingRecipient are refused. A spec that names no role posts place as observationAbout and leaves the rest to the server.
jm-rivera
force-pushed
the
feat/client-identity
branch
from
October 26, 2026 10:25
b24cd9e to
315cb4a
Compare
jm-rivera
marked this pull request as ready for review
October 26, 2026 10:26
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds two client rules on top of #24: every API request names its client in an X-DFO-Client header, and a fetched dataset comes back with its columns and rows in reading order.
The client-identity rule (
rules/client-identity.json, 18 vectors) has every API request carryX-DFO-Client: <client>/<version> (<runtime>), such asdfo-python/0.1.0 (CPython 3.14)ordfo-ts/0.1.0 (browser). A caller's headers or header provider naming it raiseheader_invalid, and no request to another origin, a signed storage URL included, carries it. Scenario records hold no request headers, so a client's replay checks that eachpathandorigin_pathrequest carries the header and eachurlrequest does not.The reading-order rule replaces column-order from #24 (
rules/reading-order.json, 10 vectors with rows). Columns run: the filled reading slots, then the other columns whose values differ between rows, then the constant columns, withstat_varlast. Rows within each member sort by the filled slots withoutvalueand the time period, then by the varying columns, then by the time period, so a CRS series of Germany's health ODA to Nigeria split by co-operation modality reads every A02 year, then every C01 year. Keys sort ascending with nulls last, ties keep served order, and members keep member order. A client hashes the file before ordering anything, so digests and refs keep the server's file.tools/validate_client_contract.pyreads each served Parquet file once throughserved_tableand checks every scenario'scolumnsand every vector's rows. Scenarios pin columns only, because dataset outcomes hold no rows; the vectors pin row order. 28 scenarios change theircolumnsand no body file changes.The base-URL rule now states every check both clients make: no userinfo (an empty one included), query, fragment, dot segment, character outside visible ASCII, or IPv4 host in a form other than four decimal numbers. A relative URL is valid only in a TypeScript client inside a browser page, and a valid value is used less its trailing slashes. Seven new vectors in
rules/base-url.jsonpin these cases. The User-Agent sentence now names every runtime outside a browser where TypeScript sends it.The role-keys rule (
rules/role-keys.json, 13 vectors) names the keys ofentities=:placefor the subject, posted asobservationAbout, andoriginanddestinationfor flows.Spec.rolesandspecRoles(spec)list a spec's roles under those keys, and a key outside them, such asobservationAboutor a column name likefinancingRecipient, raisesinvalid_argumentnaming the spec's roles.Testing: the validator reports 122 valid scenarios;
pytest tools/testswith pyarrow, 543 passed;python/client-contracttests, 144 passed.