Skip to content

feat(client-contract): identify client requests and present datasets in reading order - #25

Merged
jm-rivera merged 1 commit into
devfrom
feat/client-identity
Oct 26, 2026
Merged

jm-rivera merged 1 commit into
devfrom
feat/client-identity

Conversation

@jm-rivera

@jm-rivera jm-rivera commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

This adds two client rules on top of #24: every API request names its client in an X-DFO-Client header, and a fetched dataset comes back with its columns and rows in reading order.

The client-identity rule (rules/client-identity.json, 18 vectors) has every API request carry X-DFO-Client: <client>/<version> (<runtime>), such as dfo-python/0.1.0 (CPython 3.14) or dfo-ts/0.1.0 (browser). A caller's headers or header provider naming it raise header_invalid, and no request to another origin, a signed storage URL included, carries it. Scenario records hold no request headers, so a client's replay checks that each path and origin_path request carries the header and each url request does not.

The reading-order rule replaces column-order from #24 (rules/reading-order.json, 10 vectors with rows). Columns run: the filled reading slots, then the other columns whose values differ between rows, then the constant columns, with stat_var last. Rows within each member sort by the filled slots without value and the time period, then by the varying columns, then by the time period, so a CRS series of Germany's health ODA to Nigeria split by co-operation modality reads every A02 year, then every C01 year. Keys sort ascending with nulls last, ties keep served order, and members keep member order. A client hashes the file before ordering anything, so digests and refs keep the server's file.

tools/validate_client_contract.py reads each served Parquet file once through served_table and checks every scenario's columns and every vector's rows. Scenarios pin columns only, because dataset outcomes hold no rows; the vectors pin row order. 28 scenarios change their columns and no body file changes.

The base-URL rule now states every check both clients make: no userinfo (an empty one included), query, fragment, dot segment, character outside visible ASCII, or IPv4 host in a form other than four decimal numbers. A relative URL is valid only in a TypeScript client inside a browser page, and a valid value is used less its trailing slashes. Seven new vectors in rules/base-url.json pin these cases. The User-Agent sentence now names every runtime outside a browser where TypeScript sends it.

The role-keys rule (rules/role-keys.json, 13 vectors) names the keys of entities=: place for the subject, posted as observationAbout, and origin and destination for flows. Spec.roles and specRoles(spec) list a spec's roles under those keys, and a key outside them, such as observationAbout or a column name like financingRecipient, raises invalid_argument naming the spec's roles.

Testing: the validator reports 122 valid scenarios; pytest tools/tests with pyarrow, 543 passed; python/client-contract tests, 144 passed.

@jm-rivera
jm-rivera force-pushed the feat/reading-defaults branch from 41c47f8 to ca207f0 Compare October 26, 2026 09:31
@jm-rivera
jm-rivera force-pushed the feat/client-identity branch from 342707e to b24cd9e Compare October 26, 2026 09:39
…he base URL and role keys

The client-identity rule (rules/client-identity.json, 18 vectors) has every
API request carry X-DFO-Client: <client>/<version> (<runtime>), such as
dfo-python/0.1.0 (CPython 3.14) or dfo-ts/0.1.0 (browser). A caller's
headers or header provider naming it raise header_invalid, and no request to
another origin, a signed storage URL included, carries it. Scenario records
keep no request headers, so a client's replay checks that each path and
origin_path request carries the header and each url request does not.

The column-order rule becomes reading-order (rules/reading-order.json, 10
vectors with rows). Columns run: the filled reading slots, then the other
columns whose values differ between rows, then the constant columns, with
stat_var last. A stack puts part first and judges the groups over the
stacked table.

Each member's rows are sorted by the filled slots without value and the
time period, then by the varying columns (stat_var among them when it
varies), then by the time period, so each series' years read together.
Keys sort ascending with nulls last, ties keep served order, and members
keep member order.

The validator reads served Parquet once per file through served_table and
checks every scenario's columns and every vector's rows. 28 scenarios
change their columns only; no body file changes, so digests stay the
server's.

A base URL is valid when it is an absolute http or https URL with a host
and no userinfo (an empty one included), query, fragment, dot segment,
character outside visible ASCII, or IPv4 host in a form other than four
decimal numbers. A relative URL is valid only in a TypeScript client inside
a browser page, which resolves it against the page's address, and a valid
value is used less its trailing slashes. Seven new vectors in
rules/base-url.json pin these cases.

The User-Agent sentence now says TypeScript sends dfo-ts/<version> outside
a browser page or browser worker, in Node, Deno, Bun or a server runtime
such as Cloudflare Workers.

The role-keys rule (rules/role-keys.json, 13 vectors) names the keys of
entities=: place for the subject, posted as observationAbout, and origin
and destination for flows, posted as themselves. Spec.roles and
specRoles(spec) list a spec's roles under those keys, a flow spec takes
both directions, and a key outside the spec's roles raises
invalid_argument naming the typed key and the spec's roles, so
observationAbout and column names such as financingRecipient are refused.
A spec that names no role posts place as observationAbout and leaves the
rest to the server.
@jm-rivera
jm-rivera changed the base branch from feat/reading-defaults to dev October 26, 2026 10:25
@jm-rivera
jm-rivera force-pushed the feat/client-identity branch from b24cd9e to 315cb4a Compare October 26, 2026 10:25
@jm-rivera
jm-rivera marked this pull request as ready for review October 26, 2026 10:26
@jm-rivera
jm-rivera merged commit 26f733c into dev Oct 26, 2026
22 checks passed
@jm-rivera
jm-rivera deleted the feat/client-identity branch October 26, 2026 13:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant