Skip to content

feat(client-contract): leave out party totals by default and order columns for reading - #24

Draft
jm-rivera wants to merge 2 commits into
docs/client-waiting-rulesfrom
feat/reading-defaults
Draft

jm-rivera wants to merge 2 commits into
docs/client-waiting-rulesfrom
feat/reading-defaults

Conversation

@jm-rivera

Copy link
Copy Markdown
Contributor

A client fetch now leaves out party totals by default. With an open party role, the long layout used to hold group and whole-total rows such as DAC countries, total beside their members, so summing value over every donor to Kenya gave 15.2bn for 2022 where the members sum to 3.5bn, with no warning. The build now posts "without_party_totals": true unless the caller passes party_totals (Python) or partyTotals (TypeScript). A role the caller binds keeps what it binds, so a total named by its dcid still comes back. The wire flag is unchanged. The README states what a ref records under the default, that artifact matching ignores the flag, and that newest_version names the newest plain build, which a cut ref's content_version need not equal.

The new column-order rule (rules/column-order.json) sets the order in which a client presents a dataset's columns: time period, provider, recipient, subject, purpose, value, unit, flow type and provenance first, the label column winning over the code column, then the served order, with stat_var last. A stacked dataset puts part first. A client hashes the downloaded file before ordering, so digests and refs keep the server's bytes. The validator reads each file's served order from its Parquet schema with pyarrow and checks every recorded columns list against the rule applied to it, so a hand edit to columns fails. A missing or malformed rule file fails the validator and the capture tool.

Dev had moved 12 commits past the last recording, so every scenario is re-recorded at apis c9a1b905 and served/openapi.json is recaptured. With the flag stripped, request bodies in the 120 existing scenarios are unchanged; the remaining differences are extra running polls and four recipient labels dev renamed. Two new recorded scenarios, fetch-spec-party-totals-left-out (96 rows) and fetch-spec-party-totals-kept (111 rows), pin the default and the option. The capture tool's pacing check now keeps the first three running polls, since this build polled running five times. The tools tests need pyarrow, and ci.yml installs it.

This stacks on #23. The SDK consumes it after apis bumps its spec-kit pin.

Testing: uv run tools/validate_client_contract.py reports 122 valid scenarios; tools/tests passes 537 tests and python/client-contract 105, including fault rows that fail when the party-totals or column checks are removed.

…lumns for reading

A fetch now posts "without_party_totals": true unless the caller passes
party_totals (Python) or partyTotals (TypeScript). A role the caller binds
keeps what it binds. The wire flag is unchanged; only the client default moves.

The new column-order rule (rules/column-order.json, nine slots with vectors)
puts time period, provider, recipient, subject, purpose, value, unit, flow type
and provenance first, then the served order, with stat_var last. A client
hashes the downloaded file before it orders the columns, so digests and refs
keep the server's file.

Every scenario is re-recorded against apis dev at c9a1b905, and
served/openapi.json is recaptured. Two new recorded scenarios,
fetch-spec-party-totals-left-out and fetch-spec-party-totals-kept, pin the
default and the option.
Every accepted ref example now carries the rendering, parts, content
version and sentence of a member its scenario served, so ref-one-member
pins the plain rendering from fetch-spec-party-totals-kept and
ref-answer-member comes from answer-definite-fetch. The validator checks
both, with a fault row each.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant