Skip to content

fix: judge sampled sole values on the walked family, and never answer a 202 done - #123

Merged
jm-rivera merged 9 commits into
devfrom
fix/search-datasets-open-items
Oct 24, 2026
Merged

jm-rivera merged 9 commits into
devfrom
fix/search-datasets-open-items

Conversation

@jm-rivera

@jm-rivera jm-rivera commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Part of #101

Five fixes from the SDK end-to-end follow-ups and a Codex review of #113.

A sole slot value claimed from a recall sample was never checked against the family. _validate_sole_option_bindings knew whether a claim rested on the whole family or a sample and dropped that knowledge, and the member walk then read the family filtered by the untested claim, so a one-member sample asserting gender=Female answered the female series where the question asked for the total. A sample-only claim now carries sampled (excluded from every dump and schema, so no prompt, cache key or spec_id moves); the walk reads that slot open, and _judge_sampled_claims confirms the claim on the walked members or unbinds it. An unsettled walk leaves the claim under membership_unverified. No extra member read. The last box of #101 needs the omitted-slot fill of #99, which follows as its own PR. The paired eval caught one consequence: bind_shared_values binds the value an unexported family is told apart by (IDS debt service, financialFlowType=ServicingPayment), and it skipped a slot a sampled claim held, so the walk refuted the claim and "total debt service of Kenya" served interest payments too; a sampled claim now leaves the slot to that rule.

A submission that found a done operation could answer 202 done, which a contract client cannot follow, and for a subject served in two renderings the poll then served a stale version. OperationStore.create_or_get now resets every done row a submission finds, and the run serves what the head rule gives at run time; the rejected-version plumbing through submit_over_budget, current and spec_id, and the refresh door's dead done branch and RefreshRebuildUnservable, are removed. Because every reset goes through backlog admission, serve_mint and the definition door's _materialise_and_record now re-read the head when admission refuses and answer 200 when it serves, so a finished result is never refused 503. C187, the datasets README and docs/datasets-sync-ceiling.md state both rules. The 202 failed reply's empty error member is #117.

The Congo wording fix in #113 renamed every place to its span, so "aid from Congo to Congo" collapsed both ends into one choice. _recover_entity_surfaces renames only when no other name would carry the same text, the parsed door's one-text-one-place rule.

Corpus case dq-dac-016 "GNI of Denmark" now expects dac1.gni, pinned to the DFO series that carries NY.GNP.MKTP.CD, and its known failure under a missing issue goes. The recall gap behind it is #115 and #116.

The spec-kit pins move to 0d19172 (DevelopmentFinanceObservatory/spec-kit#20), the contract DevelopmentFinanceObservatory/sdk#9 syncs to.

A reviewer should check _judge_sampled_claims, the interleaving argument in the backlog recheck commit, and the C187 wording.

Testing: CI passed at 125f216 (one shard rerun after a timing test went over its 0.5 s bound under load; it passes locally on repeat), and the paired eval passed on that head after blocking on dq-ids-001, 002, 014, 020 and 035 at caf1154. Locally tests/engine, render, eval, datasets_api, dataset_store, platform and contract passed, as did the live-postgres store tests, ruff, ty check and the eval corpus check.

…ked family

_validate_sole_option_bindings knew whether a sole-value claim rested
on the whole family or on a recall sample, and dropped that knowledge,
so the member walk read the family filtered by a claim it never tested.
A one-member sample asserting gender=Female answered the female series
where the question asked for the total.

A claim from an incomplete population now carries sampled=True
(excluded from every dump and schema). The walk reads sampled slots
open, and _judge_sampled_claims confirms each claim on the walked
members or refutes it and unbinds the slot. An unsettled walk leaves
the claim standing under membership_unverified. No extra member read.

Refs #101

(cherry picked from commit 8214f72fdad8df9065b5c624c9a2fcb95800d401)
dq-dac-016 "GNI of Denmark" expected the World Bank family base.gni,
but the DFO series dev answers with carries the World Bank indicator
NY.GNP.MKTP.CD, and DAC1 is its only provenance for Denmark. Pin the
dac1.gni family to that series, point the case at it, and drop the
known failure, whose issue number did not exist.

(cherry picked from commit f98b2b8994c8d38de2f3dc5cd2251c36f5441e8b)
A submission reaches the operation store only when the subject's head
could not answer it, yet create_or_get returned a done row at another
version unchanged. Two renderings of one over-budget subject make that
deterministic: a csv mint after an xlsx rebuild answered 202 done, and
its poll served the old version. A client following the contract also
raises on a 202 done, which carries no manifest.

create_or_get now resets every done row to pending and enqueues it
again, and the run serves what the head rule gives at run time. This
removes the rejected-version plumbing through submit_over_budget,
current and spec_id, and the refresh door's done branch, two-pass loop
and RefreshRebuildUnservable. C187, the datasets README and
datasets-sync-ceiling state the rule.

(cherry picked from commit 37b2e6ff660ebcf0c6485acd4634988daa1ff35c)
A submission that found its operation just completed reset it and went
through backlog admission again, so with a full backlog it answered 503
although the head already served the result. serve_mint and the
definition door's _materialise_and_record now catch
OperationBacklogFull, read and judge the head once more under their own
head rule, and answer from it when it serves; otherwise the refusal
stands. Admission itself is unchanged, and the recheck costs one head
read on the 503 path only.

(cherry picked from commit 877a2ea20558a79c12b004eb08c4e96bd1082d52)
0b5ea77 renamed an entity to the span the user wrote whenever the
span's ambiguity settled it. For "aid from Congo to Congo" both ends
became "Congo", and since the parsed door reads one text as one place
per part, every option pinned both ends to the same country.
_recover_entity_surfaces now renames a name to its span only when no
other name in the part would carry the same text, so the two ends keep
the extractor's names and independent choices, while "ODA
disbursements from Germany to Congo" still reads "Congo".

(cherry picked from commit 5d42bbd6ba3bf4f0b159abebf331554d141bce4b)
…r a sampled claim

bind_shared_values binds the value an unexported family shares on a
property the exported rows also carry, since that value is what tells
the family apart (IDS debt service: financialFlowType=ServicingPayment
on a five-tuple the index shares with CRS). _unexported_shared_values
counted a sampled claim as already bound and skipped the slot, so only
the sample backed the claim; the walk then read the slot open, listed
interest payments too, and refuted it, and "total debt service of
Kenya" served both flows unbound (eval dq-ids-001, 002, 014, 020, 035).
A sampled claim now leaves the slot open to the rule, and a claim of
the shared value is kept with its flag cleared.
@github-actions github-actions Bot removed the run-eval label Oct 24, 2026
@jm-rivera
jm-rivera merged commit 6c65b07 into dev Oct 24, 2026
40 of 42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant