Skip to content

fix(presentation): match refusals to the code registry and bind routes #75

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
jm-rivera wants to merge 1 commit into
base: dev
Choose a base branch
from
Draft

fix(presentation): match refusals to the code registry and bind routes #75

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 29 additions & 20 deletions docs/conformance-presentation-api-v1.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ verified against `src/presentation_api` and the shared platform envelope it rend
| C7b | POST /api/presentation/components/{component_id}/revisions/{revision}/render takes `{"frame": {spec_id, content_version}}` and answers what a component read answers for that revision, built from its stored intent on the named frame, with `resolved_payload.provenance.inputs` naming that frame and `provenance.refreshed_from` set to the revision. It appends no revision and pins the frame for the component. It admits only the caller holding `LIVE_REFRESH_PRINCIPAL` in the `DATASETS_PRINCIPAL_HEADER` header, checked before the path or body is validated | implemented | |
| C7c | GET /api/presentation/components/{component_id}/revisions/{revision}/render/data?spec_id=&content_version= serves the C7a export of that revision rendered on the named frame, with the same `format` and `bundle` options, under the caller rule of C7b. It pins nothing | implemented | |
| C7d | C7b and C7c accept the frame the revision is stored on, a dataset frame of the same `spec_id` and rendering, or a derived frame whose recipe has the stored frame's `normal_form_hash`. Any other frame refuses `render_target_mismatch`, 409 | implemented | |
| C7e | On C7b and C7c `binding_invalid` and `constraint_violation` answer 422, and `frame_unavailable`, `release_mismatch`, and `resource_exhausted` answer 409, the statuses the code registry maps for `presentation.render`. "Refusal code detail" below gives the statuses create and revise serve | implemented | |
| C7e | On C7b and C7c every refusal answers the status the code registry maps for its code on `presentation.render`: `binding_invalid` and `constraint_violation` answer 422, and `frame_unavailable`, `release_mismatch`, and `resource_exhausted` answer 409. A component with no current revision refuses `edit_conflict`, 409. "Refusal code detail" below gives the statuses create and revise serve | implemented | |

## Request shape

Expand Down Expand Up @@ -99,7 +99,7 @@ alone.
| write_refused | 403 | do_not_retry | implemented | |
| request_invalid | 400 | headline correct_and_retry; wire `retryable` always false | implemented | |
| resource_exhausted | 400 | do_not_retry | implemented | |
| frame_unavailable | 404 or 503 per cause (see below) | refresh_then_retry (expired, ops_unsupported) / do_not_retry (revoked, missing_rendering, digest_mismatch, quarantined, unknown) / retry_after_delay (io) | implemented | |
| frame_unavailable | 404 | refresh_then_retry (expired, ops_unsupported) / do_not_retry (revoked, missing_rendering, digest_mismatch, quarantined, unknown) | implemented | |
| release_mismatch | 503 | retry_after_delay | implemented | |
| service_unavailable | 503 | retry_after_delay | implemented | |
| timeout | 504 | retry_after_delay | implemented | |
Expand All @@ -108,31 +108,40 @@ alone.
| refresh_not_permitted | 403 | do_not_retry | implemented | |
| render_target_mismatch | 409 | do_not_retry | implemented | |

### frame_unavailable outcome by reason
### FrameUnavailable outcome by reason

| Reason | HTTP | retryable |
| ------------------ | ---- | --------- |
| expired | 404 | true |
| revoked | 404 | false |
| ops_unsupported | 404 | true |
| missing_rendering | 404 | false |
| digest_mismatch | 404 | false |
| quarantined | 404 | false |
| io | 503 | true |
| unknown | 404 | false |
| any other reason | 503 | true |
A frame store that fails to answer (`io`) refuses `service_unavailable`. A reason outside this table fails classification, and the platform default answers `internal_error`.

| Reason | Code | HTTP | retryable |
| ----------------- | ------------------- | ---- | --------- |
| expired | frame_unavailable | 404 | true |
| revoked | frame_unavailable | 404 | false |
| ops_unsupported | frame_unavailable | 404 | true |
| missing_rendering | frame_unavailable | 404 | false |
| digest_mismatch | frame_unavailable | 404 | false |
| quarantined | frame_unavailable | 404 | false |
| io | service_unavailable | 503 | true |
| unknown | frame_unavailable | 404 | false |

## Registry binding

`presentation_api.codes.PRESENTATION_CODE_CATALOGUE` union
`presentation_api.codes.PRESENTATION_SERVED_PLATFORM_CODES` — the seven codes this surface mints
itself, plus the six platform codes its own handlers, its own `validation_problem` path, or the
platform default put on the wire (`not_found`, `request_invalid`, `service_unavailable`, `timeout`,
`internal_error`, `unsupported_schema_version`) — is exactly the set of active spec-kit registry
entries whose mappings name a `presentation.*` endpoint (all thirteen map onto
`presentation.write`). `tests/platform/test_registry_binding.py`'s
`presentation_api.codes.PRESENTATION_SERVED_PLATFORM_CODES`, the nine codes this surface mints
itself plus the seven platform codes its handlers, its `validation_problem` path, the
live-refresh check, or the platform default put on the wire (`not_found`, `request_invalid`,
`service_unavailable`, `timeout`, `internal_error`, `unsupported_schema_version`,
`refresh_not_permitted`), is exactly the set of active spec-kit registry entries whose mappings
name a `presentation.*` endpoint. `tests/platform/test_registry_binding.py`'s
`test_presentation_served_codes_match_registry` asserts this bijection and passes.
`PRESENTATION_SERVED_PLATFORM_CODES` is what carries `C12`'s `unsupported_schema_version` into
that comparison: it has no `CODE_RETRY_MODE` entry, since that table mirrors the baked catalogue
manifest's `retry_modes` byte for byte, so `CODE_RETRY_MODE`'s own key set alone would leave the
code out of the served set on both sides.

The surface binds `presentation.write` to create and revise, and `presentation.render` to C7b
and C7c (`Surface.registry_endpoints`). The component read shares revise's path, so it is bound
with it. The frames, component data, release, and meta routes map no registry endpoint and serve
the write routes' statuses. `tests/presentation_api/test_refusal_registry.py` asserts that every
`presentation.*` endpoint is bound and that every exemplar goes out at the status the registry
gives its code: a render door exemplar on `presentation.render`, every other on
`presentation.write`.
8 changes: 4 additions & 4 deletions packages/dfo-data-tools/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,10 +86,10 @@ graph-release = ["google-cloud-spanner>=3.57", "dfo-data-tools[coverage]"]
[tool.uv.sources]
# See the root pyproject.toml's dfo-spec-kit source comment for the
# pin-consistency invariant this rev must satisfy.
dfo-spec-kit-frame-grammar = { git = "https://github.com/DevelopmentFinanceObservatory/spec-kit.git", rev = "3616b64245c2db8af3187552e9e5de99b6d304d4", subdirectory = "python/frame-grammar" }
dfo-spec-kit-components = { git = "https://github.com/DevelopmentFinanceObservatory/spec-kit.git", rev = "3616b64245c2db8af3187552e9e5de99b6d304d4", subdirectory = "python/dfo-spec-kit-components" }
dfo-spec-kit-provenance-docs = { git = "https://github.com/DevelopmentFinanceObservatory/spec-kit.git", rev = "3616b64245c2db8af3187552e9e5de99b6d304d4", subdirectory = "python/provenance-docs" }
dfo-spec-kit-descriptions = { git = "https://github.com/DevelopmentFinanceObservatory/spec-kit.git", rev = "3616b64245c2db8af3187552e9e5de99b6d304d4", subdirectory = "python/descriptions" }
dfo-spec-kit-frame-grammar = { git = "https://github.com/DevelopmentFinanceObservatory/spec-kit.git", rev = "4f927c09040fc287f81fc5d81ce22eebe3caa2fd", subdirectory = "python/frame-grammar" }
dfo-spec-kit-components = { git = "https://github.com/DevelopmentFinanceObservatory/spec-kit.git", rev = "4f927c09040fc287f81fc5d81ce22eebe3caa2fd", subdirectory = "python/dfo-spec-kit-components" }
dfo-spec-kit-provenance-docs = { git = "https://github.com/DevelopmentFinanceObservatory/spec-kit.git", rev = "4f927c09040fc287f81fc5d81ce22eebe3caa2fd", subdirectory = "python/provenance-docs" }
dfo-spec-kit-descriptions = { git = "https://github.com/DevelopmentFinanceObservatory/spec-kit.git", rev = "4f927c09040fc287f81fc5d81ce22eebe3caa2fd", subdirectory = "python/descriptions" }

[build-system]
requires = ["uv_build>=0.11,<0.12"]
Expand Down
8 changes: 6 additions & 2 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -236,12 +236,16 @@ dfo-data-tools = { workspace = true }
# The registry refuses a part's `source_code` on search.resolve with
# `source_code_not_held`, `source_code_ambiguous`, and
# `source_code_shares_variable`.
# The pin is a commit of spec-kit's main. It registers
# The pin is spec-kit's main plus three commits that map binding_invalid,
# constraint_violation and resource_exhausted at 400, frame_unavailable at
# 404 and release_mismatch at 503 on presentation.write, and edit_conflict
# at 409 on presentation.render, and name every case frame_unavailable
# covers. Main registers
# slot_default_not_applied and the three source-code refusals, and retires
# starved_axis_binding_restored.
# Follow-up: move all five to a real spec-kit tag once one is cut, so a
# future force-push or GC on spec-kit's main can't silently break this lock.
dfo-spec-kit = { git = "https://github.com/DevelopmentFinanceObservatory/spec-kit.git", rev = "3616b64245c2db8af3187552e9e5de99b6d304d4", subdirectory = "python/spec-kit" }
dfo-spec-kit = { git = "https://github.com/DevelopmentFinanceObservatory/spec-kit.git", rev = "4f927c09040fc287f81fc5d81ce22eebe3caa2fd", subdirectory = "python/spec-kit" }

[tool.pytest.ini_options]
testpaths = ["tests"]
Expand Down
8 changes: 5 additions & 3 deletions src/presentation_api/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -398,20 +398,22 @@ The frame must be able to stand in for the frame the revision is stored on:

Any other frame is refused `render_target_mismatch` (409, not retryable). That covers another dataset or rendering, a dataset frame for a card on a derived frame and the reverse, a version or recipe that is not on record, and a `note_narration`, which binds no frame.

The render routes serve the assembler's refusals at the statuses the code registry gives `presentation.render`, which differ from the write routes' for five codes:
The render routes serve each refusal at the status the code registry gives its code on `presentation.render`, which for `binding_invalid`, `constraint_violation`, `frame_unavailable`, `release_mismatch`, and `resource_exhausted` differs from the write routes' status:

| Error Code | HTTP Status | When |
| :----------------------- | :---------- | :------------------------------------------------------------------------------------------- |
| `refresh_not_permitted` | 403 | The caller is not the live-refresh principal. |
| `render_target_mismatch` | 409 | The frame may not stand in for the stored one. |
| `not_found` | 404 | Unknown component or revision. On `/render/data`, also a `source_card`, which shows no data values. |
| `edit_conflict` | 409 | The component has no current revision. |
| `request_invalid` | 400 | A malformed id, body, `format`, or `bundle`. |
| `schema_invalid` | 400 | The stored intent fails this release's schema. |
| `binding_invalid` | 422 | The stored intent does not bind the frame: a stored binding with no value left on the frame, or a single or facet value the frame lacks while it holds several. |
| `constraint_violation` | 422 | The drawing fails a catalogue constraint on the frame's data. |
| `frame_unavailable` | 409 | The frame does not load or pin. `retryable` follows the cause, as on the write routes. |
| `release_mismatch` | 409 | The revision was minted under a newer catalogue release than this deployment's. |
| `resource_exhausted` | 409 | The drawing passes a resource maximum. |
| `service_unavailable` | 503 | A store, the frame store included, fails to answer, or the door is at capacity. |

On these routes `retryable` is true only for a refusal the same request may pass later, one whose code's retry mode is `retry_after_delay` or `refresh_then_retry` (`codes.replay_retryable`). The caller replays a stored intent and cannot change it, so `binding_invalid`, `constraint_violation`, `schema_invalid`, and `request_invalid` answer `retryable: false`.

Expand All @@ -433,7 +435,7 @@ The per-response `retryable` field is authoritative for every code. `retry_modes
| `binding_invalid` | 400 | `correct_and_retry` | Unbound selectors or invalid values. Returns `alternatives`, `truncated`, and `issues`. Each alternative pairs a legal value with the label the frame resolved for it, falling back to the value itself when the frame names none. |
| `constraint_violation` | 400 | `correct_and_retry` | Violates catalogue constraints. Returns `constraint_id`, `pointer`, and `detail`. `pointer` is expressed against the intent, for example `/scale`, `/range`, or `/change/basis`. |
| `edit_conflict` | 409 | `refresh_then_retry` | The component, a requested revision, or on create the referenced composition vanished or was rewritten mid flight, raised by a scoped or unscoped `GET` as well as by writes. |
| `frame_unavailable` | 404 / 503 | Contextual | `unknown` (404, not retryable) names a `spec_id`/`content_version` pair that was never stored. `expired` (404, retryable) names a stored version that aged out. `ops_unsupported` (404, retryable) names an `ops_`/`ocv_` pair analysis-mcp minted but never published to the platform; publishing it is the retry that succeeds. `revoked` and the other permanent reasons (`missing_rendering`, `digest_mismatch`) return 404, not retryable. `quarantined` (404, not retryable) names a `spec_id` the spec registry holds in quarantine. `io` returns 503, retry after delay. |
| `frame_unavailable` | 404 | Contextual | `unknown` (404, not retryable) names a `spec_id`/`content_version` pair that was never stored. `expired` (404, retryable) names a stored version that aged out. `ops_unsupported` (404, retryable) names an `ops_`/`ocv_` pair analysis-mcp minted but never published to the platform; publishing it is the retry that succeeds. `revoked` and the other permanent reasons (`missing_rendering`, `digest_mismatch`) return 404, not retryable. `quarantined` (404, not retryable) names a `spec_id` the spec registry holds in quarantine. A frame store that fails to answer (`io`) refuses `service_unavailable` instead. |
| `resource_exhausted` | 400 | `do_not_retry` | Selection exceeds allowed limits, including a minted component whose `component`, `provenance` and `description` subtrees serialise past `MAX_ENVELOPE_BYTES` (2 MiB, compact UTF-8 JSON), refused before any store write. Returns `limit_name` in `detail`. |
| `release_mismatch` | 503 | `retry_after_delay` | The component's head carries a newer catalogue release than this deployment writes, which happens after a rollback and clears once the newer release is deployed again, or a stored payload has a shape the release's schema does not recognise, such as a bar_chart row migration 0006 has not rewritten. |
| `request_invalid` | 400 | `correct_and_retry` | Retryable is always `false`. A malformed body field, an unparseable path identifier, or a non-positive `revision` query parameter. Also a create `write_token` reused with a different request. |
Expand All @@ -442,7 +444,7 @@ The per-response `retryable` field is authoritative for every code. `retry_modes
| `render_target_mismatch` | 409 | `do_not_retry` | A render route given a frame that may not stand in for the stored one. |
| `not_found` | 404 | `do_not_retry` | Unknown component or composition id (`DraftNotFound`), or a revision number never minted for a revision-scoped `GET`. |
| `write_refused` | 403 | `do_not_retry` | A revise without the component's write token, or a create into an existing composition without that composition's token, in `x-dfo-write-token`. Also a draft created before write tokens. `detail` ends with a fix: create a new component. |
| `service_unavailable` | 503 | `retry_after_delay` | Database failure or door thread pool backpressure exhaustion. |
| `service_unavailable` | 503 | `retry_after_delay` | Database failure, a frame store that fails to answer, or door thread pool backpressure exhaustion. |
| `timeout` | 504 | `retry_after_delay` | Execution exceeded door timeout threshold. |
| `internal_error` | 500 | `do_not_retry` | Permanent non-retryable store or system failure. |

Expand Down
Loading