Repository navigation
fix(objectql)!: floor a plugin driver on an external datasource — the engine applies the query, up to 1000 rows - #27403
Conversation
📓 Docs Drift CheckThis PR changes 2 package(s): 48 hand-written doc(s) name something this change touched — list omitted above 15 rows. Re-derive on the tree named below: ⛔ 12 release-owned page(s) also affected — read-only, see AGENTS.md Documentation Guardrails. What this run could not see
Coarse fallback — 168 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin ca6217aebd23054d8a52c062babf09289179eb56 && git checkout ca6217aebd23054d8a52c062babf09289179eb56
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin bbb49c9fe3ecfa56c843aac7a18d4ff95783047a 490c3c331a2132a8b60506d5b82127807822265d && git checkout -B drift-repro bbb49c9fe3ecfa56c843aac7a18d4ff95783047a && git merge --no-ff 490c3c331a2132a8b60506d5b82127807822265d
node scripts/docs-audit/affected-docs.mjs --json bbb49c9fe3ecfa56c843aac7a18d4ff95783047a
|
…ngine applies the query, up to 1000 rows A datasource whose schemaMode is not managed and whose driver id is not built in is served through a floored driver, stored once in the engine's drivers map. The driver is sent where, orderBy and limit 1001; the engine applies where, orderBy, offset, limit and fields itself (applyDriverQueryInMemory, over the having walker with $like/$ilike lifted into its where position). findOne and count read the same way, aggregate is hidden, writes pass through. More than 1000 rows is EXTERNAL_ROW_CAP_EXCEEDED (422); a remote 4xx on a write is EXTERNAL_WRITE_REFUSED (422); any other remote failure is ERR_DATASOURCE_UNAVAILABLE (503, reason remote). Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017ihL2dgsoHnkCiSyXF8isz
…EDED under @objectstack/objectql The two 422 refusals of the engine's floor over a plugin driver on an external datasource (ADR-0015 Tier A). Neither is an ADR-0015 gate refusal: EXTERNAL_WRITE_FORBIDDEN (403) stays the platform's own write switch. The closed ApiErrorSchema.code vocabulary gains both members; the generated references are re-recorded (gen:schema + gen:docs). The api surface is unchanged (check:api-surface). Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017ihL2dgsoHnkCiSyXF8isz
…hree-step driver path, the 1000-row cap, fail-closed owner columns The two "planned" sentences become the shipped behaviour (the floor; 503 ERR_DATASOURCE_UNAVAILABLE / 422 EXTERNAL_WRITE_REFUSED for driver failures). New passages: the engine re-applies every query (what the driver is sent, what the engine does), the driver's three steps, what the 1000-row cap means and what the floor does not cover, and that a missing owner column fails closed. The "what your driver receives" table is re-measured under the floor. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017ihL2dgsoHnkCiSyXF8isz
9f12119 to
55356b9
Compare
|
epic:#27261#1 · marchtian · session_017ihL2c5z3q4h8inrn5kwqV Seat review plus an independent full Fable review. Most of the PR is accepted as is:
Must fix
Accepted as is, with follow-ups the seat files
Same branch, same claim. The Fable review re-reads the delta after the push. |
…predicate writes; only request-shaped 4xx is a write refusal Rework round 1 on the plugin-driver floor: - registerDatasourceDef without `driver` keeps the stored driver id, so the datasource-connection service's driver-less re-registration (after a connect, restoreOldPool) no longer unwraps the floor. - updateMany / deleteMany are hidden on the floored instance, as aggregate is: a precondition write goes through update by id, and a multi-row predicate write is refused. - Only 400/404/409/412/422 from the remote is EXTERNAL_WRITE_REFUSED; a 401/403/407/429 is ERR_DATASOURCE_UNAVAILABLE (503) with its text withheld. - The in-memory sort compares a Date and an ISO string in one column as instants. - Docs and changeset follow. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017ihL2dgsoHnkCiSyXF8isz
…mber edit and delete a virtual object's rows
An explicit sharingModel is not a way out on its own ('private' is refused
the same way): it is public_read_write with a beforeUpdate hook, or a
declared owner field the driver returns under its name (columnMap only on
a built-in SQL driver). Member deletes stay refused on public_read_write
unless the driver returns created_by or a policy or share grants delete.
Matches the security-external-owner-unmapped warning (#27332).
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ihL2dgsoHnkCiSyXF8isz
…lugin-driver datasource A floored plugin driver offers no updateMany/deleteMany, so a multi:true predicate update or delete fell to the engine's generic "requires an ID or options.multi=true" — false for a caller who passed multi, and with no prescription. Both fallback arms now say the datasource is served by a plugin driver, why the engine does not run the write there, and to read the matching ids and write each by id. Every other driver keeps the generic text. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017ihL2dgsoHnkCiSyXF8isz
|
epic:#27261#1 · marchtian · session_017ihL2c5z3q4h8inrn5kwqV 维护者速读
席位意见:ACCEPT。两轮返工后,独立 Fable 完整复核 ACCEPT;第二轮的拒收措辞由席位亲读。 席位核对
|
|
epic:#27261#1 · marchtian · session_017ihL2c5z3q4h8inrn5kwqV Post-landing review owed at the landed head (seat miss recorded on #27261, correction 6501585877). Face: published schema ( |
|
epic:#27261#1 · marchtian · session_017ihL2c5z3q4h8inrn5kwqV Contract reviewServed-tier: Post-landing review at the landed head (merged 2026-10-29T07:38Z as ① Derived judgments
② Semver level
③ Boundary flags
④ Design judgment
Implemented-by: VERDICT: PASS |
Requested by 田騏 · project thread
Fixes #27277
Clause-②: yes (two codes join ERROR_CODE_LEDGER under
@objectstack/objectql:EXTERNAL_WRITE_REFUSED,EXTERNAL_ROW_CAP_EXCEEDED, both 422)Metadata-protocol: none
维护者速读
schemaMode非 managed,且driver不是内置 id)现在由引擎兜底。driver 只收到where、orderBy和limit: 1001,引擎自己再把整条查询(过滤、排序、分页、投影)作用在返回的行上。超过 1000 行 ⇒ 422EXTERNAL_ROW_CAP_EXCEEDED;远端对写入回 4xx ⇒ 422EXTERNAL_WRITE_REFUSED;其余远端失败 ⇒ 503ERR_DATASOURCE_UNAVAILABLE。DriverCapabilities没有任何下推声明(17.0.0 已退役),引擎无从得知插件 driver 执行了查询的哪一部分。一个忽略过滤的 driver 会静默返回错误的行。按裁决 Q1=B / Q2=A′ / Q3=A′ 落地。getDriverByName仍返回原实例)。受影响的只有插件 driver 外部对象:匹配超过 1000 行的读取从「静默返回」变成 422,变更集按 BREAKING 申报。回滚即 revert 两个提交。packages/objectql/src/external-driver-floor.ts(Proxy 包装、上限读取、写入错误映射),以及engine.ts的applyExternalDriverFloor(两个注册入口都调用它,drivers 映射里只放一个实例)。pnpm --filter @objectstack/objectql exec vitest run --project local src/external-driver-floor.test.ts src/driver-query-in-memory.conformance.test.ts,预期 123 passed。What changes
external-driver-floor.ts(new):needsExternalDriverFloor(def)is true only for a non-managed datasource whosedriveridresolveDriverIddoes not know.createExternalDriverFlooris idempotent per driver and returns a Proxy over it.find,findOneandcountsendwhere,orderByandlimit: 1001, neveroffsetorfields.ExternalRowCapExceededError.findOneis the same read withlimit: 1;countis its length afterwhere.aggregateis hidden (bothgetandhas), so the engine aggregates in memory.status, thenstatusCode) ⇒ExternalWriteRefusedError; any other failure ⇒DatasourceUnavailableErrorwith the new reasonremote, and the cause is logged at warn.driver-query-in-memory.ts(new):applyDriverQueryInMemory(rows, query, { fieldClasses?, jsonStored? })applies where, then orderBy, then offset/limit, then fields.idtie-break in the last key's direction, as driver-sql does.limitis read by presence, solimit: 0returns no rows.fieldskeepsid, and every row returned is a copy.groupBy,aggregations,having,search,searchFields,top) are refused by name.ReadableIdsleaf is refused (ADR-0139 D3).having-filter.ts: the walker gains awhereposition (matchesDriverWhere, refusal wording that nameswhere). It adds$likeand$ilikethrough the spec's ownlikePatternToRegExp, refusing a dangling escape, a NUL or a non-string pattern.havingand the per-aggregationfilterstill refuse both operators.external-driver-errors.ts(new): the two 422 classes, with status declared on the thrown error (thedriver-connect-errors.tspattern).driver-connect-errors.tsgains the per-request reasonremote.engine.ts:DatasourceDefgainsdriver, andregisterDatasourceDefstores it. The privateapplyExternalDriverFloor(name)runs from bothregisterDriverandregisterDatasourceDef, because either can come first. It replaces the map entry once, and unwraps it if a later definition no longer needs the floor. TheregisterDrivercollision check compares the unwrapped drivers.index.ts/core.ts: the new exports (listed in the changeset).@objectstack/objectqlinerror-code-ledger.zod.ts. The closedApiErrorSchema.codevocabulary gains both members, so the generated referencescontract.mdx,automation-api.mdxanderror-code-ledger.mdxare re-recorded withgen:schema+gen:docs.check:api-surfacereports the surface unchanged.shared/external-errors.tsis untouched, per the ruling.packages/runtime/src/plugin-driver-external-object.spike.test.ts: the P0 spike pinned the pre-floor calls, so it is re-pinned to what the remote now sees.findandcountboth arrive asfindwithlimit: 1001, andfindOneis answered from thatfind.getDriverByNamereturns the floored instance, whichunwrapExternalDriverFloormaps back to the driver.start_datetie on the paged list now breaks byid.Tests
driver-query-in-memory.conformance.test.ts: the spec tables are the oracle. It covers FILTER_LOGIC_CASES, FILTER_TEXT_CASES (every rejection row in the ADR-0112 envelope), PAGINATION_CASES (walked page by page), PAGINATION_UNORDERED_CASES and PAGINATION_ZERO_LIMIT_CASES. It also covers TEMPORAL_CASES, lowered throughlowerFilterCondition, with tokens resolved at TEMPORAL_NOW and rows in both writer forms. 106 passed.external-driver-floor.test.ts: runs through the real engine, with a driver that pushes nothing down. 17 passed. It covers:getDriverByName,datasource()andgetDriver(object)are the same object, and re-registration is idempotent);driver-where-like.test.ts:$likeand$ilikesemantics and refusals, andhavingstill refusing them. 6 passed.@objectstack/objectql: 482 files, 9071 tests.@objectstack/rest: 334 files.@objectstack/service-datasource: 58 files.@objectstack/runtime: 471 files, 7668 tests.typecheck(tsc pluscheck:test-typecheck) is green for objectql and runtime.Acceptance notes
updateManyanddeleteManypasswhereto the driver unchanged, as the ruling says (writes pass through). A plugin driver that ignoreswhereon these would write more rows than asked. Open item for the PM: floor them as read-ids-then-by-id writes, or refuse them on floored datasources.ReadableIdsleaf is refused. A controlled-by-parent external object, whose read scope composes aReadableIdsleaf, cannot be read through the floor: it is refused per ADR-0139 D3 rather than answered short.countandfindOneeach read up to 1001 rows from the remote, and a list page plus its total is two such reads. This is the price of correctness without a pushdown credential, which is the epic's P3.virtual-objects.mdxpassages (fail-closed owner/sharing column, the driver three-step progression, the 1000-row cap) and the two "planned" sentences become a follow-up after it lands.adr-0087: not-required (no-migration-prescription).Verification
Final head
9f12119993.node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstackderives 110 families for this diff. All 110 ran with their exit codes recorded before any pipe, and--ranreconciles110 derived, 110 run, 0 NOT-MEASURED, 0 UNRUN.check:docs: the generated references were stale; re-recorded in the spec commit.check:query-options-erasure: the new test'sas anyon engine query options grew the test surface from 236 to 244; the casts were removed, so it is back at 236.check-plugin-teardown-shape --self-testneeded its pinned fixture commit fetched, andcheck:dual-build-cjs-loadsneeded a builtdist. Both pass on re-run.!): run under the verify lock withVERDICT command-exit 0. Test Files 348 passed and 3 skipped of 351; Tests 2806 passed and 20 skipped. It ran one commit before the final head, which changed only the new test's casts and the generated reference docs.typecheckfor objectql and runtime, includingcheck:test-typecheck.Generated by Claude Code