Skip to content

feat(driver-odata): a declarative OData v4 datasource driver, published as a plugin driver id behind the engine floor - #27796

Merged
os-zhuang merged 8 commits into
mainfrom
claude/project-thread-wj01ct
Oct 30, 2026
Merged

os-zhuang merged 8 commits into
mainfrom
claude/project-thread-wj01ct

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Requested by 田騏 · project thread
Fixes #27741

Clause-②: no
Metadata-protocol: none

维护者速读

  • 改了什么: 新包 @objectstack/driver-odata:声明式 OData v4 数据源驱动。服务端插件 createODataDriverPlugin 把它发布成插件 driver(服务 driver.NAME,默认 driver.odata),管理员声明数据源与对象(external.remoteName 指实体集、external.columnMap 改字段名)即可实时读写 SAP / Dynamics 365 等 OData 服务,不写代码。
  • 为什么: epic [Decision] Ontology over external systems: virtual objects on ADR-0015 federation — two driver tiers (plugin code · declarative OpenAPI), pushdown/identity/catalog/cache rules, Studio and AI drafts (2026 ontology research + deep design) #27261 P2(裁决 6486145005,Tier B = 声明式 OData)。按您的指示只做 v4,代码里留了方言接缝,v2 以后另立卡加一个文件即可。
  • 怎么保证正确: 它是插件 driver id,没有加进 DRIVER_VOCABULARY,引擎的兜底层会在内存里重新应用整条查询。驱动只下推能翻译的部分,并且保证只会让服务端多返回、不会少返回($not、文本排序与否定、$like 等都留给引擎)。
  • 凭据: 只走数据源的 external.credentialsRef,经宿主的 secret store 在第一次请求时解析;缺引用、缺 store、解析不出都直接拒绝,不会无认证发请求。没有动 spec 的 schema、也没有加 DRIVER_VOCABULARY。
  • 两分钟验证: pnpm --filter @objectstack/driver-odata test,预期 6 个文件、425 条全部通过(其中 326 条是 11 组共享驱动一致性用例,在真实引擎兜底层之后、对一个会执行 $filter 的进程内 OData 服务、按大小写敏感与不敏感两种排序规则各跑一遍)。
  • 需要你做的: 本 PR 改动 3691 行,超过 3000 行门槛,按规则需要您在 GitHub 上 APPROVE 一次才能落地;其余由席位处理。
  • 额外动到的两处: packages/spec/llms.txt 的包数 72 改 73(新包让 check:llms-txt 变红,只改数字);新包加进 changeset 的 fixed 组。

What changes

A new package, packages/drivers/driver-odata (@objectstack/driver-odata, in the changeset fixed group at 18.2.0):

  • src/plugin.ts: createODataDriverPlugin(options) parses the config and registers an ODataDriver as the kernel service driver.NAME in init (driver.odata by default). This is a plugin driver id: it is not added to DRIVER_VOCABULARY (config-registry.zod.ts), so needsExternalDriverFloor holds it behind the engine floor on a schemaMode: 'external' datasource. No built-in id is argued for.
  • src/config.ts: the driver's own config contract, a strict zod schema parsed when the plugin is built: name, serviceRoot (http or https; refused when it carries userinfo or a query string), version (4.0 only), auth (none, bearer, or basic with username), pageSize, requestTimeoutMs. No credential is a config key.
  • src/odata-driver.ts: the IDataDriver.
    • Reads: find sends $filter, $orderby, $select, $top, $skip and follows @odata.nextLink until the asked number of rows; a paged read appends the entity key to $orderby so pages partition the result. count sends $count=true&$top=0 only when the whole where was translated, and refuses (status 400) otherwise. findOne is find with limit: 1.
    • Writes: POST, PATCH and DELETE by key, If-Match with the @odata.etag of the row's last read (bounded cache), only properties the entity type declares sent under their remote names (the engine's id, updated_at and other stamps are dropped). PATCH/DELETE answering 404 return null / false, the contract's not-found. upsert, the bulk verbs, transactions, execute and dropTable are refused by name with status 400.
    • Rows: annotations removed, properties renamed through external.columnMap, id set to the key (the key predicate body for a composite key). registerExternalObject records remoteName and columnMap; the engine's binding pass drives it.
    • A ReadableIds leaf (ADR-0139 D3, a read-scope semi-join) in where is refused by the driver itself with readableIdsDriverRefusal before any request: $filter cannot spell it, and the translator's usual answer to an unspellable predicate (leave it out) would read a read scope as no restriction.
    • introspectSchema: entity sets as tables, EDM type names as column types, the key as the primary key, and single-valued navigation properties (with their bound entity set) as lookup candidates for the $metadata import card.
    • supports is { transactionsUnsupported: true } and makes no pushdown claim, because DriverCapabilities has none to make.
  • src/filter.ts: the where to $filter translator. Every translation selects a superset of the engine's reading. $and drops an untranslatable member; $or with one is dropped whole; $not is never pushed down; $ne / $nin on a nullable property carry or P eq null; text gets only eq, in, contains, startswith, endswith and ASCII $icontains; a comparand the property's EDM type cannot spell is left out. The result says whether the translation was complete.
  • src/transport.ts: HTTP with the auth header, a per-request timeout, no redirect following, and same-origin nextLink / Location only (a cross-origin link is refused so the credential is never sent elsewhere). A refused response throws ODataRequestError with the HTTP status the floor reads and the service's own error.message; no message carries the URL or a header. A 401 resolves the credential once more and retries once.
  • src/dialect.ts, src/dialect-v4.ts: the dialect seam. Request headers, the collection and entity envelope, the ETag, the error body, the $metadata reader (CSDL 4.x with inheritance and namespace aliases resolved; EDMX 1.0 refused by name), literal spellings, string functions, set membership (an or chain, which 4.0 services answer) and the count option all go through one ODataDialect. Only odataV4Dialect ships; a v2 dialect is a new file.
  • packages/spec/llms.txt: the package table's count, 72 to 73 (check:llms-txt reds on a new published package; only the two numbers change).
  • .changeset/config.json: the package joins the fixed group. pnpm-lock.yaml: the new importer (fast-xml-parser 5.10.1 was already in the lockfile).
  • .changeset/27741-driver-odata.md: @objectstack/driver-odata and @objectstack/spec (for llms.txt) patch.

Credentials

The secret is the datasource's external.credentialsRef. The plugin reads the declared definition from the engine (listDatasourceDefs()) and dereferences the handle through the host's datasource secret store (options.secrets, the read half of the SecretBinder the host already gives DatasourceAdminServicePlugin). It does this at the first request, not at connect(), so it never reads the definition registry while the boot is still filling it. It fails closed on: a scheme that needs a credential with no credentialsRef, no secret store, a handle that does not resolve, an undeclared datasource, and a credentialsRef declared under scheme none. Each refuses the request before anything is sent; the floor answers 503 and logs the cause.

Tests

Six files, 425 cases. The first five run against a recorded TripPin-shaped service (src/__fixtures__/trippin-metadata.xml and recorded-service.ts: each test names the exact requests and answers them from recordings; an unrecorded request fails with the URL spelled out):

  • filter.test.ts: the pushdown translation table, 54 rows (what translates, what widens, the complete flag).
  • dialect-v4.test.ts: the CSDL reader (sets, keys, types, collections, inheritance, aliases, navigation and constraints, v2 refusal) and the envelope.
  • odata-driver.test.ts: reads, paging and nextLink, key ordering, $select/$skip, cross-origin refusal, $count, refused reads, writes with If-Match, 204 and 404 handling, refused verbs, composite keys, introspection, bearer and 401 re-resolution.
  • odata-driver.floor.test.ts: the driver behind the real engine (ObjectQL, objects bound through engine.syncSchemas()): an untranslatable predicate ($like) is applied in memory over the wider answer; count reads through the floor; a 400 read is ExternalReadRefusedError INVALID_QUERY / 400 with the service's reason; a 400 write is ExternalWriteRefusedError EXTERNAL_WRITE_REFUSED / 422; a 503 is DatasourceUnavailableError with the text withheld.
  • plugin.test.ts: the service name, config refusals, bearer and basic from the resolved credentialsRef, and the five fail-closed cases.
  • odata-driver.conformance.test.ts: all eleven shared driver case-sets (check:driver-conformance: 66 covered cells, 0 DEBT, 0 EXEMPT), 326 cases, through the real engine and its floor over an in-process OData v4 service (__fixtures__/odata-service.ts) that evaluates the $filter the driver writes and answers 400 to anything outside the grammar it reads. The read case-sets run twice, over an ordinal and a case-insensitive service, so the "never narrower" rule is checked against a collation that widens. FILTER_LOGIC, FILTER_COMPARAND_TYPE, TEMPORAL and TEMPORAL_TIME go through engine.find; PAGINATION (with server-driven paging at 5 rows) through paged engine.find walks; AGGREGATION through engine.aggregate (the floor hides aggregate); FILTER_TEXT through the floored driver, beneath the engine's declared-type door, which would refuse the case-set's text-operator-on-a-number rows before any driver is asked; READABLE_IDS is refused in the READABLE_IDS_REFUSAL envelope by the driver and by the floor, with nothing sent (the one leaf-free case, the engine's $in: [] denial, is answered); VALUE_ROUNDTRIP through the driver's own create then find.

The vitest config aliases @objectstack/core, objectql, spec and types to source.

Ablation (one-shot, not kept)

In src/filter.ts, the nullable $ne arm was replaced by the bare ne (anchor count 1, disjunction count 1 to 0). vitest run src/filter.test.ts: 1 failed, 53 passed; the failure was $ne on a nullable property. Restored with git checkout HEAD; the blob equals HEAD and git diff HEAD is empty. Tests import the source, so no dist rebuild was involved.

Acceptance notes

  • The driver id and the datasource name. The engine routes a datasource to a driver by driver.name, so one plugin instance serves one datasource, named by the plugin's name; the README follows the virtual-objects guide and keeps the datasource name, its driver and the service suffix equal. Two OData services are two plugin instances.
  • No connect-time check. connect() makes no request: the credential needs the datasource definition, which may not be registered yet at that point. $metadata is read at the first request and cached; a failed read is retried by the next one. checkHealth() asks for the service document.
  • No $batch. The bulk verbs are refused rather than looped, since a loop would claim an atomicity the transport does not give. A $batch arm is a follow-up, not filed.
  • Collation. Text equality and contains pushed down to a case- or accent-insensitive service only widen the answer, and the floor narrows it back. Text ordering and negation are not pushed down for that reason.
  • Size. 3691 changed lines (a new package with its fixtures and a 326-case conformance suite), over the 3000-line threshold, so the PR lands only with an authorized APPROVE. The LICENSE copy is 201 of those lines.
  • Paged reads under the floor carry $orderby on the key. The floor always sends a limit, so the service is asked for a key-ordered read. This is what makes find's paging contract hold when the driver is called directly; under the floor it costs nothing.

Verification

Gates were derived with node scripts/pm/dispatch-gates.mjs --commands (no paths) on the merged head bcc0a1a and reconciled with --ran: "77 derived famil(ies) accounted for — 77 run, 0 NOT-MEASURED (a DERIVED zero — all 77 recorded an exit code and none of them is 3)". All 77 exited 0. On the first pass (head 02401a7) two were red and are fixed here: check:llms-txt (the package count) and check:driver-conformance (a new driver must run every shared case-set). pnpm --filter @objectstack/driver-odata test: 6 files, 425 tests passed; typecheck exit 0; eslint packages/drivers/driver-odata exit 0. The built CJS and ESM entries load (require and import of dist/).

CI check-runs on this PR carry every family CI runs; they are cited in the os-dev report, not re-run locally.


Generated by Claude Code

@objectstack-fleet objectstack-fleet Bot added the enhancement New feature or request label Oct 29, 2026
@github-actions github-actions Bot added size/xl dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation tests tooling labels Oct 29, 2026
@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 2 package(s): @objectstack/driver-odata, @objectstack/spec, touching 128 documentable anchor(s). ⚠️ 7 changed file(s) yielded no anchor (packages/drivers/driver-odata/LICENSE, packages/drivers/driver-odata/README.md, packages/drivers/driver-odata/package.json, …), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files.

49 hand-written doc(s) name something this change touched — list omitted above 15 rows. Re-derive on the tree named below: node scripts/docs-audit/affected-docs.mjs --json 3cc4563e1e0cead21e76d3b09da732499e771ca7.

⛔ 12 release-owned page(s) also affected — read-only, see AGENTS.md Documentation Guardrails.

What this run could not see
  • 7 changed file(s) yielded no anchor (packages/drivers/driver-odata/LICENSE, packages/drivers/driver-odata/README.md, packages/drivers/driver-odata/package.json, …) — pages documenting those are invisible to this run
  • 1 cross-cutting symbol(s) contributed no route anchor: isArray (5 routes)
  • 57 name(s) were too generic to anchor anything (single lowercase words)
  • the SDK route bridge reached 58 of 210 client-bound route-ledger rows — the other 152 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 152: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 97 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
  • a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 167 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 3cc4563e1e0cead21e76d3b09da732499e771ca7 → packageMentionDocs.

Which tree this was computed on

This run read content/docs from 5958e95120f576f401e358cc206d59d9528e7911 — the merge of head 7f91132b87090f7c0bd8902f0fe17808ca63a6e0 into base 3cc4563e1e0cead21e76d3b09da732499e771ca7, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 5958e95120f576f401e358cc206d59d9528e7911 && git checkout 5958e95120f576f401e358cc206d59d9528e7911
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 3cc4563e1e0cead21e76d3b09da732499e771ca7 7f91132b87090f7c0bd8902f0fe17808ca63a6e0 && git checkout -B drift-repro 3cc4563e1e0cead21e76d3b09da732499e771ca7 && git merge --no-ff 7f91132b87090f7c0bd8902f0fe17808ca63a6e0

node scripts/docs-audit/affected-docs.mjs --json 3cc4563e1e0cead21e76d3b09da732499e771ca7

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs 3cc4563e1e0cead21e76d3b09da732499e771ca7 → pass the list as
args.docs, on the commit named under Which tree this was computed on.

`not` binds tighter than `eq` and `or` in OData, so `not P eq 1` reads as
`(not P) eq 1`, which a service refuses, and the nullable form's
`or P eq null` fell under the `not` and dropped the null rows. The
translator now writes `not (list)`, the v4 dialect's in-list is a bare
or-chain that the translator groups wherever it is an operand, and the
fixture service parses `not` at OData's precedence and refuses a
non-Boolean operand, so a number negation pushed down is exercised end
to end.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ihL2gRrVctbMWaHbABDch
@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

epic:#27261#1 · marchtian · session_017ihL2c5z3q4h8inrn5kwqV
ACCEPT · PR #27796 @ 7f91132b87090f7c0bd8902f0fe17808ca63a6e0 · 2026-10-30T00:42Z

维护者速读

  • 改了什么: 新包 @objectstack/driver-odata:声明式 OData v4 数据源驱动,以插件驱动 id driver.odata 发布。管理员声明数据源和对象(external.remoteName 指实体集,external.columnMap 改字段名),就能实时读写 SAP、Dynamics 365 这类 OData 服务,不用写代码。
  • 为什么改: epic [Decision] Ontology over external systems: virtual objects on ADR-0015 federation — two driver tiers (plugin code · declarative OpenAPI), pushdown/identity/catalog/cache rules, Studio and AI drafts (2026 ontology research + deep design) #27261 P2(裁决 6486145005,Tier B = 声明式 OData)。按您的指示只做 v4,方言接缝已留好,v2 以后另立卡。
  • 风险与回滚: 全是新包,不动 spec schema,也不进 DRIVER_VOCABULARY,引擎兜底层会在内存里重放整条查询,驱动只下推"只会多、不会少"的过滤。凭据只走 external.credentialsRef,解析失败一律拒绝。另外两处小改:packages/spec/llms.txt 包数 72→73,changeset fixed 组加入新包。还原即回滚。
  • 先看: packages/drivers/driver-odata/src/filter.ts(下推规则),transport.ts(认证、同源链接、超时)。
  • 两分钟验证: pnpm --filter @objectstack/driver-odata test,预期 439 条全部通过。
  • 需要你做的: APPROVE 一次(改动 3748 行,超过 3000 行门槛)。本 PR 不碰契约面(Clause-②: no,无 packages/spec/src/**),席位审读即审核记录;head 7f91132b87090f7c0bd8902f0fe17808ca63a6e0。

席位意见:ACCEPT。席位在 bcc0a1a3 审出一个 must-fix,已在当前 head 修复并复核。

席位核对

  1. Shape: +3746/−2, 26 files. A new package under packages/drivers/driver-odata (driver, v4 dialect behind an ODataDialect seam, $filter translator, transport, plugin, fixtures and tests), its changeset and fixed-group entry, the llms.txt package count, and the lockfile.
  2. Premise measured / seat review: read filter.ts, dialect-v4.ts (literals, quoting, inList), transport.ts and plugin.ts in full, and odata-driver.ts's read path and URL building.
    • Holds: every translation is a superset of the engine's reading. $not is never pushed. Text ordering and negation are left to the engine. The nullable $ne/$nin forms carry or P eq null.
    • Holds: @odata.nextLink is followed only on the service root's origin, and redirect: 'manual' is set, so the credential never leaves the configured host.
    • Holds: the floor's limit: EXTERNAL_ROW_CAP + 1 bounds the nextLink loop. String literals double ', and every URL segment and query value is encodeURIComponent-ed.
    • Holds: the credential is resolved at the first request, never at boot. A missing ref, store or secret fails closed, and a credentialsRef under auth: none is refused.
    • Must-fix found at bcc0a1a3, fixed at 7f91132b: single-value $nin emitted not Age eq 1. not binds tighter than eq, so a service refuses it, and on a nullable property the or Age eq null fell under the not and dropped null rows. Fix: not (LIST) always. New exact-string rows (nullable, non-nullable key, boolean). The fixture service now parses not at OData precedence. New conformance cases under both collations were reproduced red on the old line and are green on the fix.
  3. Gate check-runs on 7f91132b: 65 runs: 58 success, 7 skipped, 0 failed or pending. mergeable_state: clean. Local: 439/439, typecheck 0, eslint 0; 49 derived gates, all exit 0.
  4. Accepted deviations: packages/spec/llms.txt (a count line, outside packages/spec/src/**), required by check:llms-txt for any new published package. The PR body's 3691-line figure predates the fix; it is 3748 now.
  5. Out of scope, routed: there is no $batch arm, and bulk verbs are refused by name (a later card if a customer needs it). D7 connector-credential wiring and running objectql: publish a driver conformance kit that out-of-repo plugin drivers run in their own CI, so a contract change here is caught there — epic #27261 P2 #27744's conformance kit from this package go to whichever of service-datasource: a datasource driver resolves a borrowed connector credential through ADR-0143 D4's per-dispatch resolver (D7 runtime seam) — epic #27261 P2 #27746 / feat(objectql): a conformance kit a plugin driver outside this repository runs in its own CI #27798 lands after this PR. $metadata import (datasource import: generate object drafts from an OData $metadata document, and the Studio datasource page lists and imports its entity sets — epic #27261 P2 #27745) and the guide (docs: the external-systems guide gains the OData datasource section and the three-way choice (declarative OData, plugin driver, DataSync mirror) — epic #27261 P2 #27747) are dispatched after this lands. Skills-docs: docs: the external-systems guide gains the OData datasource section and the three-way choice (declarative OData, plugin driver, DataSync mirror) — epic #27261 P2 #27747.
  6. Landing: size-governed (check-governed-merges --pr 27796: 3748 lines > 3000; no governed path). The PR stays a draft with needs-user-decision, and review is requested from GOVERNED_APPROVERS. After an APPROVED review, the seat clears the label, then pr_ready and automerge_enable through the relay.

@os-zhuang
os-zhuang marked this pull request as ready for review October 30, 2026 01:06
@os-zhuang
os-zhuang enabled auto-merge October 30, 2026 01:07
@os-zhuang
os-zhuang added this pull request to the merge queue Oct 30, 2026
@os-zhuang
os-zhuang removed this pull request from the merge queue due to a manual request Oct 30, 2026
@os-zhuang
os-zhuang added this pull request to the merge queue Oct 30, 2026
Merged via the queue into main with commit 0a4b0fb Oct 30, 2026
74 checks passed
@os-zhuang
os-zhuang deleted the claude/project-thread-wj01ct branch October 30, 2026 01:32
This was referenced Oct 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation enhancement New feature or request size/xl tests tooling

Projects

None yet

3 participants