diff --git a/.changeset/27741-driver-odata.md b/.changeset/27741-driver-odata.md new file mode 100644 index 00000000000..3a9fe9cbb42 --- /dev/null +++ b/.changeset/27741-driver-odata.md @@ -0,0 +1,21 @@ +--- +'@objectstack/driver-odata': patch +'@objectstack/spec': patch +--- + +feat(driver-odata): a new package, `@objectstack/driver-odata`, serves an OData v4 service's entity sets as external objects through a plugin driver, with no code per entity set + +Clause-②: no +Metadata-protocol: none + +A server plugin, `createODataDriverPlugin({ name, serviceRoot, auth, pageSize, requestTimeoutMs, secrets })`, publishes an `ODataDriver` as the kernel service `driver.NAME` (`driver.odata` by default). The datasource it serves is declared as any plugin-driver datasource is (its `name` and `driver` both the plugin's `name`, `schemaMode: 'external'`), and each object binds an entity set through `external.remoteName`, renaming properties through `external.columnMap`. + +- **A plugin driver id, not a built-in one.** The id is not added to the spec's driver vocabulary, so the engine floors the driver: what the driver does not narrow, the engine applies in memory under its 1000-row cap. `supports` makes no pushdown claim. +- **Reads.** `find` sends `$filter`, `$orderby`, `$select`, `$top` and `$skip` and follows `@odata.nextLink` (same origin only). `$filter` carries comparisons, `$in`, `$between`, `$null`, `contains` / `startswith` / `endswith` / case-folded `contains`, under `and` / `or`; anything else (`$not`, `$like`, text ordering and negation, field references, collection properties) is left to the engine, so the service is only ever asked a wider question. A paged read is ordered by the entity key last. `count` sends `$count=true` when the whole `where` was translated and refuses otherwise. +- **Writes**, under the datasource's `external.allowWrites` and the object's `external.writable`: `POST`, `PATCH` and `DELETE` by key, with `If-Match` carrying the `@odata.etag` of the row's last read. Only properties the entity type declares are sent. A refused write reaches the caller as `422 EXTERNAL_WRITE_REFUSED` with the service's message; `upsert`, the bulk verbs and transactions are refused by name. +- **Keys and schema.** The key comes from `$metadata` (CSDL 4.0, inheritance and namespace aliases resolved); a row's `id` is the key's value, or the key predicate body for a composite key. `introspectSchema` lists entity sets, EDM-typed properties, keys and single-valued navigation properties as lookup candidates. +- **Credentials.** The secret is the datasource's `external.credentialsRef`, dereferenced through the host's datasource secret store (`options.secrets`, the same binder the datasource admin plugin takes) at the first request, and again once after a 401. `bearer` and `basic` are supported; a missing reference, store or secret refuses the request rather than sending it unauthenticated. The config itself refuses a `serviceRoot` that carries credentials. +- **Read scopes.** A `ReadableIds` leaf (a read-scope semi-join) is refused by the driver itself, in the shared refusal envelope, before anything is sent: `$filter` cannot spell it, and leaving it out would read it as no restriction. +- **Conformance.** The package runs all eleven shared driver case-sets, behind the real engine floor, over an in-process OData service that evaluates the `$filter` the driver writes, under an ordinal and a case-insensitive collation. +- **`@objectstack/spec`:** `llms.txt` counts the new package (73 published). +- **The dialect seam.** Headers, the response envelope, the `$metadata` reader, literal and function spellings and the count option go through one `ODataDialect`; only `odataV4Dialect` ships. A v2 dialect is a new file implementing it. diff --git a/.changeset/config.json b/.changeset/config.json index 1d5187cc93b..6a18140829c 100644 --- a/.changeset/config.json +++ b/.changeset/config.json @@ -36,6 +36,7 @@ "@objectstack/driver-mongodb", "@objectstack/driver-sqlite-wasm", "@objectstack/driver-turso", + "@objectstack/driver-odata", "@objectstack/plugin-approvals", "@objectstack/plugin-audit", "@objectstack/plugin-auth", diff --git a/packages/drivers/driver-odata/LICENSE b/packages/drivers/driver-odata/LICENSE new file mode 100644 index 00000000000..16bc23f4043 --- /dev/null +++ b/packages/drivers/driver-odata/LICENSE @@ -0,0 +1,202 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute + must include a readable copy of the attribution notices + contained within such NOTICE file, excluding those notices + that do not pertain to any part of the Derivative Works, + in at least one of the following places: within a NOTICE + text file distributed as part of the Derivative Works; within + the Source form or documentation, if provided along with + the Derivative Works; or, within a display generated by the + Derivative Works, if and wherever such third-party notices + normally appear. The contents of the NOTICE file are for + informational purposes only and do not modify the License. + You may add Your own attribution notices within Derivative + Works that You distribute, alongside or as an addendum to + the NOTICE text from the Work, provided that such additional + attribution notices cannot be construed as modifying the + License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright 2026 ObjectStack + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/packages/drivers/driver-odata/README.md b/packages/drivers/driver-odata/README.md new file mode 100644 index 00000000000..ffaa362984a --- /dev/null +++ b/packages/drivers/driver-odata/README.md @@ -0,0 +1,84 @@ +# @objectstack/driver-odata + +An OData v4 datasource driver for ObjectStack. Declare a datasource over an OData +service (SAP S/4HANA, Dynamics 365, Dataverse, any CSDL 4.0 service), bind objects to +its entity sets, and every view, form, permission and hook works on them, live. No row +is stored on the platform side, and no code is written per entity set. + +The driver is a **plugin driver**: a server plugin publishes it as the service +`driver.`. Its id is not a built-in one, so the engine holds it behind its +external-driver floor and re-applies every query to the rows the service returns. The +driver pushes down what it can translate and only ever asks the service a wider +question; see [Virtual objects](https://objectstack.ai/docs/data-modeling/virtual-objects). + +## Usage + +```typescript +import { defineStack } from '@objectstack/spec'; +import { createODataDriverPlugin } from '@objectstack/driver-odata'; + +export const plugins = [ + createODataDriverPlugin({ + name: 'erp', // the datasource it serves + serviceRoot: 'https://erp.example.com/odata/v4/sales/', + auth: { scheme: 'bearer' }, // or { scheme: 'basic', username: 'svc_reader' } + secrets: datasourceSecretBinder, // the host's datasource secret store + }), +]; + +export default defineStack({ + manifest: { id: 'com.example.sales', name: 'Sales', version: '1.0.0', type: 'app' }, + datasources: [{ + name: 'erp', + driver: 'erp', + schemaMode: 'external', + config: {}, + external: { + allowWrites: true, + credentialsRef: 'sys_secret:sec_erp', // the bearer token / basic password + validation: { onMismatch: 'warn', checkOnBoot: false }, + }, + }], + objects: [{ + name: 'sales_order', + label: 'Sales order', + datasource: 'erp', + sharingModel: 'public_read_write', + external: { + remoteName: 'SalesOrders', // the entity set + writable: true, + columnMap: { SalesOrderID: 'order_number', CustomerName: 'customer_name', NetAmount: 'net_amount' }, + }, + fields: { + order_number: { type: 'text', label: 'Order' }, + customer_name: { type: 'text', label: 'Customer' }, + net_amount: { type: 'number', label: 'Net amount' }, + }, + }], +}); +``` + +## Config + +| Key | Default | Meaning | +| :--- | :--- | :--- | +| `name` | `odata` | The datasource name, the driver name and its `driver.` service suffix. | +| `serviceRoot` | — | The service root URL (http or https, no credentials, no query). | +| `version` | `4.0` | The protocol version. Only the v4 dialect ships. | +| `auth` | `{ scheme: 'none' }` | `none`, `bearer`, or `basic` with a `username`. The secret is the datasource's `external.credentialsRef`. | +| `pageSize` | `500` | Rows asked for per server page (`Prefer: odata.maxpagesize`). | +| `requestTimeoutMs` | `30000` | Per-request timeout. | + +## What is pushed down + +`$filter`: `$eq`, `$ne`, `$gt`, `$gte`, `$lt`, `$lte`, `$between`, `$in`, `$nin`, `$null`, +`$contains`, `$startsWith`, `$endsWith` and `$icontains` (ASCII comparands), under `$and` +and `$or`. Text ordering and negation, `$not`, `$like`, `$exists`, field references and +collection-valued properties are left to the engine. Also `$orderby`, `$select`, `$top`, +`$skip`, `@odata.nextLink` paging and `$count=true`. + +## Writes + +`create`, `update` and `delete` by key, with `If-Match` from the row's last read. A +refusal by the service answers `422 EXTERNAL_WRITE_REFUSED` with its message. `upsert`, +the bulk verbs and transactions are refused by name. diff --git a/packages/drivers/driver-odata/package.json b/packages/drivers/driver-odata/package.json new file mode 100644 index 00000000000..43d0657adcc --- /dev/null +++ b/packages/drivers/driver-odata/package.json @@ -0,0 +1,68 @@ +{ + "name": "@objectstack/driver-odata", + "version": "18.2.0", + "license": "Apache-2.0", + "description": "OData v4 datasource driver for ObjectStack — serve an OData service's entity sets as external objects through a plugin driver", + "keywords": [ + "objectstack", + "driver", + "odata", + "external-datasource", + "virtual-objects", + "sap", + "dynamics-365" + ], + "type": "commonjs", + "main": "dist/index.js", + "types": "dist/index.d.ts", + "sideEffects": false, + "exports": { + ".": { + "import": { + "types": "./dist/index.d.mts", + "default": "./dist/index.mjs" + }, + "require": { + "types": "./dist/index.d.ts", + "default": "./dist/index.js" + } + } + }, + "scripts": { + "build": "tsup --config ../../../tsup.config.ts && node ../../../scripts/check-dts-emitted.mjs", + "dev": "tsc -w", + "test": "vitest run", + "typecheck": "tsc --noEmit" + }, + "dependencies": { + "@objectstack/core": "workspace:*", + "@objectstack/spec": "workspace:*", + "fast-xml-parser": "^5.10.1", + "zod": "^4.6.6" + }, + "devDependencies": { + "@objectstack/objectql": "workspace:*", + "@types/node": "^26.6.5", + "typescript": "^6.0.3", + "vitest": "^4.1.11" + }, + "author": "ObjectStack", + "repository": { + "type": "git", + "url": "git+https://github.com/objectstack-ai/objectstack.git", + "directory": "packages/drivers/driver-odata" + }, + "homepage": "https://objectstack.ai/docs", + "bugs": "https://github.com/objectstack-ai/objectstack/issues", + "publishConfig": { + "access": "public" + }, + "files": [ + "dist", + "README.md", + "CHANGELOG.md" + ], + "engines": { + "node": ">=22.0.0" + } +} diff --git a/packages/drivers/driver-odata/src/__fixtures__/odata-service.ts b/packages/drivers/driver-odata/src/__fixtures__/odata-service.ts new file mode 100644 index 00000000000..08d0171043e --- /dev/null +++ b/packages/drivers/driver-odata/src/__fixtures__/odata-service.ts @@ -0,0 +1,373 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * A small in-process OData v4 service for the conformance suite: one entity + * set over rows held in memory, answering `$metadata`, collection reads with + * `$filter` / `$orderby` / `$skip` / `$top` / `$count`, server-driven paging + * through `@odata.nextLink`, and `POST` / `PATCH` / `DELETE` by key. + * + * Its `$filter` evaluator reads the subset of the grammar the driver's + * translator writes (comparisons, `and` / `or` / `not`, parentheses, + * `contains` / `startswith` / `endswith` / `tolower`, `true` / `false` / + * `null`, quoted strings, numbers, dates, date-times and times of day). An + * expression outside that subset is answered `400`, the way a service answers + * a filter it cannot parse, so a translation change that writes something new + * fails loudly here. + * + * Two collations, because a real service's collation is its own: `ordinal` + * compares text by code unit, `case-insensitive` folds case first (as a SQL + * Server-backed service does). The engine floor must answer the same rows + * over either, which is what makes the driver's "never narrower" rule + * checkable. + */ + +import type { FetchLike } from '../transport.js'; + +export type EdmType = 'Edm.String' | 'Edm.Int32' | 'Edm.Double' | 'Edm.Boolean' | 'Edm.Date' | 'Edm.DateTimeOffset' | 'Edm.TimeOfDay'; + +export interface ServiceProperty { + name: string; + type: EdmType; + nullable?: boolean; +} + +export interface ODataServiceOptions { + entitySet: string; + /** The key is the first property. */ + properties: ServiceProperty[]; + rows: Array>; + collation?: 'ordinal' | 'case-insensitive'; + /** Rows per server page; a longer answer carries `@odata.nextLink`. */ + serverPageSize?: number; +} + +export const CONFORMANCE_ROOT = 'https://odata.conformance.test/svc/'; + +const NS = 'Conformance'; + +export function metadataFor(entitySet: string, properties: readonly ServiceProperty[]): string { + const props = properties + .map((p, i) => ``) + .join(''); + return ` + + +${props} + +`; +} + +// --------------------------------------------------------------------------- +// $filter: tokens, parse, evaluate +// --------------------------------------------------------------------------- + +type Token = + | { kind: 'punct'; value: '(' | ')' | ',' } + | { kind: 'string'; value: string } + | { kind: 'word'; value: string }; + +class FilterSyntaxError extends Error {} + +function tokenize(text: string): Token[] { + const tokens: Token[] = []; + let i = 0; + while (i < text.length) { + const ch = text[i]!; + if (ch === ' ') { i++; continue; } + if (ch === '(' || ch === ')' || ch === ',') { tokens.push({ kind: 'punct', value: ch }); i++; continue; } + if (ch === "'") { + let value = ''; + i++; + for (;;) { + if (i >= text.length) throw new FilterSyntaxError('unterminated string literal'); + if (text[i] === "'" && text[i + 1] === "'") { value += "'"; i += 2; continue; } + if (text[i] === "'") { i++; break; } + value += text[i++]; + } + tokens.push({ kind: 'string', value }); + continue; + } + const match = /^[A-Za-z0-9_.:+\-]+/.exec(text.slice(i)); + if (!match) throw new FilterSyntaxError(`unexpected character '${ch}'`); + tokens.push({ kind: 'word', value: match[0] }); + i += match[0].length; + } + return tokens; +} + +type Node = + | { kind: 'and' | 'or'; left: Node; right: Node } + | { kind: 'not'; operand: Node } + | { kind: 'compare'; op: string; left: Node; right: Node } + | { kind: 'call'; fn: string; args: Node[] } + | { kind: 'property'; name: string } + | { kind: 'literal'; text: string; quoted: boolean }; + +const COMPARISON = new Set(['eq', 'ne', 'gt', 'ge', 'lt', 'le']); +const FUNCTIONS = new Set(['contains', 'startswith', 'endswith', 'tolower']); + +function parse(text: string, types: ReadonlyMap): Node { + const tokens = tokenize(text); + let pos = 0; + const peek = () => tokens[pos]; + const isWord = (value: string) => peek()?.kind === 'word' && peek()!.value === value; + const expect = (value: string) => { + const t = tokens[pos++]; + if (!t || t.value !== value) throw new FilterSyntaxError(`expected '${value}'`); + }; + + const orExpr = (): Node => { + let left = andExpr(); + while (isWord('or')) { pos++; left = { kind: 'or', left, right: andExpr() }; } + return left; + }; + const andExpr = (): Node => { + let left = unary(); + while (isWord('and')) { pos++; left = { kind: 'and', left, right: unary() }; } + return left; + }; + // OData's precedence: `not` is unary and binds tighter than the comparisons, so + // `not P eq 1` is `(not P) eq 1`. Its operand must be Boolean, as a real service's + // parser requires; a non-Boolean one is refused (400), never read as `not (P eq 1)`. + const negation = (): Node => { + if (!isWord('not')) return operand(); + pos++; + const inner = negation(); + if (!isBoolean(inner)) throw new FilterSyntaxError("the operand of 'not' is not Boolean"); + return { kind: 'not', operand: inner }; + }; + const isBoolean = (node: Node): boolean => { + switch (node.kind) { + case 'and': case 'or': case 'not': case 'compare': return true; + case 'call': return node.fn !== 'tolower'; + case 'property': return types.get(node.name) === 'Edm.Boolean'; + case 'literal': return !node.quoted && (node.text === 'true' || node.text === 'false'); + } + }; + const unary = (): Node => { + const left = negation(); + const t = peek(); + if (t?.kind === 'word' && COMPARISON.has(t.value)) { + pos++; + return { kind: 'compare', op: t.value, left, right: operand() }; + } + return left; + }; + const operand = (): Node => { + const t = tokens[pos++]; + if (!t) throw new FilterSyntaxError('unexpected end of expression'); + if (t.kind === 'punct' && t.value === '(') { + const inner = orExpr(); + expect(')'); + return inner; + } + if (t.kind === 'string') return { kind: 'literal', text: t.value, quoted: true }; + if (t.kind === 'punct') throw new FilterSyntaxError(`unexpected '${t.value}'`); + if (FUNCTIONS.has(t.value) && peek()?.value === '(') { + pos++; + const args: Node[] = [orExpr()]; + while (peek()?.value === ',') { pos++; args.push(orExpr()); } + expect(')'); + return { kind: 'call', fn: t.value, args }; + } + if (types.has(t.value)) return { kind: 'property', name: t.value }; + if (/^[A-Za-z_]/.test(t.value) && !['true', 'false', 'null'].includes(t.value)) { + throw new FilterSyntaxError(`'${t.value}' is not a property of the entity type`); + } + return { kind: 'literal', text: t.value, quoted: false }; + }; + + const tree = orExpr(); + if (pos !== tokens.length) throw new FilterSyntaxError('trailing tokens'); + return tree; +} + +type Value = string | number | boolean | null; + +interface EvalContext { + row: Record; + types: ReadonlyMap; + fold: (text: string) => string; +} + +/** A comparable reading of a property value or literal, by the property's EDM type. */ +function comparable(value: unknown, type: EdmType | undefined): Value { + if (value === null || value === undefined) return null; + switch (type) { + case 'Edm.DateTimeOffset': return Date.parse(String(value)); + case 'Edm.TimeOfDay': { + const [h, m, s = '0'] = String(value).split(':'); + return (Number(h) * 3600 + Number(m) * 60 + Number(s)) * 1000; + } + case 'Edm.Int32': + case 'Edm.Double': return typeof value === 'number' ? value : Number(value); + case 'Edm.Boolean': return typeof value === 'boolean' ? value : value === 'true'; + default: return typeof value === 'string' ? value : String(value); + } +} + +function evaluate(node: Node, ctx: EvalContext): unknown { + switch (node.kind) { + case 'and': return evaluate(node.left, ctx) === true && evaluate(node.right, ctx) === true; + case 'or': return evaluate(node.left, ctx) === true || evaluate(node.right, ctx) === true; + case 'not': return evaluate(node.operand, ctx) !== true; + case 'property': return ctx.row[node.name] ?? null; + case 'literal': { + if (node.quoted) return node.text; + if (node.text === 'null') return null; + if (node.text === 'true') return true; + if (node.text === 'false') return false; + return node.text; + } + case 'call': { + const args = node.args.map((a) => evaluate(a, ctx)); + if (node.fn === 'tolower') return typeof args[0] === 'string' ? args[0].toLowerCase() : null; + const [subject, needle] = args; + if (typeof subject !== 'string' || typeof needle !== 'string') return false; + const s = ctx.fold(subject); + const n = ctx.fold(needle); + if (node.fn === 'contains') return s.includes(n); + if (node.fn === 'startswith') return s.startsWith(n); + return s.endsWith(n); + } + case 'compare': { + const type = typeOfSide(node.left, ctx) ?? typeOfSide(node.right, ctx); + const left = comparable(evaluate(node.left, ctx), type); + const right = comparable(evaluate(node.right, ctx), type); + // OData null semantics: null eq null is true, null ne a value is true, ordering against null is false. + if (left === null || right === null) { + if (node.op === 'eq') return left === right; + if (node.op === 'ne') return left !== right; + return false; + } + const a = typeof left === 'string' ? ctx.fold(left) : left; + const b = typeof right === 'string' ? ctx.fold(right) : right; + switch (node.op) { + case 'eq': return a === b; + case 'ne': return a !== b; + case 'gt': return a > b; + case 'ge': return a >= b; + case 'lt': return a < b; + default: return a <= b; + } + } + } +} + +function typeOfSide(node: Node, ctx: EvalContext): EdmType | undefined { + return node.kind === 'property' ? ctx.types.get(node.name) : undefined; +} + +// --------------------------------------------------------------------------- +// The service +// --------------------------------------------------------------------------- + +export interface ODataService { + fetch: FetchLike; + /** Every `$filter` the service was sent, in order. */ + filters: string[]; + rows: Array>; +} + +export function odataService(options: ODataServiceOptions): ODataService { + const { entitySet, properties } = options; + const key = properties[0]!.name; + const types = new Map(properties.map((p) => [p.name, p.type] as const)); + const fold = options.collation === 'case-insensitive' ? (t: string) => t.toLowerCase() : (t: string) => t; + const pageSize = options.serverPageSize ?? 1000; + const rows = options.rows.map((r) => ({ ...r })); + const filters: string[] = []; + const metadata = metadataFor(entitySet, properties); + + const json = (status: number, body?: unknown) => + new Response(body === undefined ? null : JSON.stringify(body), { status, headers: { 'Content-Type': 'application/json' } }); + const refused = (status: number, message: string) => json(status, { error: { code: String(status), message } }); + const project = (row: Record) => + Object.fromEntries(properties.map((p) => [p.name, row[p.name] ?? null])); + const keyOf = (segment: string): string | undefined => { + const m = new RegExp(`^${entitySet}\\('((?:[^']|'')*)'\\)$`).exec(segment); + return m ? m[1]!.replace(/''/g, "'") : undefined; + }; + + const fetch: FetchLike = async (input, init) => { + const url = new URL(input); + const path = decodeURIComponent(url.pathname.slice(new URL(CONFORMANCE_ROOT).pathname.length)); + const method = init.method ?? 'GET'; + if (method === 'GET' && path === '$metadata') return new Response(metadata, { status: 200 }); + + if (path === entitySet && method === 'GET') { + let matched = rows; + const filter = url.searchParams.get('$filter'); + if (filter !== null) { + filters.push(filter); + let tree: Node; + try { + tree = parse(filter, types); + } catch (error) { + return refused(400, `The query specified in the URI is not valid: ${(error as Error).message}`); + } + matched = rows.filter((row) => evaluate(tree, { row, types, fold }) === true); + } + const orderby = url.searchParams.get('$orderby'); + if (orderby) { + const terms = orderby.split(',').map((term) => { + const [name, dir] = term.trim().split(' '); + return { name: name!, desc: dir === 'desc', type: types.get(name!) }; + }); + matched = [...matched].sort((x, y) => { + for (const t of terms) { + const a = comparable(x[t.name], t.type); + const b = comparable(y[t.name], t.type); + if (a === b) continue; + // OData: null sorts first ascending. + const x1 = typeof a === 'string' ? fold(a) : a; + const y1 = typeof b === 'string' ? fold(b) : b; + const cmp = a === null ? -1 : b === null ? 1 : x1! < y1! ? -1 : x1! > y1! ? 1 : 0; + if (cmp === 0) continue; + return t.desc ? -cmp : cmp; + } + return 0; + }); + } + const count = matched.length; + const skip = Number(url.searchParams.get('$skip') ?? 0); + const top = url.searchParams.has('$top') ? Number(url.searchParams.get('$top')) : undefined; + const selected = matched.slice(skip, top === undefined ? undefined : skip + top); + const pageRows = selected.slice(0, pageSize); + const body: Record = { value: pageRows.map(project) }; + if (url.searchParams.get('$count') === 'true') body['@odata.count'] = count; + if (selected.length > pageSize) { + const next = new URL(url); + next.searchParams.set('$skip', String(skip + pageSize)); + if (top !== undefined) next.searchParams.set('$top', String(top - pageSize)); + body['@odata.nextLink'] = next.toString(); + } + return json(200, body); + } + + if (path === entitySet && method === 'POST') { + const created = JSON.parse(String(init.body)) as Record; + if (created[key] === undefined) return refused(400, `The key property '${key}' is required.`); + if (rows.some((r) => r[key] === created[key])) return refused(409, 'An entity with this key already exists.'); + rows.push(created); + return json(201, project(created)); + } + + const id = keyOf(path); + const index = id === undefined ? -1 : rows.findIndex((r) => String(r[key]) === id); + if (id !== undefined && method === 'GET') return index < 0 ? refused(404, 'Not found.') : json(200, project(rows[index]!)); + if (id !== undefined && method === 'PATCH') { + if (index < 0) return refused(404, 'Not found.'); + Object.assign(rows[index]!, JSON.parse(String(init.body))); + return json(200, project(rows[index]!)); + } + if (id !== undefined && method === 'DELETE') { + if (index < 0) return refused(404, 'Not found.'); + rows.splice(index, 1); + return new Response(null, { status: 204 }); + } + return refused(404, `No resource at '${path}'.`); + }; + + return { fetch, filters, rows }; +} diff --git a/packages/drivers/driver-odata/src/__fixtures__/recorded-service.ts b/packages/drivers/driver-odata/src/__fixtures__/recorded-service.ts new file mode 100644 index 00000000000..87713a7d814 --- /dev/null +++ b/packages/drivers/driver-odata/src/__fixtures__/recorded-service.ts @@ -0,0 +1,112 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * A recorded OData service for tests: each exchange is a request the driver + * is expected to send and the response the service answered. A request no + * exchange names fails the test with the request spelled out, so a + * translation change shows up as the exact URL that moved. + * + * URLs are written DECODED and relative to {@link SERVICE_ROOT} + * (`People?$filter=FirstName eq 'Russell'`), which is how they read in the + * OData specification; the matcher decodes the request before comparing. + */ + +import { readFileSync } from 'node:fs'; +import type { FetchLike } from '../transport.js'; + +export const SERVICE_ROOT = 'https://odata.example.test/TripPinRESTierService/'; + +export const TRIPPIN_METADATA = readFileSync(new URL('./trippin-metadata.xml', import.meta.url), 'utf8'); + +export interface Exchange { + method?: 'GET' | 'POST' | 'PATCH' | 'DELETE'; + /** Decoded, relative to {@link SERVICE_ROOT}. */ + url: string; + status?: number; + /** JSON-serialised unless a string. */ + body?: unknown; + headers?: Record; + /** Answer this exchange once, then fall through to the next match. */ + once?: boolean; +} + +export interface RecordedRequest { + method: string; + /** Decoded, relative to {@link SERVICE_ROOT} when under it. */ + url: string; + headers: Record; + body?: unknown; +} + +export const METADATA_EXCHANGE: Exchange = { url: '$metadata', body: TRIPPIN_METADATA, headers: { 'Content-Type': 'application/xml' } }; + +const relative = (url: string) => { + const decoded = decodeURIComponent(url); + return decoded.startsWith(SERVICE_ROOT) ? decoded.slice(SERVICE_ROOT.length) : decoded; +}; + +export function recordedService(exchanges: Exchange[]): { fetch: FetchLike; requests: RecordedRequest[] } { + const pending = [...exchanges]; + const requests: RecordedRequest[] = []; + const fetch: FetchLike = async (input, init) => { + const method = init.method ?? 'GET'; + const url = relative(input); + const headers = Object.fromEntries(Object.entries((init.headers ?? {}) as Record)); + const body = typeof init.body === 'string' ? JSON.parse(init.body) : undefined; + requests.push({ method, url, headers, ...(body !== undefined ? { body } : {}) }); + const index = pending.findIndex((e) => (e.method ?? 'GET') === method && e.url === url); + if (index < 0) throw new Error(`recorded service: no exchange recorded for ${method} ${url}`); + const exchange = pending[index]!; + if (exchange.once) pending.splice(index, 1); + const text = exchange.body === undefined + ? null + : typeof exchange.body === 'string' ? exchange.body : JSON.stringify(exchange.body); + return new Response(text, { + status: exchange.status ?? 200, + headers: { 'Content-Type': 'application/json', ...exchange.headers }, + }); + }; + return { fetch, requests }; +} + +/** TripPin's People, as the service answers them (abridged to five, Emails and AddressInfo kept). */ +export const PEOPLE = [ + { + '@odata.etag': 'W/"08D8D4E1A3B1C2F0"', + UserName: 'russellwhyte', FirstName: 'Russell', LastName: 'Whyte', MiddleName: null, + Gender: 'Male', Age: null, Emails: ['Russell@example.com', 'Russell@contoso.com'], + AddressInfo: [{ Address: '187 Suffolk Ln.', City: { Name: 'Boise', CountryRegion: 'United States', Region: 'ID' } }], + Concurrency: 0, + }, + { + '@odata.etag': 'W/"08D8D4E1A3B1C2F1"', + UserName: 'scottketchum', FirstName: 'Scott', LastName: 'Ketchum', MiddleName: null, + Gender: 'Male', Age: null, Emails: ['Scott@example.com'], + AddressInfo: [{ Address: '2817 Milton Dr.', City: { Name: 'Albuquerque', CountryRegion: 'United States', Region: 'NM' } }], + Concurrency: 1, + }, + { + '@odata.etag': 'W/"08D8D4E1A3B1C2F2"', + UserName: 'ronaldmundy', FirstName: 'Ronald', LastName: 'Mundy', MiddleName: null, + Gender: 'Male', Age: null, Emails: ['Ronald@example.com', 'Ronald@contoso.com'], + AddressInfo: [], Concurrency: 2, + }, + { + '@odata.etag': 'W/"08D8D4E1A3B1C2F3"', + UserName: 'javieralfred', FirstName: 'Javier', LastName: 'Alfred', MiddleName: null, + Gender: 'Male', Age: null, Emails: ['Javier@example.com', 'Javier@contoso.com'], + AddressInfo: [{ Address: '89 Jefferson Way Suite 2', City: { Name: 'Portland', CountryRegion: 'United States', Region: 'WA' } }], + Concurrency: 3, + }, + { + '@odata.etag': 'W/"08D8D4E1A3B1C2F4"', + UserName: 'willieashmore', FirstName: 'Willie', LastName: 'Ashmore', MiddleName: null, + Gender: 'Male', Age: null, Emails: ['Willie@example.com', 'Willie@contoso.com'], + AddressInfo: [], Concurrency: 4, + }, +] as const; + +/** A collection envelope as TripPin writes one. */ +export function page(rows: readonly unknown[], extra: Record = {}): Record { + return { '@odata.context': `${SERVICE_ROOT}$metadata#People`, value: rows, ...extra }; +} diff --git a/packages/drivers/driver-odata/src/__fixtures__/trippin-metadata.xml b/packages/drivers/driver-odata/src/__fixtures__/trippin-metadata.xml new file mode 100644 index 00000000000..4ed7b32ae51 --- /dev/null +++ b/packages/drivers/driver-odata/src/__fixtures__/trippin-metadata.xml @@ -0,0 +1,100 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/packages/drivers/driver-odata/src/config.ts b/packages/drivers/driver-odata/src/config.ts new file mode 100644 index 00000000000..af0ee99de1f --- /dev/null +++ b/packages/drivers/driver-odata/src/config.ts @@ -0,0 +1,72 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * The OData driver's own config contract. + * + * The driver is a PLUGIN driver: its id is not in the spec's + * `DRIVER_VOCABULARY`, so `DatasourceSchema` leaves the datasource's `config` + * unvalidated ("we validate what we can construct") and the engine holds the + * driver behind its external-driver floor. The contract therefore lives here, + * beside the code that reads it, and is parsed when the plugin is built — a + * wrong key refuses the boot instead of connecting to the wrong place. + * + * ⛔ No credential is a key of this config. The secret comes from the + * datasource's `external.credentialsRef`, resolved through the host's secret + * store at the first request (see `plugin.ts`); the config names only the + * scheme and, for `basic`, the user name. + */ + +import { z } from 'zod'; + +/** The default number of rows asked for per page (`Prefer: odata.maxpagesize`). */ +export const DEFAULT_PAGE_SIZE = 500; + +/** The default per-request timeout, in milliseconds. */ +export const DEFAULT_REQUEST_TIMEOUT_MS = 30_000; + +const AuthSchema = z.discriminatedUnion('scheme', [ + z.strictObject({ + scheme: z.literal('none'), + }), + z.strictObject({ + scheme: z.literal('bearer'), + }), + z.strictObject({ + scheme: z.literal('basic'), + username: z.string().min(1).describe('The user name; the password is the resolved credentialsRef secret.'), + }), +]).describe('How each request authenticates. The secret is the datasource\'s external.credentialsRef.'); + +export const ODataDriverConfigSchema = z.strictObject({ + name: z.string().regex(/^[a-z_][a-z0-9_]*$/) + .default('odata') + .describe('The datasource this driver serves; also the driver name and its `driver.` service suffix.'), + serviceRoot: z.string().url() + .refine((value) => /^https?:$/.test(new URL(value).protocol), 'serviceRoot must be an http(s) URL') + .refine((value) => { + const url = new URL(value); + return url.username === '' && url.password === ''; + }, 'serviceRoot must not carry credentials; declare external.credentialsRef on the datasource instead') + .refine((value) => new URL(value).search === '', 'serviceRoot must not carry a query string') + .describe('The OData service root, e.g. https://erp.example.com/odata/v4/sales/'), + version: z.literal('4.0').default('4.0') + .describe('The OData protocol version the service speaks. Only 4.0 is shipped; a v2 dialect is a new file.'), + auth: AuthSchema.default({ scheme: 'none' }), + pageSize: z.number().int().min(1).max(5000).default(DEFAULT_PAGE_SIZE) + .describe('Rows asked for per server page (Prefer: odata.maxpagesize).'), + requestTimeoutMs: z.number().int().min(1).max(600_000).default(DEFAULT_REQUEST_TIMEOUT_MS) + .describe('Per-request timeout in milliseconds.'), +}); + +export type ODataDriverConfigInput = z.input; +export type ODataDriverConfig = z.output; + +/** Parse the config, refusing with every issue named. */ +export function parseODataDriverConfig(input: ODataDriverConfigInput): ODataDriverConfig { + const parsed = ODataDriverConfigSchema.safeParse(input); + if (!parsed.success) { + const issues = parsed.error.issues.map((issue) => `${issue.path.join('.') || '(root)'}: ${issue.message}`); + throw new Error(`driver-odata: invalid config — ${issues.join('; ')}`); + } + return parsed.data; +} diff --git a/packages/drivers/driver-odata/src/dialect-v4.test.ts b/packages/drivers/driver-odata/src/dialect-v4.test.ts new file mode 100644 index 00000000000..390a5163002 --- /dev/null +++ b/packages/drivers/driver-odata/src/dialect-v4.test.ts @@ -0,0 +1,81 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +import { describe, expect, it } from 'vitest'; +import { odataV4Dialect, parseCsdlV4 } from './dialect-v4.js'; +import { TRIPPIN_METADATA } from './__fixtures__/recorded-service.js'; + +const NS = 'Microsoft.OData.SampleService.Models.TripPin'; + +describe('CSDL v4 $metadata reader', () => { + const model = parseCsdlV4(TRIPPIN_METADATA); + + it('reads entity sets with their qualified entity type, resolving a namespace alias', () => { + expect([...model.entitySets.keys()]).toEqual(['People', 'Photos', 'Airlines', 'Flights']); + expect(model.entitySets.get('People')?.entityType).toBe(`${NS}.Person`); + expect(model.entitySets.get('Flights')?.entityType).toBe(`${NS}.Flight`); + expect(model.entitySets.get('People')?.navigationBindings).toEqual({ Friends: 'People', BestFriend: 'People', Photo: 'Photos' }); + }); + + it('reads keys, property types, nullability and collections', () => { + const person = model.entityTypes.get(`${NS}.Person`)!; + expect(person.key).toEqual(['UserName']); + expect(person.properties.find((p) => p.name === 'UserName')).toEqual({ name: 'UserName', type: 'Edm.String', isCollection: false, nullable: false }); + expect(person.properties.find((p) => p.name === 'MiddleName')?.nullable).toBe(true); + expect(person.properties.find((p) => p.name === 'Emails')).toEqual({ name: 'Emails', type: 'Edm.String', isCollection: true, nullable: true }); + expect(person.properties.find((p) => p.name === 'Gender')?.type).toBe(`${NS}.PersonGender`); + expect(model.enumTypes.has(`${NS}.PersonGender`)).toBe(true); + }); + + it('flattens inheritance: the base key and properties first, through an alias and a full name alike', () => { + const flight = model.entityTypes.get(`${NS}.Flight`)!; + expect(flight.key).toEqual(['PlanItemId']); + expect(flight.properties.map((p) => p.name)).toEqual([ + 'PlanItemId', 'ConfirmationCode', 'StartsAt', 'EndsAt', 'Duration', 'SeatNumber', + 'FlightNumber', 'TripDate', 'TrackingId', 'Fare', 'Cancelled', + ]); + }); + + it('reads navigation properties with their target, cardinality and referential constraints', () => { + const person = model.entityTypes.get(`${NS}.Person`)!; + expect(person.navigationProperties.map((n) => [n.name, n.type, n.isCollection])).toEqual([ + ['Friends', `${NS}.Person`, true], + ['BestFriend', `${NS}.Person`, false], + ['Photo', `${NS}.Photo`, false], + ]); + const airline = model.entityTypes.get(`${NS}.Flight`)!.navigationProperties[0]!; + expect(airline).toEqual({ + name: 'Airline', type: `${NS}.Airline`, isCollection: false, nullable: false, + referentialConstraints: [{ property: 'AirlineCode', referencedProperty: 'AirlineCode' }], + }); + }); + + it('refuses a v2 (EDMX 1.0) document by name, since this package ships the v4 dialect only', () => { + const v2 = ''; + expect(() => parseCsdlV4(v2)).toThrow(/not CSDL 4\.x .*Version="1\.0".*v4 dialect only/); + }); + + it('refuses a document that is not XML', () => { + expect(() => parseCsdlV4('{"value":[]}')).toThrow(/not CSDL 4\.x/); + }); +}); + +describe('OData v4 envelope', () => { + it('reads value, nextLink and count', () => { + expect(odataV4Dialect.readCollection({ value: [{ a: 1 }], '@odata.nextLink': 'People?$skiptoken=2', '@odata.count': 7 })) + .toEqual({ rows: [{ a: 1 }], nextLink: 'People?$skiptoken=2', count: 7 }); + }); + + it('refuses a body that is not a collection', () => { + expect(() => odataV4Dialect.readCollection({ d: { results: [] } })).toThrow(/without a `value` array/); + }); + + it('reads the error body', () => { + expect(odataV4Dialect.readError({ error: { code: 'BadRequest', message: 'Bad filter' } })) + .toEqual({ code: 'BadRequest', message: 'Bad filter' }); + expect(odataV4Dialect.readError('')).toEqual({}); + }); + + it('spells set membership as an or-chain, which 4.0 services answer (4.01 added `in`)', () => { + expect(odataV4Dialect.inList('A', ['1', '2'])).toBe('A eq 1 or A eq 2'); + }); +}); diff --git a/packages/drivers/driver-odata/src/dialect-v4.ts b/packages/drivers/driver-odata/src/dialect-v4.ts new file mode 100644 index 00000000000..ea2b49a3fad --- /dev/null +++ b/packages/drivers/driver-odata/src/dialect-v4.ts @@ -0,0 +1,293 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * The OData 4.0 dialect — the one this package ships (maintainer order: v4 + * only; a v2 dialect is a sibling file implementing {@link ODataDialect}). + * + * References: OData Version 4.0 Part 1 (Protocol) §11.2.6.1 (system query + * options), Part 2 (URL Conventions) §5.1.1 (`$filter` operators and + * functions) and its ABNF for primitive literals, the JSON Format §12 (the + * collection envelope) and CSDL XML §3–§13. + */ + +import { XMLParser } from 'fast-xml-parser'; +import type { + EdmEntitySet, + EdmEntityType, + EdmNavigationProperty, + EdmProperty, + ODataCollectionPage, + ODataDialect, + ODataServiceModel, + ODataStringFunction, +} from './dialect.js'; + +type XmlNode = Record; + +const INTEGER_TYPES: ReadonlySet = new Set(['Edm.Byte', 'Edm.SByte', 'Edm.Int16', 'Edm.Int32', 'Edm.Int64']); +const DECIMAL_TYPES: ReadonlySet = new Set(['Edm.Decimal', 'Edm.Double', 'Edm.Single']); + +const INTEGER_TEXT = /^-?\d+$/; +const DECIMAL_TEXT = /^-?\d+(\.\d+)?([eE][+-]?\d+)?$/; +const GUID_TEXT = /^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$/; +const DATE_TEXT = /^\d{4}-\d{2}-\d{2}$/; +/** An instant: a date-time WITH its zone, so the remote and the engine read the same moment. */ +const DATE_TIME_OFFSET_TEXT = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}(:\d{2}(\.\d{1,12})?)?(Z|[+-]\d{2}:\d{2})$/; +const TIME_OF_DAY_TEXT = /^\d{2}:\d{2}(:\d{2}(\.\d{1,12})?)?$/; +/** An enum member name, as CSDL allows one (`SimpleIdentifier`). */ +const MEMBER_NAME = /^[A-Za-z_][A-Za-z0-9_]*$/; + +const quote = (text: string) => `'${text.replace(/'/g, "''")}'`; + +/** `Collection(X)` → `{ type: X, isCollection: true }`. */ +function unwrapCollection(type: string): { type: string; isCollection: boolean } { + const match = /^Collection\((.+)\)$/.exec(type); + return match ? { type: match[1]!, isCollection: true } : { type, isCollection: false }; +} + +const children = (node: unknown, name: string): XmlNode[] => { + if (!node || typeof node !== 'object') return []; + const value = (node as XmlNode)[name]; + return Array.isArray(value) ? (value as XmlNode[]) : []; +}; + +const attr = (node: XmlNode, name: string): string | undefined => { + const value = node[`@_${name}`]; + return typeof value === 'string' ? value : undefined; +}; + +/** The parser: every element an array (so a single child reads like many), attributes kept as text. */ +const xmlParser = new XMLParser({ + ignoreAttributes: false, + attributeNamePrefix: '@_', + removeNSPrefix: true, + parseAttributeValue: false, + parseTagValue: false, + isArray: (_name, _jpath, _isLeafNode, isAttribute) => !isAttribute, +}); + +/** Parse a CSDL 4.0 `$metadata` document into the {@link ODataServiceModel} the driver reads. */ +export function parseCsdlV4(xml: string): ODataServiceModel { + let document: XmlNode; + try { + document = xmlParser.parse(xml) as XmlNode; + } catch (error) { + throw new Error(`driver-odata: the $metadata document is not well-formed XML (${(error as Error).message})`); + } + const edmx = children(document, 'Edmx')[0]; + const version = edmx ? attr(edmx, 'Version') : undefined; + if (!edmx || !version || !version.startsWith('4.')) { + throw new Error( + `driver-odata: the $metadata document is not CSDL 4.x (edmx:Edmx Version="${version ?? 'absent'}"); ` + + 'this driver ships the OData v4 dialect only', + ); + } + const schemas = children(edmx, 'DataServices').flatMap((ds) => children(ds, 'Schema')); + + // Alias → namespace, so `Self.Person` and `Trippin.Person` name one type. + const aliases = new Map(); + for (const schema of schemas) { + const namespace = attr(schema, 'Namespace'); + const alias = attr(schema, 'Alias'); + if (namespace && alias) aliases.set(alias, namespace); + } + const qualify = (name: string): string => { + const { type, isCollection } = unwrapCollection(name); + const dot = type.lastIndexOf('.'); + const prefix = dot > 0 ? type.slice(0, dot) : ''; + const resolved = aliases.has(prefix) ? `${aliases.get(prefix)}${type.slice(dot)}` : type; + return isCollection ? `Collection(${resolved})` : resolved; + }; + + interface RawEntityType { + fullName: string; + name: string; + baseType?: string; + key: string[]; + properties: EdmProperty[]; + navigationProperties: EdmNavigationProperty[]; + } + const raw = new Map(); + const enumTypes = new Set(); + const entitySets = new Map(); + + for (const schema of schemas) { + const namespace = attr(schema, 'Namespace') ?? ''; + for (const enumType of children(schema, 'EnumType')) { + const name = attr(enumType, 'Name'); + if (name) enumTypes.add(`${namespace}.${name}`); + } + for (const entityType of children(schema, 'EntityType')) { + const name = attr(entityType, 'Name'); + if (!name) continue; + const baseType = attr(entityType, 'BaseType'); + raw.set(`${namespace}.${name}`, { + fullName: `${namespace}.${name}`, + name, + ...(baseType ? { baseType: qualify(baseType) } : {}), + key: children(entityType, 'Key') + .flatMap((key) => children(key, 'PropertyRef')) + .map((ref) => attr(ref, 'Name')) + .filter((n): n is string => typeof n === 'string'), + properties: children(entityType, 'Property').flatMap((property) => { + const propertyName = attr(property, 'Name'); + const declared = attr(property, 'Type'); + if (!propertyName || !declared) return []; + const { type, isCollection } = unwrapCollection(qualify(declared)); + return [{ name: propertyName, type, isCollection, nullable: attr(property, 'Nullable') !== 'false' }]; + }), + navigationProperties: children(entityType, 'NavigationProperty').flatMap((nav) => { + const navName = attr(nav, 'Name'); + const declared = attr(nav, 'Type'); + if (!navName || !declared) return []; + const { type, isCollection } = unwrapCollection(qualify(declared)); + return [{ + name: navName, + type, + isCollection, + nullable: attr(nav, 'Nullable') !== 'false', + referentialConstraints: children(nav, 'ReferentialConstraint').flatMap((rc) => { + const property = attr(rc, 'Property'); + const referencedProperty = attr(rc, 'ReferencedProperty'); + return property && referencedProperty ? [{ property, referencedProperty }] : []; + }), + }]; + }), + }); + } + for (const container of children(schema, 'EntityContainer')) { + for (const set of children(container, 'EntitySet')) { + const name = attr(set, 'Name'); + const entityType = attr(set, 'EntityType'); + if (!name || !entityType) continue; + const navigationBindings: Record = {}; + for (const binding of children(set, 'NavigationPropertyBinding')) { + const path = attr(binding, 'Path'); + const target = attr(binding, 'Target'); + if (path && target) navigationBindings[path] = target; + } + entitySets.set(name, { name, entityType: qualify(entityType), navigationBindings }); + } + } + } + + // Flatten inheritance: base type first, keys from the nearest type that declares one. + const entityTypes = new Map(); + const resolve = (fullName: string, seen: Set): EdmEntityType | undefined => { + const done = entityTypes.get(fullName); + if (done) return done; + const own = raw.get(fullName); + if (!own || seen.has(fullName)) return undefined; + seen.add(fullName); + const base = own.baseType ? resolve(own.baseType, seen) : undefined; + const resolved: EdmEntityType = { + fullName, + name: own.name, + key: own.key.length > 0 ? own.key : (base?.key ?? []), + properties: [...(base?.properties ?? []), ...own.properties], + navigationProperties: [...(base?.navigationProperties ?? []), ...own.navigationProperties], + }; + entityTypes.set(fullName, resolved); + return resolved; + }; + for (const fullName of raw.keys()) resolve(fullName, new Set()); + + return { version, entityTypes, entitySets, enumTypes }; +} + +const isRecord = (value: unknown): value is Record => + !!value && typeof value === 'object' && !Array.isArray(value); + +export const odataV4Dialect: ODataDialect = { + version: '4.0', + + requestHeaders() { + return { + 'OData-Version': '4.0', + 'OData-MaxVersion': '4.0', + Accept: 'application/json;odata.metadata=minimal', + }; + }, + + readCollection(body: unknown): ODataCollectionPage { + if (!isRecord(body) || !Array.isArray(body.value)) { + throw new Error('driver-odata: the service answered a collection read without a `value` array'); + } + const page: ODataCollectionPage = { rows: body.value.filter(isRecord) }; + const next = body['@odata.nextLink']; + if (typeof next === 'string' && next !== '') page.nextLink = next; + const count = body['@odata.count']; + if (typeof count === 'number') page.count = count; + else if (typeof count === 'string' && INTEGER_TEXT.test(count)) page.count = Number(count); + return page; + }, + + readEntity(body: unknown) { + if (!isRecord(body)) throw new Error('driver-odata: the service answered an entity read without an entity'); + return body; + }, + + etagOf(entity) { + const etag = entity['@odata.etag']; + return typeof etag === 'string' && etag !== '' ? etag : undefined; + }, + + isAnnotation(key) { + // `@odata.etag`, `@odata.context`, and instance annotations on a property (`Name@odata.type`). + return key.includes('@'); + }, + + readError(body) { + if (!isRecord(body) || !isRecord(body.error)) return {}; + const { code, message } = body.error; + return { + ...(typeof code === 'string' && code !== '' ? { code } : {}), + ...(typeof message === 'string' && message !== '' ? { message } : {}), + }; + }, + + parseMetadata: parseCsdlV4, + + literal(value, type, model) { + if (value === null) return 'null'; + if (type === 'Edm.String') return typeof value === 'string' ? quote(value) : undefined; + if (type === 'Edm.Boolean') return typeof value === 'boolean' ? String(value) : undefined; + if (INTEGER_TYPES.has(type)) { + if (typeof value === 'number') return Number.isSafeInteger(value) ? String(value) : undefined; + return typeof value === 'string' && INTEGER_TEXT.test(value) ? value : undefined; + } + if (DECIMAL_TYPES.has(type)) { + if (typeof value === 'number') return Number.isFinite(value) ? String(value) : undefined; + return typeof value === 'string' && DECIMAL_TEXT.test(value) ? value : undefined; + } + if (type === 'Edm.Guid') return typeof value === 'string' && GUID_TEXT.test(value) ? value : undefined; + if (type === 'Edm.Date') return typeof value === 'string' && DATE_TEXT.test(value) ? value : undefined; + if (type === 'Edm.DateTimeOffset') { + if (value instanceof Date) return Number.isNaN(value.getTime()) ? undefined : value.toISOString(); + return typeof value === 'string' && DATE_TIME_OFFSET_TEXT.test(value) ? value : undefined; + } + if (type === 'Edm.TimeOfDay') return typeof value === 'string' && TIME_OF_DAY_TEXT.test(value) ? value : undefined; + if (model.enumTypes.has(type)) { + return typeof value === 'string' && MEMBER_NAME.test(value) ? `${type}'${value}'` : undefined; + } + // Binary, Duration, spatial types, complex types: no spelling is claimed. + return undefined; + }, + + stringFunction(fn: ODataStringFunction, property: string, literal: string) { + return `${fn}(${property},${literal})`; + }, + + toLower(property) { + return `tolower(${property})`; + }, + + inList(property, literals) { + // OData 4.0 has no `in` operator (4.01 added it): an `or` chain answers on both. + return literals.map((literal) => `${property} eq ${literal}`).join(' or '); + }, + + countOptions() { + return { $count: 'true' }; + }, +}; diff --git a/packages/drivers/driver-odata/src/dialect.ts b/packages/drivers/driver-odata/src/dialect.ts new file mode 100644 index 00000000000..188083cbf85 --- /dev/null +++ b/packages/drivers/driver-odata/src/dialect.ts @@ -0,0 +1,122 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * The dialect seam: every place where OData versions spell the same thing + * differently goes through one {@link ODataDialect} object, so a v2 dialect is + * a new file beside `dialect-v4.ts`, not a rewrite of the driver. + * + * What varies between versions, and therefore lives here: + * + * - the request headers (`OData-Version`, the JSON format parameter); + * - the response envelope of a collection (`value` / `@odata.nextLink` / + * `@odata.count` in v4; `d.results` / `__next` / `__count` in v2) and of a + * single entity, and where its ETag sits; + * - the `$metadata` document (CSDL v4 under `edmx:Edmx Version="4.0"`; v2's + * EDMX 1.0 with `DataServiceVersion`); + * - literal spellings (`'text'`, bare v4 dates and GUIDs; v2's + * `datetime'…'` / `guid'…'`), the string functions (`contains(P,'x')` in v4, + * `substringof('x',P)` in v2) and set membership; + * - the count query option (`$count=true` in v4, `$inlinecount=allpages` in + * v2). + * + * What does NOT vary, and therefore stays in the driver and the filter + * translator: the `where` walk, the widening rules, paging, key handling, the + * error mapping and the write verbs. + */ + +/** A primitive or enum property of an entity type. */ +export interface EdmProperty { + name: string; + /** The qualified type name, e.g. `Edm.String`, `Trippin.PersonGender`; for a collection, its element type. */ + type: string; + isCollection: boolean; + nullable: boolean; +} + +/** A navigation property: imported as a lookup only (a single-valued one), never followed. */ +export interface EdmNavigationProperty { + name: string; + /** The qualified target entity type. */ + type: string; + isCollection: boolean; + nullable: boolean; + /** `ReferentialConstraint` pairs: the dependent property on this type → the principal property on the target. */ + referentialConstraints: Array<{ property: string; referencedProperty: string }>; +} + +export interface EdmEntityType { + /** The qualified name (`Namespace.Name`). */ + fullName: string; + name: string; + /** The key property names, the base type's when this type declares none. */ + key: string[]; + /** Own and inherited structural properties, base type first. */ + properties: EdmProperty[]; + /** Own and inherited navigation properties, base type first. */ + navigationProperties: EdmNavigationProperty[]; +} + +export interface EdmEntitySet { + name: string; + /** The qualified entity type. */ + entityType: string; + /** `NavigationPropertyBinding` path → target entity set. */ + navigationBindings: Record; +} + +/** The parts of a service's `$metadata` the driver reads. */ +export interface ODataServiceModel { + /** The CSDL version the document declared. */ + version: string; + entityTypes: Map; + entitySets: Map; + /** Qualified enum type names; a literal of one is spelled as an enum member. */ + enumTypes: Set; +} + +/** One page of a collection, as the dialect reads it out of the envelope. */ +export interface ODataCollectionPage { + rows: Array>; + nextLink?: string; + count?: number; +} + +/** A string function the translator may ask the dialect to spell. */ +export type ODataStringFunction = 'contains' | 'startswith' | 'endswith'; + +export interface ODataDialect { + /** The protocol version this dialect speaks, as the config names it. */ + readonly version: string; + /** Headers every request carries. */ + requestHeaders(): Record; + /** Read a collection response body. Throws when the body is not a collection envelope. */ + readCollection(body: unknown): ODataCollectionPage; + /** Read a single-entity response body. Throws when the body is not an entity. */ + readEntity(body: unknown): Record; + /** The entity's ETag, when the response carried one. */ + etagOf(entity: Record): string | undefined; + /** Is this key of an entity payload a protocol annotation rather than a property? */ + isAnnotation(key: string): boolean; + /** Read the error body of a failed request: the service's code and message, when present. */ + readError(body: unknown): { code?: string; message?: string }; + /** Parse a `$metadata` document. Throws when it is not this dialect's CSDL. */ + parseMetadata(xml: string): ODataServiceModel; + /** + * Spell `value` as a literal of `type` (a qualified primitive or enum type). + * `undefined` when the value cannot be spelled as that type — the caller + * then leaves the predicate to the engine instead of guessing. + */ + literal(value: unknown, type: string, model: ODataServiceModel): string | undefined; + /** Spell a string function call over a property and a string literal. */ + stringFunction(fn: ODataStringFunction, property: string, literal: string): string; + /** Spell a case-folded property reference (for `$icontains`). */ + toLower(property: string): string; + /** + * Spell set membership over already-spelled literals (non-empty). The + * result need not be grouped: the translator parenthesizes it wherever it + * is an operand (under `not`, beside `and` / `or`). + */ + inList(property: string, literals: readonly string[]): string; + /** The query options that ask a collection read to carry its total count. */ + countOptions(): Record; +} diff --git a/packages/drivers/driver-odata/src/filter.test.ts b/packages/drivers/driver-odata/src/filter.test.ts new file mode 100644 index 00000000000..a5b8c9e1d45 --- /dev/null +++ b/packages/drivers/driver-odata/src/filter.test.ts @@ -0,0 +1,116 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * The pushdown translation table: each `where` the engine can hand the + * driver, the `$filter` it becomes, and whether every predicate made it in. + * The rule every row pins is the module's: never narrower than the `where`. + */ + +import { describe, expect, it } from 'vitest'; +import { odataV4Dialect, parseCsdlV4 } from './dialect-v4.js'; +import { translateWhere, type FilterTarget } from './filter.js'; +import { TRIPPIN_METADATA } from './__fixtures__/recorded-service.js'; + +const model = parseCsdlV4(TRIPPIN_METADATA); +const NS = 'Microsoft.OData.SampleService.Models.TripPin'; + +/** A target over an entity type, with the column map the tests' objects declare (remote → local). */ +function targetOf(entityType: string, columnMap: Record): FilterTarget { + const type = model.entityTypes.get(`${NS}.${entityType}`)!; + const byField = new Map(type.properties.map((p) => [columnMap[p.name] ?? p.name, p])); + if (type.key.length === 1) byField.set('id', type.properties.find((p) => p.name === type.key[0])!); + return { model, propertyOf: (field) => byField.get(field) }; +} + +const people = targetOf('Person', { + UserName: 'user_name', FirstName: 'first_name', LastName: 'last_name', MiddleName: 'middle_name', + Gender: 'gender', Age: 'age', Emails: 'emails', AddressInfo: 'address_info', Concurrency: 'concurrency', +}); +const flights = targetOf('Flight', { + PlanItemId: 'plan_item_id', FlightNumber: 'flight_number', TripDate: 'trip_date', TrackingId: 'tracking_id', + Fare: 'fare', Cancelled: 'cancelled', StartsAt: 'starts_at', SeatNumber: 'seat_number', Duration: 'duration', +}); + +type Row = [label: string, target: FilterTarget, where: unknown, filter: string | null, complete: boolean]; + +const TABLE: Row[] = [ + // Nothing to translate. + ['no where', people, undefined, null, true], + ['empty where', people, {}, null, true], + + // Equality and its shorthand, through the column map and the `id` alias. + ['bare equality', people, { first_name: 'Russell' }, "FirstName eq 'Russell'", true], + ['$eq', people, { first_name: { $eq: 'Russell' } }, "FirstName eq 'Russell'", true], + ['quote escaping', people, { last_name: "O'Neil" }, "LastName eq 'O''Neil'", true], + ['id names the key', people, { id: 'russellwhyte' }, "UserName eq 'russellwhyte'", true], + ['null equality', people, { middle_name: null }, 'MiddleName eq null', true], + ['enum member', people, { gender: 'Female' }, `Gender eq ${NS}.PersonGender'Female'`, true], + + // Ordering and ranges on non-text types. + ['integer ordering', people, { age: { $gte: 21, $lt: 65 } }, '(Age ge 21) and (Age lt 65)', true], + ['$between', people, { age: { $between: [18, 30] } }, 'Age ge 18 and Age le 30', true], + ['integer key, integer text', flights, { id: '42' }, 'PlanItemId eq 42', true], + ['decimal', flights, { fare: { $gt: 99.5 } }, 'Fare gt 99.5', true], + ['boolean', flights, { cancelled: false }, 'Cancelled eq false', true], + ['date', flights, { trip_date: { $lte: '2026-11-02' } }, 'TripDate le 2026-11-02', true], + ['date-time with its zone', flights, { starts_at: { $gt: '2026-11-02T08:00:00Z' } }, 'StartsAt gt 2026-11-02T08:00:00Z', true], + ['Date object', flights, { starts_at: { $lt: new Date('2026-11-02T08:00:00.000Z') } }, 'StartsAt lt 2026-11-02T08:00:00.000Z', true], + ['guid', flights, { tracking_id: '5a1d6c8e-2f0b-4b5e-9f1a-0c2d3e4f5a6b' }, 'TrackingId eq 5a1d6c8e-2f0b-4b5e-9f1a-0c2d3e4f5a6b', true], + + // Negation keeps the null rows a SQL-backed service would drop. + ['$ne on a nullable property', people, { age: { $ne: 30 } }, 'Age ne 30 or Age eq null', true], + ['$ne on a non-nullable property', flights, { cancelled: { $ne: true } }, 'Cancelled ne true', true], + ['$ne null', people, { middle_name: { $ne: null } }, 'MiddleName ne null', true], + ['$nin on a nullable property', people, { age: { $nin: [1, 2] } }, 'not (Age eq 1 or Age eq 2) or Age eq null', true], + ['$nin of one value on a nullable property', people, { age: { $nin: [1] } }, 'not (Age eq 1) or Age eq null', true], + ['$nin of one value on a non-nullable property', flights, { id: { $nin: [42] } }, 'not (PlanItemId eq 42)', true], + ['$nin of one boolean', flights, { cancelled: { $nin: [true] } }, 'not (Cancelled eq true)', true], + ['$nin empty', people, { age: { $nin: [] } }, null, true], + + // Sets and nulls. + ['$in', people, { user_name: { $in: ['russellwhyte', 'scottketchum'] } }, "UserName eq 'russellwhyte' or UserName eq 'scottketchum'", true], + ['$in beside another predicate is grouped', people, { user_name: { $in: ['russellwhyte', 'scottketchum'] }, first_name: 'Russell' }, "(UserName eq 'russellwhyte' or UserName eq 'scottketchum') and (FirstName eq 'Russell')", true], + ['$in one', people, { user_name: { $in: ['russellwhyte'] } }, "UserName eq 'russellwhyte'", true], + ['$in empty', people, { user_name: { $in: [] } }, 'false', true], + ['$null true', people, { middle_name: { $null: true } }, 'MiddleName eq null', true], + ['$null false', people, { middle_name: { $null: false } }, 'MiddleName ne null', true], + + // String functions. + ['$contains', people, { last_name: { $contains: 'hy' } }, "contains(LastName,'hy')", true], + ['$startsWith', people, { last_name: { $startsWith: 'Wh' } }, "startswith(LastName,'Wh')", true], + ['$endsWith', people, { last_name: { $endsWith: 'te' } }, "endswith(LastName,'te')", true], + ['$icontains, ASCII', people, { last_name: { $icontains: 'WHY' } }, "contains(tolower(LastName),'why')", true], + + // Combinators. + ['two fields AND', people, { first_name: 'Russell', age: { $gt: 1 } }, "(FirstName eq 'Russell') and (Age gt 1)", true], + ['$and', people, { $and: [{ first_name: 'Russell' }, { last_name: 'Whyte' }] }, "(FirstName eq 'Russell') and (LastName eq 'Whyte')", true], + ['$or', people, { $or: [{ first_name: 'Russell' }, { first_name: 'Scott' }] }, "(FirstName eq 'Russell') or (FirstName eq 'Scott')", true], + + // Left to the engine — widened, never narrowed. + ['$not is never pushed down', people, { $not: { first_name: 'Russell' } }, null, false], + ['$and keeps what it can', people, { $and: [{ first_name: 'Russell' }, { $not: { age: 3 } }] }, "FirstName eq 'Russell'", false], + ['$or with one untranslatable member drops whole', people, { $or: [{ first_name: 'Russell' }, { last_name: { $like: 'Wh%' } }] }, null, false], + ['text ordering', people, { last_name: { $gt: 'M' } }, null, false], + ['text negation', people, { last_name: { $ne: 'Whyte' } }, null, false], + ['text $nin', people, { last_name: { $nin: ['Whyte'] } }, null, false], + ['one operator of two', people, { age: { $gt: 1, $exists: true } }, 'Age gt 1', false], + ['$like', people, { last_name: { $like: 'Wh%' } }, null, false], + ['$notContains', people, { last_name: { $notContains: 'x' } }, null, false], + ['$icontains, non-ASCII', people, { last_name: { $icontains: 'É' } }, null, false], + ['undeclared field', people, { nickname: 'Rus' }, null, false], + ['collection property', people, { emails: 'Russell@example.com' }, null, false], + ['complex property', people, { address_info: { $null: true } }, null, false], + ['field reference', people, { first_name: { $eq: { $field: 'last_name' } } }, null, false], + ['dotted path', people, { 'best_friend.first_name': 'Scott' }, null, false], + ['date-time without a zone', flights, { starts_at: { $gt: '2026-11-02T08:00:00' } }, null, false], + ['date-only on a date-time', flights, { starts_at: { $gt: '2026-11-02' } }, null, false], + ['text on an integer key', flights, { id: 'abc' }, null, false], + ['duration', flights, { duration: 'PT2H' }, null, false], + ['enum member that is not a name', people, { gender: "Male' or 1 eq 1" }, null, false], +]; + +describe('where → $filter (OData v4)', () => { + it.each(TABLE)('%s', (_label, target, where, filter, complete) => { + expect(translateWhere(where, target, odataV4Dialect)).toEqual({ filter, complete }); + }); +}); diff --git a/packages/drivers/driver-odata/src/filter.ts b/packages/drivers/driver-odata/src/filter.ts new file mode 100644 index 00000000000..a47de718c7c --- /dev/null +++ b/packages/drivers/driver-odata/src/filter.ts @@ -0,0 +1,228 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * `where` → `$filter`: the pushdown translator. + * + * ## The one rule: never narrower + * + * The engine holds this driver behind its external-driver floor (objectql + * `external-driver-floor.ts`): whatever the driver returns, the engine + * re-applies the whole query in memory. So a `$filter` that matches MORE rows + * than the `where` costs only transfer, and one that matches FEWER loses rows + * no layer above can recover. Every translation here is therefore a SUPERSET + * of the engine's reading, and anything that cannot be spelled as one is left + * out — the result is `null` ("no constraint"), never a guess: + * + * - a member of `$and` that cannot be translated is dropped (the rest still + * narrows); + * - an `$or` with any member that cannot be translated is dropped whole + * (dropping one branch of an `$or` would narrow it); + * - `$not` is never pushed down. A service backed by SQL evaluates `not (P gt + * 5)` under three-valued logic and drops the rows where `P` is null, which + * the engine's reading keeps — a narrowing the protocol text does not show. + * - `$ne` / `$nin` on a nullable property carry `or P eq null`, for the same + * reason: a SQL-backed `P ne 5` drops null rows. `P eq null` is answered by + * `IS NULL` on every backend, so the disjunction is a superset under either + * reading. + * - Comparisons on `Edm.String` are pushed down only where a case- or + * accent-insensitive collation can only WIDEN the answer (`eq`, `in`, + * `contains`, `startswith`, `endswith`): ordering (`gt`, `between`, …) and + * negation (`ne`, `nin`) on text are left to the engine, because a remote + * collation can order or equate strings differently from it. + * - A comparand the dialect cannot spell as the property's EDM type (a date + * without its zone on a `DateTimeOffset`, text on an integer key) is left + * out rather than coerced. + * + * What is translated, on primitive and enum properties: `$eq`, `$ne`, `$gt`, + * `$gte`, `$lt`, `$lte`, `$between`, `$in`, `$nin`, `$null`, `$contains`, + * `$startsWith`, `$endsWith`, `$icontains`, combined with `$and` / `$or`. + * Left to the engine: `$not`, `$notContains`, `$like` / `$ilike`, `$exists`, + * field references (`{ $field }`), dotted paths, collection-valued and + * complex properties, and any operator this list does not name. + */ + +import type { EdmProperty, ODataDialect, ODataServiceModel } from './dialect.js'; + +/** How the translator reads one object: a field name → the property it names, and its spelling in a URL. */ +export interface FilterTarget { + model: ODataServiceModel; + /** The remote property an engine field names, or `undefined` when it names none the translator may use. */ + propertyOf(field: string): EdmProperty | undefined; +} + +const isPlainObject = (value: unknown): value is Record => + !!value && typeof value === 'object' && !Array.isArray(value) && !(value instanceof Date); + +const ORDERING: Record = { $gt: 'gt', $gte: 'ge', $lt: 'lt', $lte: 'le' }; +const STRING_FUNCTIONS: Record = { + $contains: 'contains', + $startsWith: 'startswith', + $endsWith: 'endswith', +}; + +const and = (parts: string[]): string | null => + parts.length === 0 ? null : parts.length === 1 ? parts[0]! : parts.map((p) => `(${p})`).join(' and '); + +/** A translated `where`. */ +export interface TranslatedWhere { + /** The `$filter` expression; `null` means "no constraint". */ + filter: string | null; + /** + * `true` when every predicate of the `where` reached the expression, so the + * service's answer is the engine's (up to the service's own collation). + * `false` when one was left to the engine and the expression is wider. + */ + complete: boolean; +} + +/** + * Translate `where` into a `$filter` expression that selects a superset of + * the rows it selects. + */ +export function translateWhere(where: unknown, target: FilterTarget, dialect: ODataDialect): TranslatedWhere { + const state = { complete: true }; + const filter = translateCondition(where, target, dialect, state); + return { filter, complete: state.complete }; +} + +interface TranslationState { + complete: boolean; +} + +/** Leave a predicate to the engine: no constraint here, and the translation is no longer complete. */ +function leave(state: TranslationState): null { + state.complete = false; + return null; +} + +function translateCondition(where: unknown, target: FilterTarget, dialect: ODataDialect, state: TranslationState): string | null { + if (where === undefined || where === null) return null; + if (!isPlainObject(where)) return leave(state); + const parts: string[] = []; + for (const [key, condition] of Object.entries(where)) { + const part = translateEntry(key, condition, target, dialect, state); + if (part !== null) parts.push(part); + } + return and(parts); +} + +function translateEntry( + key: string, + condition: unknown, + target: FilterTarget, + dialect: ODataDialect, + state: TranslationState, +): string | null { + if (key === '$and') { + if (!Array.isArray(condition)) return leave(state); + return and( + condition.map((member) => translateCondition(member, target, dialect, state)).filter((p): p is string => p !== null), + ); + } + if (key === '$or') { + if (!Array.isArray(condition) || condition.length === 0) return leave(state); + const members = condition.map((member) => translateCondition(member, target, dialect, state)); + if (members.some((m) => m === null)) return leave(state); + return members.length === 1 ? members[0]! : members.map((m) => `(${m})`).join(' or '); + } + // `$not` and every other `$` key: left to the engine (see the module comment). + if (key.startsWith('$')) return leave(state); + + const property = target.propertyOf(key); + if (!property || property.isCollection) return leave(state); + + if (isPlainObject(condition) && Object.keys(condition).some((k) => k.startsWith('$'))) { + const parts: string[] = []; + for (const [op, comparand] of Object.entries(condition)) { + const part = translateOperator(op, comparand, property, target, dialect); + if (part === null) leave(state); + else if (part !== TRUE) parts.push(part); + } + return and(parts); + } + const part = translateOperator('$eq', condition, property, target, dialect); + return part === null ? leave(state) : part; +} + +/** An operator whose translation is "no constraint" by its own meaning, not by a gap (`$nin: []`). */ +const TRUE = 'true'; + +function translateOperator( + op: string, + comparand: unknown, + property: EdmProperty, + target: FilterTarget, + dialect: ODataDialect, +): string | null { + const name = property.name; + const isText = property.type === 'Edm.String'; + // A field reference, a token object, a nested condition: never spelled. + if (isPlainObject(comparand)) return null; + const lit = (value: unknown) => dialect.literal(value, property.type, target.model); + + switch (op) { + case '$eq': { + const value = lit(comparand); + return value === undefined ? null : `${name} eq ${value}`; + } + case '$ne': { + if (isText || comparand === null) return comparand === null ? `${name} ne null` : null; + const value = lit(comparand); + if (value === undefined) return null; + return property.nullable ? `${name} ne ${value} or ${name} eq null` : `${name} ne ${value}`; + } + case '$gt': + case '$gte': + case '$lt': + case '$lte': { + if (isText || comparand === null) return null; + const value = lit(comparand); + return value === undefined ? null : `${name} ${ORDERING[op]} ${value}`; + } + case '$between': { + if (isText || !Array.isArray(comparand) || comparand.length !== 2) return null; + const low = lit(comparand[0]); + const high = lit(comparand[1]); + if (low === undefined || high === undefined || comparand[0] === null || comparand[1] === null) return null; + return `${name} ge ${low} and ${name} le ${high}`; + } + case '$in': { + if (!Array.isArray(comparand)) return null; + if (comparand.length === 0) return 'false'; + const values = comparand.map(lit); + if (values.some((v) => v === undefined)) return null; + return dialect.inList(name, values as string[]); + } + case '$nin': { + if (Array.isArray(comparand) && comparand.length === 0) return TRUE; + if (isText || !Array.isArray(comparand)) return null; + const values = comparand.map(lit); + if (values.some((v) => v === undefined) || comparand.some((v) => v === null)) return null; + // `not` binds tighter than `eq` and `or`: without the parentheses `not P eq 1` reads as + // `(not P) eq 1`, which a service refuses, and the nullable form's `or P eq null` would + // fall under the `not` and drop the null rows. + const excluded = `not (${dialect.inList(name, values as string[])})`; + return property.nullable ? `${excluded} or ${name} eq null` : excluded; + } + case '$null': { + if (comparand === true) return `${name} eq null`; + if (comparand === false) return `${name} ne null`; + return null; + } + case '$contains': + case '$startsWith': + case '$endsWith': { + if (!isText || typeof comparand !== 'string') return null; + return dialect.stringFunction(STRING_FUNCTIONS[op]!, name, lit(comparand)!); + } + case '$icontains': { + // The engine folds ASCII case only. Over an ASCII comparand, `tolower` folds at least that + // much, so the answer can only widen; a non-ASCII comparand could be folded by the remote + // where the engine would not, and is left to the engine. + if (!isText || typeof comparand !== 'string' || !/^[\x20-\x7E]*$/.test(comparand)) return null; + return dialect.stringFunction('contains', dialect.toLower(name), lit(comparand.toLowerCase())!); + } + default: + return null; + } +} diff --git a/packages/drivers/driver-odata/src/index.ts b/packages/drivers/driver-odata/src/index.ts new file mode 100644 index 00000000000..3685345a751 --- /dev/null +++ b/packages/drivers/driver-odata/src/index.ts @@ -0,0 +1,34 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +export { ODataDriver } from './odata-driver.js'; +export type { ODataDriverRuntime, ODataIntrospectedSchema, ODataIntrospectedTable } from './odata-driver.js'; + +export { createODataDriverPlugin } from './plugin.js'; +export type { DatasourceDefinitionSource, ODataDriverPluginOptions, ODataSecretStore } from './plugin.js'; + +export { + DEFAULT_PAGE_SIZE, + DEFAULT_REQUEST_TIMEOUT_MS, + ODataDriverConfigSchema, + parseODataDriverConfig, +} from './config.js'; +export type { ODataDriverConfig, ODataDriverConfigInput } from './config.js'; + +// The dialect seam: a v2 dialect implements `ODataDialect` in a file of its own. +export { odataV4Dialect, parseCsdlV4 } from './dialect-v4.js'; +export type { + EdmEntitySet, + EdmEntityType, + EdmNavigationProperty, + EdmProperty, + ODataCollectionPage, + ODataDialect, + ODataServiceModel, + ODataStringFunction, +} from './dialect.js'; + +export { translateWhere } from './filter.js'; +export type { FilterTarget, TranslatedWhere } from './filter.js'; + +export { ODataRequestError } from './transport.js'; +export type { FetchLike, ODataAuth } from './transport.js'; diff --git a/packages/drivers/driver-odata/src/odata-driver.conformance.test.ts b/packages/drivers/driver-odata/src/odata-driver.conformance.test.ts new file mode 100644 index 00000000000..7362fb475a2 --- /dev/null +++ b/packages/drivers/driver-odata/src/odata-driver.conformance.test.ts @@ -0,0 +1,352 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * The shared driver conformance case-sets (`@objectstack/spec/data` + * `*-conformance.ts`), run against driver-odata the way it is served: a + * plugin driver on a `schemaMode: 'external'` datasource, behind the real + * engine's floor, over an in-process OData v4 service + * (`__fixtures__/odata-service.ts`) that evaluates the `$filter` the driver + * writes. + * + * The read case-sets run twice, over an `ordinal` service and over a + * `case-insensitive` one: a service's collation is its own, and the driver's + * rule is that it only ever asks a WIDER question, which the floor then + * narrows. Both runs must answer the case's rows exactly. + * + * - FILTER_LOGIC, FILTER_COMPARAND_TYPE, TEMPORAL, TEMPORAL_TIME: + * `engine.find` with the case's `where`. + * - FILTER_TEXT: the floored driver's `find`, beneath the engine's + * declared-type door (which refuses a text operator on a number field + * before any driver is asked), as the case-set's no-match rows require. + * - PAGINATION, PAGINATION_UNORDERED, PAGINATION_ZERO_LIMIT: a walk of + * `engine.find` pages; the service pages its answers too + * (`@odata.nextLink`). + * - AGGREGATION: `engine.aggregate` — the floor hides `aggregate`, so the + * engine aggregates the floored rows in memory. + * - READABLE_IDS: every case is refused, loudly, in the READABLE_IDS_REFUSAL + * envelope, by the driver itself and by the floored driver, with nothing + * sent: `$filter` has no tenant-walled semi-join (ADR-0139 D3's "or refused + * loudly"). + * - VALUE_ROUNDTRIP: the driver's own `create` then `find`: what was written is + * what is read back, type included. + */ + +import { describe, expect, it } from 'vitest'; +import { ObjectQL } from '@objectstack/objectql'; +import { + AGGREGATION_CASES, + AGGREGATION_ROWS, + FILTER_COMPARAND_TYPE_CASES, + FILTER_COMPARAND_TYPE_ROWS, + FILTER_LOGIC_CASES, + FILTER_LOGIC_ROWS, + FILTER_TEXT_CASES, + FILTER_TEXT_ROWS, + PAGINATION_ALL_IDS, + PAGINATION_CASES, + PAGINATION_ROWS, + PAGINATION_UNORDERED_CASES, + PAGINATION_ZERO_LIMIT_CASES, + READABLE_IDS_CASES, + READABLE_IDS_CHILD_ROWS, + READABLE_IDS_REFUSAL, + TEMPORAL_CASES, + TEMPORAL_ROWS, + TEMPORAL_TIME_CASES, + TEMPORAL_TIME_ROWS, + VALUE_ROUNDTRIP_CASES, + VALUE_ROUNDTRIP_FIELDS, + valueRoundTripDivergence, + type FilterTextCase, + type FilterTextRejectionCase, +} from '@objectstack/spec/data'; +import { parseODataDriverConfig } from './config.js'; +import { ODataDriver } from './odata-driver.js'; +import { CONFORMANCE_ROOT, odataService, type ServiceProperty } from './__fixtures__/odata-service.js'; + +type Row = Record; +const SYSTEM = { context: { isSystem: true } }; +const DATASOURCE = 'conformance'; +const OBJECT = 'conformance_row'; +const ENTITY_SET = 'Rows'; +const COLLATIONS = ['ordinal', 'case-insensitive'] as const; + +const sortedIds = (rows: Row[]) => rows.map((r) => String(r.id)).sort(); + +/** A driver over a fresh service holding `rows`, behind the engine floor, with `fields` declared on the object. */ +async function boot(opts: { + properties: ServiceProperty[]; + rows: readonly object[]; + fields: Record; + collation?: (typeof COLLATIONS)[number]; + serverPageSize?: number; +}) { + const service = odataService({ + entitySet: ENTITY_SET, + properties: opts.properties, + rows: opts.rows.map((r) => ({ ...r }) as Row), + ...(opts.collation ? { collation: opts.collation } : {}), + ...(opts.serverPageSize ? { serverPageSize: opts.serverPageSize } : {}), + }); + const driver = new ODataDriver(parseODataDriverConfig({ name: DATASOURCE, serviceRoot: CONFORMANCE_ROOT }), { fetch: service.fetch }); + const engine = new ObjectQL(); + engine.registerDriver(driver); + engine.registerDatasourceDef({ name: DATASOURCE, driver: DATASOURCE, schemaMode: 'external', external: { allowWrites: true } }); + engine.registerApp({ + id: 'conformance_pkg', + name: 'Conformance', + objects: [{ name: OBJECT, datasource: DATASOURCE, external: { remoteName: ENTITY_SET, writable: true }, fields: opts.fields }], + } as never); + await engine.syncSchemas(); + // The driver as the engine holds it: behind the floor, beneath the engine's own query doors. + const floored = engine.getDriverByName(DATASOURCE)!; + return { engine, driver, floored, service }; +} + +const text = { type: 'text' } as const; +const STRING = (name: string, nullable = true): ServiceProperty => ({ name, type: 'Edm.String', nullable }); + +describe.each(COLLATIONS)('driver-odata conformance over a %s service', (collation) => { + describe('FILTER_LOGIC_CASES', () => { + const columns = ['a', 'b', 'c', 'd', 'owner', 'status', 'parent_object', 'parent_id']; + const setup = () => boot({ + collation, + properties: [STRING('id', false), ...columns.map((c) => STRING(c))], + rows: FILTER_LOGIC_ROWS, + fields: Object.fromEntries(columns.map((c) => [c, text])), + }); + for (const c of FILTER_LOGIC_CASES) { + it(c.name, async () => { + const { engine } = await setup(); + expect(sortedIds(await engine.find(OBJECT, { where: c.filter, ...SYSTEM } as never)), c.note).toEqual([...c.expected]); + }); + } + }); + + describe('FILTER_TEXT_CASES', () => { + const setup = () => boot({ + collation, + properties: [STRING('id', false), STRING('name'), { name: 'score', type: 'Edm.Int32' }], + rows: FILTER_TEXT_ROWS, + fields: { name: text, score: { type: 'number' } }, + }); + const isRejection = (c: FilterTextCase): c is FilterTextRejectionCase => c.expectRejection === true; + for (const c of FILTER_TEXT_CASES) { + if (isRejection(c)) { + it(`${c.name} — refused in the ADR-0112 envelope`, async () => { + const { floored } = await setup(); + const err = await floored.find(OBJECT, { where: c.filter }).then(() => undefined, (e: unknown) => e); + expect(err, c.note ?? 'expected a refusal').toMatchObject({ code: c.code, status: 400 }); + for (const mention of c.mustMention) expect((err as Error).message).toContain(mention); + }); + } else { + it(c.name, async () => { + const { floored } = await setup(); + expect(sortedIds(await floored.find(OBJECT, { where: c.filter })), c.note).toEqual([...c.expected]); + }); + } + } + }); + + describe('FILTER_COMPARAND_TYPE_CASES', () => { + const setup = () => boot({ + collation, + properties: [ + STRING('id', false), { name: 'qty', type: 'Edm.Int32' }, STRING('label'), + { name: 'active', type: 'Edm.Boolean' }, STRING('note'), + ], + rows: FILTER_COMPARAND_TYPE_ROWS, + fields: { qty: { type: 'number' }, label: text, active: { type: 'boolean' }, note: text }, + }); + for (const c of FILTER_COMPARAND_TYPE_CASES) { + it(`${c.name} (${c.verdict})`, async () => { + const { engine } = await setup(); + const read = engine.find(OBJECT, { where: c.filter(), ...SYSTEM } as never); + if (c.verdict === 'matches') expect(sortedIds(await read), c.note).toEqual([...c.expected]); + else if (c.verdict === 'compiles') await expect(read, c.note).resolves.toBeInstanceOf(Array); + else { + const err = await read.then(() => undefined, (e: unknown) => e); + expect(err, c.note).toMatchObject({ code: c.code, status: 400 }); + for (const mention of c.mustMention) expect((err as Error).message).toContain(mention); + } + }); + } + }); + + // Not a shared case-set: the shared filter rows negate text, which the driver never pushes + // down. A negation on a number is pushed down, under `not`, to a service whose parser reads + // OData's precedence (`not` binds tighter than `eq`) and refuses a non-Boolean operand. + describe('negation pushed down on a non-text property', () => { + const setup = () => boot({ + collation, + properties: [ + STRING('id', false), { name: 'n', type: 'Edm.Int32' }, { name: 'm', type: 'Edm.Int32', nullable: false }, + ], + rows: [{ id: 'r1', n: 1, m: 1 }, { id: 'r2', n: 2, m: 2 }, { id: 'r3', n: null, m: 3 }], + fields: { n: { type: 'number' }, m: { type: 'number' } }, + }); + const cases: Array<[string, object, string[], string]> = [ + ['$nin of one value, nullable', { n: { $nin: [1] } }, ['r2', 'r3'], 'not (n eq 1) or n eq null'], + ['$nin of two values, nullable', { n: { $nin: [1, 2] } }, ['r3'], 'not (n eq 1 or n eq 2) or n eq null'], + ['$nin of one value, not nullable', { m: { $nin: [1] } }, ['r2', 'r3'], 'not (m eq 1)'], + ['$ne, nullable', { n: { $ne: 1 } }, ['r2', 'r3'], 'n ne 1 or n eq null'], + ]; + for (const [name, where, expected, filter] of cases) { + it(name, async () => { + const { engine, service } = await setup(); + expect(sortedIds(await engine.find(OBJECT, { where, ...SYSTEM } as never))).toEqual(expected); + expect(service.filters.join('\n')).toContain(filter); + }); + } + it('the service refuses `not` over a non-Boolean operand, as a precedence-reading parser does', async () => { + const { service } = await setup(); + const answer = await service.fetch(`${CONFORMANCE_ROOT}Rows?$filter=${encodeURIComponent('not n eq 1')}`, { method: 'GET' }); + expect(answer.status).toBe(400); + }); + }); + + describe('TEMPORAL_CASES', () => { + const setup = () => boot({ + collation, + properties: [ + STRING('id', false), { name: 'at', type: 'Edm.DateTimeOffset' }, { name: 'on', type: 'Edm.Date' }, STRING('why'), + ], + rows: TEMPORAL_ROWS.map((r) => ({ id: r.id, at: r.at, on: r.on, why: r.why })), + fields: { at: { type: 'datetime' }, on: { type: 'date' }, why: text }, + }); + for (const c of TEMPORAL_CASES) { + it(c.name, async () => { + const { engine } = await setup(); + expect(sortedIds(await engine.find(OBJECT, { where: c.filter, ...SYSTEM } as never)), c.note).toEqual([...c.expected].sort()); + }); + } + }); + + describe('TEMPORAL_TIME_CASES', () => { + const setup = () => boot({ + collation, + properties: [STRING('id', false), { name: 'at', type: 'Edm.TimeOfDay' }, STRING('why')], + rows: TEMPORAL_TIME_ROWS.map((r) => ({ id: r.id, at: r.at, why: r.why })), + fields: { at: { type: 'time' }, why: text }, + }); + for (const c of TEMPORAL_TIME_CASES) { + it(c.name, async () => { + const { engine } = await setup(); + expect(sortedIds(await engine.find(OBJECT, { where: c.filter, ...SYSTEM } as never)), c.note).toEqual([...c.expected].sort()); + }); + } + }); + + describe('PAGINATION cases', () => { + // A service page of 5 rows, so a walk crosses `@odata.nextLink`. + const setup = () => boot({ + collation, + serverPageSize: 5, + properties: [STRING('id', false), STRING('status'), { name: 'rank', type: 'Edm.Int32' }, STRING('name')], + rows: PAGINATION_ROWS, + fields: { status: text, rank: { type: 'number' }, name: text }, + }); + const walk = async (engine: ObjectQL, orderBy: unknown, pageSize: number) => { + const visited: Row[] = []; + for (let offset = 0; offset < PAGINATION_ROWS.length; offset += pageSize) { + visited.push(...await engine.find(OBJECT, { ...(orderBy ? { orderBy } : {}), limit: pageSize, offset, ...SYSTEM } as never)); + } + return visited; + }; + + for (const c of PAGINATION_CASES) { + it(`${c.name}: every row exactly once, in the requested order`, async () => { + const { engine } = await setup(); + const visited = await walk(engine, c.orderBy, c.pageSize); + expect(sortedIds(visited)).toEqual([...PAGINATION_ALL_IDS].sort()); + const whole = await engine.find(OBJECT, { orderBy: c.orderBy, ...SYSTEM } as never); + const keys = (row: Row) => c.orderBy.map(({ field }) => row[field]); + expect(visited.map(keys)).toEqual(whole.map(keys)); + }); + } + for (const c of PAGINATION_UNORDERED_CASES) { + it(`unordered — ${c.name}: every row exactly once`, async () => { + const { engine } = await setup(); + const visited = await walk(engine, undefined, c.pageSize); + expect(visited).toHaveLength(PAGINATION_ALL_IDS.length); + expect(sortedIds(visited)).toEqual([...PAGINATION_ALL_IDS].sort()); + }); + } + for (const c of PAGINATION_ZERO_LIMIT_CASES) { + it(`zero limit — ${c.name}`, async () => { + const { engine } = await setup(); + expect(await engine.find(OBJECT, { ...c.query, ...SYSTEM } as never)).toHaveLength(c.expectedRowCount); + }); + } + }); +}); + +describe('driver-odata conformance — AGGREGATION_CASES (the engine aggregates the floored rows)', () => { + const setup = () => boot({ + properties: [ + STRING('id', false), STRING('region'), STRING('stage'), { name: 'score', type: 'Edm.Int32' }, + { name: 'amount', type: 'Edm.Int32' }, { name: 'flag', type: 'Edm.Boolean' }, + ], + rows: AGGREGATION_ROWS, + fields: { region: text, stage: text, score: { type: 'number' }, amount: { type: 'number' }, flag: { type: 'boolean' } }, + }); + const byGroup = (a: { group: string | null }, b: { group: string | null }) => String(a.group).localeCompare(String(b.group)); + + for (const c of AGGREGATION_CASES) { + it(c.name, async () => { + const { engine } = await setup(); + const rows: Row[] = await engine.aggregate(OBJECT, { + aggregations: [{ function: c.function, ...(c.field ? { field: c.field } : {}), alias: 'n' }], + ...(c.groupBy ? { groupBy: [c.groupByAlias ? { field: c.groupBy, alias: c.groupByAlias } : c.groupBy] } : {}), + ...SYSTEM, + } as never); + const groupKey = c.groupByAlias ?? c.groupBy; + const actual = rows.map((r) => ({ group: groupKey ? String(r[groupKey]) : null, value: r.n })).sort(byGroup); + expect(actual, c.note ?? c.name).toEqual(c.expected.map((e) => ({ group: e.group, value: e.value })).sort(byGroup)); + }); + } +}); + +describe('driver-odata conformance — READABLE_IDS_CASES (refused loudly: $filter has no semi-join)', () => { + const setup = () => boot({ + properties: [STRING('id', false), STRING('object_name'), STRING('record_id'), STRING('label'), STRING('organization_id')], + rows: READABLE_IDS_CHILD_ROWS, + fields: { object_name: text, record_id: text, label: text, organization_id: text }, + }); + for (const c of READABLE_IDS_CASES) { + if (c.verdict === 'matches' && !c.holdsComposedLeaf) { + // The engine's denial answer carries no leaf: an ordinary `where`, answered. + it(`${c.name} — answered`, async () => { + const { floored } = await setup(); + expect(sortedIds(await floored.find(OBJECT, { where: c.where() })), c.note).toEqual([...c.expected]); + }); + continue; + } + it(`${c.name} — refused by the driver and behind the floor, nothing sent`, async () => { + const { driver, floored, service } = await setup(); + await expect(driver.find(OBJECT, { where: c.where() }), c.note).rejects.toMatchObject(READABLE_IDS_REFUSAL); + await expect(driver.count(OBJECT, { where: c.where() }), c.note).rejects.toMatchObject(READABLE_IDS_REFUSAL); + await expect(floored.find(OBJECT, { where: c.where() }), c.note).rejects.toMatchObject(READABLE_IDS_REFUSAL); + expect(service.filters).toEqual([]); + }); + } +}); + +describe('driver-odata conformance — VALUE_ROUNDTRIP_CASES (create, then read back)', () => { + // The driver alone: the case-set's field shape is the driver-level one, not an engine object's. + const columns = Object.keys(VALUE_ROUNDTRIP_FIELDS); + for (const c of VALUE_ROUNDTRIP_CASES) { + it(`${c.name} (${c.column})`, async () => { + const service = odataService({ + entitySet: ENTITY_SET, + properties: [STRING('id', false), ...columns.map((name) => STRING(name))], + rows: [], + }); + const driver = new ODataDriver(parseODataDriverConfig({ name: DATASOURCE, serviceRoot: CONFORMANCE_ROOT }), { fetch: service.fetch }); + driver.registerExternalObject({ name: OBJECT, external: { remoteName: ENTITY_SET }, fields: VALUE_ROUNDTRIP_FIELDS }); + await driver.create(OBJECT, { id: `rt_${c.name}`, label: c.name, [c.column]: c.wrote }); + const [row] = await driver.find(OBJECT, { where: { id: `rt_${c.name}` } }); + expect(valueRoundTripDivergence(c, row?.[c.column]), c.note).toBeNull(); + }); + } +}); diff --git a/packages/drivers/driver-odata/src/odata-driver.floor.test.ts b/packages/drivers/driver-odata/src/odata-driver.floor.test.ts new file mode 100644 index 00000000000..6f81436bd59 --- /dev/null +++ b/packages/drivers/driver-odata/src/odata-driver.floor.test.ts @@ -0,0 +1,130 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * The driver behind the real engine's external-driver floor: published under + * a plugin driver id on a `schemaMode: 'external'` datasource, the engine + * floors it, so a predicate the driver leaves out of `$filter` is still + * applied — in memory, over the rows the service returned — and a refusal by + * the service reaches the caller as the floor maps it. + */ + +import { describe, expect, it } from 'vitest'; +import { + DatasourceUnavailableError, + ExternalReadRefusedError, + ExternalWriteRefusedError, + ObjectQL, + needsExternalDriverFloor, +} from '@objectstack/objectql'; +import { markFilterSubtreeProvenance } from '@objectstack/spec/data'; +import { parseODataDriverConfig } from './config.js'; +import { ODataDriver } from './odata-driver.js'; +import { METADATA_EXCHANGE, PEOPLE, SERVICE_ROOT, page, recordedService, type Exchange } from './__fixtures__/recorded-service.js'; + +const DATASOURCE = 'trippin'; +const OBJECT = 'trip_person'; +const DEF = { name: DATASOURCE, driver: DATASOURCE, schemaMode: 'external', external: { allowWrites: true } } as const; +const PERSON = { + name: OBJECT, + datasource: DATASOURCE, + external: { + remoteName: 'People', + writable: true, + columnMap: { UserName: 'user_name', FirstName: 'first_name', LastName: 'last_name', Age: 'age' }, + }, + fields: { + user_name: { type: 'text' }, + first_name: { type: 'text' }, + last_name: { type: 'text' }, + age: { type: 'number' }, + }, +}; + +async function boot(exchanges: Exchange[]) { + const service = recordedService([METADATA_EXCHANGE, ...exchanges]); + const driver = new ODataDriver(parseODataDriverConfig({ name: DATASOURCE, serviceRoot: SERVICE_ROOT }), { fetch: service.fetch }); + const engine = new ObjectQL(); + engine.registerDriver(driver); + engine.registerDatasourceDef(DEF); + engine.registerApp({ id: 'trippin_pkg', name: 'TripPin', objects: [PERSON] } as never); + // The engine's DDL-free binding pass: it hands the driver each external object (`registerExternalObject`). + await engine.syncSchemas(); + return { engine, requests: service.requests }; +} + +/** A second W surname, beside TripPin's Russell Whyte. */ +const WHITE = { ...PEOPLE[0], '@odata.etag': 'W/"william"', UserName: 'williamwhite', FirstName: 'William', LastName: 'White' }; + +const sent = (requests: { url: string }[]) => requests.map((r) => r.url).filter((url) => url !== '$metadata'); + +describe('driver-odata behind the engine floor', () => { + it('is a plugin driver id, so the engine floors it', () => { + expect(needsExternalDriverFloor(DEF)).toBe(true); + }); + + it('applies in memory what it does not push down: the service is asked the wider question', async () => { + // `$like` has no `$filter` spelling, so only `startswith` reaches the service, which answers two rows. + const { engine, requests } = await boot([ + { + url: "People?$filter=startswith(LastName,'W')&$orderby=UserName asc&$top=1001", + body: page([PEOPLE[0], WHITE]), + }, + ]); + const rows = await engine.find(OBJECT, { + where: { last_name: { $startsWith: 'W' }, first_name: { $like: 'Wil%' } }, + context: { isSystem: true }, + } as never); + expect(sent(requests)).toEqual(["People?$filter=startswith(LastName,'W')&$orderby=UserName asc&$top=1001"]); + expect(rows.map((r: Record) => r.id)).toEqual(['williamwhite']); + }); + + it('counts through the floor\'s read, not the service\'s $count', async () => { + const { engine, requests } = await boot([ + { url: "People?$filter=startswith(LastName,'W')&$orderby=UserName asc&$top=1001", body: page([PEOPLE[0], WHITE]) }, + ]); + expect(await engine.count(OBJECT, { where: { last_name: { $startsWith: 'W' } }, context: { isSystem: true } } as never)).toBe(2); + expect(sent(requests)).toEqual(["People?$filter=startswith(LastName,'W')&$orderby=UserName asc&$top=1001"]); + }); + + it('a read the service refuses with 400 is INVALID_QUERY / 400 carrying the service\'s reason', async () => { + const { engine } = await boot([ + { + url: "People?$filter=FirstName eq 'x'&$orderby=UserName asc&$top=1001", + status: 400, + body: { error: { code: 'BadRequest', message: "The property 'FirstName' cannot be used in the $filter query option." } }, + }, + ]); + const err = await engine + .find(OBJECT, { where: markFilterSubtreeProvenance({ first_name: 'x' }, 'author'), context: { isSystem: true } } as never) + .then(() => undefined, (e: unknown) => e); + expect(err).toBeInstanceOf(ExternalReadRefusedError); + expect(err).toMatchObject({ code: 'INVALID_QUERY', status: 400 }); + expect((err as Error).message).toContain("cannot be used in the $filter query option"); + }); + + it('a write the service refuses with a 4xx is EXTERNAL_WRITE_REFUSED / 422 carrying the service\'s reason', async () => { + const { engine } = await boot([ + { + method: 'POST', + url: 'People', + status: 400, + body: { error: { code: 'BadRequest', message: "The property 'LastName' is required." } }, + }, + ]); + const err = await engine + .insert(OBJECT, { user_name: 'lewisblack', first_name: 'Lewis' }, { context: { isSystem: true } } as never) + .then(() => undefined, (e: unknown) => e); + expect(err).toBeInstanceOf(ExternalWriteRefusedError); + expect(err).toMatchObject({ code: 'EXTERNAL_WRITE_REFUSED', status: 422 }); + expect((err as Error).message).toContain("The property 'LastName' is required."); + }); + + it('a service failure is ERR_DATASOURCE_UNAVAILABLE / 503, its text withheld', async () => { + const { engine } = await boot([ + { url: 'People?$orderby=UserName asc&$top=1001', status: 503, body: { error: { code: 'ServiceUnavailable', message: 'backend sap-prod-07 is down' } } }, + ]); + const err = await engine.find(OBJECT, { context: { isSystem: true } } as never).then(() => undefined, (e: unknown) => e); + expect(err).toBeInstanceOf(DatasourceUnavailableError); + expect((err as Error).message).not.toContain('sap-prod-07'); + }); +}); diff --git a/packages/drivers/driver-odata/src/odata-driver.test.ts b/packages/drivers/driver-odata/src/odata-driver.test.ts new file mode 100644 index 00000000000..05182636b89 --- /dev/null +++ b/packages/drivers/driver-odata/src/odata-driver.test.ts @@ -0,0 +1,265 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * The driver against a recorded TripPin-shaped service: each test names the + * exact requests the driver sends and answers them from recordings. + */ + +import { describe, expect, it } from 'vitest'; +import { parseODataDriverConfig } from './config.js'; +import { ODataDriver } from './odata-driver.js'; +import { ODataRequestError } from './transport.js'; +import { + METADATA_EXCHANGE, + PEOPLE, + SERVICE_ROOT, + page, + recordedService, + type Exchange, +} from './__fixtures__/recorded-service.js'; + +const PERSON = { + name: 'trip_person', + external: { + remoteName: 'People', + columnMap: { + UserName: 'user_name', FirstName: 'first_name', LastName: 'last_name', MiddleName: 'middle_name', + Gender: 'gender', Age: 'age', Emails: 'emails', AddressInfo: 'address_info', Concurrency: 'concurrency', + }, + }, +}; + +function driverFor(exchanges: Exchange[], extra: { credential?: () => Promise; auth?: { scheme: 'bearer' } } = {}) { + const service = recordedService([METADATA_EXCHANGE, ...exchanges]); + const config = parseODataDriverConfig({ name: 'trippin', serviceRoot: SERVICE_ROOT, pageSize: 2, ...(extra.auth ? { auth: extra.auth } : {}) }); + const driver = new ODataDriver(config, { fetch: service.fetch, ...(extra.credential ? { credential: extra.credential } : {}) }); + driver.registerExternalObject(PERSON); + return { driver, requests: service.requests }; +} + +const urls = (requests: { method: string; url: string }[]) => + requests.filter((r) => r.url !== '$metadata').map((r) => `${r.method} ${r.url}`); + +describe('reads', () => { + it('pushes the where down, renames through the column map, drops annotations and sets id to the key', async () => { + const { driver, requests } = driverFor([ + { url: "People?$filter=FirstName eq 'Russell'", body: page([PEOPLE[0]]) }, + ]); + const rows = await driver.find('trip_person', { where: { first_name: 'Russell' } }); + expect(urls(requests)).toEqual(["GET People?$filter=FirstName eq 'Russell'"]); + expect(rows).toEqual([{ + id: 'russellwhyte', user_name: 'russellwhyte', first_name: 'Russell', last_name: 'Whyte', middle_name: null, + gender: 'Male', age: null, emails: ['Russell@example.com', 'Russell@contoso.com'], + address_info: [{ Address: '187 Suffolk Ln.', City: { Name: 'Boise', CountryRegion: 'United States', Region: 'ID' } }], + concurrency: 0, + }]); + }); + + it('asks for the page size, follows @odata.nextLink and stops at the limit', async () => { + const { driver, requests } = driverFor([ + { + url: 'People?$orderby=UserName asc&$top=3', + body: page(PEOPLE.slice(0, 2), { '@odata.nextLink': `${SERVICE_ROOT}People?$orderby=UserName asc&$top=1&$skiptoken='scottketchum'` }), + }, + { url: "People?$orderby=UserName asc&$top=1&$skiptoken='scottketchum'", body: page(PEOPLE.slice(2, 4)) }, + ]); + const rows = await driver.find('trip_person', { limit: 3 }); + expect(rows.map((r) => r.id)).toEqual(['russellwhyte', 'scottketchum', 'ronaldmundy']); + expect(urls(requests)).toEqual([ + 'GET People?$orderby=UserName asc&$top=3', + "GET People?$orderby=UserName asc&$top=1&$skiptoken='scottketchum'", + ]); + expect(requests[1]!.headers.Prefer).toBe('odata.maxpagesize=2'); + expect(requests[1]!.headers['OData-Version']).toBe('4.0'); + }); + + it('orders a paged read by the requested fields, then the key; pushes $skip and $select', async () => { + const { driver, requests } = driverFor([ + { url: 'People?$orderby=LastName desc,UserName asc&$select=UserName,FirstName&$skip=2&$top=2', body: page([]) }, + ]); + await driver.find('trip_person', { orderBy: [{ field: 'last_name', order: 'desc' }], fields: ['first_name'], offset: 2, limit: 2 }); + expect(urls(requests)).toEqual(['GET People?$orderby=LastName desc,UserName asc&$select=UserName,FirstName&$skip=2&$top=2']); + }); + + it('refuses a paged read sorted by a field the service cannot sort by, rather than paging another order', async () => { + const { driver } = driverFor([]); + await expect(driver.find('trip_person', { orderBy: [{ field: 'emails', order: 'asc' }], limit: 5 })) + .rejects.toMatchObject({ status: 400, message: expect.stringMatching(/sorts by 'emails'/) }); + }); + + it('refuses a nextLink to another origin, which would carry the credential elsewhere', async () => { + const { driver } = driverFor([ + { url: 'People', body: page([PEOPLE[0]], { '@odata.nextLink': 'https://elsewhere.example/People?$skiptoken=1' }) }, + ]); + await expect(driver.find('trip_person', {})).rejects.toThrow(/link to another origin; it is not followed/); + }); + + it('counts with $count=true when the whole where was translated', async () => { + const { driver, requests } = driverFor([ + { url: "People?$count=true&$top=0&$filter=startswith(LastName,'W')", body: page([], { '@odata.count': 2 }) }, + ]); + expect(await driver.count('trip_person', { where: { last_name: { $startsWith: 'W' } } })).toBe(2); + expect(urls(requests)).toEqual(["GET People?$count=true&$top=0&$filter=startswith(LastName,'W')"]); + }); + + it('refuses a count it could only over-count', async () => { + const { driver, requests } = driverFor([]); + await expect(driver.count('trip_person', { where: { last_name: { $like: 'W%' } } })) + .rejects.toMatchObject({ status: 400 }); + expect(urls(requests)).toEqual([]); + }); + + it('answers a refused read with the HTTP status and the service message, never the URL', async () => { + const { driver } = driverFor([ + { url: "People?$filter=FirstName eq 'x'", status: 400, body: { error: { code: 'BadRequest', message: 'The query specified in the URI is not valid.' } } }, + ]); + const error = await driver.find('trip_person', { where: { first_name: 'x' } }).catch((e: unknown) => e); + expect(error).toBeInstanceOf(ODataRequestError); + expect(error).toMatchObject({ status: 400, odataCode: 'BadRequest' }); + expect((error as Error).message).toBe("OData service refused GET on 'People' (HTTP 400): The query specified in the URI is not valid."); + expect((error as Error).message).not.toContain('example.test'); + }); + + it('names an entity set the service does not declare', async () => { + const { driver } = driverFor([]); + driver.registerExternalObject({ name: 'trip_ghost', external: { remoteName: 'Ghosts' } }); + await expect(driver.find('trip_ghost', {})).rejects.toThrow(/entity set 'Ghosts', which the service's \$metadata does not declare/); + }); + + it('reads $metadata once', async () => { + const { driver, requests } = driverFor([{ url: 'People', body: page([]) }]); + await driver.find('trip_person', {}); + await driver.find('trip_person', {}); + expect(requests.filter((r) => r.url === '$metadata')).toHaveLength(1); + }); +}); + +describe('writes', () => { + it('creates with POST, sending only declared properties under their remote names', async () => { + const created = { ...PEOPLE[0], UserName: 'lewisblack', FirstName: 'Lewis', LastName: 'Black' }; + const { driver, requests } = driverFor([{ method: 'POST', url: 'People', status: 201, body: created }]); + const row = await driver.create('trip_person', { + id: 'generated-by-engine', user_name: 'lewisblack', first_name: 'Lewis', last_name: 'Black', created_at: '2026-10-29T00:00:00Z', + }); + expect(requests[1]).toMatchObject({ + method: 'POST', url: 'People', + body: { UserName: 'lewisblack', FirstName: 'Lewis', LastName: 'Black' }, + headers: { Prefer: 'return=representation', 'Content-Type': 'application/json' }, + }); + expect(row.id).toBe('lewisblack'); + }); + + it('updates with PATCH by key, If-Match from the last read, without the key or engine stamps in the body', async () => { + const { driver, requests } = driverFor([ + { url: "People?$filter=UserName eq 'russellwhyte'", body: page([PEOPLE[0]]) }, + { method: 'PATCH', url: "People('russellwhyte')", body: { ...PEOPLE[0], LastName: 'White', '@odata.etag': 'W/"new"' } }, + ]); + await driver.find('trip_person', { where: { id: 'russellwhyte' } }); + const row = await driver.update('trip_person', 'russellwhyte', { id: 'russellwhyte', last_name: 'White', user_name: 'russellwhyte', updated_at: 'x' }); + expect(requests[2]).toMatchObject({ + method: 'PATCH', url: "People('russellwhyte')", body: { LastName: 'White' }, + headers: { 'If-Match': 'W/"08D8D4E1A3B1C2F0"' }, + }); + expect(row?.last_name).toBe('White'); + }); + + it('re-reads after a 204 PATCH and answers null for a key the service does not have', async () => { + const { driver, requests } = driverFor([ + { method: 'PATCH', url: "People('scottketchum')", status: 204 }, + { url: "People('scottketchum')", body: PEOPLE[1] }, + { method: 'PATCH', url: "People('nobody')", status: 404, body: { error: { code: 'NotFound', message: 'gone' } } }, + ]); + expect((await driver.update('trip_person', 'scottketchum', { first_name: 'Scott' }))?.id).toBe('scottketchum'); + expect(requests[1]!.headers['If-Match']).toBeUndefined(); + expect(await driver.update('trip_person', 'nobody', { first_name: 'x' })).toBeNull(); + }); + + it('carries a refused write as its status, for the floor to map', async () => { + const { driver } = driverFor([ + { method: 'PATCH', url: "People('russellwhyte')", status: 412, body: { error: { code: 'PreconditionFailed', message: 'The ETag does not match.' } } }, + ]); + await expect(driver.update('trip_person', 'russellwhyte', { last_name: 'x' })) + .rejects.toMatchObject({ status: 412, message: expect.stringContaining('The ETag does not match.') }); + }); + + it('deletes by key: true when deleted, false when the service has no such entity', async () => { + const { driver, requests } = driverFor([ + { method: 'DELETE', url: "People('russellwhyte')", status: 204 }, + { method: 'DELETE', url: "People('nobody')", status: 404 }, + ]); + expect(await driver.delete('trip_person', 'russellwhyte')).toBe(true); + expect(await driver.delete('trip_person', 'nobody')).toBe(false); + expect(urls(requests)).toEqual(["DELETE People('russellwhyte')", "DELETE People('nobody')"]); + }); + + it('refuses the verbs it does not offer by name, as a request refusal', async () => { + const { driver } = driverFor([]); + await expect(driver.bulkCreate('trip_person', [{}])).rejects.toMatchObject({ status: 400, message: expect.stringMatching(/bulkCreate .*not offered/) }); + await expect(driver.upsert('trip_person', {})).rejects.toMatchObject({ status: 400 }); + await expect(driver.beginTransaction()).rejects.toMatchObject({ status: 400 }); + }); +}); + +describe('keys', () => { + const ORDER_METADATA = ` + + + + + + `; + + it('spells a composite key as the predicate body and reads it back for a write', async () => { + const service = recordedService([ + { url: '$metadata', body: ORDER_METADATA }, + { url: 'OrderLines', body: { value: [{ OrderID: 7, Sku: "A,B'1", Quantity: 2 }] } }, + { method: 'DELETE', url: "OrderLines(OrderID=7,Sku='A,B''1')", status: 204 }, + ]); + const driver = new ODataDriver(parseODataDriverConfig({ serviceRoot: SERVICE_ROOT }), { fetch: service.fetch }); + driver.registerExternalObject({ name: 'order_line', external: { remoteName: 'OrderLines' } }); + const [row] = await driver.find('order_line', {}); + expect(row!.id).toBe("OrderID=7,Sku='A,B''1'"); + expect(await driver.delete('order_line', row!.id as string)).toBe(true); + expect(await driver.delete('order_line', 'not a predicate')).toBe(false); + }); +}); + +describe('introspection', () => { + it('lists entity sets as tables, with EDM types, the key and single-valued navigations as lookup candidates', async () => { + const { driver } = driverFor([]); + const schema = await driver.introspectSchema(); + expect(schema.dialect).toBe('odata-4.0'); + expect(Object.keys(schema.tables)).toEqual(['People', 'Photos', 'Airlines', 'Flights']); + const people = schema.tables.People!; + expect(people.columns.find((c) => c.name === 'UserName')).toEqual({ name: 'UserName', type: 'Edm.String', nullable: false, primaryKey: true }); + expect(people.columns.find((c) => c.name === 'Emails')?.type).toBe('Collection(Edm.String)'); + expect(people.navigationProperties).toEqual([ + { name: 'BestFriend', targetEntityType: 'Microsoft.OData.SampleService.Models.TripPin.Person', targetEntitySet: 'People', nullable: true }, + { name: 'Photo', targetEntityType: 'Microsoft.OData.SampleService.Models.TripPin.Photo', targetEntitySet: 'Photos', nullable: true }, + ]); + expect(schema.tables.Flights!.columns.filter((c) => c.primaryKey).map((c) => c.name)).toEqual(['PlanItemId']); + }); +}); + +describe('credentials', () => { + it('sends the resolved secret as a bearer token, and resolves again once after a 401', async () => { + let calls = 0; + const secrets = ['expired-token', 'rotated-token']; + const { driver, requests } = driverFor( + [ + { url: 'People', status: 401, once: true }, + { url: 'People', body: page([]) }, + ], + { auth: { scheme: 'bearer' }, credential: async () => secrets[calls++] }, + ); + await driver.find('trip_person', {}); + const peopleRequests = requests.filter((r) => r.url === 'People'); + expect(peopleRequests.map((r) => r.headers.Authorization)).toEqual(['Bearer expired-token', 'Bearer rotated-token']); + }); + + it('refuses to send a request when the scheme needs a credential and none resolves', async () => { + const { driver, requests } = driverFor([], { auth: { scheme: 'bearer' }, credential: async () => undefined }); + await expect(driver.find('trip_person', {})).rejects.toThrow(/resolved to nothing/); + expect(requests).toHaveLength(0); + }); +}); diff --git a/packages/drivers/driver-odata/src/odata-driver.ts b/packages/drivers/driver-odata/src/odata-driver.ts new file mode 100644 index 00000000000..1b8b6af09d8 --- /dev/null +++ b/packages/drivers/driver-odata/src/odata-driver.ts @@ -0,0 +1,621 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * `ODataDriver` — a declarative OData v4 datasource driver (epic #27261 Tier B, + * ruling 6486145005). + * + * An administrator declares a datasource served by this driver and binds + * objects to its entity sets (`object.external.remoteName`); every read and + * write of such an object is answered by the service, live. Nothing is stored + * on the platform side. + * + * ## A plugin driver, behind the engine's floor + * + * The driver is published by {@link createODataDriverPlugin} as the service + * `driver.` and is NOT a built-in id (`DRIVER_VOCABULARY`), so on a + * datasource with `schemaMode: 'external'` the engine holds it behind its + * external-driver floor: `find` is sent `where`, `orderBy` and a row cap, and + * the engine re-applies the whole query to the rows returned. This driver + * pushes down what it can translate (`filter.ts`) and is never narrower than + * the query; `supports` claims nothing about pushdown, because no such claim + * exists in `DriverCapabilities` and the floor treats every part of the query + * as unclaimed. + * + * ## What a request becomes + * + * - `find` → `GET ?$filter&$orderby&$select&$top&$skip`, following + * `@odata.nextLink` until the asked number of rows is reached. A paged read + * is ordered by the entity key after any requested order, so pages + * partition the result (`IDataDriver.find`'s contract). + * - `count` → the same read with `$count=true&$top=0`, when the whole `where` + * was translated; otherwise refused, since the service cannot count rows it + * cannot select. + * - `create` → `POST `; `update` → `PATCH ()`; + * `delete` → `DELETE ()`. `If-Match` carries the row's + * `@odata.etag` when a read of that row returned one. Only properties the + * entity type declares are sent: the engine's own stamps (`updated_at`, …) + * are not the service's. + * - The bulk verbs, `upsert`, transactions, `execute` and `dropTable` are + * refused by name. A transport without `$batch` has no atomic multi-row + * write, and offering a loop under that name would claim one. + * + * ## Rows and keys + * + * A row is the entity with its protocol annotations removed, each property + * renamed through `external.columnMap` (remote → local), and `id` set to the + * entity key: the key's value as text for a single-property key, the key + * predicate's body (`OrderID=1,ProductID=7`) for a composite one. The key is + * read from `$metadata`, which the driver reads once, at the first request. + */ + +import { findMisplacedReadableIds, readableIdsDriverRefusal } from '@objectstack/core'; +import type { DriverQuery, IDataDriver, IntrospectedSchema, IntrospectedTable } from '@objectstack/spec/contracts'; +import type { DriverOptions } from '@objectstack/spec/data'; +import type { EdmEntitySet, EdmEntityType, EdmProperty, ODataDialect, ODataServiceModel } from './dialect.js'; +import { odataV4Dialect } from './dialect-v4.js'; +import { translateWhere, type FilterTarget } from './filter.js'; +import { ODataRequestError, ODataTransport, type FetchLike, type ODataResponse } from './transport.js'; +import type { ODataDriverConfig } from './config.js'; + +type Row = Record; + +/** The dialects this package ships, by the config's `version`. */ +const DIALECTS: Record = { '4.0': odataV4Dialect }; + +/** How many ETags the driver keeps (by object and id) for `If-Match`. */ +const ETAG_CACHE_LIMIT = 10_000; + +/** What the driver is given besides its config. */ +export interface ODataDriverRuntime { + /** The `fetch` to call; the global one when absent. */ + fetch?: FetchLike; + /** The secret behind the datasource's `external.credentialsRef`; see `plugin.ts`. */ + credential?: () => Promise; +} + +/** An introspected table, with the navigation properties the import offers as lookups. */ +export interface ODataIntrospectedTable extends IntrospectedTable { + /** The qualified entity type. */ + entityType: string; + /** Single-valued navigation properties: candidates for a lookup field, never followed by reads. */ + navigationProperties: Array<{ name: string; targetEntitySet?: string; targetEntityType: string; nullable: boolean }>; +} + +export interface ODataIntrospectedSchema extends IntrospectedSchema { + tables: Record; +} + +/** What `registerExternalObject` records for an object. */ +interface ObjectDeclaration { + remoteName?: string; + /** remote property → local field */ + columnMap: Record; +} + +/** An object resolved against the service model. */ +interface Binding extends FilterTarget { + object: string; + entitySet: EdmEntitySet; + entityType: EdmEntityType; + keyProperties: EdmProperty[]; + /** remote property → local field */ + fieldOfProperty: Map; + /** `id` names the (single) key, rather than a property the column map named `id`. */ + idIsKeyAlias: boolean; +} + +/** A refusal of the request itself — the floor answers it 400 on a read, 422 on a write. */ +function refusal(message: string): ODataRequestError { + return new ODataRequestError(`driver-odata: ${message}`, 400); +} + +const isRecord = (value: unknown): value is Row => !!value && typeof value === 'object' && !Array.isArray(value); + +export class ODataDriver implements IDataDriver { + readonly name: string; + readonly version = '1.0.0'; + /** + * No pushdown claim: `DriverCapabilities` has none to make, and the engine + * floors every part of the query on a plugin driver. A REST transport has + * no transactions. + */ + readonly supports = { transactionsUnsupported: true }; + + private readonly dialect: ODataDialect; + private readonly transport: ODataTransport; + private readonly declarations = new Map(); + private readonly bindings = new Map(); + private readonly etags = new Map(); + private model?: Promise; + + constructor(config: ODataDriverConfig, runtime: ODataDriverRuntime = {}) { + const dialect = DIALECTS[config.version]; + if (!dialect) throw new Error(`driver-odata: OData version '${config.version}' has no dialect in this package`); + this.name = config.name; + this.dialect = dialect; + this.transport = new ODataTransport({ + serviceRoot: config.serviceRoot, + dialect, + auth: config.auth, + requestTimeoutMs: config.requestTimeoutMs, + pageSize: config.pageSize, + ...(runtime.fetch ? { fetch: runtime.fetch } : {}), + ...(runtime.credential ? { credential: runtime.credential } : {}), + }); + } + + // --------------------------------------------------------------------------- + // Lifecycle + // --------------------------------------------------------------------------- + + /** + * Nothing remote happens here. The credential is the datasource's + * `external.credentialsRef`, resolved at the first request: at `connect()` + * the datasource definitions may not be registered yet, and a boot must not + * record "no credential" from a registry that is still filling. + */ + async connect(): Promise {} + + async disconnect(): Promise { + this.model = undefined; + this.bindings.clear(); + this.etags.clear(); + this.transport.reset(); + } + + /** Answers whether the service document is served. */ + async checkHealth(): Promise { + try { + await this.transport.request({ method: 'GET', url: this.transport.root, subject: 'service document' }); + return true; + } catch { + return false; + } + } + + // --------------------------------------------------------------------------- + // Schema + // --------------------------------------------------------------------------- + + /** The service's `$metadata`, read once (a failed read is retried by the next call). */ + getServiceModel(): Promise { + if (!this.model) { + const pending = this.transport + .request({ method: 'GET', url: this.transport.resolve('$metadata'), subject: '$metadata', text: true }) + .then((response) => { + if (typeof response.body !== 'string') throw new Error('driver-odata: $metadata answered no document'); + return this.dialect.parseMetadata(response.body); + }); + this.model = pending; + pending.catch(() => { if (this.model === pending) this.model = undefined; }); + } + return this.model; + } + + /** + * Record an external object's binding (ADR-0015): its entity set + * (`external.remoteName`, the object name when absent) and its + * `external.columnMap`. Pure bookkeeping — the engine re-drives it after + * every metadata reload. + */ + registerExternalObject(schema: unknown): void { + if (!isRecord(schema) || typeof schema.name !== 'string') return; + const external = isRecord(schema.external) ? schema.external : {}; + const columnMap: Record = {}; + if (isRecord(external.columnMap)) { + for (const [remote, local] of Object.entries(external.columnMap)) { + if (typeof local === 'string') columnMap[remote] = local; + } + } + this.declarations.set(schema.name, { + ...(typeof external.remoteName === 'string' ? { remoteName: external.remoteName } : {}), + columnMap, + }); + this.bindings.delete(schema.name); + } + + /** External: the service owns its schema, so there is no DDL to run. */ + async syncSchema(_object: string, _schema: unknown, _options?: DriverOptions): Promise {} + + /** + * The service's entity sets as tables (ADR-0015 introspection): each + * structural property a column typed by its EDM type name, the key as the + * primary key, and single-valued navigation properties listed apart as + * lookup candidates. + */ + async introspectSchema(): Promise { + const model = await this.getServiceModel(); + const tables: Record = {}; + for (const set of model.entitySets.values()) { + const type = model.entityTypes.get(set.entityType); + if (!type) continue; + tables[set.name] = { + name: set.name, + entityType: type.fullName, + columns: type.properties.map((p) => ({ + name: p.name, + type: p.isCollection ? `Collection(${p.type})` : p.type, + nullable: p.nullable, + primaryKey: type.key.includes(p.name), + })), + navigationProperties: type.navigationProperties + .filter((nav) => !nav.isCollection) + .map((nav) => ({ + name: nav.name, + targetEntityType: nav.type, + ...(set.navigationBindings[nav.name] ? { targetEntitySet: set.navigationBindings[nav.name] } : {}), + nullable: nav.nullable, + })), + }; + } + return { tables, dialect: `odata-${this.dialect.version}`, introspectedAt: new Date().toISOString() }; + } + + // --------------------------------------------------------------------------- + // Reads + // --------------------------------------------------------------------------- + + async find(object: string, query: DriverQuery = {}, _options?: DriverOptions): Promise { + refuseReadableIds(query.where); + const binding = await this.bind(object); + const limit = typeof query.limit === 'number' ? query.limit : undefined; + const offset = typeof query.offset === 'number' ? query.offset : undefined; + if (limit === 0) return []; + + const params: Record = {}; + const { filter } = translateWhere(query.where, binding, this.dialect); + if (filter !== null) params.$filter = filter; + + const paged = limit !== undefined || offset !== undefined; + const orderby = this.orderBy(binding, query.orderBy, paged); + if (orderby) params.$orderby = orderby; + + const select = this.select(binding, query.fields); + if (select) params.$select = select; + if (offset !== undefined && offset > 0) params.$skip = String(offset); + if (limit !== undefined) params.$top = String(limit); + + const rows: Row[] = []; + let url: URL | undefined = this.collectionUrl(binding, params); + while (url) { + const response: ODataResponse = await this.transport.request({ method: 'GET', url, subject: binding.entitySet.name }); + const page = this.dialect.readCollection(response.body); + for (const entity of page.rows) rows.push(this.toRow(binding, entity)); + if (limit !== undefined && rows.length >= limit) { + rows.length = limit; + break; + } + url = page.nextLink ? this.transport.followLink(page.nextLink, url) : undefined; + } + return rows; + } + + async findOne(object: string, query: DriverQuery = {}, options?: DriverOptions): Promise { + return (await this.find(object, { ...query, limit: 1 }, options))[0] ?? null; + } + + async count(object: string, query: DriverQuery = {}, _options?: DriverOptions): Promise { + refuseReadableIds(query.where); + const binding = await this.bind(object); + const { filter, complete } = translateWhere(query.where, binding, this.dialect); + if (!complete) { + throw refusal( + `count on '${object}' carries a predicate this driver cannot send to the service, ` + + 'so the service cannot count the rows it selects', + ); + } + const params: Record = { ...this.dialect.countOptions(), $top: '0' }; + if (filter !== null) params.$filter = filter; + const url = this.collectionUrl(binding, params); + const response = await this.transport.request({ method: 'GET', url, subject: binding.entitySet.name }); + const { count } = this.dialect.readCollection(response.body); + if (count === undefined) throw new Error(`driver-odata: the service answered a count on '${binding.entitySet.name}' without one`); + return count; + } + + // --------------------------------------------------------------------------- + // Writes (ADR-0015: the engine has checked `allowWrites` and `writable`) + // --------------------------------------------------------------------------- + + async create(object: string, data: Row, _options?: DriverOptions): Promise { + const binding = await this.bind(object); + const url = this.transport.resolve(encodeURIComponent(binding.entitySet.name)); + const response = await this.transport.request({ + method: 'POST', + url, + subject: binding.entitySet.name, + body: this.toRemote(binding, data, true), + headers: { Prefer: 'return=representation' }, + }); + if (isRecord(response.body)) return this.toRow(binding, this.dialect.readEntity(response.body)); + const location = response.headers.get('OData-EntityId') ?? response.headers.get('Location'); + if (location) { + const created = await this.transport.request({ + method: 'GET', url: this.transport.followLink(location, url), subject: binding.entitySet.name, + }); + return this.toRow(binding, this.dialect.readEntity(created.body)); + } + throw new Error(`driver-odata: the service created a '${binding.entitySet.name}' entity without returning it or its location`); + } + + async update(object: string, id: string | number, data: Row, _options?: DriverOptions): Promise { + const binding = await this.bind(object); + const url = this.entityUrl(binding, id); + if (!url) return null; + const response = await this.transport.request({ + method: 'PATCH', + url, + subject: binding.entitySet.name, + body: this.toRemote(binding, data, false), + headers: { Prefer: 'return=representation', ...this.ifMatch(object, id) }, + accept: [404], + }); + if (response.status === 404) return null; + if (isRecord(response.body)) return this.toRow(binding, this.dialect.readEntity(response.body)); + const reread = await this.transport.request({ method: 'GET', url, subject: binding.entitySet.name, accept: [404] }); + return reread.status === 404 ? null : this.toRow(binding, this.dialect.readEntity(reread.body)); + } + + async delete(object: string, id: string | number, _options?: DriverOptions): Promise { + const binding = await this.bind(object); + const url = this.entityUrl(binding, id); + if (!url) return false; + const response = await this.transport.request({ + method: 'DELETE', + url, + subject: binding.entitySet.name, + headers: this.ifMatch(object, id), + accept: [404], + }); + this.etags.delete(etagKey(object, id)); + return response.status !== 404; + } + + upsert = refused('upsert (OData has no conflict-keyed write; create or update by key)'); + bulkCreate = refused('bulkCreate (no $batch support yet; write rows one at a time)'); + bulkUpdate = refused('bulkUpdate (no $batch support yet; write rows one at a time)'); + bulkDelete = refused('bulkDelete (no $batch support yet; delete rows one at a time)'); + execute = refused('execute (an OData service runs no raw commands)'); + beginTransaction = refused('beginTransaction (an OData service offers no transactions)'); + commit = refused('commit (an OData service offers no transactions)'); + rollback = refused('rollback (an OData service offers no transactions)'); + dropTable = refused('dropTable (the service owns its schema)'); + + // --------------------------------------------------------------------------- + // Internals + // --------------------------------------------------------------------------- + + /** Resolve an object against the service model: its entity set, type, key and field names. */ + private async bind(object: string): Promise { + const cached = this.bindings.get(object); + if (cached) return cached; + const model = await this.getServiceModel(); + const declaration = this.declarations.get(object) ?? { columnMap: {} }; + const setName = declaration.remoteName ?? object; + const entitySet = model.entitySets.get(setName); + if (!entitySet) { + throw new Error( + `driver-odata: object '${object}' is bound to entity set '${setName}', which the service's $metadata does not declare`, + ); + } + const entityType = model.entityTypes.get(entitySet.entityType); + if (!entityType) { + throw new Error(`driver-odata: entity set '${setName}' names entity type '${entitySet.entityType}', which $metadata does not declare`); + } + const byName = new Map(entityType.properties.map((p) => [p.name, p])); + const keyProperties = entityType.key.map((name) => byName.get(name)).filter((p): p is EdmProperty => !!p); + if (keyProperties.length === 0 || keyProperties.length !== entityType.key.length) { + throw new Error(`driver-odata: entity type '${entityType.fullName}' declares no usable key`); + } + + const fieldOfProperty = new Map(); + const propertyOfField = new Map(); + for (const property of entityType.properties) { + const field = declaration.columnMap[property.name] ?? property.name; + fieldOfProperty.set(property.name, field); + propertyOfField.set(field, property); + } + // `id` names the key, unless the column map gave the name to a property of its own. + const idIsKeyAlias = keyProperties.length === 1 && !propertyOfField.has('id'); + if (idIsKeyAlias) propertyOfField.set('id', keyProperties[0]!); + + const binding: Binding = { + object, + model, + entitySet, + entityType, + keyProperties, + fieldOfProperty, + idIsKeyAlias, + propertyOf: (field) => propertyOfField.get(field), + }; + this.bindings.set(object, binding); + return binding; + } + + private collectionUrl(binding: Binding, params: Record): URL { + const query = Object.entries(params).map(([k, v]) => `${k}=${encodeURIComponent(v)}`).join('&'); + return this.transport.resolve(`${encodeURIComponent(binding.entitySet.name)}${query ? `?${query}` : ''}`); + } + + /** `()`, or `undefined` when `id` cannot be a key of this entity type. */ + private entityUrl(binding: Binding, id: string | number): URL | undefined { + const predicate = this.keyPredicate(binding, id); + return predicate === undefined + ? undefined + : this.transport.resolve(`${encodeURIComponent(binding.entitySet.name)}(${predicate})`); + } + + private keyPredicate(binding: Binding, id: string | number): string | undefined { + const { keyProperties, model } = binding; + if (keyProperties.length === 1) { + const key = keyProperties[0]!; + const literal = this.dialect.literal(key.type === 'Edm.String' ? String(id) : id, key.type, model); + return literal === undefined ? undefined : encodeURIComponent(literal); + } + // Composite: `id` is the predicate body this driver wrote (`A=1,B='x'`); each part is re-spelled. + const parts = splitPredicate(String(id)); + if (!parts || parts.length !== keyProperties.length) return undefined; + const spelled: string[] = []; + for (const key of keyProperties) { + const raw = parts.find(([name]) => name === key.name)?.[1]; + if (raw === undefined) return undefined; + const value = key.type === 'Edm.String' ? unquote(raw) : raw; + const literal = value === undefined ? undefined : this.dialect.literal(value, key.type, model); + if (literal === undefined) return undefined; + spelled.push(`${key.name}=${encodeURIComponent(literal)}`); + } + return spelled.join(','); + } + + /** The row's `id`: the key's value, or the composite key's predicate body. */ + private idOf(binding: Binding, entity: Row): string { + const { keyProperties, model } = binding; + const values = keyProperties.map((key) => entity[key.name]); + if (values.some((v) => v === undefined || v === null)) { + throw new Error(`driver-odata: an entity of '${binding.entitySet.name}' arrived without its key`); + } + if (keyProperties.length === 1) return String(values[0]); + return keyProperties + .map((key, i) => `${key.name}=${this.dialect.literal(values[i], key.type, model) ?? String(values[i])}`) + .join(','); + } + + private toRow(binding: Binding, entity: Row): Row { + const row: Row = {}; + for (const [key, value] of Object.entries(entity)) { + if (this.dialect.isAnnotation(key)) continue; + row[binding.fieldOfProperty.get(key) ?? key] = value; + } + row.id = this.idOf(binding, entity); + const etag = this.dialect.etagOf(entity); + if (etag) this.rememberEtag(binding.object, row.id as string, etag); + return row; + } + + /** The request body: declared properties only, renamed to the service's names; the key only on create. */ + private toRemote(binding: Binding, data: Row, withKey: boolean): Row { + const body: Row = {}; + const keys = new Set(binding.keyProperties.map((k) => k.name)); + for (const [field, value] of Object.entries(data)) { + // `id` as the key's alias is the engine's name for the row, not a property to send: a key + // the service expects from the caller arrives through the field its property maps to. + if (field === 'id' && binding.idIsKeyAlias) continue; + const property = binding.propertyOf(field); + if (!property) continue; + if (keys.has(property.name) && !withKey) continue; + body[property.name] = value; + } + return body; + } + + private orderBy( + binding: Binding, + orderBy: DriverQuery['orderBy'], + paged: boolean, + ): string | undefined { + const terms: string[] = []; + const named = new Set(); + for (const term of orderBy ?? []) { + const field = (term as { field?: unknown }).field; + const property = typeof field === 'string' ? binding.propertyOf(field) : undefined; + // One field the service cannot sort by: send no order at all rather than a different one. + if (!property || property.isCollection) { + if (paged) throw refusal(`a paged read of '${binding.object}' sorts by '${String(field)}', which the service cannot sort by`); + return undefined; + } + const direction = (term as { order?: unknown }).order === 'desc' ? 'desc' : 'asc'; + terms.push(`${property.name} ${direction}`); + named.add(property.name); + } + if (paged) { + for (const key of binding.keyProperties) if (!named.has(key.name)) terms.push(`${key.name} asc`); + } + return terms.length > 0 ? terms.join(',') : undefined; + } + + private select(binding: Binding, fields: DriverQuery['fields']): string | undefined { + if (!Array.isArray(fields) || fields.length === 0) return undefined; + const names = new Set(binding.keyProperties.map((k) => k.name)); + for (const field of fields) { + const property = typeof field === 'string' ? binding.propertyOf(field) : undefined; + // A field the service does not carry: select everything rather than drop it. + if (!property) return undefined; + names.add(property.name); + } + return [...names].join(','); + } + + private ifMatch(object: string, id: string | number): Record { + const etag = this.etags.get(etagKey(object, id)); + return etag ? { 'If-Match': etag } : {}; + } + + private rememberEtag(object: string, id: string, etag: string): void { + const key = etagKey(object, id); + this.etags.delete(key); + this.etags.set(key, etag); + if (this.etags.size > ETAG_CACHE_LIMIT) { + const oldest = this.etags.keys().next().value; + if (oldest !== undefined) this.etags.delete(oldest); + } + } +} + +/** + * A `ReadableIds` leaf (ADR-0139 D3) is a read-scope predicate: "the ids of X + * this caller may read". `$filter` has no tenant-walled semi-join to spell it + * with, and leaving it out — the translator's answer to anything it cannot + * spell — would read it as no restriction. So it is refused, loudly, before + * anything is sent. + */ +function refuseReadableIds(where: unknown): void { + const site = findMisplacedReadableIds(where, false); + if (site) throw readableIdsDriverRefusal('driver-odata', site, 'an OData $filter has no form of the tenant-walled semi-join'); +} + +const etagKey = (object: string, id: string | number) => `${object}\u0000${String(id)}`; + +/** A verb this driver does not offer, refused by name (a request-shaped refusal, never an outage). */ +function refused(what: string) { + return async (..._args: unknown[]): Promise => { + throw refusal(`${what} is not offered by the OData driver`); + }; +} + +/** `A=1,B='x,y'` → `[['A','1'],['B',"'x,y'"]]`, or `undefined` when it is not a predicate body. */ +function splitPredicate(text: string): Array<[string, string]> | undefined { + const parts: string[] = []; + let current = ''; + let inQuote = false; + for (let i = 0; i < text.length; i++) { + const ch = text[i]!; + if (ch === "'" && inQuote && text[i + 1] === "'") { + current += "''"; + i++; + continue; + } + if (ch === "'") inQuote = !inQuote; + if (ch === ',' && !inQuote) { + parts.push(current); + current = ''; + continue; + } + current += ch; + } + if (inQuote) return undefined; + parts.push(current); + const pairs: Array<[string, string]> = []; + for (const part of parts) { + const eq = part.indexOf('='); + if (eq <= 0) return undefined; + pairs.push([part.slice(0, eq), part.slice(eq + 1)]); + } + return pairs; +} + +/** `'O''Neil'` → `O'Neil`; `undefined` when not a quoted literal. */ +function unquote(literal: string): string | undefined { + if (literal.length < 2 || !literal.startsWith("'") || !literal.endsWith("'")) return undefined; + return literal.slice(1, -1).replace(/''/g, "'"); +} diff --git a/packages/drivers/driver-odata/src/plugin.test.ts b/packages/drivers/driver-odata/src/plugin.test.ts new file mode 100644 index 00000000000..ea969803de0 --- /dev/null +++ b/packages/drivers/driver-odata/src/plugin.test.ts @@ -0,0 +1,82 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * The plugin: what it publishes, and where the credential comes from — the + * datasource's `external.credentialsRef`, dereferenced through the host's + * secret store at the first request, failing closed every way. + */ + +import { describe, expect, it } from 'vitest'; +import type { PluginContext } from '@objectstack/core'; +import { needsExternalDriverFloor } from '@objectstack/objectql'; +import { createODataDriverPlugin, type DatasourceDefinitionSource, type ODataDriverPluginOptions } from './plugin.js'; +import { METADATA_EXCHANGE, SERVICE_ROOT, page, recordedService } from './__fixtures__/recorded-service.js'; + +type Def = ReturnType[number]; + +async function boot(options: Omit, defs: Def[]) { + const service = recordedService([METADATA_EXCHANGE, { url: 'People', body: page([]) }]); + const plugin = createODataDriverPlugin({ serviceRoot: SERVICE_ROOT, fetch: service.fetch, ...options }); + const services = new Map([['objectql', { listDatasourceDefs: () => defs } satisfies DatasourceDefinitionSource]]); + const ctx = { + registerService: (name: string, value: unknown) => { services.set(name, value); }, + getService: (name: string) => services.get(name), + } as unknown as PluginContext; + await plugin.init!(ctx); + plugin.driver.registerExternalObject({ name: 'trip_person', external: { remoteName: 'People' } }); + return { plugin, services, requests: service.requests }; +} + +const SECRETS = { + async resolve(ref: string) { + return ref === 'sys_secret:sec_trippin' ? 'token-from-the-store' : undefined; + }, +}; + +describe('createODataDriverPlugin', () => { + it('publishes the driver as driver. — `odata` by default, a plugin driver id the engine floors', async () => { + const { services, plugin } = await boot({}, []); + expect(services.get('driver.odata')).toBe(plugin.driver); + expect(plugin.driver.name).toBe('odata'); + expect(needsExternalDriverFloor({ driver: 'odata', schemaMode: 'external' })).toBe(true); + }); + + it('refuses a config that carries a credential or names no http(s) service', () => { + expect(() => createODataDriverPlugin({ serviceRoot: 'https://user:pw@erp.example/odata/' })) + .toThrow(/serviceRoot must not carry credentials/); + expect(() => createODataDriverPlugin({ serviceRoot: 'ftp://erp.example/odata/' })).toThrow(/http\(s\) URL/); + expect(() => createODataDriverPlugin({ serviceRoot: SERVICE_ROOT, token: 'x' } as never)).toThrow(/invalid config/); + }); + + it('sends the secret behind the datasource\'s credentialsRef, resolved through the host\'s secret store', async () => { + const { plugin, requests } = await boot( + { name: 'trippin', auth: { scheme: 'bearer' }, secrets: SECRETS }, + [{ name: 'trippin', external: { credentialsRef: 'sys_secret:sec_trippin' } }], + ); + await plugin.driver.find('trip_person', {}); + expect(requests.map((r) => r.headers.Authorization)).toEqual(['Bearer token-from-the-store', 'Bearer token-from-the-store']); + }); + + it('spells basic auth from the configured user name and the resolved secret', async () => { + const { plugin, requests } = await boot( + { name: 'trippin', auth: { scheme: 'basic', username: 'svc_reader' }, secrets: SECRETS }, + [{ name: 'trippin', external: { credentialsRef: 'sys_secret:sec_trippin' } }], + ); + await plugin.driver.find('trip_person', {}); + expect(requests[0]!.headers.Authorization).toBe(`Basic ${Buffer.from('svc_reader:token-from-the-store').toString('base64')}`); + }); + + const refusals: Array<[string, Omit, Def[], RegExp]> = [ + ['no credentialsRef for a scheme that needs one', { name: 'trippin', auth: { scheme: 'bearer' }, secrets: SECRETS }, [{ name: 'trippin' }], /declares no external\.credentialsRef/], + ['no secret store', { name: 'trippin', auth: { scheme: 'bearer' } }, [{ name: 'trippin', external: { credentialsRef: 'sys_secret:sec_trippin' } }], /no secret store was passed/], + ['a handle that does not resolve', { name: 'trippin', auth: { scheme: 'bearer' }, secrets: SECRETS }, [{ name: 'trippin', external: { credentialsRef: 'sys_secret:gone' } }], /could not be resolved or decrypted/], + ['an undeclared datasource', { name: 'trippin', auth: { scheme: 'bearer' }, secrets: SECRETS }, [], /is not declared/], + ['a credentialsRef the scheme would ignore', { name: 'trippin', secrets: SECRETS }, [{ name: 'trippin', external: { credentialsRef: 'sys_secret:sec_trippin' } }], /auth scheme is 'none'/], + ]; + + it.each(refusals)('fails closed, sending nothing, on %s', async (_label, options, defs, message) => { + const { plugin, requests } = await boot(options, defs); + await expect(plugin.driver.find('trip_person', {})).rejects.toThrow(message); + expect(requests).toHaveLength(0); + }); +}); diff --git a/packages/drivers/driver-odata/src/plugin.ts b/packages/drivers/driver-odata/src/plugin.ts new file mode 100644 index 00000000000..3af587fc77c --- /dev/null +++ b/packages/drivers/driver-odata/src/plugin.ts @@ -0,0 +1,111 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * The server plugin that publishes an {@link ODataDriver} as the kernel + * service `driver.` — a PLUGIN driver id, deliberately not a member of + * the spec's `DRIVER_VOCABULARY`. On a datasource with `schemaMode: + * 'external'` the engine therefore floors it (`needsExternalDriverFloor`): + * whatever the driver does not narrow, the engine applies in memory, under + * its row cap. + * + * ## The credential + * + * The secret is the datasource's `external.credentialsRef` — never a key of + * the config, never inline. The plugin finds the datasource definition in the + * engine (`listDatasourceDefs()`) and dereferences the handle through the + * host's datasource secret store: the same `SecretBinder` the host gives + * `DatasourceAdminServicePlugin` (its `resolve`). It does so at the FIRST + * request, not at boot, so it never reads the definition registry while it is + * still filling; a 401 makes it resolve again, which picks up a rotated + * secret. + * + * Fail-closed, every way: an auth scheme with no `credentialsRef` on the + * datasource, a `credentialsRef` with no secret store, a handle that does not + * resolve, and a `credentialsRef` declared on a datasource whose scheme is + * `none` all refuse the request (the engine answers 503 and logs the cause) + * instead of sending it unauthenticated. + */ + +import type { Plugin, PluginContext } from '@objectstack/core'; +import { parseODataDriverConfig, type ODataDriverConfigInput } from './config.js'; +import { ODataDriver } from './odata-driver.js'; +import type { FetchLike } from './transport.js'; + +/** The host's datasource secret store: the read half of `SecretBinder`. */ +export interface ODataSecretStore { + resolve(credentialsRef: string): Promise; +} + +/** The one engine read this plugin makes: the declared datasource definitions. */ +export interface DatasourceDefinitionSource { + listDatasourceDefs(): Array<{ name: string; external?: { credentialsRef?: string } }>; +} + +export interface ODataDriverPluginOptions extends ODataDriverConfigInput { + /** The secret store `external.credentialsRef` resolves through. Required for `bearer` and `basic`. */ + secrets?: ODataSecretStore; + /** The `fetch` the driver calls; the global one when absent. */ + fetch?: FetchLike; +} + +/** The engine's service name. */ +const ENGINE_SERVICE = 'objectql'; + +export function createODataDriverPlugin(options: ODataDriverPluginOptions): Plugin & { readonly driver: ODataDriver } { + const { secrets, fetch, ...input } = options; + const config = parseODataDriverConfig(input); + let ctxRef: PluginContext | undefined; + + const credential = async (): Promise => { + if (!ctxRef) throw new Error(`driver-odata: datasource '${config.name}' was used before its plugin initialised`); + const engine = ctxRef.getService(ENGINE_SERVICE); + if (!engine || typeof engine.listDatasourceDefs !== 'function') { + throw new Error(`driver-odata: datasource '${config.name}': the engine offers no datasource definitions to read its credentialsRef from`); + } + const def = engine.listDatasourceDefs().find((d) => d.name === config.name); + if (!def) { + throw new Error(`driver-odata: datasource '${config.name}' is not declared, so it has no credentialsRef`); + } + const ref = def.external?.credentialsRef; + if (config.auth.scheme === 'none') { + if (ref) { + throw new Error( + `driver-odata: datasource '${config.name}' declares external.credentialsRef but its auth scheme is 'none'; ` + + "set auth.scheme to 'bearer' or 'basic', or remove the credentialsRef", + ); + } + return undefined; + } + if (!ref) { + throw new Error( + `driver-odata: datasource '${config.name}' uses auth scheme '${config.auth.scheme}' and declares no external.credentialsRef`, + ); + } + if (!secrets) { + throw new Error( + `driver-odata: datasource '${config.name}' declares a credentialsRef, but no secret store was passed to the plugin (options.secrets)`, + ); + } + const secret = await secrets.resolve(ref); + if (!secret) { + throw new Error(`driver-odata: datasource '${config.name}': its credentialsRef could not be resolved or decrypted`); + } + return secret; + }; + + const driver = new ODataDriver(config, { + ...(fetch ? { fetch } : {}), + credential, + }); + + return { + name: `com.objectstack.driver-odata.${config.name}`, + version: '1.0.0', + driver, + // Registered in init, not start: the engine adopts `driver.*` services once every plugin has initialised. + init: async (ctx: PluginContext) => { + ctxRef = ctx; + ctx.registerService(`driver.${config.name}`, driver); + }, + }; +} diff --git a/packages/drivers/driver-odata/src/transport.ts b/packages/drivers/driver-odata/src/transport.ts new file mode 100644 index 00000000000..6ba9e313c5f --- /dev/null +++ b/packages/drivers/driver-odata/src/transport.ts @@ -0,0 +1,202 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * HTTP to one OData service: authentication, timeout, same-origin links and + * the error shape the engine's floor reads. + * + * ## What a failure looks like to the engine + * + * The floor (objectql `external-driver-floor.ts`) reads `status` on a thrown + * error. So a response the service REFUSED is an {@link ODataRequestError} + * carrying the HTTP status and the service's own message (`error.message` of + * the OData error body): a 400 / 404 / 422 on a read reaches the caller as + * `400 INVALID_QUERY`, a request-shaped 4xx on a write as + * `422 EXTERNAL_WRITE_REFUSED`, and anything else — a 401, a 429, a 5xx — as + * `503 ERR_DATASOURCE_UNAVAILABLE` with this message only in the server log. + * A network failure, a timeout, and a credential that cannot be resolved throw + * with no status at all, which the floor also answers 503. + * + * ⛔ No message built here carries the service URL, a header or a credential: + * a read refusal's message may be shown to the caller. + */ + +import type { ODataDialect } from './dialect.js'; + +/** The `fetch` this transport calls — the global one unless the host passes its own. */ +export type FetchLike = (input: string, init: RequestInit) => Promise; + +/** The auth scheme, as the config names it. */ +export type ODataAuth = { scheme: 'none' } | { scheme: 'bearer' } | { scheme: 'basic'; username: string }; + +/** A refusal or failure answered by the service, with the HTTP status the engine's floor reads. */ +export class ODataRequestError extends Error { + readonly status: number; + /** The OData error body's `code`, when the service sent one. Not an ADR-0112 code. */ + readonly odataCode?: string; + + constructor(message: string, status: number, odataCode?: string) { + super(message); + this.name = 'ODataRequestError'; + this.status = status; + if (odataCode !== undefined) this.odataCode = odataCode; + } +} + +export interface ODataTransportOptions { + serviceRoot: string; + dialect: ODataDialect; + auth: ODataAuth; + requestTimeoutMs: number; + pageSize: number; + fetch?: FetchLike; + /** + * The secret behind the datasource's `external.credentialsRef`. Called at + * the first request that needs it and again after a 401, never at boot. + */ + credential?: () => Promise; +} + +export interface ODataResponse { + status: number; + headers: Headers; + /** The parsed JSON body; the text when the body is not JSON; `undefined` when empty. */ + body: unknown; +} + +export interface ODataRequest { + method: 'GET' | 'POST' | 'PATCH' | 'DELETE'; + url: URL; + /** What the request is about, for messages: an entity set name or `$metadata`. */ + subject: string; + body?: unknown; + headers?: Record; + /** Answer these statuses instead of throwing (e.g. 404 on a by-key read). */ + accept?: readonly number[]; + /** Read the body as text, not JSON (`$metadata`). */ + text?: boolean; +} + +export class ODataTransport { + readonly root: URL; + private readonly opts: ODataTransportOptions; + private readonly fetchImpl: FetchLike; + private secret?: Promise; + + constructor(opts: ODataTransportOptions) { + this.opts = opts; + this.root = new URL(opts.serviceRoot.endsWith('/') ? opts.serviceRoot : `${opts.serviceRoot}/`); + const fetchImpl = opts.fetch ?? (globalThis.fetch as FetchLike | undefined); + if (typeof fetchImpl !== 'function') { + throw new Error('driver-odata: no fetch implementation is available; pass `fetch` in the driver options'); + } + this.fetchImpl = fetchImpl; + } + + /** A URL under the service root. */ + resolve(path: string): URL { + return new URL(path, this.root); + } + + /** + * A link the service returned (`@odata.nextLink`, `Location`), resolved + * against the request it came with. ⛔ A link to another origin is refused: + * following it would send this datasource's credential to a host the + * operator never configured. + */ + followLink(link: string, base: URL): URL { + const url = new URL(link, base); + if (url.origin !== this.root.origin) { + throw new Error('driver-odata: the service returned a link to another origin; it is not followed'); + } + return url; + } + + /** Forget the resolved credential, so the next request resolves it again. */ + reset(): void { + this.secret = undefined; + } + + async request(req: ODataRequest): Promise { + let response = await this.send(req); + if (response.status === 401 && this.opts.auth.scheme !== 'none') { + // The credential may have been rotated in the secret store: resolve it once more. + this.reset(); + response = await this.send(req); + } + const body = await readBody(response, req.text === true); + if (response.ok || req.accept?.includes(response.status)) { + return { status: response.status, headers: response.headers, body }; + } + const { code, message } = this.opts.dialect.readError(body); + throw new ODataRequestError( + `OData service refused ${req.method} on '${req.subject}' (HTTP ${response.status})` + + (message ? `: ${message}` : ''), + response.status, + code, + ); + } + + private async send(req: ODataRequest): Promise { + const headers: Record = { + ...this.opts.dialect.requestHeaders(), + ...(req.text ? { Accept: 'application/xml' } : {}), + ...(req.method === 'GET' && !req.text ? { Prefer: `odata.maxpagesize=${this.opts.pageSize}` } : {}), + ...(req.body !== undefined ? { 'Content-Type': 'application/json' } : {}), + ...req.headers, + }; + const authorization = await this.authorization(); + if (authorization) headers.Authorization = authorization; + try { + return await this.fetchImpl(req.url.toString(), { + method: req.method, + headers, + ...(req.body !== undefined ? { body: JSON.stringify(req.body) } : {}), + redirect: 'manual', + signal: AbortSignal.timeout(this.opts.requestTimeoutMs), + }); + } catch (error) { + const reason = error instanceof Error && error.name === 'TimeoutError' + ? `timed out after ${this.opts.requestTimeoutMs} ms` + : 'could not be reached'; + throw new Error(`driver-odata: ${req.method} on '${req.subject}' ${reason}`, { cause: error }); + } + } + + private async authorization(): Promise { + const { auth } = this.opts; + if (!this.opts.credential) { + if (auth.scheme === 'none') return undefined; + throw new Error( + `driver-odata: auth scheme '${auth.scheme}' needs the datasource's external.credentialsRef, ` + + 'and no credential source was wired', + ); + } + // Asked even under `none`, so a source can refuse a declared credential the scheme would ignore. + if (!this.secret) { + const pending = this.opts.credential(); + this.secret = pending; + // A failed resolution is not cached: the next request tries again. + pending.catch(() => { if (this.secret === pending) this.secret = undefined; }); + } + const secret = await this.secret; + if (auth.scheme === 'none') return undefined; + if (!secret) { + this.secret = undefined; + throw new Error(`driver-odata: the credential for auth scheme '${auth.scheme}' resolved to nothing`); + } + return auth.scheme === 'bearer' + ? `Bearer ${secret}` + : `Basic ${Buffer.from(`${auth.username}:${secret}`, 'utf8').toString('base64')}`; + } +} + +async function readBody(response: Response, asText: boolean): Promise { + const text = await response.text(); + if (text === '') return undefined; + if (asText) return text; + try { + return JSON.parse(text); + } catch { + return text; + } +} diff --git a/packages/drivers/driver-odata/tsconfig.json b/packages/drivers/driver-odata/tsconfig.json new file mode 100644 index 00000000000..b644b61c34c --- /dev/null +++ b/packages/drivers/driver-odata/tsconfig.json @@ -0,0 +1,14 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + // module re-set from the base's NodeNext: no "type": "module" here, so NodeNext would compile src as CommonJS and refuse its `import.meta` (TS1470, measured). + "module": "ES2020", + "moduleResolution": "bundler", + "outDir": "./dist", + "types": ["node"], + "lib": ["ES2022"], + "rootDir": "./src" + }, + "include": ["src/**/*"], + "exclude": ["node_modules", "dist"] +} diff --git a/packages/drivers/driver-odata/vitest.config.ts b/packages/drivers/driver-odata/vitest.config.ts new file mode 100644 index 00000000000..1398455e8e5 --- /dev/null +++ b/packages/drivers/driver-odata/vitest.config.ts @@ -0,0 +1,32 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +import { defineConfig } from 'vitest/config'; +import path from 'path'; + +export default defineConfig({ + test: { + // A late console.* must not redden a green suite (#10374): vitest's worker + // forwards console output over RPC and discards the promise, and a write + // landing after teardown's rpcDone() snapshot is rejected into an unhandled + // error — a fully green run that exits 1. Disarming removes the mechanism. + // Mechanism + measured costs: examples/app-showcase/vitest.config.ts. + // Enforced repo-wide by scripts/check-console-intercept-disarm.mjs. + disableConsoleIntercept: true, + environment: 'node', + env: { OS_REGISTRY_LOG: 'warn' }, + }, + resolve: { + // The tests are a verdict about the source in the checkout, never a sibling's build + // (scripts/check-test-source-alias.mjs). Each spec namespace is `src//index.ts`. + alias: [ + { find: /^@objectstack\/core$/, replacement: path.resolve(__dirname, '../../core/src/index.ts') }, + { find: /^@objectstack\/objectql$/, replacement: path.resolve(__dirname, '../../objectql/src/index.ts') }, + { + find: /^@objectstack\/spec\/([a-z-]+)$/, + replacement: `${path.resolve(__dirname, '../../spec/src')}/$1/index.ts`, + }, + { find: /^@objectstack\/spec$/, replacement: path.resolve(__dirname, '../../spec/src/index.ts') }, + { find: /^@objectstack\/types$/, replacement: path.resolve(__dirname, '../../types/src/index.ts') }, + ], + }, +}); diff --git a/packages/spec/llms.txt b/packages/spec/llms.txt index 653cb904e4d..70c1f893bf4 100644 --- a/packages/spec/llms.txt +++ b/packages/spec/llms.txt @@ -178,9 +178,9 @@ function registerObject(rawConfig: unknown) { --- -## 7. Package Ecosystem (72 packages) +## 7. Package Ecosystem (73 packages) -The workspace publishes 72 packages under the `@objectstack` scope. The table +The workspace publishes 73 packages under the `@objectstack` scope. The table below is a curated entry-point list, not the full set — drivers, connectors, triggers, plugins and kernel-managed services each form their own family. diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 28ec90b597d..2e8147f8f49 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -1064,6 +1064,34 @@ importers: specifier: ^4.1.11 version: 4.1.12(@opentelemetry/api@1.9.1)(@types/node@26.6.5)(@vitest/coverage-v8@4.1.12)(@vitest/ui@4.1.10)(happy-dom@21.0.4)(jsdom@30.1.1(@noble/hashes@2.4.0))(msw@2.15.0(@types/node@26.6.5)(typescript@6.0.3))(vite@8.3.5(@types/node@26.6.5)(esbuild@0.28.2)(jiti@2.7.0)(tsx@4.23.15(patch_hash=4d1d35da1809be2a945519cb26890e48107810b9b66c6913a6a89abb5ad6a1e6))(yaml@2.9.1)) + packages/drivers/driver-odata: + dependencies: + '@objectstack/core': + specifier: workspace:* + version: link:../../core + '@objectstack/spec': + specifier: workspace:* + version: link:../../spec + fast-xml-parser: + specifier: ^5.10.1 + version: 5.10.1 + zod: + specifier: ^4.6.6 + version: 4.6.7 + devDependencies: + '@objectstack/objectql': + specifier: workspace:* + version: link:../../objectql + '@types/node': + specifier: ^26.6.5 + version: 26.6.5 + typescript: + specifier: ^6.0.3 + version: 6.0.3 + vitest: + specifier: ^4.1.11 + version: 4.1.12(@opentelemetry/api@1.9.1)(@types/node@26.6.5)(@vitest/coverage-v8@4.1.12)(@vitest/ui@4.1.10)(happy-dom@21.0.4)(jsdom@30.1.1(@noble/hashes@2.4.0))(msw@2.15.0(@types/node@26.6.5)(typescript@6.0.3))(vite@8.3.5(@types/node@26.6.5)(esbuild@0.28.2)(jiti@2.7.0)(tsx@4.23.15(patch_hash=4d1d35da1809be2a945519cb26890e48107810b9b66c6913a6a89abb5ad6a1e6))(yaml@2.9.1)) + packages/drivers/driver-sql: dependencies: '@objectstack/core':