Repository navigation
feat(connector-openapi): an offline normalize-openapi script — Swagger 2.0 to 3.0, YAML to JSON, $ref bundled and inlined, absolute server, sha256 and an include list - #27436
Conversation
…ependencies for the offline normalizer Claude-Session: https://claude.ai/code/session_017xRBcVBKAFk1ntXSjvPt7U Co-authored-by: Claude <noreply@anthropic.com>
…er 2.0 to 3.0, YAML to JSON, bundled and dereferenced, absolute server, sha256 and an include list Claude-Session: https://claude.ai/code/session_017xRBcVBKAFk1ntXSjvPt7U Co-authored-by: Claude <noreply@anthropic.com>
…ize-openapi script Claude-Session: https://claude.ai/code/session_017xRBcVBKAFk1ntXSjvPt7U Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift Check
What this run could not see
Coarse fallback — 2 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): |
|
epic:#27293#1 · sunhaolin · session_017xRBcVBKAFk1ntXSjvPt7U 席位核对ACCEPT at head
The dev's three open questions, decided by the seat (implementation parameters, inside triage's ruling 6487372008; no product or contract fork):
Out of scope: Landing: not governed: the seat lands it now ( |
Fixes #27373
Clause-②: no
Metadata-protocol: none
维护者速读
connector-openapi包内新增离线脚本normalize-openapi(只在仓库里,不进 npm 包),把厂商的 Swagger 2.0 / YAML / 多文件 OpenAPI 文档转成openapiprovider 直接能加载的单个 3.x JSON,并打印写入字节的 sha256 和按模式生成的include列表。$ref,运行时按设计拒收 2.0、并静默丢掉$ref形式的参数/请求体/响应;没有这一步,每个作者都得手工转换,而且会悄悄转错。src/与发布的dist/字节不变),只新增两个 devDependency(Scalar 的@scalar/json-magic、@scalar/openapi-upgrader,均 MIT);回滚即 revert 本 PR。scripts/normalize-openapi.mts的Dereferencer.follow/stub(在递归处与--max-depth处把 schema 引用换成带说明的类型桩,这是有损的取舍);②fillEmptyConsumes(空consumes默认拒收);③package.json的依赖与typecheck行。pnpm --filter @objectstack/connector-openapi exec vitest run scripts/normalize-openapi.test.ts,预期Tests 13 passed (13)。What changes
packages/connectors/connector-openapi/scripts/normalize-openapi.mts(new): one script, a small CLI and an exportednormalizeOpenApi():pnpm --filter @objectstack/connector-openapi normalize-openapi --in FILE_OR_URL --out FILE.json [--include-pattern REGEX]... [--max-depth N] [--server-url URL] [--empty-consumes MEDIA_TYPE].$ref(sibling files of a file document, confined to its directory; URLs only for a URL document);$refreachable frompaths/webhooks, then drops thecomponentssections only references used (securitySchemesstay, they are referenced by name);servers[0].urlabsolute (server variables take their defaults; a relative URL resolves against the source URL, or--server-url; a relative URL in a file document is refused until--server-urlis given);--include-pattern, emits theproviderConfig.includelist: every operation whose action key, or whoseMETHOD /path, matches. The key is computed with the provider's ownoperationActionKey(imported fromsrc/openapi-connector.ts, not re-derived), so the list and the provider can never disagree;JSON.stringify(doc, null, 2)plus a newline and prints a JSON summary whosesha256is the hash of exactly those bytes, with the cut counts andrefsLeft.Exit codes: 0 written, 1 refused (message names the remedy), 2 usage.
$refleft" and "every schema fully inlined" cannot both hold for Stripe: measured onspec3.json(3.0.0, 431 paths, 612 operations, 117 back-edges among component schemas), every one of the 612 operations expands past 2,000,000 JSON nodes when only true recursion is cut. So a SCHEMA reference becomes a stub when it is recursive, or deeper than--max-depthnamed-schema hops (default 2); a$refinside a vendorx-extension value is stubbed too, since its target is dropped withcomponents. The stub keeps the target'stype,nullable,title,descriptionand says which reference it replaced and why. Structural references (parameters, responses, request bodies, headers, path items) are always inlined. Literal data (example, a schema'sdefault/enum/const/examples, Example Objects) is copied, never walked; a PROPERTY nameddefaultis still a schema.consumes: []clears the global list, so a faithful conversion gives that request body no media type and therefore no schema: the body would vanish from the action silently. DocuSign's eSignature document does this on 187 of its 414 operations. The script refuses and names the operations;--empty-consumes application/jsonfills them, and the summary counts them.packages/connectors/connector-openapi/tsconfig.scripts.json(new) putsscripts/**in a tsc program, named by the package'stypecheckscript (precedentpackages/spec/tsconfig.scripts.json); without it the new directory would be TypeScript no program reads (check:type-check-coverageSOURCES_COVERED / TESTS_COVERED). This file is outside the claim's declared file surface (one package-root config); it is the only such path.package.json:normalize-openapiscript entry, thetypecheckline, two devDependencies.pnpm-lock.yaml: regenerated bypnpm install --lockfile-only(38 added lines, nothing else moved).content/docs/automation/connectors.mdx: one paragraph and one command where the page says "convert it to OpenAPI 3.x offline".openapi-provider.tsandopenapi-connector.tsare untouched; the provider still refuses every non-3.x document.The new third-party dependencies (read from the npm registry 2026-10-29)
@scalar/openapi-upgraderupgrade(doc, '3.0'))@scalar/helpers,@scalar/openapi-types)@scalar/json-magic$ref(files, URLs, YAML/JSON parse)@scalar/helpers,pathe,undici,yaml)Both live in the
scalar/scalarmonorepo and release roughly weekly (38 and 74 published versions). Both are pre-1.0, so the caret ranges pin the minor. The lockfile adds four packages in all (@scalar/helpers0.16.4 and@scalar/openapi-types0.9.8 transitively;pathe,undici,yamlwere already in the tree). Weekly counts come from the registry search endpoint (registry.npmjs.org/-/v1/search), becauseapi.npmjs.orgis refused by this container's egress proxy.Why two packages and not one:
@scalar/openapi-parser(MIT, 0.29.18, 2026-10-28) would be the one-package option, but its ownupgrade()targets 3.1 only (the card's scope is 2.0 → 3.0), itsload()is deprecated in favour of@scalar/json-magic/bundle, and it depends on both of these packages plus a validator (ajv). So the two direct dependencies are a strictly smaller tree. Alternatives read and rejected:swagger2openapi7.0.8 (BSD-3-Clause, last release 2021-07-07),@apidevtools/swagger-parser13.1.0 (MIT; parses and dereferences, does not convert 2.0),@redocly/openapi-core2.72.0 (MIT; bundles, does not convert 2.0, 13 direct deps). The inlining itself is this script's own code: no library inlines a recursive document into finite JSON, which is the cut policy above.Tests
scripts/normalize-openapi.test.ts, 13 cases, no network (a URL input is served by an injected fetch). Every normalized output is loaded through the REAL provider (createOpenApiProviderFactory, file-pathspecvialoadPackageFile), and "no$ref" is asserted on both the document and the materialized connector definition.scripts/fixtures/swagger2-esignature-shape.json(Swagger 2.0,host/basePath, body parameter, an operation-levelconsumes: [], a recursive definition) andscripts/fixtures/openapi3-billing-shape.yamlplusbilling-shared.yaml(OpenAPI 3.0 YAML across two files, customer ↔ subscription recursion, a vendorx-expansionResourcesextension holding$refs, sharedresponses.defaultand parameter, a property nameddefault).defaultas property vs literal; extension stub; action keys equal the provider's; the generatedincludelist materializes exactly those actions; zero-match and invalid patterns refused;--max-depth 0stubs; relative server resolved against a source URL, variables filled, refused from a file without--server-url; non-OpenAPI and HTTP 404 refused; the CLI's printed sha256 equals the hash of the file it wrote; exit codes 1 and 2 write nothing.inSchema &&guard on the schema-literal keys (soresponses.defaultis read as literal data) turned 2 of 13 red ("bundles the external file…", "inlines up to --max-depth…");scripts/ablation-replace.mjsreported anchor 1 → 0, blob613cae347a0e→0cd0bc79a1fd, restored to the HEAD blob withgit diff HEADempty.Acceptance notes
stripe/openapiopenapi/spec3.json(8,317,513 bytes, 3.0.0) → 63,317,655 bytes,refsLeft0, cuts 754 recursive / 29,831 max-depth / 8,164 extension; the provider materializes 612 actions with 0$refin the definition. The same document by URL with--include-pattern '^GET /v1/customers'gives the byte-identical output (same sha256) and anincludelist of 22 keys; the provider with thatincludematerializes exactly 22 actions. DocuSigndocusign/OpenAPI-Specificationsesignature.rest.swagger-v2.1.json(4,846,375 bytes, Swagger 2.0) is refused without--empty-consumes(187 operations named); with--empty-consumes application/json→ OpenAPI 3.0.4, 52,884,345 bytes,refsLeft0, serverhttps://www.docusign.net/restapi; the provider materializes 414 actions, 0$ref, 192 with a body input.sha256sumof the written file for both real outputs (Stripe0ea3c37a…b2, DocuSignceee5238…37), and the CLI test pins it.filesisdist,README.md,CHANGELOG.md; README and CHANGELOG are untouched; afterpnpm --filter @objectstack/connector-openapi build,normalizeOpenApi/normalize-openapioccur 0 times indist/*while the positive controlcreateOpenApiProviderFactoryoccurs 4 times indist/index.js. Labelledskip-changeset.--max-depth(whole document, no include): depth 0 → Stripe 6.1 MB / DocuSign 1.6 MB; 1 → 14.8 / 4.9 MB; 2 (default) → 63.3 / 52.9 MB; 3 → 361 / 160 MB. The default is a trade-off the reviewer may want to change; it is one constant (DEFAULT_MAX_DEPTH).bin). A pipeline in another repository cannot run it from npm; publishing it (abin, the two packages moved todependencies) would widen the published surface and is left to a ruling.parametersare not hoisted into operations (the 2.0 upgrader hoists them for 2.0 input). The generator does not read path-item-level parameters at all, so such a parameter is missing from the action'sinputSchema; that is a runtime defect, reported separately rather than worked around here.discriminator.mappingvalues are strings, not$refkeys; aftercomponentsis dropped they name schemas no longer in the document. The generator never readsdiscriminator.@scalar/openapi-upgraderlets the globalproduceswin over an operation-level one (document.produces ?? operationItem.produces); harmless for both real documents, noted only.Verification
All local readings at head
f9a4f408.Package test and typecheck (head f9a4f40)
Derived gate families
node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstackderived 89 commands for this diff; each was run with its exit code captured before any pipe. 86 exited 0. 3 exited 3 (PREREQUISITE NOT MET, nothing measured, none reads these paths):check-plugin-teardown-shape.mjs --self-test(shallow clone),@objectstack/spec check:skill-examples(client packages unbuilt),check:dual-build-cjs-loads(workspace unbuilt).--ranreconciliation: "89 derived famil(ies) accounted for — 86 run, 3 NOT-MEASURED (3 DERIVED from a recorded exit 3)". The full rerun at the final head is in the report comment on #27373.Real-document runs (local, once)
Generated by Claude Code