From 2378230698fdb4eea3d6af449f0c05c90c0451c7 Mon Sep 17 00:00:00 2001 From: Pete Koomen Date: Tue, 21 Jul 2026 08:13:48 -0700 Subject: [PATCH 1/7] Add the distribution plan: CLI releases, install scripts, npm packages Co-Authored-By: Claude Fable 5 --- notes/distribution-plan.md | 207 +++++++++++++++++++++++++++++++++++++ 1 file changed, 207 insertions(+) create mode 100644 notes/distribution-plan.md diff --git a/notes/distribution-plan.md b/notes/distribution-plan.md new file mode 100644 index 0000000..f73e984 --- /dev/null +++ b/notes/distribution-plan.md @@ -0,0 +1,207 @@ +# Distributing Scratchwork: CLI binaries + npm packages + +**Goal:** anyone can install the `scratchwork` CLI with one command and deploy their own +server from published packages: + +1. CLI binaries versioned and released on GitHub Releases; +2. `https://scratchwork.dev/install.sh` (humans) and `https://scratchwork.dev/install.md` + (agents) as the install entry points; +3. the server packages (`@scratchwork/shared`, `@scratchwork/server-core`, + `@scratchwork/server-deploy-{aws,cloudflare,local}`) published to npm. + +Status legend: `[ ]` not started · `[~]` in progress · `[x]` done + +## Where we are today + +- **CLI:** `cli/build.js` compiles `src/index.ts` into a single self-contained binary + (`bun build --compile`) with the renderer shell embedded — host platform only, no + target matrix. Version comes from `cli/package.json` (`0.1.0`) via `import pkg from + "../package.json"`, already wired to `scratchwork version`, `--version`, and `-v`. + So the binary already embeds its version at build time; nothing new needed there. +- **Packages:** every workspace is `private: true` at `0.1.0`, inter-deps use + `workspace:*`, and `exports` point at TypeScript source (`./src/*.ts`). Nothing is + publishable as-is. +- **Automation:** the only workflow is `ci.yml` (exactly `bun run ci`). No release + workflow, no tags, no changelog. +- **Docs already promise the endpoint:** `README.md` and `docs/index.md` both say + `curl -fsSL https://scratchwork.dev/install.sh | bash`, but nothing produces or + serves `install.sh`, and there are no releases to download. +- **scratchwork.dev is itself a published project.** The homepage domains serve an + ordinary project (`homepageDomains`/`homepageProject`, see + `deploy/cloudflare-vanilla`); `docs/` is the homepage content, published with + `scratchwork publish`. So the install assets ship by adding files to `docs/` and + republishing — no server change needed to host them. +- **Raw markdown serving already works:** requesting a `.md` path directly returns the + raw markdown as `text/plain` (`RawMarkdownServed` in `shared/src/site/serve.ts`), + while extensionless routes get the rendered shell. `install.md` therefore serves + agent-readable raw markdown for free. `.sh` content-type needs verifying in + `shared/src/site/content.ts` (any type works for `curl | bash`, but `text/plain` + beats a download prompt for humans who open it in a browser). + +## Decisions (recommendations inline; flag disagreement before Phase 1) + +1. **Lockstep versioning.** One version for the whole repo: CLI binaries and all npm + packages share it, and a git tag `vX.Y.Z` on main is the single release trigger. + Independent per-package versions buy nothing at this stage and complicate the + `workspace:*` story. Source of truth: the `version` field in each `package.json`, + kept in lockstep by a version-bump script and a mechanized ci check. +2. **npm packages ship TypeScript source, not built JS.** The published server packages + are consumed by Bun projects (the deploy projects run `bun deploy.ts`) and by + bundlers (wrangler/esbuild), both of which consume TS directly. Shipping `src/` as-is + means zero build infrastructure and the published artifact equals the repo. Declare + the constraint honestly: `"engines": { "bun": ">=1.2" }` and a README note. Revisit + with a build step + `.d.ts` only if non-Bun Node consumers materialize. +3. **What publishes where.** GitHub Releases: the CLI binary only. npm: `shared`, + `server/core`, `server/deploy-aws`, `server/deploy-cloudflare`, `server/deploy-local`. + Never published: `renderer` (embedded in the CLI), `deploy/*` (per-domain instances — + they become the template users copy), `e2e`, the `server` tooling workspace. +4. **Cross-compile on one runner.** `bun build --compile --target=...` cross-compiles + from a single Linux runner; no per-OS build matrix. Targets: `bun-darwin-arm64`, + `bun-darwin-x64`, `bun-linux-x64`, `bun-linux-arm64`. Windows and musl (Alpine) + are explicit non-goals for v0; install.sh says so clearly when it can't match. +5. **Install destination `~/.local/bin`, no sudo.** Overridable with + `SCRATCHWORK_INSTALL_DIR`; version pinnable with `SCRATCHWORK_VERSION`. The script + never escalates privileges. + +## Resolved questions (Pete, 2026-07-20) + +- **Repo visibility:** `github.com/scratch/scratchwork` is public (verified via + `gh repo view`), so GitHub Release assets are publicly downloadable — install.sh can + fetch them directly. +- **npm scope:** Pete owns the `@scratchwork` org and the local npm CLI is + authenticated (`npm whoami` → `koomen`, org owner). First releases publish from + Pete's machine; moving publishing into the release workflow (granular `NPM_TOKEN` in + Actions secrets) is a follow-up once the manual loop is boring. +- **macOS code signing:** unsigned binaries are accepted for v0. `curl | bash` installs + don't set the quarantine attribute; revisit notarization only if browser downloads + become a supported path. + +## Plan + +### Phase 1 — Version plumbing + +`[ ]` A `bun scripts/set-version.ts ` script that stamps `version` in the root +and every workspace `package.json` in lockstep. No other duty — tagging and changelog +stay manual and visible. + +`[ ]` Mechanized lockstep check in `bun run ci` (natural home: `check-boundaries.ts` or +a sibling script): every workspace `version` equals the root version. A drifted bump +fails the gate. + +`[ ]` `CHANGELOG.md` at root, maintained by hand per release, newest first. The release +workflow copies the top section into the GitHub Release notes. + +Acceptance: `bun scripts/set-version.ts 0.2.0` updates every package.json; `bun run ci` +fails if any one is edited out of lockstep. + +### Phase 2 — Cross-platform CLI builds + GitHub Release workflow + +`[ ]` Extend `cli/build.js` with a target matrix mode (e.g. `bun build.js --all-targets`): +builds the renderer once, then `bun build --compile --target=bun--` per target +into `cli/dist/scratchwork--`. Default (no flag) behavior stays exactly as +today so `bun run ci` cost doesn't change. + +`[ ]` `scripts/package-release.ts`: tars each binary as +`scratchwork-v--.tar.gz` (binary named `scratchwork` inside) and +writes `checksums.txt` (SHA-256 of each archive). Tarball rather than bare binary so +the executable bit survives and the name inside is stable. + +`[ ]` `.github/workflows/release.yml`, triggered by tags matching `v*`: +1. checkout, pinned Bun, `bun install --frozen-lockfile`; +2. `bun run ci` (the same one gate — release never ships what ci wouldn't pass); +3. assert the tag matches the package.json version (fail loudly on mismatch); +4. build all targets, package, `gh release create` with archives + `checksums.txt` + and the changelog section as notes. +`ci.yml` is untouched; the gate stays exactly `bun run ci`. + +Acceptance: pushing tag `v0.2.0` produces a GitHub Release with four archives + +checksums, and `gh release download v0.2.0` on a mac yields a runnable binary that +prints `0.2.0` from `scratchwork --version`. + +### Phase 3 — install.sh and install.md on scratchwork.dev + +`[ ]` `docs/install.sh` — POSIX sh, `set -euf`. Detects `uname -s`/`-m`, maps to a +release target (clear error for unsupported platforms, mentioning Windows/musl +explicitly), downloads from the stable no-API URL +`https://github.com/scratch/scratchwork/releases/latest/download/` (or the +pinned `SCRATCHWORK_VERSION`), verifies SHA-256 against `checksums.txt` (uses +`shasum -a 256` or `sha256sum`, whichever exists), installs to +`$SCRATCHWORK_INSTALL_DIR` (default `~/.local/bin`), and prints PATH guidance only when +the directory isn't on `$PATH`. Re-running upgrades in place. No sudo, ever. + +`[ ]` `docs/install.md` — the agent-facing page: what Scratchwork is (one paragraph), +the one-liner, the manual steps (exact URL pattern per platform, checksum verification, +chmod, PATH), version pinning, uninstall (`rm` one binary), and a pointer to +`scratchwork --help`. Written to be executed by an agent without fetching anything else. + +`[ ]` Verify/extend the `.sh` content-type mapping in `shared/src/site/content.ts` +(want `text/plain; charset=utf-8` or `text/x-shellscript`), with a serving test. Add a +test asserting a published `install.md` round-trips raw (the `RawMarkdownServed` path) +— that behavior is now load-bearing for distribution. + +`[ ]` Mechanized checks for the script itself inside `bun run ci`: `sh -n docs/install.sh` +(syntax) plus a unit test driving the platform-mapping + download against a local HTTP +fixture standing in for GitHub (same hermetic spirit as the e2e OAuth stand-in). No +network in ci. + +`[ ]` Release step: republish the homepage project after each release +(`scratchwork publish docs --project www ...`). Manual at first, listed in RELEASING.md; +automating it in release.yml (server credentials as secrets) is a follow-up once the +manual loop is boring. + +Acceptance: with a release published, `curl -fsSL https://scratchwork.dev/install.sh | bash` +on clean macOS and Linux machines installs the latest binary and `scratchwork --version` +prints the released version; `curl https://scratchwork.dev/install.md` returns raw +markdown an agent can follow end-to-end. + +### Phase 4 — npm packages + +`[ ]` Make the five packages publishable: drop `private: true`; add `license`, +`repository` (with `directory`), `engines.bun`, `files` (src + README, no tests), and a +short README each (what it is, that it ships TS source for Bun, minimal usage). Root, +`renderer`, `cli`, `server` (tooling), `deploy/*`, and `e2e` stay private. + +`[ ]` Verify `bun publish` rewrites `workspace:*` to the concrete lockstep version in +the published tarball (it should; check with `bun pm pack` + inspect). If it doesn't, +the release script rewrites versions at publish time. + +`[ ]` Dry-run check in ci or release workflow: `bun pm pack` each publishable package +and assert the tarball contains `src/`, no test files, and no `workspace:` strings. + +`[ ]` A `scripts/publish-packages.ts` release step that runs `bun publish` for each +package in dependency order (shared → server-core → deploy adapters), refusing to run +on a dirty tree or when the checked-out tag doesn't match the lockstep version. Run +locally with Pete's authenticated npm CLI for the first releases. Follow-up (not this +plan): move it into `release.yml` with a granular `NPM_TOKEN` and `--provenance` +(free supply-chain attestation from Actions OIDC now that the repo is public). + +`[ ]` Consumer walkthrough in `server/README.md` (or `docs/`): "deploy your own" — a +fresh directory, `bun add @scratchwork/server-deploy-cloudflare`, copy the config shape +from `deploy/cloudflare-vanilla`, one command deploy. A `scratchwork server init` +scaffolder is explicitly future work, not this plan. + +Acceptance: in a fresh directory outside the repo, `bun add +@scratchwork/server-deploy-cloudflare` + the documented config typechecks and deploys a +working server at the released version. + +### Phase 5 — Release process doc + +`[ ]` `RELEASING.md` at root: bump with `set-version.ts` → update `CHANGELOG.md` → PR → +merge → tag `vX.Y.Z` → workflow does the GitHub Release → run +`scripts/publish-packages.ts` locally for npm → republish homepage → smoke-test +install.sh from a clean machine. Short enough to actually be followed. + +`[ ]` First real release: `v0.2.0` end-to-end, following RELEASING.md as written and +fixing the doc where reality disagrees. + +## Invariant compliance notes + +- Release/packaging scripts under `scripts/` and the workflow are deploy tooling — + outside the Effect-boundary lint scope (`cli/src`, `server/**/src`, `shared/src`), + same as the existing root scripts. `install.sh` is not TypeScript and lives in + `docs/`, which is published content, deliberately outside the gate — but its syntax + check and mapping test (Phase 3) run inside `bun run ci` so the gate still covers it. +- Any serving change for `.sh`/`.md` content types touches `shared/src/site` — plain + mechanized-test territory (invariant 1 applies; no new async boundaries expected). +- No new auth surface, routes, or storage semantics anywhere in this plan + (invariants 3–6 untouched). From c576b6c5ca73eff2ac86a742b52b0a6629e20ed4 Mon Sep 17 00:00:00 2001 From: Peter Koomen Date: Tue, 21 Jul 2026 09:40:16 -0700 Subject: [PATCH 2/7] Implement the distribution plan: releases, install scripts, npm packages MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 1 — version plumbing: scripts/set-version.ts stamps the lockstep version across root + all workspaces; scripts/check-versions.ts enforces it in the gate; CHANGELOG.md seeds the release-notes flow. Phase 2 — CLI releases: cli/build.js --all-targets cross-compiles the four release targets; scripts/package-release.ts produces tar.gz archives + checksums.txt; .github/workflows/release.yml (tag v*) runs the same one gate, asserts tag == lockstep version, and publishes the GitHub Release with the CHANGELOG section as notes. Phase 3 — install entry points: docs/install.sh (POSIX, checksum-verified, ~/.local/bin, no sudo; discovers the latest version from checksums.txt so no GitHub API call) and docs/install.md (agent-readable manual steps). scripts/check-install-sh.ts drives the full install loop against a local HTTP fixture in ci; a shared serving test pins .sh-as-text/plain and raw .md round-tripping. Phase 4 — npm packages: the five publishable packages (shared, server-core, server-deploy-{aws,cloudflare,local}) build to dist/ (tsc ESM + declarations) and stage publish-shaped tarballs; scripts/check-npm-pack.ts verifies tarball shape, imports every package under plain Node (Bun for deploy-local), and typechecks a NodeNext consumer — all hermetically in ci. scripts/publish-packages.ts publishes from a clean tagged checkout. Discovered prerequisites shipped along the way: cross-package imports moved from relative ../../../shared paths to @scratchwork/* specifiers with real workspace deps; relative imports in publishable packages carry .ts extensions for rewriteRelativeImportExtensions (d.ts specifiers post-processed); the Bun-only svg text import became a generated module (freshness-checked); the server tooling workspace dissolved into server/core/src/deploy/ with proc.ts ported to node:child_process so deploys run under plain Node; MIT LICENSE added (packages can't publish without one — flag if you want a different license). Phase 5 — RELEASING.md documents the loop end to end. The first real release (v0.2.0) is the remaining unchecked plan item. Co-Authored-By: Claude Fable 5 --- .github/workflows/release.yml | 48 +++ .gitignore | 4 + AGENTS.md | 4 +- CHANGELOG.md | 18 + LICENSE | 21 ++ RELEASING.md | 28 ++ bun.lock | 14 +- cli/build.js | 33 +- cli/package.json | 1 + cli/src/api.ts | 2 +- cli/src/assets.d.ts | 7 +- cli/src/auth.ts | 4 +- cli/src/commands/login.ts | 2 +- cli/src/commands/projects.ts | 2 +- cli/src/commands/publish.ts | 10 +- cli/src/commands/template.ts | 2 +- cli/src/dev/diagnostics.ts | 8 +- cli/src/dev/live-reload.ts | 2 +- cli/src/dev/server.ts | 4 +- cli/src/dev/site-files.ts | 6 +- cli/src/dev/target.ts | 4 +- cli/src/errors.ts | 2 +- cli/src/project-config.ts | 2 +- cli/src/renderer/default.ts | 2 +- deploy/cloudflare-access/tsconfig.json | 1 - deploy/cloudflare-vanilla/tsconfig.json | 1 - deploy/generic-aws/tsconfig.json | 1 - deploy/local-dev/tsconfig.json | 1 - docs/install.md | 87 +++++ docs/install.sh | 95 +++++ e2e/tsconfig.json | 3 +- notes/distribution-plan.md | 95 +++-- package.json | 3 +- renderer/build.js | 6 +- scripts/build-packages.ts | 132 +++++++ scripts/check-boundaries.ts | 2 + scripts/check-generated-fresh.ts | 5 +- scripts/check-install-sh.ts | 150 ++++++++ scripts/check-npm-pack.ts | 173 +++++++++ scripts/check-versions.ts | 24 ++ scripts/each-workspace.ts | 26 +- scripts/generate-assets.ts | 26 ++ scripts/package-release.ts | 57 +++ scripts/publish-packages.ts | 46 +++ scripts/release-notes.ts | 32 ++ scripts/release-targets.ts | 7 + scripts/set-version.ts | 29 ++ scripts/workspaces.ts | 30 ++ server/README.md | 37 ++ server/core/README.md | 23 ++ server/core/package.json | 17 +- server/core/src/access.d.ts | 41 +++ server/core/src/access.js | 202 +++++++++++ server/core/src/access.ts | 2 +- server/core/src/api-routes.ts | 24 +- server/core/src/app.ts | 30 +- server/core/src/assets.d.ts | 4 - server/core/src/auth.ts | 18 +- server/core/src/cloudflare-jwt.ts | 4 +- server/core/src/config.d.ts | 81 +++++ server/core/src/config.js | 335 ++++++++++++++++++ server/core/src/config.ts | 4 +- server/{scripts => core/src/deploy}/env.ts | 0 server/core/src/deploy/proc.ts | 60 ++++ .../src/deploy}/server-settings.ts | 4 +- server/core/src/error-pages.ts | 4 +- server/core/src/google-jwt.ts | 4 +- server/core/src/http.ts | 4 +- server/core/src/index.ts | 18 +- server/core/src/jwt-rs256.ts | 2 +- server/core/src/object-site-files.ts | 10 +- server/core/src/publish-request.ts | 6 +- server/core/src/routes.ts | 2 +- server/core/src/share-request.ts | 6 +- server/core/src/site-records.ts | 4 +- server/core/src/site-store.ts | 22 +- server/core/src/storage.ts | 4 +- .../test/deploy-env.test.ts} | 2 +- .../test/deploy-server-settings.test.ts} | 2 +- server/core/tsconfig.build.json | 19 + server/deploy-aws/README.md | 15 + server/deploy-aws/package.json | 13 +- server/deploy-aws/src/deploy.ts | 6 +- server/deploy-aws/src/handler.ts | 4 +- server/deploy-aws/src/index.ts | 6 +- server/deploy-aws/tsconfig.build.json | 20 ++ server/deploy-cloudflare/README.md | 12 +- server/deploy-cloudflare/package.json | 14 +- server/deploy-cloudflare/src/deploy.ts | 6 +- server/deploy-cloudflare/src/index.ts | 6 +- server/deploy-cloudflare/src/local-worker.ts | 4 +- server/deploy-cloudflare/src/worker.ts | 4 +- server/deploy-cloudflare/tsconfig.build.json | 20 ++ server/deploy-local/README.md | 12 +- server/deploy-local/package.json | 13 +- server/deploy-local/src/index.ts | 4 +- server/deploy-local/src/run.ts | 2 +- server/deploy-local/tsconfig.build.json | 20 ++ server/deploy-local/tsconfig.json | 1 - server/package.json | 16 - server/scripts/proc.ts | 52 --- server/tsconfig.json | 22 -- shared/README.md | 19 + shared/package.json | 16 +- shared/src/assets/figure-svg.generated.ts | 3 + shared/src/crypto/digest.ts | 2 +- shared/src/publish/api.ts | 4 +- shared/src/publish/bundle.ts | 4 +- shared/src/site/components.ts | 2 +- shared/src/site/default-renderer.generated.js | 6 +- shared/src/site/files.ts | 2 +- shared/src/site/html.ts | 2 +- shared/src/site/renderer.ts | 6 +- shared/src/site/routing.ts | 8 +- shared/src/site/serve.ts | 10 +- shared/test/serve-install-assets.test.ts | 73 ++++ shared/tsconfig.build.json | 16 + 117 files changed, 2348 insertions(+), 354 deletions(-) create mode 100644 .github/workflows/release.yml create mode 100644 CHANGELOG.md create mode 100644 LICENSE create mode 100644 RELEASING.md create mode 100644 docs/install.md create mode 100644 docs/install.sh create mode 100644 scripts/build-packages.ts create mode 100644 scripts/check-install-sh.ts create mode 100644 scripts/check-npm-pack.ts create mode 100644 scripts/check-versions.ts create mode 100644 scripts/generate-assets.ts create mode 100644 scripts/package-release.ts create mode 100644 scripts/publish-packages.ts create mode 100644 scripts/release-notes.ts create mode 100644 scripts/release-targets.ts create mode 100644 scripts/set-version.ts create mode 100644 scripts/workspaces.ts create mode 100644 server/core/README.md create mode 100644 server/core/src/access.d.ts create mode 100644 server/core/src/access.js delete mode 100644 server/core/src/assets.d.ts create mode 100644 server/core/src/config.d.ts create mode 100644 server/core/src/config.js rename server/{scripts => core/src/deploy}/env.ts (100%) create mode 100644 server/core/src/deploy/proc.ts rename server/{scripts => core/src/deploy}/server-settings.ts (99%) rename server/{scripts/env.test.ts => core/test/deploy-env.test.ts} (98%) rename server/{scripts/server-settings.test.ts => core/test/deploy-server-settings.test.ts} (98%) create mode 100644 server/core/tsconfig.build.json create mode 100644 server/deploy-aws/README.md create mode 100644 server/deploy-aws/tsconfig.build.json create mode 100644 server/deploy-cloudflare/tsconfig.build.json create mode 100644 server/deploy-local/tsconfig.build.json delete mode 100644 server/package.json delete mode 100644 server/scripts/proc.ts delete mode 100644 server/tsconfig.json create mode 100644 shared/README.md create mode 100644 shared/src/assets/figure-svg.generated.ts create mode 100644 shared/test/serve-install-assets.test.ts create mode 100644 shared/tsconfig.build.json diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml new file mode 100644 index 0000000..a04e40e --- /dev/null +++ b/.github/workflows/release.yml @@ -0,0 +1,48 @@ +# Release: triggered by pushing a tag vX.Y.Z (see RELEASING.md). Runs the same +# one gate as ci.yml — a release never ships what `bun run ci` wouldn't pass — +# then cross-compiles the CLI target matrix, packages the archives, and +# publishes a GitHub Release with the matching CHANGELOG.md section as notes. +# npm publishing stays a manual local step for now (scripts/publish-packages.ts). +name: release + +on: + push: + tags: ["v*"] + +# Creates the GitHub Release and uploads its assets. +permissions: + contents: write + +jobs: + release: + runs-on: ubuntu-latest + timeout-minutes: 30 + steps: + - uses: actions/checkout@v4 + - uses: oven-sh/setup-bun@v2 + with: + bun-version-file: .bun-version + - run: bun install --frozen-lockfile + - run: bun run ci + - name: Assert the tag matches the lockstep version + run: | + version="$(bun -e 'console.log(JSON.parse(await Bun.file("package.json").text()).version)')" + if [ "v${version}" != "${GITHUB_REF_NAME}" ]; then + echo "tag ${GITHUB_REF_NAME} does not match package.json version v${version}" >&2 + echo "bump with \`bun scripts/set-version.ts\` and re-tag" >&2 + exit 1 + fi + - name: Cross-compile the CLI target matrix + run: cd cli && bun build.js --all-targets + - name: Package archives and checksums + run: bun scripts/package-release.ts + - name: Extract release notes from CHANGELOG.md + run: bun scripts/release-notes.ts "${GITHUB_REF_NAME#v}" > release/notes.md + - name: Create the GitHub Release + env: + GH_TOKEN: ${{ github.token }} + run: | + gh release create "${GITHUB_REF_NAME}" \ + release/*.tar.gz release/checksums.txt \ + --title "${GITHUB_REF_NAME}" \ + --notes-file release/notes.md diff --git a/.gitignore b/.gitignore index 1ceb2e4..d2625fb 100644 --- a/.gitignore +++ b/.gitignore @@ -2,6 +2,10 @@ node_modules/ dist/ +# Release staging written by scripts/package-release.ts and +# scripts/build-packages.ts; assets upload to GitHub Releases / npm, never git. +release/ + # Local environment files. Keep .env.example files committed. **/.env **/.env.* diff --git a/AGENTS.md b/AGENTS.md index 9aa3f82..f52d12e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -15,9 +15,9 @@ Package names are `@scratchwork/` for everything under `server/` and `deplo importable by both cli and server. - `cli/` — the `scratchwork` CLI, built with Effect: dev server with hot reload, publishing, login. -- `server/` — deploy tooling scripts only; the server itself lives in its sub-workspaces: +- `server/` — the server workspaces (the directory itself is not a workspace): - `server/core/` — platform-neutral publishing server core: auth, routing, storage - contracts. + contracts, and the deploy-script tooling (`src/deploy/`). - `server/deploy-aws/` — AWS Lambda + S3/DynamoDB adapters. - `server/deploy-cloudflare/` — Cloudflare Worker + R2/D1 adapters. - `server/deploy-local/` — local Bun deploy target. diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..5c8b621 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,18 @@ +# Changelog + +Newest first. Maintained by hand per release; the release workflow copies the +top released section into the GitHub Release notes (scripts/release-notes.ts). + +## Unreleased + +- Distribution: cross-platform CLI binaries on GitHub Releases, `install.sh` / + `install.md` served from scratchwork.dev, and the server packages + (`@scratchwork/shared`, `@scratchwork/server-core`, + `@scratchwork/server-deploy-{aws,cloudflare,local}`) published to npm as + built JS + type declarations. + +## v0.1.0 + +- Pre-distribution baseline: the `scratchwork` CLI (dev server, publish, + share), the single-file Markdown renderer, and the publishing server for + local, AWS, and Cloudflare deploy targets. diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..9c1baf6 --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 Peter Koomen + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/RELEASING.md b/RELEASING.md new file mode 100644 index 0000000..af89bd2 --- /dev/null +++ b/RELEASING.md @@ -0,0 +1,28 @@ +# Releasing Scratchwork + +One version for the whole repo (CLI binaries + npm packages), one tag per +release. Short enough to actually follow: + +1. **Bump:** `bun scripts/set-version.ts X.Y.Z` (stamps root + every + workspace; `bun run ci` enforces lockstep). +2. **Changelog:** retitle the `## Unreleased` section of `CHANGELOG.md` to + `## vX.Y.Z` and start a fresh `## Unreleased` above it. +3. **PR + merge:** open a PR with the bump, let ci pass, merge to main. +4. **Tag:** `git tag vX.Y.Z && git push origin vX.Y.Z` on the merge commit. + `.github/workflows/release.yml` runs the full gate, asserts the tag matches + the lockstep version, cross-compiles the four CLI targets, and publishes + the GitHub Release with archives + `checksums.txt` and the changelog + section as notes. +5. **npm:** from a clean checkout of the tag, with an authenticated npm CLI: + `bun scripts/publish-packages.ts` (add `--dry-run` first if in doubt). + Publishes `@scratchwork/shared`, `@scratchwork/server-core`, and the three + `@scratchwork/server-deploy-*` packages in dependency order. +6. **Homepage:** republish the install entry points so + `https://scratchwork.dev/install.sh` serves the current docs: + `scratchwork publish docs --project www` (with the production server + credentials). Automating this in release.yml is a follow-up. +7. **Smoke:** on a machine (or clean container) without Scratchwork: + `curl -fsSL https://scratchwork.dev/install.sh | bash` and check + `scratchwork --version` prints X.Y.Z. + +Fix this document in the same PR whenever reality disagrees with it. diff --git a/bun.lock b/bun.lock index a210556..f61cafc 100644 --- a/bun.lock +++ b/bun.lock @@ -1,6 +1,5 @@ { "lockfileVersion": 1, - "configVersion": 0, "workspaces": { "": { "name": "scratchwork", @@ -22,6 +21,7 @@ "@effect/printer": "0.49.0", "@effect/printer-ansi": "0.49.0", "@effect/typeclass": "0.40.0", + "@scratchwork/shared": "workspace:*", "effect": "3.21.4", }, "devDependencies": { @@ -105,19 +105,12 @@ "esbuild": "^0.21.5", }, }, - "server": { - "name": "scratchwork-server", - "version": "0.1.0", - "devDependencies": { - "@types/bun": "^1.3.14", - "typescript": "^6.0.3", - }, - }, "server/core": { "name": "@scratchwork/server-core", "version": "0.1.0", "dependencies": { "@effect/platform": "0.96.2", + "@scratchwork/shared": "workspace:*", "effect": "3.21.4", }, "devDependencies": { @@ -147,6 +140,7 @@ "dependencies": { "@effect/platform": "0.96.2", "@scratchwork/server-core": "workspace:*", + "@scratchwork/shared": "workspace:*", "cloudflare": "^6.5.0", "effect": "3.21.4", }, @@ -632,8 +626,6 @@ "scratchwork-renderer": ["scratchwork-renderer@workspace:renderer"], - "scratchwork-server": ["scratchwork-server@workspace:server"], - "semver": ["semver@7.8.5", "", { "bin": { "semver": "bin/semver.js" } }, "sha512-Y7/KDsb8LjooZpwaqGyulO6DQlksgCncchHGk+sZIY4SBvUocMBEFH5Ur1fI4dV+Jvl0w6cjvucaIi40puRioA=="], "sharp": ["sharp@0.34.5", "", { "dependencies": { "@img/colour": "^1.0.0", "detect-libc": "^2.1.2", "semver": "^7.7.3" }, "optionalDependencies": { "@img/sharp-darwin-arm64": "0.34.5", "@img/sharp-darwin-x64": "0.34.5", "@img/sharp-libvips-darwin-arm64": "1.2.4", "@img/sharp-libvips-darwin-x64": "1.2.4", "@img/sharp-libvips-linux-arm": "1.2.4", "@img/sharp-libvips-linux-arm64": "1.2.4", "@img/sharp-libvips-linux-ppc64": "1.2.4", "@img/sharp-libvips-linux-riscv64": "1.2.4", "@img/sharp-libvips-linux-s390x": "1.2.4", "@img/sharp-libvips-linux-x64": "1.2.4", "@img/sharp-libvips-linuxmusl-arm64": "1.2.4", "@img/sharp-libvips-linuxmusl-x64": "1.2.4", "@img/sharp-linux-arm": "0.34.5", "@img/sharp-linux-arm64": "0.34.5", "@img/sharp-linux-ppc64": "0.34.5", "@img/sharp-linux-riscv64": "0.34.5", "@img/sharp-linux-s390x": "0.34.5", "@img/sharp-linux-x64": "0.34.5", "@img/sharp-linuxmusl-arm64": "0.34.5", "@img/sharp-linuxmusl-x64": "0.34.5", "@img/sharp-wasm32": "0.34.5", "@img/sharp-win32-arm64": "0.34.5", "@img/sharp-win32-ia32": "0.34.5", "@img/sharp-win32-x64": "0.34.5" } }, "sha512-Ou9I5Ft9WNcCbXrU9cMgPBcCK8LiwLqcbywW3t4oDV37n1pzpuNLsYiAV8eODnjbtQlSDwZ2cUEeQz4E54Hltg=="], diff --git a/cli/build.js b/cli/build.js index f5f77e3..0d7fee2 100644 --- a/cli/build.js +++ b/cli/build.js @@ -8,28 +8,41 @@ * compiler embeds into the binary, so it runs anywhere with no renderer * source or dist on disk. * + * With --all-targets, cross-compiles the release target matrix instead + * (decision 4 in notes/distribution-plan.md) into cli/dist/scratchwork--. + * The default (no flag) build is unchanged so `bun run ci` cost stays the same. + * * Requires the renderer's deps to be installed (cd ../renderer && bun install). */ import { buildDist } from "../renderer/build.js"; +import { RELEASE_TARGETS } from "../scripts/release-targets"; import { mkdirSync } from "node:fs"; import { fileURLToPath } from "node:url"; import { dirname, join } from "node:path"; const here = dirname(fileURLToPath(import.meta.url)); const OUT_DIR = join(here, "dist"); -const OUT_BIN = join(OUT_DIR, "scratchwork"); // 1. Build the renderer shell the binary will embed through shared/src/site. await buildDist(); -// 2. Compile src/index.ts into a binary. +// 2. Compile src/index.ts — host binary by default, the matrix with --all-targets. mkdirSync(OUT_DIR, { recursive: true }); -const proc = Bun.spawnSync( - ["bun", "build", join(here, "src", "index.ts"), "--compile", "--outfile", OUT_BIN], - { cwd: here, stdout: "inherit", stderr: "inherit" }, -); -if (!proc.success) { - console.error("cli build: `bun build --compile` failed"); - process.exit(1); +const builds = process.argv.includes("--all-targets") + ? RELEASE_TARGETS.map((target) => ({ + outfile: join(OUT_DIR, `scratchwork-${target}`), + extraArgs: [`--target=bun-${target}`], + })) + : [{ outfile: join(OUT_DIR, "scratchwork"), extraArgs: [] }]; + +for (const { outfile, extraArgs } of builds) { + const proc = Bun.spawnSync( + ["bun", "build", join(here, "src", "index.ts"), "--compile", ...extraArgs, "--outfile", outfile], + { cwd: here, stdout: "inherit", stderr: "inherit" }, + ); + if (!proc.success) { + console.error(`cli build: \`bun build --compile ${extraArgs.join(" ")}\` failed`); + process.exit(1); + } + console.log(`Built ${outfile.slice(dirname(here).length + 1)} (standalone binary, renderer shell embedded)`); } -console.log(`Built cli/dist/scratchwork (standalone binary, renderer shell embedded)`); diff --git a/cli/package.json b/cli/package.json index d8fcc79..395c80b 100644 --- a/cli/package.json +++ b/cli/package.json @@ -20,6 +20,7 @@ "@effect/printer": "0.49.0", "@effect/printer-ansi": "0.49.0", "@effect/typeclass": "0.40.0", + "@scratchwork/shared": "workspace:*", "effect": "3.21.4" }, "files": [ diff --git a/cli/src/api.ts b/cli/src/api.ts index a05a842..a970c12 100644 --- a/cli/src/api.ts +++ b/cli/src/api.ts @@ -29,7 +29,7 @@ import * as Either from "effect/Either"; import * as Option from "effect/Option"; import type * as ParseResult from "effect/ParseResult"; import * as Schema from "effect/Schema"; -import { ApiErrorBodySchema, CLI_TOKEN_EXCHANGE_PATH, ScratchworkApi } from "../../shared/src/publish/api"; +import { ApiErrorBodySchema, CLI_TOKEN_EXCHANGE_PATH, ScratchworkApi } from "@scratchwork/shared/publish/api"; import { readAuthToken, readCfToken } from "./auth"; import { CliError, errorMessage } from "./errors"; diff --git a/cli/src/assets.d.ts b/cli/src/assets.d.ts index 3916936..8cb95c7 100644 --- a/cli/src/assets.d.ts +++ b/cli/src/assets.d.ts @@ -1,17 +1,12 @@ /* * Type declarations for Bun's `with { type: "text" }` imports, which load - * markdown, SVG, and text assets as plain strings. + * markdown and text assets as plain strings. */ declare module "*.md" { const text: string; export default text; } -declare module "*.svg" { - const text: string; - export default text; -} - declare module "*.txt" { const text: string; export default text; diff --git a/cli/src/auth.ts b/cli/src/auth.ts index fb1f1f4..0a809e0 100644 --- a/cli/src/auth.ts +++ b/cli/src/auth.ts @@ -12,8 +12,8 @@ import * as Effect from "effect/Effect"; import * as Encoding from "effect/Encoding"; import * as Schema from "effect/Schema"; import { homedir } from "node:os"; -import { sha256Base64Url } from "../../shared/src/crypto/digest"; -import { isLoopbackHost } from "../../shared/src/util/url"; +import { sha256Base64Url } from "@scratchwork/shared/crypto/digest"; +import { isLoopbackHost } from "@scratchwork/shared/util/url"; import { CliError, errorMessage } from "./errors"; const AUTH_FILE = "auth.json"; diff --git a/cli/src/commands/login.ts b/cli/src/commands/login.ts index e38144a..0017de2 100644 --- a/cli/src/commands/login.ts +++ b/cli/src/commands/login.ts @@ -18,7 +18,7 @@ import * as Effect from "effect/Effect"; import { type CliTokenRequest, type CliTokenResponse, -} from "../../../shared/src/publish/api"; +} from "@scratchwork/shared/publish/api"; import { apiClient, mapApiErrors } from "../api"; import { generateLoginProof, diff --git a/cli/src/commands/projects.ts b/cli/src/commands/projects.ts index ac64240..a2e1cf9 100644 --- a/cli/src/commands/projects.ts +++ b/cli/src/commands/projects.ts @@ -13,7 +13,7 @@ import * as Effect from "effect/Effect"; import * as Stream from "effect/Stream"; import * as Either from "effect/Either"; import * as Encoding from "effect/Encoding"; -import { isSafeProjectIdentifier } from "../../../shared/src/site/identifiers"; +import { isSafeProjectIdentifier } from "@scratchwork/shared/site/identifiers"; import { apiClient, mapApiErrors } from "../api"; import { readAuthToken } from "../auth"; import { SKIPPED_DIRECTORIES } from "../dev/target"; diff --git a/cli/src/commands/publish.ts b/cli/src/commands/publish.ts index da67614..6bbc83d 100644 --- a/cli/src/commands/publish.ts +++ b/cli/src/commands/publish.ts @@ -11,14 +11,14 @@ import * as Path from "@effect/platform/Path"; import * as Console from "effect/Console"; import * as Effect from "effect/Effect"; import * as Encoding from "effect/Encoding"; -import { decodedBase64ByteLength } from "../../../shared/src/encoding/base64"; +import { decodedBase64ByteLength } from "@scratchwork/shared/encoding/base64"; import { type PublishRequestBody, type PublishResponse, -} from "../../../shared/src/publish/api"; -import { PUBLISH_BUNDLE_VERSION, type PublishBundle } from "../../../shared/src/publish/bundle"; -import { isSafeProjectIdentifier, slugifyIdentifier } from "../../../shared/src/site/identifiers"; -import { isSafeSitePath, type SitePath } from "../../../shared/src/site/paths"; +} from "@scratchwork/shared/publish/api"; +import { PUBLISH_BUNDLE_VERSION, type PublishBundle } from "@scratchwork/shared/publish/bundle"; +import { isSafeProjectIdentifier, slugifyIdentifier } from "@scratchwork/shared/site/identifiers"; +import { isSafeSitePath, type SitePath } from "@scratchwork/shared/site/paths"; import { apiClient, mapApiErrors } from "../api"; import { readAuthToken } from "../auth"; import { openBrowser } from "../browser"; diff --git a/cli/src/commands/template.ts b/cli/src/commands/template.ts index 1d62eb8..2484755 100644 --- a/cli/src/commands/template.ts +++ b/cli/src/commands/template.ts @@ -8,7 +8,7 @@ import * as FileSystem from "@effect/platform/FileSystem"; import * as Path from "@effect/platform/Path"; import * as Console from "effect/Console"; import * as Effect from "effect/Effect"; -import { markMarkdownRenderer } from "../../../shared/src/site/marker"; +import { markMarkdownRenderer } from "@scratchwork/shared/site/marker"; import { CliError } from "../errors"; import { loadShell } from "../renderer/default"; import type { TemplateConfig } from "../types"; diff --git a/cli/src/dev/diagnostics.ts b/cli/src/dev/diagnostics.ts index d4740ea..5493c5f 100644 --- a/cli/src/dev/diagnostics.ts +++ b/cli/src/dev/diagnostics.ts @@ -7,13 +7,13 @@ import * as Effect from "effect/Effect"; import { collectComponentNames, componentFileCandidates, -} from "../../../shared/src/site/components"; -import { SiteFiles } from "../../../shared/src/site/files"; -import type { SitePath } from "../../../shared/src/site/paths"; +} from "@scratchwork/shared/site/components"; +import { SiteFiles } from "@scratchwork/shared/site/files"; +import type { SitePath } from "@scratchwork/shared/site/paths"; import type { RendererSource, SiteServeEvent, -} from "../../../shared/src/site/serve"; +} from "@scratchwork/shared/site/serve"; import { errorMessage } from "../errors"; import { problem, status } from "./output"; import type { DevState } from "./types"; diff --git a/cli/src/dev/live-reload.ts b/cli/src/dev/live-reload.ts index d94b8b1..4c02592 100644 --- a/cli/src/dev/live-reload.ts +++ b/cli/src/dev/live-reload.ts @@ -11,7 +11,7 @@ import * as Effect from "effect/Effect"; import * as Option from "effect/Option"; import * as Ref from "effect/Ref"; import * as Stream from "effect/Stream"; -import type { HtmlTransform } from "../../../shared/src/site/html"; +import type { HtmlTransform } from "@scratchwork/shared/site/html"; import { errorMessage, CliError } from "../errors"; import { status } from "./output"; import type { DevState, ReloadPayload } from "./types"; diff --git a/cli/src/dev/server.ts b/cli/src/dev/server.ts index a9758b2..758a3bd 100644 --- a/cli/src/dev/server.ts +++ b/cli/src/dev/server.ts @@ -8,8 +8,8 @@ import * as HttpServerRequest from "@effect/platform/HttpServerRequest"; import * as HttpServerResponse from "@effect/platform/HttpServerResponse"; import * as BunHttpServer from "@effect/platform-bun/BunHttpServer"; import * as Effect from "effect/Effect"; -import { serveRequest } from "../../../shared/src/site/serve"; -import FIGURE_SVG from "../../../shared/assets/figure.svg" with { type: "text" }; +import { serveRequest } from "@scratchwork/shared/site/serve"; +import { FIGURE_SVG } from "@scratchwork/shared/assets/figure-svg.generated"; import { CliError, errorMessage } from "../errors"; import { loadShell } from "../renderer/default"; import * as SiteFilesLive from "./site-files"; diff --git a/cli/src/dev/site-files.ts b/cli/src/dev/site-files.ts index 87e02ef..0ffc06e 100644 --- a/cli/src/dev/site-files.ts +++ b/cli/src/dev/site-files.ts @@ -10,9 +10,9 @@ import * as HttpServerResponse from "@effect/platform/HttpServerResponse"; import * as Path from "@effect/platform/Path"; import * as Effect from "effect/Effect"; import * as Layer from "effect/Layer"; -import { SiteFileError, SiteFiles } from "../../../shared/src/site/files"; -import type { SitePath } from "../../../shared/src/site/paths"; -import { isWithinRoot } from "../../../shared/src/util/fs"; +import { SiteFileError, SiteFiles } from "@scratchwork/shared/site/files"; +import type { SitePath } from "@scratchwork/shared/site/paths"; +import { isWithinRoot } from "@scratchwork/shared/util/fs"; /** Builds a SiteFiles service that reads from the given site root directory. */ export function layer( diff --git a/cli/src/dev/target.ts b/cli/src/dev/target.ts index 565d7a9..407df0b 100644 --- a/cli/src/dev/target.ts +++ b/cli/src/dev/target.ts @@ -7,8 +7,8 @@ import * as FileSystem from "@effect/platform/FileSystem"; import type { PlatformError } from "@effect/platform/Error"; import * as Path from "@effect/platform/Path"; import * as Effect from "effect/Effect"; -import { isMarkedMarkdownRenderer } from "../../../shared/src/site/marker"; -import { isSafeSitePath } from "../../../shared/src/site/paths"; +import { isMarkedMarkdownRenderer } from "@scratchwork/shared/site/marker"; +import { isSafeSitePath } from "@scratchwork/shared/site/paths"; import { CliError } from "../errors"; import type { DevTarget } from "./types"; diff --git a/cli/src/errors.ts b/cli/src/errors.ts index d1965e6..5ac75b8 100644 --- a/cli/src/errors.ts +++ b/cli/src/errors.ts @@ -4,7 +4,7 @@ */ import * as Data from "effect/Data"; -export { errorMessage } from "../../shared/src/util/errors"; +export { errorMessage } from "@scratchwork/shared/util/errors"; /** A user-facing CLI failure: `message` is printed to stderr, `code` becomes the exit code. */ export class CliError extends Data.TaggedError("CliError")<{ diff --git a/cli/src/project-config.ts b/cli/src/project-config.ts index 039afd9..b5f120e 100644 --- a/cli/src/project-config.ts +++ b/cli/src/project-config.ts @@ -15,7 +15,7 @@ import * as Option from "effect/Option"; import * as Schema from "effect/Schema"; import * as Either from "effect/Either"; import * as Predicate from "effect/Predicate"; -import { isSafeProjectIdentifier } from "../../shared/src/site/identifiers"; +import { isSafeProjectIdentifier } from "@scratchwork/shared/site/identifiers"; import { resolveProjectByPath, type ResolvedProjectRef } from "./api"; import { normalizeServerUrl } from "./auth"; import { CliError } from "./errors"; diff --git a/cli/src/renderer/default.ts b/cli/src/renderer/default.ts index 38ddbe6..1cd635a 100644 --- a/cli/src/renderer/default.ts +++ b/cli/src/renderer/default.ts @@ -18,7 +18,7 @@ import { fileURLToPath } from "node:url"; import { defaultRendererHtml, defaultRendererSourceHash, -} from "../../../shared/src/site/default-renderer.generated.js"; +} from "@scratchwork/shared/site/default-renderer.generated.js"; const rendererRootUrl = new URL("../../../renderer/", import.meta.url); const rendererRoot = fileURLToPath(rendererRootUrl); diff --git a/deploy/cloudflare-access/tsconfig.json b/deploy/cloudflare-access/tsconfig.json index fd04b7e..4a03b30 100644 --- a/deploy/cloudflare-access/tsconfig.json +++ b/deploy/cloudflare-access/tsconfig.json @@ -21,7 +21,6 @@ "local.ts", "cloudflare-config.ts", "server-config.ts", - "../../server/core/src/assets.d.ts", "../../shared/src/**/*.ts", "../../shared/**/*.js" ] diff --git a/deploy/cloudflare-vanilla/tsconfig.json b/deploy/cloudflare-vanilla/tsconfig.json index fd04b7e..4a03b30 100644 --- a/deploy/cloudflare-vanilla/tsconfig.json +++ b/deploy/cloudflare-vanilla/tsconfig.json @@ -21,7 +21,6 @@ "local.ts", "cloudflare-config.ts", "server-config.ts", - "../../server/core/src/assets.d.ts", "../../shared/src/**/*.ts", "../../shared/**/*.js" ] diff --git a/deploy/generic-aws/tsconfig.json b/deploy/generic-aws/tsconfig.json index 0860dc6..c7ecfbc 100644 --- a/deploy/generic-aws/tsconfig.json +++ b/deploy/generic-aws/tsconfig.json @@ -20,7 +20,6 @@ "deploy.ts", "local.ts", "server-config.ts", - "../../server/core/src/assets.d.ts", "../../shared/src/**/*.ts", "../../shared/**/*.js" ] diff --git a/deploy/local-dev/tsconfig.json b/deploy/local-dev/tsconfig.json index 10e2e7d..e76bd50 100644 --- a/deploy/local-dev/tsconfig.json +++ b/deploy/local-dev/tsconfig.json @@ -18,7 +18,6 @@ }, "include": [ "local.ts", - "../../server/core/src/assets.d.ts", "../../shared/src/**/*.ts", "../../shared/**/*.js" ] diff --git a/docs/install.md b/docs/install.md new file mode 100644 index 0000000..a53fa6f --- /dev/null +++ b/docs/install.md @@ -0,0 +1,87 @@ +# Installing the scratchwork CLI + +Scratchwork publishes static HTML and Markdown, publicly and privately: a dev +server with hot reload (`scratchwork dev`), one-command publishing +(`scratchwork publish`), and link sharing (`scratchwork share`). This page is +the complete manual install procedure — written so a human or an agent can +follow it end-to-end without fetching anything else. + +## One-liner + +```sh +curl -fsSL https://scratchwork.dev/install.sh | bash +``` + +Installs the latest release to `~/.local/bin/scratchwork`. No sudo, ever. +Re-running upgrades in place. + +- Pin a version: `SCRATCHWORK_VERSION=0.2.0 curl -fsSL https://scratchwork.dev/install.sh | bash` +- Change the destination: set `SCRATCHWORK_INSTALL_DIR` (default `~/.local/bin`) + +## Supported platforms + +Prebuilt binaries exist for exactly these targets: + +| OS | Architecture | Release asset suffix | +| -------------- | --------------- | -------------------- | +| macOS | arm64 (Apple) | `darwin-arm64` | +| macOS | x64 (Intel) | `darwin-x64` | +| Linux (glibc) | x64 | `linux-x64` | +| Linux (glibc) | arm64 | `linux-arm64` | + +Windows and musl-libc Linux (e.g. Alpine) are not supported yet. + +## Manual install + +Releases live at `https://github.com/scratch/scratchwork/releases`. Each +release `vX.Y.Z` carries one archive per target plus a `checksums.txt`: + +``` +https://github.com/scratch/scratchwork/releases/download/vX.Y.Z/scratchwork-vX.Y.Z--.tar.gz +https://github.com/scratch/scratchwork/releases/download/vX.Y.Z/checksums.txt +``` + +The latest release is always reachable without knowing its version at +`https://github.com/scratch/scratchwork/releases/latest/download/checksums.txt` +— the asset names inside carry the version number. + +Steps (example: macOS arm64, version 0.2.0): + +```sh +curl -fsSLO https://github.com/scratch/scratchwork/releases/download/v0.2.0/scratchwork-v0.2.0-darwin-arm64.tar.gz +curl -fsSLO https://github.com/scratch/scratchwork/releases/download/v0.2.0/checksums.txt + +# Verify: the computed digest must match the asset's line in checksums.txt. +shasum -a 256 -c <(grep darwin-arm64 checksums.txt) # Linux: sha256sum -c ... + +tar -xzf scratchwork-v0.2.0-darwin-arm64.tar.gz # extracts one file: scratchwork +mkdir -p ~/.local/bin +mv scratchwork ~/.local/bin/scratchwork +chmod +x ~/.local/bin/scratchwork +``` + +Make sure `~/.local/bin` is on your `PATH`: + +```sh +export PATH="$HOME/.local/bin:$PATH" +``` + +## Verify the install + +```sh +scratchwork --version +``` + +## Uninstall + +```sh +rm ~/.local/bin/scratchwork +``` + +That's everything — the binary is fully self-contained (the renderer is +embedded) and writes no other files at install time. + +## Next steps + +Run `scratchwork --help`, or start with `scratchwork dev` in a directory +containing Markdown files. Full documentation: https://scratchwork.dev diff --git a/docs/install.sh b/docs/install.sh new file mode 100644 index 0000000..869ce3c --- /dev/null +++ b/docs/install.sh @@ -0,0 +1,95 @@ +#!/bin/sh +# Installs the scratchwork CLI from GitHub Releases. +# +# curl -fsSL https://scratchwork.dev/install.sh | bash +# +# Environment: +# SCRATCHWORK_VERSION pin a version (e.g. 0.2.0); default: latest release +# SCRATCHWORK_INSTALL_DIR install destination; default: ~/.local/bin +# SCRATCHWORK_DOWNLOAD_BASE override the release download base URL (used by +# the hermetic ci test; default: GitHub Releases) +# +# The script never escalates privileges. Re-running upgrades in place. +# Agent-readable manual steps: https://scratchwork.dev/install.md + +set -euf + +base="${SCRATCHWORK_DOWNLOAD_BASE:-https://github.com/scratch/scratchwork/releases}" +install_dir="${SCRATCHWORK_INSTALL_DIR:-$HOME/.local/bin}" +version="${SCRATCHWORK_VERSION:-}" + +fail() { + printf 'install.sh: %s\n' "$1" >&2 + exit 1 +} + +# ── Map uname to a release target ─────────────────────────────────────────── +os="$(uname -s)" +arch="$(uname -m)" +case "$os" in + Darwin) os=darwin ;; + Linux) os=linux ;; + *) fail "unsupported operating system: $os. Prebuilt binaries cover macOS and glibc Linux only — Windows is not supported yet. See https://github.com/scratch/scratchwork for building from source." ;; +esac +case "$arch" in + arm64|aarch64) arch=arm64 ;; + x86_64|amd64) arch=x64 ;; + *) fail "unsupported architecture: $arch. Prebuilt binaries cover arm64 and x64 only." ;; +esac +if [ "$os" = linux ] && command -v ldd >/dev/null 2>&1 && ldd --version 2>&1 | grep -qi musl; then + fail "musl libc detected (Alpine?). Prebuilt binaries are glibc-only for now — musl is an explicit non-goal for v0." +fi + +command -v curl >/dev/null 2>&1 || fail "curl is required" +command -v tar >/dev/null 2>&1 || fail "tar is required" +if command -v sha256sum >/dev/null 2>&1; then + sha256() { sha256sum "$1" | cut -d' ' -f1; } +elif command -v shasum >/dev/null 2>&1; then + sha256() { shasum -a 256 "$1" | cut -d' ' -f1; } +else + fail "sha256sum or shasum is required to verify the download" +fi + +tmp="$(mktemp -d)" +trap 'rm -rf "$tmp"' EXIT INT TERM + +# ── Fetch checksums.txt; its asset names carry the release version, so the +# "latest" case needs no GitHub API call ───────────────────────────────── +if [ -n "$version" ]; then + checksums_url="$base/download/v$version/checksums.txt" +else + checksums_url="$base/latest/download/checksums.txt" +fi +curl -fsSL "$checksums_url" -o "$tmp/checksums.txt" || fail "could not download $checksums_url" +if [ -z "$version" ]; then + version="$(sed -n 's/^.*scratchwork-v\([^-][^-]*\)-.*\.tar\.gz$/\1/p' "$tmp/checksums.txt" | head -n 1)" + [ -n "$version" ] || fail "could not read the latest version from checksums.txt" +fi + +asset="scratchwork-v$version-$os-$arch.tar.gz" +expected="$(grep -F " $asset" "$tmp/checksums.txt" | cut -d' ' -f1 || true)" +[ -n "$expected" ] || fail "release v$version has no prebuilt binary for $os-$arch" + +# ── Download, verify, extract, install ────────────────────────────────────── +printf 'Downloading scratchwork v%s (%s-%s)...\n' "$version" "$os" "$arch" +curl -fsSL "$base/download/v$version/$asset" -o "$tmp/$asset" || fail "could not download $base/download/v$version/$asset" +actual="$(sha256 "$tmp/$asset")" +[ "$actual" = "$expected" ] || fail "checksum mismatch for $asset: expected $expected, got $actual" + +tar -xzf "$tmp/$asset" -C "$tmp" +[ -f "$tmp/scratchwork" ] || fail "archive $asset did not contain a scratchwork binary" +mkdir -p "$install_dir" +chmod 755 "$tmp/scratchwork" +mv -f "$tmp/scratchwork" "$install_dir/scratchwork" + +printf 'Installed %s\n' "$install_dir/scratchwork" +"$install_dir/scratchwork" --version >/dev/null || fail "the installed binary failed to run" +printf 'scratchwork %s is ready.\n' "$("$install_dir/scratchwork" --version)" + +case ":$PATH:" in + *":$install_dir:"*) ;; + *) + printf '\n%s is not on your PATH. Add it, e.g.:\n' "$install_dir" + printf ' export PATH="%s:$PATH"\n' "$install_dir" + ;; +esac diff --git a/e2e/tsconfig.json b/e2e/tsconfig.json index 95f9959..2040229 100644 --- a/e2e/tsconfig.json +++ b/e2e/tsconfig.json @@ -18,7 +18,6 @@ }, "include": [ "src/**/*.ts", - "test/**/*.ts", - "../server/core/src/assets.d.ts" + "test/**/*.ts" ] } diff --git a/notes/distribution-plan.md b/notes/distribution-plan.md index f73e984..1fb6043 100644 --- a/notes/distribution-plan.md +++ b/notes/distribution-plan.md @@ -45,12 +45,13 @@ Status legend: `[ ]` not started · `[~]` in progress · `[x]` done Independent per-package versions buy nothing at this stage and complicate the `workspace:*` story. Source of truth: the `version` field in each `package.json`, kept in lockstep by a version-bump script and a mechanized ci check. -2. **npm packages ship TypeScript source, not built JS.** The published server packages - are consumed by Bun projects (the deploy projects run `bun deploy.ts`) and by - bundlers (wrangler/esbuild), both of which consume TS directly. Shipping `src/` as-is - means zero build infrastructure and the published artifact equals the repo. Declare - the constraint honestly: `"engines": { "bun": ">=1.2" }` and a README note. Revisit - with a build step + `.d.ts` only if non-Bun Node consumers materialize. +2. **npm packages ship built JS + `.d.ts`, not TypeScript source.** (Pete, 2026-07-21; + supersedes the earlier ship-TS-source recommendation.) Node deliberately refuses to + type-strip files under `node_modules` (`ERR_UNSUPPORTED_NODE_MODULES_TYPE_STRIPPING`), + so TS-source packages would be Bun/bundler-only — and users should be able to deploy + with plain Node if they want. Each publishable package gets a `tsc` emit (ESM JS + + declarations) into `dist/`, and the published `exports` point at `dist/`. In-repo + development is unchanged: workspace resolution keeps consuming `src/*.ts` directly. 3. **What publishes where.** GitHub Releases: the CLI binary only. npm: `shared`, `server/core`, `server/deploy-aws`, `server/deploy-cloudflare`, `server/deploy-local`. Never published: `renderer` (embedded in the CLI), `deploy/*` (per-domain instances — @@ -80,15 +81,15 @@ Status legend: `[ ]` not started · `[~]` in progress · `[x]` done ### Phase 1 — Version plumbing -`[ ]` A `bun scripts/set-version.ts ` script that stamps `version` in the root +`[x]` A `bun scripts/set-version.ts ` script that stamps `version` in the root and every workspace `package.json` in lockstep. No other duty — tagging and changelog stay manual and visible. -`[ ]` Mechanized lockstep check in `bun run ci` (natural home: `check-boundaries.ts` or +`[x]` Mechanized lockstep check in `bun run ci` (natural home: `check-boundaries.ts` or a sibling script): every workspace `version` equals the root version. A drifted bump fails the gate. -`[ ]` `CHANGELOG.md` at root, maintained by hand per release, newest first. The release +`[x]` `CHANGELOG.md` at root, maintained by hand per release, newest first. The release workflow copies the top section into the GitHub Release notes. Acceptance: `bun scripts/set-version.ts 0.2.0` updates every package.json; `bun run ci` @@ -96,17 +97,17 @@ fails if any one is edited out of lockstep. ### Phase 2 — Cross-platform CLI builds + GitHub Release workflow -`[ ]` Extend `cli/build.js` with a target matrix mode (e.g. `bun build.js --all-targets`): +`[x]` Extend `cli/build.js` with a target matrix mode (e.g. `bun build.js --all-targets`): builds the renderer once, then `bun build --compile --target=bun--` per target into `cli/dist/scratchwork--`. Default (no flag) behavior stays exactly as today so `bun run ci` cost doesn't change. -`[ ]` `scripts/package-release.ts`: tars each binary as +`[x]` `scripts/package-release.ts`: tars each binary as `scratchwork-v--.tar.gz` (binary named `scratchwork` inside) and writes `checksums.txt` (SHA-256 of each archive). Tarball rather than bare binary so the executable bit survives and the name inside is stable. -`[ ]` `.github/workflows/release.yml`, triggered by tags matching `v*`: +`[x]` `.github/workflows/release.yml`, triggered by tags matching `v*`: 1. checkout, pinned Bun, `bun install --frozen-lockfile`; 2. `bun run ci` (the same one gate — release never ships what ci wouldn't pass); 3. assert the tag matches the package.json version (fail loudly on mismatch); @@ -120,7 +121,7 @@ prints `0.2.0` from `scratchwork --version`. ### Phase 3 — install.sh and install.md on scratchwork.dev -`[ ]` `docs/install.sh` — POSIX sh, `set -euf`. Detects `uname -s`/`-m`, maps to a +`[x]` `docs/install.sh` — POSIX sh, `set -euf`. Detects `uname -s`/`-m`, maps to a release target (clear error for unsupported platforms, mentioning Windows/musl explicitly), downloads from the stable no-API URL `https://github.com/scratch/scratchwork/releases/latest/download/` (or the @@ -129,22 +130,22 @@ pinned `SCRATCHWORK_VERSION`), verifies SHA-256 against `checksums.txt` (uses `$SCRATCHWORK_INSTALL_DIR` (default `~/.local/bin`), and prints PATH guidance only when the directory isn't on `$PATH`. Re-running upgrades in place. No sudo, ever. -`[ ]` `docs/install.md` — the agent-facing page: what Scratchwork is (one paragraph), +`[x]` `docs/install.md` — the agent-facing page: what Scratchwork is (one paragraph), the one-liner, the manual steps (exact URL pattern per platform, checksum verification, chmod, PATH), version pinning, uninstall (`rm` one binary), and a pointer to `scratchwork --help`. Written to be executed by an agent without fetching anything else. -`[ ]` Verify/extend the `.sh` content-type mapping in `shared/src/site/content.ts` +`[x]` Verify/extend the `.sh` content-type mapping in `shared/src/site/content.ts` (want `text/plain; charset=utf-8` or `text/x-shellscript`), with a serving test. Add a test asserting a published `install.md` round-trips raw (the `RawMarkdownServed` path) — that behavior is now load-bearing for distribution. -`[ ]` Mechanized checks for the script itself inside `bun run ci`: `sh -n docs/install.sh` +`[x]` Mechanized checks for the script itself inside `bun run ci`: `sh -n docs/install.sh` (syntax) plus a unit test driving the platform-mapping + download against a local HTTP fixture standing in for GitHub (same hermetic spirit as the e2e OAuth stand-in). No network in ci. -`[ ]` Release step: republish the homepage project after each release +`[x]` Release step: republish the homepage project after each release (`scratchwork publish docs --project www ...`). Manual at first, listed in RELEASING.md; automating it in release.yml (server credentials as secrets) is a follow-up once the manual loop is boring. @@ -156,37 +157,47 @@ markdown an agent can follow end-to-end. ### Phase 4 — npm packages -`[ ]` Make the five packages publishable: drop `private: true`; add `license`, -`repository` (with `directory`), `engines.bun`, `files` (src + README, no tests), and a -short README each (what it is, that it ships TS source for Bun, minimal usage). Root, +`[x]` A build step per publishable package: `tsc` emit of ESM JS + `.d.ts` into +`dist/`, driven by one shared script (`scripts/build-packages.ts` or per-package +`build`). Published `exports`/`types` point at `dist/`; in-repo `exports` keep pointing +at `src/*.ts` so workspace dev needs no build. Use `publishConfig` to swap the fields at +publish time (verify `bun publish`/`bun pm pack` applies `publishConfig.exports`; if +not, the release script rewrites package.json at pack time). `dist/` stays gitignored; +the release flow builds before packing. + +`[x]` Make the five packages publishable: drop `private: true`; add `license`, +`repository` (with `directory`), `files` (dist + README, no tests or src), and a short +README each (what it is, works under Node ≥ 22 or Bun, minimal usage). Root, `renderer`, `cli`, `server` (tooling), `deploy/*`, and `e2e` stay private. -`[ ]` Verify `bun publish` rewrites `workspace:*` to the concrete lockstep version in +`[x]` Verify `bun publish` rewrites `workspace:*` to the concrete lockstep version in the published tarball (it should; check with `bun pm pack` + inspect). If it doesn't, the release script rewrites versions at publish time. -`[ ]` Dry-run check in ci or release workflow: `bun pm pack` each publishable package -and assert the tarball contains `src/`, no test files, and no `workspace:` strings. +`[x]` Dry-run check in ci or release workflow: build, then `bun pm pack` each +publishable package and assert the tarball contains `dist/` with `.js` + `.d.ts`, its +`exports` resolve to `dist/`, and it has no test files and no `workspace:` strings. -`[ ]` A `scripts/publish-packages.ts` release step that runs `bun publish` for each +`[x]` A `scripts/publish-packages.ts` release step that runs `bun publish` for each package in dependency order (shared → server-core → deploy adapters), refusing to run on a dirty tree or when the checked-out tag doesn't match the lockstep version. Run locally with Pete's authenticated npm CLI for the first releases. Follow-up (not this plan): move it into `release.yml` with a granular `NPM_TOKEN` and `--provenance` (free supply-chain attestation from Actions OIDC now that the repo is public). -`[ ]` Consumer walkthrough in `server/README.md` (or `docs/`): "deploy your own" — a +`[x]` Consumer walkthrough in `server/README.md` (or `docs/`): "deploy your own" — a fresh directory, `bun add @scratchwork/server-deploy-cloudflare`, copy the config shape from `deploy/cloudflare-vanilla`, one command deploy. A `scratchwork server init` scaffolder is explicitly future work, not this plan. Acceptance: in a fresh directory outside the repo, `bun add @scratchwork/server-deploy-cloudflare` + the documented config typechecks and deploys a -working server at the released version. +working server at the released version — and the same walkthrough works with plain +Node/npm (no Bun installed), since the packages are built JS. ### Phase 5 — Release process doc -`[ ]` `RELEASING.md` at root: bump with `set-version.ts` → update `CHANGELOG.md` → PR → +`[x]` `RELEASING.md` at root: bump with `set-version.ts` → update `CHANGELOG.md` → PR → merge → tag `vX.Y.Z` → workflow does the GitHub Release → run `scripts/publish-packages.ts` locally for npm → republish homepage → smoke-test install.sh from a clean machine. Short enough to actually be followed. @@ -194,6 +205,36 @@ install.sh from a clean machine. Short enough to actually be followed. `[ ]` First real release: `v0.2.0` end-to-end, following RELEASING.md as written and fixing the doc where reality disagrees. +## Discovered work (implementation, 2026-07-21) + +Building Phase 4 surfaced prerequisites the plan hadn't named — all shipped in +the same PR: + +- **Package-specifier imports.** server/core, the deploy packages, and the CLI + imported shared via relative `../../../shared/src/...` paths, which escape + the package root and can't publish. All cross-package imports now use + `@scratchwork/shared/...` / `@scratchwork/server-core/...` specifiers with + explicit workspace dependencies; shared's exports map is the pattern pair + `"./*.js"`/`"./*"` → `src`, swapped to `dist` at staging time. +- **`.ts` import extensions.** Relative imports inside the five publishable + packages carry explicit `.ts` extensions so tsc's + `rewriteRelativeImportExtensions` can emit Node-runnable `.js` specifiers. + tsc does not rewrite them in declaration output, so build-packages.ts + post-processes the emitted `.d.ts`. +- **figure.svg module.** The Bun-only ``import ... with { type: "text" }`` + svg imports became a generated module + (`shared/src/assets/figure-svg.generated.ts`, scripts/generate-assets.ts, + freshness-checked in ci) so published code contains no loader-specific syntax. +- **Server tooling workspace dissolved.** `server/scripts/{env,proc,server-settings}.ts` + were imported from published deploy `src/` via relative escapes; they moved + into `server/core/src/deploy/` (published as + `@scratchwork/server-core/deploy/*`), proc.ts was ported from Bun.spawn to + node:child_process so deploys run under plain Node, and the now-empty + `server` tooling workspace was removed. +- **LICENSE.** The repo had none; publishing requires one. Added MIT at the + root (flagged in the PR for veto), `license`/`repository`/`engines` in the + five package manifests. + ## Invariant compliance notes - Release/packaging scripts under `scripts/` and the workflow are deploy tooling — diff --git a/package.json b/package.json index 604e5e0..6078cd9 100644 --- a/package.json +++ b/package.json @@ -8,7 +8,6 @@ "renderer", "shared", "cli", - "server", "server/core", "server/deploy-aws", "server/deploy-cloudflare", @@ -19,7 +18,7 @@ "scripts": { "build": "cd cli && bun run build", "build:renderer": "cd renderer && bun run build", - "ci": "bun scripts/check-boundaries.ts && bun scripts/each-workspace.ts ci && bun scripts/check-generated-fresh.ts", + "ci": "bun scripts/check-boundaries.ts && bun scripts/check-versions.ts && bun scripts/each-workspace.ts ci && bun scripts/check-install-sh.ts && bun scripts/check-npm-pack.ts && bun scripts/generate-assets.ts && bun scripts/check-generated-fresh.ts", "deploy:generic-aws": "cd deploy/generic-aws && bun run deploy", "deploy:cloudflare-vanilla": "cd deploy/cloudflare-vanilla && bun run deploy", "deploy:cloudflare-access": "cd deploy/cloudflare-access && bun run deploy", diff --git a/renderer/build.js b/renderer/build.js index e07b767..49d35bd 100644 --- a/renderer/build.js +++ b/renderer/build.js @@ -202,8 +202,10 @@ export async function buildDist() { OUT_SHARED, [ "// AUTO-GENERATED by renderer/build.js — do not edit.", - `export const defaultRendererSourceHash = ${JSON.stringify(sourceHash)};`, - `export const defaultRendererHtml = ${JSON.stringify(html)};`, + "// The JSDoc casts keep tsc declaration emit at `string` instead of a", + "// megabyte-scale literal type when shared is built for npm publishing.", + `export const defaultRendererSourceHash = /** @type {string} */ (${JSON.stringify(sourceHash)});`, + `export const defaultRendererHtml = /** @type {string} */ (${JSON.stringify(html)});`, "export default defaultRendererHtml;", "", ].join("\n"), diff --git a/scripts/build-packages.ts b/scripts/build-packages.ts new file mode 100644 index 0000000..2299374 --- /dev/null +++ b/scripts/build-packages.ts @@ -0,0 +1,132 @@ +#!/usr/bin/env bun +/* + * Builds the five publishable npm packages (notes/distribution-plan.md + * Phase 4) and stages publishable directories under release/packages/: + * + * 1. tsc emit (ESM JS + .d.ts) into each package's dist/, in dependency + * order (shared → server-core → deploy adapters) so each build resolves + * its workspace deps against their already-built dist via tsconfig paths. + * 2. Post-process emitted .d.ts: tsc's rewriteRelativeImportExtensions + * rewrites `.ts` specifiers to `.js` in JS output but NOT in declaration + * output, so the same rewrite is applied here — a shipped d.ts referencing + * "./x.ts" would fail consumers' typechecking. + * 3. Stage release/packages// with dist/, README.md, the root LICENSE, + * and a publish-shaped package.json: exports/files pointed at dist/, + * workspace:* pinned to the lockstep version, dev-only fields dropped. + * (Neither bun nor npm applies publishConfig.exports, and `bun publish` + * only rewrites workspace:* when run from the workspace itself, so the + * staging transform owns both.) + * + * scripts/check-npm-pack.ts packs and verifies the staged output in ci; + * scripts/publish-packages.ts publishes it. + */ +import { cpSync, existsSync, mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { join } from "node:path"; +import { repoRoot } from "./workspaces"; + +/** Publishable packages in dependency order (dir is repo-relative). */ +export const PUBLISHABLE = [ + "shared", + "server/core", + "server/deploy-local", + "server/deploy-aws", + "server/deploy-cloudflare", +] as const; + +const rootVersion = JSON.parse(readFileSync(join(repoRoot, "package.json"), "utf8")).version as string; + +/** Maps an in-repo exports target (./src/X.ts, ./src/*.js) to its published shape. */ +function publishedExportTarget(target: string): { types: string; default: string } { + if (!target.startsWith("./src/")) { + throw new Error(`exports target ${target} does not point into src/ — extend the staging transform`); + } + const inDist = "./dist/" + target.slice("./src/".length); + if (inDist.endsWith(".ts")) { + const base = inDist.slice(0, -3); + return { types: `${base}.d.ts`, default: `${base}.js` }; + } + if (inDist.endsWith(".js")) { + const base = inDist.slice(0, -3); + return { types: `${base}.d.ts`, default: `${base}.js` }; + } + throw new Error(`exports target ${target} has an unexpected extension — extend the staging transform`); +} + +/** The publish-shaped package.json: dist exports, pinned deps, dev fields dropped. */ +function publishManifest(pkg: Record): Record { + const exports = Object.fromEntries( + Object.entries(pkg.exports as Record).map(([key, target]) => [ + key, + publishedExportTarget(target), + ]), + ); + const dependencies = Object.fromEntries( + Object.entries((pkg.dependencies as Record) ?? {}).map(([name, range]) => [ + name, + range === "workspace:*" ? rootVersion : range, + ]), + ); + const { name, version, type, description, license, repository, engines, keywords } = pkg as Record; + return { + name, + version, + type, + description, + license, + repository, + engines, + ...(keywords ? { keywords } : {}), + exports, + files: ["dist"], + dependencies, + }; +} + +/** Rewrites relative `.ts` import specifiers to `.js` in an emitted .d.ts. */ +function rewriteDeclarationSpecifiers(text: string): string { + return text.replace(/((?:from\s*|import\s*\()\s*["'])(\.{1,2}\/[^"']*)\.ts(["'])/g, "$1$2.js$3"); +} + +export function buildAndStage(): string[] { + const staged: string[] = []; + for (const dir of PUBLISHABLE) { + const packageDir = join(repoRoot, dir); + rmSync(join(packageDir, "dist"), { recursive: true, force: true }); + const tsc = Bun.spawnSync(["bunx", "tsc", "-p", "tsconfig.build.json"], { + cwd: packageDir, + stdout: "inherit", + stderr: "inherit", + }); + if (!tsc.success) { + console.error(`build-packages: tsc failed for ${dir}`); + process.exit(1); + } + for (const file of new Bun.Glob("**/*.d.ts").scanSync({ cwd: join(packageDir, "dist") })) { + const path = join(packageDir, "dist", file); + const text = readFileSync(path, "utf8"); + const rewritten = rewriteDeclarationSpecifiers(text); + if (rewritten !== text) writeFileSync(path, rewritten); + } + + const pkg = JSON.parse(readFileSync(join(packageDir, "package.json"), "utf8")); + const staging = join(repoRoot, "release", "packages", dir.replaceAll("/", "-")); + rmSync(staging, { recursive: true, force: true }); + mkdirSync(staging, { recursive: true }); + cpSync(join(packageDir, "dist"), join(staging, "dist"), { recursive: true }); + writeFileSync(join(staging, "package.json"), JSON.stringify(publishManifest(pkg), null, 2) + "\n"); + cpSync(join(repoRoot, "LICENSE"), join(staging, "LICENSE")); + const readme = join(packageDir, "README.md"); + if (!existsSync(readme)) { + console.error(`build-packages: ${dir} has no README.md — every published package ships one`); + process.exit(1); + } + cpSync(readme, join(staging, "README.md")); + staged.push(staging); + console.log(`staged ${staging.slice(repoRoot.length + 1)} (${pkg.name}@${rootVersion})`); + } + return staged; +} + +if (import.meta.main) { + buildAndStage(); +} diff --git a/scripts/check-boundaries.ts b/scripts/check-boundaries.ts index d19a758..61307ef 100644 --- a/scripts/check-boundaries.ts +++ b/scripts/check-boundaries.ts @@ -38,6 +38,8 @@ const ASYNC_BOUNDARIES: Readonly> = { "server/core/src/google-jwt.ts": "Google OAuth token-endpoint POST + JWKS fetch — the identity-provider edge", "server/core/src/cloudflare-jwt.ts": "Cloudflare Access JWKS fetch — the identity-provider edge", "cli/src/commands/login-callback-server.ts": "Bun.serve loopback listener — platform entrypoint for the login callback", + "server/core/src/deploy/env.ts": "deploy tooling — dotenv loading for deploy scripts (plain Promise-based, runs once on a developer's machine)", + "server/core/src/deploy/proc.ts": "deploy tooling — node:child_process spawning for deploy scripts (Promise-based, runs once on a developer's machine)", "server/deploy-aws/src/handler.ts": "AWS Lambda entrypoint — the platform's contract is Promise-based", "server/deploy-aws/src/s3-storage.ts": "AWS SDK S3 Promise APIs wrapped into the ObjectStorage service", "server/deploy-aws/src/deploy.ts": "deploy tooling — deliberately plain Promise-based script code (runs once on a developer's machine)", diff --git a/scripts/check-generated-fresh.ts b/scripts/check-generated-fresh.ts index 6f21d57..167e514 100644 --- a/scripts/check-generated-fresh.ts +++ b/scripts/check-generated-fresh.ts @@ -13,7 +13,10 @@ const root = dirname(dirname(fileURLToPath(import.meta.url))); // Every generated file committed to the repo. dist/ outputs are gitignored // and don't belong here. -const artifacts = ["shared/src/site/default-renderer.generated.js"]; +const artifacts = [ + "shared/src/site/default-renderer.generated.js", + "shared/src/assets/figure-svg.generated.ts", +]; const status = Bun.spawnSync( ["git", "status", "--porcelain", "--", ...artifacts], diff --git a/scripts/check-install-sh.ts b/scripts/check-install-sh.ts new file mode 100644 index 0000000..c9b0211 --- /dev/null +++ b/scripts/check-install-sh.ts @@ -0,0 +1,150 @@ +#!/usr/bin/env bun +/* + * Mechanized checks for docs/install.sh (notes/distribution-plan.md Phase 3), + * run inside the root `bun run ci`: + * + * 1. `sh -n` syntax check. + * 2. Full install loop against a local HTTP fixture standing in for GitHub + * Releases (no network): latest install, pinned-version install, + * checksum-mismatch rejection, and uname→target platform mapping via a + * fake `uname` on PATH. The "binaries" are tiny sh scripts that answer + * `--version`, so the loop exercises download → verify → extract → + * install → run without real release assets. + */ +import { chmodSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { repoRoot } from "./workspaces"; + +const script = join(repoRoot, "docs", "install.sh"); +const failures: string[] = []; + +// ── 1. Syntax ─────────────────────────────────────────────────────────────── +const syntax = Bun.spawnSync(["sh", "-n", script], { stderr: "pipe" }); +if (!syntax.success) { + failures.push(`sh -n docs/install.sh failed:\n${syntax.stderr.toString()}`); +} + +// ── 2. Hermetic fixture ───────────────────────────────────────────────────── +const work = mkdtempSync(join(tmpdir(), "install-sh-check-")); +const TARGETS = ["darwin-arm64", "darwin-x64", "linux-x64", "linux-arm64"]; + +/** Builds a fake release: per-target tarballs of an sh "binary" + checksums.txt. */ +function makeRelease(version: string): { dir: string; checksums: string } { + const dir = join(work, `release-v${version}`); + mkdirSync(dir, { recursive: true }); + const lines: string[] = []; + for (const target of TARGETS) { + const staging = join(dir, `staging-${target}`); + mkdirSync(staging); + writeFileSync(join(staging, "scratchwork"), `#!/bin/sh\necho "${version} (${target})"\n`); + chmodSync(join(staging, "scratchwork"), 0o755); + const archive = join(dir, `scratchwork-v${version}-${target}.tar.gz`); + const tar = Bun.spawnSync(["tar", "-czf", archive, "-C", staging, "scratchwork"]); + if (!tar.success) throw new Error(`fixture tar failed for ${archive}`); + const digest = new Bun.CryptoHasher("sha256").update(readFileSync(archive)).digest("hex"); + lines.push(`${digest} scratchwork-v${version}-${target}.tar.gz`); + } + const checksums = lines.join("\n") + "\n"; + writeFileSync(join(dir, "checksums.txt"), checksums); + return { dir, checksums }; +} + +const latest = makeRelease("9.9.9"); +const pinned = makeRelease("8.8.8"); +const releases: Record = { "9.9.9": latest, "8.8.8": pinned }; + +let tamperChecksums = false; +const server = Bun.serve({ + port: 0, + hostname: "127.0.0.1", + fetch(request) { + const path = new URL(request.url).pathname; + const latestMatch = path === "/latest/download/checksums.txt"; + const versioned = path.match(/^\/download\/v([^/]+)\/(.+)$/); + const version = latestMatch ? "9.9.9" : versioned?.[1]; + const file = latestMatch ? "checksums.txt" : versioned?.[2]; + const release = version != null ? releases[version] : undefined; + if (release == null || file == null) return new Response("not found", { status: 404 }); + if (file === "checksums.txt" && tamperChecksums) { + const tampered = readFileSync(join(release.dir, file), "utf8").replace(/^[0-9a-f]{8}/gm, "00000000"); + return new Response(tampered); + } + return new Response(Bun.file(join(release.dir, file))); + }, +}); +const base = `http://127.0.0.1:${server.port}`; + +// A fake uname for platform-mapping cases; real uname answers otherwise. +const fakeBin = join(work, "fake-bin"); +mkdirSync(fakeBin); +writeFileSync( + join(fakeBin, "uname"), + `#!/bin/sh\ncase "\${1:-}" in\n -s) echo "$FAKE_UNAME_S" ;;\n -m) echo "$FAKE_UNAME_M" ;;\n *) echo "Fake" ;;\nesac\n`, +); +chmodSync(join(fakeBin, "uname"), 0o755); + +interface RunResult { + code: number; + stdout: string; + stderr: string; +} + +let caseNumber = 0; +function runInstall(env: Record): RunResult { + const installDir = join(work, `install-${caseNumber++}`); + const proc = Bun.spawnSync(["sh", script], { + env: { + PATH: env.FAKE_UNAME_S ? `${fakeBin}:${process.env.PATH}` : process.env.PATH, + HOME: work, + SCRATCHWORK_DOWNLOAD_BASE: base, + SCRATCHWORK_INSTALL_DIR: installDir, + ...env, + }, + stdout: "pipe", + stderr: "pipe", + }); + return { code: proc.exitCode, stdout: proc.stdout.toString(), stderr: proc.stderr.toString() }; +} + +function expect(name: string, condition: boolean, detail: RunResult) { + if (!condition) { + failures.push(`${name}\n exit=${detail.code}\n stdout: ${detail.stdout.trim()}\n stderr: ${detail.stderr.trim()}`); + } +} + +// Latest install on the real host platform. +let result = runInstall({}); +expect("latest install should succeed and run the binary", result.code === 0 && result.stdout.includes("9.9.9"), result); + +// Pinned version install. +result = runInstall({ SCRATCHWORK_VERSION: "8.8.8" }); +expect("pinned install should fetch v8.8.8", result.code === 0 && result.stdout.includes("8.8.8"), result); + +// Checksum tampering must be rejected. +tamperChecksums = true; +result = runInstall({}); +expect("tampered checksums must fail with a mismatch", result.code !== 0 && result.stderr.includes("checksum mismatch"), result); +tamperChecksums = false; + +// Platform mapping: unsupported OS and architecture fail with clear errors... +result = runInstall({ FAKE_UNAME_S: "FreeBSD", FAKE_UNAME_M: "x86_64" }); +expect("unsupported OS must fail before downloading", result.code !== 0 && result.stderr.includes("unsupported operating system"), result); +result = runInstall({ FAKE_UNAME_S: "Linux", FAKE_UNAME_M: "riscv64" }); +expect("unsupported architecture must fail", result.code !== 0 && result.stderr.includes("unsupported architecture"), result); + +// ...and supported uname spellings map to the right release asset. +result = runInstall({ FAKE_UNAME_S: "Linux", FAKE_UNAME_M: "aarch64" }); +expect("Linux/aarch64 must map to the linux-arm64 asset", result.code === 0 && result.stdout.includes("(linux-arm64)"), result); +result = runInstall({ FAKE_UNAME_S: "Darwin", FAKE_UNAME_M: "arm64" }); +expect("Darwin/arm64 must map to the darwin-arm64 asset", result.code === 0 && result.stdout.includes("(darwin-arm64)"), result); + +server.stop(true); +rmSync(work, { recursive: true, force: true }); + +if (failures.length > 0) { + console.error(`check-install-sh: ${failures.length} failure(s)\n`); + for (const failure of failures) console.error(failure + "\n"); + process.exit(1); +} +console.log("check-install-sh: syntax ok; latest, pinned, checksum-rejection, and platform-mapping cases pass against the local fixture"); diff --git a/scripts/check-npm-pack.ts b/scripts/check-npm-pack.ts new file mode 100644 index 0000000..1fdcfc7 --- /dev/null +++ b/scripts/check-npm-pack.ts @@ -0,0 +1,173 @@ +#!/usr/bin/env bun +/* + * Mechanized dry-run of npm publishing (notes/distribution-plan.md Phase 4), + * run inside the root `bun run ci`. No network. + * + * 1. Build + stage every publishable package (scripts/build-packages.ts) + * and `bun pm pack` each staged directory. + * 2. Assert tarball shape: dist JS + matching d.ts, README, LICENSE; a + * manifest with no workspace:/private/scripts/devDependencies, every + * exports target present in the tarball, and the lockstep version. + * 3. Assemble a consumer node_modules from the packed tarballs (external + * deps symlinked from the repo's node_modules, standing in for a + * registry install) and import every package's entrypoint under plain + * Node — except deploy-local, which requires Bun at runtime and is + * imported under Bun instead. + * 4. Typecheck a tiny consumer under NodeNext resolution against the + * shipped declarations — the strictest consumer tsc configuration. + */ +import { cpSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, rmSync, symlinkSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { buildAndStage } from "./build-packages"; +import { repoRoot } from "./workspaces"; + +const failures: string[] = []; +const rootVersion = JSON.parse(readFileSync(join(repoRoot, "package.json"), "utf8")).version as string; + +const staged = buildAndStage(); + +interface PackedPackage { + readonly name: string; + readonly stagingDir: string; + readonly tarball: string; + readonly entries: readonly string[]; + readonly manifest: Record; +} + +function pack(stagingDir: string): PackedPackage | null { + const packed = Bun.spawnSync(["bun", "pm", "pack"], { cwd: stagingDir, stdout: "pipe", stderr: "pipe" }); + if (!packed.success) { + failures.push(`bun pm pack failed in ${stagingDir}:\n${packed.stderr.toString()}`); + return null; + } + const tarballName = readdirSync(stagingDir).find((file) => file.endsWith(".tgz")); + if (tarballName == null) { + failures.push(`bun pm pack produced no tarball in ${stagingDir}`); + return null; + } + const tarball = join(stagingDir, tarballName); + const list = Bun.spawnSync(["tar", "-tzf", tarball], { stdout: "pipe", stderr: "pipe" }); + const entries = list.stdout.toString().trim().split("\n"); + const manifestText = Bun.spawnSync(["tar", "-xzOf", tarball, "package/package.json"], { stdout: "pipe" }); + const manifest = JSON.parse(manifestText.stdout.toString()); + return { name: manifest.name, stagingDir, tarball, entries, manifest }; +} + +function checkTarball(pkg: PackedPackage): void { + const { name, entries, manifest } = pkg; + for (const required of ["package/package.json", "package/README.md", "package/LICENSE"]) { + if (!entries.includes(required)) failures.push(`${name}: tarball is missing ${required}`); + } + const jsFiles = entries.filter((entry) => entry.startsWith("package/dist/") && entry.endsWith(".js")); + if (jsFiles.length === 0) failures.push(`${name}: tarball has no built JS under dist/`); + for (const js of jsFiles) { + const declaration = js.slice(0, -3) + ".d.ts"; + if (!entries.includes(declaration)) failures.push(`${name}: ${js} has no matching ${declaration}`); + } + const offenders = entries.filter((entry) => /\.test\.|\/test\/|\.ts$/.test(entry) && !entry.endsWith(".d.ts")); + if (offenders.length > 0) failures.push(`${name}: tarball ships test or source files: ${offenders.join(", ")}`); + + const raw = JSON.stringify(manifest); + if (raw.includes("workspace:")) failures.push(`${name}: manifest still contains workspace: ranges`); + for (const field of ["private", "scripts", "devDependencies"]) { + if (field in manifest) failures.push(`${name}: manifest ships dev-only field "${field}"`); + } + if (manifest.version !== rootVersion) failures.push(`${name}: version ${manifest.version} != lockstep ${rootVersion}`); + if (manifest.license !== "MIT" || manifest.repository == null) failures.push(`${name}: missing license/repository metadata`); + for (const [subpath, target] of Object.entries(manifest.exports as Record)) { + for (const kind of ["types", "default"] as const) { + const file = target[kind]; + // Pattern exports are spot-checked by the import smoke below instead. + if (file.includes("*")) continue; + if (!entries.includes(`package/${file.slice(2)}`)) { + failures.push(`${name}: exports["${subpath}"].${kind} points at ${file}, which is not in the tarball`); + } + } + } +} + +/** Lays out a consumer install: tarballs extracted into node_modules/@scratchwork, + * every other dependency symlinked from the repo root's node_modules. */ +function assembleConsumer(packages: readonly PackedPackage[]): string { + const consumer = mkdtempSync(join(tmpdir(), "scratchwork-consumer-")); + const nodeModules = join(consumer, "node_modules"); + mkdirSync(join(nodeModules, "@scratchwork"), { recursive: true }); + for (const entry of readdirSync(join(repoRoot, "node_modules"))) { + if (entry === "@scratchwork" || entry.startsWith(".")) continue; + symlinkSync(join(repoRoot, "node_modules", entry), join(nodeModules, entry)); + } + for (const pkg of packages) { + const extractDir = join(consumer, "extract", pkg.name); + mkdirSync(extractDir, { recursive: true }); + const untar = Bun.spawnSync(["tar", "-xzf", pkg.tarball, "-C", extractDir]); + if (!untar.success) { + failures.push(`${pkg.name}: could not extract ${pkg.tarball}`); + continue; + } + cpSync(join(extractDir, "package"), join(nodeModules, pkg.name), { recursive: true }); + } + rmSync(join(consumer, "extract"), { recursive: true, force: true }); + return consumer; +} + +function importSmoke(consumer: string, runtime: "node" | "bun", specifier: string): void { + const result = Bun.spawnSync( + [runtime, "--input-type=module", "-e", `await import(${JSON.stringify(specifier)});`].filter( + (arg) => runtime === "node" || arg !== "--input-type=module", + ), + { cwd: consumer, stdout: "pipe", stderr: "pipe" }, + ); + if (!result.success) { + failures.push(`${specifier}: import under ${runtime} failed:\n${result.stderr.toString().slice(0, 2000)}`); + } +} + +const packages = staged.map(pack).filter((pkg): pkg is PackedPackage => pkg != null); +for (const pkg of packages) checkTarball(pkg); + +if (failures.length === 0) { + const consumer = assembleConsumer(packages); + importSmoke(consumer, "node", "@scratchwork/shared/publish/api"); + importSmoke(consumer, "node", "@scratchwork/shared/site/serve"); + importSmoke(consumer, "node", "@scratchwork/server-core"); + importSmoke(consumer, "node", "@scratchwork/server-core/deploy/server-settings"); + importSmoke(consumer, "node", "@scratchwork/server-deploy-aws"); + importSmoke(consumer, "node", "@scratchwork/server-deploy-cloudflare"); + importSmoke(consumer, "bun", "@scratchwork/server-deploy-local"); + + // Consumer typecheck under NodeNext — the strictest resolution a user runs. + writeFileSync( + join(consumer, "consumer.ts"), + [ + 'import { ScratchworkApi } from "@scratchwork/shared/publish/api";', + 'import { contentType } from "@scratchwork/shared/site/content";', + 'import type { ScratchworkServerConfig } from "@scratchwork/server-core/deploy/server-settings";', + 'import { deployServer } from "@scratchwork/server-deploy-cloudflare";', + "const config: ScratchworkServerConfig = { auth: \"oauth\", appDomain: \"app.example.com\" };", + "void ScratchworkApi; void contentType; void deployServer; void config;", + "", + ].join("\n"), + ); + const tsc = Bun.spawnSync( + [ + "bunx", "tsc", "--noEmit", "--strict", "--skipLibCheck", + "--module", "nodenext", "--moduleResolution", "nodenext", "--target", "esnext", + join(consumer, "consumer.ts"), + ], + { cwd: consumer, stdout: "pipe", stderr: "pipe" }, + ); + if (!tsc.success) { + failures.push(`consumer typecheck under NodeNext failed:\n${tsc.stdout.toString().slice(0, 2000)}`); + } + rmSync(consumer, { recursive: true, force: true }); +} + +if (failures.length > 0) { + console.error(`check-npm-pack: ${failures.length} failure(s)\n`); + for (const failure of failures) console.error(failure + "\n"); + process.exit(1); +} +console.log( + `check-npm-pack: ${packages.length} tarballs verified (shape, manifest, Node/Bun import smoke, NodeNext consumer typecheck)`, +); diff --git a/scripts/check-versions.ts b/scripts/check-versions.ts new file mode 100644 index 0000000..8c4bd15 --- /dev/null +++ b/scripts/check-versions.ts @@ -0,0 +1,24 @@ +#!/usr/bin/env bun +/* + * Mechanizes the lockstep-version rule (notes/distribution-plan.md decision 1): + * every workspace package.json carries exactly the root version. A drifted + * bump fails the gate; bump with `bun scripts/set-version.ts `. + */ +import { readFileSync } from "node:fs"; +import { join } from "node:path"; +import { repoRoot, workspaceDirs } from "./workspaces"; + +const rootVersion = JSON.parse(readFileSync(join(repoRoot, "package.json"), "utf8")).version as string; +const drifted: string[] = []; +const dirs = workspaceDirs(); +for (const dir of dirs) { + const pkg = JSON.parse(readFileSync(join(repoRoot, dir, "package.json"), "utf8")); + if (pkg.version !== rootVersion) drifted.push(`${dir}: ${pkg.version}`); +} +if (drifted.length > 0) { + console.error(`check-versions: root is ${rootVersion} but these workspaces drifted:\n`); + for (const line of drifted) console.error(` ${line}`); + console.error("\nStamp them back into lockstep with `bun scripts/set-version.ts `."); + process.exit(1); +} +console.log(`check-versions: root and all ${dirs.length} workspaces at ${rootVersion}`); diff --git a/scripts/each-workspace.ts b/scripts/each-workspace.ts index 25fbc8d..53c177d 100644 --- a/scripts/each-workspace.ts +++ b/scripts/each-workspace.ts @@ -10,12 +10,11 @@ * interleave. Every workspace runs even if another fails, and all failures * are reported at the end. */ -import { existsSync, readdirSync, readFileSync } from "node:fs"; -import { dirname, join } from "node:path"; -import { fileURLToPath } from "node:url"; +import { readFileSync } from "node:fs"; +import { join } from "node:path"; import { createPool, runPooled } from "./pool"; +import { repoRoot as root, workspaceDirs } from "./workspaces"; -const root = dirname(dirname(fileURLToPath(import.meta.url))); const script = process.argv[2]; const runnable = ["typecheck", "test", "ci"]; if (!script || !runnable.includes(script)) { @@ -29,25 +28,6 @@ if (!script || !runnable.includes(script)) { // e2e bundles the CLI (which embeds renderer artifacts), so it starts after cli. const runAfter: Record = { cli: ["renderer"], e2e: ["cli"] }; -/** Expands the root workspaces globs (literal dirs and trailing "/*") into workspace dirs. */ -function workspaceDirs(): string[] { - const rootPackage = JSON.parse(readFileSync(join(root, "package.json"), "utf8")); - const dirs: string[] = []; - for (const pattern of rootPackage.workspaces as string[]) { - if (pattern.endsWith("/*")) { - const parent = pattern.slice(0, -2); - for (const entry of readdirSync(join(root, parent), { withFileTypes: true })) { - if (entry.isDirectory() && existsSync(join(root, parent, entry.name, "package.json"))) { - dirs.push(join(parent, entry.name)); - } - } - } else { - dirs.push(pattern); - } - } - return dirs; -} - const dirs = workspaceDirs(); const missing = dirs.filter((dir) => { const pkg = JSON.parse(readFileSync(join(root, dir, "package.json"), "utf8")); diff --git a/scripts/generate-assets.ts b/scripts/generate-assets.ts new file mode 100644 index 0000000..dff2f23 --- /dev/null +++ b/scripts/generate-assets.ts @@ -0,0 +1,26 @@ +#!/usr/bin/env bun +/* + * Regenerates shared/src/assets/figure-svg.generated.ts from + * shared/assets/figure.svg. The module form (instead of importing the .svg + * with a Bun `with { type: "text" }` attribute) keeps the publishable + * packages free of loader-specific syntax so their tsc-built dist runs under + * plain Node. Runs in the root `bun run ci`; freshness is enforced by + * scripts/check-generated-fresh.ts. + */ +import { mkdirSync, readFileSync, writeFileSync } from "node:fs"; +import { dirname, join } from "node:path"; +import { repoRoot } from "./workspaces"; + +const svg = readFileSync(join(repoRoot, "shared", "assets", "figure.svg"), "utf8"); +const out = join(repoRoot, "shared", "src", "assets", "figure-svg.generated.ts"); +mkdirSync(dirname(out), { recursive: true }); +writeFileSync( + out, + [ + "// Generated by scripts/generate-assets.ts from shared/assets/figure.svg — do not edit.", + "// Regenerate with: bun scripts/generate-assets.ts", + `export const FIGURE_SVG: string = ${JSON.stringify(svg)};`, + "", + ].join("\n"), +); +console.log(`generated ${out.slice(repoRoot.length + 1)} (${svg.length} chars of svg)`); diff --git a/scripts/package-release.ts b/scripts/package-release.ts new file mode 100644 index 0000000..560ef39 --- /dev/null +++ b/scripts/package-release.ts @@ -0,0 +1,57 @@ +#!/usr/bin/env bun +/* + * Packages the cross-compiled CLI binaries (cli/dist/scratchwork--, + * built by `cd cli && bun build.js --all-targets`) into release/ at the repo + * root: + * + * scratchwork-v--.tar.gz one per target, containing a + * single file named `scratchwork` + * checksums.txt SHA-256 of each archive, in + * `sha256sum` format + * + * Tarball rather than bare binary so the executable bit survives and the name + * inside is stable. checksums.txt is also how install.sh discovers the latest + * version (its asset names carry the version), so its name stays unversioned. + */ +import { chmodSync, cpSync, existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { repoRoot } from "./workspaces"; +// RELEASE_TARGETS is exported by cli/build.js, but importing it would run the +// build; keep the list in scripts/release-targets.ts, shared by both. +import { RELEASE_TARGETS } from "./release-targets"; + +const version = JSON.parse(readFileSync(join(repoRoot, "package.json"), "utf8")).version as string; +const releaseDir = join(repoRoot, "release"); +rmSync(releaseDir, { recursive: true, force: true }); +mkdirSync(releaseDir, { recursive: true }); + +const checksums: string[] = []; +for (const target of RELEASE_TARGETS) { + const binary = join(repoRoot, "cli", "dist", `scratchwork-${target}`); + if (!existsSync(binary)) { + console.error(`package-release: missing ${binary} — run \`cd cli && bun build.js --all-targets\` first`); + process.exit(1); + } + const archiveName = `scratchwork-v${version}-${target}.tar.gz`; + const staging = mkdtempSync(join(tmpdir(), "scratchwork-release-")); + try { + cpSync(binary, join(staging, "scratchwork")); + chmodSync(join(staging, "scratchwork"), 0o755); + const tar = Bun.spawnSync( + ["tar", "-czf", join(releaseDir, archiveName), "-C", staging, "scratchwork"], + { stdout: "inherit", stderr: "inherit" }, + ); + if (!tar.success) { + console.error(`package-release: tar failed for ${archiveName}`); + process.exit(1); + } + } finally { + rmSync(staging, { recursive: true, force: true }); + } + const digest = new Bun.CryptoHasher("sha256").update(readFileSync(join(releaseDir, archiveName))).digest("hex"); + checksums.push(`${digest} ${archiveName}`); + console.log(`release/${archiveName}`); +} +writeFileSync(join(releaseDir, "checksums.txt"), checksums.join("\n") + "\n"); +console.log(`release/checksums.txt (${checksums.length} archives, version ${version})`); diff --git a/scripts/publish-packages.ts b/scripts/publish-packages.ts new file mode 100644 index 0000000..96438da --- /dev/null +++ b/scripts/publish-packages.ts @@ -0,0 +1,46 @@ +#!/usr/bin/env bun +/* + * Publishes the five packages to npm from their staged directories, in + * dependency order (notes/distribution-plan.md Phase 4). Run locally with an + * authenticated npm CLI after the GitHub Release exists (see RELEASING.md): + * + * bun scripts/publish-packages.ts [--dry-run] + * + * Refuses to run on a dirty tree or when HEAD isn't the tag matching the + * lockstep version, so what's published is exactly what's tagged. Uses + * `npm publish --access public` (scoped packages default to restricted). + * Moving this into release.yml with a granular NPM_TOKEN and --provenance is + * a follow-up once the manual loop is boring. + */ +import { readFileSync } from "node:fs"; +import { join } from "node:path"; +import { buildAndStage } from "./build-packages"; +import { repoRoot } from "./workspaces"; + +const dryRun = process.argv.includes("--dry-run"); +const version = JSON.parse(readFileSync(join(repoRoot, "package.json"), "utf8")).version as string; + +const status = Bun.spawnSync(["git", "status", "--porcelain"], { cwd: repoRoot, stdout: "pipe" }); +if (status.stdout.toString().trim() !== "") { + console.error("publish-packages: working tree is dirty — publish only from a clean checkout of the release tag"); + process.exit(1); +} +const tags = Bun.spawnSync(["git", "tag", "--points-at", "HEAD"], { cwd: repoRoot, stdout: "pipe" }); +const expected = `v${version}`; +if (!tags.stdout.toString().split("\n").map((tag) => tag.trim()).includes(expected)) { + console.error(`publish-packages: HEAD is not tagged ${expected} — tag the release first (see RELEASING.md)`); + process.exit(1); +} + +const staged = buildAndStage(); +for (const stagingDir of staged) { + const name = JSON.parse(readFileSync(join(stagingDir, "package.json"), "utf8")).name as string; + const args = ["npm", "publish", "--access", "public", ...(dryRun ? ["--dry-run"] : [])]; + console.log(`\n${name}@${version}: ${args.join(" ")}`); + const publish = Bun.spawnSync(args, { cwd: stagingDir, stdout: "inherit", stderr: "inherit" }); + if (!publish.success) { + console.error(`publish-packages: publish failed for ${name} — packages published before it remain live`); + process.exit(1); + } +} +console.log(`\npublished ${staged.length} packages at ${version}${dryRun ? " (dry run)" : ""}`); diff --git a/scripts/release-notes.ts b/scripts/release-notes.ts new file mode 100644 index 0000000..86327c4 --- /dev/null +++ b/scripts/release-notes.ts @@ -0,0 +1,32 @@ +#!/usr/bin/env bun +/* + * Prints the CHANGELOG.md section for one version to stdout — the release + * workflow pipes it into the GitHub Release notes. Fails if the section is + * missing, so a release can't ship with empty notes. + * + * Usage: bun scripts/release-notes.ts + */ +import { readFileSync } from "node:fs"; +import { join } from "node:path"; +import { repoRoot } from "./workspaces"; + +const version = process.argv[2]; +if (!version) { + console.error("usage: bun scripts/release-notes.ts "); + process.exit(1); +} +const changelog = readFileSync(join(repoRoot, "CHANGELOG.md"), "utf8"); +const lines = changelog.split("\n"); +const start = lines.findIndex((line) => line.trim() === `## v${version}`); +if (start === -1) { + console.error(`release-notes: CHANGELOG.md has no "## v${version}" section — write one before releasing`); + process.exit(1); +} +let end = lines.length; +for (let i = start + 1; i < lines.length; i++) { + if (lines[i].startsWith("## ")) { + end = i; + break; + } +} +console.log(lines.slice(start + 1, end).join("\n").trim()); diff --git a/scripts/release-targets.ts b/scripts/release-targets.ts new file mode 100644 index 0000000..10e5e2d --- /dev/null +++ b/scripts/release-targets.ts @@ -0,0 +1,7 @@ +/* + * The CLI release target matrix (decision 4 in notes/distribution-plan.md): + * cross-compiled by `cd cli && bun build.js --all-targets`, packaged by + * scripts/package-release.ts, and mapped from `uname` by docs/install.sh. + * Windows and musl (Alpine) are explicit non-goals for v0. + */ +export const RELEASE_TARGETS = ["darwin-arm64", "darwin-x64", "linux-x64", "linux-arm64"] as const; diff --git a/scripts/set-version.ts b/scripts/set-version.ts new file mode 100644 index 0000000..35c656a --- /dev/null +++ b/scripts/set-version.ts @@ -0,0 +1,29 @@ +#!/usr/bin/env bun +/* + * Stamps the lockstep version into the root package.json and every workspace + * package.json. One version for the whole repo (see notes/distribution-plan.md + * decision 1): CLI binaries and all npm packages share it, and the git tag + * vX.Y.Z on main is the release trigger. This script's only duty is stamping — + * tagging and CHANGELOG.md stay manual and visible (see RELEASING.md). + * + * Usage: bun scripts/set-version.ts + */ +import { readFileSync, writeFileSync } from "node:fs"; +import { join } from "node:path"; +import { repoRoot, workspaceDirs } from "./workspaces"; + +const version = process.argv[2]; +if (!version || !/^\d+\.\d+\.\d+(-[0-9A-Za-z.-]+)?$/.test(version)) { + console.error("usage: bun scripts/set-version.ts "); + process.exit(1); +} + +const manifests = ["", ...workspaceDirs()].map((dir) => join(repoRoot, dir, "package.json")); +for (const manifest of manifests) { + const pkg = JSON.parse(readFileSync(manifest, "utf8")); + const previous = pkg.version; + pkg.version = version; + writeFileSync(manifest, JSON.stringify(pkg, null, 2) + "\n"); + console.log(`${manifest.slice(repoRoot.length + 1)}: ${previous} -> ${version}`); +} +console.log(`\nstamped ${manifests.length} manifests to ${version}`); diff --git a/scripts/workspaces.ts b/scripts/workspaces.ts new file mode 100644 index 0000000..9ec6ddb --- /dev/null +++ b/scripts/workspaces.ts @@ -0,0 +1,30 @@ +/* + * Workspace discovery shared by the root scripts: expands the root + * package.json workspaces globs (literal dirs and trailing "/*") into the + * list of workspace directories, so every script derives the same list and a + * new workspace joins them all automatically. + */ +import { existsSync, readdirSync, readFileSync } from "node:fs"; +import { dirname, join } from "node:path"; +import { fileURLToPath } from "node:url"; + +export const repoRoot = dirname(dirname(fileURLToPath(import.meta.url))); + +/** Expands the root workspaces globs into workspace dirs (repo-relative). */ +export function workspaceDirs(): string[] { + const rootPackage = JSON.parse(readFileSync(join(repoRoot, "package.json"), "utf8")); + const dirs: string[] = []; + for (const pattern of rootPackage.workspaces as string[]) { + if (pattern.endsWith("/*")) { + const parent = pattern.slice(0, -2); + for (const entry of readdirSync(join(repoRoot, parent), { withFileTypes: true })) { + if (entry.isDirectory() && existsSync(join(repoRoot, parent, entry.name, "package.json"))) { + dirs.push(join(parent, entry.name)); + } + } + } else { + dirs.push(pattern); + } + } + return dirs; +} diff --git a/server/README.md b/server/README.md index 50e1c2e..67cf24c 100644 --- a/server/README.md +++ b/server/README.md @@ -190,3 +190,40 @@ SCRATCHWORK_SESSION_SECRET=use-at-least-32-random-bytes Cloudflare deploy attaches non-secret auth values to the Worker as plain-text bindings and uploads `SCRATCHWORK_GOOGLE_CLIENT_SECRET` plus `SCRATCHWORK_SESSION_SECRET` through the Workers secrets API. AWS deploy sends `SCRATCHWORK_*` values to Lambda environment variables. Set `SCRATCHWORK_APP_URL` and `SCRATCHWORK_CONTENT_URL` (or the `appDomain`/`contentDomain` config values) behind a custom domain or proxy so publish responses return the public URL. Use the same value for both on a single-host server. + +## Deploy your own (from npm) + +The server packages are published to npm as built JavaScript with type +declarations, so you can deploy a Scratchwork server from a fresh directory +without cloning this repository. Example for Cloudflare: + +```sh +mkdir my-scratchwork && cd my-scratchwork +bun add @scratchwork/server-deploy-cloudflare # or: npm install @scratchwork/server-deploy-cloudflare +``` + +Copy the config shape from this repo's `deploy/cloudflare-vanilla` project — +a `server-config.ts` describing your domains and auth, a `cloudflare-config.ts` +naming the Worker/bucket/database, and a two-line `deploy.ts`: + +```ts +// deploy.ts +import { deployServer } from "@scratchwork/server-deploy-cloudflare"; +import { config } from "./cloudflare-config"; + +await deployServer(config, { envFile: ".env" }); +``` + +Put `CLOUDFLARE_API_TOKEN` and the auth secrets in `.env`, then run the deploy +with either runtime: + +```sh +bun deploy.ts # Bun +node deploy.ts # Node 24+ runs your own TypeScript files directly +``` + +The AWS equivalent starts from `deploy/generic-aws` and +`@scratchwork/server-deploy-aws`; a local single-machine server uses +`@scratchwork/server-deploy-local` under Bun. A `scratchwork server init` +scaffolder is future work — for now the `deploy/*` projects are the templates +you copy. diff --git a/server/core/README.md b/server/core/README.md new file mode 100644 index 0000000..30002e1 --- /dev/null +++ b/server/core/README.md @@ -0,0 +1,23 @@ +# @scratchwork/server-core + +The platform-neutral core of the +[Scratchwork](https://github.com/scratch/scratchwork) publishing server: +auth, routing, and the storage contracts (`PrimitiveDb`, `ObjectStorage`) +that deploy targets implement. Written with +[Effect](https://effect.website). + +You normally consume it through a deploy package rather than directly: + +- `@scratchwork/server-deploy-aws` — AWS Lambda + S3/DynamoDB +- `@scratchwork/server-deploy-cloudflare` — Cloudflare Worker + R2/D1 +- `@scratchwork/server-deploy-local` — local Bun server + +```ts +import { makeApp } from "@scratchwork/server-core"; +import { PrimitiveDb } from "@scratchwork/server-core/db"; +import { ObjectStorage } from "@scratchwork/server-core/storage"; +``` + +Published as built ESM JavaScript with type declarations; works under Node ≥ 22 +or Bun. See the [repo](https://github.com/scratch/scratchwork)'s +`server/README.md` for the deploy-your-own walkthrough. MIT license. diff --git a/server/core/package.json b/server/core/package.json index 6f04121..ce51461 100644 --- a/server/core/package.json +++ b/server/core/package.json @@ -2,13 +2,27 @@ "name": "@scratchwork/server-core", "version": "0.1.0", "type": "module", - "private": true, "description": "Platform-neutral Scratchwork publishing server core.", + "license": "MIT", + "repository": { + "type": "git", + "url": "git+https://github.com/scratch/scratchwork.git", + "directory": "server/core" + }, + "engines": { + "node": ">=22" + }, + "files": [ + "dist" + ], "exports": { ".": "./src/index.ts", "./app": "./src/app.ts", "./config": "./src/config.ts", "./db": "./src/db.ts", + "./deploy/env": "./src/deploy/env.ts", + "./deploy/proc": "./src/deploy/proc.ts", + "./deploy/server-settings": "./src/deploy/server-settings.ts", "./storage": "./src/storage.ts" }, "scripts": { @@ -18,6 +32,7 @@ }, "dependencies": { "@effect/platform": "0.96.2", + "@scratchwork/shared": "workspace:*", "effect": "3.21.4" }, "devDependencies": { diff --git a/server/core/src/access.d.ts b/server/core/src/access.d.ts new file mode 100644 index 0000000..f8988a9 --- /dev/null +++ b/server/core/src/access.d.ts @@ -0,0 +1,41 @@ +import * as Effect from "effect/Effect"; +/** Access expression: "public", "private", or a comma-separated list of emails and @domains. */ +export type AccessGroup = "public" | "private" | string; +declare const AccessGroupError_base: new = {}>(args: import("effect/Types").VoidIfEmpty<{ readonly [P in keyof A as P extends "_tag" ? never : P]: A[P]; }>) => import("effect/Cause").YieldableError & { + readonly _tag: "AccessGroupError"; +} & Readonly; +/** Raised when an access expression cannot be parsed. */ +export declare class AccessGroupError extends AccessGroupError_base<{ + readonly message: string; +}> { +} +/** The identity an access group is matched against. */ +export interface AccessPrincipal { + readonly email: string; +} +/** Normalizes and validates Scratchwork's shared access-expression syntax. */ +export declare function normalizeAccessGroup(group: string): Effect.Effect; +/** Returns true when the principal is included by the access group. */ +export declare function accessGroupMatches(group: AccessGroup, principal: AccessPrincipal | null | undefined): boolean; +export { isSafeProjectIdentifier } from "@scratchwork/shared/site/identifiers"; +/** Returns true when a name is reserved and cannot be claimed as a project. */ +export declare function isReservedSlug(value: string): boolean; +/** Applies share/revoke deltas to an access group: adds and removes individual email and + * @domain grants. Targets must be single email/@domain terms — never `public` or + * `private`, which are set through publish/unpublish. `public` has no grant list, so + * grants cannot be added to it, while removals against it are no-ops (there is no grant + * to remove; the caller's warning machinery reports the retained access). Removals match + * exact terms only; an email left covered by a remaining domain grant is the caller's to + * detect (see accessGroupMatches). Removing the last grant yields `private`. */ +export declare function accessGroupModify(group: AccessGroup, changes: { + readonly add: ReadonlyArray; + readonly remove: ReadonlyArray; +}): Effect.Effect; +/** Lists a grant group's individual email/@domain terms for API responses. `private` + * (no grants) and `public` (no grant list) yield an empty list; an unparsable group + * also yields [] rather than leaking a malformed stored value. */ +export declare function accessGroupTerms(group: AccessGroup): ReadonlyArray; +/** Checks that every explicit email/domain share target sits within an allowed domain list. */ +export declare function accessGroupUsesOnlyDomains(group: AccessGroup, domains: ReadonlySet): boolean; +/** Returns true for a plausible lowercase DNS domain. */ +export declare function safeDomain(value: string): boolean; diff --git a/server/core/src/access.js b/server/core/src/access.js new file mode 100644 index 0000000..565b9a5 --- /dev/null +++ b/server/core/src/access.js @@ -0,0 +1,202 @@ +import * as Data from "effect/Data"; +import * as Effect from "effect/Effect"; +/** Raised when an access expression cannot be parsed. */ +export class AccessGroupError extends Data.TaggedError("AccessGroupError") { +} +/** Normalizes and validates Scratchwork's shared access-expression syntax. */ +export function normalizeAccessGroup(group) { + const terms = parseAccessGroup(group); + return terms == null + ? Effect.fail(new AccessGroupError({ message: explainInvalidAccessGroup(group) })) + : Effect.succeed(serializeTerms(terms)); +} +/** Returns true when the principal is included by the access group. */ +export function accessGroupMatches(group, principal) { + const terms = parseAccessGroup(group); + if (terms == null) + return false; + return terms.some((term) => termMatches(term, principal)); +} +export { isSafeProjectIdentifier } from "@scratchwork/shared/site/identifiers"; +/** Names that cannot be claimed as projects. Projects live at single top-level path + * segments, so a project name is also a root path on the content host: server-owned + * routes, host-wide root files (a project named "robots.txt" would control crawl policy + * for the whole host), and prefixes held back for possible future namespace features + * (gh/g and the auth-provider names) are all off limits. Names are permanent once + * claimed, so reserve before shipping, not after. This is route policy, not identifier + * grammar — the CLI must not hardcode it. Separately, the identifier grammar requires an + * alphanumeric first character, so every "_"- and "."-prefixed name (including + * ".well-known") is unclaimable without an entry here. */ +const RESERVED_ROUTE_SLUGS = new Set([ + // Server-owned routes. + "api", "auth", "health", "favicon.ico", "favicon.svg", + // Host-wide root files. + "robots.txt", "sitemap.xml", "ads.txt", "app-ads.txt", "security.txt", + // Future namespace prefixes. + "gh", "g", + // Auth/identity providers, same future-namespace rationale. + "github", "gitlab", "gl", "bitbucket", "bb", "google", "microsoft", "ms", "apple", + "okta", "auth0", "x", "twitter", "facebook", "fb", "linkedin", "li", "slack", "discord", +]); +/** Returns true when a name is reserved and cannot be claimed as a project. */ +export function isReservedSlug(value) { + return RESERVED_ROUTE_SLUGS.has(value.toLowerCase()); +} +/** Applies share/revoke deltas to an access group: adds and removes individual email and + * @domain grants. Targets must be single email/@domain terms — never `public` or + * `private`, which are set through publish/unpublish. `public` has no grant list, so + * grants cannot be added to it, while removals against it are no-ops (there is no grant + * to remove; the caller's warning machinery reports the retained access). Removals match + * exact terms only; an email left covered by a remaining domain grant is the caller's to + * detect (see accessGroupMatches). Removing the last grant yields `private`. */ +export function accessGroupModify(group, changes) { + const current = parseAccessGroup(group); + if (current == null) { + return Effect.fail(new AccessGroupError({ message: explainInvalidAccessGroup(group) })); + } + const additions = grantTerms(changes.add); + const removals = grantTerms(changes.remove); + if (additions instanceof AccessGroupError) + return Effect.fail(additions); + if (removals instanceof AccessGroupError) + return Effect.fail(removals); + if (current.some((term) => term._tag === "Public")) { + if (additions.length > 0) { + return Effect.fail(new AccessGroupError({ + message: 'This group is "public", which has no per-account grants to edit. Make the project private first (scratchwork publish --private or unpublish).', + })); + } + return Effect.succeed("public"); + } + const removedKeys = new Set(removals.map((term) => serializeTerms([term]))); + const terms = dedupeTerms([ + ...current.filter((term) => term._tag !== "Private" && !removedKeys.has(serializeTerms([term]))), + ...additions, + ]); + return Effect.succeed(terms.length === 0 ? "private" : serializeTerms(terms)); +} +/** Lists a grant group's individual email/@domain terms for API responses. `private` + * (no grants) and `public` (no grant list) yield an empty list; an unparsable group + * also yields [] rather than leaking a malformed stored value. */ +export function accessGroupTerms(group) { + const terms = parseAccessGroup(group); + if (terms == null) + return []; + return terms + .filter((term) => term._tag === "Email" || term._tag === "Domain") + .map((term) => serializeTerms([term])); +} +/** Parses share/revoke targets, each of which must be one email or @domain term. */ +function grantTerms(targets) { + const terms = []; + for (const target of targets) { + const parsed = parseAccessGroup(target); + const term = parsed != null && parsed.length === 1 ? parsed[0] : null; + if (term == null || (term._tag !== "Email" && term._tag !== "Domain")) { + return new AccessGroupError({ + message: `Invalid share target: ${target} (expected an email address or an @domain group like @example.com)`, + }); + } + terms.push(term); + } + return terms; +} +/** Checks that every explicit email/domain share target sits within an allowed domain list. */ +export function accessGroupUsesOnlyDomains(group, domains) { + if (domains.size === 0) + return true; + const terms = parseAccessGroup(group); + if (terms == null) + return false; + for (const term of terms) { + if (term._tag === "Public") + return false; + if (term._tag === "Private") + continue; + const domain = term._tag === "Domain" ? term.domain : term.email.split("@")[1]; + if (domain == null || !domains.has(domain)) + return false; + } + return true; +} +/** The error message for an unparsable access expression: what was given, what is accepted. */ +function explainInvalidAccessGroup(group) { + return `Invalid access group "${group}": expected "public", "private", or a comma-separated list of email addresses and @domain groups, like "alice@example.com,@example.com"`; +} +/** Parses a comma-separated access expression into validated terms, or null when invalid. */ +function parseAccessGroup(group) { + const text = group.trim().toLowerCase(); + if (text === "") + return null; + const parts = text.split(",").map((part) => part.trim()).filter((part) => part !== ""); + if (parts.length === 0) + return null; + const terms = []; + for (const part of parts) { + if (part === "public") { + terms.push({ _tag: "Public" }); + } + else if (part === "private") { + terms.push({ _tag: "Private" }); + } + else if (part.startsWith("@") && safeDomain(part.slice(1))) { + terms.push({ _tag: "Domain", domain: part.slice(1) }); + } + else if (safeEmail(part)) { + terms.push({ _tag: "Email", email: part }); + } + else { + return null; + } + } + if (terms.some((term) => term._tag === "Public") && terms.length > 1) + return null; + if (terms.some((term) => term._tag === "Private") && terms.length > 1) + return null; + return dedupeTerms(terms); +} +/** Returns true when the principal satisfies one term. */ +function termMatches(term, principal) { + if (term._tag === "Public") + return true; + if (term._tag === "Private" || principal == null) + return false; + const email = principal.email.toLowerCase(); + if (term._tag === "Email") + return email === term.email; + return email.endsWith(`@${term.domain}`); +} +/** Renders parsed terms back into the canonical expression string. */ +function serializeTerms(terms) { + if (terms.length === 1 && terms[0]._tag === "Public") + return "public"; + if (terms.length === 1 && terms[0]._tag === "Private") + return "private"; + return terms.map((term) => term._tag === "Email" + ? term.email + : term._tag === "Domain" + ? `@${term.domain}` + : term._tag.toLowerCase()).join(","); +} +/** Removes duplicate terms while preserving first-seen order. */ +function dedupeTerms(terms) { + const seen = new Set(); + const result = []; + for (const term of terms) { + const key = serializeTerms([term]); + if (seen.has(key)) + continue; + seen.add(key); + result.push(term); + } + return result; +} +/** Returns true for a plausible lowercase email with a safe domain. */ +function safeEmail(value) { + const [local, domain, extra] = value.split("@"); + return extra == null && local != null && local !== "" && domain != null && safeDomain(domain) && !/[,\s<>]/.test(local); +} +/** Returns true for a plausible lowercase DNS domain. */ +export function safeDomain(value) { + return /^[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?(?:\.[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?)+$/.test(value); +} diff --git a/server/core/src/access.ts b/server/core/src/access.ts index f32ea30..7e46bf0 100644 --- a/server/core/src/access.ts +++ b/server/core/src/access.ts @@ -36,7 +36,7 @@ export function accessGroupMatches(group: AccessGroup, principal: AccessPrincipa return terms.some((term) => termMatches(term, principal)); } -export { isSafeProjectIdentifier } from "../../../shared/src/site/identifiers"; +export { isSafeProjectIdentifier } from "@scratchwork/shared/site/identifiers"; /** Names that cannot be claimed as projects. Projects live at single top-level path * segments, so a project name is also a root path on the content host: server-owned diff --git a/server/core/src/api-routes.ts b/server/core/src/api-routes.ts index e968007..2cf5f53 100644 --- a/server/core/src/api-routes.ts +++ b/server/core/src/api-routes.ts @@ -39,8 +39,8 @@ import { type EndpointSuccess, type ProjectInfo, type ScratchworkEndpointName, -} from "../../../shared/src/publish/api"; -import { accessGroupTerms, isSafeProjectIdentifier } from "./access"; +} from "@scratchwork/shared/publish/api"; +import { accessGroupTerms, isSafeProjectIdentifier } from "./access.ts"; import { Auth, AuthError, @@ -49,9 +49,9 @@ import { decryptCliCloudflareToken, verifyCliCodeExchange, type AuthUser, -} from "./auth"; -import { ServerConfig, type ServerConfigShape } from "./config"; -import { PrimitiveDb } from "./db"; +} from "./auth.ts"; +import { ServerConfig, type ServerConfigShape } from "./config.ts"; +import { PrimitiveDb } from "./db.ts"; import { appBaseUrl, contentBaseUrl, @@ -60,11 +60,11 @@ import { projectUrl, publishedUrl, rejectCrossOriginApiRequest, -} from "./http"; -import { MAX_PUBLISH_BODY_BYTES, normalizePublishRequest } from "./publish-request"; -import { projectForRequest } from "./routes"; -import { MAX_SHARE_BODY_BYTES, validateShareChanges } from "./share-request"; -import { type SiteRecord } from "./site-records"; +} from "./http.ts"; +import { MAX_PUBLISH_BODY_BYTES, normalizePublishRequest } from "./publish-request.ts"; +import { projectForRequest } from "./routes.ts"; +import { MAX_SHARE_BODY_BYTES, validateShareChanges } from "./share-request.ts"; +import { type SiteRecord } from "./site-records.ts"; import { canReadProject, projectRole, @@ -73,8 +73,8 @@ import { SiteStoreError, type LoadedSite, type ProjectRole, -} from "./site-store"; -import { StorageError } from "./storage"; +} from "./site-store.ts"; +import { StorageError } from "./storage.ts"; /** Failures any API handler may raise. */ type RouteError = HttpError | AuthError | SiteStoreError | StorageError; diff --git a/server/core/src/app.ts b/server/core/src/app.ts index 4fd9962..6bd2774 100644 --- a/server/core/src/app.ts +++ b/server/core/src/app.ts @@ -12,17 +12,17 @@ import type * as HttpApp from "@effect/platform/HttpApp"; import * as HttpServerRequest from "@effect/platform/HttpServerRequest"; import * as HttpServerResponse from "@effect/platform/HttpServerResponse"; import * as Effect from "effect/Effect"; -import { SiteFiles } from "../../../shared/src/site/files"; -import { servePath } from "../../../shared/src/site/serve"; -import { defaultRendererHtml } from "../../../shared/src/site/default-renderer.generated.js"; -import FIGURE_SVG from "../../../shared/assets/figure.svg" with { type: "text" }; -import { isSafeProjectIdentifier } from "./access"; -import { dispatchApiRoute, requireReadableSite } from "./api-routes"; -import { Auth, AuthError, type AuthShape, type AuthUser } from "./auth"; -import { ServerConfig, type ServerConfigShape } from "./config"; -import { PrimitiveDb } from "./db"; -import { projectAccessCookie, projectAccessCookieValues } from "./cookies"; -import { acceptsHtmlPage, errorPageResponse, errorResponse } from "./error-pages"; +import { SiteFiles } from "@scratchwork/shared/site/files"; +import { servePath } from "@scratchwork/shared/site/serve"; +import { defaultRendererHtml } from "@scratchwork/shared/site/default-renderer.generated.js"; +import { FIGURE_SVG } from "@scratchwork/shared/assets/figure-svg.generated"; +import { isSafeProjectIdentifier } from "./access.ts"; +import { dispatchApiRoute, requireReadableSite } from "./api-routes.ts"; +import { Auth, AuthError, type AuthShape, type AuthUser } from "./auth.ts"; +import { ServerConfig, type ServerConfigShape } from "./config.ts"; +import { PrimitiveDb } from "./db.ts"; +import { projectAccessCookie, projectAccessCookieValues } from "./cookies.ts"; +import { acceptsHtmlPage, errorPageResponse, errorResponse } from "./error-pages.ts"; import { appBaseUrl, contentBaseUrl, @@ -31,10 +31,10 @@ import { requestBaseUrl, sameOrigin, securityHeaders, -} from "./http"; -import { projectForRequest, routeRest } from "./routes"; -import { canReadProject, SiteStore, SiteStoreError, type LoadedSite } from "./site-store"; -import { StorageError } from "./storage"; +} from "./http.ts"; +import { projectForRequest, routeRest } from "./routes.ts"; +import { canReadProject, SiteStore, SiteStoreError, type LoadedSite } from "./site-store.ts"; +import { StorageError } from "./storage.ts"; const NO_STORE = "no-store, must-revalidate"; /** diff --git a/server/core/src/assets.d.ts b/server/core/src/assets.d.ts deleted file mode 100644 index fcfbf9f..0000000 --- a/server/core/src/assets.d.ts +++ /dev/null @@ -1,4 +0,0 @@ -declare module "*.svg" { - const text: string; - export default text; -} diff --git a/server/core/src/auth.ts b/server/core/src/auth.ts index 3935442..9f0b196 100644 --- a/server/core/src/auth.ts +++ b/server/core/src/auth.ts @@ -18,12 +18,12 @@ import * as Either from "effect/Either"; import * as Encoding from "effect/Encoding"; import * as Layer from "effect/Layer"; import * as Schema from "effect/Schema"; -import { sha256Base64Url } from "../../../shared/src/crypto/digest"; -import { errorMessage } from "../../../shared/src/util/errors"; -import { accessGroupMatches } from "./access"; -import * as AuthCrypto from "./auth-crypto"; -import { verifyCloudflareAccessToken } from "./cloudflare-jwt"; -import { ServerConfig, type AuthConfig, type CloudflareAccessAuthConfig, type OAuthAuthConfig } from "./config"; +import { sha256Base64Url } from "@scratchwork/shared/crypto/digest"; +import { errorMessage } from "@scratchwork/shared/util/errors"; +import { accessGroupMatches } from "./access.ts"; +import * as AuthCrypto from "./auth-crypto.ts"; +import { verifyCloudflareAccessToken } from "./cloudflare-jwt.ts"; +import { ServerConfig, type AuthConfig, type CloudflareAccessAuthConfig, type OAuthAuthConfig } from "./config.ts"; import { clearOauthStateCookie, clearSessionCookie, @@ -32,13 +32,13 @@ import { oauthStateToken, sessionCookie, STATE_TTL_SECONDS, -} from "./cookies"; +} from "./cookies.ts"; import { postAuthorizationCodeGrant, verifyGoogleIdToken, type GoogleIdTokenClaims, -} from "./google-jwt"; -import { timingSafeEqual } from "./tokens"; +} from "./google-jwt.ts"; +import { timingSafeEqual } from "./tokens.ts"; const GOOGLE_AUTHORIZE_URL = "https://accounts.google.com/o/oauth2/v2/auth"; const GOOGLE_TOKEN_URL = "https://oauth2.googleapis.com/token"; diff --git a/server/core/src/cloudflare-jwt.ts b/server/core/src/cloudflare-jwt.ts index fb618b3..e78b78e 100644 --- a/server/core/src/cloudflare-jwt.ts +++ b/server/core/src/cloudflare-jwt.ts @@ -9,8 +9,8 @@ */ import * as Data from "effect/Data"; import * as Effect from "effect/Effect"; -import { errorMessage } from "../../../shared/src/util/errors"; -import { CLOCK_SKEW_SECONDS, verifyRs256Jwt, verifyRs256JwtWithJwks } from "./jwt-rs256"; +import { errorMessage } from "@scratchwork/shared/util/errors"; +import { CLOCK_SKEW_SECONDS, verifyRs256Jwt, verifyRs256JwtWithJwks } from "./jwt-rs256.ts"; /** Raised when an Access token fails signature or claim validation. */ export class CloudflareJwtError extends Data.TaggedError("CloudflareJwtError")<{ diff --git a/server/core/src/config.d.ts b/server/core/src/config.d.ts new file mode 100644 index 0000000..893b314 --- /dev/null +++ b/server/core/src/config.d.ts @@ -0,0 +1,81 @@ +import * as Context from "effect/Context"; +import * as Effect from "effect/Effect"; +import * as Layer from "effect/Layer"; +import { type AccessGroup } from "./access.ts"; +/** An environment-variable map from any platform (process.env, Worker vars, Lambda env). */ +export type EnvVars = Readonly>; +/** Parsed server configuration. */ +export interface ServerConfigShape { + readonly port: number; + /** Public origin of the app host (auth routes and API), when configured. */ + readonly appUrl?: string; + /** Public origin of the content host (published sites), when configured. */ + readonly contentUrl?: string; + /** Origins served from the homepage project (first is canonical, the rest 308 to it). + * Empty when the server has no homepage. */ + readonly homepageUrls: ReadonlyArray; + /** Name of the project served on the homepage origins; set iff homepageUrls is non-empty. */ + readonly homepageProject?: string; + /** false: no project may be public — existing public projects read as private. */ + readonly allowPublicProjects: boolean; + /** When non-empty, share grants must fall inside these domains; grants outside them + * stop conferring access. */ + readonly allowedShareDomains: ReadonlySet; + /** true: publishers choose globally-unique project names (first-writer-wins). + * false: the server assigns a random slug on first publish. */ + readonly usersCanSetProjectNames: boolean; + readonly auth: AuthConfig; +} +/** Session-signing and allow-list settings shared by every auth mode. */ +export interface AuthConfigCommon { + readonly sessionSecret: string; + readonly allowedUsers: AccessGroup; + readonly sessionTtlSeconds: number; +} +/** Authorization-server endpoints used by the OAuth login flow. Overridable only + * by the loopback-gated local test configuration; production always uses Google's. */ +export interface OAuthProviderEndpoints { + readonly authorizeUrl: string; + readonly tokenUrl: string; + readonly jwksUrl: string; +} +/** Built-in Google OAuth: the server runs the login flow itself. */ +export interface OAuthAuthConfig extends AuthConfigCommon { + readonly mode: "oauth"; + readonly clientId: string; + readonly clientSecret: string; + /** Provider endpoints supplied only by the hermetic local test provider. */ + readonly localEndpoints?: OAuthProviderEndpoints; +} +/** Cloudflare Access: the server sits behind an Access application that authenticates + * users at the edge and injects a signed Cf-Access-Jwt-Assertion header. */ +export interface CloudflareAccessAuthConfig extends AuthConfigCommon { + /** Team origin the assertions are issued by, like "https://myteam.cloudflareaccess.com". */ + readonly teamDomain: string; + /** Audience (AUD) tag of the Access application protecting this server. */ + readonly audience: string; + /** Public signing keys supplied only by the offline local Access simulator. */ + readonly localJwks?: ReadonlyArray; + readonly mode: "cloudflare-access"; +} +/** Auth settings. Auth cannot be disabled. */ +export type AuthConfig = OAuthAuthConfig | CloudflareAccessAuthConfig; +declare const ServerConfig_base: Context.TagClass; +/** Service tag for server configuration. */ +export declare class ServerConfig extends ServerConfig_base { +} +declare const ServerConfigError_base: new = {}>(args: import("effect/Types").VoidIfEmpty<{ readonly [P in keyof A as P extends "_tag" ? never : P]: A[P]; }>) => import("effect/Cause").YieldableError & { + readonly _tag: "ServerConfigError"; +} & Readonly; +/** Raised when environment configuration is missing or malformed. */ +export declare class ServerConfigError extends ServerConfigError_base<{ + readonly message: string; +}> { +} +/** Builds a ServerConfig layer from an explicit environment map. */ +export declare function makeServerConfigLayer(env: EnvVars): Layer.Layer; +/** Parses all server runtime configuration from environment variables. */ +export declare function readServerConfig(env: EnvVars): Effect.Effect; +export {}; diff --git a/server/core/src/config.js b/server/core/src/config.js new file mode 100644 index 0000000..770b748 --- /dev/null +++ b/server/core/src/config.js @@ -0,0 +1,335 @@ +import * as Context from "effect/Context"; +import * as Data from "effect/Data"; +import * as Effect from "effect/Effect"; +import * as Layer from "effect/Layer"; +import * as Schema from "effect/Schema"; +import { isLoopbackHost } from "@scratchwork/shared/util/url"; +import { isReservedSlug, isSafeProjectIdentifier, normalizeAccessGroup, safeDomain } from "./access.js"; +/** Service tag for server configuration. */ +export class ServerConfig extends Context.Tag("@scratchwork/server/Config")() { +} +/** Raised when environment configuration is missing or malformed. */ +export class ServerConfigError extends Data.TaggedError("ServerConfigError") { +} +/** Builds the uniform rejection for one env var: the value given and what is accepted. */ +function invalidValue(name, value, expected) { + return new ServerConfigError({ message: `Invalid ${name} "${value}": expected ${expected}` }); +} +/** Builds a ServerConfig layer from an explicit environment map. */ +export function makeServerConfigLayer(env) { + return Layer.effect(ServerConfig, readServerConfig(env)); +} +/** Parses all server runtime configuration from environment variables. */ +export function readServerConfig(env) { + return Effect.gen(function* () { + const portValue = env.PORT ?? env.SCRATCHWORK_PORT ?? "3001"; + const port = parsePort(portValue); + if (port == null) { + return yield* Effect.fail(invalidValue("PORT", portValue, 'an integer between 1 and 65535, like "3001"')); + } + const appUrl = yield* readPublicUrl(env.SCRATCHWORK_APP_URL || urlFromDomain(env.SCRATCHWORK_APP_DOMAIN), "SCRATCHWORK_APP_URL"); + const contentUrl = yield* readPublicUrl(env.SCRATCHWORK_CONTENT_URL || urlFromDomain(env.SCRATCHWORK_CONTENT_DOMAIN), "SCRATCHWORK_CONTENT_URL"); + yield* rejectRetiredEnvVars(env); + return { + port, + appUrl, + contentUrl, + ...(yield* readHomepage(env, appUrl, contentUrl)), + allowPublicProjects: yield* readBoolean(env.SCRATCHWORK_ALLOW_PUBLIC_PROJECTS, true, "SCRATCHWORK_ALLOW_PUBLIC_PROJECTS"), + allowedShareDomains: yield* readDomainSet(env.SCRATCHWORK_ALLOWED_SHARE_DOMAINS, "SCRATCHWORK_ALLOWED_SHARE_DOMAINS"), + usersCanSetProjectNames: yield* readBoolean(env.SCRATCHWORK_USERS_CAN_SET_PROJECT_NAMES, true, "SCRATCHWORK_USERS_CAN_SET_PROJECT_NAMES"), + auth: yield* readAuthConfig(env, appUrl), + }; + }); +} +/** Validates and normalizes one configured public origin (appUrl or contentUrl). */ +function readPublicUrl(value, name) { + if (value == null || value === "") + return Effect.succeed(undefined); + try { + const url = new URL(value); + if (url.pathname !== "/" || url.search !== "" || url.hash !== "") { + return Effect.fail(invalidValue(name, value, 'a bare origin with no path, query, or fragment, like "https://example.com"')); + } + // Loopback (including *.localhost, which resolves locally on modern systems) + // may use http so local runs get real hostname-per-role URLs. + if (url.protocol !== "https:" && !(url.protocol === "http:" && isLoopbackHost(url.hostname))) { + return Effect.fail(invalidValue(name, value, 'an https URL, like "https://example.com" (plain http is allowed only for loopback hosts, like "http://localhost:3001")')); + } + return Effect.succeed(url.origin); + } + catch { + return Effect.fail(invalidValue(name, value, 'a URL, like "https://example.com"')); + } +} +/** Parses the optional server-homepage settings: the origins served from the homepage + * project (bare domains or full origins, comma-separated; the first is canonical) and + * the project's name. Set both or neither. The homepage origins must be distinct from + * the app and content origins, since the hostname decides the routing model. */ +function readHomepage(env, appUrl, contentUrl) { + return Effect.gen(function* () { + const domains = env.SCRATCHWORK_HOMEPAGE_DOMAINS?.trim() || undefined; + const project = env.SCRATCHWORK_HOMEPAGE_PROJECT?.trim() || undefined; + if (domains == null && project == null) + return { homepageUrls: [] }; + if (domains == null || project == null) { + return yield* Effect.fail(new ServerConfigError({ + message: "SCRATCHWORK_HOMEPAGE_DOMAINS and SCRATCHWORK_HOMEPAGE_PROJECT must be set together: the domains say where the homepage is served, the project says which project it is", + })); + } + if (!isSafeProjectIdentifier(project) || isReservedSlug(project)) { + return yield* Effect.fail(invalidValue("SCRATCHWORK_HOMEPAGE_PROJECT", project, 'a publishable project name, like "home"')); + } + const homepageUrls = []; + for (const item of csvItems(domains)) { + const url = yield* readPublicUrl(urlFromDomain(item), "SCRATCHWORK_HOMEPAGE_DOMAINS"); + if (url == null || homepageUrls.includes(url)) + continue; + if (url === appUrl || url === contentUrl) { + return yield* Effect.fail(invalidValue("SCRATCHWORK_HOMEPAGE_DOMAINS", item, "a homepage origin distinct from the app and content origins, which use path-based routing")); + } + homepageUrls.push(url); + } + if (homepageUrls.length === 0) { + return yield* Effect.fail(invalidValue("SCRATCHWORK_HOMEPAGE_DOMAINS", domains, 'a comma-separated list of domains or origins, like "example.com,www.example.com"')); + } + return { homepageUrls, homepageProject: project }; + }); +} +/** Parses auth settings from environment variables. Auth cannot be disabled, and the + * mode must be chosen explicitly: a server either runs built-in Google OAuth or sits + * behind Cloudflare Access. */ +function readAuthConfig(env, appUrl) { + return Effect.gen(function* () { + const authMode = (env.SCRATCHWORK_AUTH ?? "").toLowerCase(); + if (authMode === "") { + return yield* Effect.fail(new ServerConfigError({ + message: 'SCRATCHWORK_AUTH is required: set it to "oauth" or "cloudflare-access".', + })); + } + if (authMode !== "oauth" && authMode !== "cloudflare-access") { + return yield* Effect.fail(invalidValue("SCRATCHWORK_AUTH", authMode, '"oauth" or "cloudflare-access"')); + } + if (authMode === "cloudflare-access") + return yield* readCloudflareAccessConfig(env, appUrl); + return yield* readOAuthConfig(env, appUrl); + }); +} +/** Parses required Google OAuth settings from environment variables. */ +function readOAuthConfig(env, appUrl) { + return Effect.gen(function* () { + const clientId = env.SCRATCHWORK_GOOGLE_CLIENT_ID ?? env.GOOGLE_CLIENT_ID; + const clientSecret = env.SCRATCHWORK_GOOGLE_CLIENT_SECRET ?? env.GOOGLE_CLIENT_SECRET; + const sessionSecret = env.SCRATCHWORK_SESSION_SECRET; + if (!clientId || !clientSecret || !sessionSecret) { + const missing = [ + clientId ? null : "SCRATCHWORK_GOOGLE_CLIENT_ID", + clientSecret ? null : "SCRATCHWORK_GOOGLE_CLIENT_SECRET", + sessionSecret ? null : "SCRATCHWORK_SESSION_SECRET", + ].filter((name) => name != null); + return yield* Effect.fail(new ServerConfigError({ + message: `OAuth mode requires ${missing.join(", ")}: create OAuth credentials at https://console.cloud.google.com/apis/credentials and generate a session secret with "openssl rand -hex 32".`, + })); + } + return { + mode: "oauth", + clientId, + clientSecret, + ...(yield* readLocalOAuthEndpoints(env, appUrl)), + ...(yield* readCommonAuthConfig(env, sessionSecret)), + }; + }); +} +/** Parses the hermetic test provider's endpoints. Like the local Cloudflare JWKS + * override, the LOCAL prefix and the loopback gates keep this out of production + * configuration: the variables are accepted only when the app origin is loopback, + * and every endpoint must itself be a loopback URL. Set all three or none. */ +function readLocalOAuthEndpoints(env, appUrl) { + const names = [ + "SCRATCHWORK_LOCAL_OAUTH_AUTHORIZE_URL", + "SCRATCHWORK_LOCAL_OAUTH_TOKEN_URL", + "SCRATCHWORK_LOCAL_OAUTH_JWKS_URL", + ]; + const values = names.map((name) => env[name] || undefined); + if (values.every((value) => value == null)) + return Effect.succeed({}); + if (appUrl == null || !isLiteralLoopbackHost(new URL(appUrl).hostname)) { + return Effect.fail(new ServerConfigError({ + message: "SCRATCHWORK_LOCAL_OAUTH_* endpoints are accepted only when SCRATCHWORK_APP_URL uses a loopback host", + })); + } + if (values.some((value) => value == null)) { + return Effect.fail(new ServerConfigError({ message: `Set all of ${names.join(", ")} or none of them` })); + } + for (const [index, value] of values.entries()) { + if (!isLoopbackUrl(value)) { + return Effect.fail(invalidValue(names[index], value, 'a loopback URL, like "http://127.0.0.1:4300/authorize"')); + } + } + const [authorizeUrl, tokenUrl, jwksUrl] = values; + return Effect.succeed({ localEndpoints: { authorizeUrl, tokenUrl, jwksUrl } }); +} +/** Returns true for an http(s) URL whose host is loopback. */ +function isLoopbackUrl(value) { + try { + const url = new URL(value); + return (url.protocol === "http:" || url.protocol === "https:") && isLiteralLoopbackHost(url.hostname); + } + catch { + return false; + } +} +/** Narrow host gate for credential-bearing local OAuth endpoints. Unlike the + * general local-development helper, this excludes 0.0.0.0 and *.localhost so a + * client secret is sent only to an explicit loopback listener. */ +function isLiteralLoopbackHost(hostname) { + const host = hostname.toLowerCase(); + return host === "localhost" || host === "127.0.0.1" || host === "::1" || host === "[::1]"; +} +/** Parses required Cloudflare Access settings from environment variables. */ +function readCloudflareAccessConfig(env, appUrl) { + return Effect.gen(function* () { + const teamDomainValue = env.SCRATCHWORK_CF_ACCESS_TEAM_DOMAIN?.trim() || undefined; + const audience = env.SCRATCHWORK_CF_ACCESS_AUD?.trim() || undefined; + const sessionSecret = env.SCRATCHWORK_SESSION_SECRET; + if (teamDomainValue == null || audience == null || !sessionSecret) { + const missing = [ + teamDomainValue != null ? null : "SCRATCHWORK_CF_ACCESS_TEAM_DOMAIN", + audience != null ? null : "SCRATCHWORK_CF_ACCESS_AUD", + sessionSecret ? null : "SCRATCHWORK_SESSION_SECRET", + ].filter((name) => name != null); + return yield* Effect.fail(new ServerConfigError({ + message: `Cloudflare Access mode requires ${missing.join(", ")}: copy the team domain and the application Audience (AUD) tag from the Cloudflare Zero Trust dashboard, and generate a session secret with "openssl rand -hex 32".`, + })); + } + const teamDomain = normalizeCfTeamDomain(teamDomainValue); + if (teamDomain == null) { + return yield* Effect.fail(invalidValue("SCRATCHWORK_CF_ACCESS_TEAM_DOMAIN", teamDomainValue, 'a Cloudflare Access team domain, like "myteam" or "myteam.cloudflareaccess.com"')); + } + return { + mode: "cloudflare-access", + teamDomain, + audience, + ...(yield* readLocalCloudflareJwks(env.SCRATCHWORK_LOCAL_CF_ACCESS_JWKS, appUrl)), + ...(yield* readCommonAuthConfig(env, sessionSecret)), + }; + }); +} +/** A JWKS document: JSON with a non-empty array of object-shaped keys. */ +const LocalJwksSchema = Schema.parseJson(Schema.Struct({ + keys: Schema.Array(Schema.Object).pipe(Schema.filter((keys) => keys.length > 0 || "keys must not be empty")), +})); +/** Parses the generated public JWKS used by the offline Access simulator. Keeping the + * override behind a LOCAL-prefixed variable prevents it from becoming part of normal + * Cloudflare deployment configuration. */ +function readLocalCloudflareJwks(value, appUrl) { + if (value == null || value === "") + return Effect.succeed({}); + if (appUrl == null || !isLoopbackHost(new URL(appUrl).hostname)) { + return Effect.fail(new ServerConfigError({ + message: "SCRATCHWORK_LOCAL_CF_ACCESS_JWKS is accepted only when SCRATCHWORK_APP_URL uses a loopback host", + })); + } + return Schema.decodeUnknown(LocalJwksSchema)(value).pipe(Effect.map((parsed) => ({ localJwks: parsed.keys })), Effect.mapError(() => invalidValue("SCRATCHWORK_LOCAL_CF_ACCESS_JWKS", value, "a generated JWKS JSON document with at least one public key"))); +} +/** Parses the auth settings every mode shares: the session-signing secret (validated + * for length), the allow-list, and the session lifetime. */ +function readCommonAuthConfig(env, sessionSecret) { + return Effect.gen(function* () { + if (new TextEncoder().encode(sessionSecret).byteLength < 32) { + return yield* Effect.fail(new ServerConfigError({ + message: 'SCRATCHWORK_SESSION_SECRET must be at least 32 bytes: generate one with "openssl rand -hex 32"', + })); + } + return { + sessionSecret, + allowedUsers: yield* readAccessGroup(env.SCRATCHWORK_ALLOWED_USERS ?? groupFromLegacyAuthAllowLists(env.SCRATCHWORK_AUTH_ALLOWED_EMAILS, env.SCRATCHWORK_AUTH_ALLOWED_DOMAINS), "public", "SCRATCHWORK_ALLOWED_USERS"), + sessionTtlSeconds: yield* readPositiveInteger(env.SCRATCHWORK_AUTH_SESSION_SECONDS, 60 * 60 * 24 * 30, "SCRATCHWORK_AUTH_SESSION_SECONDS"), + }; + }); +} +/** Normalizes the configured Cloudflare Access team domain — "myteam", + * "myteam.cloudflareaccess.com", or a full https URL — to its https origin. */ +function normalizeCfTeamDomain(value) { + const host = value + .toLowerCase() + .replace(/^https:\/\//, "") + .replace(/[/?#].*$/, ""); + const domain = host.includes(".") ? host : `${host}.cloudflareaccess.com`; + return safeDomain(domain) ? `https://${domain}` : null; +} +/** Retired variables and their replacements. These carried access policy, so silently + * ignoring one could leave a server more open than its operator intended — refuse to + * start instead. */ +const RETIRED_ENV_VARS = [ + ["SCRATCHWORK_SHARE_ALLOWED_DOMAINS", "SCRATCHWORK_ALLOWED_SHARE_DOMAINS"], +]; +/** Fails when a retired environment variable is still set. */ +function rejectRetiredEnvVars(env) { + for (const [name, replacement] of RETIRED_ENV_VARS) { + if (env[name]) { + return Effect.fail(new ServerConfigError({ message: `${name} is no longer supported: use ${replacement} instead` })); + } + } + return Effect.void; +} +/** Parses one access-group environment value with a fallback expression. */ +function readAccessGroup(value, fallback, name) { + return normalizeAccessGroup(value == null || value === "" ? fallback : value).pipe(Effect.mapError((cause) => new ServerConfigError({ message: `${name}: ${cause.message}` }))); +} +/** Parses one boolean environment value with a fallback. */ +function readBoolean(value, fallback, name) { + if (value == null || value === "") + return Effect.succeed(fallback); + const normalized = value.trim().toLowerCase(); + if (normalized === "true") + return Effect.succeed(true); + if (normalized === "false") + return Effect.succeed(false); + return Effect.fail(invalidValue(name, value, '"true" or "false"')); +} +/** Expands a bare domain env value into an https origin. */ +function urlFromDomain(value) { + if (value == null || value === "") + return undefined; + return value.includes("://") ? value : `https://${value}`; +} +/** Folds the legacy SCRATCHWORK_AUTH_ALLOWED_EMAILS/_DOMAINS variables into one expression. */ +function groupFromLegacyAuthAllowLists(emails, domains) { + const parts = [ + ...csvItems(emails), + ...csvItems(domains).map((domain) => domain.startsWith("@") ? domain : `@${domain}`), + ]; + return parts.length === 0 ? undefined : parts.join(","); +} +/** Parses a valid TCP port number. */ +function parsePort(value) { + const port = Number(value); + return Number.isInteger(port) && port >= 1 && port <= 65535 ? port : null; +} +/** Parses one positive-integer environment value with a fallback. */ +function readPositiveInteger(value, fallback, name) { + if (value == null || value === "") + return Effect.succeed(fallback); + const number = Number(value); + if (Number.isInteger(number) && number > 0) + return Effect.succeed(number); + return Effect.fail(invalidValue(name, value, 'a positive integer number of seconds, like "86400"')); +} +/** Parses a comma-separated domain allow-list environment value into a lowercase set. */ +function readDomainSet(value, name) { + const domains = csvItems(value).map((domain) => domain.replace(/^@/, "")); + if (domains.some((domain) => !safeDomain(domain))) { + return Effect.fail(invalidValue(name, value ?? "", 'a comma-separated list of domains, like "example.com,corp.example.com"')); + } + return Effect.succeed(new Set(domains)); +} +/** Splits a comma-separated env value into trimmed lowercase items. */ +function csvItems(value) { + if (value == null || value === "") + return []; + return value + .split(",") + .map((item) => item.trim().toLowerCase()) + .filter((item) => item !== ""); +} diff --git a/server/core/src/config.ts b/server/core/src/config.ts index 0de08ef..36924b4 100644 --- a/server/core/src/config.ts +++ b/server/core/src/config.ts @@ -3,8 +3,8 @@ import * as Data from "effect/Data"; import * as Effect from "effect/Effect"; import * as Layer from "effect/Layer"; import * as Schema from "effect/Schema"; -import { isLoopbackHost } from "../../../shared/src/util/url"; -import { isReservedSlug, isSafeProjectIdentifier, normalizeAccessGroup, safeDomain, type AccessGroup } from "./access"; +import { isLoopbackHost } from "@scratchwork/shared/util/url"; +import { isReservedSlug, isSafeProjectIdentifier, normalizeAccessGroup, safeDomain, type AccessGroup } from "./access.ts"; /** An environment-variable map from any platform (process.env, Worker vars, Lambda env). */ export type EnvVars = Readonly>; diff --git a/server/scripts/env.ts b/server/core/src/deploy/env.ts similarity index 100% rename from server/scripts/env.ts rename to server/core/src/deploy/env.ts diff --git a/server/core/src/deploy/proc.ts b/server/core/src/deploy/proc.ts new file mode 100644 index 0000000..d487f58 --- /dev/null +++ b/server/core/src/deploy/proc.ts @@ -0,0 +1,60 @@ +/* + * Process spawning for the deploy scripts. Uses node:child_process (not + * Bun.spawn) so published deploy tooling runs under plain Node as well as Bun. + */ +import { spawn } from "node:child_process"; + +/** Options for one spawned command. */ +export interface RunOptions { + readonly allowFailure?: boolean; + readonly capture?: boolean; + readonly cwd?: string; +} + +/** Exit status and captured output of one spawned command. */ +export interface RunResult { + readonly ok: boolean; + readonly stdout: string; + readonly stderr: string; +} + +/** Creates a command runner that uses one captured deploy environment. */ +export function createRunner(commandEnv: Record) { + return { + run: (command: string, args: ReadonlyArray, options: RunOptions = {}) => + run(command, args, commandEnv, options), + }; +} + +/** Spawns one command and optionally captures output or tolerates failure. */ +function run( + command: string, + args: ReadonlyArray, + commandEnv: Record, + options: RunOptions, +): Promise { + return new Promise((resolve, reject) => { + const child = spawn(command, args, { + cwd: options.cwd, + env: commandEnv, + stdio: [ + "inherit", + options.capture ? "pipe" : "inherit", + options.capture || options.allowFailure ? "pipe" : "inherit", + ], + }); + let stdout = ""; + let stderr = ""; + child.stdout?.setEncoding("utf8").on("data", (chunk: string) => (stdout += chunk)); + child.stderr?.setEncoding("utf8").on("data", (chunk: string) => (stderr += chunk)); + child.on("error", reject); + child.on("close", (exitCode) => { + if (exitCode !== 0 && !options.allowFailure) { + if (stderr) process.stderr.write(stderr); + reject(new Error(`${command} ${args.join(" ")} failed with exit code ${exitCode}`)); + return; + } + resolve({ ok: exitCode === 0, stdout, stderr }); + }); + }); +} diff --git a/server/scripts/server-settings.ts b/server/core/src/deploy/server-settings.ts similarity index 99% rename from server/scripts/server-settings.ts rename to server/core/src/deploy/server-settings.ts index c3d2a3d..80e5d86 100644 --- a/server/scripts/server-settings.ts +++ b/server/core/src/deploy/server-settings.ts @@ -5,8 +5,8 @@ */ import * as Effect from "effect/Effect"; import * as Either from "effect/Either"; -import { readServerConfig } from "../core/src/config"; -import type { DeployEnv } from "./env"; +import { readServerConfig } from "../config.ts"; +import type { DeployEnv } from "./env.ts"; /** The `server` section of a deploy project's config, mapped onto SCRATCHWORK_* env vars. */ export interface ScratchworkServerConfig { diff --git a/server/core/src/error-pages.ts b/server/core/src/error-pages.ts index 4b6c506..d4b514f 100644 --- a/server/core/src/error-pages.ts +++ b/server/core/src/error-pages.ts @@ -6,8 +6,8 @@ */ import type * as HttpServerRequest from "@effect/platform/HttpServerRequest"; import * as HttpServerResponse from "@effect/platform/HttpServerResponse"; -import FIGURE_SVG from "../../../shared/assets/figure.svg" with { type: "text" }; -import { errorJson, securityHeaders } from "./http"; +import { FIGURE_SVG } from "@scratchwork/shared/assets/figure-svg.generated"; +import { errorJson, securityHeaders } from "./http.ts"; /** One link rendered as a button on an error page. */ export interface ErrorPageAction { diff --git a/server/core/src/google-jwt.ts b/server/core/src/google-jwt.ts index c02bbf0..096208b 100644 --- a/server/core/src/google-jwt.ts +++ b/server/core/src/google-jwt.ts @@ -10,8 +10,8 @@ import * as Data from "effect/Data"; import * as Effect from "effect/Effect"; import * as Option from "effect/Option"; import * as Schema from "effect/Schema"; -import { errorMessage } from "../../../shared/src/util/errors"; -import { CLOCK_SKEW_SECONDS, verifyRs256Jwt } from "./jwt-rs256"; +import { errorMessage } from "@scratchwork/shared/util/errors"; +import { CLOCK_SKEW_SECONDS, verifyRs256Jwt } from "./jwt-rs256.ts"; const GOOGLE_JWKS_URL = "https://www.googleapis.com/oauth2/v3/certs"; diff --git a/server/core/src/http.ts b/server/core/src/http.ts index d404cf2..3377838 100644 --- a/server/core/src/http.ts +++ b/server/core/src/http.ts @@ -2,8 +2,8 @@ import * as HttpServerRequest from "@effect/platform/HttpServerRequest"; import * as HttpServerResponse from "@effect/platform/HttpServerResponse"; import * as Data from "effect/Data"; import * as Effect from "effect/Effect"; -import { isLoopbackHost } from "../../../shared/src/util/url"; -import type { ServerConfigShape } from "./config"; +import { isLoopbackHost } from "@scratchwork/shared/util/url"; +import type { ServerConfigShape } from "./config.ts"; /** Generic HTTP failure; `status` becomes the response status and `message` the error body. */ export class HttpError extends Data.TaggedError("HttpError")<{ diff --git a/server/core/src/index.ts b/server/core/src/index.ts index 8733d17..7eaa3b9 100644 --- a/server/core/src/index.ts +++ b/server/core/src/index.ts @@ -5,7 +5,7 @@ */ // The HTTP app — one Effect handling every request; provide the services below to run it. -export { app } from "./app"; +export { app } from "./app.ts"; // Server configuration, parsed from SCRATCHWORK_* environment variables. export { @@ -18,10 +18,10 @@ export { type EnvVars, type OAuthAuthConfig, type ServerConfigShape, -} from "./config"; +} from "./config.ts"; // The auth service (Google OAuth or Cloudflare Access) and signed session tokens. -export { Auth, AuthError, AuthLive, createSessionToken, makeAuth, type AuthShape, type AuthUser } from "./auth"; +export { Auth, AuthError, AuthLive, createSessionToken, makeAuth, type AuthShape, type AuthUser } from "./auth.ts"; // Access-group expressions ("public" | "private" | emails/@domains) and identifier helpers. export { @@ -32,7 +32,7 @@ export { isSafeProjectIdentifier, type AccessGroup, type AccessPrincipal, -} from "./access"; +} from "./access.ts"; // Versioned JSON key-value contract with an in-memory implementation for tests/local runs. export { @@ -44,7 +44,7 @@ export { type JsonValue, type PrimitiveDbRecord, type PrimitiveDbShape, -} from "./db"; +} from "./db.ts"; // Blob-store contract with a local-filesystem implementation. export { @@ -56,7 +56,7 @@ export { type PutObjectOptions, type PutObjectResult, type StoredObject, -} from "./storage"; +} from "./storage.ts"; // The site store: publishing, loading, and access policy over storage + db. export { @@ -72,8 +72,8 @@ export { type PublishResult, type ShareRole, type SiteStoreShape, -} from "./site-store"; +} from "./site-store.ts"; // Persisted record shapes and route matching for published sites. -export { type SiteFileObject, type SiteRecord, type SiteRevisionRecord } from "./site-records"; -export { projectForRequest, routeRest } from "./routes"; +export { type SiteFileObject, type SiteRecord, type SiteRevisionRecord } from "./site-records.ts"; +export { projectForRequest, routeRest } from "./routes.ts"; diff --git a/server/core/src/jwt-rs256.ts b/server/core/src/jwt-rs256.ts index 785e364..4820bd3 100644 --- a/server/core/src/jwt-rs256.ts +++ b/server/core/src/jwt-rs256.ts @@ -13,7 +13,7 @@ import * as Either from "effect/Either"; import * as Encoding from "effect/Encoding"; import * as Predicate from "effect/Predicate"; import * as Schema from "effect/Schema"; -import { toArrayBuffer } from "../../../shared/src/encoding/bytes"; +import { toArrayBuffer } from "@scratchwork/shared/encoding/bytes"; /** Tolerated clock difference for exp/nbf/iat claim checks. */ export const CLOCK_SKEW_SECONDS = 300; diff --git a/server/core/src/object-site-files.ts b/server/core/src/object-site-files.ts index d3f8a99..7b54590 100644 --- a/server/core/src/object-site-files.ts +++ b/server/core/src/object-site-files.ts @@ -1,10 +1,10 @@ import * as HttpServerResponse from "@effect/platform/HttpServerResponse"; import * as Effect from "effect/Effect"; -import { contentType } from "../../../shared/src/site/content"; -import { SiteFileError, SiteFiles } from "../../../shared/src/site/files"; -import type { SitePath } from "../../../shared/src/site/paths"; -import type { SiteRevisionRecord } from "./site-records"; -import type { ObjectStorageShape } from "./storage"; +import { contentType } from "@scratchwork/shared/site/content"; +import { SiteFileError, SiteFiles } from "@scratchwork/shared/site/files"; +import type { SitePath } from "@scratchwork/shared/site/paths"; +import type { SiteRevisionRecord } from "./site-records.ts"; +import type { ObjectStorageShape } from "./storage.ts"; /** Builds a SiteFiles service backed by revision metadata and object storage. */ export function makeObjectSiteFiles( diff --git a/server/core/src/publish-request.ts b/server/core/src/publish-request.ts index ca8c4e7..26132f0 100644 --- a/server/core/src/publish-request.ts +++ b/server/core/src/publish-request.ts @@ -5,9 +5,9 @@ * and normalizes the open path. */ import * as Effect from "effect/Effect"; -import { decodedBase64ByteLength } from "../../../shared/src/encoding/base64"; -import { type PublishRequestBody } from "../../../shared/src/publish/api"; -import { HttpError } from "./http"; +import { decodedBase64ByteLength } from "@scratchwork/shared/encoding/base64"; +import { type PublishRequestBody } from "@scratchwork/shared/publish/api"; +import { HttpError } from "./http.ts"; /** Maximum accepted request body size (base64-encoded JSON, larger than the content caps). */ export const MAX_PUBLISH_BODY_BYTES = 30 * 1024 * 1024; diff --git a/server/core/src/routes.ts b/server/core/src/routes.ts index e414438..a55df43 100644 --- a/server/core/src/routes.ts +++ b/server/core/src/routes.ts @@ -3,7 +3,7 @@ * path segment — its globally unique name — so a request path maps to at most one project * plus a site-file remainder. No storage or service dependencies. */ -import { isSafeProjectIdentifier } from "./access"; +import { isSafeProjectIdentifier } from "./access.ts"; /** Maps a request pathname to the only project it can belong to: its decoded first * segment. Returns null when the path has no segments or the decoded segment is not a diff --git a/server/core/src/share-request.ts b/server/core/src/share-request.ts index a105558..45d5961 100644 --- a/server/core/src/share-request.ts +++ b/server/core/src/share-request.ts @@ -6,9 +6,9 @@ * place. */ import * as Effect from "effect/Effect"; -import type { ShareRequest } from "../../../shared/src/publish/api"; -import { HttpError } from "./http"; -import type { ShareChanges } from "./site-store"; +import type { ShareRequest } from "@scratchwork/shared/publish/api"; +import { HttpError } from "./http.ts"; +import type { ShareChanges } from "./site-store.ts"; /** Maximum accepted share request body size. */ export const MAX_SHARE_BODY_BYTES = 64 * 1024; diff --git a/server/core/src/site-records.ts b/server/core/src/site-records.ts index 2a638d3..5038875 100644 --- a/server/core/src/site-records.ts +++ b/server/core/src/site-records.ts @@ -8,8 +8,8 @@ * pointer is the single server-wide claim on the project name. */ import * as Schema from "effect/Schema"; -import { isSafeSitePath } from "../../../shared/src/site/paths"; -import { isSafeProjectIdentifier } from "./access"; +import { isSafeSitePath } from "@scratchwork/shared/site/paths"; +import { isSafeProjectIdentifier } from "./access.ts"; /** DB namespace of mutable project pointers, keyed by bare project name. */ export const PROJECTS_NAMESPACE = "projects"; diff --git a/server/core/src/site-store.ts b/server/core/src/site-store.ts index 9d2b13c..27503af 100644 --- a/server/core/src/site-store.ts +++ b/server/core/src/site-store.ts @@ -12,8 +12,8 @@ import * as Encoding from "effect/Encoding"; import * as Layer from "effect/Layer"; import * as ParseResult from "effect/ParseResult"; import * as Schema from "effect/Schema"; -import type { PublishResponse } from "../../../shared/src/publish/api"; -import { contentType } from "../../../shared/src/site/content"; +import type { PublishResponse } from "@scratchwork/shared/publish/api"; +import { contentType } from "@scratchwork/shared/site/content"; import { accessGroupMatches, accessGroupModify, @@ -21,9 +21,9 @@ import { isReservedSlug, isSafeProjectIdentifier, type AccessGroup, -} from "./access"; -import type { AuthUser } from "./auth"; -import type { ServerConfigShape } from "./config"; +} from "./access.ts"; +import type { AuthUser } from "./auth.ts"; +import type { ServerConfigShape } from "./config.ts"; import { PrimitiveDb, PrimitiveDbConflict, @@ -31,9 +31,9 @@ import { type JsonValue, type PrimitiveDbRecord, type PrimitiveDbShape, -} from "./db"; -import { makeObjectSiteFiles } from "./object-site-files"; -import type { PublishRequest } from "./publish-request"; +} from "./db.ts"; +import { makeObjectSiteFiles } from "./object-site-files.ts"; +import type { PublishRequest } from "./publish-request.ts"; import { blobObjectKey, OWNER_INDEX_NAMESPACE, @@ -48,9 +48,9 @@ import { type SiteOwner, type SiteRecord, type SiteRevisionRecord, -} from "./site-records"; -import { ObjectStorage, sha256Hex, StorageError, type ObjectStorageShape } from "./storage"; -import { randomRevisionId, randomSlug } from "./tokens"; +} from "./site-records.ts"; +import { ObjectStorage, sha256Hex, StorageError, type ObjectStorageShape } from "./storage.ts"; +import { randomRevisionId, randomSlug } from "./tokens.ts"; /** How many random-name candidates a create attempts before giving up. */ const RANDOM_NAME_ATTEMPTS = 3; diff --git a/server/core/src/storage.ts b/server/core/src/storage.ts index c76c3ff..f04813a 100644 --- a/server/core/src/storage.ts +++ b/server/core/src/storage.ts @@ -4,8 +4,8 @@ import * as Context from "effect/Context"; import * as Data from "effect/Data"; import * as Effect from "effect/Effect"; import * as Layer from "effect/Layer"; -import { sha256Hex as sha256HexDigest } from "../../../shared/src/crypto/digest"; -import { isWithinRoot } from "../../../shared/src/util/fs"; +import { sha256Hex as sha256HexDigest } from "@scratchwork/shared/crypto/digest"; +import { isWithinRoot } from "@scratchwork/shared/util/fs"; /** Raised when a storage backend cannot complete a read or write. */ export class StorageError extends Data.TaggedError("StorageError")<{ diff --git a/server/scripts/env.test.ts b/server/core/test/deploy-env.test.ts similarity index 98% rename from server/scripts/env.test.ts rename to server/core/test/deploy-env.test.ts index 4af906a..978e8a4 100644 --- a/server/scripts/env.test.ts +++ b/server/core/test/deploy-env.test.ts @@ -2,7 +2,7 @@ import { describe, expect, test } from "bun:test"; import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; -import { loadDeployEnv } from "./env"; +import { loadDeployEnv } from "../src/deploy/env.ts"; describe("loadDeployEnv", () => { test("loads server, package, explicit env files with shell precedence", async () => { diff --git a/server/scripts/server-settings.test.ts b/server/core/test/deploy-server-settings.test.ts similarity index 98% rename from server/scripts/server-settings.test.ts rename to server/core/test/deploy-server-settings.test.ts index e5f0b4e..64e4c6d 100644 --- a/server/scripts/server-settings.test.ts +++ b/server/core/test/deploy-server-settings.test.ts @@ -1,5 +1,5 @@ import { describe, expect, test } from "bun:test"; -import { serverConfigEnv, serverConfigEnvNames, validateDeploymentConfig, type ScratchworkServerConfig } from "./server-settings"; +import { serverConfigEnv, serverConfigEnvNames, validateDeploymentConfig, type ScratchworkServerConfig } from "../src/deploy/server-settings.ts"; describe("serverConfigEnvNames", () => { test("pins the exact non-secret variable set forwarded to cloud deploys", () => { diff --git a/server/core/tsconfig.build.json b/server/core/tsconfig.build.json new file mode 100644 index 0000000..bb46d92 --- /dev/null +++ b/server/core/tsconfig.build.json @@ -0,0 +1,19 @@ +// Publish build (scripts/build-packages.ts): tsc emit into dist/. The paths +// mapping resolves @scratchwork/shared against its already-built dist so this +// package's emit consumes the same declarations consumers will (and tsc never +// pulls workspace .ts sources from outside rootDir). Build order: shared first. +{ + "extends": "./tsconfig.json", + "compilerOptions": { + "noEmit": false, + "declaration": true, + "outDir": "dist", + "rootDir": "src", + "rewriteRelativeImportExtensions": true, + "paths": { + "@scratchwork/shared/*": ["../../shared/dist/*"] + } + }, + "include": ["src/**/*.ts"], + "exclude": ["src/**/*.test.ts"] +} diff --git a/server/deploy-aws/README.md b/server/deploy-aws/README.md new file mode 100644 index 0000000..9eaaa90 --- /dev/null +++ b/server/deploy-aws/README.md @@ -0,0 +1,15 @@ +# @scratchwork/server-deploy-aws + +AWS deployment package for the +[Scratchwork](https://github.com/scratch/scratchwork) publishing server: +Lambda handler plus S3 (`ObjectStorage`) and DynamoDB (`PrimitiveDb`) +adapters for `@scratchwork/server-core`. + +```ts +import { handler } from "@scratchwork/server-deploy-aws/handler"; +``` + +Published as built ESM JavaScript with type declarations; works under Node ≥ 22 +or Bun. To deploy your own server, start from the walkthrough in the +[repo](https://github.com/scratch/scratchwork)'s `server/README.md` and the +`deploy/generic-aws` template. MIT license. diff --git a/server/deploy-aws/package.json b/server/deploy-aws/package.json index 5d7d4d9..0cfb1d4 100644 --- a/server/deploy-aws/package.json +++ b/server/deploy-aws/package.json @@ -2,8 +2,19 @@ "name": "@scratchwork/server-deploy-aws", "version": "0.1.0", "type": "module", - "private": true, "description": "AWS Lambda + S3 deployment package for the Scratchwork server.", + "license": "MIT", + "repository": { + "type": "git", + "url": "git+https://github.com/scratch/scratchwork.git", + "directory": "server/deploy-aws" + }, + "engines": { + "node": ">=22" + }, + "files": [ + "dist" + ], "exports": { ".": "./src/index.ts", "./dynamodb-db": "./src/dynamodb-db.ts", diff --git a/server/deploy-aws/src/deploy.ts b/server/deploy-aws/src/deploy.ts index ee827f4..115835f 100644 --- a/server/deploy-aws/src/deploy.ts +++ b/server/deploy-aws/src/deploy.ts @@ -6,8 +6,8 @@ import { mkdir, rm, writeFile } from "node:fs/promises"; import { dirname, join } from "node:path"; import { fileURLToPath } from "node:url"; -import { definedEnv, loadDeployEnv, type DeployEnv } from "../../scripts/env"; -import { createRunner, type RunOptions, type RunResult } from "../../scripts/proc"; +import { definedEnv, loadDeployEnv, type DeployEnv } from "@scratchwork/server-core/deploy/env"; +import { createRunner, type RunOptions, type RunResult } from "@scratchwork/server-core/deploy/proc"; import { deployArgv, homepagePublishHint, @@ -17,7 +17,7 @@ import { validateDeploymentConfig, type DeployServerOptions, type ScratchworkServerConfig, -} from "../../scripts/server-settings"; +} from "@scratchwork/server-core/deploy/server-settings"; /** Deploy options and server settings, shared with the other deploy packages. */ export type { DeployServerOptions as AwsDeployOptions, ScratchworkServerConfig }; diff --git a/server/deploy-aws/src/handler.ts b/server/deploy-aws/src/handler.ts index 47d825b..24099fd 100644 --- a/server/deploy-aws/src/handler.ts +++ b/server/deploy-aws/src/handler.ts @@ -6,8 +6,8 @@ import type { APIGatewayProxyStructuredResultV2, Context as LambdaContext, } from "aws-lambda"; -import { DynamoDbPrimitiveDbLive } from "./dynamodb-db"; -import { S3ObjectStorageLive } from "./s3-storage"; +import { DynamoDbPrimitiveDbLive } from "./dynamodb-db.ts"; +import { S3ObjectStorageLive } from "./s3-storage.ts"; const env = process.env as EnvVars; /** The full service graph for the Lambda runtime, built from process.env. */ diff --git a/server/deploy-aws/src/index.ts b/server/deploy-aws/src/index.ts index fdc402b..d8cd0bc 100644 --- a/server/deploy-aws/src/index.ts +++ b/server/deploy-aws/src/index.ts @@ -5,6 +5,6 @@ export { type AwsDeployResult, type AwsDeployServerConfig, type ScratchworkServerConfig, -} from "./deploy"; -export { DynamoDbPrimitiveDbLive, makeDynamoDbPrimitiveDb, readDynamoDbPrimitiveDbConfig, type DynamoDbPrimitiveDbConfig } from "./dynamodb-db"; -export { S3ObjectStorageLive } from "./s3-storage"; +} from "./deploy.ts"; +export { DynamoDbPrimitiveDbLive, makeDynamoDbPrimitiveDb, readDynamoDbPrimitiveDbConfig, type DynamoDbPrimitiveDbConfig } from "./dynamodb-db.ts"; +export { S3ObjectStorageLive } from "./s3-storage.ts"; diff --git a/server/deploy-aws/tsconfig.build.json b/server/deploy-aws/tsconfig.build.json new file mode 100644 index 0000000..91d8bd4 --- /dev/null +++ b/server/deploy-aws/tsconfig.build.json @@ -0,0 +1,20 @@ +// Publish build (scripts/build-packages.ts): tsc emit into dist/, resolving +// workspace dependencies against their already-built dist (see +// server/core/tsconfig.build.json for why). Build order: shared, core, then this. +{ + "extends": "./tsconfig.json", + "compilerOptions": { + "noEmit": false, + "declaration": true, + "outDir": "dist", + "rootDir": "src", + "rewriteRelativeImportExtensions": true, + "paths": { + "@scratchwork/shared/*": ["../../shared/dist/*"], + "@scratchwork/server-core": ["../core/dist/index"], + "@scratchwork/server-core/*": ["../core/dist/*"] + } + }, + "include": ["src/**/*.ts"], + "exclude": ["src/**/*.test.ts"] +} diff --git a/server/deploy-cloudflare/README.md b/server/deploy-cloudflare/README.md index e2caaae..9ed6717 100644 --- a/server/deploy-cloudflare/README.md +++ b/server/deploy-cloudflare/README.md @@ -1,8 +1,14 @@ # Cloudflare deploy package -Deploys the Scratchwork server as a Cloudflare Worker backed by R2 and D1. It can -also run that exact Worker locally through Wrangler/workerd, with persistent local R2 -and D1 bindings. +Deploys the [Scratchwork](https://github.com/scratch/scratchwork) server as a +Cloudflare Worker backed by R2 and D1. It can also run that exact Worker locally +through Wrangler/workerd, with persistent local R2 and D1 bindings. + +Published to npm as `@scratchwork/server-deploy-cloudflare`: built ESM +JavaScript with type declarations, works under Node ≥ 22 or Bun (the Worker +itself runs on Cloudflare's runtime). To deploy your own server, start from the +walkthrough in the repository's `server/README.md` and the +`deploy/cloudflare-vanilla` template. MIT license. ## Local Cloudflare runtime diff --git a/server/deploy-cloudflare/package.json b/server/deploy-cloudflare/package.json index 63ccddb..854b302 100644 --- a/server/deploy-cloudflare/package.json +++ b/server/deploy-cloudflare/package.json @@ -2,8 +2,19 @@ "name": "@scratchwork/server-deploy-cloudflare", "version": "0.1.0", "type": "module", - "private": true, "description": "Cloudflare Worker + R2 deployment package for the Scratchwork server.", + "license": "MIT", + "repository": { + "type": "git", + "url": "git+https://github.com/scratch/scratchwork.git", + "directory": "server/deploy-cloudflare" + }, + "engines": { + "node": ">=22" + }, + "files": [ + "dist" + ], "exports": { ".": "./src/index.ts", "./d1-db": "./src/d1-db.ts", @@ -21,6 +32,7 @@ "dependencies": { "@effect/platform": "0.96.2", "@scratchwork/server-core": "workspace:*", + "@scratchwork/shared": "workspace:*", "cloudflare": "^6.5.0", "effect": "3.21.4" }, diff --git a/server/deploy-cloudflare/src/deploy.ts b/server/deploy-cloudflare/src/deploy.ts index f320e9d..da335ab 100644 --- a/server/deploy-cloudflare/src/deploy.ts +++ b/server/deploy-cloudflare/src/deploy.ts @@ -10,8 +10,8 @@ import Cloudflare, { APIError, toFile } from "cloudflare"; import { mkdir, readFile, writeFile } from "node:fs/promises"; import { dirname, join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; -import { copyEnv, definedEnv, loadDeployEnv, type DeployEnv } from "../../scripts/env"; -import { createRunner } from "../../scripts/proc"; +import { copyEnv, definedEnv, loadDeployEnv, type DeployEnv } from "@scratchwork/server-core/deploy/env"; +import { createRunner } from "@scratchwork/server-core/deploy/proc"; import { deployArgv, homepagePublishHint, @@ -23,7 +23,7 @@ import { type DeployServerOptions, type ResolvedScratchworkServerConfig, type ScratchworkServerConfig, -} from "../../scripts/server-settings"; +} from "@scratchwork/server-core/deploy/server-settings"; /** Worker compatibility date shared by production deploys and conformance lanes. */ export const DEFAULT_CLOUDFLARE_COMPATIBILITY_DATE = "2026-06-01"; diff --git a/server/deploy-cloudflare/src/index.ts b/server/deploy-cloudflare/src/index.ts index c440f80..5159194 100644 --- a/server/deploy-cloudflare/src/index.ts +++ b/server/deploy-cloudflare/src/index.ts @@ -11,6 +11,6 @@ export { type CloudflareR2BucketConfig, type CloudflareRouteConfig, type ScratchworkServerConfig, -} from "./deploy"; -export { D1PrimitiveDbLive, makeD1PrimitiveDb, type D1DatabaseBinding, type D1PreparedStatementBinding, type D1PrimitiveDbOptions } from "./d1-db"; -export { R2ObjectStorageLive, type R2BucketBinding } from "./r2-storage"; +} from "./deploy.ts"; +export { D1PrimitiveDbLive, makeD1PrimitiveDb, type D1DatabaseBinding, type D1PreparedStatementBinding, type D1PrimitiveDbOptions } from "./d1-db.ts"; +export { R2ObjectStorageLive, type R2BucketBinding } from "./r2-storage.ts"; diff --git a/server/deploy-cloudflare/src/local-worker.ts b/server/deploy-cloudflare/src/local-worker.ts index cfd1747..9494ed7 100644 --- a/server/deploy-cloudflare/src/local-worker.ts +++ b/server/deploy-cloudflare/src/local-worker.ts @@ -4,8 +4,8 @@ * and D1 bindings; this wrapper supplies the one edge feature Miniflare cannot emulate. */ import * as Encoding from "effect/Encoding"; -import { toArrayBuffer } from "../../../shared/src/encoding/bytes"; -import worker from "./worker"; +import { toArrayBuffer } from "@scratchwork/shared/encoding/bytes"; +import worker from "./worker.ts"; interface LocalAccessEnv extends Record { readonly SCRATCHWORK_LOCAL_CF_ACCESS_PRIVATE_JWK: string; diff --git a/server/deploy-cloudflare/src/worker.ts b/server/deploy-cloudflare/src/worker.ts index 2ac6a6b..4125440 100644 --- a/server/deploy-cloudflare/src/worker.ts +++ b/server/deploy-cloudflare/src/worker.ts @@ -1,8 +1,8 @@ import * as HttpApp from "@effect/platform/HttpApp"; import { AuthLive, app, makeServerConfigLayer, SiteStoreLive, type EnvVars } from "@scratchwork/server-core"; import * as Layer from "effect/Layer"; -import { D1PrimitiveDbLive, type D1DatabaseBinding } from "./d1-db"; -import { R2ObjectStorageLive, type R2BucketBinding } from "./r2-storage"; +import { D1PrimitiveDbLive, type D1DatabaseBinding } from "./d1-db.ts"; +import { R2ObjectStorageLive, type R2BucketBinding } from "./r2-storage.ts"; /** The Worker environment: the two service bindings plus string config vars. */ interface CloudflareEnv extends Record { diff --git a/server/deploy-cloudflare/tsconfig.build.json b/server/deploy-cloudflare/tsconfig.build.json new file mode 100644 index 0000000..91d8bd4 --- /dev/null +++ b/server/deploy-cloudflare/tsconfig.build.json @@ -0,0 +1,20 @@ +// Publish build (scripts/build-packages.ts): tsc emit into dist/, resolving +// workspace dependencies against their already-built dist (see +// server/core/tsconfig.build.json for why). Build order: shared, core, then this. +{ + "extends": "./tsconfig.json", + "compilerOptions": { + "noEmit": false, + "declaration": true, + "outDir": "dist", + "rootDir": "src", + "rewriteRelativeImportExtensions": true, + "paths": { + "@scratchwork/shared/*": ["../../shared/dist/*"], + "@scratchwork/server-core": ["../core/dist/index"], + "@scratchwork/server-core/*": ["../core/dist/*"] + } + }, + "include": ["src/**/*.ts"], + "exclude": ["src/**/*.test.ts"] +} diff --git a/server/deploy-local/README.md b/server/deploy-local/README.md index f2c1fae..75dfe50 100644 --- a/server/deploy-local/README.md +++ b/server/deploy-local/README.md @@ -1,8 +1,14 @@ # Local Scratchwork deploy -This package runs the shared Scratchwork server app as a local deploy target, -with local file storage and an in-memory database. It lets any deploy project -under `deploy/` run its server config locally. +This package runs the shared +[Scratchwork](https://github.com/scratch/scratchwork) server app as a local +deploy target, with local file storage and an in-memory database. It lets any +deploy project under `deploy/` run its server config locally. + +Published to npm as `@scratchwork/server-deploy-local`: built ESM JavaScript +with type declarations. Requires Bun ≥ 1.2 at runtime (it serves with +`@effect/platform-bun`); the other deploy packages are runtime-neutral. +MIT license. Run the generic local development server from the repo root: diff --git a/server/deploy-local/package.json b/server/deploy-local/package.json index 50c3baf..57812be 100644 --- a/server/deploy-local/package.json +++ b/server/deploy-local/package.json @@ -2,8 +2,19 @@ "name": "@scratchwork/server-deploy-local", "version": "0.1.0", "type": "module", - "private": true, "description": "Local Bun deploy target for the Scratchwork server.", + "license": "MIT", + "repository": { + "type": "git", + "url": "git+https://github.com/scratch/scratchwork.git", + "directory": "server/deploy-local" + }, + "engines": { + "bun": ">=1.2" + }, + "files": [ + "dist" + ], "exports": { ".": "./src/index.ts" }, diff --git a/server/deploy-local/src/index.ts b/server/deploy-local/src/index.ts index ebf7326..27ac0be 100644 --- a/server/deploy-local/src/index.ts +++ b/server/deploy-local/src/index.ts @@ -1,2 +1,2 @@ -export { runLocalServer, type RunLocalServerOptions } from "./run"; -export type { ScratchworkServerConfig } from "../../scripts/server-settings"; +export { runLocalServer, type RunLocalServerOptions } from "./run.ts"; +export type { ScratchworkServerConfig } from "@scratchwork/server-core/deploy/server-settings"; diff --git a/server/deploy-local/src/run.ts b/server/deploy-local/src/run.ts index 7f60ea5..ca272db 100644 --- a/server/deploy-local/src/run.ts +++ b/server/deploy-local/src/run.ts @@ -13,7 +13,7 @@ import { import * as Console from "effect/Console"; import * as Effect from "effect/Effect"; import * as Layer from "effect/Layer"; -import { serverConfigEnvEntries, type ScratchworkServerConfig } from "../../scripts/server-settings"; +import { serverConfigEnvEntries, type ScratchworkServerConfig } from "@scratchwork/server-core/deploy/server-settings"; /** Options for a local run: the deploy project's `server` settings plus an env override * for tests. Using the shared config shape lets a deploy project run one config module diff --git a/server/deploy-local/tsconfig.build.json b/server/deploy-local/tsconfig.build.json new file mode 100644 index 0000000..91d8bd4 --- /dev/null +++ b/server/deploy-local/tsconfig.build.json @@ -0,0 +1,20 @@ +// Publish build (scripts/build-packages.ts): tsc emit into dist/, resolving +// workspace dependencies against their already-built dist (see +// server/core/tsconfig.build.json for why). Build order: shared, core, then this. +{ + "extends": "./tsconfig.json", + "compilerOptions": { + "noEmit": false, + "declaration": true, + "outDir": "dist", + "rootDir": "src", + "rewriteRelativeImportExtensions": true, + "paths": { + "@scratchwork/shared/*": ["../../shared/dist/*"], + "@scratchwork/server-core": ["../core/dist/index"], + "@scratchwork/server-core/*": ["../core/dist/*"] + } + }, + "include": ["src/**/*.ts"], + "exclude": ["src/**/*.test.ts"] +} diff --git a/server/deploy-local/tsconfig.json b/server/deploy-local/tsconfig.json index ab7ca7b..357a46a 100644 --- a/server/deploy-local/tsconfig.json +++ b/server/deploy-local/tsconfig.json @@ -19,7 +19,6 @@ "include": [ "src/**/*.ts", "../scripts/**/*.ts", - "../core/src/assets.d.ts", "../../shared/src/**/*.ts", "../../shared/**/*.js" ] diff --git a/server/package.json b/server/package.json deleted file mode 100644 index a27a10d..0000000 --- a/server/package.json +++ /dev/null @@ -1,16 +0,0 @@ -{ - "name": "scratchwork-server", - "version": "0.1.0", - "type": "module", - "private": true, - "description": "Scratchwork server deploy tooling (scripts/). The server itself lives in the core and deploy-* workspaces, each with its own typecheck/test/ci.", - "scripts": { - "ci": "bun run typecheck && bun run test", - "test": "bun test scripts/", - "typecheck": "tsc -p tsconfig.json" - }, - "devDependencies": { - "@types/bun": "^1.3.14", - "typescript": "^6.0.3" - } -} diff --git a/server/scripts/proc.ts b/server/scripts/proc.ts deleted file mode 100644 index 1c7c545..0000000 --- a/server/scripts/proc.ts +++ /dev/null @@ -1,52 +0,0 @@ -/** Options for one spawned command. */ -export interface RunOptions { - readonly allowFailure?: boolean; - readonly capture?: boolean; - readonly cwd?: string; -} - -/** Exit status and captured output of one spawned command. */ -export interface RunResult { - readonly ok: boolean; - readonly stdout: string; - readonly stderr: string; -} - -/** Creates a command runner that uses one captured deploy environment. */ -export function createRunner(commandEnv: Record) { - return { - run: (command: string, args: ReadonlyArray, options: RunOptions = {}) => - run(command, args, commandEnv, options), - }; -} - -/** Spawns one command and optionally captures output or tolerates failure. */ -async function run( - command: string, - args: ReadonlyArray, - commandEnv: Record, - options: RunOptions, -): Promise { - const proc = Bun.spawn([command, ...args], { - cwd: options.cwd, - env: commandEnv, - stdout: options.capture ? "pipe" : "inherit", - stderr: options.capture || options.allowFailure ? "pipe" : "inherit", - }); - const [stdout, stderr, exitCode] = await Promise.all([ - read(proc.stdout), - read(proc.stderr), - proc.exited, - ]); - if (exitCode !== 0 && !options.allowFailure) { - if (stderr) process.stderr.write(stderr); - throw new Error(`${command} ${args.join(" ")} failed with exit code ${exitCode}`); - } - return { ok: exitCode === 0, stdout, stderr }; -} - -/** Reads a spawned process stream into a string. */ -async function read(stream: ReadableStream | null | undefined): Promise { - if (stream == null) return ""; - return new Response(stream).text(); -} diff --git a/server/tsconfig.json b/server/tsconfig.json deleted file mode 100644 index f5c3bad..0000000 --- a/server/tsconfig.json +++ /dev/null @@ -1,22 +0,0 @@ -{ - "compilerOptions": { - "allowArbitraryExtensions": true, - "allowImportingTsExtensions": true, - "allowJs": true, - "baseUrl": ".", - "checkJs": false, - "ignoreDeprecations": "6.0", - "lib": ["ESNext", "DOM"], - "module": "Preserve", - "moduleDetection": "force", - "moduleResolution": "bundler", - "noEmit": true, - "resolveJsonModule": true, - "skipLibCheck": true, - "strict": true, - "target": "ESNext", - "types": ["bun"], - "verbatimModuleSyntax": true - }, - "include": ["scripts/**/*.ts"] -} diff --git a/shared/README.md b/shared/README.md new file mode 100644 index 0000000..e4568ca --- /dev/null +++ b/shared/README.md @@ -0,0 +1,19 @@ +# @scratchwork/shared + +Code shared between the [Scratchwork](https://github.com/scratch/scratchwork) +CLI and server: the publish API contract (Effect Schemas + `HttpApi`), site +serving helpers, and small utilities. You normally don't depend on this +directly — it comes in as a dependency of `@scratchwork/server-core` and the +deploy packages. + +Published as built ESM JavaScript with type declarations; works under Node ≥ 22 +or Bun. Every module is importable by subpath, e.g.: + +```ts +import { ScratchworkApi } from "@scratchwork/shared/publish/api"; +import { contentType } from "@scratchwork/shared/site/content"; +``` + +Part of the Scratchwork repository — see the +[repo](https://github.com/scratch/scratchwork) for development, and +`server/README.md` there for the deploy-your-own walkthrough. MIT license. diff --git a/shared/package.json b/shared/package.json index 000696e..726b6d0 100644 --- a/shared/package.json +++ b/shared/package.json @@ -2,10 +2,22 @@ "name": "@scratchwork/shared", "version": "0.1.0", "type": "module", - "private": true, "description": "Code shared between the Scratchwork CLI and server: the publish API contract, site serving helpers, and small utilities.", + "license": "MIT", + "repository": { + "type": "git", + "url": "git+https://github.com/scratch/scratchwork.git", + "directory": "shared" + }, + "engines": { + "node": ">=22" + }, + "files": [ + "dist" + ], "exports": { - "./*": "./src/*" + "./*.js": "./src/*.js", + "./*": "./src/*.ts" }, "scripts": { "ci": "bun run typecheck && bun run test", diff --git a/shared/src/assets/figure-svg.generated.ts b/shared/src/assets/figure-svg.generated.ts new file mode 100644 index 0000000..299eede --- /dev/null +++ b/shared/src/assets/figure-svg.generated.ts @@ -0,0 +1,3 @@ +// Generated by scripts/generate-assets.ts from shared/assets/figure.svg — do not edit. +// Regenerate with: bun scripts/generate-assets.ts +export const FIGURE_SVG: string = "\n"; diff --git a/shared/src/crypto/digest.ts b/shared/src/crypto/digest.ts index e48032e..f9efb2b 100644 --- a/shared/src/crypto/digest.ts +++ b/shared/src/crypto/digest.ts @@ -7,7 +7,7 @@ * and callers wrap these helpers exactly once with Effect.tryPromise. */ import * as Encoding from "effect/Encoding"; -import { toArrayBuffer } from "../encoding/bytes"; +import { toArrayBuffer } from "../encoding/bytes.ts"; /** Computes the base64url SHA-256 digest of a UTF-8 string (PKCE S256). */ export async function sha256Base64Url(value: string): Promise { diff --git a/shared/src/publish/api.ts b/shared/src/publish/api.ts index 35cc615..8aeb6cb 100644 --- a/shared/src/publish/api.ts +++ b/shared/src/publish/api.ts @@ -17,8 +17,8 @@ import * as HttpApiEndpoint from "@effect/platform/HttpApiEndpoint"; import * as HttpApiGroup from "@effect/platform/HttpApiGroup"; import * as HttpApiSchema from "@effect/platform/HttpApiSchema"; import * as Schema from "effect/Schema"; -import { isSafeProjectIdentifier } from "../site/identifiers"; -import { PublishBundleSchema } from "./bundle"; +import { isSafeProjectIdentifier } from "../site/identifiers.ts"; +import { PublishBundleSchema } from "./bundle.ts"; export { PublishBundleSchema }; diff --git a/shared/src/publish/bundle.ts b/shared/src/publish/bundle.ts index be861b0..c41b4df 100644 --- a/shared/src/publish/bundle.ts +++ b/shared/src/publish/bundle.ts @@ -5,8 +5,8 @@ * request schema in api.ts) and the CLI decodes clone downloads through it. */ import * as Schema from "effect/Schema"; -import { decodedBase64ByteLength } from "../encoding/base64"; -import { isSafeSitePath } from "../site/paths"; +import { decodedBase64ByteLength } from "../encoding/base64.ts"; +import { isSafeSitePath } from "../site/paths.ts"; /** Version number both sides must agree on before reading a bundle. */ export const PUBLISH_BUNDLE_VERSION = 1; diff --git a/shared/src/site/components.ts b/shared/src/site/components.ts index f172599..6eeebfd 100644 --- a/shared/src/site/components.ts +++ b/shared/src/site/components.ts @@ -5,7 +5,7 @@ * this file in sync with renderer/src/components.js so dev diagnostics agree * with what the renderer actually loads. */ -import { dirnameSitePath, joinSitePath, type SitePath } from "./paths"; +import { dirnameSitePath, joinSitePath, type SitePath } from "./paths.ts"; /** * Blanks out `inline code` spans in a line so their contents are not scanned. diff --git a/shared/src/site/default-renderer.generated.js b/shared/src/site/default-renderer.generated.js index 6e752f2..63ef191 100644 --- a/shared/src/site/default-renderer.generated.js +++ b/shared/src/site/default-renderer.generated.js @@ -1,4 +1,6 @@ // AUTO-GENERATED by renderer/build.js — do not edit. -export const defaultRendererSourceHash = "67ac94fe85f68eb4448c9aa641ff5d49be3986d4b5ce67c6675c8f6474cdf2d4"; -export const defaultRendererHtml = "\n\n\n \n \n \n \n\n \n \n \n \n \n \n\n
\n\n \n \n \n\n"; +// The JSDoc casts keep tsc declaration emit at `string` instead of a +// megabyte-scale literal type when shared is built for npm publishing. +export const defaultRendererSourceHash = /** @type {string} */ ("8b9fd970c123a0ad2b0f05d7af21540468918ef4c7eece877ef7778e2297d1c7"); +export const defaultRendererHtml = /** @type {string} */ ("\n\n\n \n \n \n \n\n \n \n \n \n \n \n\n
\n\n \n \n \n\n"); export default defaultRendererHtml; diff --git a/shared/src/site/files.ts b/shared/src/site/files.ts index bb244a2..d75cb58 100644 --- a/shared/src/site/files.ts +++ b/shared/src/site/files.ts @@ -8,7 +8,7 @@ import type * as HttpServerResponse from "@effect/platform/HttpServerResponse"; import * as Context from "effect/Context"; import * as Data from "effect/Data"; import type * as Effect from "effect/Effect"; -import type { SitePath } from "./paths"; +import type { SitePath } from "./paths.ts"; /** Failure reading a site file, tagged with why (forbidden, missing, or IO error). */ export class SiteFileError extends Data.TaggedError("SiteFileError")<{ diff --git a/shared/src/site/html.ts b/shared/src/site/html.ts index d5a5b0c..27ac7d8 100644 --- a/shared/src/site/html.ts +++ b/shared/src/site/html.ts @@ -4,7 +4,7 @@ * served — for example to inject the live-reload script. */ import * as Effect from "effect/Effect"; -import type { SitePath } from "./paths"; +import type { SitePath } from "./paths.ts"; /** Where the HTML being transformed came from: a static file or a renderer shell. */ export interface HtmlContext { diff --git a/shared/src/site/renderer.ts b/shared/src/site/renderer.ts index 816e6e2..4becb6c 100644 --- a/shared/src/site/renderer.ts +++ b/shared/src/site/renderer.ts @@ -4,9 +4,9 @@ * fallback shell (the one embedded in the CLI/server) is used. */ import * as Effect from "effect/Effect"; -import { SiteFileError, SiteFiles } from "./files"; -import { isMarkedMarkdownRenderer } from "./marker"; -import { dirnameSitePath, joinSitePath, type SitePath } from "./paths"; +import { SiteFileError, SiteFiles } from "./files.ts"; +import { isMarkedMarkdownRenderer } from "./marker.ts"; +import { dirnameSitePath, joinSitePath, type SitePath } from "./paths.ts"; /** The renderer shell to serve for a Markdown route, and where it came from. */ export type MarkdownRenderer = diff --git a/shared/src/site/routing.ts b/shared/src/site/routing.ts index b3da87b..3806c38 100644 --- a/shared/src/site/routing.ts +++ b/shared/src/site/routing.ts @@ -7,9 +7,9 @@ */ import * as Data from "effect/Data"; import * as Effect from "effect/Effect"; -import { extensionOf } from "./content"; -import { SiteFileError, SiteFiles } from "./files"; -import { isMarkedMarkdownRenderer } from "./marker"; +import { extensionOf } from "./content.ts"; +import { SiteFileError, SiteFiles } from "./files.ts"; +import { isMarkedMarkdownRenderer } from "./marker.ts"; import { basenameSitePath, dirnameSitePath, @@ -17,7 +17,7 @@ import { joinSitePath, stripExtension, type SitePath, -} from "./paths"; +} from "./paths.ts"; /** The favicon a browser requests by default when a page names none. */ export const FAVICON_ICO_PATH = "favicon.ico"; diff --git a/shared/src/site/serve.ts b/shared/src/site/serve.ts index af76c9b..5cd64cf 100644 --- a/shared/src/site/serve.ts +++ b/shared/src/site/serve.ts @@ -9,20 +9,20 @@ import * as HttpServerRequest from "@effect/platform/HttpServerRequest"; import * as HttpServerResponse from "@effect/platform/HttpServerResponse"; import * as Effect from "effect/Effect"; -import { contentType, defaultCacheControl, isMarkdownPath } from "./content"; -import { SiteFileError, SiteFiles } from "./files"; -import { applyHtmlTransforms, type HtmlTransform } from "./html"; +import { contentType, defaultCacheControl, isMarkdownPath } from "./content.ts"; +import { SiteFileError, SiteFiles } from "./files.ts"; +import { applyHtmlTransforms, type HtmlTransform } from "./html.ts"; import { resolveMarkdownRenderer, type MarkdownRenderer, -} from "./renderer"; +} from "./renderer.ts"; import { FAVICON_SVG_PATH, parseRouteRequest, resolveRoute, SiteRouteError, type ResolvedRoute, -} from "./routing"; +} from "./routing.ts"; /** Which renderer shell answered a Markdown route, for logging/diagnostics. */ export type RendererSource = diff --git a/shared/test/serve-install-assets.test.ts b/shared/test/serve-install-assets.test.ts new file mode 100644 index 0000000..ff4fb63 --- /dev/null +++ b/shared/test/serve-install-assets.test.ts @@ -0,0 +1,73 @@ +/* + * Distribution-serving behavior (notes/distribution-plan.md Phase 3): the + * install entry points on scratchwork.dev are ordinary published files, so + * these two behaviors are load-bearing for `curl | bash`: + * + * - a published .sh file serves as text/plain (curl-able, readable in a + * browser, never a download prompt); + * - requesting a .md path directly round-trips the raw markdown bytes + * (the RawMarkdownServed path), so install.md is agent-readable as-is. + */ +import { describe, expect, test } from "bun:test"; +import * as HttpServerResponse from "@effect/platform/HttpServerResponse"; +import * as Effect from "effect/Effect"; +import * as Layer from "effect/Layer"; +import { SiteFileError, SiteFiles } from "../src/site/files"; +import { servePath, type SiteServeEvent } from "../src/site/serve"; + +const INSTALL_SH = "#!/bin/sh\nset -euf\necho 'fake installer'\n"; +const INSTALL_MD = "# Installing\n\n```sh\ncurl -fsSL https://scratchwork.dev/install.sh | bash\n```\n"; + +const files: Record = { + "install.sh": INSTALL_SH, + "install.md": INSTALL_MD, +}; + +const notFound = (path: string) => + new SiteFileError({ path: path as never, reason: "NotFound", message: `not found: ${path}` }); + +const siteFilesLayer = Layer.succeed(SiteFiles, { + exists: (path) => Effect.succeed(files[path as string] != null), + readText: (path) => + files[path as string] != null ? Effect.succeed(files[path as string]) : Effect.fail(notFound(path as string)), + readBytes: (path) => + files[path as string] != null + ? Effect.succeed(new TextEncoder().encode(files[path as string])) + : Effect.fail(notFound(path as string)), + fileResponse: (path, options) => + files[path as string] != null + ? Effect.succeed( + HttpServerResponse.text(files[path as string], { + contentType: options?.contentType, + headers: options?.headers, + }), + ) + : Effect.fail(notFound(path as string)), +}); + +function serve(pathname: string) { + const events: SiteServeEvent[] = []; + return Effect.runPromise( + servePath(pathname, "", { + rendererFallback: Effect.succeed(null), + onServeEvent: (event) => Effect.sync(() => void events.push(event)), + }).pipe(Effect.provide(siteFilesLayer)), + ).then((response) => ({ web: HttpServerResponse.toWeb(response), events })); +} + +describe("distribution install assets", () => { + test("a published .sh file serves as text/plain with its exact bytes", async () => { + const { web } = await serve("/install.sh"); + expect(web.status).toBe(200); + expect(web.headers.get("content-type")).toBe("text/plain; charset=utf-8"); + expect(await web.text()).toBe(INSTALL_SH); + }); + + test("a .md path requested directly round-trips raw markdown", async () => { + const { web, events } = await serve("/install.md"); + expect(web.status).toBe(200); + expect(web.headers.get("content-type")).toBe("text/markdown; charset=utf-8"); + expect(await web.text()).toBe(INSTALL_MD); + expect(events).toContainEqual({ _tag: "RawMarkdownServed", path: "install.md" }); + }); +}); diff --git a/shared/tsconfig.build.json b/shared/tsconfig.build.json new file mode 100644 index 0000000..5d7e6df --- /dev/null +++ b/shared/tsconfig.build.json @@ -0,0 +1,16 @@ +// Publish build (scripts/build-packages.ts): per-file tsc emit of ESM JS + +// declarations into dist/, mirroring src/ so the package.json exports +// patterns map 1:1. rewriteRelativeImportExtensions turns the in-repo `.ts` +// specifiers into `.js` in the emitted output so dist runs under plain Node. +{ + "extends": "./tsconfig.json", + "compilerOptions": { + "noEmit": false, + "declaration": true, + "outDir": "dist", + "rootDir": "src", + "rewriteRelativeImportExtensions": true + }, + "include": ["src/**/*.ts", "src/**/*.js"], + "exclude": ["src/**/*.test.ts"] +} From aa126b665a1ebf3e14bed9ab04a9980f8c134d47 Mon Sep 17 00:00:00 2001 From: Pete Koomen Date: Tue, 21 Jul 2026 15:38:42 -0700 Subject: [PATCH 3/7] Fix the check-install-sh deadlock: spawn the install script async MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bun.spawnSync blocks the event loop that the Bun.serve fixture needs to answer the install script's curl (which has no timeout), so the check hung forever — locally and in the GitHub CI run for this branch, which died at the workflow's 20-minute timeout. Spawning async with awaited stdout/stderr/exit keeps the fixture responsive; the check now runs in about a second. Co-Authored-By: Claude Fable 5 --- scripts/check-install-sh.ts | 27 +++++++++++++++++---------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/scripts/check-install-sh.ts b/scripts/check-install-sh.ts index c9b0211..85eff71 100644 --- a/scripts/check-install-sh.ts +++ b/scripts/check-install-sh.ts @@ -91,9 +91,11 @@ interface RunResult { } let caseNumber = 0; -function runInstall(env: Record): RunResult { +// Async spawn is load-bearing: the install script curls the Bun.serve fixture in +// this same process, and spawnSync would block the event loop that serves it. +async function runInstall(env: Record): Promise { const installDir = join(work, `install-${caseNumber++}`); - const proc = Bun.spawnSync(["sh", script], { + const proc = Bun.spawn(["sh", script], { env: { PATH: env.FAKE_UNAME_S ? `${fakeBin}:${process.env.PATH}` : process.env.PATH, HOME: work, @@ -104,7 +106,12 @@ function runInstall(env: Record): RunResult { stdout: "pipe", stderr: "pipe", }); - return { code: proc.exitCode, stdout: proc.stdout.toString(), stderr: proc.stderr.toString() }; + const [stdout, stderr, code] = await Promise.all([ + new Response(proc.stdout).text(), + new Response(proc.stderr).text(), + proc.exited, + ]); + return { code, stdout, stderr }; } function expect(name: string, condition: boolean, detail: RunResult) { @@ -114,29 +121,29 @@ function expect(name: string, condition: boolean, detail: RunResult) { } // Latest install on the real host platform. -let result = runInstall({}); +let result = await runInstall({}); expect("latest install should succeed and run the binary", result.code === 0 && result.stdout.includes("9.9.9"), result); // Pinned version install. -result = runInstall({ SCRATCHWORK_VERSION: "8.8.8" }); +result = await runInstall({ SCRATCHWORK_VERSION: "8.8.8" }); expect("pinned install should fetch v8.8.8", result.code === 0 && result.stdout.includes("8.8.8"), result); // Checksum tampering must be rejected. tamperChecksums = true; -result = runInstall({}); +result = await runInstall({}); expect("tampered checksums must fail with a mismatch", result.code !== 0 && result.stderr.includes("checksum mismatch"), result); tamperChecksums = false; // Platform mapping: unsupported OS and architecture fail with clear errors... -result = runInstall({ FAKE_UNAME_S: "FreeBSD", FAKE_UNAME_M: "x86_64" }); +result = await runInstall({ FAKE_UNAME_S: "FreeBSD", FAKE_UNAME_M: "x86_64" }); expect("unsupported OS must fail before downloading", result.code !== 0 && result.stderr.includes("unsupported operating system"), result); -result = runInstall({ FAKE_UNAME_S: "Linux", FAKE_UNAME_M: "riscv64" }); +result = await runInstall({ FAKE_UNAME_S: "Linux", FAKE_UNAME_M: "riscv64" }); expect("unsupported architecture must fail", result.code !== 0 && result.stderr.includes("unsupported architecture"), result); // ...and supported uname spellings map to the right release asset. -result = runInstall({ FAKE_UNAME_S: "Linux", FAKE_UNAME_M: "aarch64" }); +result = await runInstall({ FAKE_UNAME_S: "Linux", FAKE_UNAME_M: "aarch64" }); expect("Linux/aarch64 must map to the linux-arm64 asset", result.code === 0 && result.stdout.includes("(linux-arm64)"), result); -result = runInstall({ FAKE_UNAME_S: "Darwin", FAKE_UNAME_M: "arm64" }); +result = await runInstall({ FAKE_UNAME_S: "Darwin", FAKE_UNAME_M: "arm64" }); expect("Darwin/arm64 must map to the darwin-arm64 asset", result.code === 0 && result.stdout.includes("(darwin-arm64)"), result); server.stop(true); From 032e87c5f87d3a93e90a2d86179550b1ca38e03d Mon Sep 17 00:00:00 2001 From: Pete Koomen Date: Tue, 21 Jul 2026 16:16:01 -0700 Subject: [PATCH 4/7] Add a /release skill that drives the RELEASING.md loop MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Encodes the release procedure as an agent skill: preflight (version choice, changelog content, gh/npm auth), bump PR, merge + tag, npm publish from a clean tag checkout, homepage republish, and a container smoke test of the real install path. Irreversible steps (tag push, npm publish, homepage republish) sit behind explicit confirmation gates. RELEASING.md stays the normative doc and now points at the skill. The release/ gitignore rule is now anchored to the repo root — unanchored it also swallowed .claude/skills/release/. Co-Authored-By: Claude Fable 5 --- .claude/skills/release/SKILL.md | 86 +++++++++++++++++++++++++++++++++ .gitignore | 4 +- RELEASING.md | 4 +- 3 files changed, 92 insertions(+), 2 deletions(-) create mode 100644 .claude/skills/release/SKILL.md diff --git a/.claude/skills/release/SKILL.md b/.claude/skills/release/SKILL.md new file mode 100644 index 0000000..6c115cf --- /dev/null +++ b/.claude/skills/release/SKILL.md @@ -0,0 +1,86 @@ +--- +name: release +description: Cut a Scratchwork release end to end — version bump PR, tag, GitHub Release, npm publish, homepage republish, install smoke test. Use when asked to release, ship, or cut vX.Y.Z. +--- + +# Release + +Drive the release loop in `RELEASING.md` (the normative doc — read it first; if it +and this skill disagree, follow it and fix the drift in the same PR). One lockstep +version for the whole repo, one tag per release. + +**Confirmation gates:** pushing the tag, publishing to npm, and republishing the +homepage are public and effectively irreversible. Pause and confirm with the user +before each one, unless they explicitly asked for an unattended release up front. +Everything else (branches, PRs, dry runs) proceeds without asking. + +## Procedure + +### 1. Preflight + +- Start from a clean, up-to-date `main` (`git status`, `git pull`). +- Determine the version. If the user didn't give one, propose it from the + `## Unreleased` section of `CHANGELOG.md` (breaking → minor while pre-1.0, + otherwise patch) and confirm. +- `## Unreleased` must have real content; if it's empty, stop and ask what this + release is. +- `gh auth status` and `npm whoami` must both succeed. Fail fast, not at step 5. + +### 2. Bump PR + +- Branch: `git checkout -b release-vX.Y.Z`. +- `bun scripts/set-version.ts X.Y.Z` (stamps root + every workspace). +- In `CHANGELOG.md`: retitle `## Unreleased` to `## vX.Y.Z` and add a fresh empty + `## Unreleased` above it. +- `bun run ci` locally (enforces lockstep; requires Docker running for the e2e + LocalStack lane). +- Commit, push, `gh pr create`. Watch CI with `gh pr checks --watch`. + +### 3. Merge and tag + +- **Gate:** confirm the user is ready to ship, then merge the PR + (`gh pr merge --merge`), pull `main`, and verify HEAD carries the bump + (`package.json` version is X.Y.Z). +- **Gate passed above covers the tag too:** `git tag vX.Y.Z && git push origin vX.Y.Z`. +- The tag triggers `.github/workflows/release.yml`. Watch it + (`gh run watch --exit-status`); on success verify the Release exists with the four + `*.tar.gz` archives + `checksums.txt`: `gh release view vX.Y.Z`. + +### 4. npm publish + +- Publish from a clean checkout of the tag, never the working tree. Use a scratch + clone/worktree checked out at `vX.Y.Z` (the script itself refuses dirty trees and + untagged HEADs). +- Dry-run first: `bun scripts/publish-packages.ts --dry-run` — review the five + package names and the version. +- **Gate:** confirm, then `bun scripts/publish-packages.ts`. If a mid-sequence + publish fails, report which packages went live and which didn't; already-published + versions cannot be re-pushed, so the fix is forward (patch release), not retry. + +### 5. Homepage republish + +- **Gate:** confirm, then `scratchwork publish docs --project www` with production + credentials, so `https://scratchwork.dev/install.sh` serves the current docs. + This needs an interactive `scratchwork login` against production — if not logged + in, hand this step to the user rather than skipping it silently. + +### 6. Smoke test + +Run the real install path in a clean container (no Scratchwork, no repo): + +```sh +docker run --rm debian:stable-slim sh -c ' + apt-get update -qq && apt-get install -y -qq curl ca-certificates >/dev/null && + curl -fsSL https://scratchwork.dev/install.sh | bash && + "$HOME/.local/bin/scratchwork" --version' +``` + +Pass = it prints `X.Y.Z`. If the homepage step was deferred, smoke against the +GitHub Release directly instead by setting `SCRATCHWORK_DOWNLOAD_BASE` to +`https://github.com/scratch/scratchwork/releases` inside the container. + +## Report + +End with: version shipped, PR and Release URLs, the five npm package versions, +whether the homepage was republished, and the smoke-test output. List any step that +was skipped or handed to the user — never imply an unfinished release is done. diff --git a/.gitignore b/.gitignore index d2625fb..aada4e1 100644 --- a/.gitignore +++ b/.gitignore @@ -4,7 +4,9 @@ dist/ # Release staging written by scripts/package-release.ts and # scripts/build-packages.ts; assets upload to GitHub Releases / npm, never git. -release/ +# Anchored to the repo root so directories like .claude/skills/release/ stay +# trackable. +/release/ # Local environment files. Keep .env.example files committed. **/.env diff --git a/RELEASING.md b/RELEASING.md index af89bd2..8fe0b90 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -1,7 +1,9 @@ # Releasing Scratchwork One version for the whole repo (CLI binaries + npm packages), one tag per -release. Short enough to actually follow: +release. Short enough to actually follow — or ask an agent to run the +`/release` skill (`.claude/skills/release/SKILL.md`), which drives this loop +with confirmation gates at the irreversible steps: 1. **Bump:** `bun scripts/set-version.ts X.Y.Z` (stamps root + every workspace; `bun run ci` enforces lockstep). From 0f48022dd982cefdc4287b609efa916f59816ad5 Mon Sep 17 00:00:00 2001 From: Pete Koomen Date: Tue, 21 Jul 2026 16:43:24 -0700 Subject: [PATCH 5/7] Move the homepage project from docs/ to scratchwork.dev/ The directory was already the published scratchwork.dev site: the landing page (index.md with the logo and project description), its components, and the install.sh / install.md entry points. Naming it after the domain makes that role explicit. References updated across README, renderer README, RELEASING.md, the release skill (publish command now scratchwork.dev --project www --public), AGENTS.md's outside-the-gate list, and the install-sh check's script path. Co-Authored-By: Claude Fable 5 --- .claude/skills/release/SKILL.md | 4 ++-- AGENTS.md | 7 +++++-- README.md | 4 ++-- RELEASING.md | 6 +++--- renderer/README.md | 6 +++--- {docs => scratchwork.dev}/components/Counter.js | 0 {docs => scratchwork.dev}/components/Highlight.js | 0 .../components/MadeWithScratchwork.js | 0 {docs => scratchwork.dev}/index.md | 0 {docs => scratchwork.dev}/install.md | 0 {docs => scratchwork.dev}/install.sh | 0 {docs => scratchwork.dev}/scratchwork-logo.svg | 0 scripts/check-install-sh.ts | 7 ++++--- scripts/release-targets.ts | 2 +- 14 files changed, 20 insertions(+), 16 deletions(-) rename {docs => scratchwork.dev}/components/Counter.js (100%) rename {docs => scratchwork.dev}/components/Highlight.js (100%) rename {docs => scratchwork.dev}/components/MadeWithScratchwork.js (100%) rename {docs => scratchwork.dev}/index.md (100%) rename {docs => scratchwork.dev}/install.md (100%) rename {docs => scratchwork.dev}/install.sh (100%) rename {docs => scratchwork.dev}/scratchwork-logo.svg (100%) diff --git a/.claude/skills/release/SKILL.md b/.claude/skills/release/SKILL.md index 6c115cf..47968e1 100644 --- a/.claude/skills/release/SKILL.md +++ b/.claude/skills/release/SKILL.md @@ -59,8 +59,8 @@ Everything else (branches, PRs, dry runs) proceeds without asking. ### 5. Homepage republish -- **Gate:** confirm, then `scratchwork publish docs --project www` with production - credentials, so `https://scratchwork.dev/install.sh` serves the current docs. +- **Gate:** confirm, then `scratchwork publish scratchwork.dev --project www --public` + with production credentials, so `https://scratchwork.dev/install.sh` serves the current content. This needs an interactive `scratchwork login` against production — if not logged in, hand this step to the user rather than skipping it silently. diff --git a/AGENTS.md b/AGENTS.md index f52d12e..676eabd 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -26,8 +26,11 @@ Package names are `@scratchwork/` for everything under `server/` and `deplo - `e2e/` — full-loop publish e2e: real server (local-dev, miniflare, LocalStack) driven by the real CLI, hermetic OAuth provider standing in for Google. -`examples/` and `notes/` are deliberately outside the gate: examples are user-facing -sample content, notes are working documents. Nothing in them is imported by shipped code. +`examples/`, `notes/`, and `scratchwork.dev/` are deliberately outside the gate: +examples are user-facing sample content, notes are working documents, and +`scratchwork.dev/` is the published homepage project (landing page plus the +`install.sh` / `install.md` entry points — the install script alone is exercised by +`scripts/check-install-sh.ts` in the gate). Nothing in them is imported by shipped code. ## Build and test diff --git a/README.md b/README.md index 19ce861..61e6bf3 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,5 @@

- Scratchwork + Scratchwork

Scratchwork is a local development tool for static HTML and Markdown artifacts created by your coding agent. @@ -58,7 +58,7 @@ changing to another directory. ## Working with Markdown -Scratchwork renders Markdown with an embedded default renderer, and Markdown files can reference React components from nearby component files. See [`docs/index.md`](docs/index.md) for live examples. +Scratchwork renders Markdown with an embedded default renderer, and Markdown files can reference React components from nearby component files. See [`scratchwork.dev/index.md`](scratchwork.dev/index.md) for live examples. To use the docs page as a starting point for your own project, run: diff --git a/RELEASING.md b/RELEASING.md index 8fe0b90..0584367 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -20,9 +20,9 @@ with confirmation gates at the irreversible steps: Publishes `@scratchwork/shared`, `@scratchwork/server-core`, and the three `@scratchwork/server-deploy-*` packages in dependency order. 6. **Homepage:** republish the install entry points so - `https://scratchwork.dev/install.sh` serves the current docs: - `scratchwork publish docs --project www` (with the production server - credentials). Automating this in release.yml is a follow-up. + `https://scratchwork.dev/install.sh` serves the current content: + `scratchwork publish scratchwork.dev --project www --public` (with the + production server credentials). Automating this in release.yml is a follow-up. 7. **Smoke:** on a machine (or clean container) without Scratchwork: `curl -fsSL https://scratchwork.dev/install.sh | bash` and check `scratchwork --version` prints X.Y.Z. diff --git a/renderer/README.md b/renderer/README.md index 9d68be4..a9cd4e2 100644 --- a/renderer/README.md +++ b/renderer/README.md @@ -45,9 +45,9 @@ name → component map; consulted **before** lazy-loading `components/*.js`). The renderer embeds **no** images — no favicon, no logo, no Scratchwork-specific components. Branding lives with the project content instead: -- The sample project (`docs/`) ships `scratchwork-logo.svg` (figure + wordmark) +- The sample project (`scratchwork.dev/`) ships `scratchwork-logo.svg` (figure + wordmark) and uses it directly in `index.md`, plus a `MadeWithScratchwork` component - under `docs/components/`. + under `scratchwork.dev/components/`. - The `scratchwork dev` server serves the Scratchwork figure mark as the **default favicon** when a project ships none of its own (see `../cli/src/dev/server.ts` and `../cli/assets/figure.svg`). @@ -125,7 +125,7 @@ statements (not full MDX), footnotes, and definition lists. There is no CSS framework. Rendered markdown is styled by `prose.css` (scoped to `.scratchwork-prose`). Components style themselves with inline `style` or a small -scoped `\n \n \n \n \n\n
\n\n \n \n \n\n"); export default defaultRendererHtml; From 1c93dd881513c3913a7a91dbcd2813a33d56336b Mon Sep 17 00:00:00 2001 From: Pete Koomen Date: Tue, 21 Jul 2026 17:37:32 -0700 Subject: [PATCH 7/7] Wrap the landing page quick-start prompt to avoid horizontal scroll Co-Authored-By: Claude Fable 5 --- scratchwork.dev/www/index.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/scratchwork.dev/www/index.md b/scratchwork.dev/www/index.md index 8f7e964..1d687c8 100644 --- a/scratchwork.dev/www/index.md +++ b/scratchwork.dev/www/index.md @@ -20,7 +20,9 @@ Publish your work publicly to share it with the world, or privately to share it Just ask your agent: ``` -Create a simple "hello world" website and publish it to scratchwork.dev. Give everyone with an email @example.com permission to read it. +Create a simple "hello world" website and publish it to +scratchwork.dev. Give everyone with an email @example.com +permission to read it. ``` ## Slow start