Skip to content

Add Buildkite skeleton mirroring GHA preflight#312

Draft
mokagio wants to merge 40 commits into
trunkfrom
mokagio/buildkite-ci-skeleton
Draft

Add Buildkite skeleton mirroring GHA preflight#312
mokagio wants to merge 40 commits into
trunkfrom
mokagio/buildkite-ci-skeleton

Conversation

@mokagio
Copy link
Copy Markdown

@mokagio mokagio commented May 18, 2026

image

Note

The description below is out of date

Rationale

First step of the GHA → Buildkite CI migration described in docs/buildkite-migration-plan.md.

Lands the smallest useful Buildkite pipeline: a mirror of the cow-changed-preflight job from .github/workflows/ci.yml.
Picked it because it's the simplest job in our GHA matrix (Linux-only, no caching, no artifacts, one script), which makes it the cleanest candidate to prove the BK ↔ GHA duplication pattern works before porting heavier jobs.

Intentional tradeoffs

  • .xcode-version is included even though this step is Linux-only.
    The a8c shared-pipeline-vars convention reads IMAGE_ID from it,
    and macOS steps added in later migration phases will need it.
    Better to land the full scaffolding now than restructure later.
  • The a8c-ci-toolkit plugin is pinned in shared-pipeline-vars even though this Linux step doesn't strictly need it.
    Same reasoning — keep the scaffolding shape consistent with other a8c repos.
  • PR base discovery uses BUILDKITE_PULL_REQUEST_BASE_BRANCH in place of GHA's github.base_ref.
    Otherwise the command script is a verbatim port of the GHA step.

Gotchas

  • The GHA ci.yml workflow does not run on this PR.
    Its paths: filter only matches crates/**, scripts/** (excluding dev/), runtime/**, etc., and does not include .buildkite/** or .xcode-version.
    Comparing the BK output against GHA for this PR isn't possible;
    parity verification has to come from a follow-up PR that touches a tracked path,
    or by re-running the BK preflight against a trunk push.
  • Buildkite pipeline is already registered for Automattic/forkpress (build [codex] Cover incomplete generated media metadata #69 was the first build kicked off by this PR — the pipeline has 68 prior builds).
    I had assumed Phase 0 question Easy way to run 10 agents on my site. #2 from the migration plan was still open;
    it's already answered.

How to test

  • Local syntax checks (already done):
    • bash -n .buildkite/commands/cow-changed-preflight.sh
    • bash -n .buildkite/shared-pipeline-vars
    • Sourcing shared-pipeline-vars from the worktree root resolves to IMAGE_ID=xcode-26.1 and CI_TOOLKIT_PLUGIN=automattic/a8c-ci-toolkit#6.1.0.
  • End-to-end: the Buildkite build linked from the GitHub check (Buildkite / COW changed preflight) runs the same scripts/dev/cow-changed-test-plan.sh the GHA step runs, against origin/trunk as the base.

Posted by Claude (Opus 4.7) on behalf of @mokagio with approval.

First step of the GHA → Buildkite CI migration laid out in
`docs/buildkite-migration-plan.md`.

Mirrors the simplest job in `.github/workflows/ci.yml`,
`cow-changed-preflight`,
so the two pipelines can run in parallel and we can verify Buildkite's
output matches GHA's before porting heavier jobs.

The shared a8c scaffolding (`.xcode-version`,
`.buildkite/shared-pipeline-vars` exporting `IMAGE_ID` and
`CI_TOOLKIT_PLUGIN`,
`automattic/a8c-ci-toolkit` plugin)
is included now so subsequent steps can layer on without restructuring.
`.xcode-version` is unused by this Linux-only step,
but is kept because `shared-pipeline-vars` resolves `IMAGE_ID` from it
and macOS steps added in later phases will need it.

The command script reproduces the GHA step verbatim,
except for how the PR base ref is discovered:
`BUILDKITE_PULL_REQUEST_BASE_BRANCH` in place of `github.base_ref`.

---

Generated with the help of Claude Code, https://claude.com/claude-code

Co-Authored-By: Claude Code Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 18, 2026 05:56
@mokagio mokagio self-assigned this May 18, 2026
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds initial Buildkite CI scaffolding as the first step of the GHA → Buildkite migration, mirroring the existing GitHub Actions cow-changed-preflight job logic.

Changes:

  • Add a minimal Buildkite pipeline with a single “COW changed preflight” step.
  • Add a Buildkite command script that installs PHP, determines PR base, and runs scripts/dev/cow-changed-test-plan.sh.
  • Add shared pipeline variables (including IMAGE_ID derived from .xcode-version) for Buildkite pipeline interpolation.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
.xcode-version Introduces Xcode version value used to derive Buildkite IMAGE_ID.
.buildkite/shared-pipeline-vars Exports IMAGE_ID and pinned CI toolkit plugin for pipeline interpolation.
.buildkite/pipeline.yml Defines the initial Buildkite pipeline step mirroring GHA preflight.
.buildkite/commands/cow-changed-preflight.sh Implements the Buildkite-side preflight command sequence (PHP install, base fetch, run plan).
Comments suppressed due to low confidence (1)

.xcode-version:2

  • .xcode-version currently has a trailing empty line, which causes $(sed ... .xcode-version) (used in .buildkite/shared-pipeline-vars) to expand to "26.1 " with a trailing space via command-substitution newline collapsing. That produces an IMAGE_ID with whitespace (e.g., xcode-26.1 ) and can break agent/image selection. Remove the extra blank line so the file contains a single version line.
26.1


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +7 to +8
XCODE_VERSION=$(sed -E 's/^~> ?//' .xcode-version)

Build #69 hung at `sudo apt-get update`'s password prompt for 5 minutes
before timing out and failing the job:
the a8c BK Linux agent runs as `buildkite-agent` without passwordless
sudo,
unlike the GHA `ubuntu-24.04` runner that the original step was written
for.

For this PR specifically the change set is `.buildkite/**` and
`.xcode-version`,
so the preflight plan is empty
(`git diff --check` only, confirmed by a local `--list` run against
`origin/trunk`)
and PHP is never invoked.
Detect availability and continue when it isn't there;
land a proper PHP toolchain via the Docker plugin in a follow-up so the
step can cover the full set of changes.

---

Generated with the help of Claude Code, https://claude.com/claude-code

Co-Authored-By: Claude Code Opus 4.7 (1M context) <noreply@anthropic.com>
@mokagio mokagio marked this pull request as draft May 18, 2026 06:07
mokagio and others added 23 commits May 18, 2026 16:13
Smallest end-to-end Linux build the Buildkite pipeline can run.
Goal: confirm push → BK Linux agent → `cargo` succeeds before adding
the heavier path
(static PHP runtime bundle,
musl target,
caching,
artifact upload).

Excludes `forkpress-cli` from the build because its `build.rs` requires
the static PHP runtime bundle produced by `scripts/build-dist.sh`,
which would add 3-5 minutes of compilation and pull in toolchain
dependencies that aren't needed for a skeleton.
The non-CLI workspace members compile against the stock `rust:1.85`
image with no extra system packages.

Reuses the `docker#v5.13.0` plugin pattern from `Automattic/release-toolkit`
(`DOCKER_PLUGIN` and `DOCKER_RUST_IMAGE` exported from
`shared-pipeline-vars`),
which sidesteps the no-passwordless-sudo constraint on the a8c BK
`default` Linux queue
(observed on build #69 of this same PR).

---

Generated with the help of Claude Code, https://claude.com/claude-code

Co-Authored-By: Claude Code Opus 4.7 (1M context) <noreply@anthropic.com>
Build #72 failed with `redb@4.1.0 requires rustc 1.89`;
the previous pin (1.85) was the Rust version that introduced edition
2024 but is now well below the floor of our transitive deps.

1.95 is the current Rust stable and is what GHA's
`dtolnay/rust-toolchain@stable` resolves to in `ci.yml`,
so the two pipelines build with the same compiler.

---

Generated with the help of Claude Code, https://claude.com/claude-code

Co-Authored-By: Claude Code Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a `linux-tests` step mirroring the first two cargo-test invocations
from GHA's `linux-cow-e2e` job
(`cargo test --workspace --exclude forkpress-cli` and
`cargo test -p forkpress-core --features dev-experiments`),
and makes `linux-build` `depends_on` it
so the build only runs when tests pass.

Establishes the per-platform `<platform>-tests` → `<platform>-build`
pattern;
mac and windows variants will follow the same shape when their queues
get wired up.
The `forkpress-cli` test invocations
(which need `FORKPRESS_RUNTIME_BUNDLE=/dev/null` to skip the static PHP
runtime bundle)
land in a follow-up once cargo caching is in place,
since each cold compile pulls down ~150 crates.

---

Generated with the help of Claude Code, https://claude.com/claude-code

Co-Authored-By: Claude Code Opus 4.7 (1M context) <noreply@anthropic.com>
`forkpress-git` integration tests shell out to `git init`,
which the `-slim` variant of the Rust image doesn't ship
(it extends `debian:bookworm-slim`, which omits VCS and dev tools to
minimise size).
Build #74 surfaced this as `No such file or directory (os error 2)`
from `git init --bare`.

The non-slim `rust:1.95-bookworm` extends `buildpack-deps:bookworm`,
which bakes in git, curl, build-essential, and the other tools the
forkpress test suite assumes are present on a Linux dev host.
Trades ~300 MB of image size for not having to hand-install missing
binaries on every step.

---

Generated with the help of Claude Code, https://claude.com/claude-code

Co-Authored-By: Claude Code Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors the remaining cargo-test invocations from GHA `linux-cow-e2e`
that don't need the static PHP runtime bundle,
plus `make test-release` (a meta-preflight that `bash -n`'s
`scripts/build-dist.sh` and `make -n`'s the release targets).
`FORKPRESS_RUNTIME_BUNDLE=/dev/null` tells `forkpress-cli`'s `build.rs`
to skip the 3-5 min static-PHP compile and stamp the binary with an
"external" runtime ID;
tests still cover the CLI's Rust surface,
just without a real embedded runtime archive.

Still outstanding for full Linux parity:
`make test-cow-fast` (needs PHP + SQLite in the image),
the production `cargo build --release` against the static-PHP dist
bundle on the musl target,
and the COW / CAS e2e suites that consume the built binary.
Each lands in its own follow-up.

---

Generated with the help of Claude Code, https://claude.com/claude-code

Co-Authored-By: Claude Code Opus 4.7 (1M context) <noreply@anthropic.com>
Installs `php-cli` and `php-sqlite3` at the top of the step (we run as
root inside the docker container, so apt works without sudo),
then runs `make test-cow-fast` which exercises 16 PHP scripts under
`tests/cow/` against SQLite-backed COW fixtures.

Last non-build-dist chunk of GHA's `linux-cow-e2e`.
Still outstanding for full Linux parity:
the production `cargo build --release` against the static-PHP dist
bundle on the musl target,
and the COW / CAS e2e suites that consume the built binary.

`apt-get update` adds ~15s per build,
acceptable for the skeleton.
A custom image with PHP baked in is the natural follow-up once we add
caching.

---

Generated with the help of Claude Code, https://claude.com/claude-code

Co-Authored-By: Claude Code Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the toy `cargo build --workspace --exclude forkpress-cli`
walking skeleton with the production path GHA `linux-cow-e2e` runs:
install heavy build deps,
add the musl Rust target,
run `scripts/build-dist.sh` to compile the static PHP runtime bundle
(3-5 min),
`cargo build --release` the `forkpress` CLI against the bundle for
`x86_64-unknown-linux-musl`,
then `tests/cow/e2e.sh` against the produced binary.

Caching is intentionally absent for now,
per the migration plan's "exclude caching" scope;
expect each build to pay the full static-PHP compile cost until a
follow-up step lands a `.build` cache.

---

Generated with the help of Claude Code, https://claude.com/claude-code

Co-Authored-By: Claude Code Opus 4.7 (1M context) <noreply@anthropic.com>
`static-php-cli` (vendored by `scripts/build-dist.sh`) refuses to run
on PHP < 8.4
(parse error on a newer-syntax `ConsoleApplication.php`).
Debian Bookworm only ships PHP 8.2;
Debian Trixie ships PHP 8.4,
so `rust:1.95-trixie` is the smallest change that unblocks the static
PHP runtime build.

`make test-cow-fast` and the cargo-test paths still run on the same
image,
and both stages of the Linux pipeline keep using the matching apt
package names
(`php-cli`, `php-sqlite3`, `automake`, ...) which are unchanged between
Bookworm and Trixie.

---

Generated with the help of Claude Code, https://claude.com/claude-code

Co-Authored-By: Claude Code Opus 4.7 (1M context) <noreply@anthropic.com>
`static-php-cli doctor` refused to proceed without them and dropped to
an interactive "Do you want to fix it?" prompt,
which the BK agent can't answer.
GHA's `ubuntu-24.04` runner has these baked in;
`rust:1.95-trixie` doesn't.

---

Generated with the help of Claude Code, https://claude.com/claude-code

Co-Authored-By: Claude Code Opus 4.7 (1M context) <noreply@anthropic.com>
`./bin/spc doctor` drops to an interactive
"Do you want to fix it?" prompt when it finds missing tools
(observed for `musl-wrapper` on the BK `rust:1.95-trixie` image).
laravel/prompts can't be answered without a TTY,
so CI hangs and is eventually killed.

`--auto-fix` is a documented spc option
(`InputOption::VALUE_OPTIONAL`)
that installs the missing items non-interactively.
GHA gets away without it because `actions/cache@v4` restores a populated
`.build/` directory and the doctor never needs to fix anything;
on a cold cache GHA would hit the same hang.

---

Generated with the help of Claude Code, https://claude.com/claude-code

Co-Authored-By: Claude Code Opus 4.7 (1M context) <noreply@anthropic.com>
Revert the `--auto-fix` shortcut on `scripts/build-dist.sh`
(`tests/release/build-dist-preflight.sh` explicitly forbids it —
the project's policy is that the operator must supply prereqs so the
release artifact's tooling is auditable).

Instead,
in `linux-build.sh`,
run the same install steps `LinuxMuslCheck`'s fix-items would run:
build musl-1.2.5 from source into `/usr/local/musl/`,
and unpack the static-php.dev prebuilt `x86_64-musl-toolchain.tgz`
into the same prefix.
`doctor` then finds both checks already satisfied and proceeds
without prompting.
Skipping spc's CVE-2025-26519 patches on musl since the musl install
here only acts as a build-time wrapper for cross-compilation,
not a shipped runtime.

On GHA these directories are restored from the `.build/` cache;
we don't cache yet (per migration scope),
so we pay the install cost on every build.

---

Generated with the help of Claude Code, https://claude.com/claude-code

Co-Authored-By: Claude Code Opus 4.7 (1M context) <noreply@anthropic.com>
The manual musl-wrapper + musl-cross-make install only covered two of
spc's `doctor` checks;
once those passed,
the next pkg-config check kicked off another interactive "fix it?"
prompt the BK agent can't answer.
Trying to pre-install each item individually is a long whack-a-mole —
spc has many platform-conditional checks.

Switch to running spc's `doctor` ourselves with `--auto-fix=yes` before
`scripts/build-dist.sh` runs.
The script-level policy
(`tests/release/build-dist-preflight.sh` forbids `--auto-fix` inside
`build-dist.sh`)
holds because `build-dist.sh` itself stays untouched;
its `doctor` run is a no-op once we've already installed everything.

Clone spc to the same `BUILD_DIR/static-php-cli` path `build-dist.sh`
uses,
pin to the same `SPC_REF`,
`composer install --no-dev`,
then `./bin/spc doctor --auto-fix`.

---

Generated with the help of Claude Code, https://claude.com/claude-code

Co-Authored-By: Claude Code Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors the tail of GHA `linux-cow-e2e`:
build a second static-PHP runtime bundle with the dev profile,
`cargo build --release` `forkpress-dev` with the `dev-experiments`
feature,
then run `experiments/cas/tests/e2e.sh` against the produced binary.

Adds another ~5-10 min uncached static-PHP compile per build.
Cache layer is out of scope per the migration plan.

This closes 1:1 functional parity with `linux-cow-e2e`'s job
(modulo caching);
remaining `ci.yml` work is the macOS matrix and `windows-cow-check`.

---

Generated with the help of Claude Code, https://claude.com/claude-code

Co-Authored-By: Claude Code Opus 4.7 (1M context) <noreply@anthropic.com>
BK build #94 preflight failed with `permission denied` while cleaning
the checkout dir:
the docker plugin runs our scripts as root in the container,
so every file `cargo` / `static-php-cli` writes through the mounted
workspace ends up root-owned on the host.
The agent's later `git clean` runs as `buildkite-agent` and can't
remove root-owned artifacts,
so checkout cleanup loops forever and subsequent builds on that agent
break.

Trap on EXIT to chown the workspace back to the dir's host owner
before the container goes away.

---

Generated with the help of Claude Code, https://claude.com/claude-code

Co-Authored-By: Claude Code Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors GHA `mac-cow-e2e` matrix entry for `aarch64-apple-darwin`:
single BK step running on the `mac` queue
(Apple Silicon VM under the `xcode-26.1` image),
no Docker plugin since the mac queue runs native commands.
Step content reproduces the full GHA job in order —
rustup install,
cargo unit tests (workspace excl. CLI; forkpress-core dev; CLI with
`FORKPRESS_RUNTIME_BUNDLE=/dev/null`; forkpress-dev with same),
`make test-release`,
`scripts/dev/install-macos-runtime-tools.sh` for brew deps,
`make test-cow-fast`,
`scripts/build-dist.sh`,
`cargo build --release forkpress`,
and `tests/cow/e2e.sh` with `FORKPRESS_FORCE_MACOS_APFS_SPARSEBUNDLE=1`.

Also adds `depends_on: cow-changed-preflight` to `linux-tests`
so the GHA `needs: cow-changed-preflight` gate is preserved across
all per-platform jobs.

Intel mac (`x86_64-apple-darwin` on `macos-15-intel` in GHA) is left
out — no Intel queue surfaced in the a8c BK queue list,
so it's a Phase-0 infra question.
TODO comment in `pipeline.yml` flags this.

---

Generated with the help of Claude Code, https://claude.com/claude-code

Co-Authored-By: Claude Code Opus 4.7 (1M context) <noreply@anthropic.com>
Build #100 cascaded `waiting_failed` to every job because preflight
failed (dirty agent state from previous builds before the chown fix
landed) and every other step depended on it.
Removing the `depends_on: cow-changed-preflight` so independent jobs
can run on whichever agent BK assigns them;
that lets jobs landing on clean agents succeed and start cleaning up
behind themselves via the chown trap,
instead of being held hostage to preflight's bad luck.

GHA's `needs: cow-changed-preflight` semantics will need a different
treatment later
(maybe a soft gate, or a separate orchestrator step) —
worth revisiting once the agent pollution settles.

---

Generated with the help of Claude Code, https://claude.com/claude-code

Co-Authored-By: Claude Code Opus 4.7 (1M context) <noreply@anthropic.com>
`xcode-26.1` isn't in the a8c BK mac VM image registry
(BK error `Unable to find remote image: xcode-26.1`),
so the mac aarch64 step failed at agent startup.

`xcode-26.4.1` is the image the `Automattic/workspace` pipeline uses
(per engram memory dated 2026-05-09);
it's a known-good a8c BK mac image.

---

Generated with the help of Claude Code, https://claude.com/claude-code

Co-Authored-By: Claude Code Opus 4.7 (1M context) <noreply@anthropic.com>
Build #150 hung at the interactive `pkg-config is not installed`
prompt for both `Linux build` and `Mac aarch64 cow-e2e`:

- Linux pre-ran `doctor --auto-fix` only inside `.build/$TARGET` (the
  prod runtime path), but the subsequent dev runtime build uses its own
  `.build/$TARGET-dev` checkout with a separate `pkgroot/`.
  spc's `findPkgConfig` only scans `PKG_ROOT_PATH/bin/`, never `$PATH`,
  so the dev `doctor` saw no pkg-config and waited for input.
- Mac never pre-ran `doctor --auto-fix` at all; even with brew's
  `pkg-config` in `$PATH`, spc's check is path-bound and misses it.

Loop the pre-run over both dist names on Linux,
and add a single pre-run on Mac before `build-dist.sh`.

---

Generated with the help of Claude Code, https://claude.com/claude-code

Co-Authored-By: Claude Code Opus 4.7 (1M context) <noreply@anthropic.com>
`tests/cow/e2e.sh` invokes `node` at line 371 to extract a WP admin
nonce from inline HTML.
GHA's `macos-14` runner ships node out of the box;
the BK mac VM (`xcode-26.4.1` image) doesn't.
Build #166 reached e2e (after all cargo tests, brew tools install, PHP
runtime build, cargo release, and APFS sparsebundle setup) only to
trip on `node: command not found` and bail.

---

Generated with the help of Claude Code, https://claude.com/claude-code

Co-Authored-By: Claude Code Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors GHA `windows-cow-check` end-to-end:
install Rust via `rustup-init.exe`,
add the MSVC target,
run `cargo test --workspace --exclude forkpress-cli`,
run `cargo test -p forkpress-cli --bin forkpress` with an empty
`FORKPRESS_RUNTIME_BUNDLE` file,
parse-check each of the five PowerShell scripts under
`scripts/windows/`,
and run `tests/windows/installer-error-surface.ps1`.

Code signing is intentionally out of scope per the migration plan;
the `sign.ps1` script is still parse-checked here,
which mirrors GHA exactly.

Native PowerShell execution on the BK `windows` queue —
no Docker plugin
(Windows BK agents don't run Linux containers).

---

Generated with the help of Claude Code, https://claude.com/claude-code

Co-Authored-By: Claude Code Opus 4.7 (1M context) <noreply@anthropic.com>
Build #172 failed with `rustup: The term ... is not recognized` after
the `cargo` `Get-Command` check returned truthy:
the agent has some `cargo.exe` on PATH (possibly chocolatey) without a
matching `rustup`,
so the install branch was skipped and the immediate `rustup target add`
line threw.

Drop the conditional and let rustup-init run every time;
it's a no-op when the toolchain is already in-place,
and the explicit `%USERPROFILE%\.cargo\bin` prepend ensures the
rustup-managed `cargo.exe` wins over any prior install on PATH.

---

Generated with the help of Claude Code, https://claude.com/claude-code

Co-Authored-By: Claude Code Opus 4.7 (1M context) <noreply@anthropic.com>
The default `-fxdq` cleanup (`-x` removes ignored paths) trips on
`.build/` content left root-owned by previous docker-plugin runs on the
same `default`-queue agent, which fails the checkout phase before the
step's script — and its chown-back trap — gets a chance to run.

Dropping `-x` from `BUILDKITE_GIT_CLEAN_FLAGS` keeps cleanup focused on
files git tracks; ignored content under `.build/` is left in place,
which is harmless for cargo workspace tests and gets overwritten by the
docker container in `linux-build`.

---

Generated with the help of Claude Code, https://claude.ai/code

Co-Authored-By: Claude Code Opus 4.7 <noreply@anthropic.com>
GHA `mac-cow-e2e` stops at the COW e2e and never surfaces the built
binary anywhere visible — release-publish is the only path that emits a
mac binary. Add a `buildkite-agent artifact upload` after the
`cargo build --release` so each CI run produces a downloadable mac
forkpress binary as a smoke artifact (no signing/notarisation; that
stays in the release pipeline).

---

Generated with the help of Claude Code, https://claude.ai/code

Co-Authored-By: Claude Code Opus 4.7 <noreply@anthropic.com>
mokagio and others added 10 commits May 19, 2026 09:20
GHA `windows-cow-check` runs cargo test but never produces a release
binary; `forkpress.exe` is only built inside release-publish. Add a
`cargo build --release` + `buildkite-agent artifact upload` after the
existing checks so every CI run surfaces a Windows binary as a
downloadable smoke artifact.

Build uses the same empty `FORKPRESS_RUNTIME_BUNDLE` shim as the test
step — static PHP is not built on Windows in CI, and release-publish
embeds the prebuilt bundle separately. The artifact is a sanity check
that the Windows build path stays green, not a shippable artifact.

---

Generated with the help of Claude Code, https://claude.ai/code

Co-Authored-By: Claude Code Opus 4.7 <noreply@anthropic.com>
Mirrors the `build` job in `.github/workflows/docs.yml`: npm ci plus
npm run validate (astro check, node --test, astro build), running on
`public.ecr.aws/docker/library/node:24-bookworm` inside the BK docker
plugin. Built site uploaded as a BK artifact for preview.

The GHA `deploy` job stays in GHA — it's GitHub-Pages-specific and
push-to-trunk only.

GHA workflow filters on docs paths; BK has no native path filter so we
run on every build. Cost is ~60s; cheap enough to skip the filter for
now.

---

Generated with the help of Claude Code, https://claude.ai/code

Co-Authored-By: Claude Code Opus 4.7 <noreply@anthropic.com>
The docs-validate step ends with `buildkite-agent artifact upload` to
surface the built docs site. The docker plugin doesn't mount the agent
binary by default, so the command fails with `command not found`.
Set `mount-buildkite-agent: true` so the artifact upload resolves.

---

Generated with the help of Claude Code, https://claude.ai/code

Co-Authored-By: Claude Code Opus 4.7 <noreply@anthropic.com>
static-php-cli queries the GitHub releases API for every dependency.
Unauthenticated requests share the BK agent's NAT IP rate limit (60/hr)
and the API starts returning 403 after a handful of consecutive builds.
spc treats that as a download failure and falls over.

The BK agent already loads `GITHUB_TOKEN` from a8c secrets in the
environment hook, but the docker plugin's `propagate-environment: true`
only forwards `BUILDKITE_*` variables. List `GITHUB_TOKEN` explicitly in
the docker plugin's `environment` array so the container inherits it
and spc hits the 5000/hr authenticated quota.

---

Generated with the help of Claude Code, https://claude.ai/code

Co-Authored-By: Claude Code Opus 4.7 <noreply@anthropic.com>
Add `artifact_paths` to the `linux-build` step so the production and
dev musl binaries surface in the BK UI alongside the mac and Windows
artifacts. YAML-level `artifact_paths` runs regardless of step status,
so a failed e2e still leaves the binary downloadable for debugging.

---

Generated with the help of Claude Code, https://claude.ai/code

Co-Authored-By: Claude Code Opus 4.7 <noreply@anthropic.com>
Mirror the Linux pattern: `mac-aarch64-tests` runs the cargo unit tests,
preflight, and fast COW PHP suite. `mac-aarch64-build` (depends_on the
tests step) does the static-PHP runtime bundle, the release cargo build,
and the APFS-sparsebundle COW e2e. The aarch64 binary is uploaded via
YAML `artifact_paths` rather than inline `buildkite-agent artifact
upload`, so it ships even when a later e2e step fails.

Both steps re-run `install-macos-runtime-tools.sh` because BK doesn't
pin a step to the agent that ran its dependency.

---

Generated with the help of Claude Code, https://claude.ai/code

Co-Authored-By: Claude Code Opus 4.7 <noreply@anthropic.com>
a8c BK has no Intel mac queue, so the GHA `macos-15-intel` matrix entry
can't be mirrored end-to-end. Add a build-only step on the existing
`mac` (aarch64) queue that adds the x86_64-apple-darwin rustup target
and cross-compiles forkpress with an empty runtime bundle, mirroring
the Windows smoke pattern.

The resulting binary is uploaded via `artifact_paths` for inspection
but isn't run — Rosetta isn't installed on the mac VMs and embedding
the static PHP runtime cross-arch from spc is non-trivial.
release-publish remains the source of truth for shippable Intel
binaries.

---

Generated with the help of Claude Code, https://claude.ai/code

Co-Authored-By: Claude Code Opus 4.7 <noreply@anthropic.com>
Mirror the Linux/Mac aarch64 pattern on Windows: `windows-tests` does
the cargo unit tests, PowerShell syntax checks, and the
installer-error-surface harness. `windows-build` (depends_on tests)
does the release cargo build. The Windows binary is uploaded via YAML
`artifact_paths` instead of an inline `buildkite-agent artifact
upload`, so it ships even if a later step fails.

While here, switch `docs-validate` to `artifact_paths` for the built
site too and drop the now-unused `mount-buildkite-agent` option on its
docker plugin — `artifact_paths` is handled by the agent on the host
and doesn't need the buildkite-agent binary mounted into the
container.

---

Generated with the help of Claude Code, https://claude.ai/code

Co-Authored-By: Claude Code Opus 4.7 <noreply@anthropic.com>
Three repeated chunks now live in `.buildkite/commands/_lib/`:

- `docker-chown-trap.sh` — chown-back-on-exit for any docker-bound
  step that writes to the bind-mounted workspace. Sourced by
  `linux-tests.sh`, `linux-build.sh`, `docs-validate.sh`.
- `install-rust.sh` — idempotent rustup-init + cargo env source for
  the mac runners. Sourced by all three mac scripts.
- `spc-doctor-prerun.sh` — `spc_doctor_prerun <dist_name>` clones
  the pinned spc revision into `.build/<dist_name>/static-php-cli/`
  and runs `doctor --auto-fix`. Used by `linux-build.sh` (twice,
  once per profile) and `mac-aarch64-build.sh`.

`SPC_REF` lives in the helper so the two callers stay in lock-step;
override via env when bumping in isolation.

---

Generated with the help of Claude Code, https://claude.ai/code

Co-Authored-By: Claude Code Opus 4.7 <noreply@anthropic.com>
Mirror the mac `_lib/install-rust.sh` pattern on Windows. The
duplicated rustup-init + PATH-prepend block in `windows-tests.ps1`
and `windows-build.ps1` is now a dot-sourced helper that takes the
target triple as a parameter.

---

Generated with the help of Claude Code, https://claude.ai/code

Co-Authored-By: Claude Code Opus 4.7 <noreply@anthropic.com>
@mokagio mokagio force-pushed the mokagio/buildkite-ci-skeleton branch from 58249aa to 2a6ebe4 Compare May 19, 2026 04:07
@mokagio mokagio requested a review from Copilot May 19, 2026 04:08
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 7 comments.

Comment thread .buildkite/pipeline.yml Outdated
Comment on lines +18 to +20
- label: ":crab: Linux tests (workspace, excl. CLI)"
key: linux-tests
command: .buildkite/commands/linux-tests.sh
Comment thread .buildkite/pipeline.yml Outdated
Comment thread .buildkite/pipeline.yml Outdated
Comment on lines +43 to +48
- label: ":crab: Linux build (workspace, excl. CLI)"
key: linux-build
depends_on: linux-tests
command: .buildkite/commands/linux-build.sh
artifact_paths:
- "target/x86_64-unknown-linux-musl/release/forkpress"
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot apply changes based on this feedback

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — updated the label from :crab: Linux build (workspace, excl. CLI) to :crab: Linux build (workspace + CLI) in commit e562884, matching the pattern already used for the Linux tests step.

Comment thread .buildkite/commands/_lib/spc-doctor-prerun.sh Outdated
git -C "$spc_dir" checkout --detach FETCH_HEAD
(
cd "$spc_dir"
composer install --no-dev --no-interaction --quiet

host_uid="$(stat -c %u .)"
host_gid="$(stat -c %g .)"
trap 'chown -R "$host_uid:$host_gid" . 2>/dev/null || true' EXIT

CI_TOOLKIT_PLUGIN_VERSION='6.1.0'
DOCKER_PLUGIN_VERSION='v5.13.0'
RUST_IMAGE_TAG='1.95-trixie'
mokagio and others added 2 commits May 19, 2026 14:17
The a8c BK agent forces the cleanup flags regardless of any per-step
`BUILDKITE_GIT_CLEAN_FLAGS` override — the warning
`Ignored BUILDKITE_GIT_CLEAN_FLAGS` shows on every Linux job. Earlier
builds where the per-step env "worked" were just landing on agents
that happened to be free of pollution.

Root cause: `DOCKER_USERNS_REMAP=true` on the agent host means
container root is remapped to an unprivileged subuid. Files written
by the docker step end up owned by 100000-ish on the host, and the
chown trap (running inside the userns-remapped container) couldn't
see the real `buildkite-agent` UID to restore ownership.

`userns: host` on the docker plugin keeps the container in the host
namespace: container root = host root, the chown trap chowns to
`buildkite-agent` for real, and the next checkout's cleanup
succeeds. Drop the now-ineffective per-step env overrides while
here.

---

Generated with the help of Claude Code, https://claude.ai/code

Co-Authored-By: Claude Code Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
mokagio and others added 2 commits May 19, 2026 14:58
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
PR comment flagged that `spc-doctor-prerun.sh`'s `composer install`
omits the `--ignore-platform-reqs` flag `scripts/build-dist.sh` uses,
so the pre-run can fail on a host whose system PHP is older than the
constraint floating in spc's composer.lock — even when the real
build would have succeeded.

Extract the ref pin, clone/refresh, and composer install into
`scripts/shared/static-php-cli.sh`. Both `scripts/build-dist.sh` and
`.buildkite/commands/_lib/spc-doctor-prerun.sh` source it now, so
they can no longer drift on revision, repo URL, or composer flags.
The marker-file write moves inside the helper since both callers
expect it at the same path.

---

Generated with the help of Claude Code, https://claude.ai/code

Co-Authored-By: Claude Code Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants