ci: enforce immutable installs and scope the docker bake cache

[jordan-simonovski]

Two CI-time improvements. The headline non-change: I measured caching node_modules and it loses, so it was deliberately not added — restoring the ~1.8G tree costs ~29.6s (plus a 241M cache download) versus a ~23.2s warm yarn install. What did ship is smaller and safe.

What changed

• yarn install → yarn install --immutable across the CI install steps (main.yml lint/unit/integration/clickhouse-static-build, knip.yml, and the e2e-setup action). This fails CI on lockfile drift — the same class of bug that would otherwise slip through.
• The integration job's docker buildx bake cache goes from an unscoped type=gha to type=gha,mode=max,scope=ci-integration, so its layers actually persist between runs.

Key decisions

• node_modules cache dropped on evidence. The restore-and-decompress of a 1.8G tree is I/O-bound and slower than yarn's warm-cache linking, so it would make CI slower, not faster. Kept only the existing setup-node yarn download cache.
• There is no e2e bake step to scope — the e2e stack uses docker compose pull of published images.

Impact

• Marginally faster/more-reliable installs; better docker layer reuse on the integration job.
• release.yml and claude.yml have their own yarn install steps left intentionally out of scope (this PR targets the PR-time CI path).
• CI only — no changeset.
• Built off main, so it will conflict in main.yml with the lint and integration PRs; on resolution, carry forward --immutable and the mode=max,scope=ci-integration bake cache.

Implementation detail

--immutable is the Yarn 4 flag (repo pins yarn@4.13.0). Local dev install paths (scripts/, Makefile) are untouched. The single build: target in docker-compose.ci.yml means the shared * bake selector can't cause a scope collision (verified with buildx bake --print).

[changeset-bot]

⚠️ No Changeset found

Latest commit: 2efcd8c

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments

Project	Deployment	Actions	Updated (UTC)
hyperdx-oss	Ignored		Jul 16, 2026 5:24am
hyperdx-storybook	Ignored		Jul 16, 2026 5:24am

[github-actions]

🔴 Tier 4 — Critical

Touches auth, data models, config, tasks, OTel pipeline, ClickHouse, or CI/CD.

Why this tier:

• Critical-path files (1):
  • .github/workflows/main.yml

Review process: Deep review from a domain expert. Synchronous walkthrough may be required.
SLA: Schedule synchronous review within 2 business days.

Stats

• Production files changed: 1
• Production lines changed: 2
• Branch: jordansimonovski/ci-caching
• Author: jordan-simonovski

To override this classification, remove the review/tier-4 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

[greptile-apps]

Greptile Summary

This PR tightens CI hygiene by switching all PR-path yarn install calls to --immutable (the Yarn 4 lockfile-enforcement flag) and scoping the integration job's Docker BuildKit GHA cache to prevent cross-run layer pollution.

• yarn install --immutable is added to six install steps across main.yml (lint, unit, integration, clickhouse-static-build), knip.yml, and the e2e-setup composite action — any PR that forgets to commit an updated yarn.lock will now fail CI rather than silently diverging.
• The integration job's docker buildx bake cache switches from the unscoped type=gha to type=gha,mode=max,scope=ci-integration, so intermediate image layers are written (mode=max) and the scope prevents cache entries from bleeding into or being polluted by other workflow runs.

Confidence Score: 5/5

Safe to merge — all changes are additive CI guardrails with no effect on application code or runtime behavior.

Every change is a single-flag addition (--immutable) or a cache parameter refinement. The --immutable flag correctly matches the Yarn 4 API the repo already pins (yarn@4.13.0). The mode=max + scope=ci-integration cache options are well-formed BuildKit GHA cache directives; mode is correctly placed on cache-to only (it's a write-time option). The first post-merge run will be a cold Docker build, but that's a one-time startup cost and is expected. No application code, dependencies, or published artifacts are touched.

No files require special attention.

Important Files Changed

Filename	Overview
.github/workflows/main.yml	Adds --immutable to all four yarn install steps (lint, unit, integration, clickhouse-static-build) and scopes the integration job's buildx bake GHA cache with mode=max and scope=ci-integration
.github/actions/e2e-setup/action.yml	Single-line change adding --immutable to the composite action's yarn install step; no other logic altered
.github/workflows/knip.yml	Single-line change adding --immutable to the knip job's yarn install step; no other logic altered

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    PR([Pull Request / Push]) --> lint & unit & integration & clickhouse & knip & e2e

    subgraph lint["lint job"]
        L1[Setup Node / yarn cache] --> L2["yarn install --immutable"]
        L2 --> L3[make ci-build / ci-lint]
    end

    subgraph unit["unit job"]
        U1[Setup Node / yarn cache] --> U2["yarn install --immutable"]
        U2 --> U3[make ci-build / ci-unit]
    end

    subgraph integration["integration job"]
        I1[Setup Node / yarn cache] --> I2["yarn install --immutable"]
        I2 --> I3[Expose GitHub Runtime]
        I3 --> I4["docker buildx bake\ntype=gha,mode=max,scope=ci-integration"]
        I4 --> I5[make ci-build / ci-int]
    end

    subgraph clickhouse["clickhouse-static-build job"]
        C1[Setup Node / yarn cache] --> C2["yarn install --immutable"]
        C2 --> C3[yarn build:clickhouse]
    end

    subgraph knip["knip job"]
        K1[Setup Node / yarn cache] --> K2["yarn install --immutable"]
        K2 --> K3[yarn knip --reporter json]
    end

    subgraph e2e["e2e-setup action"]
        E1[Setup Node / yarn cache] --> E2["yarn install --immutable"]
        E2 --> E3[npx nx ci:build]
        E3 --> E4[docker compose pull / up]
    end

    I4 <-->|"Scoped GHA cache\nci-integration"| GHACache[(GitHub Actions Cache)]

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    PR([Pull Request / Push]) --> lint & unit & integration & clickhouse & knip & e2e

    subgraph lint["lint job"]
        L1[Setup Node / yarn cache] --> L2["yarn install --immutable"]
        L2 --> L3[make ci-build / ci-lint]
    end

    subgraph unit["unit job"]
        U1[Setup Node / yarn cache] --> U2["yarn install --immutable"]
        U2 --> U3[make ci-build / ci-unit]
    end

    subgraph integration["integration job"]
        I1[Setup Node / yarn cache] --> I2["yarn install --immutable"]
        I2 --> I3[Expose GitHub Runtime]
        I3 --> I4["docker buildx bake\ntype=gha,mode=max,scope=ci-integration"]
        I4 --> I5[make ci-build / ci-int]
    end

    subgraph clickhouse["clickhouse-static-build job"]
        C1[Setup Node / yarn cache] --> C2["yarn install --immutable"]
        C2 --> C3[yarn build:clickhouse]
    end

    subgraph knip["knip job"]
        K1[Setup Node / yarn cache] --> K2["yarn install --immutable"]
        K2 --> K3[yarn knip --reporter json]
    end

    subgraph e2e["e2e-setup action"]
        E1[Setup Node / yarn cache] --> E2["yarn install --immutable"]
        E2 --> E3[npx nx ci:build]
        E3 --> E4[docker compose pull / up]
    end

    I4 <-->|"Scoped GHA cache\nci-integration"| GHACache[(GitHub Actions Cache)]

Loading

_{Reviews (1): Last reviewed commit: "ci: enforce immutable installs and scope..." | Re-trigger Greptile}

[github-actions]

Deep Review

✅ No critical issues found.

This is a CI-only change across three workflow files: yarn install → yarn install --immutable (Yarn 4.13.0) on the PR-time install steps, and a scoped GHA cache (type=gha,mode=max,scope=ci-integration) on the integration job's docker buildx bake. No production code paths are touched. The --immutable flag is the correct Yarn 4 spelling, crazy-max/ghaction-github-runtime@v4 runs before the bake so type=gha has its token, and docker-compose.ci.yml exposes a single buildable target so the * bake selector cannot cause a scope collision. No P0/P1/P2 findings.

🔵 P3 nitpicks (2)

• .github/workflows/main.yml:68 — the bake cache-from uses only the new scoped type=gha,scope=ci-integration, so the first integration run after merge cold-misses every layer and rebuilds from scratch before the cache warms.
  • Fix: optionally add a second unscoped --set *.cache-from="type=gha" for one transition run, or accept the one-time cold build since the scoped cache self-heals afterward.
  • _reliability
• .github/workflows/main.yml:24 — the --immutable rollout is partial: PR-time install steps here, in knip.yml, and in e2e-setup/action.yml now enforce lockfile immutability while release.yml and claude.yml keep bare yarn install, leaving the enforcement boundary implicit and easy to propagate incorrectly when copying a job template.
  • Fix: add a short comment marking the PR-time-vs-non-PR distinction, or centralize the install step in a shared composite action so the flag is set in one place.
  • _{maintainability}

---

Reviewers (5): correctness, maintainability, testing, reliability, agent-native.

Testing gaps: none — the change is self-verifying by running CI; --immutable exercises lockfile-drift detection on every run and the cache scope affects only layer hit/miss performance, not testable behavior.

Residual risk: mode=max stores all intermediate layers under the ci-integration scope; GitHub's ~10GB per-repo cache budget is shared across the many scopes already used by docker-build.yml, release.yml, and release-nightly.yml, so this can nudge LRU eviction and cache hit-rate — a performance concern, not a correctness one. Separately, --immutable will hard-fail installs on any package.json/yarn.lock drift, which is the intended enforcement but will surface as a CI failure for automated dependency bumps that do not regenerate the lockfile.

[github-actions]

E2E Test Results

✅ All tests passed • 239 passed • 3 skipped • 1534s

Status	Count
✅ Passed	239
❌ Failed	0
⚠️ Flaky	2
⏭️ Skipped	3

Tests ran across 4 shards in parallel.

View full report →

[jordan-simonovski]

Thanks — no P0/P1/P2, and the two notes are expected:

• First integration run cold-misses the new scope: correct and self-resolving — the ci-integration scope warms on the first run post-merge, then subsequent runs hit it.
• Partial --immutable rollout (release.yml/claude.yml still bare): intentional and noted in the PR description — this PR targets the PR-time CI path. Extending immutability to the release/claude workflows is a fine follow-up but out of scope here.

[jordan-simonovski]

Distributed into the PRs that own each file, to remove this PR's main.yml conflicts and cut the PR count:

• lint --immutable → ci(lint): nx cache + affected, plus the escape-hatch ratchet #2661 (owns the lint job)
• integration/unit/clickhouse --immutable + knip --immutable + integration bake mode=max,scope=ci-integration → ci: faster test feedback — split API tests, parallelise common-utils, shard integration #2653 (owns the integration job + is the main.yml infra PR)
• e2e-setup --immutable → test(e2e): determinism cleanup + e2e performance (workers, 8-core, Chromium cache) #2655 (now owns e2e-setup via the Chromium-cache fold)

Nothing dropped; the node_modules-cache decision (dropped on the timing gate) doesn't need a home. Closing.