Skip to content

CI: cache package Electron native rebuild outputs#23274

Merged
Sewer56 merged 4 commits into
Nexus-Mods:masterfrom
Sewer56:ci/opt-package-native-cache
May 28, 2026
Merged

CI: cache package Electron native rebuild outputs#23274
Sewer56 merged 4 commits into
Nexus-Mods:masterfrom
Sewer56:ci/opt-package-native-cache

Conversation

@Sewer56
Copy link
Copy Markdown
Member

@Sewer56 Sewer56 commented May 20, 2026

Part of LAZ-411

Depends on:

This is package-specific version of:

Summary

Cache Electron native rebuild outputs in the shared package composite action.

Because change is in .github/actions/package/action.yml, both workflows benefit:

  • package.yml
  • nightly.yml

Change

Package action now:

  1. Installs dependencies with VORTEX_DEFER_ELECTRON_REBUILD=1, then builds Vortex
  2. Deploys to src/main/dist with VORTEX_SKIP_ELECTRON_REBUILD=1; native modules are skipped throughout
  3. Restores cached native modules into src/main/dist/node_modules; cache hit means skip rebuild
  4. On cache miss, rebuilds native modules separately via npx electron-rebuild (native build headers are also cached to speed this up)
  5. Packages the installer

Cache keys are derived from pnpm-lock.yaml, pnpm-workspace.yaml, src/main/package.json, and src/main/postinstall.mjs so dependency bumps invalidate correctly.

Cache miss behaviour

On cache miss, actions/cache restores nothing and electron-rebuild runs a full rebuild, identical to baseline. Cache populates on first run; subsequent runs hit. GitHub evicts stale caches automatically.

Measurement

Baseline package runs before this optimisation:

Dist-cache-only warm measured runs:

Result:

  • Package Vortex median: 832s → 651s (-181s)
  • packaged native cache hit on all warm measured runs

@Sewer56 Sewer56 force-pushed the ci/opt-package-native-cache branch 2 times, most recently from 5e84e4a to ae8af8a Compare May 20, 2026 23:06
@Sewer56 Sewer56 marked this pull request as ready for review May 20, 2026 23:42
@Sewer56 Sewer56 requested a review from a team as a code owner May 20, 2026 23:42
IDCs
IDCs previously approved these changes May 21, 2026
Sewer56 added 3 commits May 21, 2026 14:27
Split the shared package composite action so packaged electron-rebuild outputs can use a dedicated actions/cache entry. This action is used by both package.yml and nightly.yml.

This branch is based on the separate Electron rebuild controls PR. Controls alone do not optimize package.yml; this commit owns the package behavior changes:

- install dependencies with VORTEX_ELECTRON_REBUILD=defer
- cache native build headers
- build the workspace without running a workspace Electron native rebuild
- publish src/main/dist with VORTEX_ELECTRON_REBUILD=skip
- restore/cache packaged native outputs under src/main/dist/node_modules
- explicitly rebuild packaged native modules in src/main/dist
- run electron-builder packaging from src/main

Only the packaged/dist native output cache is needed. An initial version cached both workspace node_modules/.pnpm outputs and packaged src/main/dist/node_modules outputs, but probe runs on Sewer56/Vortex showed the workspace Electron rebuild/cache is not required for package.yml to build successfully. Removing the workspace rebuild/cache still passed publish and installer packaging.

The packaged/dist cache is still required. pnpm deploy creates src/main/dist/node_modules from the content-addressable store and dist virtual store, so it does not preserve Electron-rebuilt workspace virtual-store outputs.

Cold/no-hit dist-cache-only probe still passed, but spent about 97s from the packaged cache step to the installer step:

- https://github.com/Sewer56/Vortex/actions/runs/26091864064

The original two-cache warmup spent about 67s in the packaged rebuild before the packaged cache existed:

- https://github.com/Sewer56/Vortex/actions/runs/26067243939

Warm packaged cache hits reduce that rebuild path to about 1-2s.

The explicit dist rebuild calls node ./node_modules/@electron/rebuild/lib/cli.js instead of pnpm --dir src/main/dist exec electron-rebuild because pnpm exec in the deployed dist package can try dependency resolution there and fail on peer-suffixed specs. Calling the installed CLI directly avoids that resolver path.

Baseline package.yml runs before this optimization:

- https://github.com/Sewer56/Vortex/actions/runs/26065180899
- https://github.com/Sewer56/Vortex/actions/runs/26065183051
- https://github.com/Sewer56/Vortex/actions/runs/26065185231

Package Vortex median was 814s.

Original two-cache measured runs:

- https://github.com/Sewer56/Vortex/actions/runs/26067806598
- https://github.com/Sewer56/Vortex/actions/runs/26067807434
- https://github.com/Sewer56/Vortex/actions/runs/26067808289

Package Vortex median was 642s (-172s vs baseline).

Dist-cache-only warm measured runs:

- https://github.com/Sewer56/Vortex/actions/runs/26092460250
- https://github.com/Sewer56/Vortex/actions/runs/26093157889
- https://github.com/Sewer56/Vortex/actions/runs/26093159302

Package Vortex median was 633s (-181s vs baseline). Packaged native cache hit on all warm measured runs.

Cleaned-stack validation after rebase:

- https://github.com/Sewer56/Vortex/actions/runs/26099063004
- https://github.com/Sewer56/Vortex/actions/runs/26099063777
- https://github.com/Sewer56/Vortex/actions/runs/26100726058
- https://github.com/Sewer56/Vortex/actions/runs/26115728616
- https://github.com/Sewer56/Vortex/actions/runs/26115728667

These runs succeeded with packaged cache hit true where measured. Package Vortex steps were 685s, 649s, and 661s in the first three validation runs.
pnpm --dir src/main run package works but is fragile - breaks silently
if the directory moves. pnpm -F @vortex/main is the idiomatic workspace
filter that resolves the same script by package name.

Semantically equivalent: both run only the 'package' script without
dependency resolution, preserving the native rebuild optimization.

Using pnpm nx run would be WRONG here - it resolves the dependency chain
(package -> publish -> rimraf ./dist), destroying the native rebuilds.
- Replace pnpm --dir with pnpm -F for package/package:nosign scripts
  (--dir breaks silently if directory moves; -F uses package name)

- Guard native rebuild on cache miss (skip @electron/rebuild
  entirely when packaged-native-cache hits, instead of running
  unconditionally for a wasteful tree scan)

- Use npx electron-rebuild instead of direct node path
  (pnpm exec fails in dist/ with LOCKFILE_CONFIG_MISMATCH since
  deployed dist has no pnpm workspace context; npx resolves from
  node_modules/.bin without pnpm's lockfile validation)
@Sewer56 Sewer56 force-pushed the ci/opt-package-native-cache branch from ae8af8a to f87cbd5 Compare May 21, 2026 13:38
IDCs
IDCs previously approved these changes May 27, 2026
@Sewer56 Sewer56 dismissed IDCs’s stale review May 27, 2026 21:09

The merge-base changed after approval.

IDCs
IDCs previously approved these changes May 28, 2026
@Sewer56 Sewer56 dismissed IDCs’s stale review May 28, 2026 05:36

The merge-base changed after approval.

@IDCs
Copy link
Copy Markdown
Contributor

IDCs commented May 28, 2026

@Sewer56 - you'll need to rebase/merge before anyone can approve this PR as the merge-base branch no longer exists.

@Sewer56
Copy link
Copy Markdown
Member Author

Sewer56 commented May 28, 2026

@IDCs I'm actually not sure what happened here.
Because both the branch I was PR-ing from, and the master branch on Nexus-Mods actually did exist.

The only thing that changed was that #23270 was merged; which somehow ended up invalidating this.

Oddly, just merging master and re-pushing it fixed it.

@Sewer56 Sewer56 merged commit ed155b8 into Nexus-Mods:master May 28, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants