Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,13 @@ cargo test --offline --test e2e push_workflow_succeeds # one e2e test
cargo run -- -C tests/fixtures/repo --host -n # try the CLI against the fixture repo
```

- `tests/e2e.rs` runs the built binary (through `bin()`, which adds `--no-history`) with `--host --no-cache-server` against `tests/fixtures/repo` (needs `bash`, `node` and `git` on PATH, no Docker). Assertions grep stdout, so changing log/summary line formats (`"<wf> / <job>: <result>"`, masking `***`) can break them.
- `tests/e2e.rs` runs the built binary (through `bin()`, which adds `--no-history`) with `--host --no-cache-server` against `tests/fixtures/repo` (needs `bash`, `node` and `git` on PATH, no Docker). Assertions grep stdout, so changing log/summary line formats (`"<wf> / <job>: <result>"`, the closing `🏁`/`❌ N workflows, M jobs: …` line, masking `***`) can break them.
- The e2e test binary bakes in absolute paths via `CARGO_BIN_EXE_runthrough` / `CARGO_MANIFEST_DIR`. If it fails with `run runthrough: ... NotFound` after the repo directory was moved or renamed, `touch tests/e2e.rs` (or `cargo clean`) to force a rebuild.
- Exit codes: `0` all jobs succeeded/skipped, `1` a job failed, `2` an internal error (`anyhow` error from `run()`).

## Architecture

Pipeline: `main.rs` (CLI + config assembly) → `workflow.rs` (load YAML) → `filters.rs` (`on.<event>` branch/tag/path/type filters) → `planner.rs` (stages, matrix) → `runner.rs` (execute) using `expr.rs`, `actions.rs`, `backend.rs`, `commands.rs`, `matchers.rs`, `context.rs`, `servers.rs`; `info.rs` serves the exit-early modes (`--graph`, `--man-page`, `--list-options`, `--bug-report`) and `--watch`'s fingerprint; `history.rs` and `dashboard.rs` record runs and serve `--dashboard`; `markers.rs` formats the job and step status lines the runner logs and `history.rs` parses back.
Pipeline: `main.rs` (CLI + config assembly) → `workflow.rs` (load YAML) → `filters.rs` (`on.<event>` branch/tag/path/type filters) → `planner.rs` (stages, matrix) → `runner.rs` (execute) using `expr.rs`, `actions.rs`, `backend.rs`, `commands.rs`, `matchers.rs`, `context.rs`, `servers.rs`; `info.rs` serves the exit-early modes (`--graph`, `--man-page`, `--list-options`, `--bug-report`, `--install-hook`) and `--watch`'s fingerprint; `history.rs` and `dashboard.rs` record runs and serve `--dashboard`; `markers.rs` formats the job and step status lines the runner logs and `history.rs` parses back.

- **`main.rs`** parses CLI (clap derive; flag names follow act, see `docs/act-parity.md`), discovers workflows (`workflow_paths`: `-W`, or `.github/workflows` plus `.runthrough/workflows`; recursively unless `--no-recurse`), filters by event and its `on.<event>` filters (an explicit `-j` or a single workflow file bypasses both), handles `-l`/`-n`/`-g`/`--validate`, merges `.secrets`/`.vars`/`.env`/`.input` files with CLI flags (flags win), and builds `runner::Config`. Arguments from `.actrc` files are spliced in before the real ones (`actrc_args`). `Cli` is `pub` so `info.rs` and `servers.rs` read it directly.
- **`workflow.rs`** validates every workflow (`schema_errors`: required keys and value types; `strict_check` adds unknown keys under `--strict`), then deserializes it with `serde_yaml`; many fields stay as raw `serde_yaml::Value` because they may contain `${{ }}` and are evaluated later. `yaml_to_json` / `yaml_scalar_to_string` bridge YAML into the JSON values the expression engine uses.
Expand All @@ -38,13 +38,13 @@ Pipeline: `main.rs` (CLI + config assembly) → `workflow.rs` (load YAML) → `f
- `exec_with` is the single point that spawns step processes: it creates fresh `GITHUB_OUTPUT`/`ENV`/`PATH`/`STATE`/`STEP_SUMMARY` files under `files/<seq>/`, injects env (runner → job → step), parses workflow commands from output, and applies masking. `Target::Backend` runs in the job's backend; `Target::Container` runs a Docker action or `docker://` step as a sibling container started from the host (`run_container_step`), with env passed as `-e`.
- `Logger` prints through `emit`, which ignores stdout write errors so a closed pipe never skips cleanup.
- `resolve_platform` maps `runs-on` labels to an image or the host (`-self-hosted` sentinel; `ubuntu*`/`linux` → `DEFAULT_IMAGE`; `self-hosted` and `macos*` on macOS → host).
- `actions/checkout` is special-cased (`checkout`) unless `--no-skip-checkout`: the own repo is copied (no-op with `--bind`), another `repository` is shallow-cloned.
- `actions/checkout` is special-cased (`checkout`) unless `--no-skip-checkout`: the own repo is copied (no-op with `--bind`; only the HEAD commit with `--committed`, which also reads the repository's workflows, called ones included, from that commit exported by `actions::export_head` into `Config.committed`), another `repository` is shallow-cloned.
- **`backend.rs`**: `Backend::Host` or `Backend::Container`, driven through a CLI chosen by `Engine` (`docker`, or Apple's `container` via `--engine container`); no container API crate. Engine differences are handled here, as `Engine` methods the runner asks (`quiet_args`, `host_gateway_arg`, `has_network_aliases`, `has_health_checks`, `has_host_network`, `needs_temp_volume`, `is_remote`): Apple `container` gets `--progress none`, has no `--network-alias` (the runner writes service IPs into the job container's `/etc/hosts`) and no health checks (`take_health` strips `--health-*` and `wait_healthy` polls the command with `exec`); `host_address` is `host.docker.internal` for Docker and the network's `ipv4Gateway` for Apple. The job scratch dir is bind-mounted at the **same absolute path** in the container, so host and container paths are interchangeable throughout the runner. Exceptions are the job's volumes (`JobRun.job_volumes`, mounted over those paths): RUNNER_TEMP for Docker jobs, and, where Docker shares host files through a VM (`Engine::shares_files_through_vm`, i.e. Docker Desktop outside Linux) and without `--bind` or the engine socket (whose jobs hand workspace paths to the engine), the workspace; the host does not see writes there, and `sync_in` copies checkouts into it. With a remote Docker (`engine_is_remote`: `DOCKER_HOST=ssh://`/`tcp://`, or `RUNTHROUGH_REMOTE_ENGINE=1` to force it locally) `Config.copy_mode` replaces the mounts with per-job volumes at those same paths; `JobRun::sync_in` and `exec_with` keep them in sync via `docker cp` (temp/ and files/<seq> before each step, files/<seq> back after, checkout results and cached actions once).
- **`actions.rs`**: parses `uses:` (`Local`, `Remote`, `Docker`), loads `action.yml`, and `fetch_remote` git-clones into the action cache (`$XDG_CACHE_HOME/runthrough/actions` or `~/.cache/...`), serialized by a global mutex; `--offline` only uses the cache.
- **`commands.rs`**: parsing of `GITHUB_ENV`-style files (including heredoc delimiters), `::command::` lines, and secret masking.
- **`matchers.rs`**: problem matchers (`::add-matcher::`). `JobRun.matchers` is the job-wide list (shared with forks); `exec_with` feeds each output line to a per-step copy and logs matches as annotations.
- **`context.rs`**: `RepoInfo::detect` shells out to `git` to fill the `github` context (repo slug from `--remote-name`, sha, ref, default branch); `host` (`--github-instance`) drives the server/API URLs. `event_ref_and_sha` then overrides ref/sha from the event payload the way GitHub does (`refs/pull/N/merge`, release tags, push `ref`/`after`, …), and `GITHUB_*_URL` from the caller's environment win.
- **`history.rs`**: `Recorder` (in `Config.history`, unless `--no-history` or `--insecure-secrets`) writes each run to `<cache>/runs/<id>/`: `meta.json` from `main.rs` (masked argv, repo, event, jobs by stage, and `config`: env/vars/inputs masked, secret names, platforms), every `Logger::line` after masking into `log.jsonl`, and `result.json` at the end. `Summary` / `Job::feed` rebuild jobs and steps from the log's status lines (`⭐`, `✅`/`❌`/`⚠️`/`🛑`, `⏭️`, `🚀`/`🏁`), formatted and parsed by `markers.rs`, whose tests check each line parses back.
- **`history.rs`**: `Recorder` (in `Config.history`, unless `--no-history` or `--insecure-secrets`) writes each run to `<cache>/runs/<id>/`: `meta.json` from `main.rs` (masked argv, repo, event, jobs by stage, and `config`: env/vars/inputs masked, secret names, platforms), every `Logger::line` after masking into `log.jsonl`, and `result.json` at the end. `Summary` / `Job::feed` rebuild jobs and steps from the log's status lines (`⭐`, `✅`/`❌`/`⚠️`/`🛑`, `⏭️`, `🚀`/`🏁`, `ℹ️` notes on what a local run does not reproduce) and keeps step summaries recorded beside them, formatted and parsed by `markers.rs`, whose tests check each line parses back.
- **`dashboard.rs`**: `--dashboard` serves `dashboard.html` (one self-contained page, no build step) and a JSON API (`/api/runs`, `/api/runs/<id>`, `/api/runs/<id>/job?workflow=&job=`) through `servers::serve`.
- **`servers.rs`**: std-only HTTP/1.1 servers jobs reach through `ACTIONS_CACHE_URL` (`@actions/cache` v1 API, on by default, entries persisted in `index.json`) and `ACTIONS_RESULTS_URL` (`@actions/artifact` v2 Twirp API plus Azure-blob-style block uploads, with `--artifact-server-path`; the toolkit sends proto field names such as `workflow_run_backend_id`) plus `ACTIONS_RUNTIME_URL` (the v1 API of the v3 actions, files under `v3/<run>/`, gzipped chunks stored with a `.gz__` suffix). `ACTIONS_RUNTIME_TOKEN` is a JWT signed with a per-run key (`RuntimeAuth`, or the caller's own token) whose `scp` claim carries the run/job IDs the toolkit parses; the artifact API refuses calls without it, except the signed block URLs. The cache API lives under a random per-run prefix (`cache_token`), lowercases keys, and garbage-collects entries (`CacheStore::gc`) on start and after each upload. They bind `127.0.0.1`, or `0.0.0.0` for Apple containers (reached via the vmnet gateway); a failure to start the cache server only disables it.

Expand Down
10 changes: 8 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,14 @@ runthrough --watch # rerun whenever a file changes
runthrough --graph # draw the job graph
runthrough --dashboard # browse recorded runs at http://127.0.0.1:7766
runthrough --validate --strict # check the workflows, then exit
runthrough --committed # rehearse the HEAD commit, leaving uncommitted changes out
runthrough --install-hook # add a git pre-push hook that runs the push workflows first
```

A run ends with one line for all of its workflows: jobs by result, the time it took and, when it was recorded, its dashboard URL. With `--watch`, each rerun also says which jobs changed result since the last one.

`--install-hook` writes `.git/hooks/pre-push` (or the hook in `core.hooksPath`), never over an existing hook. It runs `runthrough --committed --quiet push`, so the commit being pushed is rehearsed rather than the working tree (its checkout and the workflows themselves come from HEAD), and a failed job stops the push, as does pushing a commit other than HEAD, which the hook does not rehearse; `git push --no-verify` skips it once. Add flags such as `--host` to the hook's `runthrough` line.

Workflows are read from `.github/workflows` and, laid out the same way, `.runthrough/workflows` (for workflows meant only for local runs); `-W` reads a single file or directory instead.

Flags follow act's names (`--matrix`, `--reuse`, `--json`, `--input-file`, …); `runthrough --help` lists them and [docs/act-parity.md](docs/act-parity.md) tracks parity with act. Day-to-day operation and troubleshooting are in [RUNBOOK.md](RUNBOOK.md).
Expand All @@ -47,7 +53,7 @@ Flags win over files. Arguments from `.actrc` files are read before the command

### Dashboard

Every run is recorded (with secrets masked) under `~/.cache/runthrough/runs`, and `runthrough --dashboard` serves a local web UI over them in the style of GitHub's Actions tab: the runs list (optionally grouped by repository or date), a run's job graph by stage with its matrix legs and annotations, and each job's steps with their logs, refreshed while a run is in progress. A failed job's page has **Copy for LLM**, which copies the job's context, its steps and the last 200 lines of each failed step as Markdown to paste into an LLM. **Resources** lists the ports runthrough listens on and the Docker containers, networks and volumes it created, with the run each belongs to; ones whose run has ended are marked left behind, with a command to remove them. `--dashboard-addr` / `--dashboard-port` change where it listens; `--no-history` skips recording a run, and runs with `--insecure-secrets` are never recorded. The newest 100 runs are kept.
Every run is recorded (with secrets masked) under `~/.cache/runthrough/runs`, and `runthrough --dashboard` serves a local web UI over them in the style of GitHub's Actions tab: the runs list (optionally grouped by repository or date), a run's job graph by stage with its matrix legs and annotations, and each job's steps with their logs, refreshed while a run is in progress. A run's summary page shows the jobs' step summaries (`GITHUB_STEP_SUMMARY`, as Markdown) and, as annotations, what the run did not reproduce locally; **Copy rehearsal note** copies the jobs and their results as Markdown for a pull request. A failed job's page has **Copy for LLM**, which copies the job's context, its steps and the last 200 lines of each failed step as Markdown to paste into an LLM. **Resources** lists the ports runthrough listens on and the Docker containers, networks and volumes it created, with the run each belongs to; ones whose run has ended are marked left behind, with a command to remove them. `--dashboard-addr` / `--dashboard-port` change where it listens; `--no-history` skips recording a run, and runs with `--insecure-secrets` are never recorded. The newest 100 runs are kept.

### Exit codes

Expand All @@ -72,7 +78,7 @@ Steps see `RUNTHROUGH=true` and `ACT=true` (also in the `env` context, so workfl

## Not supported

The cache service v2 API (`ACTIONS_CACHE_SERVICE_V2`). `permissions` and `jobs.<id>.snapshot` are validated but have no local effect; `environment` is reported (name and `url`) without environment secrets or protection rules. `concurrency` groups span one `runthrough` invocation, and a job cancelled by `cancel-in-progress` stops before its next step.
A job using one of these logs a note (`ℹ️`) when it starts. The cache service v2 API (`ACTIONS_CACHE_SERVICE_V2`). `permissions` and `jobs.<id>.snapshot` are validated but have no local effect; `environment` is reported (name and `url`) without environment secrets or protection rules. `concurrency` groups span one `runthrough` invocation, and a job cancelled by `cancel-in-progress` stops before its next step.

## Platforms

Expand Down
1 change: 1 addition & 0 deletions docs/favicon.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
16 changes: 13 additions & 3 deletions docs/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,15 @@
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="runthrough runs your GitHub Actions workflows locally: an act-style runner written in Rust.">
<title>runthrough</title>
<link rel="icon" href="favicon.svg" type="image/svg+xml">
<meta property="og:type" content="website">
<meta property="og:title" content="runthrough: the full rehearsal before you push">
<meta property="og:description" content="Run your GitHub Actions workflows locally, in Docker, Apple container or on the host. An act-style runner in one Rust binary.">
<meta property="og:image" content="https://simota.github.io/runthrough/og.png">
<meta property="og:image:width" content="1200">
<meta property="og:image:height" content="630">
<meta property="og:image:alt" content="RUNTHROUGH on a strip of yellow spike tape over a dark stage: the full rehearsal before you push.">
<meta name="twitter:card" content="summary_large_image">
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Big+Shoulders+Stencil+Display:wght@700;900&family=Instrument+Sans:wght@400;500;600&family=JetBrains+Mono:wght@400;600&display=swap">
<style>
Expand Down Expand Up @@ -295,6 +304,7 @@ <h2>Call it from the wings</h2>
<div class="cue"><span class="n">Q6</span><code>runthrough --graph</code><p>Draw the job graph.</p></div>
<div class="cue"><span class="n">Q7</span><code>runthrough --validate --strict</code><p>Check the workflows, including unknown keys, then exit.</p></div>
<div class="cue"><span class="n">Q8</span><code>runthrough --dashboard</code><p>Browse recorded runs at <code>127.0.0.1:7766</code>.</p></div>
<div class="cue"><span class="n">Q9</span><code>runthrough --install-hook</code><p>Rehearse before every push: a pre-push hook runs the <code>push</code> workflows on the commit being pushed.</p></div>
</div>
</div>
</section>
Expand Down Expand Up @@ -422,7 +432,7 @@ <h4>Stage 2</h4>
<div class="section-head">
<span class="tape">Understudies</span>
<h2>What stays different from GitHub</h2>
<p>A local rehearsal can't do everything the real stage does. These are the known gaps.</p>
<p>A local rehearsal can't do everything the real stage does. These are the known gaps, and a run says so when a job uses one of them.</p>
</div>
<div class="gaps">
<div><strong>Cache service v2</strong>The <code>ACTIONS_CACHE_SERVICE_V2</code> API is not served. The v1 API is.</div>
Expand All @@ -438,7 +448,7 @@ <h2>What stays different from GitHub</h2>
<div class="section-head">
<span class="tape">Curtain call</span>
<h2>Exit codes you can script</h2>
<p>Use runthrough as a pre-push hook or in your own scripts. Secrets come from <code>-s</code> or <code>.secrets</code>, variables from <code>--var</code> or <code>.vars</code>.</p>
<p><code>runthrough --install-hook</code> makes it a pre-push hook; use it in your own scripts too. Secrets come from <code>-s</code> or <code>.secrets</code>, variables from <code>--var</code> or <code>.vars</code>.</p>
</div>
<div class="exits">
<div class="exit"><b>0</b><p>Every job succeeded or was skipped.</p></div>
Expand All @@ -449,7 +459,7 @@ <h2>Exit codes you can script</h2>
</section>

<footer class="wrap foot">
<span>runthrough 0.1.0 · MIT license</span>
<span>runthrough · MIT license</span>
<a href="https://github.com/simota/runthrough">github.com/simota/runthrough</a>
</footer>

Expand Down
31 changes: 31 additions & 0 deletions docs/og.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
<!doctype html>
<!-- Source of og.png, the share image (1200×630). Render it again after an edit:
chrome --headless --screenshot=docs/og.png --window-size=1200,630 docs/og.html -->
<html lang="en">
<head>
<meta charset="utf-8">
<title>runthrough share image</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Big+Shoulders+Stencil+Display:wght@900&family=Instrument+Sans:wght@500&family=JetBrains+Mono:wght@600&display=block">
<style>
html, body { margin: 0; width: 1200px; height: 630px; overflow: hidden; }
body {
background: radial-gradient(circle at 70% 18%, rgba(255, 214, 120, .38), transparent 42%), #101214;
color: #d8dde2; font-family: "Instrument Sans", "Helvetica Neue", Arial, sans-serif;
display: grid; align-content: center; gap: 36px; padding: 0 96px; box-sizing: border-box;
}
.tape {
justify-self: start; background: #f3c21c; color: #1a1a17; transform: rotate(-2deg);
font: 900 150px/1 "Big Shoulders Stencil Display", "Arial Narrow", Impact, sans-serif;
letter-spacing: .02em; text-transform: uppercase; padding: 22px 48px 14px;
clip-path: polygon(1% 0, 99% 4%, 100% 50%, 98% 100%, 0 96%, 0 45%);
}
p { margin: 0; font-size: 46px; font-weight: 500; }
code { font: 600 28px/1 "JetBrains Mono", monospace; color: #7d858e; }
</style>
</head>
<body>
<div class="tape">runthrough</div>
<p>The full rehearsal before you push.</p>
<code>$ runthrough # GitHub Actions, run locally</code>
</body>
</html>
Binary file added docs/og.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading