diff --git a/README.md b/README.md index 83a6c16..6cf19f8 100644 --- a/README.md +++ b/README.md @@ -23,6 +23,10 @@ Local CI parses your workflow file, runs selected jobs in containers with pre-bu | **[mikefarah/yq](https://github.com/mikefarah/yq) v4+** | YAML parsing -- **not** pip `yq` (PyPI) or [kislyuk/yq](https://github.com/kislyuk/yq) | | [act](https://github.com/nektos/act) | Run GitHub Actions locally | +### Host platforms + +Local CI is **Linux-container-first**: workflows run Linux containers unless your jobs target Windows runners. On macOS or Windows, Docker Desktop provides the Linux runtime (use the **WSL2** backend on Windows). Setup varies by OS and Docker install; see **[Cross-platform prerequisites](cli/Usage%20Guide.md#cross-platform-prerequisites)** in the User Guide for per-OS installs, Docker caveats, a **preflight checklist**, and notes on **Windows, WSL2, and parallelism**. + ### yq (mikefarah/yq v4 or newer) Several incompatible tools share the name `yq`. Local CI requires **[mikefarah/yq](https://github.com/mikefarah/yq)** **v4+** (`yq --version` output should reference `mikefarah` / `github.com/mikefarah` or report `version v4`). The wrong install fails at runtime with unclear errors. If no suitable binary is on `PATH`, the CLI uses a PyYAML fallback with limited expression support and emits a warning. diff --git a/cli/Usage Guide.md b/cli/Usage Guide.md index 535de32..8f2d31f 100644 --- a/cli/Usage Guide.md +++ b/cli/Usage Guide.md @@ -4,6 +4,14 @@ - [Introduction](#introduction) - [Installation](#installation) + - [Prerequisites](#prerequisites) + - [Cross-platform prerequisites](#cross-platform-prerequisites) + - [Docker runtime differences](#docker-runtime-differences) + - [Preflight checklist (before your first run)](#preflight-checklist-before-your-first-run) + - [Windows, WSL2, and parallelism](#windows-wsl2-and-parallelism) + - [Install for usage](#install-for-usage) + - [Install for development](#install-for-development) + - [Verify installation](#verify-installation) - [Quick Start](#quick-start) - [Configuration](#configuration) - [Creating a Config File](#creating-a-config-file) @@ -52,9 +60,56 @@ Key features: | Tool | Purpose | Install | |------|---------|---------| | Python 3.10+ | Runtime | [python.org](https://www.python.org/downloads/) | -| Docker | Container execution | [Docker Desktop](https://www.docker.com/products/docker-desktop/) | -| **mikefarah/yq v4+** | YAML parsing (not pip `yq` / kislyuk) | **Windows:** `winget install MikeFarah.yq` or `choco install yq` — **macOS:** `brew install yq` — **Linux:** `sudo snap install yq` or [release binary](https://github.com/mikefarah/yq/releases) — see [yq#install](https://github.com/mikefarah/yq#install) | -| act | Local GitHub Actions | `choco install act-cli` / `brew install act` / `curl -s https://raw.githubusercontent.com/nektos/act/master/install.sh \| sudo bash` | +| Docker | Container execution | Linux: Docker Engine; macOS/Windows: [Docker Desktop](https://www.docker.com/products/docker-desktop/) (Windows: WSL2 backend recommended) | +| **[mikefarah/yq](https://github.com/mikefarah/yq) v4+** | YAML parsing | Not pip `yq` / kislyuk; see [yq#install](https://github.com/mikefarah/yq#install) and [Cross-platform prerequisites](#cross-platform-prerequisites) | +| act | Local GitHub Actions | `brew install act` / install script / Windows: `act-cli` ([Chocolatey](https://community.chocolatey.org/packages/act-cli)); see below | + +### Cross-platform prerequisites + +Local CI runs **Linux** workflow jobs in Docker via [act](https://github.com/nektos/act). Your machine may be Linux, macOS, or Windows; whether runs succeed depends on **Docker** and **act**, not only Python. + +#### Linux (native) + +- **Docker:** Install Docker Engine + CLI from your distribution or [Docker Engine install](https://docs.docker.com/engine/install/). +- **act:** e.g. `curl -s https://raw.githubusercontent.com/nektos/act/master/install.sh \| sudo bash` or a distro package if maintained. +- **yq:** Install **[mikefarah/yq](https://github.com/mikefarah/yq) v4+** (Snap, release binary, or package that ships that binary). Verify with `yq --version` (output should reference mikefarah or report `version v4`). + +#### macOS (Intel and Apple silicon) + +- **Docker:** [Docker Desktop for Mac](https://www.docker.com/products/docker-desktop/). +- **act / yq:** Commonly `brew install act` and `brew install yq` (Homebrew provides mikefarah `yq`). +- **Apple silicon (arm64):** Workflow images are often **linux/amd64**. If image pulls or act fails with architecture errors, pull or run with an explicit platform, e.g. `docker pull --platform linux/amd64 `. Use Docker Desktop settings for Rosetta / QEMU emulation when prompted. + +#### Windows (Docker Desktop and WSL2) + +- **Docker:** Install [Docker Desktop](https://www.docker.com/products/docker-desktop/) and use the **WSL2** backend. Enable **WSL integration** for the distro where you open terminals so `docker` talks to the same engine Local CI will use. +- **act:** On Windows, Chocolatey installs the executable as **`act-cli`** (see discovery logic in [`localci/core/executor.py`](localci/core/executor.py)). Ensure `act` or `act-cli` is on `PATH`. You can use `winget install nektos.act` where it exposes `act`. +- **yq:** Prefer `winget install MikeFarah.yq`, `choco install yq`, or a binary from [mikefarah/yq releases](https://github.com/mikefarah/yq/releases). +- **Avoid mixed environments:** Run `docker`, `act`, and `localci` from the **same** environment (all on Windows, or all inside one WSL distro). Mixing a Windows `docker` CLI with a daemon only reachable inside WSL2 commonly yields "Docker daemon is not running" or connection errors. + +### Docker runtime differences + +- **Socket / API:** On Linux, Docker usually listens on a Unix socket (often `unix:///var/run/docker.sock`). On Windows, Docker Desktop typically exposes the engine via the named pipe **`\\.\pipe\docker_engine`**. The `docker` CLI hides most of this; problems usually show up as connection errors if the CLI and daemon use different contexts. See [Docker contexts](https://docs.docker.com/engine/context/working-with-contexts/). +- **CPU architecture:** When the host CPU architecture does not match the image (`amd64` vs `arm64`), you may need **`docker pull --platform linux/amd64`** (or `arm64`) or equivalent build settings so act starts a compatible image. +- **WSL2:** Bind mounts that cross the Windows/Linux filesystem boundary can be slow or permission-sensitive. Prefer cloning projects under the Linux filesystem for the distro integrated with Docker (e.g. under `\\wsl$\...`) when using WSL2 heavily. See [Docker Desktop WSL 2 backend](https://docs.docker.com/desktop/wsl/). + +### Preflight checklist (before your first run) + +Run these checks before `localci run`: + +```bash +python --version # expect Python 3.10+ +docker version +docker info # daemon must be reachable (no fatal errors) +act --version || act-cli --version # Windows: often act-cli +yq --version # confirm mikefarah / v4.x +localci --version +localci analyze path/to/your/workflow.yml # parse-only smoke test +``` + +### Windows, WSL2, and parallelism + +There is no separate code path named "solo pool" in this repository; parallel jobs use a [`ThreadPoolExecutor`](https://docs.python.org/3/library/concurrent.futures.html#threadpoolexecutor) whose size comes from config (`parallel.max_jobs`) or **`localci run --parallel`**. On **Windows with Docker Desktop + WSL2**, all Linux containers share resources in one VM; **high parallelism** can saturate CPU, RAM, or disk I/O and cause flaky failures or Docker errors. If that happens, reduce **`parallel.max_jobs`** in `.localci.yml` or pass **`--parallel N`** with a smaller **N**. ### Install for usage diff --git a/cli/localci/cli/run.py b/cli/localci/cli/run.py index bbcc1b3..f58ab30 100644 --- a/cli/localci/cli/run.py +++ b/cli/localci/cli/run.py @@ -146,6 +146,9 @@ def run( cfg = ctx.obj["config"] effective_timeout = timeout or cfg.execution.timeout + # parallel.max_jobs / --parallel cap concurrent act processes. On Windows with + # Docker Desktop + WSL2, many containers share one Linux VM; very high values + # can contend on CPU/RAM/disk (see Usage Guide: Windows, WSL2, parallelism). effective_parallel = parallel or cfg.parallel.max_jobs effective_keep_containers = ( keep_containers if keep_containers is not None else cfg.execution.keep_containers diff --git a/docs/Design Guide.md b/docs/Design Guide.md index e01a5de..de60fb0 100644 --- a/docs/Design Guide.md +++ b/docs/Design Guide.md @@ -638,6 +638,8 @@ Examples: **Recommendation**: Use Windows host for maximum compatibility (supports both Windows and Linux containers). +For concrete install commands, Docker Desktop/WSL2 notes, and a preflight checklist before your first run, see **[Cross-platform prerequisites](../cli/Usage%20Guide.md#cross-platform-prerequisites)** in the Usage Guide. + ### G. Benefits and Limitations #### Benefits