diff --git a/DESIGN.md b/DESIGN.md index 04d2e57d..36528a37 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -1695,10 +1695,10 @@ Implements `sysforge update` — the update manager. The iteration scope is the **Phase 2 — Source sync** (`_sync_sources`). Ensures every iterated package has an up-to-date local PKGBUILD. Skipped entirely with `--offline` (and with `--install-only`, which forces offline since no rebuild will happen). VCS pkgbases (`-git`/`-svn`/`-hg`/`-bzr`) are filtered out of the request batch when `--devel` is **not** in effect — Phase 3 returns `DEVEL` for those packages without rebuilding, so the purge / clone / fetch / pkgver-resolve work is wasted; `--cleansrc` therefore never touches VCS checkouts unless `--devel` is also passed. Explicit positional pkgnames do not override this — the skip is uniform, mirroring the existing Phase 3 build-step skip. Pacman-class repo packages are likewise excluded (their upgrade detection runs through `checkupdates_map` in Phase 3 and the upgrade itself is dispatched as one terminal `sudo pacman -Syu`). The remaining requests delegate to the `source_sync.SourceSyncScheduler` singleton (`get_scheduler(...)`), issuing one `SyncRequest` per package — AUR sources go through the RPC short-circuit and `aur_clone`/`git_fetch_and_compare`, repo sources skip the RPC short-circuit (no AUR-RPC equivalent for the Arch packaging repo) and clone via `pkgctl_checkout` then refresh via `git_fetch_and_compare`; for repo sources only, a `STATUS_DIVERGED` outcome on a clean working tree triggers a hard-reset to `FETCH_HEAD` because pkgctl checkouts carry no user commits worth preserving (dirty trees are still respected and stay diverged): -1. **RPC-first.** The scheduler batches one `aur_info` call for all AUR packages in the run. For every package whose cached `rpc_version` / `rpc_last_modified` still matches the local HEAD metadata, the request short-circuits to `STATUS_UP_TO_DATE` — no `git fetch` executes at all. +1. **RPC-first.** The scheduler batches one `aur_info` call for all AUR packages in the run. For every package whose checkout `origin` is that pkgbase's own AUR repo (`source_sync._rpc_vouches`, strict: a fork, another package's repo or a missing origin never qualifies) and whose cached `rpc_version` / `rpc_last_modified` still matches the local HEAD metadata, the request short-circuits to `STATUS_UP_TO_DATE` — no `git fetch` executes at all. 2. **Clone on miss.** Missing / empty / non-git dirs hit `aur_clone` through the shared rate limiter. -3. **Shallow fetch + compare.** Everything else runs `git_fetch_and_compare` (depth-1 fetch, non-destructive HEAD compare). VCS packages (`-git`/`-svn`/`-hg`/`-bzr`) reaching the scheduler are always force-fetched regardless of RPC metadata — but this only happens under `--devel`, since `_sync_sources` filters them out otherwise (see above). -4. **Divergence is surfaced, not fixed.** A local-plus-upstream divergence (e.g. force-push, local commits) yields `STATUS_DIVERGED`; the build proceeds against the local PKGBUILD and an operator can opt into `--cleansrc` on the next run. +3. **Fetch + compare.** Everything else runs `git_fetch_and_compare` (a full-history fetch, then a HEAD compare). VCS packages (`-git`/`-svn`/`-hg`/`-bzr`), which reach the scheduler only under `--devel`, go through the same RPC short-circuit as any other package (`3.5.0-B144`): `--devel` asks whether *upstream* moved, and Phase 3 answers that itself (`peek_upstream_commit`'s `ls-remote`, `evaluate_vcs_pkgver`'s `pkgver()`), so an AUR packaging repo the RPC reports unchanged is not re-fetched (before B144 every VCS package was force-fetched, one fetch per package for nothing). Two consequences are handled in the scheduler. Under the source freeze a VCS package that would short-circuit returns `STATUS_FROZEN` instead, because its version check needs upstream, which the freeze denies; that keeps `update --devel --frozen` failing through `_raise_if_frozen` instead of reporting `DEVEL_EVAL_FAILED` and exiting 0. And the RPC fields are cached only when the entry vouches for the checkout (`_record_rpc`, else `SourceMetaCache.forget_rpc`), so a checkout fetched from a fork and later pointed back at the AUR fetches once before it can short-circuit. +4. **Divergence on a clean tree is reset; on a dirty tree it is surfaced.** When upstream rewrote history (a force-push, common on the AUR) and the working tree is clean per the VCS-aware `git_is_dirty`, the checkout is hard-reset to `FETCH_HEAD`. A dirty tree carries real operator edits: it stays `STATUS_DIVERGED`, the build proceeds against the local PKGBUILD, and the operator can opt into `--cleansrc` on the next run. 5. **Rate-limit aware.** AUR `Retry-After` is honoured; runs where the remaining penalty exceeds `[aur] rate_limit_abort_s` mark all pending packages `STATUS_RATE_LIMITED` rather than hanging for minutes. Statuses treated as sync failures for the downstream buildability filter: `STATUS_FAILED`, `STATUS_RATE_LIMITED`, `STATUS_PURGE_REFUSED` (collected in `_SYNC_BLOCKING_STATUSES`, dispatched per-package via `_SYNC_STATUS_TO_ACTION`). `STATUS_DIVERGED` is a warning, not a blocker. `sync_failures` carries `(status, error_message)` so `_check_one_pkgbase` can map each blocking status to a distinct user-facing action: `STATUS_FAILED` → `PULL_FAILED`, `STATUS_RATE_LIMITED` → `RATE_LIMITED`, `STATUS_PURGE_REFUSED` → `PURGE_REFUSED` (cleansrc path). @@ -1886,7 +1886,7 @@ Implements `sysforge state` — a small read/repair namespace for `build_state.t - **`state orphans`** — scans `PKGDEST` for `.pkg.tar*` artifacts whose pkgname is installed AND whose version is strictly older than the installed version (the **superseded** category). Read-only by default; `--delete` deletes after a y/N prompt (`--yes` skips). The detection primitive `pacman.detect_orphan_artifacts(pkgdest, installed)` returns `{"superseded": [...]}`. - **Files whose pkgname is *not* installed are intentionally NOT surfaced.** They could be a build kept on purpose (a kernel artifact whose source has local commits the user wants to keep available for later install, a test build, etc.) and we can't safely tell the difference. Per the load-bearing rule: if `--delete` wouldn't safely delete a file, don't list it. Users who want a broader view can use `paccache`/manual inspection. - Files whose `.PKGINFO` can't be read or whose filename doesn't parse are silently skipped — never deleted. -- **`state failed`** — lists the `[failures]` table of `build_state.toml`: pkgbase, failure timestamp, diagnosis signature, the diagnosed fix command (when `build_diag` matched a known signature), and the truncated error tail. Read-only by default. `--names` (`3.1.0-F2`) prints only the pkgbases, one per line, with no header, no pager and no output at all when nothing failed, so `sysforge build $(sysforge state failed --names)` is the supported retry spelling; it is verb-local on purpose (this is the one listing whose output is already a set of package names), not a CLI-wide machine-output mode, and a distinct name because the global `--quiet` already means verbosity 0. `--clear PKGBASE` removes one entry and `--clear-all` removes all (both rewrite `build_state.toml`, so they take the sentinel like `state repair`). Failures are recorded by `sysforge update`'s build fan-out (`_record_build_failure`) and auto-clear on the next successful build of the same pkgbase (`BuildState.record` pops the matching failure) or once the package is removed (`3.5.0-B120`: `install_reconcile.clear_removed_failures`, see §`install_reconcile.py`; `state forget` also clears the failure of a pkgbase it leaves untracked), so the list stays a live view of *currently-broken* packages. The listing applies the removal pass in memory over the open hook window, so a `pacman -R` shows at once, before the next `update` clears the window for good. +- **`state failed`** — lists the `[failures]` table of `build_state.toml` (build failures, and since `3.5.0-F23` review-gate refusals, marked `kind = "review"` and shown with `review` in the signature column): pkgbase, failure timestamp, diagnosis signature, the diagnosed fix command (when `build_diag` matched a known signature), and the truncated error tail. Read-only by default. `--names` (`3.1.0-F2`) prints only the pkgbases, one per line, with no header, no pager and no output at all when nothing failed, for scripts; the retry spelling is `sysforge build --failed` (`3.5.0-F23`), which reads the same table and is a clean no-op when it is empty; it is verb-local on purpose (this is the one listing whose output is already a set of package names), not a CLI-wide machine-output mode, and a distinct name because the global `--quiet` already means verbosity 0. `--clear PKGBASE` removes one entry and `--clear-all` removes all (both rewrite `build_state.toml`, so they take the sentinel like `state repair`). Failures are recorded by `sysforge update`'s build fan-out (`_record_build_failure`) and auto-clear on the next successful build of the same pkgbase (`BuildState.record` pops the matching failure) or once the package is removed (`3.5.0-B120`: `install_reconcile.clear_removed_failures`, see §`install_reconcile.py`; `state forget` also clears the failure of a pkgbase it leaves untracked), so the list stays a live view of *currently-broken* packages. The listing applies the removal pass in memory over the open hook window, so a `pacman -R` shows at once, before the next `update` clears the window for good. - **`state forget PKG…`** — deletes the named packages' `build_state.toml` records so `sysforge update` stops maintaining them (the "hand it back to pacman" escape hatch for the durable-by-default tracking model — see §Package Manifest). A name matching a pkgbase forgets every split-package member sharing it (`BuildState.delete` per pkgname). The *installed* artifact is untouched and still carries the `sf-build` pacman group, so `pacman -Syu` won't replace it; reverting fully to the stock repo binary is a separate `pacman -S `. Rewrites `build_state.toml`, so it takes the sentinel like `state repair`. It also records the decision in the opt-out record (`forget_packages(opt_out=VIA_FORGET)`, `3.4.0-F4`): deleting the record alone did not stop `update`, whose adoption sources (every foreign package, a non-inert override, `repo_mode = "build_from_source"`) re-adopted the package on the next run. The record is keyed by the stock pkgbase and lists every member in both spellings (renamed and stock), so a split set opts out together and `update` matches a candidate by name without a `%BASE%` lookup. Only an explicit adoption clears it: `build `, `update ` on a mutating run, and `packages add `; passive sources never do. (Uninstalling a package already auto-stops tracking via `sync_with_installed`'s prune and writes no opt-out: a removed package is outside the installed set, and a later reinstall is a fresh decision. `forget` covers the keep-installed-but-unmanage case.) - **`state profiles`** (`3.2.0-F11`) — the one inspection/reclamation surface for the shared profile store (`/var/cache/sysforge`): one row per non-empty store — method, or method/target for the per-package methods (`pgo`, `autofdo`, `propeller`, `bolt`) — with size, age and the `3.2.0-B16` collected-version sidecar where present (`makepkg_pgo.list_profile_stores`). Read-only by default. `--delete [/]` (`makepkg_pgo.resolve_purge_target`: an unknown method or a target that is not one plain path component is refused, never guessed) deletes one store after a confirmation (default No) and, without a TTY, refuses unless `--yes` is given, because a collected profile needs a workload to regenerate; it takes the sentinel like `state failed --clear`. **Nothing purges implicitly**: a merged profile's existence is the durable PGO opt-in (`mesa_pgo.reuse_profdata`), so removing a package must not delete it — `revert-to-stock` and `uninstall` instead print one `ui()` line per non-empty store they leave behind (`verbs.shared.profile_store_names` reads the pre-rename `origin_pkgbase` before the record is forgotten; `state_cmd.orphaned_profile_lines` → `makepkg_pgo.stores_for_package`) with the `--purge` spelling that reclaims it. - **Pagination.** `state list`, `state orphans`, `state failed` and `state profiles` pipe their output through `$PAGER` (or `less -RF` / `more` as fallbacks) when stdout is a TTY. `--no-pager` disables. The pager wrapper `_maybe_pager(use_pager)` lives in `state_cmd.py` and degrades gracefully when no pager binary is available. @@ -2526,7 +2526,7 @@ Flow per request: 7. **Repo-track pin (`source = "repo"` only).** Gated by `config.resolve_repo_track` (`sysforge.toml [build] repo_track`, default `"stable"`) — a `"main"` track never pins and this step is a no-op. `_pin_repo_checkout(pkgbase, pkgbuild_dir)` looks up `pacman.get_repo_candidate_version(sync_db_name or pkgbase)` — the sync-DB read cross-checked against a refreshed side copy, so a sync DB that has not been refreshed since the last repo push cannot pin an already-superseded release (3.1.0-B3) — and, when found, runs `pkgctl_switch_version(pkgbuild_dir, version)` to check out the matching release tag (detached HEAD); no candidate in the sync DB is **not** an error — it warns and leaves the checkout on `main` (the package may live only in a custom repo not tracked by pacman). A `pkgctl_switch_version` failure returns `STATUS_FAILED` with the error attached. Two call sites: **clone-pin** — immediately after a fresh `pkgctl_checkout` in `_clone`, before the head is recorded in `SourceMetaCache`. **Fetch-repin** — in `_fetch`, on every subsequent sync of an already-pinned checkout: because a pinned checkout is a detached HEAD with no tracking branch, `git_fetch_and_compare` reports `no_tracking` for it; the repo-stable branch treats that as the steady state, refreshes tags with a plain `git fetch --tags origin` (`_fetch_repo_tags` — `git_fetch_and_compare` would otherwise bail before ever seeing new tags), then falls through to re-run `_pin_repo_checkout` so the checkout follows the sync DB's release forward. If the work-tree is dirty (`git_is_dirty(is_vcs=…)`) the re-pin is skipped and the result is `STATUS_DIVERGED` with `error="local edits present — not re-pinning"` — real operator work is never silently discarded by the pin logic. When the pin moves HEAD, the delta is reported as `STATUS_FETCHED` (`head_before`/`head_after` bracket the switch) rather than `STATUS_UP_TO_DATE`. Singletons: -- `get_scheduler(*, state_dir=None, offline=False, cleansrc=False, cleansrc_force=False, force_devel=False, min_fetch_interval_ms=None, rate_limit_abort_s=None, fetch_timeout=None, clone_timeout=None)` — returns the process-wide scheduler, constructing it on first call. Subsequent calls with the same args are memoised; dedup keys: `(pkgbase)` — any given pkgbase is synced at most once per process. `cleansrc_force=True` implies `cleansrc=True` and propagates to `purge_src(force=True)` so `STATUS_PURGE_REFUSED` cannot occur — the operator has explicitly opted in to overwriting local work. `force_devel` only gates the forced-fetch behaviour for VCS pkgbases that *reach* the scheduler; the higher-level filter in `update.py:_sync_sources` is what keeps VCS pkgbases out of the request batch entirely when `--devel` is off (so `--cleansrc` never purges a `-git` tree the user hasn't opted in to rebuild). +- `get_scheduler(*, state_dir=None, offline=False, cleansrc=False, cleansrc_force=False, min_fetch_interval_ms=None, rate_limit_abort_s=None, fetch_timeout=None, clone_timeout=None)` — returns the process-wide scheduler, constructing it on first call. Subsequent calls with the same args are memoised; dedup keys: `(pkgbase)` — any given pkgbase is synced at most once per process. `cleansrc_force=True` implies `cleansrc=True` and propagates to `purge_src(force=True)` so `STATUS_PURGE_REFUSED` cannot occur — the operator has explicitly opted in to overwriting local work. The scheduler knows nothing of `--devel` (`3.5.0-B144` dropped its `force_devel`, which only forced VCS packages past the RPC short-circuit): the filter in `update_sync._sync_sources` keeps VCS pkgbases out of the request batch entirely when `--devel` is off (so `--cleansrc` never purges a `-git` tree the user hasn't opted in to rebuild), and once in, a VCS package short-circuits like any other. The short-circuit also requires `_rpc_vouches`: `aur.aur_origin_pkgbase(pkgbuild_dir)` (strict, unlike the fail-open `is_aur_checkout` used for the AUR-disabled gate) must equal the entry's `PackageBase`, since the RPC is looked up by name and cannot vouch for a fork, another package's repo or a checkout with no origin. RPC fields are cached only when the entry vouches (`_record_rpc`; otherwise `SourceMetaCache.forget_rpc` clears them, as `update` preserves unset fields). Under the source freeze a VCS package that would short-circuit returns `STATUS_FROZEN`, since its version check needs upstream. - `reset_scheduler()` — test-only hook. Tests that need fresh state call this between runs. ### `build_state.py` @@ -2551,7 +2551,7 @@ Read by `sysforge update` for version drift detection (every installed AUR packa On the write path, after a successful build `makepkg_wrapper.py` derives `pkgver`/`pkgrel`/`epoch` from the produced `.pkg.tar.*` filenames rather than the static PKGBUILD parse. The static parser intentionally leaves shell parameter-expansion forms (e.g. `${_ver/[a-z]/.${_ver//[0-9.]/}}`) untouched so it never produces a misleading partial substitution, but a built package's filename always carries the fully resolved version. Falling back to filenames prevents source-built entries from storing literal `$...` strings that would mismatch every subsequent vercmp and cause the package to be flagged for rebuild on every `sysforge update` run. -**Build failures** live in a reserved top-level `[failures]` table (keyed by pkgbase), held apart from the per-package install mirror so `all_packages()` / `sync_with_installed()` stay a clean superset of `pacman -Q` (the `failures` key is popped into a private dict on load and re-serialized separately; a package literally named `failures` would collide but none exists in practice). Each entry carries `failed_at` (ISO 8601 UTC), `error` (the failure message tail — last ~6 lines / 600 chars), and optionally `pkgver`, `signature`, and `fix_cmd` (the latter two from `build_diag` postflight diagnosis when a known pattern matched). API: `record_failure(pkgbase, *, error, pkgver=None, signature=None, fix_cmd=None, failed_at=None)`, `clear_failure(pkgbase) -> bool`, `all_failures() -> {pkgbase: record}`. A successful `record()` calls `clear_failure(pkgbase)` so the failure list self-heals on the next good build; a removal clears it too (`3.5.0-B120`, `install_reconcile.clear_removed_failures`), as does `forget_packages` for a pkgbase it leaves with no tracked entry. `sysforge update`'s build fan-out records failures via `_record_build_failure` (opening a fresh `BuildState` so loop-time success writes aren't clobbered, and pulling `signature`/`fix_cmd` from the exception's `.diagnosis`, attached by `makepkg_wrapper`). Surfaced by `sysforge state failed`. +**Build failures** live in a reserved top-level `[failures]` table (keyed by pkgbase), held apart from the per-package install mirror so `all_packages()` / `sync_with_installed()` stay a clean superset of `pacman -Q` (the `failures` key is popped into a private dict on load and re-serialized separately; a package literally named `failures` would collide but none exists in practice). Each entry carries `failed_at` (ISO 8601 UTC), `error` (the failure message tail — last ~6 lines / 600 chars), and optionally `pkgver`, `signature`, and `fix_cmd` (the latter two from `build_diag` postflight diagnosis when a known pattern matched), and `kind = "review"` for a package refused at the review gate rather than one that failed to build (`3.5.0-F23`: `build_core._record_review_refusal`, called where `build_and_install` adds to `outcome.review_refused`, both for a target's own change and for a refused dependency, so an unattended run's refusals outlive it). `sysforge build --failed` (`build_cmd._add_recorded_failures`) builds every entry of either kind; the doctor `state` axis words a review entry as a refusal with `build --failed` as its remediation. API: `record_failure(pkgbase, *, error, pkgver=None, signature=None, fix_cmd=None, failed_at=None, kind=None)`, `clear_failure(pkgbase) -> bool`, `all_failures() -> {pkgbase: record}`. A successful `record()` calls `clear_failure(pkgbase)` so the failure list self-heals on the next good build; a removal clears it too (`3.5.0-B120`, `install_reconcile.clear_removed_failures`), as does `forget_packages` for a pkgbase it leaves with no tracked entry. `sysforge update`'s build fan-out records failures via `_record_build_failure` (opening a fresh `BuildState` so loop-time success writes aren't clobbered, and pulling `signature`/`fix_cmd` from the exception's `.diagnosis`, attached by `makepkg_wrapper`). Surfaced by `sysforge state failed`. **Reverted builds** live in a second reserved table, `[reverted]`, keyed by artifact filename and held apart the same way (popped on load, serialized after `[failures]`, excluded from the lost-update diff). When a refused install reverts an entry, `revert_record(pkgname, prior, *, artifact=)` keeps the entry it undid there (with its `pkgname`), so the build's facts survive while its artifact waits in `PKGDEST`. `restore_reverted(artifact) -> pkgname | None` reinstates it once pacman installs that exact file. It is the only route by which a reused ("already built") artifact gets a record, because `build_and_install`'s `AlreadyBuilt` branch installs without calling `makepkg_wrapper` (3.3.0-B2). Only an entry recorded since `prior` is kept: a second refusal finds the entry already reverted (equal to `prior` once the carried-forward `_BUILD_FACTS` are ignored) and leaves the first stash alone. A later `record()` for the pkgname drops its stash (a newer build supersedes it; installing the stale artifact must not roll back to it), as does `sync_with_installed` pruning the package. Per-package fields serialize in `_ENTRY_KEYS` order, shared by both tables. diff --git a/completions/_sysforge b/completions/_sysforge index b5c57647..389b852c 100644 --- a/completions/_sysforge +++ b/completions/_sysforge @@ -161,6 +161,7 @@ _sysforge_build() { $_sysforge_help_spec \ '(-m --makepkg)'{-m,--makepkg}'[extra makepkg flags, appended after profile flags]:flags' \ '--interactive[run makepkg without --noconfirm, attached to the terminal]' \ + '--failed[also build every package `state failed` lists]' \ '--profile-conf[alternate profiles.toml]:file:_files' \ '--toolchain[build with this toolchain as a whole]:toolchain:(gcc llvm)' \ '(--reset-toolchain)--save-toolchain[also save --toolchain for these packages]' \ diff --git a/completions/sysforge.bash b/completions/sysforge.bash index 5847b864..251961d8 100644 --- a/completions/sysforge.bash +++ b/completions/sysforge.bash @@ -206,7 +206,7 @@ _sysforge_installed_pkg_names() { _sysforge_build() { _sysforge_flag_arg && return - local flags="-m --makepkg --interactive --profile-conf \ + local flags="-m --makepkg --interactive --failed --profile-conf \ --toolchain --save-toolchain --reset-toolchain --cc --cxx --ld \ --no-pkg-log --log-dir --persist-log --cache-report --abi-check \ --no-update --cleansrc --cleansrc-force --no-llvm-preflight \ diff --git a/docs/design/09-primitives-layer.md b/docs/design/09-primitives-layer.md index 5deada88..513dfc01 100644 --- a/docs/design/09-primitives-layer.md +++ b/docs/design/09-primitives-layer.md @@ -631,7 +631,7 @@ Flow per request: 7. **Repo-track pin (`source = "repo"` only).** Gated by `config.resolve_repo_track` (`sysforge.toml [build] repo_track`, default `"stable"`) — a `"main"` track never pins and this step is a no-op. `_pin_repo_checkout(pkgbase, pkgbuild_dir)` looks up `pacman.get_repo_candidate_version(sync_db_name or pkgbase)` — the sync-DB read cross-checked against a refreshed side copy, so a sync DB that has not been refreshed since the last repo push cannot pin an already-superseded release (3.1.0-B3) — and, when found, runs `pkgctl_switch_version(pkgbuild_dir, version)` to check out the matching release tag (detached HEAD); no candidate in the sync DB is **not** an error — it warns and leaves the checkout on `main` (the package may live only in a custom repo not tracked by pacman). A `pkgctl_switch_version` failure returns `STATUS_FAILED` with the error attached. Two call sites: **clone-pin** — immediately after a fresh `pkgctl_checkout` in `_clone`, before the head is recorded in `SourceMetaCache`. **Fetch-repin** — in `_fetch`, on every subsequent sync of an already-pinned checkout: because a pinned checkout is a detached HEAD with no tracking branch, `git_fetch_and_compare` reports `no_tracking` for it; the repo-stable branch treats that as the steady state, refreshes tags with a plain `git fetch --tags origin` (`_fetch_repo_tags` — `git_fetch_and_compare` would otherwise bail before ever seeing new tags), then falls through to re-run `_pin_repo_checkout` so the checkout follows the sync DB's release forward. If the work-tree is dirty (`git_is_dirty(is_vcs=…)`) the re-pin is skipped and the result is `STATUS_DIVERGED` with `error="local edits present — not re-pinning"` — real operator work is never silently discarded by the pin logic. When the pin moves HEAD, the delta is reported as `STATUS_FETCHED` (`head_before`/`head_after` bracket the switch) rather than `STATUS_UP_TO_DATE`. Singletons: -- `get_scheduler(*, state_dir=None, offline=False, cleansrc=False, cleansrc_force=False, force_devel=False, min_fetch_interval_ms=None, rate_limit_abort_s=None, fetch_timeout=None, clone_timeout=None)` — returns the process-wide scheduler, constructing it on first call. Subsequent calls with the same args are memoised; dedup keys: `(pkgbase)` — any given pkgbase is synced at most once per process. `cleansrc_force=True` implies `cleansrc=True` and propagates to `purge_src(force=True)` so `STATUS_PURGE_REFUSED` cannot occur — the operator has explicitly opted in to overwriting local work. `force_devel` only gates the forced-fetch behaviour for VCS pkgbases that *reach* the scheduler; the higher-level filter in `update.py:_sync_sources` is what keeps VCS pkgbases out of the request batch entirely when `--devel` is off (so `--cleansrc` never purges a `-git` tree the user hasn't opted in to rebuild). +- `get_scheduler(*, state_dir=None, offline=False, cleansrc=False, cleansrc_force=False, min_fetch_interval_ms=None, rate_limit_abort_s=None, fetch_timeout=None, clone_timeout=None)` — returns the process-wide scheduler, constructing it on first call. Subsequent calls with the same args are memoised; dedup keys: `(pkgbase)` — any given pkgbase is synced at most once per process. `cleansrc_force=True` implies `cleansrc=True` and propagates to `purge_src(force=True)` so `STATUS_PURGE_REFUSED` cannot occur — the operator has explicitly opted in to overwriting local work. The scheduler knows nothing of `--devel` (`3.5.0-B144` dropped its `force_devel`, which only forced VCS packages past the RPC short-circuit): the filter in `update_sync._sync_sources` keeps VCS pkgbases out of the request batch entirely when `--devel` is off (so `--cleansrc` never purges a `-git` tree the user hasn't opted in to rebuild), and once in, a VCS package short-circuits like any other. The short-circuit also requires `_rpc_vouches`: `aur.aur_origin_pkgbase(pkgbuild_dir)` (strict, unlike the fail-open `is_aur_checkout` used for the AUR-disabled gate) must equal the entry's `PackageBase`, since the RPC is looked up by name and cannot vouch for a fork, another package's repo or a checkout with no origin. RPC fields are cached only when the entry vouches (`_record_rpc`; otherwise `SourceMetaCache.forget_rpc` clears them, as `update` preserves unset fields). Under the source freeze a VCS package that would short-circuit returns `STATUS_FROZEN`, since its version check needs upstream. - `reset_scheduler()` — test-only hook. Tests that need fresh state call this between runs. ### `build_state.py` @@ -656,7 +656,7 @@ Read by `sysforge update` for version drift detection (every installed AUR packa On the write path, after a successful build `makepkg_wrapper.py` derives `pkgver`/`pkgrel`/`epoch` from the produced `.pkg.tar.*` filenames rather than the static PKGBUILD parse. The static parser intentionally leaves shell parameter-expansion forms (e.g. `${_ver/[a-z]/.${_ver//[0-9.]/}}`) untouched so it never produces a misleading partial substitution, but a built package's filename always carries the fully resolved version. Falling back to filenames prevents source-built entries from storing literal `$...` strings that would mismatch every subsequent vercmp and cause the package to be flagged for rebuild on every `sysforge update` run. -**Build failures** live in a reserved top-level `[failures]` table (keyed by pkgbase), held apart from the per-package install mirror so `all_packages()` / `sync_with_installed()` stay a clean superset of `pacman -Q` (the `failures` key is popped into a private dict on load and re-serialized separately; a package literally named `failures` would collide but none exists in practice). Each entry carries `failed_at` (ISO 8601 UTC), `error` (the failure message tail — last ~6 lines / 600 chars), and optionally `pkgver`, `signature`, and `fix_cmd` (the latter two from `build_diag` postflight diagnosis when a known pattern matched). API: `record_failure(pkgbase, *, error, pkgver=None, signature=None, fix_cmd=None, failed_at=None)`, `clear_failure(pkgbase) -> bool`, `all_failures() -> {pkgbase: record}`. A successful `record()` calls `clear_failure(pkgbase)` so the failure list self-heals on the next good build; a removal clears it too (`3.5.0-B120`, `install_reconcile.clear_removed_failures`), as does `forget_packages` for a pkgbase it leaves with no tracked entry. `sysforge update`'s build fan-out records failures via `_record_build_failure` (opening a fresh `BuildState` so loop-time success writes aren't clobbered, and pulling `signature`/`fix_cmd` from the exception's `.diagnosis`, attached by `makepkg_wrapper`). Surfaced by `sysforge state failed`. +**Build failures** live in a reserved top-level `[failures]` table (keyed by pkgbase), held apart from the per-package install mirror so `all_packages()` / `sync_with_installed()` stay a clean superset of `pacman -Q` (the `failures` key is popped into a private dict on load and re-serialized separately; a package literally named `failures` would collide but none exists in practice). Each entry carries `failed_at` (ISO 8601 UTC), `error` (the failure message tail — last ~6 lines / 600 chars), and optionally `pkgver`, `signature`, and `fix_cmd` (the latter two from `build_diag` postflight diagnosis when a known pattern matched), and `kind = "review"` for a package refused at the review gate rather than one that failed to build (`3.5.0-F23`: `build_core._record_review_refusal`, called where `build_and_install` adds to `outcome.review_refused`, both for a target's own change and for a refused dependency, so an unattended run's refusals outlive it). `sysforge build --failed` (`build_cmd._add_recorded_failures`) builds every entry of either kind; the doctor `state` axis words a review entry as a refusal with `build --failed` as its remediation. API: `record_failure(pkgbase, *, error, pkgver=None, signature=None, fix_cmd=None, failed_at=None, kind=None)`, `clear_failure(pkgbase) -> bool`, `all_failures() -> {pkgbase: record}`. A successful `record()` calls `clear_failure(pkgbase)` so the failure list self-heals on the next good build; a removal clears it too (`3.5.0-B120`, `install_reconcile.clear_removed_failures`), as does `forget_packages` for a pkgbase it leaves with no tracked entry. `sysforge update`'s build fan-out records failures via `_record_build_failure` (opening a fresh `BuildState` so loop-time success writes aren't clobbered, and pulling `signature`/`fix_cmd` from the exception's `.diagnosis`, attached by `makepkg_wrapper`). Surfaced by `sysforge state failed`. **Reverted builds** live in a second reserved table, `[reverted]`, keyed by artifact filename and held apart the same way (popped on load, serialized after `[failures]`, excluded from the lost-update diff). When a refused install reverts an entry, `revert_record(pkgname, prior, *, artifact=)` keeps the entry it undid there (with its `pkgname`), so the build's facts survive while its artifact waits in `PKGDEST`. `restore_reverted(artifact) -> pkgname | None` reinstates it once pacman installs that exact file. It is the only route by which a reused ("already built") artifact gets a record, because `build_and_install`'s `AlreadyBuilt` branch installs without calling `makepkg_wrapper` (3.3.0-B2). Only an entry recorded since `prior` is kept: a second refusal finds the entry already reverted (equal to `prior` once the carried-forward `_BUILD_FACTS` are ignored) and leaves the first stash alone. A later `record()` for the pkgname drops its stash (a newer build supersedes it; installing the stale artifact must not roll back to it), as does `sync_with_installed` pruning the package. Per-package fields serialize in `_ENTRY_KEYS` order, shared by both tables. diff --git a/docs/design/verbs/state.md b/docs/design/verbs/state.md index c3674395..6b92dbce 100644 --- a/docs/design/verbs/state.md +++ b/docs/design/verbs/state.md @@ -8,7 +8,7 @@ Implements `sysforge state` — a small read/repair namespace for `build_state.t - **`state orphans`** — scans `PKGDEST` for `.pkg.tar*` artifacts whose pkgname is installed AND whose version is strictly older than the installed version (the **superseded** category). Read-only by default; `--delete` deletes after a y/N prompt (`--yes` skips). The detection primitive `pacman.detect_orphan_artifacts(pkgdest, installed)` returns `{"superseded": [...]}`. - **Files whose pkgname is *not* installed are intentionally NOT surfaced.** They could be a build kept on purpose (a kernel artifact whose source has local commits the user wants to keep available for later install, a test build, etc.) and we can't safely tell the difference. Per the load-bearing rule: if `--delete` wouldn't safely delete a file, don't list it. Users who want a broader view can use `paccache`/manual inspection. - Files whose `.PKGINFO` can't be read or whose filename doesn't parse are silently skipped — never deleted. -- **`state failed`** — lists the `[failures]` table of `build_state.toml`: pkgbase, failure timestamp, diagnosis signature, the diagnosed fix command (when `build_diag` matched a known signature), and the truncated error tail. Read-only by default. `--names` (`3.1.0-F2`) prints only the pkgbases, one per line, with no header, no pager and no output at all when nothing failed, so `sysforge build $(sysforge state failed --names)` is the supported retry spelling; it is verb-local on purpose (this is the one listing whose output is already a set of package names), not a CLI-wide machine-output mode, and a distinct name because the global `--quiet` already means verbosity 0. `--clear PKGBASE` removes one entry and `--clear-all` removes all (both rewrite `build_state.toml`, so they take the sentinel like `state repair`). Failures are recorded by `sysforge update`'s build fan-out (`_record_build_failure`) and auto-clear on the next successful build of the same pkgbase (`BuildState.record` pops the matching failure) or once the package is removed (`3.5.0-B120`: `install_reconcile.clear_removed_failures`, see §`install_reconcile.py`; `state forget` also clears the failure of a pkgbase it leaves untracked), so the list stays a live view of *currently-broken* packages. The listing applies the removal pass in memory over the open hook window, so a `pacman -R` shows at once, before the next `update` clears the window for good. +- **`state failed`** — lists the `[failures]` table of `build_state.toml` (build failures, and since `3.5.0-F23` review-gate refusals, marked `kind = "review"` and shown with `review` in the signature column): pkgbase, failure timestamp, diagnosis signature, the diagnosed fix command (when `build_diag` matched a known signature), and the truncated error tail. Read-only by default. `--names` (`3.1.0-F2`) prints only the pkgbases, one per line, with no header, no pager and no output at all when nothing failed, for scripts; the retry spelling is `sysforge build --failed` (`3.5.0-F23`), which reads the same table and is a clean no-op when it is empty; it is verb-local on purpose (this is the one listing whose output is already a set of package names), not a CLI-wide machine-output mode, and a distinct name because the global `--quiet` already means verbosity 0. `--clear PKGBASE` removes one entry and `--clear-all` removes all (both rewrite `build_state.toml`, so they take the sentinel like `state repair`). Failures are recorded by `sysforge update`'s build fan-out (`_record_build_failure`) and auto-clear on the next successful build of the same pkgbase (`BuildState.record` pops the matching failure) or once the package is removed (`3.5.0-B120`: `install_reconcile.clear_removed_failures`, see §`install_reconcile.py`; `state forget` also clears the failure of a pkgbase it leaves untracked), so the list stays a live view of *currently-broken* packages. The listing applies the removal pass in memory over the open hook window, so a `pacman -R` shows at once, before the next `update` clears the window for good. - **`state forget PKG…`** — deletes the named packages' `build_state.toml` records so `sysforge update` stops maintaining them (the "hand it back to pacman" escape hatch for the durable-by-default tracking model — see §Package Manifest). A name matching a pkgbase forgets every split-package member sharing it (`BuildState.delete` per pkgname). The *installed* artifact is untouched and still carries the `sf-build` pacman group, so `pacman -Syu` won't replace it; reverting fully to the stock repo binary is a separate `pacman -S `. Rewrites `build_state.toml`, so it takes the sentinel like `state repair`. It also records the decision in the opt-out record (`forget_packages(opt_out=VIA_FORGET)`, `3.4.0-F4`): deleting the record alone did not stop `update`, whose adoption sources (every foreign package, a non-inert override, `repo_mode = "build_from_source"`) re-adopted the package on the next run. The record is keyed by the stock pkgbase and lists every member in both spellings (renamed and stock), so a split set opts out together and `update` matches a candidate by name without a `%BASE%` lookup. Only an explicit adoption clears it: `build `, `update ` on a mutating run, and `packages add `; passive sources never do. (Uninstalling a package already auto-stops tracking via `sync_with_installed`'s prune and writes no opt-out: a removed package is outside the installed set, and a later reinstall is a fresh decision. `forget` covers the keep-installed-but-unmanage case.) - **`state profiles`** (`3.2.0-F11`) — the one inspection/reclamation surface for the shared profile store (`/var/cache/sysforge`): one row per non-empty store — method, or method/target for the per-package methods (`pgo`, `autofdo`, `propeller`, `bolt`) — with size, age and the `3.2.0-B16` collected-version sidecar where present (`makepkg_pgo.list_profile_stores`). Read-only by default. `--delete [/]` (`makepkg_pgo.resolve_purge_target`: an unknown method or a target that is not one plain path component is refused, never guessed) deletes one store after a confirmation (default No) and, without a TTY, refuses unless `--yes` is given, because a collected profile needs a workload to regenerate; it takes the sentinel like `state failed --clear`. **Nothing purges implicitly**: a merged profile's existence is the durable PGO opt-in (`mesa_pgo.reuse_profdata`), so removing a package must not delete it — `revert-to-stock` and `uninstall` instead print one `ui()` line per non-empty store they leave behind (`verbs.shared.profile_store_names` reads the pre-rename `origin_pkgbase` before the record is forgotten; `state_cmd.orphaned_profile_lines` → `makepkg_pgo.stores_for_package`) with the `--purge` spelling that reclaims it. - **Pagination.** `state list`, `state orphans`, `state failed` and `state profiles` pipe their output through `$PAGER` (or `less -RF` / `more` as fallbacks) when stdout is a TTY. `--no-pager` disables. The pager wrapper `_maybe_pager(use_pager)` lives in `state_cmd.py` and degrades gracefully when no pager binary is available. diff --git a/docs/design/verbs/update.md b/docs/design/verbs/update.md index 5f8e5aa8..95794bf0 100644 --- a/docs/design/verbs/update.md +++ b/docs/design/verbs/update.md @@ -10,10 +10,10 @@ Implements `sysforge update` — the update manager. The iteration scope is the **Phase 2 — Source sync** (`_sync_sources`). Ensures every iterated package has an up-to-date local PKGBUILD. Skipped entirely with `--offline` (and with `--install-only`, which forces offline since no rebuild will happen). VCS pkgbases (`-git`/`-svn`/`-hg`/`-bzr`) are filtered out of the request batch when `--devel` is **not** in effect — Phase 3 returns `DEVEL` for those packages without rebuilding, so the purge / clone / fetch / pkgver-resolve work is wasted; `--cleansrc` therefore never touches VCS checkouts unless `--devel` is also passed. Explicit positional pkgnames do not override this — the skip is uniform, mirroring the existing Phase 3 build-step skip. Pacman-class repo packages are likewise excluded (their upgrade detection runs through `checkupdates_map` in Phase 3 and the upgrade itself is dispatched as one terminal `sudo pacman -Syu`). The remaining requests delegate to the `source_sync.SourceSyncScheduler` singleton (`get_scheduler(...)`), issuing one `SyncRequest` per package — AUR sources go through the RPC short-circuit and `aur_clone`/`git_fetch_and_compare`, repo sources skip the RPC short-circuit (no AUR-RPC equivalent for the Arch packaging repo) and clone via `pkgctl_checkout` then refresh via `git_fetch_and_compare`; for repo sources only, a `STATUS_DIVERGED` outcome on a clean working tree triggers a hard-reset to `FETCH_HEAD` because pkgctl checkouts carry no user commits worth preserving (dirty trees are still respected and stay diverged): -1. **RPC-first.** The scheduler batches one `aur_info` call for all AUR packages in the run. For every package whose cached `rpc_version` / `rpc_last_modified` still matches the local HEAD metadata, the request short-circuits to `STATUS_UP_TO_DATE` — no `git fetch` executes at all. +1. **RPC-first.** The scheduler batches one `aur_info` call for all AUR packages in the run. For every package whose checkout `origin` is that pkgbase's own AUR repo (`source_sync._rpc_vouches`, strict: a fork, another package's repo or a missing origin never qualifies) and whose cached `rpc_version` / `rpc_last_modified` still matches the local HEAD metadata, the request short-circuits to `STATUS_UP_TO_DATE` — no `git fetch` executes at all. 2. **Clone on miss.** Missing / empty / non-git dirs hit `aur_clone` through the shared rate limiter. -3. **Shallow fetch + compare.** Everything else runs `git_fetch_and_compare` (depth-1 fetch, non-destructive HEAD compare). VCS packages (`-git`/`-svn`/`-hg`/`-bzr`) reaching the scheduler are always force-fetched regardless of RPC metadata — but this only happens under `--devel`, since `_sync_sources` filters them out otherwise (see above). -4. **Divergence is surfaced, not fixed.** A local-plus-upstream divergence (e.g. force-push, local commits) yields `STATUS_DIVERGED`; the build proceeds against the local PKGBUILD and an operator can opt into `--cleansrc` on the next run. +3. **Fetch + compare.** Everything else runs `git_fetch_and_compare` (a full-history fetch, then a HEAD compare). VCS packages (`-git`/`-svn`/`-hg`/`-bzr`), which reach the scheduler only under `--devel`, go through the same RPC short-circuit as any other package (`3.5.0-B144`): `--devel` asks whether *upstream* moved, and Phase 3 answers that itself (`peek_upstream_commit`'s `ls-remote`, `evaluate_vcs_pkgver`'s `pkgver()`), so an AUR packaging repo the RPC reports unchanged is not re-fetched (before B144 every VCS package was force-fetched, one fetch per package for nothing). Two consequences are handled in the scheduler. Under the source freeze a VCS package that would short-circuit returns `STATUS_FROZEN` instead, because its version check needs upstream, which the freeze denies; that keeps `update --devel --frozen` failing through `_raise_if_frozen` instead of reporting `DEVEL_EVAL_FAILED` and exiting 0. And the RPC fields are cached only when the entry vouches for the checkout (`_record_rpc`, else `SourceMetaCache.forget_rpc`), so a checkout fetched from a fork and later pointed back at the AUR fetches once before it can short-circuit. +4. **Divergence on a clean tree is reset; on a dirty tree it is surfaced.** When upstream rewrote history (a force-push, common on the AUR) and the working tree is clean per the VCS-aware `git_is_dirty`, the checkout is hard-reset to `FETCH_HEAD`. A dirty tree carries real operator edits: it stays `STATUS_DIVERGED`, the build proceeds against the local PKGBUILD, and the operator can opt into `--cleansrc` on the next run. 5. **Rate-limit aware.** AUR `Retry-After` is honoured; runs where the remaining penalty exceeds `[aur] rate_limit_abort_s` mark all pending packages `STATUS_RATE_LIMITED` rather than hanging for minutes. Statuses treated as sync failures for the downstream buildability filter: `STATUS_FAILED`, `STATUS_RATE_LIMITED`, `STATUS_PURGE_REFUSED` (collected in `_SYNC_BLOCKING_STATUSES`, dispatched per-package via `_SYNC_STATUS_TO_ACTION`). `STATUS_DIVERGED` is a warning, not a blocker. `sync_failures` carries `(status, error_message)` so `_check_one_pkgbase` can map each blocking status to a distinct user-facing action: `STATUS_FAILED` → `PULL_FAILED`, `STATUS_RATE_LIMITED` → `RATE_LIMITED`, `STATUS_PURGE_REFUSED` → `PURGE_REFUSED` (cleansrc path). diff --git a/docs/guides/getting-started.md b/docs/guides/getting-started.md index 95655463..8d41044c 100644 --- a/docs/guides/getting-started.md +++ b/docs/guides/getting-started.md @@ -97,6 +97,21 @@ from Arch's own packaging repos are accepted without asking. For unattended runs `[build] review = "accept"` in `packages.toml` (accepted changes are listed in the summary) or pass `--no-review`; the [freeze guide](freeze.md) explains the trade-off. +### Finishing what a run left behind + +sysforge remembers every package whose build failed and every package it refused at review, +until it builds. `sysforge state failed` lists them, with the fix sysforge diagnosed when it +recognised the failure. Once you have dealt with them (reviewed the change, added an override +in `profiles.toml` or `packages.toml`, installed a missing key), build exactly those again: + +```bash +sysforge build --failed +``` + +At a terminal that also shows you each refused change to review. Nothing else is rebuilt, and +each entry clears as its package builds. With nothing recorded it does nothing. This is the +usual follow-up to an unattended `update`. + At a terminal, `update` first prints its plan (what it will rebuild and why, what it will upgrade, and the protections in force) and asks `Proceed? [Y/n]`. `--yes`, or `confirm_plan = false` in `packages.toml`, skips the question. `sysforge setup` asks about diff --git a/docs/release-notes/unreleased.md b/docs/release-notes/unreleased.md index db42598e..60a5ac07 100644 --- a/docs/release-notes/unreleased.md +++ b/docs/release-notes/unreleased.md @@ -114,6 +114,10 @@ https://keepachangelog.com/en/1.1.0/ - **`3.5.0-F21` — `--unattended`: never prompt, even at a terminal.** A new global flag, a `[ui] unattended` key in `sysforge.toml` and a `SYSFORGE_UNATTENDED` environment variable make every question get the answer it gets when there is no terminal, the way a timer or cron run already behaves. That is useful for an overnight `update` left running in a terminal, or a script run over `ssh -t`. Safe defaults apply where there is one: an unreviewed PKGBUILD change is refused, no PGP key is imported, the kernel config review is skipped, and `update` proceeds without asking `Proceed?`. Anything that removes packages or data still refuses unless you pass `--yes`, so this is not a way to say yes to everything. sysforge does not ask for your sudo password either: an unattended run needs credentials sudo has already cached, or a `NOPASSWD` rule, and otherwise stops before changing anything, as it does with no terminal. The flag wins over the variable, and the variable over the key; `SYSFORGE_UNATTENDED=0` gets the questions back for one run when the key is on. `build --interactive` and `update --interactive` are unchanged: they still give makepkg and pacman the terminal. A message that refuses or skips something because it could not ask now gives the reason in brackets, `(unattended run)` or `(no terminal)`. Before, it always said there was no terminal, which was wrong for an unattended run at a terminal; `--yes` help now says it is needed in both cases. +--- + +- **`3.5.0-F23` — `sysforge build --failed` retries only what the last runs could not finish.** After an `update`, especially an unattended one, you can fix what went wrong and then rebuild just those packages: every package whose build failed, and every package refused at review because the run could not show you its changed PKGBUILD. Refusals used to appear only in that run's summary; they are now remembered like build failures, listed by `sysforge state failed` (marked `review`) and by `sysforge doctor --state`, and each clears once its package builds. At a terminal, `build --failed` shows each refused change for review first, and the overrides you added in the meantime apply. With nothing recorded it does nothing and exits 0. The update summary now suggests it under `Failed:` and `Refused at review:`. It replaces `sysforge build $(sysforge state failed --names)`, which still works. + ## Changed - **`3.0.0-F3` — `update` no longer builds a changed source nobody looked at.** The PKGBUILD review gate, which diffs a package's whole source tree since the last build you accepted, used to accept every change during `update` with a logged line, and accepted silently whenever there was no terminal, so a timer-driven `update` built and installed whatever had arrived. Now a changed source is shown to you at a terminal (view, accept, skip or abort, as `--review` did), and with no terminal (a timer, cron, a pipe) that package is refused: the rest of the run still builds, the summary lists it under "Refused at review" and the run exits 1. Changes from Arch's own packaging repos are accepted with a logged line, since the same people publish the binary packages you already install; `[build] review_trusted_sources` in `packages.toml` changes that list, and `--review` reviews everything. `build` and the toolchain stage's rebuild of libLLVM consumers follow the same rules, and `build` gains `--review`. To keep the old unattended behaviour, set `[build] review = "accept"` (every accepted change is still listed in the summary) or pass `--no-review`. The config comment explains the risk: a build sandbox limits what a build can read, but it does not inspect what gets installed. @@ -903,6 +907,10 @@ https://keepachangelog.com/en/1.1.0/ - **`3.5.0-B142` — the `artifact` commands work in your first session.** Run before anything else after installing, `sysforge artifact adopt` (or any command that saves the artifact list) ended in a Python traceback, `PermissionError` on `/var/lib/sysforge`: the artifact commands found SysForge's state directory on their own and never opened it to you the way `build` does until the `sysforge` group applies at your next login. They now go through the same step, and the read-only ones still never ask for `sudo`. +--- + +- **`3.5.0-B144` — `update --devel` no longer re-fetches every `-git` package's AUR repo, and a fork is never mistaken for the AUR package of the same name.** `--devel` checks whether *upstream* has new commits, and the version check does that itself (`git ls-remote`, or the PKGBUILD's `pkgver()`). Source sync also fetched every VCS package's AUR packaging repo, even when the AUR reported it unchanged: one `git fetch` per package, which adds up when you track many `-git` packages, and more when the AUR is slow. Those repos are now fetched only when the AUR says they changed, or your copy moved, as already happened without `--devel`. With the source freeze on, such a package is still reported as held by the freeze, so `update --devel --frozen` exits 1 as before. Separately, that check matched packages by name only, so a package you cloned from your own fork, when the AUR has one of the same name, could go unfetched until the AUR's copy changed. It now applies only to a checkout cloned from that package's own AUR repo; a fork, another package's repo or a checkout with no `origin` is always fetched, and switching a checkout from a fork back to the AUR fetches once before the check applies again. + ## Security - **`3.5.0-B37` — the root pacman hook no longer follows links in the state directory.** The hook helper runs as root in every pacman transaction and wrote into `/var/lib/sysforge/sentinels`, whose parent any `sysforge` group member can write to. A member could swap `sentinels` for a symlink to another directory, which the next transaction made group-writable, or symlink one of its files so the hook appended to any file as root. The helper now enters the directory and acts only if it is the real `sentinels`, owned by root. It appends only to a regular file, opened so that a symlink is refused and a planted FIFO cannot stall the transaction. As before, it never fails a transaction. diff --git a/man/sysforge.1 b/man/sysforge.1 index 9836869a..8c81988d 100644 --- a/man/sysforge.1 +++ b/man/sysforge.1 @@ -239,13 +239,18 @@ Used by: \fBconfig clean\fR, \fBconfig reset\fR.\& .PP .SS build .PP -\fBsysforge build\fR [\fIOPTIONS\fR] \fIPKGBUILD.\&.\&.\&\fR +\fBsysforge build\fR [\fIOPTIONS\fR] \fI[PKGBUILD.\&.\&.\&]\fR .PP Build a package from a PKGBUILD.\& .PP \fBPKGBUILD\fR .RS 4 -One or more packages to build (path, directory, or bare package name).\& +One or more packages to build (path, directory, or bare package name).\& Optional with \-\-failed.\& +.PP +.RE +\fB\-\-failed\fR +.RS 4 +Also build every package recorded by `sysforge state failed`: those whose last build failed, and those refused at the review gate (for example by an unattended run).\& Each entry clears when its package builds.\& With nothing recorded and no PKGBUILD given, does nothing.\& .PP .RE \fB\-\-makepkg\fR, \fB\-m\fR \fIFLAGS\fR @@ -878,11 +883,11 @@ Show which systemd\-boot loader entry boots each installed kernel, who owns it ( .PP \fBsysforge state failed\fR [\fIOPTIONS\fR] .PP -List packages whose last build failed (recorded in build_state.\&toml), with any diagnosed fix.\& Entries auto\-clear on the next successful build, or once the package is removed.\& +List packages whose last build failed or that were refused at the review gate (recorded in build_state.\&toml), with any diagnosed fix.\& `sysforge build \-\-failed` retries them all.\& Entries auto\-clear on the next successful build, or once the package is removed.\& .PP \fB\-\-names\fR .RS 4 -Print only the failed pkgbases, one per line, with no header and no pager (empty when none), for retrying them: sysforge build $(sysforge state failed \-\-names).\& +Print only the recorded pkgbases, one per line, with no header and no pager (empty when none), for scripts.\& To retry them, use `sysforge build \-\-failed`.\& .PP .RE \fB\-\-clear\fR \fIPKGBASE\fR diff --git a/sysforge/CLAUDE.md b/sysforge/CLAUDE.md index d854634b..05aed06c 100644 --- a/sysforge/CLAUDE.md +++ b/sysforge/CLAUDE.md @@ -164,7 +164,8 @@ Mechanism lives in the cited §DESIGN section. both container channels — the `-I` closure and `local_repo.sync`; the ABI gate keeps the narrower `source_built_packages`. A `rust`/`cargo` target swaps `llvm_linked` for the repo LLVM via `-I` only (`fetch_repo_packages`), never by dropping LLVM from the local repo (2026-09-11 mesa). §makepkg-wrapper. -- **Build failures**: reserved `[failures]` namespace in `build_state.toml`; diagnostics in +- **Build failures**: reserved `[failures]` namespace in `build_state.toml` (review refusals too, + `kind = "review"`, via `build_core._record_review_refusal`; `build --failed` retries both, 3.5.0-F23); diagnostics in `build_diag.py` (`_MATCHERS`, on `strip_ansi`-cleaned lines). §Build failures. - **Refused-install revert/restore**: `build_core._revert_uninstalled_state` stashes the undone entry in the reserved `[reverted]` namespace (keyed by artifact filename); `install_built` restores it via diff --git a/sysforge/build_cmd.py b/sysforge/build_cmd.py index a6d2be7e..d3c762a3 100644 --- a/sysforge/build_cmd.py +++ b/sysforge/build_cmd.py @@ -210,6 +210,26 @@ def _cli_toolchain(args) -> str | None: return toolchain_for_compilers(cc, cxx) if cc and cxx else None +def _add_recorded_failures(args) -> None: + """Append every pkgbase in build_state.toml's ``[failures]`` table to + ``args.pkgbuilds`` (3.5.0-F23), after any named ones and without repeats. + + That table holds build failures and, since F23, review refusals, so this is + "everything the last runs could not finish". Sorted for a stable order; + ``build_and_install`` topo-sorts the batch itself. + """ + from sysforge.pipeline.state import resolve_state_dir + from sysforge.primitives.build_state import BuildState + + state_dir, _ = resolve_state_dir(getattr(args, "state_dir", None)) + recorded = sorted(BuildState(state_dir).all_failures()) + named = list(args.pkgbuilds or []) + have = {_pkg_to_name(p) for p in named} + args.pkgbuilds = named + [pb for pb in recorded if pb not in have] + if recorded: + _log.info(f"--failed: {len(recorded)} recorded package(s): {', '.join(recorded)}") + + class BuildVerb(Verb): """Build one or more packages using their matched profiles.""" @@ -221,9 +241,15 @@ class BuildVerb(Verb): def add_parser(cls, sub): p = sub.add_parser("build", help="Build a package from a PKGBUILD.") p.add_argument( - "pkgbuilds", nargs="+", metavar="PKGBUILD", - help="One or more packages to build (path, directory, or bare package name).", + "pkgbuilds", nargs="*", metavar="PKGBUILD", + help="One or more packages to build (path, directory, or bare package name). " + "Optional with --failed.", ) + p.add_argument("--failed", action="store_true", dest="failed", + help="Also build every package recorded by `sysforge state failed`: those " + "whose last build failed, and those refused at the review gate (for " + "example by an unattended run). Each entry clears when its package " + "builds. With nothing recorded and no PKGBUILD given, does nothing.") p.add_argument("--makepkg", "-m", metavar="FLAGS", help="Additional makepkg flags, appended after profile makepkg_flags. " "Usually optional: bare makepkg short flags are forwarded " @@ -329,6 +355,16 @@ def journal_target(self, args) -> str | None: return journal.pkg_target(getattr(args, "packages", None) or []) def pre_check(self, args) -> PreCheckResult: + if getattr(args, "failed", False): + _add_recorded_failures(args) + if not args.pkgbuilds: + _log.ui("nothing to retry: no failed or review-refused packages recorded") + return PreCheckResult(skip_reason="build --failed: nothing recorded") + if not args.pkgbuilds: + return PreCheckResult( + blocker="build needs at least one PKGBUILD, or --failed to retry the " + "packages `sysforge state failed` lists", + exit_code=2) if args.no_pkg_log and args.log_dir: print( "[SYSFORGE] Warning: --log-dir has no effect when --no-pkg-log is set.", diff --git a/sysforge/build_core.py b/sysforge/build_core.py index 65ec46b1..1e2ec3ab 100644 --- a/sysforge/build_core.py +++ b/sysforge/build_core.py @@ -240,6 +240,22 @@ def _record_build_failure(state_dir, target, exc) -> None: _log.warn(f"Failed to record build failure for {target.pkgbase!r}: {e}") +def _record_review_refusal(state_dir, pkgbase: str, why: str) -> None: + """Persist a review-gate refusal as a ``kind = "review"`` failure (3.5.0-F23). + + Without this an unattended run's refusals lived only in its summary, so + nothing could retry them afterwards. Same shape and posture as + :func:`_record_build_failure`: a fresh BuildState, best-effort, cleared by + the next successful build of the pkgbase. + """ + try: + bs_fail = BuildState(_state_dir_path(state_dir)) + bs_fail.record_failure(pkgbase, error=f"refused at review: {why}", kind="review") + bs_fail.save() + except Exception as e: # pragma: no cover - defensive + _log.warn(f"Failed to record review refusal for {pkgbase!r}: {e}") + + # --------------------------------------------------------------------------- # Dependency preparation # --------------------------------------------------------------------------- @@ -1138,6 +1154,7 @@ def build_and_install( if decision == DECISION_REFUSED: outcome.review_refused[t.pkgbase] = ( change_summary(pkgbuild_dir, reviewed) or "unreviewed source change") + _record_review_refusal(state_dir, t.pkgbase, outcome.review_refused[t.pkgbase]) continue if decision == DECISION_ACCEPT: # Saved now, not after the build (3.5.0-B8): accepting is the @@ -1232,6 +1249,7 @@ def build_and_install( hit = names & dep_refused.keys() if hit: outcome.review_refused[t.pkgbase] = dep_refused[sorted(hit)[0]] + _record_review_refusal(state_dir, t.pkgbase, outcome.review_refused[t.pkgbase]) else: still.append(t) targets = still diff --git a/sysforge/fetch.py b/sysforge/fetch.py index 2a35cc31..6f3a6fc5 100644 --- a/sysforge/fetch.py +++ b/sysforge/fetch.py @@ -64,7 +64,6 @@ def cmd_fetch(args) -> int: scheduler = get_scheduler( cleansrc=cleansrc, cleansrc_force=cleansrc_force, - force_devel=getattr(args, "devel", False), repo_track=resolve_repo_track(load_sysforge_toml().get("build", {})), ) diff --git a/sysforge/primitives/aur.py b/sysforge/primitives/aur.py index 8086e874..102fca9d 100644 --- a/sysforge/primitives/aur.py +++ b/sysforge/primitives/aur.py @@ -273,6 +273,30 @@ def is_aur_checkout(pkgbuild_dir) -> bool: _AUR_HOST = urllib.parse.urlsplit(AUR_GIT_BASE).hostname + +def aur_origin_pkgbase(pkgbuild_dir) -> str | None: + """The AUR pkgbase this checkout's ``origin`` clones, else ``None``. + + Strict, unlike :func:`is_aur_checkout` (which fails open for the + AUR-disabled gate): no checkout, no origin, an unreadable one or another + host all give ``None``. The source-sync short-circuit uses it to decide + whether the AUR RPC entry of a name can vouch for a checkout (3.5.0-B144). + """ + d = Path(pkgbuild_dir) + if not (d / ".git").exists(): + return None + try: + proc = run.probe(["git", "-C", str(d), "remote", "get-url", "origin"]) + except OSError: + return None + url = (proc.stdout or "").strip() if proc.returncode == 0 else "" + if not url or _url_host(url) != _AUR_HOST: + return None + path = (urllib.parse.urlsplit(url).path if "://" in url + else url.split(":", 1)[1]).strip("/") + name = path.removesuffix(".git") + return name if name and "/" not in name else None + # Where `pkgctl repo clone` checks out an official package: one project per # pkgbase under this group. _REPO_HOST = "gitlab.archlinux.org" diff --git a/sysforge/primitives/build_state.py b/sysforge/primitives/build_state.py index 1f2869c8..ff45d337 100644 --- a/sysforge/primitives/build_state.py +++ b/sysforge/primitives/build_state.py @@ -548,13 +548,17 @@ def record_failure(self, pkgbase: str, *, error, pkgver: str | None = None, signature: str | None = None, fix_cmd: str | None = None, - failed_at: str | None = None) -> None: + failed_at: str | None = None, + kind: str | None = None) -> None: """Record a build failure for ``pkgbase`` in the ``[failures]`` table. ``error`` is the failure message/exception (stored truncated to its tail). ``signature`` / ``fix_cmd`` come from postflight diagnosis (`build_diag`) when a known pattern matched. Overwrites any prior failure for the same pkgbase. ``failed_at`` defaults to now (ISO-8601). + ``kind = "review"`` marks a package refused at the review gate rather + than one that failed to build (3.5.0-F23); a build failure has no kind. + Both are retried by ``sysforge build --failed``. """ entry = { "failed_at": failed_at or _now_iso(), @@ -566,6 +570,8 @@ def record_failure(self, pkgbase: str, *, error, entry["signature"] = signature if fix_cmd: entry["fix_cmd"] = fix_cmd + if kind: + entry["kind"] = kind self._failures[pkgbase] = entry def clear_failure(self, pkgbase: str) -> bool: @@ -745,12 +751,13 @@ def _serialize(self) -> str: lines.append(f'{key} = "{val}"') lines.append("") if self._failures: - lines.append("# Build failures (cleared on the next successful build).") + lines.append("# Build failures and review refusals (kind = \"review\"); " + "each clears on the next successful build.") lines.append("") for pkgbase, entry in sorted(self._failures.items()): escaped = pkgbase.replace("\\", "\\\\").replace('"', '\\"') lines.append(f'[{_FAILURES_KEY}."{escaped}"]') - for key in ("failed_at", "pkgver", "signature", "fix_cmd", "error"): + for key in ("failed_at", "kind", "pkgver", "signature", "fix_cmd", "error"): if key in entry: val = _toml_escape(entry[key]) lines.append(f'{key} = "{val}"') diff --git a/sysforge/primitives/source_meta.py b/sysforge/primitives/source_meta.py index c9c89451..7ffb4454 100644 --- a/sysforge/primitives/source_meta.py +++ b/sysforge/primitives/source_meta.py @@ -123,6 +123,15 @@ def update( if last_fetch_at is not None: entry["last_fetch_at"] = last_fetch_at + def forget_rpc(self, pkgbase: str) -> None: + """Drop the cached AUR RPC fields for pkgbase, which :meth:`update` + cannot do (it preserves unset fields). Without them the source-sync + short-circuit cannot match, so the next sync fetches (3.5.0-B144).""" + entry = self._data.get(pkgbase) + if entry: + for key in ("rpc_version", "rpc_last_modified", "rpc_package_base"): + entry.pop(key, None) + def mark_rpc_sync(self, timestamp: str | None = None) -> None: """Record the timestamp of the last successful RPC batch query.""" self._meta["last_rpc_at"] = timestamp or _now_iso() diff --git a/sysforge/primitives/source_sync.py b/sysforge/primitives/source_sync.py index 465f346c..19bb7774 100644 --- a/sysforge/primitives/source_sync.py +++ b/sysforge/primitives/source_sync.py @@ -46,7 +46,7 @@ class SourceSyncScheduler: get_scheduler(*, state_dir, offline=False, cleansrc=False, min_fetch_interval_ms=None, rate_limit_abort_s=None, fetch_timeout=None, clone_timeout=None, - force_devel=False, repo_track="stable") -> SourceSyncScheduler + repo_track="stable") -> SourceSyncScheduler ``repo_track`` governs ``source = "repo"`` checkouts: ``"stable"`` (default) pins them to pacman's sync-DB release tag after every clone/fetch (detached @@ -71,6 +71,7 @@ class SourceSyncScheduler: GitFetchOutcome, aur_clone, aur_info, + aur_origin_pkgbase, git_fetch_and_compare, git_is_dirty, is_aur_checkout, @@ -85,6 +86,7 @@ class SourceSyncScheduler: from sysforge.primitives.net_policy import ( KIND_AUR_CLONE, KIND_REPO_CHECKOUT, + KIND_SOURCE_FETCH, NetworkFrozen, get_policy, ) @@ -181,7 +183,6 @@ def __init__( offline: bool = False, cleansrc: bool = False, cleansrc_force: bool = False, - force_devel: bool = False, repo_track: str = "stable", min_fetch_interval_ms: int | None = None, rate_limit_abort_s: float | None = None, @@ -195,7 +196,6 @@ def __init__( # dirty-tree refusal bypassed at purge_src. self.cleansrc = cleansrc or cleansrc_force self.cleansrc_force = cleansrc_force - self.force_devel = force_devel self.rate_limit_abort_s = float(rate_limit_abort_s or 120.0) self.fetch_timeout = fetch_timeout if fetch_timeout is not None else 30 self.clone_timeout = clone_timeout if clone_timeout is not None else 60 @@ -398,13 +398,29 @@ def _sync_one(self, req: SyncRequest) -> SyncResult: # Repo packages have no AUR-RPC equivalent — they fall through to # the generic fetch path, which works because pkgctl-clones are # plain git repos with a tracking branch. + # --devel is not a reason to fetch here (3.5.0-B144): it asks whether + # *upstream* moved, which the version check answers itself (ls-remote, + # pkgver()). Whether the AUR packaging repo moved is what the RPC says, + # for a VCS package as for any other. The RPC is looked up by name, so + # it only vouches for a checkout of that pkgbase's AUR repo + # (_rpc_vouches): a fork, another package's repo or a missing origin + # always fetches. meta = self.cache.get(pkgbase) rpc_entry = self._rpc_entry(pkgbase) if req.source != "repo" else None local_head = _head_commit(pkgbuild_dir) - force = req.force_fetch or self.cleansrc or ( - self.force_devel and _is_vcs(pkgbase) - ) - if not force and self._can_short_circuit(rpc_entry, meta, local_head): + force = req.force_fetch or self.cleansrc + if (not force and self._can_short_circuit(rpc_entry, meta, local_head) + and _rpc_vouches(pkgbuild_dir, rpc_entry, pkgbase)): + if _is_vcs(pkgbase): + # A VCS package is synced only under --devel, whose version + # check needs upstream; the freeze denies that there, so say + # so here, as the fetch this skips used to (3.5.0-B144). + try: + get_policy().check(KIND_SOURCE_FETCH, pkgbase) + except NetworkFrozen as e: + return SyncResult(pkgbase=pkgbase, status=STATUS_FROZEN, + head_before=local_head, head_after=local_head, + error=str(e)) # Refresh last_fetch_at so the entry doesn't look stale. self.cache.update(pkgbase, last_fetch_at=_now_iso()) return SyncResult( @@ -431,6 +447,21 @@ def _reclone_blocked(self, pkgbase: str, source: str | None) -> SyncResult | Non error=f"--cleansrc kept the checkout: {e}") return None + def _record_rpc(self, pkgbase: str, pkgbuild_dir: Path, rpc_entry: dict | None) -> None: + """Cache the RPC entry beside the head just synced, only if it vouches + for the checkout; otherwise clear any cached one. AUR metadata kept + beside a fork's head would let a later switch of origin to the AUR + skip its first fetch and keep building the fork (3.5.0-B144).""" + if rpc_entry is not None and _rpc_vouches(pkgbuild_dir, rpc_entry, pkgbase): + self.cache.update( + pkgbase, + rpc_version=rpc_entry.get("Version"), + rpc_last_modified=rpc_entry.get("LastModified"), + rpc_package_base=rpc_entry.get("PackageBase"), + ) + else: + self.cache.forget_rpc(pkgbase) + def _can_short_circuit( self, rpc_entry: dict | None, meta: dict | None, local_head: str | None, ) -> bool: @@ -480,11 +511,9 @@ def _clone(self, pkgbase: str, pkgbuild_dir: Path, head = _head_commit(pkgbuild_dir) rpc_entry = self._rpc_entry(pkgbase) if source != "repo" else None + self._record_rpc(pkgbase, pkgbuild_dir, rpc_entry) self.cache.update( pkgbase, - rpc_version=(rpc_entry or {}).get("Version"), - rpc_last_modified=(rpc_entry or {}).get("LastModified"), - rpc_package_base=(rpc_entry or {}).get("PackageBase"), head_commit=head, is_vcs=_is_vcs(pkgbase), last_fetch_at=_now_iso(), @@ -629,11 +658,9 @@ def _fetch( ) if outcome.status in ("up_to_date", "fetched"): + self._record_rpc(pkgbase, pkgbuild_dir, rpc_entry) self.cache.update( pkgbase, - rpc_version=(rpc_entry or {}).get("Version"), - rpc_last_modified=(rpc_entry or {}).get("LastModified"), - rpc_package_base=(rpc_entry or {}).get("PackageBase"), head_commit=outcome.head_after, is_vcs=_is_vcs(pkgbase), last_fetch_at=_now_iso(), @@ -661,6 +688,19 @@ def _fetch( ) +def _rpc_vouches(pkgbuild_dir: Path, rpc_entry: dict | None, pkgbase: str) -> bool: + """Whether the AUR RPC entry describes this checkout (3.5.0-B144). + + The entry is matched by name, so it describes the AUR package of that + name. It speaks for the checkout only when ``origin`` is that package's + AUR repo: not a fork, not another package's repo (an exact name match can + be a member of a different pkgbase), not a checkout with no origin. + """ + if rpc_entry is None: + return False + return aur_origin_pkgbase(pkgbuild_dir) == rpc_entry.get("PackageBase", pkgbase) + + def _head_commit(pkgbuild_dir: Path) -> str | None: r = run.probe(["git", "-C", str(pkgbuild_dir), "rev-parse", "HEAD"]) if r.returncode != 0: @@ -718,7 +758,6 @@ def get_scheduler( offline: bool = False, cleansrc: bool = False, cleansrc_force: bool = False, - force_devel: bool = False, repo_track: str = "stable", min_fetch_interval_ms: int | None = None, rate_limit_abort_s: float | None = None, @@ -728,7 +767,7 @@ def get_scheduler( """Return the per-process scheduler, constructing it on first call. Subsequent calls update mutable runtime flags (``offline`` / ``cleansrc`` - / ``cleansrc_force`` / ``force_devel`` / ``repo_track``) on the existing + / ``cleansrc_force`` / ``repo_track``) on the existing instance so callers in different commands (e.g. `sysforge fetch` after `sysforge update`) can share the metadata cache without re-reading @@ -744,7 +783,6 @@ def get_scheduler( offline=offline, cleansrc=cleansrc, cleansrc_force=cleansrc_force, - force_devel=force_devel, repo_track=repo_track, min_fetch_interval_ms=min_fetch_interval_ms, rate_limit_abort_s=rate_limit_abort_s, @@ -756,7 +794,6 @@ def get_scheduler( _scheduler.offline = offline or _scheduler.offline _scheduler.cleansrc = cleansrc or cleansrc_force or _scheduler.cleansrc _scheduler.cleansrc_force = cleansrc_force or _scheduler.cleansrc_force - _scheduler.force_devel = force_devel or _scheduler.force_devel if repo_track != "stable": _scheduler.repo_track = repo_track return _scheduler diff --git a/sysforge/primitives/state_probe.py b/sysforge/primitives/state_probe.py index be4e5998..0a9a6b50 100644 --- a/sysforge/primitives/state_probe.py +++ b/sysforge/primitives/state_probe.py @@ -156,15 +156,24 @@ def collect_state_result(state_dir: Path | str | None = None, when = rec.get("failed_at", "?") fix_cmd = rec.get("fix_cmd") signature = rec.get("signature") - detail = f"{pkgbase} build failed ({when})" - if signature: - detail += f" [{signature}]" - if msg: - detail += f": {msg}" - remediation = fix_cmd or ( - f"inspect with `sysforge log {pkgbase}`; clears on the next successful build " - "or once the package is removed" - ) + if rec.get("kind") == "review": + # Refused at the review gate, usually by an unattended run (3.5.0-F23): + # nothing failed, the change is waiting for a person. + detail = f"{pkgbase} refused at review ({when})" + if msg: + detail += f": {msg.removeprefix('refused at review: ')}" + remediation = ("review and build it at a terminal: `sysforge build --failed`; " + "clears once it builds or the package is removed") + else: + detail = f"{pkgbase} build failed ({when})" + if signature: + detail += f" [{signature}]" + if msg: + detail += f": {msg}" + remediation = fix_cmd or ( + f"inspect with `sysforge log {pkgbase}`; clears on the next successful " + "build or once the package is removed" + ) # Intentional-revert escape hatch: the warning is correct but # dead-ends a user who deliberately went back to the repo package. remediation += ( diff --git a/sysforge/state_cmd.py b/sysforge/state_cmd.py index 1b9633a1..4818633f 100644 --- a/sysforge/state_cmd.py +++ b/sysforge/state_cmd.py @@ -573,7 +573,9 @@ def cmd_state_failed(args): rows.append(( pkgbase, rec.get("failed_at", ""), - rec.get("signature", ""), + # A review refusal (3.5.0-F23) has no diagnosis signature; name + # the kind there so it reads apart from a build failure. + rec.get("signature", "") or ("review" if rec.get("kind") == "review" else ""), err, rec.get("fix_cmd", ""), )) @@ -600,7 +602,10 @@ def cmd_state_failed(args): f" {'':<{max_base}} {'':<{max_when}} " f"{'':<{max_sig}} {log.green(f'fix: {fix}')}" ) - print(f"\n {len(rows)} failed package(s)") + n_review = sum(1 for rec in failures.values() if rec.get("kind") == "review") + print(f"\n {len(rows)} failed package(s)" + + (f", {n_review} of them refused at review" if n_review else "")) + print(" retry them with `sysforge build --failed`") print( " (entries clear on the next successful build or on removal; " "or use --clear PKGBASE / --clear-all)" @@ -994,17 +999,18 @@ class StateFailedVerb(Verb): @classmethod def add_parser(cls, sub): p = sub.add_parser("failed", - help="List packages whose last build failed (recorded in build_state.toml), " - "with any diagnosed fix. Entries auto-clear on the next successful build, " - "or once the package is removed.") + help="List packages whose last build failed or that were refused at the " + "review gate (recorded in build_state.toml), with any diagnosed fix. " + "`sysforge build --failed` retries them all. Entries auto-clear on the " + "next successful build, or once the package is removed.") p.add_argument("--state-dir", metavar="DIR", dest="state_dir", help="Override state directory.") p.add_argument("--no-pager", action="store_true", dest="no_pager", help="Don't pipe output through $PAGER (default: paginate when stdout is a TTY).") p.add_argument("--names", action="store_true", dest="names", - help="Print only the failed pkgbases, one per line, with no header and no " - "pager (empty when none), for retrying them: " - "sysforge build $(sysforge state failed --names).") + help="Print only the recorded pkgbases, one per line, with no header and no " + "pager (empty when none), for scripts. To retry them, use " + "`sysforge build --failed`.") p.add_argument("--clear", metavar="PKGBASE", dest="clear", help="Clear the recorded failure for PKGBASE and exit.") p.add_argument("--clear-all", action="store_true", dest="clear_all", diff --git a/sysforge/update_summary.py b/sysforge/update_summary.py index 8b7acfb8..6022ae37 100644 --- a/sysforge/update_summary.py +++ b/sysforge/update_summary.py @@ -328,8 +328,8 @@ def _section(label: str, lines: list[str]) -> None: if summary.review_refused: _section("Refused at review:", [ f"{pb}: {why}" for pb, why in summary.review_refused.items()]) - emit(" run `sysforge update` at a terminal to review them " - "([build] review in packages.toml)") + emit(" run `sysforge build --failed` at a terminal to review and build " + "them ([build] review in packages.toml)") if summary.review_accepted: _section("Accepted without review:", list(summary.review_accepted)) @@ -355,6 +355,8 @@ def _section(label: str, lines: list[str]) -> None: if summary.failed_pkgs: _section("Failed:", [_fmt_pkg(summary, pb) for pb in summary.failed_pkgs]) + emit(" `sysforge state failed` shows any diagnosed fix; after applying it, " + "`sysforge build --failed` retries only these") if summary.layout_findings: _section("Payload layout (installed, but never read by its consumer):", diff --git a/sysforge/update_sync.py b/sysforge/update_sync.py index 380ccc44..b3c34f15 100644 --- a/sysforge/update_sync.py +++ b/sysforge/update_sync.py @@ -87,7 +87,6 @@ def _sync_sources( offline=offline, cleansrc=cleansrc, cleansrc_force=cleansrc_force, - force_devel=force_devel, repo_track=resolve_repo_track(sysforge_toml.get("build", {})), min_fetch_interval_ms=aur_cfg.get("min_fetch_interval_ms", 500), rate_limit_abort_s=aur_cfg.get("rate_limit_abort_s", 120.0), diff --git a/tests/conftest.py b/tests/conftest.py index 0a6d237b..6de99e59 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -1098,7 +1098,7 @@ def all(self): return {} class _FakeScheduler: - offline = cleansrc = cleansrc_force = force_devel = False + offline = cleansrc = cleansrc_force = False cache = _FakeCache() def _ensure_rpc(self, bases, members=None): diff --git a/tests/data/surface.json b/tests/data/surface.json index fc974315..20a51a5a 100644 --- a/tests/data/surface.json +++ b/tests/data/surface.json @@ -88,6 +88,7 @@ "--cleansrc": null, "--cleansrc-force": null, "--cxx": null, + "--failed": null, "--force": null, "--interactive": null, "--ld": null, diff --git a/tests/test_build_core.py b/tests/test_build_core.py index 998eaa12..b96a8013 100644 --- a/tests/test_build_core.py +++ b/tests/test_build_core.py @@ -1556,6 +1556,12 @@ def test_review_refused_target_dropped_rest_build(tmp_path, monkeypatch): assert outcome.review_refused == {"foo": "unreviewed source change (a → b)"} assert not outcome.aborted assert calls == [bar.pkgbuild_path] + # 3.5.0-F23: the refusal outlives the run, for `sysforge build --failed`. + from sysforge.primitives.build_state import BuildState + rec = BuildState(tmp_path / "state").all_failures() + assert rec["foo"]["kind"] == "review" + assert "a → b" in rec["foo"]["error"] + assert "bar" not in rec def test_review_trusted_source_passes_trusted_true(tmp_path): diff --git a/tests/test_retry_failed.py b/tests/test_retry_failed.py new file mode 100644 index 00000000..cf13bf71 --- /dev/null +++ b/tests/test_retry_failed.py @@ -0,0 +1,133 @@ +# SPDX-FileCopyrightText: 2026 Keith Raghubar +# +# SPDX-License-Identifier: MIT + +""" +Retrying what a run could not finish (3.5.0-F23). + +A build failure was already recorded in build_state.toml's ``[failures]`` +table. A package refused at the review gate (an unreviewed source change on a +run that could not ask, usually unattended) is now recorded there too, with +``kind = "review"``, so it survives the run. ``sysforge build --failed`` +rebuilds every recorded entry; a successful build clears it, as before. +""" +from pathlib import Path +from types import SimpleNamespace + +import pytest + +from cli_args import cli_args + +import sysforge.build_cmd as build_cmd +import sysforge.build_core as build_core +from sysforge.primitives.build_state import BuildState +from sysforge.state_cmd import cmd_state_failed + + +def _seed(state_dir: Path, pkgbase: str, **fields): + state_dir.mkdir(parents=True, exist_ok=True) + bs = BuildState(state_dir) + bs.record_failure(pkgbase, **fields) + bs.save() + + +# --------------------------------------------------------------------------- +# build_state: review refusals are failure entries with kind = "review" +# --------------------------------------------------------------------------- + +def test_review_kind_survives_a_save_and_reload(tmp_path): + _seed(tmp_path, "foo-git", error="unreviewed source change", kind="review") + entry = BuildState(tmp_path).all_failures()["foo-git"] + assert entry["kind"] == "review" + assert entry["error"] == "unreviewed source change" + + +def test_a_build_failure_carries_no_kind(tmp_path): + _seed(tmp_path, "foo-git", error="[build_failed] exit 4") + assert "kind" not in BuildState(tmp_path).all_failures()["foo-git"] + + +def test_record_review_refusal_persists_it(tmp_path): + build_core._record_review_refusal(tmp_path, "foo-git", "PKGBUILD changed") + entry = BuildState(tmp_path).all_failures()["foo-git"] + assert entry["kind"] == "review" + assert "PKGBUILD changed" in entry["error"] + + +# --------------------------------------------------------------------------- +# state failed lists both kinds, telling them apart +# --------------------------------------------------------------------------- + +def _state_args(state_dir: Path, **kw): + base = dict(state_dir=str(state_dir), no_pager=True, clear=None, + clear_all=False, names=False) + base.update(kw) + return SimpleNamespace(**base) + + +def test_state_failed_marks_a_review_refusal(tmp_path, capsys): + _seed(tmp_path, "foo-git", error="PKGBUILD changed", kind="review") + _seed(tmp_path, "bar-git", error="[build_failed] exit 4") + cmd_state_failed(_state_args(tmp_path)) + out = capsys.readouterr().out + assert "foo-git" in out and "refused at review" in out + assert "bar-git" in out + assert "sysforge build --failed" in out + + +# --------------------------------------------------------------------------- +# build --failed +# --------------------------------------------------------------------------- + +def test_build_takes_failed_without_package_names(): + args = cli_args("build", "--failed") + assert args.failed is True + assert args.pkgbuilds == [] + + +def test_build_needs_packages_or_failed(): + verb = build_cmd.BuildVerb() + pre = verb.pre_check(cli_args("build")) + assert pre.blocker and "--failed" in pre.blocker + assert pre.exit_code == 2 + + +def test_failed_adds_every_recorded_entry(tmp_path): + _seed(tmp_path, "foo-git", error="PKGBUILD changed", kind="review") + _seed(tmp_path, "bar-git", error="[build_failed] exit 4") + args = cli_args("build", "--failed", "--state-dir", str(tmp_path)) + build_cmd._add_recorded_failures(args) + assert args.pkgbuilds == ["bar-git", "foo-git"] + + +def test_failed_keeps_named_packages_and_dedups(tmp_path): + _seed(tmp_path, "foo-git", error="[build_failed] exit 4") + args = cli_args("build", "foo-git", "baz", "--failed", "--state-dir", str(tmp_path)) + build_cmd._add_recorded_failures(args) + assert args.pkgbuilds == ["foo-git", "baz"] + + +def test_failed_with_nothing_recorded_is_a_clean_no_op(tmp_path, capsys): + verb = build_cmd.BuildVerb() + pre = verb.pre_check(cli_args("build", "--failed", "--state-dir", str(tmp_path))) + assert pre.blocker is None + assert pre.skip_reason is not None + cap = capsys.readouterr() + assert "nothing to retry" in cap.out + cap.err + + +# --------------------------------------------------------------------------- +# The update summary points at it +# --------------------------------------------------------------------------- + +@pytest.mark.parametrize("field", ["failed_pkgs", "review_refused"]) +def test_update_summary_suggests_build_failed(field): + from sysforge.update_summary import ResultSummary, _print_result_summary + summary = ResultSummary() + if field == "failed_pkgs": + summary.failed_pkgs = ["foo-git"] + else: + summary.review_refused = {"foo-git": "PKGBUILD changed"} + lines = [] + _print_result_summary(summary, emit=lines.append) + assert "sysforge build --failed" in "\n".join(lines) diff --git a/tests/test_source_sync.py b/tests/test_source_sync.py index 051835dd..47b5ca04 100644 --- a/tests/test_source_sync.py +++ b/tests/test_source_sync.py @@ -5,7 +5,8 @@ SourceSyncScheduler RPC short-circuit when version+last_modified+head all match RPC miss triggers shallow fetch - VCS packages always fetch when --devel is on + VCS packages short-circuit under --devel like any other (3.5.0-B144) + a non-AUR origin never short-circuits missing dir → _clone path divergence surfaced (not fixed) rate-limited status propagates + aborts remainder @@ -19,6 +20,8 @@ from pathlib import Path from unittest.mock import patch +import pytest + from sysforge.primitives.aur import GitFetchOutcome from sysforge.primitives.source_sync import ( STATUS_CLONED, @@ -46,6 +49,17 @@ def _make_repo(parent: Path, name: str) -> Path: return d +def _make_aur_repo(parent: Path, name: str, origin: str | None = "aur") -> Path: + """A real git checkout whose origin is the AUR repo of ``name`` (the default), + any other URL, or absent (``None``): the short-circuit reads it (3.5.0-B144).""" + d = _make_repo(parent, name) + subprocess.run(["git", "init", "-q", str(d)], check=True) + if origin is not None: + url = f"https://aur.archlinux.org/{name}.git" if origin == "aur" else origin + subprocess.run(["git", "-C", str(d), "remote", "add", "origin", url], check=True) + return d + + def _scheduler(tmp_path, **kwargs) -> SourceSyncScheduler: kwargs.setdefault("fetch_timeout", 30) kwargs.setdefault("clone_timeout", 60) @@ -57,7 +71,7 @@ def _scheduler(tmp_path, **kwargs) -> SourceSyncScheduler: # --------------------------------------------------------------------------- def test_rpc_short_circuit_when_cache_matches(tmp_path): - pkg = _make_repo(tmp_path, "htop") + pkg = _make_aur_repo(tmp_path, "htop") sched = _scheduler(tmp_path) sched._rpc_done = True sched._rpc_map = {"htop": {"Version": "3.3-1", "LastModified": 1000}} @@ -105,7 +119,7 @@ def test_split_pkgbase_resolves_its_rpc_entry_through_a_member(tmp_path): def test_split_pkgbase_short_circuits_and_records_its_rpc_version(tmp_path): - pkg = _make_repo(tmp_path, "lib32-libdbusmenu") + pkg = _make_aur_repo(tmp_path, "lib32-libdbusmenu") sched = _scheduler(tmp_path) sched.cache.update("lib32-libdbusmenu", rpc_version="16.04.0-6", rpc_last_modified=1700, head_commit="abc") @@ -129,7 +143,7 @@ def test_plain_pkgbase_is_still_keyed_by_name(tmp_path): def test_rpc_miss_triggers_fetch(tmp_path): - pkg = _make_repo(tmp_path, "htop") + pkg = _make_aur_repo(tmp_path, "htop") sched = _scheduler(tmp_path) sched._rpc_done = True sched._rpc_map = {"htop": {"Version": "3.4-1", "LastModified": 2000}} @@ -153,26 +167,155 @@ def test_rpc_miss_triggers_fetch(tmp_path): assert entry["head_commit"] == "def" -def test_vcs_pkg_force_fetches_under_devel(tmp_path): - pkg = _make_repo(tmp_path, "mesa-git") - sched = _scheduler(tmp_path, force_devel=True) +def _cached_vcs(tmp_path, *, rpc=("v1-1", 100), origin="aur", package_base="mesa-git"): + """A VCS pkg (synced only under --devel) whose cache matches RPC + ``v1-1``/``100`` at head ``abc``, checked out from ``origin``.""" + pkg = _make_aur_repo(tmp_path, "mesa-git", origin=origin) + sched = _scheduler(tmp_path) sched._rpc_done = True - sched._rpc_map = {"mesa-git": {"Version": "v1-1", "LastModified": 100}} + sched._rpc_map = {"mesa-git": {"Version": rpc[0], "LastModified": rpc[1], + "PackageBase": package_base}} sched.cache.update( "mesa-git", rpc_version="v1-1", rpc_last_modified=100, head_commit="abc", ) + return pkg, sched - outcome = GitFetchOutcome(status="up_to_date", head_before="abc", head_after="abc") - with patch("sysforge.primitives.source_sync._head_commit", return_value="abc"), \ + +def _request_vcs(pkg, sched, *, head="abc"): + outcome = GitFetchOutcome(status="up_to_date", head_before=head, head_after=head) + with patch("sysforge.primitives.source_sync._head_commit", return_value=head), \ patch("sysforge.primitives.source_sync.git_fetch_and_compare", return_value=outcome) as fetch: result = sched.request(SyncRequest(pkgbase="mesa-git", pkgbuild_dir=pkg)) + return result, fetch + + +def test_vcs_pkg_short_circuits_under_devel_when_the_aur_repo_is_unchanged(tmp_path): + """--devel checks upstream in the version check (ls-remote / pkgver()); + re-fetching an unchanged AUR packaging repo bought nothing (3.5.0-B144).""" + pkg, sched = _cached_vcs(tmp_path) + result, fetch = _request_vcs(pkg, sched) + fetch.assert_not_called() + assert result.status == STATUS_UP_TO_DATE + + +def test_vcs_pkg_fetches_under_devel_after_an_aur_push(tmp_path): + pkg, sched = _cached_vcs(tmp_path, rpc=("v1-1", 200)) + _, fetch = _request_vcs(pkg, sched) + fetch.assert_called_once() + - # --devel bypasses the RPC short-circuit for VCS packages. +def test_vcs_pkg_fetches_under_devel_when_the_local_head_moved(tmp_path): + pkg, sched = _cached_vcs(tmp_path) + _, fetch = _request_vcs(pkg, sched, head="local-commit") fetch.assert_called_once() + + +def test_scp_form_aur_origin_short_circuits(tmp_path): + """The form AUR maintainers clone with is still the AUR.""" + pkg, sched = _cached_vcs(tmp_path, origin="aur@aur.archlinux.org:mesa-git.git") + _, fetch = _request_vcs(pkg, sched) + fetch.assert_not_called() + + +@pytest.mark.parametrize("origin", [ + "https://github.com/someone/mesa-git.git", # a fork of the same name + "https://aur.archlinux.org/other-pkg.git", # the AUR, but another package + None, # no origin to compare at all +]) +def test_rpc_only_vouches_for_the_aur_repo_of_that_pkgbase(tmp_path, origin): + """The RPC entry is matched by name, so it describes the AUR package + ``mesa-git``; it cannot vouch for a checkout of anything else (3.5.0-B144).""" + pkg, sched = _cached_vcs(tmp_path, origin=origin) + _, fetch = _request_vcs(pkg, sched) + fetch.assert_called_once() + + +def test_rpc_entry_of_another_pkgbase_does_not_vouch(tmp_path): + """An exact name match can be a member of a different AUR pkgbase.""" + pkg, sched = _cached_vcs(tmp_path, package_base="mesa-all-git") + _, fetch = _request_vcs(pkg, sched) + fetch.assert_called_once() + + +def test_a_fork_fetch_does_not_cache_the_aur_metadata(tmp_path): + """Switching a checkout's origin from a fork back to the AUR must not + leave AUR metadata cached beside the fork's head, or the next run would + skip the fetch and keep building the fork (3.5.0-B144).""" + pkg, sched = _cached_vcs(tmp_path, origin="https://github.com/someone/mesa-git.git") + _, fetch = _request_vcs(pkg, sched, head="fork-head") + fetch.assert_called_once() + assert sched.cache.get("mesa-git").get("rpc_version") is None + subprocess.run(["git", "-C", str(pkg), "remote", "set-url", "origin", + "https://aur.archlinux.org/mesa-git.git"], check=True) + sched._results.clear() + _, fetch = _request_vcs(pkg, sched, head="fork-head") + fetch.assert_called_once() + + +def test_vcs_skip_under_the_source_freeze_reports_frozen(tmp_path): + """A VCS package that skips the fetch still needs upstream in the version + check, which the freeze denies: report it frozen here, as the forced fetch + used to, so `update --devel --frozen` still exits 1 (3.5.0-B144).""" + from sysforge.primitives import net_policy + from sysforge.primitives.source_sync import STATUS_FROZEN + pkg, sched = _cached_vcs(tmp_path) + net_policy.set_policy(net_policy.NetPolicy(frozen=True, thawed=frozenset())) + try: + result, fetch = _request_vcs(pkg, sched) + finally: + net_policy.reset_policy() + fetch.assert_not_called() + assert result.status == STATUS_FROZEN + + +def test_vcs_skip_under_the_freeze_honours_thaw(tmp_path): + from sysforge.primitives import net_policy + pkg, sched = _cached_vcs(tmp_path) + net_policy.set_policy(net_policy.NetPolicy(frozen=True, thawed=frozenset({"mesa-git"}))) + try: + result, fetch = _request_vcs(pkg, sched) + finally: + net_policy.reset_policy() + fetch.assert_not_called() assert result.status == STATUS_UP_TO_DATE +def test_repo_source_never_consults_the_rpc(tmp_path): + """pkgctl checkouts have no RPC entry, so they always fetch and the origin + is not even looked at.""" + pkg = _make_aur_repo(tmp_path, "htop", origin="https://gitlab.archlinux.org/archlinux/packaging/packages/htop.git") + sched = _scheduler(tmp_path) + sched._rpc_done = True + sched._rpc_map = {"htop": {"Version": "3.3-1", "LastModified": 1000, "PackageBase": "htop"}} + sched.cache.update("htop", rpc_version="3.3-1", rpc_last_modified=1000, head_commit="abc") + outcome = GitFetchOutcome(status="up_to_date", head_before="abc", head_after="abc") + with patch("sysforge.primitives.source_sync._head_commit", return_value="abc"), \ + patch("sysforge.primitives.source_sync.aur_origin_pkgbase", + side_effect=AssertionError("origin read for a repo source")), \ + patch.object(sched, "_pin_repo_checkout", return_value=None), \ + patch("sysforge.primitives.source_sync.git_fetch_and_compare", + return_value=outcome) as fetch: + sched.request(SyncRequest(pkgbase="htop", pkgbuild_dir=pkg, source="repo")) + fetch.assert_called_once() + + +def test_non_vcs_pkg_from_a_non_aur_origin_fetches(tmp_path): + pkg = _make_aur_repo(tmp_path, "htop", origin="https://github.com/someone/htop.git") + sched = _scheduler(tmp_path) + sched._rpc_done = True + sched._rpc_map = {"htop": {"Version": "3.3-1", "LastModified": 1000}} + sched.cache.update( + "htop", rpc_version="3.3-1", rpc_last_modified=1000, head_commit="abc", + ) + outcome = GitFetchOutcome(status="up_to_date", head_before="abc", head_after="abc") + with patch("sysforge.primitives.source_sync._head_commit", return_value="abc"), \ + patch("sysforge.primitives.source_sync.git_fetch_and_compare", + return_value=outcome) as fetch: + sched.request(SyncRequest(pkgbase="htop", pkgbuild_dir=pkg, source="git")) + fetch.assert_called_once() + + def test_force_fetch_bypasses_short_circuit(tmp_path): pkg = _make_repo(tmp_path, "htop") sched = _scheduler(tmp_path) diff --git a/tests/test_state_probe.py b/tests/test_state_probe.py index e9f3fd51..683562ec 100644 --- a/tests/test_state_probe.py +++ b/tests/test_state_probe.py @@ -204,3 +204,16 @@ def test_current_state_has_no_legacy_finding(monkeypatch, tmp_path): monkeypatch.setenv("SYSFORGE_PROFILE_STORE", str(tmp_path)) bs = _FakeBS({"mesa-sysforge": {"build_mode": "pgo"}}) assert state_probe._check_legacy_state(bs) == [] + + +def test_review_refusal_is_not_reported_as_a_build_failure(monkeypatch): + """A refusal recorded by an unattended run (3.5.0-F23) says what happened + and how to finish it, not that a build failed.""" + _setup(monkeypatch, failures={ + "foo-git": {"failed_at": "2026-10-24T00:00:00Z", "kind": "review", + "error": "refused at review: PKGBUILD changed"}, + }) + f = state_probe.collect_state_findings(installed={})[0] + assert "refused at review" in f.message + assert "build failed" not in f.message + assert "sysforge build --failed" in f.remediation diff --git a/tests/test_update.py b/tests/test_update.py index 9255e9a0..54eca1d9 100644 --- a/tests/test_update.py +++ b/tests/test_update.py @@ -1249,6 +1249,23 @@ def test_devel_flag_triggers_vcs_rebuild(update_scenario): assert len(builds) == 1 +def test_devel_rebuilds_when_only_upstream_moved(update_scenario): + """A new upstream commit does not touch the AUR PKGBUILD, so source sync + short-circuits on the unchanged RPC and fetches nothing (3.5.0-B144). The + version check resolves pkgver() against upstream regardless, so the + package still rebuilds.""" + update_scenario.add_pkg( + "neovim-git", "pkgname=neovim-git\npkgver=r1234.gabcdef\npkgrel=1\n") + update_scenario.fake_sync({"neovim-git": "up_to_date"}) + update_scenario.fake_vcs_pkgver("neovim-git", "r5678.g9999999-1") + builds = update_scenario.run( + _make_args(devel=True, offline=False), + installed={"neovim-git": "r1234.gabcdef-1"}, + foreign={"neovim-git": "r1234.gabcdef-1"}, + ) + assert len(builds) == 1 + + def test_devel_skips_uptodate_vcs(update_scenario): """--devel + resolved pkgver equal to installed → build does NOT run.""" update_scenario.add_pkg( @@ -1438,7 +1455,7 @@ def test_sync_sources_skips_vcs_without_devel(tmp_path, monkeypatch): seen: list[str] = [] class _FakeScheduler: - offline = cleansrc = cleansrc_force = force_devel = False + offline = cleansrc = cleansrc_force = False cache = MagicMock() def _ensure_rpc(self, bases, members=None): # noqa: ARG002 pass @@ -1478,7 +1495,7 @@ def test_sync_sources_includes_vcs_under_devel(tmp_path, monkeypatch): seen: list[str] = [] class _FakeScheduler: - offline = cleansrc = cleansrc_force = force_devel = False + offline = cleansrc = cleansrc_force = False cache = MagicMock() def _ensure_rpc(self, bases, members=None): # noqa: ARG002 pass @@ -1510,7 +1527,7 @@ def test_sync_sources_primes_the_rpc_with_split_members(tmp_path, monkeypatch): reqs: list = [] class _FakeScheduler: - offline = cleansrc = cleansrc_force = force_devel = False + offline = cleansrc = cleansrc_force = False cache = MagicMock() def _ensure_rpc(self, bases, members=None): primed.append((bases, members)) diff --git a/tests/test_update_summary.py b/tests/test_update_summary.py index c31f5659..bd00a5d7 100644 --- a/tests/test_update_summary.py +++ b/tests/test_update_summary.py @@ -347,5 +347,5 @@ def test_refused_and_accepted_sections(monkeypatch): assert "1 refused at review" in lines[0] assert "Refused at review:" in joined assert "foo-git: unreviewed source change (a → b)" in joined - assert "run `sysforge update` at a terminal to review them" in joined + assert "run `sysforge build --failed` at a terminal to review and build them" in joined assert "Accepted without review:" in joined and "bar" in joined