diff --git a/.claude/agents/pkgbuild-reviewer.md b/.claude/agents/pkgbuild-reviewer.md index 6a7981ec..253fb67b 100644 --- a/.claude/agents/pkgbuild-reviewer.md +++ b/.claude/agents/pkgbuild-reviewer.md @@ -20,7 +20,7 @@ You review changes to: - `sysforge/primitives/vcs_pkgver.py` — `[epoch:]pkgver-pkgrel` resolution for VCS sources - `sysforge/primitives/source_meta.py` — `source_meta.toml` schema - `sysforge/primitives/build_state.py` — `build_state.toml` schema -- `sysforge/update.py`, `sysforge/fetch.py`, `sysforge/converge.py` — call sites +- `sysforge/update.py`, `sysforge/fetch.py` — call sites Reject the review request if the diff falls outside this scope; the user wants a generalist reviewer instead. diff --git a/.claude/skills/pkgbuild-spec-check/SKILL.md b/.claude/skills/pkgbuild-spec-check/SKILL.md index 7722912d..3384df54 100644 --- a/.claude/skills/pkgbuild-spec-check/SKILL.md +++ b/.claude/skills/pkgbuild-spec-check/SKILL.md @@ -70,7 +70,7 @@ Trigger this whenever a change touches: - [ ] **Source sync goes through the scheduler** (CLAUDE.md gotcha #3). Any new code path that needs a fresh PKGBUILD calls `sysforge.primitives.source_sync.get_scheduler().request(SyncRequest(...))` — never `git pull` directly. - [ ] **`build_state.toml` invariants.** If you touch what's parsed from PKGBUILD, check whether the cached `build_state.toml` schema needs to evolve too. -- [ ] **Tests.** `tests/test_parser.py`, `tests/test_patcher.py`, `tests/test_pkgbuild_meta.py` (if present), and `tests/test_aur.py` are the regression nets — add or update cases that exercise the spec detail you just changed. +- [ ] **Tests.** `tests/test_parser.py`, `tests/test_patcher.py` and `tests/test_aur.py` are the regression nets — add or update cases that exercise the spec detail you just changed. ## Reporting diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 00000000..769767e6 --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,11 @@ + + +## Checklist + +CI checks most of this; the boxes are for what it cannot see. + +- [ ] Release-note entry in `docs/release-notes/unreleased.md`, led by its roadmap ID +- [ ] If a user-facing surface changed: `make surface` committed, and its diff matches what I meant to change +- [ ] If a name was removed or renamed: the old one is in `tools/retired_names.toml` (config keys: `RETIRED_KEYS`) +- [ ] The guide for this feature (`docs/guides/`) still describes it correctly +- [ ] `make pre-release` (or `make ci-local`) passes locally diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index a4f995fc..ac083824 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -1,12 +1,22 @@ # Suite, lint and check-* gates on a clean Arch container (3.4.0-DEV1). -# Contributor-only, never shipped. Advisory: no branch protection until it has -# stayed green for one release cycle. The job body is tools/ci.sh; reproduce a -# red run locally with `make ci-local`. +# Contributor-only, never shipped. Required: a ruleset on main makes this +# `gates` job a required status check and changes arrive by pull request, so a +# red run blocks the merge. The job body is tools/ci.sh; reproduce a red run +# locally with `make ci-local`. name: ci on: + # A branch is checked through its pull request; push runs only for main, so a + # PR's commit is not tested twice. push: + branches: [main] pull_request: + workflow_dispatch: + +# A newer push to the same PR cancels the run it supersedes. +concurrency: + group: ci-${{ github.event.pull_request.number || github.ref }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} permissions: contents: read diff --git a/CLAUDE.md b/CLAUDE.md index b0477c6c..e1709318 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -24,6 +24,8 @@ make check-design # guard: DESIGN.md in sync with sources make roadmap-view SORT=effort # print Planned table sorted by a column (read-only) make check-shipped # guard: etc/sysforge, PKGBUILD*, hooks, completions, manpage parity make check-standards +make surface # regenerate tests/data/surface.json after any user-facing change +make check-surface # guard: snapshot current; surface changes vs main named in release notes make sync-config # adopt new shipped defaults into untracked live config (add-only) make container-smoke[-cachyos] # packaging + portability checks in a container (needs make vm-pkg-stable first) make ci-local # replay the CI job (.github/workflows/ci.yml) in a clean Arch container @@ -75,7 +77,8 @@ Shipped-file edits must pass `make check-shipped`; doc/design edits `make check- profiles`, profile store paths) updates its guide in the same change. Order: after README. Concept pages (`docs/concepts/*.md`) explain the technique, not sysforge: no sysforge vocabulary outside the closing "In sysforge" section, upstream sources cited, no numbers - measured on one machine; guides link them (and the glossary) rather than re-explaining. + measured on one machine (`tests/test_concept_pages.py` checks the closing sections, the upstream + links and the vocabulary); guides link them (and the glossary) rather than re-explaining. - **`DOC` IDs are for user-facing docs only** (`docs/guides/`, `docs/concepts/`, README, man page) and file under `## Changed` (patch; lint-enforced). Internal docs (design, CLAUDE.md, ROADMAP) get no ID; wrong `--help`/man output is a `B`, since it is CLI behaviour. @@ -83,11 +86,14 @@ Shipped-file edits must pass `make check-shipped`; doc/design edits `make check- dev docs) and file their release-note entry under `## Changed` only, like `DOC` (lint-enforced, `3.4.0-STD1`). Touching `sysforge/` or a shipped file makes it `B`/`F`. - **Completions stay in lockstep with the CLI** (`completions/_sysforge` + bash) in the same change. +- **A user-facing change regenerates `tests/data/surface.json`** (`make surface`) and names what + changed in the release notes; `make check-surface` (a required CI gate) enforces both. - **Removing or renaming a flag, token or public name adds it to `tools/retired_names.toml`** in the same change (unless the old spelling is still valid elsewhere); `make check-standards` then finds every leftover, comments included. Written-down `sysforge …` commands and `§` references are checked too (§Documentation consistency guards). Tests build `BuildOptions(...)`, - `RunOptions(...)` or `build_parser().parse_args([...])`, never a `SimpleNamespace` stand-in. + `RunOptions(...)` or `cli_args("state", "list", ...)` (`tests/cli_args.py`), never a + `SimpleNamespace` stand-in. - **CLI verbs go through the Verb framework**: `Verb` subclass dispatched by `verbs.runner.run_verb` (`pre_check`/`execute`/`post_validate`; `requires_sentinel=True` if it mutates). Its flags live on the class (`add_parser` classmethod, ending `set_defaults(verb_cls=cls)`); register it in diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 14868087..5553af14 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -14,18 +14,41 @@ just as valuable — many features here started as workflow pain points. ## Ground rules for PRs -- `make test` and `make lint` must pass. Use the Makefile targets, not direct - `pytest`/`ruff` invocations. -- **`DESIGN.md` is generated** — edit the sources under `docs/design/*.md`, then run +Changes reach `main` by pull request, and the CI `gates` job is a required check: it runs +every gate in a clean Arch container, and a red run blocks the merge. Run the same set +locally with `make pre-release`, or reproduce CI exactly with `make ci-local` (podman). +`make install-hooks` adds a pre-commit hook running the fast gates, so most failures show +up before you push. + +- Use the Makefile targets, not direct `pytest`/`ruff` invocations. +- **`DESIGN.md` is generated**: edit the sources under `docs/design/*.md`, then run `make design`. Never edit `DESIGN.md` directly. Doc update order: `docs/design/*.md` → README.md → CLAUDE.md. -- CLI surface changes (new verb, flag, help text) must update - `completions/_sysforge` + `completions/sysforge.bash` and regenerate the man page - (`make man`) **in the same change**. `make check-shipped` verifies this. - New logic that branches on the resolved compiler (gcc vs llvm/clang) ships with tests for **both** paths. - Keep behavior changes and refactors in separate commits where practical. +### Changing anything a user can see + +A flag, command, config key, environment variable, `--json` field or log name is a +*surface*. Change one and, in the same PR: + +| Also update | Enforced by | +|---|---| +| Both completion scripts (`completions/_sysforge`, `completions/sysforge.bash`) | `make check-shipped` (`completion_commands`) | +| The man page: `make man` (generated from the parser and the env/journal tables) | `make check-shipped` (`manpage`) | +| `tests/data/surface.json`: `make surface`; its diff is what a reviewer reads | `make check-surface` | +| A release-note entry in `docs/release-notes/unreleased.md` naming what changed | `make check-surface`, `make check-standards` | +| **Removing or renaming** a name: add the old one to `tools/retired_names.toml` (a config key: `RETIRED_KEYS` in `sysforge/primitives/config_schema.py`) | `make check-surface`, `make check-standards` (`retired_names`) | +| A **new config key**: document it in the shipped `etc/sysforge/*.toml` (and its twin under `tests/data/etc/sysforge/`) | `tests/test_config_keys_documented.py`, `make check-shipped` | +| A **new environment variable**: a row in `sysforge/primitives/env_vars.py` | `tests/test_env_vars.py` | +| A **new `--json` field**: its schema under `tests/data/json-schema/` | `tests/test_json_output.py` | +| The user guide for the feature (`docs/guides/`) | review only | + +In tests, build the real objects: `BuildOptions(...)`, `RunOptions(...)`, or +`cli_args("state", "list", ...)` from `tests/cli_args.py` for a parsed command line. +`SimpleNamespace` stand-ins drift from the real thing (`tests/test_stand_ins.py`). + ## Getting started ```bash diff --git a/DESIGN.md b/DESIGN.md index 54c12fb1..afaf2b31 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -496,6 +496,17 @@ The non-`profiles.toml` configs (`packages.toml`, `toolchain.toml`, `kernel.toml **Adopting the `.sfnew` residue — `sysforge config merge`.** Hand-merging the `.sfnew` companions is the only remaining manual step, so `sysforge config merge` (verb `config-merge`, `config_cmd.py`) is a pacdiff-style interactive driver over them. It scans the resolved config dir (`--config-dir` override, else `paths.CONFIG_DIR`) for `*.sfnew` — plus pacman's own `*.pacnew`/`*.pacsave` for sysforge config files on a packaged install — and for each presents: `[v]iew` (a `difflib` unified diff through `$PAGER` via `maybe_pager`, prefixed by a **relocated-section banner** naming any header the diff shows as both removed and added — a header on both a `-` and a `+` line is present in both files, so the pair is a move, not a deletion; `difflib` has no move detection of its own), `[m]erge` (launch the resolved diff/merge tool with `live new`, then re-loop), `[s]kip`, `[r]emove` (drop the companion once the live file is satisfactory), `[o]verwrite` (copy the companion over the live file verbatim — guarded by a confirm and a "discards your local values" warning, never the default, intended for the `.pacnew` accept-maintainer case), and a[b]ort. Because a `.sfnew` is the *verbatim shipped file*, there is **no blind "accept theirs"** as a primary action — the safe path is merge-then-remove. The verb edits config files in place but never builds/installs, so it carries **no sentinel**. The diff/merge tool resolves through one home shared with the reconfigure editor chain — `primitives/editor.resolve_merge_tool` (`SYSFORGE_MERGE` > `sysforge.toml [ui].merge` > `$DIFFPROG` > `vimdiff`) launched via `run_tty_argv` (the `/dev/tty` passthrough, also used by `resolve_editor`'s callers, which additionally runs the child inside a `"child"` `yield_terminal` so an active progress bar's reserved bottom row is released before a full-screen editor takes the terminal). `--dry-run` reports the companion→target pairs without prompting (scripting/CI). +### Checking and repairing a live config + +Neither an upgrade nor `sync-config` ever removes anything from a live file, so a key sysforge stopped reading stays there, silently ignored, and a misspelt key is never reported. `primitives/config_schema.py` (`3.5.0-F18`) is the one home for what a live file may contain and for the edits that bring one back in line: + +- **The schema.** `KNOWN_SECTIONS`/`KNOWN_TOP_KEYS` (moved here from `tools/check_shipped.py`, which now imports them, so the runtime and the release gate read one definition) name each file's sections and top-level keys. `OPEN_SECTIONS` are the ones whose keys the user names (a profile's flag variables, `[[rules]]`, `[[package]]`, `[group.*]`, `[[kconfig]]`, ...); nothing under them is judged. In every other section a key is known when the **shipped default documents it**, active or as a commented `# key = ...` example, so documenting a key is what makes it valid; a commented example table covers only its own comment block, so a blank line returns to the last real header. `tests/test_config_keys_documented.py` keeps that sound: every key the code reads from a fixed section (chained `.get`/`[]` off a config-like object, or a local assigned from one) and every schema top-level key must be documented in the shipped default, or `config clean --unknown` would delete a working setting. It found `profiles.toml`'s `extends_system` (read from a user file) undocumented. The first run found `toolchain.toml`'s `reuse_unchanged` and `profile_store` (read by the stage and by `resolve_profile_store_root`) missing from the old list; `profile_store` is now also documented in the shipped file. +- **What was retired.** `RETIRED_KEYS` (renamed or dropped keys: `[git] pull_timeout`, the per-package `pkgbuild_patch`) and `RETIRED_VALUES` (`repo_mode = "profiled"`, `build_mode = "patched_pkgbuild"`), each with the ID that retired it. +- **The shipped defaults.** `shipped_text(name)` reads the checkout's `etc/sysforge`, or on an installed system the copy the wheel carries (`pyproject.toml` force-includes `etc/sysforge` as `sysforge/_data/config-defaults`): `/etc/sysforge` holds the user's own copies, kept across upgrades by `backup=()`. +- **Edits.** Line-based, so comments and layout survive, over a small scanner that knows tables, array-of-tables entries and multi-line values. Every edited text is re-parsed with `tomllib` and refused if it would not parse; files are written through `atomic_write.replace_file`. + +Three `config` verbs and a doctor axis sit on it (`config_cmd.py`): **`config check`** reports retired and unknown settings with their line, exiting 1 when there are any (`--json` prints one `config-check` document instead, `tests/data/json-schema/config-check.schema.json`: each issue's file, line, kind, setting, message and fix); **`config clean`** renames a retired key (or drops it when its replacement is already set in the same table, saying so) and rewrites a retired value, and with `--unknown` also removes unknown keys and sections; **`config reset FILE [KEY...]`** puts each `SECTION.KEY` back to the shipped line (removing it when the default leaves it unset, adding it when the file lacks it), refuses keys inside array-of-tables entries, and with no KEY replaces the whole file, keeping the old one as `FILE.bak-