feat(agent): per-tool create-legacy-rule recipes with --tool

.changeset/legacy-linter-recipes.md

@@ -5,3 +5,5 @@
 Rules for a linter the repository already runs now follow a fixed order. The agent first looks for a rule the linter already ships, checking each tool's catch-all rules by name (`no-restricted-syntax`, `banned-api`, `noRestrictedImports`, and others). It writes a new custom rule in that linter only when the repository already has five or more of its own to model it on. Otherwise the request becomes a Taskless rule. `route` (topic v8) no longer asks the user to choose between a linter rule and a Taskless rule, and it describes each linter `detect` found in the repository. During onboarding (topic v11), the agent only enables rules the repository's installed packages already ship.
 
 `detect --json` reports `customRules` for ESLint, Biome, pylint, flake8, and Ruff: how many rules the repository wrote itself for that linter, and where they are. A count that cannot be read without running a JavaScript config is reported as `"unknown"`.
+
+`taskless agent create-legacy-rule --tool=<name>` serves a recipe for one linter, named as `detect` reports it: `ruff`, `eslint` (including typescript-eslint), `biome`, `pylint`, or `flake8`. Each recipe names the tool's catch-all rules, says how to confirm a rule exists in the installed version (`ruff rule`, `biome explain`, `pylint --help-msg`), and covers the traps that make a rule silently not fire, such as Ruff preview rules and ESLint 10's removal of `.eslintrc`. The ESLint recipe also lists eight well-known plugins, each with a docs link. Other tool names fall back to the generic recipe. `ast-grep`, `vale`, `prettier`, and `black` are refused with a pointer to the right recipe. `route` now names the `--tool` command for each linter it describes.

.github/workflows/legacy-linter-contract.yml

@@ -0,0 +1,58 @@
+# Pins what the `create-legacy-rule` recipes say about Ruff, pylint, flake8,
+# and Biome, by asking each linter at a pinned version. See
+# `packages/cli/test/legacy-linter-vendor-contract.test.ts`.
+#
+# Separate from `Validate` on purpose: it downloads linters from PyPI and npm,
+# and a registry outage must not block every merge. A red run here means a
+# recipe sentence stopped being true, or a pinned version moved.
+name: Legacy linter contract
+
+on:
+  # No `branches:` filter, for the reason `validate.yml` gives.
+  pull_request:
+    types: [opened, synchronize, reopened, ready_for_review]
+    paths:
+      - "packages/cli/src/agent/create-legacy-rule*.md"
+      - "packages/cli/test/legacy-linter-vendor-contract.test.ts"
+      - ".github/workflows/legacy-linter-contract.yml"
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+env:
+  TASKLESS_TELEMETRY_DISABLED: "1"
+
+jobs:
+  contract:
+    name: Legacy linter contract
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+
+      - name: Install pnpm
+        uses: pnpm/action-setup@ea17c68df8912ef543352723c149a84f56e3d413 # v6.1.0
+
+      - name: Setup Node
+        uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
+        with:
+          node-version-file: .nvmrc
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
+
+      - name: Install dependencies
+        run: pnpm install --frozen-lockfile
+
+      # The suite's global setup snapshots the built CLI and refuses to start
+      # without one, even for a test file that never spawns it.
+      - name: Build
+        run: pnpm build
+
+      - name: Test
+        working-directory: packages/cli
+        env:
+          TASKLESS_LEGACY_LINTER_CONTRACT: "1"
+        run: pnpm vitest run test/legacy-linter-vendor-contract.test.ts

openspec/changes/legacy-linter-recipes/design.md → openspec/changes/archive/2026-10-22-legacy-linter-recipes/design.md

@@ -40,7 +40,7 @@ Constraints that shape the approach:
 
 **The ESLint plugin list is curated and linked, and the recipe never lists individual rules.** Eight plugins, each with a summary and a docs link. This narrows the standing "no catalog" line instead of removing it, because a catalog of rules goes stale with every plugin release, while a list of plugins changes rarely.
 
-**Vendor tests pin recipe claims against real installs.** Each claim a recipe makes about tool behavior gets a test that runs the tool, in the same spirit as the existing Vale and ast-grep vendor-contract tests: preview status in the `ruff rule` JSON, `--help-msg` failing without `--load-plugins`, `biome explain` output, and `banned-modules` in `flake8-tidy-imports --help`. They run through `uvx` and `pnpm dlx` at pinned versions, so a version bump that breaks a claim fails a test instead of shipping a wrong recipe.
+**Vendor tests pin recipe claims against real installs.** Each claim a recipe makes about tool behavior gets a test that runs the tool, in the same spirit as the existing Vale and ast-grep vendor-contract tests: preview status in the `ruff rule` JSON, `--help-msg` failing without `--load-plugins`, `biome explain` output, and `banned-modules` in `flake8-tidy-imports --help`. They run through `uvx` and `pnpm dlx` at pinned versions, so a version bump that breaks a claim fails a test instead of shipping a wrong recipe. Because they download packages, they are opt-in (`TASKLESS_LEGACY_LINTER_CONTRACT=1`) and run in their own `legacy-linter-contract` workflow on changes to the legacy recipes, not in `Validate`: a registry outage must not block every merge.
 
 ## Research record
 

openspec/changes/legacy-linter-recipes/tasks.md → openspec/changes/archive/2026-10-22-legacy-linter-recipes/tasks.md

@@ -15,21 +15,21 @@
 
 ## 3. The `--tool` flag (PR 2)
 
-- [ ] 3.1 Collect `create-legacy-rule.<tool>.md` variants at build time with the same glob mechanism as `.anonymous`, and add `--tool` to the agent command's value flags. Resolve `--tool` before `--anonymous`. Verify tests: a known tool serves its variant, an unknown tool falls back to the generic recipe, `--tool` as `--tool x` and `--tool=x` both parse, and the positional parser does not read the value as a topic.
-- [ ] 3.2 Refuse `ast-grep`, `vale`, `prettier`, `black`, and `anonymous` with a non-zero exit that names the right recipe. Verify a test per refused name.
-- [ ] 3.3 Make the `route` paragraphs name `create-legacy-rule --tool=<name>`. Verify the "Shipped recipes name only commands that exist" check passes for every rendered paragraph.
+- [x] 3.1 Collect `create-legacy-rule.<tool>.md` variants at build time with the same glob mechanism as `.anonymous`, and add `--tool` to the agent command's value flags. Resolve `--tool` before `--anonymous`. Verify tests: a known tool serves its variant, an unknown tool falls back to the generic recipe, `--tool` as `--tool x` and `--tool=x` both parse, and the positional parser does not read the value as a topic.
+- [x] 3.2 Refuse `ast-grep`, `vale`, `prettier`, `black`, and `anonymous` with a non-zero exit that names the right recipe. Verify a test per refused name.
+- [x] 3.3 Make the `route` paragraphs name `create-legacy-rule --tool=<name>`. Verify the "Shipped recipes name only commands that exist" check passes for every rendered paragraph.
 
 ## 4. Per-tool recipes (PR 2)
 
-- [ ] 4.1 Write `create-legacy-rule.ruff.md`: `banned-api` first, web discovery confirmed with `ruff rule`, preview status checked, the 0.16 default-set change, `extend` and nested configs, and how to run Ruff. Verify recipe tests for the Ruff scenarios.
-- [ ] 4.2 Write `create-legacy-rule.eslint.md`: core rules and the `no-restricted-*` catch-alls first (including `@typescript-eslint/`), the eight curated plugins each with a summary and docs link, installed plugins preferred, the ESLint 10 peer gap and `import-x`, flat config against `.eslintrc`, and how to run ESLint. Verify recipe tests that every plugin has a summary and URL and that no individual plugin rules are listed.
-- [ ] 4.3 Write `create-legacy-rule.biome.md`: an existing rule first, `noRestrictedImports`/`noRestrictedGlobals` as the catch-alls, web discovery confirmed with `biome explain`, nested configs and `extends`, and how to run Biome. Verify the recipe tests.
-- [ ] 4.4 Write `create-legacy-rule.pylint.md`: `--list-msgs`/`--help-msg` with the `--load-plugins` trap, `deprecated-modules`/`preferred-modules`/`bad-functions`, config search order, and how to run pylint. Verify the recipe tests.
-- [ ] 4.5 Write `create-legacy-rule.flake8.md`: no discovery step, `banned-modules` from `flake8-tidy-imports` (a dependency that needs consent), the three config files, and how to run flake8. Verify the recipe tests.
-- [ ] 4.6 Add vendor tests that run each tool at a pinned version and assert every behavioral claim the recipes make: the `preview` field in `ruff rule --output-format json`, `ruff rule TID251`, `biome explain` output, `pylint --help-msg` failing without and succeeding with `--load-plugins`, and `--banned-modules` in `flake8 --help` with the plugin. Verify they pass locally and in CI, and fail when a claim is edited to be false.
-- [ ] 4.7 Extend the changeset with `--tool` and the five recipes. Run `pnpm typecheck` and `pnpm lint`; both pass.
+- [x] 4.1 Write `create-legacy-rule.ruff.md`: `banned-api` first, web discovery confirmed with `ruff rule`, preview status checked, the 0.16 default-set change, `extend` and nested configs, and how to run Ruff. Verify recipe tests for the Ruff scenarios.
+- [x] 4.2 Write `create-legacy-rule.eslint.md`: core rules and the `no-restricted-*` catch-alls first (including `@typescript-eslint/`), the eight curated plugins each with a summary and docs link, installed plugins preferred, the ESLint 10 peer gap and `import-x`, flat config against `.eslintrc`, and how to run ESLint. Verify recipe tests that every plugin has a summary and URL and that no individual plugin rules are listed.
+- [x] 4.3 Write `create-legacy-rule.biome.md`: an existing rule first, `noRestrictedImports`/`noRestrictedGlobals` as the catch-alls, web discovery confirmed with `biome explain`, nested configs and `extends`, and how to run Biome. Verify the recipe tests.
+- [x] 4.4 Write `create-legacy-rule.pylint.md`: `--list-msgs`/`--help-msg` with the `--load-plugins` trap, `deprecated-modules`/`preferred-modules`/`bad-functions`, config search order, and how to run pylint. Verify the recipe tests.
+- [x] 4.5 Write `create-legacy-rule.flake8.md`: no discovery step, `banned-modules` from `flake8-tidy-imports` (a dependency that needs consent), the three config files, and how to run flake8. Verify the recipe tests.
+- [x] 4.6 Add vendor tests that run each tool at a pinned version and assert every behavioral claim the recipes make: the `preview` field in `ruff rule --output-format json`, `ruff rule TID251`, `biome explain` output, `pylint --help-msg` failing without and succeeding with `--load-plugins`, and `--banned-modules` in `flake8 --help` with the plugin. Verify they pass locally and in CI, and fail when a claim is edited to be false.
+- [x] 4.7 Extend the changeset with `--tool` and the five recipes. Run `pnpm typecheck` and `pnpm lint`; both pass.
 
 ## 5. Archive (PR 2, tip)
 
-- [ ] 5.1 Run the pre-archive check from CLAUDE.md (commit, keep the SHA, `pnpm openspec archive legacy-linter-recipes -y`, confirm every prior scenario of `cli-rule-routing`, `cli-agent`, `cli-detect`, and `cli-onboard` survives, then reset to the SHA).
-- [ ] 5.2 Archive for real. Verify `pnpm openspec validate --strict` passes on the result.
+- [x] 5.1 Run the pre-archive check from CLAUDE.md (commit, keep the SHA, `pnpm openspec archive legacy-linter-recipes -y`, confirm every prior scenario of `cli-rule-routing`, `cli-agent`, `cli-detect`, and `cli-onboard` survives, then reset to the SHA).
+- [x] 5.2 Archive for real. Verify `pnpm openspec validate --strict` passes on the result.

openspec/specs/cli-agent/spec.md

@@ -8,11 +8,14 @@ Defines the `agent` subcommand for the `@taskless/cli` package, including recipe
 
 ### Requirement: Agent subcommand serves recipes for commands
 
-The CLI SHALL support an `agent` subcommand that accepts at most one positional argument identifying a topic AND an optional `--anonymous` boolean flag. Topics SHALL be addressed by a single token; the subcommand SHALL NOT join multiple positionals into a topic key. When a topic is provided, the subcommand SHALL look up a matching recipe file embedded at build time using the following resolution order:
+The CLI SHALL support an `agent` subcommand that accepts at most one positional argument identifying a topic, an optional `--anonymous` boolean flag, AND an optional `--tool <name>` value flag. Topics SHALL be addressed by a single token; the subcommand SHALL NOT join multiple positionals into a topic key. When a topic is provided, the subcommand SHALL look up a matching recipe file embedded at build time using the following resolution order:
 
-1. If `--anonymous` is set AND `<topic>.anonymous.md` exists in the embedded map, return that file.
-2. Otherwise, return `<topic>.md`.
-3. If neither exists, exit with code 1 and an error message suggesting `taskless agent` for the topic index.
+1. If `--tool` is set AND `<topic>.<tool>.md` exists in the embedded map, return that file.
+2. If `--anonymous` is set AND `<topic>.anonymous.md` exists in the embedded map, return that file.
+3. Otherwise, return `<topic>.md`.
+4. If none of these exists, exit with code 1 and an error message suggesting `taskless agent` for the topic index.
+
+On a topic that has tool variants, `--tool` SHALL refuse, with a non-zero exit and a message naming the recipe to use instead, a tool that has its own authoring recipe (`ast-grep`, `vale`) or that cannot hold a rule (`prettier`, `black`). An unrecognized tool name SHALL fall back to `<topic>.md`.
 
 When no positional argument is provided, the subcommand SHALL print a topic index containing a one-paragraph human slug followed by a topic disambiguation table mapping topic names to their summaries.
 
@@ -39,6 +42,33 @@ The subcommand is named for its reader. It serves agents fetching a procedure, n
 - **WHEN** a user runs `taskless help check`
 - **THEN** the CLI SHALL NOT print recipe text for `check`
 
+#### Scenario: A tool variant is served
+
+- **WHEN** a user runs `taskless agent create-legacy-rule --tool=ruff`
+- **AND** `create-legacy-rule.ruff.md` exists
+- **THEN** the CLI SHALL print that file's contents
+
+#### Scenario: An unknown tool falls back to the generic recipe
+
+- **WHEN** a user runs `taskless agent create-legacy-rule --tool=rubocop`
+- **AND** no `create-legacy-rule.rubocop.md` exists
+- **THEN** the CLI SHALL print `create-legacy-rule.md`
+
+#### Scenario: A tool with its own recipe is refused
+
+- **WHEN** a user runs `taskless agent create-legacy-rule --tool=vale`
+- **THEN** the CLI SHALL exit non-zero and name `create-vale-rule` as the recipe to use
+
+#### Scenario: A refused tool name is ignored on a topic without tool variants
+
+- **WHEN** a user runs `taskless agent route --tool=vale`
+- **THEN** the CLI SHALL print `route.md` and exit zero
+
+#### Scenario: An unknown topic is not masked by a refused tool
+
+- **WHEN** a user runs `taskless agent no-such-topic --tool=vale`
+- **THEN** the CLI SHALL exit non-zero with the unknown-topic error, not the tool refusal
+
 ### Requirement: Recipe files are embedded at build time
 
 Recipe files SHALL be located at `packages/cli/src/agent/` as `.md` files. The Vite build SHALL embed these files into the CLI bundle via `import.meta.glob` with raw imports. A recipe file SHALL exist for every registered command and subcommand.

openspec/specs/cli-detect/spec.md

@@ -270,3 +270,29 @@ A linter that ships with a language toolchain rather than being declared as a de
 
 - **WHEN** a root CI or hook config names a linter only in a comment, a step's `name`, or another field that is not a command
 - **THEN** `detect --json` SHALL NOT report that linter from it
+
+### Requirement: Detect counts the repository's own rules per linter
+
+`detect --json` SHALL report, for each detected linter that can host custom rules, how many custom rules the repository itself authors for it, and the evidence for that count. A custom rule is one whose logic the repository wrote:
+
+- ESLint: rules in a local plugin or rule directory, including plugins defined inline in the config or imported from a file in the repository.
+- Biome: entries in the configuration's `plugins` list.
+- pylint: modules named by `load-plugins` that live in the repository.
+- flake8: checkers registered under `[flake8:local-plugins]`, or a `flake8.extension` entry point the repository publishes.
+
+Optional extensions shipped with the tool, third-party plugin packages, presets, and rule selection SHALL NOT count. Ruff has no custom-rule mechanism and SHALL report a count of zero. The count SHALL be derived from files on disk without executing anything, consistent with detect's other signals.
+
+#### Scenario: Local ESLint rules are counted
+
+- **WHEN** the repository has a local ESLint plugin defining six rules
+- **THEN** `detect --json` SHALL report a custom-rule count of six for ESLint with the plugin's path as evidence
+
+#### Scenario: A built-in pylint extension does not count
+
+- **WHEN** `load-plugins` names only `pylint.extensions.*` modules
+- **THEN** `detect --json` SHALL report a custom-rule count of zero for pylint
+
+#### Scenario: Ruff reports no custom rules
+
+- **WHEN** the repository configures Ruff, however extensively
+- **THEN** `detect --json` SHALL report a custom-rule count of zero for Ruff