Skip to content

Commit df0e1a0

Browse files
feat(qa): retest-owed reader, touched:SHA selector and the standing regression cadence (#27647)
domain:devx#1 · os-elon-musk · session_01LKHDniRQSWjRJyDtacwpEd <!-- claude-projects-attribution: {"github_login":"os-elon-musk"} --> _Requested by **Elon** · [project thread](https://claude.ai/code/project/chan_01WedDzV26nniwxv7jC88B9D?thread=cmsg_01WedDzV26nniwxv7jC88B9D27iKcauY7daF8J3vRY9Ucx)_ Fixes #27383 ## What changes 1. **`scripts/qa/retest-owed.mjs` (new, read-only).** Lists the closed issues that carry a `QA-source: #RECORD · area.item · clause` line and closed after the subject commit time of the newest `qa-run` record that judged that item. Each row names the item, its fix card(s), the record that last judged it, and whether a later record re-judged it. `--json`, `--as-of ISO` (replays the past), `--repo`, and an offline `--self-test` (57 assertions in 9 batteries, with the battery roster and floor copied from qa-rollup). It writes nothing to GitHub. - A record judges an item only when its title selector is that item with a judged count, or a body line names the item with a pass/partial/fail word and no not-run word. Records that only *name* an item are shown as `unconfirmed: #N names it`, so a missed judgment errs toward "owed", never toward "re-judged". - `smoke:npm-VERSION · CELL` locators are re-judged by the next `smoke:npm-*` record. - Nothing is dropped: malformed `QA-source:` lines, unrecognized locators and cross-repo records are listed under **Not read** with the reason. Cards closed `not_planned`/`duplicate` are listed under **Closed without a fix**. - Time is three-valued. A record whose subject sha cannot be resolved makes the row **Unknown**, not owed and not re-judged. - Endpoints are repo-scoped only (closed issues since the oldest record, `GET /repos/…/commits/SHA` for subject times). The listing follows the `next` cursor carried onto the `/repos` path, because the `/repositories/ID` form of that link answers 403 from a cloud session and `page=100` answers 422. 2. **`touched:SHA` selector in `scripts/checklist-select.mjs`.** Selects the runnable items whose `source[]` paths or `automated.ref` files changed in `SHA..HEAD`, from `git diff --no-renames --name-only SHA HEAD`. - The `#symbol` anchor and trailing prose are cut, and `dir/{a,b}.ts` brace lists are expanded. - A cited path matches a changed path that holds it as a whole run of `/` segments. That covers the same path, a path under a cited directory (prefix match), or a longer path ending in it. A bare basename matches only for a source-file extension. - The CLI refuses a malformed or unfetched sha with exit 2 and the fetch remedy, and names citations whose file no longer exists. - It is listed in the header's selector table and in the skill's §0 table. The self-test battery floor goes from 31 to 45 cases (14 new, each load-bearing). 3. **`.claude/skills/checklist-test/SKILL.md`**, with the cadence stated as the maintainer chose it: - a new §6 "standing regression", covering the weekly run on `main` HEAD (one `qa-run` record per selector: retest-owed list, `touched:` the previous weekly subject, `priority:P0`, `surface:api`, with Supersede keeping at most one open per selector), the post-publish `smoke:npm-VERSION` in the shape of the previous smoke record, where its fix cards are re-judged, and the retest-owed reader; - §5 gains a fourth retest trigger and keeps "hand-feel reruns are not triggers"; - verification retest is now "not the fifth trigger"; - the creation of the scheduled tasks stays with the dispatching PM session, not repo code. 4. **`scripts/pm/check-skill-line-ratchet.mjs`**: the checklist-test ceiling moves **259 → 277**, on top of #27642's 246 → 259 (ruling 6500750765, +13). Step one is **259 → 276** under **#27377 comment 6501246005** (the maintainer's four options and 「这些都应该写到 checklist skills 中」, verbatim), **+17** measured: 20 ruled lines (the §0 `touched:` row, trigger 4, the §5 npm smoke locator exception, §6) less 3 repaid by rejoining §5 wraps broken mid-word. Step two is **276 → 277** under ruling **6500750765**, **+1**: retest trigger 2's security-hardening clause. No line is over 120 bytes. (Seat refresh of this paragraph, per contract review 6508135176 should-fix 2.) RUNNER.md is unchanged; nothing in it needed to move. ## Why QA-extracted fix cards land between release candidates. Nothing listed which checklist items those landings left un-re-judged, so §5 trigger 2 depended on memory. `touched:` gives the weekly run an incremental pool. The maintainer ruled the cadence on #27377 and asked for it to be written into the checklist skill. ## Verification **Anchor proof on #25807** (live, 214 API reads, 0 writes; corpus: 160 `qa-run` records, 136 with a canonical title; 17,967 issues scanned since 2026-08-11; 171 cards with a `QA-source:` line; 32 of 32 subject shas resolved): - `--as-of 2026-10-29T00:00:00Z` (before wave 3) gives 20 owed rows. All 22 fix cards the card names are accounted for: - **16 owed**: #25934, #26452, #26453, #26459, #26461, #26462, #26463, #26465, #26474, #26477, #26478, #26479, #26480 (12 item rows; `ux-conventions.write-feedback` carries two cards) and #26630, #26631, #26632 (`smoke:npm-18.2.0` cells D3, C3, F5). - **5 under Not read**: #26640, #26641, #26642, #26644, #26646, because their line is `QA-source: #26639 · observation N`, two fields where the contract has three. - **#26707 absent**: its body has no `QA-source:` line (it uses `Batch-of:`). - The other 5 owed rows the card did not name: #23946, #23950, #24576, #24782, #24784. I spot-checked three of them, and later records hid or recorded the item not-run: `access-security.no-active-org-session-semantics` is hidden as blocked in #25227 and #26481, `identity-auth.email-verification-loop` is not-run in #24538, and `platform-core.shell-nav-personalization` is not-run in #24557. They are owed. - Without `--as-of` (2026-10-29 15:41Z), there are 15 owed rows. The wave-3 records re-judged all 12 item rows above. The three `smoke:npm-18.2.0` rows stay owed until the next npm smoke. Seven new rows are wave-3 fix cards closed today: #27395, #27445, #27448, #27488, #27490, #27495, #27510. **Local gates**, each exit code captured before any pipe; the change set was derived by `dispatch-gates.mjs`: - `node scripts/checklist-select.mjs --self-test`: 45 cases pass - `node scripts/qa/retest-owed.mjs --self-test`: 57 assertions in 9 batteries - `node scripts/check-platform-checklist.mjs` and `pnpm check:platform-checklist`: OK, 352 items - `pnpm check:pm-skill-ratchet`: 0 - `pnpm check:pm-skill-id-lint`: 0 - `pnpm check:pm-dispatch-gates`: fast tier, 1937 cases - `pnpm check:entry-guard`: 0 - `pnpm check:parse-guard`: 0 - `pnpm check:agent-test-spelling`: 0 - `pnpm check:bash32-floor`: 0 - `pnpm check:ci-filter-parity`: 0 - `pnpm check:cli-command-ids`: 0 - `pnpm check:cross-package-test-inputs`: 0 - `pnpm check:doc-authoring`: 0 - `check:doc-formula-expressions`: 0, after building formula and lint - `pnpm check:pm-governed-merges`: 0 - `pnpm check:pnpm-filter-targets`: 0 - `pnpm check:pnpm-flag-spellings`: 0 - `pnpm check:ratchet-remedy-authority`: 0 - `pnpm check:scripts-symbol-anchors`: 0, after the second commit - `pnpm check:skill-frame-sync`: 0 - `pnpm check:nul-bytes`: 0 - `check-declaration-mirrors` (and its self-test): 0 - `check-self-test-wired` (and its self-test): 0 - `check-self-test-workflow-commands` (and its self-test): 0 - `check-whole-set-label-write` (and its self-test): 0 - `plan-test-core-shards --self-test`: 0 - `check-harness-current --self-test`: 0 - `check-skills-token-ratchet`: 0 - `check-published-list-mirrors`: 0 - `check-commit-card-trailers`: 0 - eslint on the three scripts: 0 The other families run in CI. Clause-②: no Metadata-protocol: none _Keep one value on each of the two lines above, each line starting with its key. [Human Minimum](https://github.com/objectstack-ai/objectstack/blob/main/CONTRIBUTING.md#human-minimum) item 5 says what they mean._ ## Human Minimum Each item is one rule from [CONTRIBUTING.md › Human Minimum](https://github.com/objectstack-ai/objectstack/blob/main/CONTRIBUTING.md#human-minimum), which links the AGENTS.md text that states it. - [x] 1. Schemas start as Zod; `packages/spec` holds no runtime logic. (No schema touched.) - [x] 2. Every name I added uses the casing for its kind. - [x] 3. No formatter ran; `pnpm lint` passes. (eslint on the touched scripts.) - [x] 4. One issue, one branch off an up-to-date `main`, one pull request. - [x] 5. A change that publishes carries a changeset, and both declaration lines are filled in here and in the changeset. (Nothing publishes: `skip-changeset`.) - [x] 6. `packages/spec` touched: its generated artifacts are regenerated and committed. (Not touched.) - [x] 7. No hand edits to `content/docs/references/`, `packages/*/CHANGELOG.md` or `content/docs/releases/`. - [x] 8. Title, body and comments are in English. - [x] The *Before Submitting* list in CONTRIBUTING.md is done. ## Acceptance notes - **qa-rollup pagination.** `scripts/qa/qa-rollup.mjs` follows GitHub's `next` link verbatim. From a cloud session that link (`/repositories/ID/…`) answers 403 on page 2. The new reader carries the cursor onto the `/repos` path instead. Not fixed here, because it is out of scope. Carrier: the Devx seat has taken it as its own card. - **Self-test wiring.** `retest-owed --self-test` is not run by CI, which is the same posture as `qa-rollup --self-test`. Wiring it in means a one-word addition to the `check:platform-checklist` alias, left for the seat to decide, since this card excludes CI changes. - **Record bodies vary.** The body rule does not see judgments written as aggregate prose ("every item not listed below passed") or with short ids. Those rows come out owed, with an `unconfirmed: #N names it` pointer, so the error is on the side of re-running. --- _Generated by [Claude Code](https://claude.ai/code/session_016WV4d1oYpq8eEjYF141fC3)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 869fa76 commit df0e1a0

5 files changed

Lines changed: 1176 additions & 17 deletions

File tree

‎.claude/skills/checklist-test/SKILL.md‎

Lines changed: 27 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,7 @@ node scripts/checklist-select.mjs <selector> --json
4848
| `surface:api` | 全部 API 面测试项(便宜 —— 不需要浏览器构建) |
4949
| `since:v17` | 某个 release 引入的全部项(release-sweep 过滤器) |
5050
| **`file:packages/plugins/plugin-approvals/src/approval-service.ts`** | **`source[]` 引用该文件的项 ——「测覆盖这个文件的一切」**(带 `/` 或代码扩展名的裸路径也自动按 `file:` 解析) |
51+
| `touched:<sha>` | `source[]` 或 `automated.ref` 引用的文件在 `<sha>..HEAD` 里改过的项(增量回归) |
5152
| `all` | 整份清单 |
5253

5354
`--json` 给出可运行清单(id · priority · surface · revision)。**Blocked 项默认排
@@ -169,8 +170,7 @@ expected-fail 探针 —— 永不在真实缺陷上把条款打成绿。
169170

170171
立单不是终点,收口是每一轮运行的一部分。RUNNER 的抽取义务条款给的是**抽什么**;本节
171172
给的是**一张记录什么时候才算关闭**,以及关闭这件事在别处怎么被读到。零新标签 ——
172-
下面
173-
每个状态都用现有载体承载。
173+
下面每个状态都用现有载体承载。
174174

175175
**开着的 `qa-run` 数 = 未清的收口债**,与裸 `finding` 计数同一构造:它是可直接读的健
176176
康指标,所以⛔ 永不靠关掉记录让它变好看,只靠把下面几件事做完。
@@ -215,29 +215,47 @@ QA-source: #运行记录号 · area.item · 条款
215215
`platform-core.console-login`)—— 解析器、run 记录的逐条款表、抽取卡三处同一把定位
216216
符,才互相 grep 得到;⛔ 不写只有 item 段的短形。**写进 GitHub 正文/评论时⛔ 不要用尖
217217
括号占位符**:`<`+字母会被写入层当标签吃掉,一行溯源就此消失在库里 —— 占位用中文词
218-
或
219-
直接填实值。
218+
或直接填实值。**唯一例外是 npm 冒烟**(§6):定位字段写冒烟选择器、条款写格号,如
219+
`QA-source: #运行记录号 · smoke:npm-18.2.0 · C3` —— 下一轮冒烟凭这一行重判它。
220220

221221
**Supersede 协议** —— 重跑覆盖了旧记录的地面时,旧记录关闭(`not_planned` + 一行
222222
`Superseded-by:` 指向新记录),但**先把新一轮没有重新判定的判定结转过去**:结转行
223223
(`Carried-forward:`,引旧记录号)落在**新记录**上,supersede 行落在**旧记录**上。⛔
224224
永不用「新的一轮跑过了」直接关掉旧记录:没被重新覆盖的判定会随它一起消失,而记录被
225-
关
226-
闭这件事在别处不留任何痕迹。
225+
关闭这件事在别处不留任何痕迹。
227226

228-
**重测触发,三条且只有三条**:
227+
**重测触发,四条且只有四条**:
229228

230229
1. **发版候选** ⇒ 完整 Tier 1 重跑(§1 的分层)。
231230
2. **QA 抽取的修复卡落地** ⇒ **接受该卡的座位**把受影响的项追加到波次锚卡的重测清
232-
单 —— 与跨仓联动同一纪律:第一产者是接受座位。
231+
单 —— 与跨仓联动同一纪律:第一产者是接受座位。安全加固没有卡:落地它的 PM 会话自
232+
己重跑该项,⛔ 不在任何卡上点名它。
233233
3. **维护者点名目标**。
234+
4. **常驻回归**(维护者所定,§6)⇒ 每周回归与发版后 npm 冒烟。
234235

235236
⛔ 其它一切都不是重测触发:凭手感「再跑一遍」只增加收口债,不增加覆盖。
236237

237-
**验证重测**不是第四条触发:由同一 PR 对该项的清单编辑(§3)引发,⛔ 不立 `qa-run` 记录。
238+
**验证重测**不是第五条触发:由同一 PR 对该项的清单编辑(§3)引发,⛔ 不立 `qa-run` 记录。
238239
逐项判定表进 PR 正文,该项 `history` 行的 `ref` 指向该 PR;`partial`/`blocked` 的缺口进 PR
239240
正文与 `fixtures.knownGaps`;产品缺陷 `fail` 照常抽卡、溯源指向该 PR;条款过期在同一 diff 修。
240241

242+
## 6. 常驻回归 —— 两次发版之间也重跑
243+
244+
维护者所定的两条例行(裁定评论 6501246005);定时任务由派发的 PM 会话建,⛔ 不是仓库代码,也
245+
不是合并门禁。
246+
247+
- **每周回归**,对 `main` HEAD,四张 `qa-run` 记录,一个选择器一张:
248+
- 欠测清单:标题选择器写 `owed:weekly`(⛔ 不带冒号的词会被读成区),清单里每个 id
249+
按 §0 裸 id 逐项解析;
250+
- `touched:<上周 touched 记录的 subject sha>`;首周取最近一张发版全量重跑记录的 subject;
251+
- `priority:P0` 与 `surface:api`。
252+
§5 的 Supersede 让每条例行至多开着一张:选择器周周相同的三张直接适用;`touched:` 的 sha
253+
每周都变,本周那张照样 supersede 上周那张,判定结转照 §5。
254+
- **发版后 npm 冒烟**:每次发布后跑 `smoke:npm-<版本>`,形状照上一张同类记录;上一轮冒烟抽出
255+
的修复卡在这里重判(它们的溯源行见 §5 的冒烟例外)。
256+
- **欠测清单**由 `node scripts/qa/retest-owed.mjs` 只读算出(⛔ 不写 GitHub):
257+
QA 抽取的修复卡关闭晚于该项最新判定的 subject,该项即欠测 —— 就是 §5 触发 2 的清单。
258+
241259
## Guardrails
242260

243261
- **不伪造覆盖。** 缺 fixture → `blocked(fixture)` 带理由;console 没构建 → 构建它

‎package.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -118,7 +118,7 @@
118118
"check:adr-symbol-anchors": "node scripts/check-adr-symbol-anchors.mjs --self-test && node scripts/check-adr-symbol-anchors.mjs",
119119
"check:scripts-symbol-anchors": "node scripts/check-scripts-symbol-anchors.mjs --self-test && node scripts/check-scripts-symbol-anchors.mjs",
120120
"check:spec-docblock-symbol-anchors": "node scripts/check-spec-docblock-symbol-anchors.mjs --self-test && node scripts/check-spec-docblock-symbol-anchors.mjs",
121-
"check:platform-checklist": "node scripts/checklist-select.mjs --self-test && node scripts/check-platform-checklist.mjs --self-test && node scripts/check-platform-checklist.mjs",
121+
"check:platform-checklist": "node scripts/checklist-select.mjs --self-test && node scripts/qa/qa-rollup.mjs --self-test && node scripts/qa/retest-owed.mjs --self-test && node scripts/check-platform-checklist.mjs --self-test && node scripts/check-platform-checklist.mjs",
122122
"gen:checklist-status": "node scripts/gen-checklist-status.mjs --self-test && node scripts/gen-checklist-status.mjs",
123123
"check:org-identifier": "node scripts/check-org-identifier.mjs --self-test && node scripts/check-org-identifier.mjs",
124124
"check:runner-env-posture": "node scripts/check-runner-env-posture.mjs --self-test && node scripts/check-runner-env-posture.mjs",

0 commit comments

Comments
 (0)