Repository navigation
Commit cf218ee
test: seven suites keep their temp databases and scaffolds off the shard's shared disk (memory-backed TMPDIR) (#27379)
domain:server#3 · os-bill · session_01Utp6CXc5hequf76uJfzrdK
<!-- claude-projects-attribution: {"github_login":"os-bill"} -->
_Requested by **Bill** · [project
thread](https://claude.ai/code/project/chan_016iMXWSZDzqWRWvWzdNo8wg?thread=cmsg_016iMXWSZDzqWRWvWzdNo8wgKURKuRGF1ewEVVZKx7U1i4)_
Fixes #27306
Clause-②: no
Metadata-protocol: none
## 维护者速读
-
改了什么:create-objectstack、cli、runtime、driver-turso、driver-sql、service-package、service-automation
七个包的测试配置,把测试临时目录(TMPDIR)指到内存盘 /dev/shm;新增仓根共享助手
`vitest.memory-tmp.mts`,driver-sqlite-wasm 也改读它。不动任何产品代码、用例、断言或超时。
- 为什么改:这些测试把 SQLite 库或脚手架目录写在系统临时目录里,CI 分片上磁盘被邻居测试占满时,fsync
和删目录要排队,用例或收尾钩子就超时;main 上两次 `Test Core (5/8)` 红灯落在 create-objectstack。
- 风险与回滚:只影响测试进程的临时目录;没有可写 /dev/shm 的机器(macOS、Windows)照旧用系统临时目录。回滚即
revert 本 PR。
-
先看:`vitest.memory-tmp.mts`(理由与开关)、`packages/create-objectstack/vitest.config.ts`(与
main 红灯同形的那一包)、`packages/cli/vitest.config.ts`(两个 project 各自带 env)。
- 两分钟验证:`pnpm --filter create-objectstack test`,应 19 个文件全绿;在 4 个 `dd
conv=fsync` 写入者旁跑同一命令,改前 2 个文件因 afterAll 超时失败,改后全绿。
- 需要你做的:无,席位落地。
## What changes
- **`vitest.memory-tmp.mts`** (new, repo root, beside
`vitest.preset.mts`): `memoryBackedTmpEnv()` returns `{ TMPDIR:
'/dev/shm' }` when `/dev/shm` is writable and `{}` otherwise. It holds
the rationale once. It is deliberately not a key of the shared preset,
which every re-exporter inherits.
- **Seven configs spread it into `test.env`**: `create-objectstack`,
`cli`, `runtime`, `driver-turso`, `driver-sql`, `service-package` and
`service-automation`. `cli` and `runtime` declare vitest `projects`,
where a root-level `env` is inert, so each project's own `env` carries
it beside `OS_REGISTRY_LOG`. `driver-turso` re-exported the preset, so
it now restates the preset's one setting in its own config, the shape
`driver-sqlite-wasm` took in #27245.
- **`driver-sqlite-wasm`** (the #27245 precedent) now reads the same
helper instead of its private copy. Its behaviour is unchanged.
- Each config records its own before and after reading in a comment.
No product source changed. No case, assertion, fixture, timeout or
budget changed, and no retry was added (the #27076 ruling).
## Why
`os.tmpdir()` reads `TMPDIR`, and a child process inherits it. Every
database file and scaffold tree these suites write was measured to sit
under `os.tmpdir()` (the card asked for this reading per package). So
the one setting moves all of it off the runner's shared disk. On a Test
Core shard, a case that fsyncs, or a hook that unlinks a tree, waits for
the other suites' I/O, so its wall time measures the neighbours rather
than the code.
**create-objectstack, stated in three parts, as the seat asked:**
1. **The failing case in main's two reds is NOT MEASURED.** Push CI runs
40713919581 and 40718500281 (`Test Core (5/8)`) annotate only
`packages/create-objectstack … pnpm run test exited (1)`. Their full job
logs and artifacts are served from a storage host that this cloud
container's egress proxy refuses (403). The log API returns only the
last 5,000 lines, and the failing segment sits near line 33,848 of
42,326.
2. **The local signature matches, but it is not proven to be the same
failure.** Beside four `dd conv=fsync` writers, all 301 cases passed
while two FILES failed with `Hook timed out in 10000ms`. The timeouts
were in the `afterAll` that removes the scaffolded tree
(`created-summary.test.ts:134`,
`scaffold-skills-single-copy.test.ts:289`). That gives a red package
with no failed test named, which is the shape of the annotation.
3. **The mechanism is metadata disk load, not fsync.** `strace -f -c`
over the whole suite counted 4 fsync, 4,018 unlink and 6,202 rmdir
calls. The scaffolds' creation and removal is what waits on the loaded
disk. The same isolation fixes it, because every path is under
`os.tmpdir()`.
For the other packages, the mechanism is SQLite durability I/O. Under
strace, one `cli` file
(`organization-ownership-apply.generation-1.test.ts`) made 3,388 fsync
calls. One `runtime` file
(`package-revert-stored-members.integration.test.ts`) made 784 fsync and
982 unlink calls, the second being SQLite's per-transaction journal
deletes.
## Tests
Each package was measured three ways on the same files: idle on the
base, beside four `dd if=/dev/zero bs=1M count=256 conv=fsync` writers
on the same disk on the base, and beside the same writers on the head.
All runs were on tree `objectstack@6d7634a` (base) and this branch
(head), in a 4 vCPU cloud container with `/` on `/dev/vda` and
`/dev/shm` on tmpfs, using `vitest run --maxWorkers=2`. Four synthetic
writers are a stress, not a replay of a shard, so a red here means
"exposed", not "will go red". The absolute seconds are shared-box
readings.
| Package (files measured) | Idle, base | Under load, base | Under load,
head |
|:--|:--|:--|:--|
| `create-objectstack` (all 19) | 19/19 files, 21 s | **2/19 files
fail** (afterAll hook timeouts, 0/301 cases), 236 s | 19/19 files, 20 s
|
| `cli` (the 15 `organization-ownership-*` files) | 232/232, 68 s |
**145/232 cases time out**, 7 files, 1,809 s | 232/232, 59 s |
| `runtime` (51 files using SQLite under `os.tmpdir()`) | 684/684, 102 s
| **25/684 time out**, 7 files (package revert, uninstall,
duplicate-adopt, list-commits, audit-meta-item), 501 s | 684/684, 110 s
|
| `driver-turso` (14 files using `os.tmpdir()`) | 502/502, 17 s |
**21/502 time out**, 4 files (inherited-members, external-object-parity,
drift-detection, migration-entries), 211 s | 502/502, 17 s |
| `driver-sql` (23 files using `os.tmpdir()`) | 192/192, 28 s | **3/192
time out** (posture conformance, reclaim-space, declared-index retired
keys), 98 s | 192/192, 28 s |
| `service-package` (2 files) | 11/11, slowest case 0.17 s | 11/11
green, slowest case **3.04 s** | 11/11, slowest case 0.15 s |
| `service-automation` (4 files) | 86/86, slowest case 0.31 s | 86/86
green, slowest case **4.35 s** of its 5 s budget | 86/86, slowest case
0.34 s |
Every failure under load on the base is `Test timed out` or `Hook timed
out`. None is an assertion failure, and none fails idle.
`service-package` and `service-automation` stayed green in this load run
(the card's survey had 5 of 11 and 1 of 8 failing). They are treated
because the card lists them as exposed and the readings put them at 61%
and 87% of budget for no change in code.
Full package suites on the head: each affected package's own suite,
idle, on the final head `110f05aa` (after merging `origin/main`), unless
noted:
- `cli` unit layer (`vitest run --project unit`): 346/346 files,
5,032/5,032 tests. The integration layer is CI's; its 14 measured
`organization-ownership-*` integration files are in the table above.
- `runtime` (`pnpm test`): 471 files, 7,668 passed and 19 skipped.
- On `fb2d18d7`, before the merge of `main`, which touched no file of
these five: `driver-turso` 95 files, 2,492 passed and 34 skipped;
`driver-sql` 236 files passed and 12 skipped, 3,847 tests passed and 214
skipped; `service-package` 10/10 files, 127/127; `service-automation`
217/217 files, 2,768/2,768; `create-objectstack` 19/19 files, 301/301;
`driver-sqlite-wasm` 39/39 files, 746/746.
## Acceptance notes
- `service-print` (the card's Chromium-profile exception) and the other
fsyncing packages in the card's survey (`dogfood`, `plugin-auth`,
`verify`, `service-datasource`, `plugin-security`,
`plugin-pinyin-search`, `rest`) are not touched. The card's own table
measured the last four of those green under load, and `dogfood`,
`verify` and `rest` were not in its exposure table.
- Some `cli` tests deliberately scaffold under
`packages/cli/node_modules/…` or `packages/cli/tmp` (they need the
workspace's module resolution) and are not moved by `TMPDIR`. They were
not in the measured failing family, and this PR leaves them alone.
- The survey's evidence file
(`pm-orders/server/evidence/27076/fsync-survey.md`) was not reachable
from this account, so every reading above was re-measured here.
- No changeset: the diff touches only vitest configs and a root test
helper, none of which is in any package's published `files`. `Check
Changeset` reports whatever it reports, and the seat decides on the
label.
## Verification
All on the final head `110f05aa`.
<details><summary>Derived gate families: 45 derived, 45 run, all exit
0</summary>
`node scripts/pm/dispatch-gates.mjs --commands --repo
objectstack-ai/objectstack` derived 45 families from the 9 changed
paths. Each was run and its exit code recorded, then reconciled with
`--ran`: `✓ dispatch-gates --ran: 45 derived famil(ies) accounted for —
45 run, 0 NOT-MEASURED`. Two notes. `check-plugin-teardown-shape.mjs
--self-test` first exited 3 on a shallow clone (its pinned fixture
commit was unreachable) and exited 0 after `git fetch --unshallow`.
`check:dts-closure` and `check:dual-build-cjs-loads` read built output,
so they were run after a full `turbo run build`.
</details>
<details><summary>Lint</summary>
`eslint --no-inline-config` over the nine touched files: exit 0. The
repo-wide `pnpm lint` is CI's.
</details>
<details><summary>How the load was made</summary>
Four loops of `dd if=/dev/zero of=FILE bs=1M count=256 conv=fsync` on
the same disk as `/tmp`, started 3 s before the suite and stopped after
it. Each base/head pair used the same file list and `--maxWorkers=2`.
The per-file and per-case durations come from vitest's JSON reporter.
</details>
## Human Minimum
- [x] 1. Schemas start as Zod; `packages/spec` holds no runtime logic.
(not touched)
- [x] 2. Every name I added uses the casing for its kind.
- [x] 3. No formatter ran; `pnpm lint` passes. (eslint on the nine
touched files, exit 0; the repo-wide run is CI's)
- [x] 4. One issue, one branch off an up-to-date `main`, one pull
request.
- [x] 5. A change that publishes carries a changeset, and both
declaration lines are filled in here and in the changeset. (publishes
nothing)
- [x] 6. `packages/spec` touched: its generated artifacts are
regenerated and committed. (not touched)
- [x] 7. No hand edits to `content/docs/references/`,
`packages/*/CHANGELOG.md` or `content/docs/releases/`.
- [x] 8. Title, body and comments are in English.
- [ ] The *Before Submitting* list in CONTRIBUTING.md is done. (the
local subset is above; the full farm is CI's)
---
_Generated by [Claude
Code](https://claude.ai/code/session_01Utp6CZiAsVSvj5E1XZbWrG)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent ad2c286 commit cf218ee
10 files changed
Lines changed: 168 additions & 25 deletions
File tree
- packages
- cli
- create-objectstack
- drivers
- driver-sqlite-wasm
- driver-sql
- driver-turso
- runtime
- services
- service-automation
- service-package
- scripts
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
651 | 651 | | |
652 | 652 | | |
653 | 653 | | |
| 654 | + | |
654 | 655 | | |
655 | 656 | | |
656 | 657 | | |
| |||
989 | 990 | | |
990 | 991 | | |
991 | 992 | | |
| 993 | + | |
| 994 | + | |
| 995 | + | |
| 996 | + | |
| 997 | + | |
| 998 | + | |
| 999 | + | |
| 1000 | + | |
| 1001 | + | |
992 | 1002 | | |
993 | 1003 | | |
994 | 1004 | | |
| |||
999 | 1009 | | |
1000 | 1010 | | |
1001 | 1011 | | |
1002 | | - | |
| 1012 | + | |
| 1013 | + | |
| 1014 | + | |
1003 | 1015 | | |
1004 | 1016 | | |
1005 | 1017 | | |
| |||
1012 | 1024 | | |
1013 | 1025 | | |
1014 | 1026 | | |
1015 | | - | |
| 1027 | + | |
| 1028 | + | |
| 1029 | + | |
1016 | 1030 | | |
1017 | 1031 | | |
1018 | 1032 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
| 4 | + | |
4 | 5 | | |
5 | 6 | | |
6 | 7 | | |
| |||
38 | 39 | | |
39 | 40 | | |
40 | 41 | | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
41 | 54 | | |
42 | 55 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
| 5 | + | |
5 | 6 | | |
6 | 7 | | |
7 | 8 | | |
| |||
14 | 15 | | |
15 | 16 | | |
16 | 17 | | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
17 | 27 | | |
18 | 28 | | |
19 | 29 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | | - | |
4 | 3 | | |
| 4 | + | |
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| |||
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
30 | | - | |
31 | | - | |
32 | | - | |
33 | | - | |
34 | | - | |
35 | | - | |
36 | | - | |
37 | | - | |
38 | | - | |
39 | | - | |
40 | | - | |
41 | | - | |
42 | | - | |
43 | | - | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
44 | 33 | | |
45 | 34 | | |
46 | 35 | | |
| |||
51 | 40 | | |
52 | 41 | | |
53 | 42 | | |
54 | | - | |
| 43 | + | |
55 | 44 | | |
56 | 45 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | | - | |
4 | | - | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
| 37 | + | |
37 | 38 | | |
38 | 39 | | |
39 | 40 | | |
| |||
231 | 232 | | |
232 | 233 | | |
233 | 234 | | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
234 | 244 | | |
235 | 245 | | |
236 | 246 | | |
| |||
242 | 252 | | |
243 | 253 | | |
244 | 254 | | |
245 | | - | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
246 | 258 | | |
247 | 259 | | |
248 | 260 | | |
| |||
258 | 270 | | |
259 | 271 | | |
260 | 272 | | |
261 | | - | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
262 | 276 | | |
263 | 277 | | |
264 | 278 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
| 3 | + | |
3 | 4 | | |
4 | 5 | | |
5 | 6 | | |
| |||
38 | 39 | | |
39 | 40 | | |
40 | 41 | | |
41 | | - | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
42 | 49 | | |
43 | 50 | | |
44 | 51 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | | - | |
| 3 | + | |
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
| 9 | + | |
9 | 10 | | |
10 | 11 | | |
11 | 12 | | |
| |||
19 | 20 | | |
20 | 21 | | |
21 | 22 | | |
22 | | - | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
23 | 30 | | |
24 | 31 | | |
25 | 32 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
0 commit comments