diff --git a/.agents/docs/2026-10-22-issues-59-63-evidence/.gitattributes b/.agents/docs/2026-10-22-issues-59-63-evidence/.gitattributes new file mode 100644 index 00000000..500129dd --- /dev/null +++ b/.agents/docs/2026-10-22-issues-59-63-evidence/.gitattributes @@ -0,0 +1,2 @@ +# Unified diff context for an empty line is a literal single space. +engine-memory-*.patch whitespace=-blank-at-eol diff --git a/.agents/docs/2026-10-22-issues-59-63-evidence/HANDOFF.md b/.agents/docs/2026-10-22-issues-59-63-evidence/HANDOFF.md new file mode 100644 index 00000000..6c7978e3 --- /dev/null +++ b/.agents/docs/2026-10-22-issues-59-63-evidence/HANDOFF.md @@ -0,0 +1,37 @@ +# PR65 同步交接 + +此前评论里的“本地修复”没有 commit/push，这是交接缺口。本次将其实际提交到 `fix/17-connection-writer-encoding`，并附带[可恢复的精简证据包](portable-handoff/README.md)。接手者不需要访问原机器。 + +获取：`gh pr checkout 65 --repo Sunrisepeak/mcpp-language-server`；或者在已有仓库中 `git fetch origin fix/17-connection-writer-encoding` 后检查该分支。不要在 `release/0.0.16` 分支上寻找未合入的修复。 + +## 已同步文件 + +- `src/orchestrator/workspace.cpp`、`src/project/scan.cpp/.cppm`：成功 producer 回答确认 run-start 源结构，防止条件 import 导致 body-only 保存重复 emit；代数/身份/stamp、失败、inflight、ABA 和未知预处理均保守处理。仅确认扫描 opt-in 保留预处理文本。 +- `tests/test_answer_routing.cpp`、`tests/test_scan.cpp`、`conformance/fixtures/mcpp-watched-source-structure/`：对应原生及完整 70 项 LSP 对照。 +- `src/server/session.cpp/.cppm`、`tests/test_session.cpp`：拒绝已有普通文件作为 workspace directory，保留原目录回退顺序。 +- `editors/zed/tests/{smoke.sh,lsp_frames.py}`、CI workflows：真实目录与 mcppls 自身 clangd 初始化/进程检查、Zed PATH clangd、Termux 官方包源、首失败日志/producer envelope 保留。 +- `conformance/fixtures/mcpp-target-windows-real/`：真实 std 依赖修正，原 validator 未放宽。 +- `src/bin/conformance.cpp`、`tests/test_conformance.cpp`：有界 completion 请求时间线、最慢支持点及已有前后 report 的覆盖边界。 +- 本目录 `engine-memory-{proposal.md,proposal.patch,native-canary.patch,proposal-source.json}`：完整四份本地引擎提案，**未应用到引擎、未编译 canary、未发布或修改 payload lock**。 + +## 实际验证与限制 + +修复是在 `e01e1c1` 基础上的 dirty snapshot 验证，包中保留 exact patch、未跟踪 fixture/test 快照、修订和二进制 hash；不能将这些记录冒充新 commit 的远端 CI。 + +| 验证 | dev | release | +|---|---|---| +| 完整 workspace | 6 members / 58 programs / 0 failed | 6 members / 58 programs / 0 failed | +| 原 31 组 LSP + 新 fixture | 32 组 / 0 failed | 32 组 / 0 failed | +| 新 watched-source fixture | 70 checks / 0 failed | 70 checks / 0 failed | +| 重点原生 | scan 23/146、Workspace 29/399、Session 3/12，均 0 fail | 同左 | + +重点 Session 测试最初两处 string/JSON 比较编译失败，已修为显式 `.get()`；首次日志和修订均保留。规范校验 316 rules、0 failure、4 项既有 pending。新源版本 CI、真实 Windows 双目标链、新产品完整性能验收、VSIX 实际升级与发布验证仍未完成。 + +## 未解决的问题 + +- 原 `1341938` nightly 四项失败仍成立：mcpp U5/U6 慢请求、U15 在 U6 内严格 60 秒窗口 13/min、xlings idle RSS +25.33%、two-core timeout/U7a 超时；不由后来 PASS 抹去。 +- 新 trace 四阶段为 25 checks / 1 failure。本次 U5/U6 未复现历史慢请求；U7a 107.247s 来自 setup 阶段 config AST 崩溃后的 120s 隔离。其他七个 importer <6s，恢复后 AST 约 .730s；底层 crash 仍未归因。不要通过缩短冷却或改预算制造通过。 +- 引擎 .4 的周期 memo JSON 在版本判断前构造，独立无效分配已证明；尚不能归因全部 RSS 增长。引擎提案只作为候选，不代表已获授权实施。 +- 远端旧 `e01e1c1` CI/nightly 仍失败：Windows schema precheck、Termux 镜像 404、Zed 默认模式 API403 及 file-as-folder 产品缺陷；新修订需实际新 CI。 + +当前要求是同步并交接，PR 继续 draft，**不合并、不发布 0.0.17、不修改其他仓库或 issue 状态**。接手先核验本提交 diff 与证据，再按简报继续调查。发布暂缓；不能因为后来 CI 通过就自动发布。原两个实际 0.0.16 升级 profile 不重置。用户 `.zcodeignore`、`.agents/docs/2026-10.12-implementation-status-tmp` 未纳入提交。 diff --git a/.agents/docs/2026-10-22-issues-59-63-evidence/README.md b/.agents/docs/2026-10-22-issues-59-63-evidence/README.md index 9e0d041b..b4b4f65a 100644 --- a/.agents/docs/2026-10-22-issues-59-63-evidence/README.md +++ b/.agents/docs/2026-10-22-issues-59-63-evidence/README.md @@ -1,6 +1,8 @@ # #59 / #63 验证简报 -日期：2026-10-22 JST。**R1 产品补修本地已通过；新 CI 的 CPU 采样测试与原九阶段 warm 超时仍待解决，PR 保持草稿。** +交接获取修正：源码、四份引擎提案及精简证据现随 PR 分支提交。见 [HANDOFF.md](HANDOFF.md) 和[证据恢复入口](portable-handoff/README.md)，不再要求接手者拥有原机器的 `.payload-cache`。两 profile 完整 workspace 各 58 程序、LSP 各 32 组（含 70 项 fixture）均已通过；新源版本远端 CI 与性能问题仍待验证。 + +日期：2026-10-22 JST。**PR #64 已合入；PR #65 的产品补修在提交 `1341938` 通过 PR CI 与本地原九阶段。最终范围审计补充真实 mcpp Windows 双目标链，新增验收仍待原生 CI；nightly 和正式发布/升级尚未完成。** 本目录只收结论、[复现步骤](reproduce.md)和 [SHA-256 索引](raw-index.tsv)。45 份原始记录共 26,769 行，已移出 PR；失败、对照、日志与 core 均保留。源码边界见 [设计方案](../2026-10-22-issues-59-63-target-and-reliability-design.md)。 @@ -56,6 +58,59 @@ R1 补修本地 dev/release 全 workspace 均为 6 members / 57 programs / 0 fai 同一新产品的原性能首轮 mcpp warm ready **15.219 s > 12 s**，真实失败；176 已有模块零重建/零首次构建、零错误/重启。producer/model/首次导航很快，慢段位于引擎 prerequisite reuse 检查与扫描。该段采样约 0.343 core；前检查点约 1.353 core，总 CPU 工作近似相当，但缺当时 IO、调度及宿主 pressure 证据，不归因外部负载或认定产品无关。其余阶段继续，不能将旧九阶段通过或后来诊断通过替代本轮失败。 +## 合入检查点与发布前发送补修 + +`38366b9` 的 PR CI 39720450692 为 61 success / 1 expected skip；proot 首次下载 VS Code 超时，单项原样重试后实际 LSP/editor 通过，首日志保留。nightly 39720447763 为 62 success / 4 expected skips，全部八组 UX 的 34 阶段 / 207 检查实际通过。固定 observer 对照四组 cold→firstwarm 全部退出 0，warm 为 3.92/4.06/4.16/3.98 秒，没有证明观察器导致首次超时。 + +另一次预登记的完整九阶段最终验收为 9 exit 0 / 53 checks / 0 failed，保留原 CPU、pins、producer、cache lineage 和门槛；mcpp warm 3.824 秒，U8 undo 均为 0。原 600 秒 xlings idle 为 0.004819 core、RSS 1258.30→1121.86 MiB，有限工作单列。此前 15.219 秒失败仍未归因，未因新通过改写。该检查点于 `fd869bd` squash 合入；原始材料及小型终态摘要保存在 `.payload-cache/59-63-performance/light-observer-final-acceptance-f6ec554/`。 + +随后 draft Release 39732244483 的 macOS dev job 119670268987 在原 `test_process.cpp` 发送检查失败：4 MiB 入队应小于 2 秒；CPU oracle 测试通过。旧断言未打印实际耗时，不能给出精确超时值或归因宿主。代码确认 JSON 编码仍在调用线程同步执行；补修将拥有所有权的 JSON 快照交给 writer 编码、写入，保持 FIFO、UTF-8 和两种 framing，并隔离 writer 编码异常。原 4 MiB / 2 秒及 stop 10 秒门槛不变；新产品必须重新验收，不能借用前述产品通过。见[复现第 11 节](reproduce.md#11-发送队列的编码与快照)。 + +## 新增状态一致性回归与证据边界 + +`072c230` 的 CI 39735383721 有真实 U15-edits 失败：最忙一分钟 13 次状态转换，超过原 12 次门槛；169 次采样、总 28 次转换、两次配置重启、零 error。其余 cold/warm/typing/graph/config undo/faults 通过不抵消该失败。原 artifact 只有汇总，旋转 server 日志也没有完整 status 轨迹，**不能确定这 13 次的原因**。 + +独立回归证明同步 apply 的一致性缺陷：真实 clangd 在安装新 primer/安排重启前会调用 `host.status_changed()`，workspace 此时已采用新 profile。相同回调时序的定制 core 在旧产品上仅一个断言失败，暴露了引擎尚未 commit 的新 profile/ready；无回调对照通过。补修只合并同一同步 apply 栈内的状态通知，所有参与引擎完成后发布；不增加延时、不扩展 hold、不隐藏异步状态。修后 dev/release 各 28 cases / 357 assertions 通过，其他验证仍在进行。这一独立缺陷不能冒充首个 13 次失败的归因。 + +runner 新增紧凑 `stateTransitionTrace` 与 `busiestMinuteWindow`：保存初始状态、重启边界和实际计数的转换，带整数纳秒及采样索引，能按原严格 `<60s` 窗口重算。只保留 issue code，不复制完整 status 或路径；原采样、计数和门槛不变。完整新产品性能链与平台 CI/nightly 尚未完成。见[复现第 12 节](reproduce.md#12-同步-apply-的状态快照与时间线证据)。 + +## 补修产品最终本地验收与真实 Windows 生产链边界 + +提交 `1341938` 的 PR CI 39742414940 为 **46 success / 1 条件跳过**。两种 profile 各 6 members / 57 programs、31 组真实 LSP、原完整 build-programs 与实际 VS Code（44 passing / 1 平台专用 pending）均零失败。原九阶段单次完整运行 **9 exit 0 / 53 checks / 0 failed**，fixture、CPU、producer、cache lineage 与预算保持原值；mcpp/xlings warm ready 为 **6.572 / 2.435 秒 <12 秒**，已有模块分别 176/111，均零重建；xlings 另有 39 首次构建。U8 撤销均零重建，U15 edits 最忙一分钟为 4/2 次。 + +原 600 秒 xlings idle 为 **0.004954 core**，RSS **1108.40→1129.73 MiB（+1.92%）**；之前 62.021 秒有限工作单列，不混入空闲值。独立重算全部状态轨迹和严格 `<60s` 窗口，9 个 observer 均正常结束，303 个观测到的 PID/birth 身份已退出；1 秒采样可能漏掉瞬时子进程。终态摘要在 `.payload-cache/59-63-performance/light-observer-final-acceptance-status-1341938/terminal-summary.json`，原 13/min 与 15.219 秒失败及其未归因边界不变。 + +范围审计发现：原 `mcpp-target-windows` 是 mock producer + 真实 clangd，它证明目标传递和条件声明，但不能代替设计要求的真实 mcpp 原生双目标生产链。新增 `mcpp-target-windows-real` 使用真实 `mcpp build --configure-only --target` 生成两份完整 CDB，不改写条目；同一会话切换 i686→x64→i686，校验 auto 来源、实际引擎命令、32/64/32 条件声明、导航与诊断。Windows 原生 CI 实际通过前，该项保持待验收；没有把 mock 通过扩大为真实生产链通过。 + +## 134 nightly 首次失败与后续修复边界 + +`1341938` 的 nightly [39742450508](https://github.com/Sunrisepeak/mcpp-language-server/actions/runs/39742450508) 最终为 **58 success / 4 failure / 4 skipped**，不能由本地九阶段或 PR CI 通过替代： + +| 首次失败 | 原始门槛与观测 | 当前结论 | +|---|---|---| +| mcpp round 2 | U5 p95 2.374 s / max 8.264 s；U6 max 6.384 s，超过 1/3/3 s 预算 | 未归因。server completion max 8.262 s 与客户端一致，不能只归为 runner 接收延迟 | +| mcpp round 1 | U15 edits 最忙一分钟 13 次 >12 | 重算跨度 59,999,840,473 ns，确实在严格 `<60s` 内；全部支持点位于 U6 autosave，不能归给后续 U8 | +| xlings two-core | U5 timeout 1；U7a p95/max 1.53 s >1.5 s | 保留实际失败，best effort 不等于通过 | +| xlings round 1 | 原 600 s idle CPU 0.006108 core；RSS 948.086→1188.250 MiB，增长 25.33% >10% | idle 前 126.419 s 有限工作单列；idle 无引擎活动事件。独立 memo 分配问题不能冒充本次全部增长的精确归因 | + +原日志和四组 measure 保存在本地 `.payload-cache/59-63-real-target-integration/first134-nightly-*`；U15 的逐点重算为 `first134-nightly-mcpp-round1-window-recompute.json`。PR 仅保留本简报和复现入口，不提交完整原始输出。 + +新的 completion trace runner 已在 dev/release 各通过 11 cases / 443 assertions；旧 runner 跑新增测试恰有两项失败，原九项通过。trace 冻结原采样回答集合后才构建 JSON，保留最多 256 条有序请求及覆盖全部请求的最慢支持点，不改变采样、超时或门槛。使用旧产品和新 trace runner 的单次四阶段诊断已结束：25 checks / 1 failure，168 个观测 PID/birth 身份均退出。U5/U6 本次 max 为 0.861/0.707 s，未复现历史慢请求，不能解释或抹去原失败；唯一 U7a failure 是 importer diagnostics 107.25 s >33 s。完整证据显示一个 importer 在保存前 clangd AST 崩溃后被隔离，其他七个约 5.6 s 内重发，迟到 importer 约两分钟后恢复；底层崩溃与安全恢复条件仍在审查。**这不是产品性能修复或新产品验收**。 + +另有独立真实 LSP 反例：合法 `#if 0` import 被 producer 正确排除，旧产品却因 lexical/model 差异，对 body-only 和相同字节保存重复 emit。新 fixture 在旧产品上为 11 次 emit、14 个失败检查（包括计数级联，非 14 个独立缺陷）；修复须确认成功生成开始时的结构，保留失败重试、生成中变化、预处理变化、target/producer/manifest 的重载。产品修改仍在验证，**不宣称它解释了历史 13/min**。 + +`e01e1c1` 的 Windows job 在 schema precheck 提前失败：新真实目标 fixture 未使用 `std`，实际 producer 不输出原契约要求的 `mcpp:std`。fixture 已补真实 `import std` 和类型断言；Linux 上真实 pinned producer 的 schema 对照通过，原 validator 未放宽；Windows 真正双目标 LSP 验收仍待新 CI。新增 artifact 保存 schema 阶段最初的 producer envelope，即使尚未进入 LSP fixture 也可审查。 + +## e01 CI 与 Zed/Termux 的新增失败 + +`e01e1c1` 的 PR CI [39752365858](https://github.com/Sunrisepeak/mcpp-language-server/actions/runs/39752365858) 最终 **45 success / 1 failure / 1 skipped**；唯一失败为上述 Windows schema precheck。nightly [39752360539](https://github.com/Sunrisepeak/mcpp-language-server/actions/runs/39752360539) 尚未结束，另已保存两个实际失败： + +- Termux 随机镜像对 `proot-distro_5.6.0_all.deb` 返回 404，在 Debian login 和任何产品验证前退出。修订固定[官方主源](https://github.com/termux/termux-packages/wiki/Mirrors)，不再吞 update 失败，原 bind/LSP 检查保留；静态 shell/YAML 检查通过，实际容器验证待新 CI。 +- Zed 默认模式 240 秒内没有自己的 clangd，日志记录 GitHub API 403。CI 修订预装 PATH clangd，仍要求 Zed 按默认设置亲自启动独立进程；这不再声称测试自动下载。 +- 同一 Zed artifact 还证明独立产品兼容缺口：`workspaceFolders` 指向真实 `.cppm` 文件，而 `rootUri` 是目录；mcppls 采用文件作为 clangd cwd，出现 `engine-start-failed`。修订忽略已有普通文件的 folder 项并沿原目录回退顺序选择，不把文件的 parent 猜成项目。新 native 回归及真实 Zed 复验待执行。 + +原 smoke 仅需任意 diagnostics，模块层错误也可满足，故新 smoke 同时要求 mcppls 自身 clangd 完成协议初始化并存活，不能用 Zed 的另一个 clangd 代替；这仍不宣称全部 BMI 准备完成。原始 initialize、server 日志和首次失败完整 job log 保存在本地 `zed-smoke-default-first/` 和 `nightly-{termux,zed}-*-first.log`。artifact 将补收自动诊断包，以便下一次启动失败保留更精确的错误。上述源修改均尚未经过新产品完整验证。 + ## 历史失败与反证仍保留 | 记录 | 结论边界 | @@ -82,4 +137,4 @@ R1 补修本地 dev/release 全 workspace 均为 6 members / 57 programs / 0 fai - [raw-index.tsv](raw-index.tsv) 为逐文件大小与 SHA，不按通过筛选；本地历史快照 `.payload-cache/59-63-evidence-raw/checkpoint-20261021T202025Z-5e19005.tar.gz`，SHA `674b5072ddfb27204aef5bc2de61cf13ad38176f3e5a79c00a5cda382e15a8c0`。 - 快照后的记录独立保存在上述目录、envelope 与 CI artifacts，不混入旧索引。较完整的历史叙述还可从 Git 的 `bcc9cbe` 版简报读取。 - 已结束日志的 `.log.gz` 为无损压缩，`compressed-logs.jsonl` 记录原始字节数/SHA/解压核验；core 保留本地。早期丢失的 `/tmp` 跟踪不冒充完整证据。 -- 空间不足只清理本项目过期、无进程引用的构建中间文件；当前冻结、升级 profile、性能缓存、日志及 core 不动。清理范围与保留二进制 SHA 另有 JSON 记录。 +- 空间不足只清理本项目过期、无进程引用的构建中间文件。2026-10-22 另清理 32 个历史独立性能 cache 内的 10,170 个 PCM/对象文件，释放 11,077,361,664 B；保留这些运行的 JSON、日志和输入，历史缓存已不能据此复用重跑。当前诊断与验收缓存、冻结二进制、升级 profile、日志和 core 不动。逐文件元数据和进程引用检查见本地 `cleanup-2026-10-22-obsolete-performance-{plan,report}.json`。 diff --git a/.agents/docs/2026-10-22-issues-59-63-evidence/engine-memory-native-canary.patch b/.agents/docs/2026-10-22-issues-59-63-evidence/engine-memory-native-canary.patch new file mode 100644 index 00000000..28f38a47 --- /dev/null +++ b/.agents/docs/2026-10-22-issues-59-63-evidence/engine-memory-native-canary.patch @@ -0,0 +1,126 @@ +--- a/clang-tools-extra/clangd/ModuleMemoStore.h ++++ b/clang-tools-extra/clangd/ModuleMemoStore.h +@@ -44,6 +44,10 @@ + llvm::json::Value moduleMemosJSON(); + size_t adoptModuleMemos(const llvm::json::Value &Memos); + ++/// Native-test seams. Poll is allowed only after flush joined the worker. ++size_t moduleMemosJSONBuildCountForTest(); ++bool pollModuleMemoStoreForTest(); ++ + } // namespace clang::clangd + + #endif +--- a/clang-tools-extra/clangd/ModuleMemoStore.cpp ++++ b/clang-tools-extra/clangd/ModuleMemoStore.cpp +@@ -17,6 +17,7 @@ + #include "llvm/Support/MemoryBuffer.h" + #include "llvm/Support/Path.h" + #include "llvm/Support/raw_ostream.h" ++#include + #include + #include + #include +@@ -24,6 +25,8 @@ + + namespace clang::clangd { + namespace { ++ ++std::atomic JSONBuilds{0}; + + constexpr int FormatVersion = 2; + constexpr llvm::StringLiteral FileName = "module-memos.json"; +@@ -392,6 +395,7 @@ + // ---- the whole -------------------------------------------------------------- + + Value memos(uint64_t &Verdicts, uint64_t &Scans) { ++ JSONBuilds.fetch_add(1, std::memory_order_relaxed); + auto VerdictSnapshot = getModuleValidationCache().snapshot(Verdicts); + auto ScanSnapshot = getModuleDependencyScanCache().snapshot(Scans); + Array VerdictsJSON; +@@ -443,6 +447,13 @@ + public: + explicit Store(std::string Directory) : Directory(std::move(Directory)) {} + ++ bool pollForTest() { ++ if (Worker.joinable()) ++ return false; // never race the periodic writer with the test ++ write(); ++ return true; ++ } ++ + void start() { + Worker = std::thread([this] { + load(); +@@ -578,7 +589,17 @@ + S->flush(); + } + ++size_t moduleMemosJSONBuildCountForTest() { ++ return JSONBuilds.load(std::memory_order_relaxed); ++} ++ ++bool pollModuleMemoStoreForTest() { ++ Store *S = theStore(); ++ return S && S->pollForTest(); ++} ++ + llvm::json::Value moduleMemosJSON() { ++ + uint64_t Verdicts, Scans; + return memos(Verdicts, Scans); + } +--- a/clang-tools-extra/clangd/unittests/PrerequisiteModulesTest.cpp ++++ b/clang-tools-extra/clangd/unittests/PrerequisiteModulesTest.cpp +@@ -1 +1,2 @@ + #include "ModuleMemoStore.h" ++#include "llvm/Support/MemoryBuffer.h" +--- a/clang-tools-extra/clangd/unittests/PrerequisiteModulesTest.cpp ++++ b/clang-tools-extra/clangd/unittests/PrerequisiteModulesTest.cpp +@@ -1 +1,46 @@ ++// Run this lifecycle canary in an isolated process: Store is a process singleton. ++// Poll uses the real periodic write path, after flush has joined its worker. ++TEST_F(PrerequisiteModulesTests, UnchangedMemosDoNotConstructJSON) { ++ writeTestFile(TestDir, "quiet-input.cpp", "int before;\n"); ++ writeTestFile(TestDir, "quiet.pcm", "identity-only test payload"); ++ const auto Input = getFullPath("quiet-input.cpp"); ++ const auto BMI = getFullPath("quiet.pcm"); ++ const auto File = getFullPath("quiet-memos/module-memos.json"); ++ auto Identity = ModuleValidationCache::identityOf(Input); ++ ASSERT_TRUE(Identity); ++ getModuleValidationCache().markFresh(BMI, "quiet-canary", {{Input, *Identity}}); ++ const auto Before = moduleMemosJSONBuildCountForTest(); ++ startModuleMemoStore(getFullPath("quiet-memos")); ++ flushModuleMemoStore(); ++ const auto First = moduleMemosJSONBuildCountForTest(); ++ EXPECT_EQ(First, Before + 1); ++ std::string InitialBytes; ++ { ++ auto InitialFile = llvm::MemoryBuffer::getFile(File); ++ ASSERT_TRUE(InitialFile); ++ InitialBytes = (*InitialFile)->getBuffer().str(); ++ } // close the mapping before a later rename, including on Windows ++ for (unsigned I = 0; I != 10; ++I) ++ ASSERT_TRUE(pollModuleMemoStoreForTest()); ++ EXPECT_EQ(moduleMemosJSONBuildCountForTest(), First); ++ { ++ auto QuietFile = llvm::MemoryBuffer::getFile(File); ++ ASSERT_TRUE(QuietFile); ++ EXPECT_EQ((*QuietFile)->getBuffer(), InitialBytes); ++ } ++ // Dirty data still builds JSON and reaches disk on the same periodic path. ++ writeTestFile(TestDir, "quiet-input.cpp", "int after_changed;\n"); ++ Identity = ModuleValidationCache::identityOf(Input); ++ ASSERT_TRUE(Identity); ++ getModuleValidationCache().markFresh(BMI, "quiet-canary", {{Input, *Identity}}); ++ ASSERT_TRUE(pollModuleMemoStoreForTest()); ++ EXPECT_EQ(moduleMemosJSONBuildCountForTest(), First + 1); ++ auto ChangedFile = llvm::MemoryBuffer::getFile(File); ++ ASSERT_TRUE(ChangedFile); ++ EXPECT_NE((*ChangedFile)->getBuffer(), InitialBytes); ++ // Final flush deliberately keeps the pre-fix snapshot behavior. ++ flushModuleMemoStore(); ++ EXPECT_EQ(moduleMemosJSONBuildCountForTest(), First + 2); ++} ++ + TEST_F(PrerequisiteModulesTests, MemosKeptForTheNextSession) { diff --git a/.agents/docs/2026-10-22-issues-59-63-evidence/engine-memory-proposal-source.json b/.agents/docs/2026-10-22-issues-59-63-evidence/engine-memory-proposal-source.json new file mode 100644 index 00000000..f40a5715 --- /dev/null +++ b/.agents/docs/2026-10-22-issues-59-63-evidence/engine-memory-proposal-source.json @@ -0,0 +1,10 @@ +{ + "engineForkCommit": "92a4c630f4691890e6133c87d3e4591bc5fd8258", + "engineVersion": "23.1.0-mcppls.4", + "pinned0030SHA256": "091fa15f06a736955f331c3025bf73bcd891bce7de81024c8f3826588f0b9986", + "runtimePatchSHA256": "143bebe8e0d335620ddfe18350dd8a7c22629d50050bbcb1c3f512ec4f3b0122", + "nativeCanaryPatchSHA256": "3c12e2a2753ff3560a9238d59ac78d36c0ac8a3ccf33e15fc09de770cfa1fbbf", + "status": "local proposal only; no LLVM build, no engine checkout changes, no registration/comment/publication", + "contextValidation": "Store.cpp/header reconstructed exactly from added-file content of pinned0030; cache snapshot contexts from same patch; full released source git apply --check pending authorization", + "shutdownPolicy": "Periodic writes get early-version fast path; flush forces original snapshot path because ClangdMain permits lingering cache writers" +} diff --git a/.agents/docs/2026-10-22-issues-59-63-evidence/engine-memory-proposal.md b/.agents/docs/2026-10-22-issues-59-63-evidence/engine-memory-proposal.md new file mode 100644 index 00000000..34375093 --- /dev/null +++ b/.agents/docs/2026-10-22-issues-59-63-evidence/engine-memory-proposal.md @@ -0,0 +1,74 @@ +# 引擎 memo 空闲分配：本地修复提案 + +状态：供 review、随 PR 同步的交接候选，当前不实施引擎修复。未编译 LLVM、未修改任何引擎工作树，未发引擎评论、PR 或 release。原设计首段限定“只修改 mcppls 仓库”；本提案只是存放在本仓库的评审材料。根任务尚未决定是否跨仓实施；先确认引擎修改范围是本提案的 scope 建议，不是 skill 强制的审批步骤。用户既有指令禁止 GitHub issue 评论及状态变更；即使后续允许修改引擎，也不自动授权登记上游问题或对外发布，相关外发仍须另获明确授权。不能用修改 payload lock 代替尚未完成的引擎修复和验收。 + +## 已证明的缺陷与尚未证明的归因 + +精确发布输入为 fork `92a4c630f4691890e6133c87d3e4591bc5fd8258`、`23.1.0-mcppls.4`。原始 `0030` 位于本地 `engine-memory-review/pinned-0030.patch`，其 SHA 与提案 SHA 记录于 [engine-memory-proposal-source.json](engine-memory-proposal-source.json)。 + +`Store::write()` 先调用 `memos(Verdicts, Scans)`，复制两个缓存的 snapshot、构造完整 JSON，之后才判断版本是否与 `Written*` 相同。因此，每个无变化的 30 秒周期也会构造并销毁整份 JSON；“没有写文件”不等于没有分配。该控制流缺陷可直接由发布源码证明，与任何一次 RSS 曲线是否复现无关。 + +首次 `1341938` nightly xlings round1 的原 U13 仍为 FAIL：600 秒 CPU `0.0061084758` core，RSS `948.0859375 → 1188.25 MiB`，增长 `25.3314655%`；原 CPU `0.01`、RSS `10%` 门槛不变。已知有限工作 `126.418657918s` 已单列，前后 preparation/index/request 状态一致，记录的 engine activity 为零。原始 measure 是 `.payload-cache/59-63-real-target-integration/first134-nightly-xlings-round1-measures/ux-xlings-1-long.json`。 + +这些数据**尚不能证明这次 +25.3% 全部来自 memo JSON**：原 artifact 没有分配栈、逐 PID smaps/allocator 归属或序列化期间 heap 证据；当前 activity 计数也不是全引擎分配计数。提案修复一个独立、已知的无效分配路径，不能先承诺修复全部 U13，更不能把之后的 PASS 覆盖首次 FAIL。 + +## 最小运行时改动 + +[engine-memory-proposal.patch](engine-memory-proposal.patch) 仅涉及三个发布源码文件，21 行新增、2 行修改： + +1. `ModuleValidationCache::changeCount()` 在 `Mutex` 内读取 `Changes`。 +2. `ModuleDependencyScanCache::changeCount()` 在 `Mu` 内读取 `Stores`，与现有 `snapshot()` 读取同一计数。 +3. 周期 `Store::write()` 在调用 `memos()` 前比较两个 cheap counter。均等于最近**成功发布**的版本则立即返回；有变化仍走现有 snapshot、JSON、临时文件和 rename 流程，并保留现有 snapshot 后的版本判断。 + +不改变 30 秒周期、JSON 格式、持久化开关、缓存准入、BMI/输入验证、RSS 测量起点、预算或观察频率。计数器采样不遍历 Entries，不复制 manifest，也不构造 JSON。两缓存分别短暂加锁，不同时持有两把锁；无需引入新的跨缓存锁或 Linux 专用 allocator 操作。 + +当前发布补丁中，validation 的插入/判 stale 递增 `Changes`；scan 的 `Entries.push_front` 与 `++Stores` 位于同一 `Mu` 临界区（已有 `0023` 发布补丁上下文）。实施时仍需审计完整发布树的所有持久化可见变化：这次优化复用原先写入决策的版本语义，不应顺便声称修复未计数的 eviction、adopt 或 eligibility 变化。 + +## 并发、失败与退出边界 + +| 时序 | 行为及保证 | +|---|---| +| 无变化 | 至多读取两个计数；直接返回，不调用 `memos()`。 | +| 变化恰好发生在 cheap 检查之后 | 这一 tick 可返回；`Written*` 没有前移，下一 tick 仍可发现变化。不是事件被消费掉。 | +| 构造 JSON 时又发生变化 | 成功 rename 后只保存 `snapshot()` 捕获的两个版本，不能重新读取 live counter 标记为 written。后续变化保持 dirty。 | +| 两个 snapshot 之间有写入 | 沿用原来两个独立 snapshot 的语义，不承诺跨两个缓存的事务原子性；较新的版本会再次被持久化。 | +| create/write/flush/rename 失败 | 沿用原分支，`Written*` 不更新；即便下一轮没有新缓存变化，已有 dirty 版本仍会重试。不能在 cheap 检查时提前更新 written。 | +| 最终 flush | 先 join Store worker，再显式 `write(CheckForChanges=false)`，保留原实现先做 snapshot 的退出路径。周期 fast path 不用于最终 flush。 | + +最后一项有必要：发布版 `ClangdMain` 在 `flushModuleMemoStore()` 之后仍注释可能存在 lingering background threads。Store 的 join 只停止持久化 worker，不能假定所有缓存生产者已经停止。若最终 flush 也使用 early-return，会新增“读到旧 counter 后并发产生变化就直接退出”的窗口。因此本提案保留旧最终 snapshot 路径。 + +也必须说明其上限：若缓存生产者在最终 snapshot **之后**仍产生新条目，旧实现本身就没有全量退出保证。本提案不修改 shutdown 顺序，不应宣称持久化了进程最后一个后台动作。若用户要求这个更强契约，需另审 producer drain → final snapshot 的生命周期排序；不能靠循环到版本稳定、无限等待或改周期补上。 + +## 原生 canary 与负对照 + +[engine-memory-native-canary.patch](engine-memory-native-canary.patch) 是独立的拟议测试补丁，尚未编译。它在真正的 `memos()` 入口计数，而非在成功写文件处计数；增加 test-only accessor，以及 **Store worker 已 join 后**才能调用的周期 write 测试入口。没有新的用户配置或 LSP 字段。 + +`UnchangedMemosDoNotConstructJSON` 使用真实缓存和真实 Store write： + +1. 制造一个 dirty validation 条目，启动 Store 后 flush/join，确认构造次数增加一次、文件可读。 +2. 无版本变化时调用周期 write 入口十次，要求构造计数完全不增加，磁盘 bytes 相同。旧实现虽然也不重写文件，却会再构造十次 JSON，因此同一 canary 必须在仅含测试补丁的 `.4` 实现上 FAIL。 +3. 再次 markFresh，要求下一次周期 write 构造一次并改变持久化文件 bytes；证明 fast path 不把后续 dirty 数据吞掉。 +4. 最终 flush 仍构造一次 snapshot，验证有意保留的退出策略。读取磁盘的 MemoryBuffer 在 rename 前销毁，避免 Windows 映射阻止替换。 + +Store 是进程 singleton，该生命周期 canary 应在独立原生测试进程执行；不能靠等待 30 秒、RSS 阈值或仅比较 mtime 来证明没有 JSON 构造。测试补丁可单独应用于旧实现作负对照；其 poll 入口拒绝在 worker 仍 joinable 时运行，避免测试制造双 writer。 + +合入前还必须补齐这些有控制屏障和故障注入的原生测试，不能以 sleep 近似并发顺序： + +- 在 cheap counter 读取与返回之间发布一次变化：下一次 poll 或强制 flush 包含该版本。 +- 在 snapshot 取得版本后、rename 成功前发布变化：written 仅推进至 snapshot 版本，下一次 poll 仍构造并保存新版本。 +- 分别注入 create/stream flush/rename 失败：文件保持上一有效版本；不发生新 cache mutation 时下一次 poll 仍重试。 +- producer 静止后最后一次变化由 flush 持久化；producer 在 snapshot 后继续变化的场景必须显式表述为旧 shutdown 契约边界，不写成已保证。 +- 覆盖 scan 与 validation 两个缓存的 mutation/counter 对应关系、从旧会话 adopt 后的无变化读取，以及 dirty scan 重试；若发现未计数的持久化可见变化，先修正版本协议，不能用 advisory cache 为漏测开脱。 + +## 验证与发布顺序 + +以下是跨仓实施范围获确认后的候选流程；当前只保留本地提案，外发及正式发布另按用户授权执行： + +1. 在精确发布源树检查补丁上下文，实施最小修复、原生负对照与上述并发/失败测试；仅在本地记录独立 UP 候选及已有输入/persistence 事项关联。用户既有指令禁止 GitHub issue 评论及状态变更，不能因扩大引擎修改范围而自动向 issue #24 登记；如确需外发，另获明确授权。当前没有发出任何登记或评论。 +2. 对 payload.lock 已列出的 linux-x64、linux-arm64、darwin-arm64、win32-x64 构建并运行相关原生测试；并发测试在支持的 Linux 工具链做 TSAN。新 canary 不以 Linux-only skip 规避 Windows/macOS。 +3. 先比较旧/新引擎的 JSON 构造次数；再在相同项目 pin、producer、CPU、原阶段 lineage 下运行原 U13 600 秒、0.01 core、10% RSS 预算，以及原完整场景。保留所有失败及 allocator/进程归属证据，不关闭 persistence，不延长起始等待，不改 RSS 算法，也不挑一次 PASS 当根因证明。 +4. 为候选修复分配下一未使用 engine 版本（例如仅在尚未占用时使用 `23.1.0-mcppls.5`），先完成真实跨平台构建与原生验证，准备本地或经授权的 draft 资产及 engine.json，记录 fork/LLVM commit、features、build/binary identity。此时尚不正式发布引擎，不能重用 `.4` 名称替换资产。 +5. mcppls 先 pin 上述候选资产，更新 `packaging/payload.lock.json` 候选中的 engine 版本、四个平台 URL/file/size/SHA256；从候选分发位置独立下载并按 lock 校验，刷新 payload manifest/identity，并逐条复查本地记录的 canary/WA 去留。保持 persistent-module-memos feature，kit 是否变化由实际 ABI/内容决定，不凭推测重发。 +6. 使用候选锁构建、验证完整 payload 与编辑器/发布产物，跑 dev/release native、跨平台 CI 和完整原性能验收；这些验证均使用尚未正式发布的候选资产，不依赖先发布引擎才能验证。通过后再按已获批准的发布流程正式发布相同字节的引擎资产，核对正式下载的 SHA/identity 与候选一致，再进行 mcppls 合入及正式 release。若正式 URL 变化则只迁移 lock URL 并复核下载，不重建或替换已验证字节；资产字节变化必须重新验证。更换 engine identity 导致旧 memo 候选不被采纳是兼容性边界，需按新引擎 cold→warm lineage 验证，不能要求复用已不兼容 BMI 来制造零重建。 + +当前只验证了提案补丁的 unified-diff 格式（`git apply --stat`）。Store 新文件内容从 pinned `0030` 完整重建；两个 header 的 snapshot 上下文来自同一发布补丁。没有在真实 LLVM 工作树执行 `git apply --check`、编译或运行测试，不能把本地提案称为已可发布修复。 diff --git a/.agents/docs/2026-10-22-issues-59-63-evidence/engine-memory-proposal.patch b/.agents/docs/2026-10-22-issues-59-63-evidence/engine-memory-proposal.patch new file mode 100644 index 00000000..719bbb61 --- /dev/null +++ b/.agents/docs/2026-10-22-issues-59-63-evidence/engine-memory-proposal.patch @@ -0,0 +1,51 @@ +--- a/clang-tools-extra/clangd/ModuleValidationCache.h ++++ b/clang-tools-extra/clangd/ModuleValidationCache.h +@@ -1,3 +1,9 @@ ++ /// Samples the same publication counter as snapshot(), without copying entries. ++ uint64_t changeCount() { ++ std::lock_guard Lock(Mutex); ++ return Changes; ++ } ++ + /// For the store (ModuleMemoStore.h): every entry, by key, and how many + /// entries have been made so far. + std::vector>> +--- a/clang-tools-extra/clangd/ModuleDependencyScanCache.h ++++ b/clang-tools-extra/clangd/ModuleDependencyScanCache.h +@@ -1,3 +1,9 @@ ++ /// Samples the same publication counter as snapshot(), without copying scans. ++ uint64_t changeCount() { ++ std::lock_guard Lock(Mu); ++ return Stores.load(); ++ } ++ + /// What the store writes: every live scan whose manifest can be replayed + /// in another session, and the kept ones not yet replayed in this one. + std::vector snapshot(uint64_t &StoredSoFar) { +--- a/clang-tools-extra/clangd/ModuleMemoStore.cpp ++++ b/clang-tools-extra/clangd/ModuleMemoStore.cpp +@@ -466,7 +466,7 @@ + Wake.notify_all(); + if (Worker.joinable()) + Worker.join(); +- write(); ++ write(/*CheckForChanges=*/false); + } + + private: +@@ -508,7 +508,14 @@ + .count()); + } + +- void write() { ++ void write(bool CheckForChanges = true) { ++ // Poll publication counters before copying snapshots or building JSON. ++ // Each read uses the same lock as its snapshot. A racing mutation remains ++ // dirty for the next tick. flush() retains the original snapshot path. ++ if (CheckForChanges && ++ getModuleValidationCache().changeCount() == WrittenVerdicts && ++ getModuleDependencyScanCache().changeCount() == WrittenScans) ++ return; + uint64_t Verdicts = 0, Scans = 0; + Value Root = memos(Verdicts, Scans); + if (Verdicts == WrittenVerdicts && Scans == WrittenScans) diff --git a/.agents/docs/2026-10-22-issues-59-63-evidence/portable-handoff/README.md b/.agents/docs/2026-10-22-issues-59-63-evidence/portable-handoff/README.md new file mode 100644 index 00000000..da79a610 --- /dev/null +++ b/.agents/docs/2026-10-22-issues-59-63-evidence/portable-handoff/README.md @@ -0,0 +1,21 @@ +# PR65 的交接证据包 + +约 1 MB 的证据包只保留在 PR 的提交历史里，不进入主分支（0.0.17 发布设计 D5）。它位于提交 +`fcb3450e7960d093e93cfb0aca9ae2ad7f007d70` 的本目录：`evidence.tar.gz`（1,002,457 字节，222 个文件，SHA-256 +`af07e0600be611c59f8943c492c37743e1ab3640282d35a6b8044a4a7c95f25f`），以及逐文件记录路径、字节数和 SHA-256 的 +`manifest.json`、整包记录 `archive.json` 和校验恢复脚本 `verify.py`。 + +该提交是 PR #65（`refs/pull/65/head`）和 0.0.17 release PR 的祖先，GitHub 在合并后仍保留这两个 ref。获取与校验： + +```bash +git fetch origin pull/65/head +git worktree add /tmp/pr65-evidence fcb3450e7960d093e93cfb0aca9ae2ad7f007d70 +cd /tmp/pr65-evidence +python3 .agents/docs/2026-10-22-issues-59-63-evidence/portable-handoff/verify.py +python3 .agents/docs/2026-10-22-issues-59-63-evidence/portable-handoff/verify.py --extract --into /tmp/pr65-evidence-files +``` + +包含：两 profile 全 workspace 与 32 组 LSP 的实际终态和日志、70 项 fixture 的结果、旧产品重复 emit 反例、四项首次 +nightly 性能失败、Windows/Termux/Zed 首次失败与原始 LSP、trace runner 对照、U7a 四阶段 measure/observer/provenance、 +崩溃栈及 parser context、符号化结果、memo 控制与原发布补丁、测试前源码快照。不含 ELF、PCM/对象/index 缓存、完整 +项目工作副本、30 MB 的 memo seed 和全部 verbose server 日志。 diff --git a/.agents/docs/2026-10-22-issues-59-63-evidence/reproduce.md b/.agents/docs/2026-10-22-issues-59-63-evidence/reproduce.md index 6888b018..df5191d7 100644 --- a/.agents/docs/2026-10-22-issues-59-63-evidence/reproduce.md +++ b/.agents/docs/2026-10-22-issues-59-63-evidence/reproduce.md @@ -166,3 +166,51 @@ Windows 原生环境仍运行完整 `ux-modules-windows` fixture，以原 `asked `mcpp test process` 和 `mcpp test process --profile release` 保留原 `busy CPU > 0.5 s`、`idle CPU < 0.2 s` 断言。旧测试固定等 2 秒 wall time，不能保证获得 0.5 秒 CPU；PR/nightly 的 macOS release 首轮分别只读到 0.09/0.08 秒增量。它们仍是真失败，不能据此排除产品 sampler 的问题。 子进程 READY 后由父进程先采样，再发送 START；子进程用独立 native process CPU 时钟确认累计至少 0.75 秒并发送 DONE，保持进程存活供父进程再次采样。父进程的工作与握手共用原 30 秒期限；sampler 每次调用另有既有界限（macOS 的 ps 为 2 秒），清理也可能越过工作期限，迟到采样仍失败。记录 native work 和两组 before/after；独立工作未完成、时钟不可用或 subject 读数不足均失败。Linux oracle 直接 native clock syscall，macOS 直接 libSystem 的 Darwin CPU clock（ID 12），不使用 subject `/proc`/`ps` 读取器；当前 OpenKAL 的 `std::clock` 实际走 monotonic，不能充当 CPU oracle。该改动只涉及测试，不更改产品 sampler 或性能输入/预算。 + +## 11. 发送队列的编码与快照 + +运行两种 profile 的完整 `mcpp test process`。原 nonreader 测试仍发送 64 条、每条 64 KiB，要求全部 send 在 2 秒内返回、stop 在 10 秒内结束；现在同时记录实际耗时。Release 39732244483 的原 macOS dev 失败日志保存在 `.payload-cache/59-63-delivery/release-macos-dev-first-failure.log`，不得以另一次通过或放宽时限替代它。 + +双 framing 的真实 echo 子进程回归检查 FIFO、发送后修改或销毁原 JSON 的快照独立性、UTF-8 replacement；退出的 peer 必须拒绝继续发送并完成 stop。新 writer 在锁外使用既有 dump/encode_frame，编码异常必须转换成连接失败，不能逃出线程；没有添加生产测试 hook 或伪造损坏 JSON 来触发 OOM。该异常隔离有代码审查，不能宣称已经完成原生 OOM 故障注入。 + +## 12. 同步 apply 的状态快照与时间线证据 + +以 `mcpp test answer_routing --profile release` 运行 Workspace 的确定性回归。先在旧 `072c230` 产品上只加入相同测试：同步 apply 回调组应恰好失败一项，输出尚未 commit 就发布的新 profile；无回调组及最终通知断言均通过。再应用产品补修，dev/release 都应通过。真实 clangd 的对应回调是 apply 内 `update_quarantine_issue_()` 调用 host；该对照证明状态提交边界，不证明原 CI 最忙一分钟的具体原因。保留两次日志、退出码、测试 patch 和 workspace 源码 SHA。 + +使用匹配 profile 的新 runner 执行 `mcpp test conformance`，其中 fake LSP 故意触发状态转换超预算，要求失败 measure 仍保留可重算的 trace/window。重复 progress 不算转换；restart 不计转换且重置前态；整数纳秒保留严格 60 秒边界。重新运行原 UX 场景时，先登记用途、产品/runner SHA、fixture/pin、CPU 和缓存谱系；不改 `maxChangesPerMinute=12`、采样或时长。首次失败保留，单次后续通过不能自行证明原因或稳定性；实际结果按源版本独立汇报。 + +## 13. 补全延迟的请求支持点 + +`MCPPLS_CONFORMANCE="$runner" mcpp test test_conformance --profile "$build_profile"` +覆盖原采样结束后才到达的回复、错误回复以及超过 256 个请求的截断。 +新 runner 在原统计完成后、额外 probe/report 前冻结回答集合，再输出请求编号、 +相对发送/回答纳秒、结果数量和 incomplete；不复制候选文本或路径。前 256 条之外 +的最慢成功/已回答请求单独保留，不能因截断丢失 max 的支持点。 +旧 runner 跑相同新增测试应失败，原有测试仍通过。 + +真实 UX 使用第 2 节原命令、输入、CPU 和 cache lineage，不增加轮询或修改预算。 +每个 typing check 的 serverStats 来自已有前后 report，recent distributions 是 +最近采样而非该 check 的精确增量；slow journal 有缺失和覆盖边界，不能据晚期 +report 的较快 p95 判断先前慢请求来自客户端。保留第一次 measure、完整 server +日志及 SHA；只有四阶段进程树全部退出后才分析日志，不在测量期间扫描 PCM。 + +## 14. 被条件编译排除的 import 与保存重载 + +完成对应 profile 构建后，运行 `mcpp test test_scan --profile "$build_profile"` +和 `mcpp test test_answer_routing --profile "$build_profile"`，然后执行完整 fixture： + +```bash +"$runner" run --server "$server" --payload "$audit_dir/released/payload" \ + --fixture conformance/fixtures/mcpp-watched-source-structure \ + --workspace-dir "$audit_dir/watched-source-workspace" \ + --cache-dir "$audit_dir/watched-source-cache" \ + --timeout 60 --measure "$audit_dir/watched-source.json" --verbose \ + > "$audit_dir/watched-source.log" 2>&1 +``` + +fixture 的 README 区分旧 28 项负对照与扩展后的 70 项。旧 `1341938` 对原 28 项 +有 11 次 emit、14 个失败检查，其中包含计数级联。扩展项保留原断言，并验证 +预处理指令变化、producer 失败后重试、生成期间结构再次变化以及 target/provider +变化。Workspace 单元回归另覆盖尚未送达的 watch 与 A→B→A 字节恢复；确认需要 +开始时结构及未变化的文件 stamp，不读取完成时结构来冒充 producer 已处理输入。 +这些是独立的正确性对照，不把它们当作历史 U15 13/min 的已证实归因。 diff --git a/.agents/docs/2026-10-22-issues-59-63-target-and-reliability-design.md b/.agents/docs/2026-10-22-issues-59-63-target-and-reliability-design.md index 1b5c33ed..26dddad8 100644 --- a/.agents/docs/2026-10-22-issues-59-63-target-and-reliability-design.md +++ b/.agents/docs/2026-10-22-issues-59-63-target-and-reliability-design.md @@ -254,28 +254,19 @@ nightly 的 proot-distro login 增加 `/mnt/scripts:/mnt/scripts` 绑定或将 - 上游 mcpp 基线 `0e79128a`：`src/build/compile_commands.cppm`（根 CDB 发布语义）；`src/build/execute.cppm`（内部记录及限制）；`src/build/prepare/toolchain.cpp`（默认目标）；`src/cli.cppm`（emit 的 target/profile 参数）。这些是核查事实，不是新增稳定协议承诺。 - 当次复现临时材料位于 `/tmp/mcppls-issue-audit/`；临时目录不作为长期验收依赖，实施时转为仓库内 fixture。此前口头归因以本文 §2 的证据分级为准。 -## 13. 实施检查点（最终验收尚未完成） +## 13. 当前实施与验收状态 -- 已实现目标配置、自动提取、当前 producer 确认、动态切换及状态展示；补齐旧 CDB 回退校验和冷启动拒绝错误架构的竞态。 -- 已修 UTF-8 artifact 输出、Termux 脚本绑定及 polling watch 注册初期漏事件；有界 CDB 读取不依赖事前 stat 维持内存上限。 -- 当前 Linux 工作区检查点：dev/release 各 40 个单测执行体通过，22 组目标/身份/polling conformance 通过，规范和仓库检查通过。二进制 hash、范围及原始记录索引见 [验证简报](2026-10-22-issues-59-63-evidence/README.md#validation)。 -- 真实 VS Code E2E 通过：44 passing、1 项平台专用 pending；准确 release-profile 二进制 hash 与验证范围见简报。发布 VSIX 的安装/升级仍需单独验证。 -- U15 已以真实模块完成活动修复停顿判断。原 xlings cold/warm/edits/long 完整链路曾通过，U13 保留 600 秒、0.01 core、10% RSS 门槛；该次 CPU 为 0.007208 core，RSS 净变化为 -50.28%。另一个无文件/无索引的隔离实验定位到内置引擎未变 memo JSON 的周期性分配，属于已知未修局限；原 gate 通过不能证明该局限消失。详见 [简报的 memo 结论](2026-10-22-issues-59-63-evidence/README.md#memo)。最终重放中的失败也在简报中保留。 -- 原 mcpp 四核 cold/warm/edits 已通过；U8 在该链路撤销时新增实际构建为零。两逻辑核的完整 edits/faults 验收仍在进行，不以窄场景替代。不会通过放宽既有预算结束性能验收。 -- 尚未完成：全平台 CI、Termux 实际环境、性能最终重放、最终自审、squash 合入、release 与下载/升级验证。单 PR 将保持草稿直到验收完整。 +实现与验收分开记录。详细历史、原失败、复现命令与证据边界统一见[验证简报](2026-10-22-issues-59-63-evidence/README.md)，不将完整原始日志放入 PR；先前检查点仍可从 Git 历史核对。 -后续自审补充：target-only 设置更新仅调度 mcpp 根，不重跑其他 provider；无关 provider 的模型比较不纳入 mcpp 选择身份。真实 LSP 回归在修前观察到额外 model-unchanged，修后不再出现。轮询生命周期对照：四次移除根目录，修前遗留四条线程，修后回到相同线程基线。对应原始记录由简报的索引定位，不在 PR 内展开 JSON。 - -只读交叉自审另发现 clang-cl 探测遗漏目标参数，会把正确 i686/ARM64 命令读成宿主 x64 并被新校验拒收。已补目标参数传递及有区分力的 host/target 单测，保留 cl.exe 原行为；补修后 Linux dev/release 各 40 个执行体通过，仍需最终提交的原生 CI 验收。 - -U8 两逻辑核完整重放随后确实未通过：undo 窗口有 28 次实际构建。原 release key 全部保留，最终 release 引擎也能复用它们。进一步 PCM 元数据与进程时序证明：旧 dev 引擎退出前读到了原地更新的 release CDB，产生 dev 依赖闭包与 release 命令混合的新 key。这里同时存在实验与驱动缺陷：runner 的 ready 判断在 dev 准备 `done=0 / wanted=140` 时已允许撤销；驱动在旧进程未完全退出时发布新 CDB。 - -补修任务：runner 在改变配置前、撤销前及撤销后检查明确的有限工作状态并有界等待，不用 CPU/RSS 或固定延时选择起点，保留等待活动与原零重建门槛；驱动对需要重启的计划保留最新待发布快照，旧进程 reaped 后才发布 CDB 和生成 prime 文件，再启动新进程。普通无需重启的增量更新仍直接发布。写入失败不得记成功或启动读取旧配置的新引擎，应保留待发布计划供明确重试。新增 native regression 后先以旧实现确认失败，再统一编译、测试并重跑原两逻辑核链路；目前这组补修尚未验收。 - -原两核完整链暴露另一个调度缺陷：普通模型 apply 重置 implementation sweep 完成状态后，没有待处理队列或新的 prime completion 时不会再调用 pump。900 s drain 最终同时看到 sweep 未完成、未调度，不能视为空闲。已在成功增量 apply 后调用既有 pump，遵循原工作/安静期判断；native 同计划重应用对照在旧实现有 4 个断言失败，补修 dev/release 各 40 个执行体及 engine 65 cases / 650 assertions 全通过，仍待完整新字节链及最终 CI。历史原链结果与独立 runner SIGSEGV 均保留在验证简报，不能由这一项修复替代验收。 - -最终自审补充恢复边界：首份或普通增量 CDB 单次写失败后，用户修复磁盘直接重启，不应依赖另一次 apply 才恢复接受流量和模块/implementation 元数据。start_process 在无接收流量状态下，发布保留的完整快照后重新应用，再启动；保留首次写失败提前返回，不让旧进程采用尚未发布的状态。新增只失败一次直接恢复的原生对照覆盖首份与增量计划，旧实现 5 个断言失败，补修 dev/release 各 40 执行体通过；其余最终验收继续。 - -原生 Windows CI 暴露 CDB 中 8.3 短名与工作区长名被当成不同项目的问题。目标提取、有效目标校验和模型 profile 现统一使用平台规范路径判断归属，保留原编译参数和嵌套项目边界。另修未信任 mcpp 根对扫描回退模型错误执行 producer 选择代数校验引起的无限重载；可信根的过期结果拒收保持有效。新 Workspace 对照在旧 guard 有 6 个断言失败；补修 dev/release 各 40 执行体通过（engine 67/693，target 9/92），24 组真实 LSP 通过。新字节原生 CI、完整性能与发布升级仍待验收，结果统一记入简报。 - -workspace 声明成员自身的别名身份也必须保留：原路径/一层 glob 匹配失败时，以声明成员目录的规范身份再核对，按一次归属快照缓存，保持嵌套与外部项目排除。对应旧实现行为对照为 8 个断言失败；正式结果与最终字节统一由简报更新。 +| 项目 | 当前事实 | 尚需完成 | +|---|---|---| +| 跟随 target、项目配置、producer 身份与缓存 | 已实现；包含 Windows 路径别名、workspace 成员归属、未信任根回退、错误架构与迟到结果拒收 | 最终提交的全部平台 CI；真实 mcpp Windows i686→x64→i686 生产链 | +| UTF-8、连接写入、状态一致性与恢复 | 已实现 UTF-8 artifact、队列写入、同步 apply 后发布状态、只读接口 rename 保护、implementation sweep 与磁盘恢复 | 新产品完整 workspace、LSP、编辑器与发布验证 | +| 源文件重复 discovery | 独立旧产品反例已证明；新实现确认有效 run-start 结构、文件 stamp、代数与 producer 身份，预处理文本仅按需采集，未知 import/header-unit 语法保守重查 | 完整 70 项 LSP fixture 与最终 CI；不能将其冒充历史 U15 的已证实归因 | +| Zed 与 Termux | 已修正 Termux 官方包源/内层绑定；新增 Zed file-as-folder 过滤及自身引擎初始化/进程检查；预装 PATH clangd 保留 Zed 默认独立启动 | 实际新 CI 的安装、原生 Zed 及产品检查，不用静态检查代替 | +| 新增原生回归 | dev/release 均通过 scan 23 cases/146 assertions、Workspace 29/399、Session 3/12；Session 首次测试编译错误已修正并保留日志 | 本地两 profile 全 workspace 各 58 程序及 LSP 各 32 组已通过，实际结果见交接；新 commit 仍需远端验证 | +| 平台与性能 | `1341938` 曾有原本地九阶段 53 项与 PR CI 通过，但其 nightly 有四项真实失败；`e01e1c1` CI/nightly 仍失败。一次新 trace 诊断为 25 项/1 失败，U7a 受保存前 AST 崩溃后的隔离影响 | 原预算不变的新产品完整性能链与所有 CI；历史慢请求、RSS 增长和底层 AST 崩溃未被完整归因 | +| 引擎范围 | [memo 提案](2026-10-22-issues-59-63-evidence/engine-memory-proposal.md) 仅供本地 review；周期无变化仍构造完整 JSON 的缺陷已证明，未修改或发布引擎 | 当前仅同步 mcppls 工作和交接，不实施引擎修复；不发布 issue 评论或变更状态，不承诺修复全部 RSS 增长 | +| 合入与交付 | PR64 已合入，发布预检后补修集中在 draft PR65；0.0.17 尚未发布，两个原 0.0.16 升级环境保持原状 | 最终 review、精确提交 squash、新 main CI/draft release、12 个实际下载资产、本地运行及两个原 profile 升级、正式发布后下载/OpenVSX 校验，最后提供上传目录 | + +依赖顺序仍为：源码冻结 → 两 profile 全量验证 → 最终源版本 CI/nightly 与完整性能验收 → 最终 review/squash → 新 main 与候选 release 验证 → 实际升级 → 正式发布及下载回验。不能以一次通过覆盖原失败，不能通过预算、等待起点、缓存谱系或冷却缩短制造通过。 diff --git a/.agents/docs/2026-10-23-0.0.17-part2-design.md b/.agents/docs/2026-10-23-0.0.17-part2-design.md new file mode 100644 index 00000000..25395293 --- /dev/null +++ b/.agents/docs/2026-10-23-0.0.17-part2-design.md @@ -0,0 +1,352 @@ +# 0.0.17 part2：PR #65 之后的收口设计 + +日期：2026-10-23。状态：**第二版，已纳入 D1–D5 决定和自我 review，待 review**。基线为 PR #65 head `fcb3450`（draft，分支 `fix/17-connection-writer-encoding`），main `fd869bd`（#64 已 squash 合入，版本号已是 0.0.17，未打 tag）。引擎仍是 `23.1.0-mcppls.4`（mcppls-clangd v0.0.16），引擎仓库还没有 memo 修复的 PR。 + +本文不重复 [#59/#63 设计](2026-10-22-issues-59-63-target-and-reliability-design.md)和[证据简报](2026-10-22-issues-59-63-evidence/README.md)。本文做四件事：记录已定的决定，核对 `fcb3450` 的实际状态，用现有 nightly artifact 重新比较 0.0.16 与候选版本，再给出每个工作项的方案与验收方式。§10 记录第二版相对初稿改了什么、为什么改。 + +## 0. 结论 + +1. **发布目标是 0.0.17 全绿**（D4）。这意味着在最终 head 上，PR CI 和 nightly 的**每一个 job** 都要通过，包括 continue-on-error 的 termux 和 two-core。对照这个目标，初稿漏了两项必须处理的工作：U13 RSS 和 two-core（§3、§5）。 +2. `fcb3450` 的 CI 失败**没有一项来自 mcppls 服务端**：6 项是仓库规范检查，1 项是 Windows fixture 用了旧版 mcpp，另 1 项是 runner 证据工具在 Windows 上的断言失败。按 D5 处理证据包后，规范检查的失败会随之消失（§4.1）。 +3. 用现有 artifact 对比 0.0.16（6 晚 18 轮）和候选版本（4 个 head 12 轮）后： + - **U15**：0.0.16 就在预算边缘（busiest minute 10–12，预算 12），候选版本是 10–13。这不是回归，而是一直没有余量。W4 必须做出明显余量。 + - **U13 RSS**：失败的两次和通过的两次分成两组，差别在于测量期间分配器有没有把内存还回去（§3.2）。这个增长在 0.0.16 上多半也存在，只是被旧的测量起点掩盖了。要稳定通过，引擎 memo 修复（W7）实际上在关键路径上。 + - **U5/U6**：8.26 s 的离群值在 12 轮里只出现 1 次，6+6 次 A/B 分辨不出这么低的发生率。D2 改为看"分布是否整体变差"加"离群值的机制"（§4.6）。 +4. D3（引擎 `.5` 不绑定）和 D4（全绿）之间有一个需要你确认的交界：如果到最终门槛只剩 U13 因为引擎没就绪而红，按哪条执行？我的建议见 §6 E1。 + +## 1. 已定的决定 + +| # | 问题 | 决定 | 对方案的影响 | +|---|---|---|---| +| D1 | W4（autosave reload）和 W5（隔离期）是否进入 0.0.17 | **进入** | 两项都是 P1，都要有确定性 fixture，并且旧产品必须在 fixture 上失败 | +| D2 | U5/U6 慢补全是否阻断 0.0.17 | **按 A/B 结果定** | A/B 的方法改为同一个 runner 只换 server，判定规则见 §4.6 | +| D3 | 引擎 `.5`（memo）是否绑定 0.0.17 | **不绑定** | 但在 D4 下它是 U13 稳定通过的前提，所以现在就开 PR，不再"有空再做"（§4.7） | +| D4 | nightly 放行标准 | **最好 0.0.17 全绿** | "全绿"的定义见 §5；0.0.16 就有的问题不能再"只记录、不阻断" | +| D5 | 约 1 MB 的证据包是否进 main | **只留在 PR 的 commit 历史里，不进 main** | 做法见 §4.12；`verify.py` 随之离开主分支的树，W1 的 allowlist 问题不需要处理 | + +## 2. `fcb3450` 现状核对 + +PR CI [39766141074](https://github.com/Sunrisepeak/mcpp-language-server/actions/runs/39766141074)：37 success / 7 failure / 2 skipped（real Zed、release checks 按条件跳过）。 + +| 失败 | 实际原因（已读日志核对） | 处理 | +|---|---|---| +| build and unit tests ×6 | `test_check`：`portable-handoff/verify.py` 已提交，但不在 `tools/devtools/scripts.allow` 里 | 按 D5 处理后，`verify.py` 不在 head 的树里，检查自然通过（§4.1） | +| windows-2022 dev 另有 `test_conformance` | 合并树中的 466/467/470 行：overflow trace 的 `slowestSuccessful`、`slowestAnswered` 与 `completion.max` 不相等；最慢那一项的 index < 256 | W3（§4.3） | +| conformance (win32-x64, 1 of 2) | `mcpp-target-windows-real` 用了 PATH 上的 mcpp `2026.9.30.2`。`-Xmicrosoft-windows-sdk-root "…\10\"` 末尾的 `\"` 被当成转义引号，之后的参数整体错位。上游 2026.10.5.2 已修复 | W2（§4.2） | + +`fcb3450` 还**没有跑过 nightly**。它已经包含三项 CI 环境修复，需要在 nightly 上实际跑过才能确认： + +- **termux**：`e01e1c1` 失败的原因是随机镜像 `linux.domainesia.com` 对 `proot-distro_5.6.0` 返回 404。`fcb3450` 改为固定使用官方源 `packages.termux.dev`。 +- **Zed 默认模式**：`e01e1c1` 失败的原因是 Zed 下载 clangd 时被 GitHub API 限流（403）。`fcb3450` 改为预装 PATH 上的 clangd，这样仍走 Zed 默认的查找逻辑，但不再依赖下载。 +- **失败阶段的完整日志**：nightly 会在后续阶段覆盖日志之前，把失败阶段的 server 日志保存下来。two-core 和 U5 离群值的归因都需要这些日志（§4.6、§4.10）。 + +## 3. 0.0.16 与候选版本：用现有 artifact 重新比较 + +数据来源：0.0.16（`d3c3a94`）定时 nightly 10-15、16、17、18、19、20、21 的 ux artifact，以及候选版本 `38366b9`、`072c230`、`1341938`、`e01e1c1` 的 workflow_dispatch nightly。注意：候选版本的 runner 改过（U13 测量窗口、日志保留），所以两组数字只有在同一 runner 下才能严格比较（§4.6 的 A/B）。 + +### 3.1 各 job 的结果 + +| job | 0.0.16（8 晚） | 候选版本（4 次 nightly） | 归属 | +|---|---|---|---| +| ux-xlings 1（U13） | 8/8 红，原因是 CPU 0.02–0.12 core | 2/4 红，原因是 RSS +73.9%、+25.3%；CPU 都通过（0.006–0.008，预算 0.01） | §3.2，W7 | +| ux-mcpp 1/2/3（U15） | 0 次超预算，但 busiest minute 为 10–12，预算 12 | 2/4 次出现 13（`072c230` r3、`1341938` r1） | W4 | +| ux-mcpp 2（U5/U6） | max 0.52–0.84 s，18 轮都通过 | 1 次 8.26 s / 6.38 s（`1341938` r2，同一会话内 U5、U6 都出现）；其余 11 轮 max 0.57–1.01 s | W6 | +| ux two-core | 8/8 红 | 3/4 红（`38366b9` 全绿）：U5 有 1–2 次 server 侧 request-timeout，预算 0；U7a p95 1.53/1.71 s，预算 1.5 s | W10 | +| ux-mcpp 1（U9） | 1/8 红：kill clangd 之后，首次回答用了 121.35 s | 未复现 | W5 | +| termux | 8/8 红 | `e01e1c1` 因镜像 404 失败 | `fcb3450` 已修，待 nightly | +| Zed 默认模式 | — | `e01e1c1` 因 GitHub API 403 失败 | `fcb3450` 已修，待 nightly | +| self-hosting、startup timing | 8/8、4/8 红 | 通过 | 主要靠 CI/runner 修复，不能算作产品改善的证据 | + +### 3.2 U13 RSS：失败与通过分成两组 + +| run | 结果 | 有限工作时长 | idle 开始时 RSS | idle 结束时 RSS | 增长 | idle CPU | +|---|---|---|---|---|---|---| +| `38366b9` | 通过 | 70.1 s | 1391.8 MB | 660.3 MB | −52.6% | 0.0059 | +| `e01e1c1` | 通过 | 80.1 s | 1494.3 MB | 675.4 MB | −54.8% | 0.0080 | +| `072c230` | 失败 | 122.4 s | 624.7 MB | 1086.6 MB | +73.9% | 0.0060 | +| `1341938` | 失败 | 126.4 s | 948.1 MB | 1188.3 MB | +25.3% | 0.0061 | + +解读： + +- 在通过的两次里，测量开始时 RSS 很高，期间 clangd 把内存还给了系统（trim），所以最后约为 670 MB。 +- 在失败的两次里，这次 trim 发生在有限工作期间，测量开始时 RSS 已经很低。之后 10 分钟空闲期间 RSS 又涨回 1.1–1.2 GB，期间再也没有 trim。 +- 两组的 idle CPU 都在 0.006 左右。600 s × 0.006 core ≈ 3.6 CPU 秒，与 memo 线程每 30 s 构造一次约 30 MB JSON、共 20 次的量级相符。 + +所以初稿的假设（空闲期间没有 LSP 消息，就不会 trim；memo 线程反复分配又释放的大块内存被分配器保留）得到了**间接支持**，但还没有被证明。两点推论： + +1. **"0.0.16 没有 RSS 问题"不成立**。0.0.16 的测量窗口开始得更早，起点通常处在高位，所以增长被掩盖了。引擎同是 `.4`，增长多半一直存在。 +2. 在 mcppls 侧，只有"挑测量起点"能让 U13 变绿，而这正是 #59 review 明确禁止的做法。要稳定变绿，只能消掉空闲期的分配：引擎 memo 修复（W7）。 + +## 4. 工作项与方案 + +分级：**P0** 不做就不能合并；**P1** 本版修复；**P2** 并行推进，不阻塞。D4 之后不再有"记录为已知问题"的 P3。 + +| # | 项目 | 级别 | 仓库 | +|---|---|---|---| +| W1 | 本地验收流程（干净树） | P0 | mcppls | +| W2 | 真实 Windows fixture 使用正确的 mcpp | P0 | mcppls（CI） | +| W3 | runner overflow trace 的 Windows 断言 | P0 | mcppls（runner） | +| W4 | autosave 下正在输入的文件每次保存都 reload（U15） | P1 | mcppls | +| W5 | crash/exit 之后的隔离期（U7a、U9） | P1 | mcppls | +| W6 | U5/U6 慢补全：取证、A/B、D2 判定 | P1 | mcppls | +| W7 | 引擎 memo 周期性构造 JSON（U13 RSS） | **P1（D4 的关键路径，按 D3 不绑定）** | mcppls-clangd → mcppls | +| W8 | 引擎崩溃可符号化 | P2 | mcppls-clangd | +| W9 | 发布流程 | P0（合并后） | mcppls | +| W10 | two-core：server 超时与 U7a p95 | **P1（从 P3 升级）** | mcppls | +| W11 | termux、Zed 的 CI 环境修复在 nightly 上确认 | P0（只验证） | mcppls（CI） | +| W12 | 证据包只留在 PR 历史（D5） | P0 | mcppls | + +### 4.1 W1：本地验收必须基于干净树（P0） + +HANDOFF 里"本地 6 members / 58 programs / 0 failed"是在 dirty snapshot 上跑出来的，`verify.py` 是之后才提交的。所以本地全绿和远端 `test_check` 失败并不矛盾。规则：本地报告必须写明 commit SHA，并且 `git status --porcelain` 为空；dirty snapshot 上的结果只能作参考。 + +初稿里"把 `verify.py` 加进 `scripts.allow`"这一项取消：按 D5，它会离开 head 的树（§4.12）。 + +### 4.2 W2：真实 Windows fixture 使用正确的 mcpp（P0） + +问题：CI 里有两个 mcpp（见 `.github/versions.env`）。一个是固定的 `MCPP_VERSION=2026.9.30.2`，它是默认版本、在 PATH 上；另一个是从 release 安装的 `MCPP_BUILD_PROGRAMS_VERSION=2026.10.11.1`，只有 `mcpp-build-programs` 会触发安装。`mcpp-target-windows-real` 写的是 `{env:MCPPLS_REAL_MCPP|mcpp}`，CI 没有设置这个变量，所以用了旧版。 + +方案（不改 validator，不改 fixture 断言）： + +1. fixture 在 `scenario.json` 中自己声明 `"requires-mcpp": ">=2026.10.5.2"`。runner 在 prepare 之前读取实际 producer 的版本；不满足时直接失败，报"fixture 需要 mcpp ≥ X，实际是 Y"，而不是让 clang 报一串看不懂的路径错误。 +2. CI 的"The mcpp that describes build programs"步骤改为：只要本矩阵项里有 fixture 声明了 `requires-mcpp`，或者包含 `mcpp-build-programs`，就安装。两个环境变量合并成一个 `MCPPLS_REAL_MCPP`，`mcpp-build-programs` 也改用它。schema 预检查选择 `emit=` 时，读的也是同一份声明。 +3. 默认 `MCPP_VERSION` 这次不升级。升级会改变全部 fixture 的基线，放到 0.0.18 单独评估。 +4. 在 `docs/20-projects.md` 中补一句："Windows MSVC 下使用 `import std` 需要 mcpp ≥ 2026.10.5.2。" + +验收：win32-x64 conformance 上，`mcpp-target-windows-real` 实际跑通 i686→x64→i686。这是 #63 设计要求的原生生产链，**到目前为止从未通过过**。 + +### 4.3 W3：runner overflow trace 的 Windows 断言（P0） + +**更正初稿的诊断**。读代码（`src/bin/conformance.cpp` 约 2773–2832 行）可以确认：统计和 trace 遍历的是同一组 `completions` id、同一个 `asyncRequests`，中间也不会 pump 客户端；而且 `typed["answered"] == total`（471 行）是通过的。所以"两边集合不一致"这个解释**不成立**。 + +对 470 行，目前有一个候选解释：mock 的计数器 `completions == 270`（`tests/test_conformance.cpp:120`）统计的是 mock 收到的**所有** completion，包括打字开始之前，runner 为等首个回答而发出的那些请求。Windows 上等首个回答需要的轮次更多，于是被故意放慢的那个请求在客户端的 index 就落到了 256 以下。466/467 两行（最慢的不等于 max）目前还无法解释。 + +方案： + +1. 先让测试在失败时打印诊断信息：trace 的 `slowest*`（index、elapsedNanoseconds）、`completion.max`、`total`、打字开始之前 mock 已经收到的 completion 数，以及 elapsed 最大的前 3 项。在 Windows CI 上跑一次，拿到数据。 +2. 如果候选解释成立：mock 改为从打字开始之后才计数（例如由 runner 在打字开始时发一个标记，或者改用 request 内容里的位置），不改 runner 的产品逻辑。 +3. 466/467 等数据出来再定。在原因查清之前，不放宽断言，也不按平台跳过。 + +### 4.4 W4：正在输入的文件，autosave 每次保存都 reload（P1，U15） + +**证据**：最忙一分钟的 13 次转换全部是 `ready→loading→ready`，每次 loading 只有 0.65–1.0 s，间隔约 2 s。它们全都发生在 U6 autosave 期间，脚本反复在 `src/build/execute.cppm` 第 17 行输入 `import mcpp.build.`。这和 `fcb3450` 修掉的"条件 import 导致重复 discovery"不是同一个问题：这里每次保存都是真实的结构变化。0.0.16 上同一指标是 10–12，所以这个问题一直存在，只是刚好落在预算以内。 + +**机制**（`src/orchestrator/workspace.cpp`）：watched-file 的结构变化会调用 `schedule_reload(fromEdits=true)`，等待时间是 `clamp(lastProducerMs, 1.5s, 60s)`，而且后续的变化会把它往后推。producer 很快时，窗口就是 1.5 s；autosave 大约每 2 s 保存一次，每次都超过窗口，于是各触发一次 reload。 + +**方案**： + +1. **复用已有的状态**，不新建原语：workspace 里已经有 `editedAt`（每个打开文件最后一次 didChange 的时间，import-hang plan §5）和 `EDIT_SETTLE = 2000ms`（F13，replan 用）。新增一个辅助函数 `edit_quiet_at(path) = editedAt[path] + EDIT_SETTLE`。 +2. 保存触发结构 reload 时，如果文件是打开的，并且还没到 `edit_quiet_at`，就令 `reloadAt = max(现有等待时间, edit_quiet_at(path))`；之后每次输入和保存都继续往后推。没有打开的文件（checkout、外部工具写入）以及停止输入之后的保存，行为不变。 +3. 上限：从第一次被推迟算起，最多推迟 `5 × EDIT_SETTLE = 10 s`，避免一直不停输入时模型永远不更新。 +4. 本版**不**合并 `BUILD_PROGRAM_QUIET`（它针对磁盘上的变化）和 `DOOM_RETRY_QUIET`（3 s，在引擎层，靠 `touchedAt_` 判断）。合并它们会改变两处现有行为，放到 0.0.18（§7 原则 1）。 +5. 不改 status 的语义，不加 hold，不改 U15 的 12/min 预算。 + +**验收**： + +- 新增确定性 fixture `mcpp-autosave-import-typing`：mock producer，10 Hz 在 import 行输入，每 1 s autosave 一次，持续 20 s。旧产品的 producer 调用和状态转换应 ≥ 8 次，新产品 ≤ 2 次；停止输入之后，模型在 EDIT_SETTLE + 1.5 s 内反映最终的 import。 +- 现有 70 项 watched-source fixture、U8 和 U15 场景保持不变。 +- nightly 上 U15 的 busiest minute **≤ 8**（预算仍是 12）。没有余量的"通过"不算通过。 + +### 4.5 W5：crash/exit 之后的隔离期（P1，U7a、U9） + +**证据**： + +- U9（0.0.16 nightly 10-17）：测试主动 kill clangd，没有 crash context。probe 的首个回答等了 121.35 s，正好对应 `Quarantine::FIRST_TERM = 2 min`。 +- U7a（`u7a-postrun-analysis.md`）：clangd 在构建 `config.cppm` 的 AST 时崩溃，**有 crash context**（Build AST，栈里有 realloc），文件被隔离 120.216 s。在这期间，它依赖的 `mcpp.log`、`mcpp.config` 已经重建成功；交回之后，AST 只用了 0.73 s。 +- 代码：`settle_exit_` 对所有 suspect 一律执行 `set_aside_(…, Reclaim::no)`。`sources_changed()` 只释放 `moduleFailed` 类型的隔离，其他隔离只有在文件自身变化或期满时才交回。 + +**方案**（两条新规则；现有规则不变）： + +1. **原因不明的退出，第一次不隔离**（对应 U9）：没有 crash context 指认的 suspect 文件，只记一次 strike。同一个文件在 10 min 内第二次成为 suspect，才隔离。被 crash context 指认的文件，照旧立即隔离。U7a 属于这一类，所以这条规则**帮不了 U7a**。 +2. **依赖闭包变化之后，交回重试一次**（对应 U7a）： + - 隔离时记录该文件 import 闭包的身份。闭包从 mcppls 自己的计划里取；身份用的是 mcppls 观察到的每个模块 `module-built` 事件的序号。引擎只报告"Built module X to …"，不报告 BMI 的内容哈希，所以不需要改引擎。 + - 之后，闭包里的模块全部构建成功、并且身份与记录的不同时，交回 clangd 重试一次。每个不同的身份只重试一次。重试之后再崩溃，按现有规则把期限翻倍。闭包里有模块构建失败时，不交回。 +3. 现有规则不变：文件自身变化时交回、期满交回、`MAX_TERM = 16 min`。最坏情况由引擎现有的崩溃计数兜底：5 min 内退出 5 次，就改为退避重启（1、2、4、8 分钟），而不是放弃。初稿把这里写成了"放弃"，已更正。 + +**安全边界**：最坏情况是依赖每真实变化一次，clangd 就多重启一次，并且受崩溃计数约束。不缩短 `FIRST_TERM`，不做无条件交回，不改 U7a/U9 的预算。U7a 背后的崩溃本身（AST 构建时 realloc）是引擎的 bug，靠 W8 符号化之后才能定位，不在本版修复范围内。 + +**验收**（确定性，使用 mock 或 fake-crash 引擎）： + +- kill 一次 → probe 在 25 s 预算内由 clangd 回答；同一文件在 10 min 内两次成为 suspect → 隔离。 +- 指认了文件的 crash → 隔离；无关文件保存 → 不交回；依赖重建但身份相同 → 不交回；身份变化 → 交回一次；再崩溃 → 期限翻倍，同一身份不会再交回；依赖构建失败 → 不交回。 +- 旧产品必须在"kill 一次"和"闭包身份变化"这两个用例上失败。 + +### 4.6 W6：U5/U6 慢补全（P1，D2 判定） + +**已有数据**（§3.1）： + +- 0.0.16 的 18 轮：U5 max 0.52–0.84 s（中位约 0.76），U6 max 0.47–0.76 s。 +- 候选版本的 11 轮（不含离群那一轮）：U5 max 0.57–1.01 s（中位约 0.89），U6 max 0.53–0.96 s。 +- 离群值是 1/12：`1341938` r2 中，U5 为 8.26 s、U6 为 6.38 s，同一会话里两项都慢，像是会话级的事件，不像单个请求的问题。 +- 典型 max 大约上移了 0.13 s，离 3 s 预算还很远，但值得确认它是不是来自产品本身。 + +**A/B 方法**（改进初稿）： + +- 同一个 runner、同一个 fixture、同一种 runner 机器，只换 `--server/--payload`：一边是 v0.0.16 release 的 `payload-linux-x64.tar.gz`，一边是候选版本的 payload。这样可以排除 runner 改动的影响。 +- 实现方式：nightly 的 workflow_dispatch 增加输入 `payload-from`。ux-mcpp 的 edits 阶段各跑 6 次。如果新 runner 读取了 0.0.16 不提供的报告字段，runner 必须把它们当成 null 处理（`serverCompletionEvidence` 已经是这样），并在结果里记下缺了哪些字段。 +- 顺带跑 U13：ux-xlings 的 long 阶段各 3 次，用来验证 §3.2 的推论（0.0.16 在新测量窗口下同样会出现 RSS 增长）。 + +**D2 判定规则**： + +1. 典型分布（每轮的 p95 和 max）在 A/B 中明显变差（候选版本的中位数比 0.0.16 高出两边合并标准差的 2 倍以上）→ 阻断，先找原因。 +2. 离群值：6+6 次分辨不出约 1/12 的发生率，所以不靠计数下结论。在最终 head 的 nightly（§5）中再次出现 → 阻断，用 `fcb3450` 保留的失败阶段日志归因。没有再出现 → 记入 #59，但取证改动照样随 0.0.17 发布。 + +**取证**（这是 server 改动，但只增加观察，不改变行为）：server 已经记录每个请求的 `engineSentAt/engineRepliedAt`、按方法统计的 `engineP95Ms/overheadP95Ms`，以及 ≥ 5 s 的 `slow-request` journal。在此基础上扩展，而不是另起一个环形缓冲： + +- `slow-request` 增加 `queueMs`（收到 → 发给引擎）、`engineMs`、`answerLayerMs`，以及发送时这个文件的 clangd fileStatus（是否正在构建 preamble 或模块）。 +- 对 completion，把阈值从 5 s 降到 1 s，最多保留 32 条。 + +预设假设 H1：import 行的编辑使 preamble 失效，之后的函数体补全要等 preamble 和模块前置重建。取证结果应该能直接证实或否定它。修复方向（取证之后再定，现在不实施）：answer layer 在 preamble 重建期间使用上一代的答案（作为 standin 标注出来），或者在引擎侧让补全使用旧 preamble。 + +### 4.7 W7：引擎 memo 周期性构造 JSON（P1，D4 关键路径，按 D3 不绑定） + +已证明的部分：`.4` 的 `ModuleMemoStore::write()` 每 30 s 先构造完整的 JSON，然后才判断有没有变化。隔离实验（只载入 30 MB memo，零次写入）中，RSS 在 180 s 内从 177 MiB 涨到 267 MiB。采纳[引擎提案](2026-10-22-issues-59-63-evidence/engine-memory-proposal.md)：先比较两个缓存的廉价 changeCount；最后一次 flush 仍然强制写 snapshot。 + +§3.2 的数据让它从 P2 升为 D4 的关键路径。验证"分配器保留"这一推论： + +- runner 在 idle 窗口两端各发一次 clangd 的 `$/memoryUsage`。 +- MemoryTree 基本不变而 RSS 上涨 → 是分配器保留，修复点在 memo 写入；MemoryTree 本身在涨 → 是真实的数据增长，要另找原因。 +- 如果打了补丁之后仍然出现"未 trim 的增长"，再考虑在引擎侧做第二步：memo flush 之后，如果确实分配过，就调用一次 `malloc_trim`。本版不预先做这一步。 + +**流程**（沿用既定做法：引擎先在产品 CI 中验证，再合并和发布）： + +1. **现在就**在 mcppls-clangd 开 PR：补丁 + 原生 canary（旧实现必须失败）。 +2. mcppls 通过 `packaging/engine-candidate.json` 用这个 PR 的 CI 构建，跑完整 CI 和 U13 各 3 轮。 +3. 两边都绿之后，发布 mcppls-clangd v0.0.17（`23.1.0-mcppls.5`），切换 `payload.lock.json`，删除候选文件。 + +按 D3，0.0.17 不等它。但要做到 D4，它必须赶上，所以它从第一天就并行开工。 + +### 4.8 W8：引擎崩溃可符号化（P2） + +U7a 四个崩溃地址的符号化结果全是 `??`，因为 Linux/macOS 的 release ELF 是 stripped 的，只有 Windows 有 `win-symbols.yml`。方案： + +- 引擎 release 为 linux-x64/arm64 发布 `objcopy --only-keep-debug` 生成的独立调试文件（按 build-id 命名），macOS 发布 dSYM。 +- mcppls 的 incident 记录补上 build-id 和模块内偏移，以后可以离线符号化。 +- 用户下载的包大小不变。可以和 W7 放在同一个引擎版本里发布，但不阻塞。 + +### 4.9 W9：发布流程（P0，合并后） + +沿用 #59/#63 设计 §13 的顺序： + +1. 最终源码 CI + nightly（§5）。 +2. squash 合并。 +3. main CI 与 draft release。 +4. 逐一下载 12 个实际资产并校验。 +5. 用两个原 0.0.16 profile 实际升级。 +6. 正式发布，并再次下载回验。 + +另加一步：合并后核对 main 不含证据包（§4.12 第 4 条）。 + +### 4.10 W10：two-core（P1，从 P3 升级） + +D4 的"全绿"包括 continue-on-error 的 job。two-core 在 `38366b9` 上通过过，说明可以做到。失败的情况分两类： + +- U5 有 1–2 次 server 侧 `request-timeout`，预算 0。U5 的 p95 0.71–0.76 s、max 0.83–0.98 s 都很低，所以超时的**不是补全本身**，而是同一时间段里的其他请求（`clangd.cpp` 中按 deferred/held 或 pending 超时统计，事件里带有 method 和 file）。现有 artifact 只有总数，没有 method。 +- U7a 的 p95 为 1.53/1.71 s，预算 1.5 s，超出很少。 + +方案： + +1. 先在 `fcb3450` 之后的 nightly 中，从保存下来的失败阶段日志里查出超时的 method 和原因：`waited too long for …` 还是 `did not answer … in time (while …)`。 +2. 如果是 mcppls 的等待上限在两核机器上太紧（例如等 "clangd to accept requests"），在 mcppls 中修复；如果是 clangd 在两核上确实答不过来，记录证据，并回到 §6 E2。 +3. 不改 two-core 的预算，也不把它从矩阵里拿掉。 + +### 4.11 W11：termux、Zed 的 CI 环境修复（P0，只验证） + +`fcb3450` 已经包含修复（§2）。在最终 head 的 nightly 上，termux 和 Zed 默认模式都要实际通过。如果再遇到镜像或 API 问题，修环境，然后重跑整个 nightly，不对单个 job 豁免。 + +### 4.12 W12：证据包只留在 PR 历史（P0，D5） + +`fcb3450` 提交了 `portable-handoff/`（`evidence.tar.gz` 1,002,457 B、`manifest.json` 1121 行、`archive.json`、`verify.py`）。main 的 `fd869bd` 不含这些文件，#64 是 15 个提交 squash 成 1 个。做法： + +1. **不改写 `fcb3450` 之前的历史**（不 rebase、不 force-push 越过它）。GitHub 的 `refs/pull/65/head` 在合并之后依然保留，`fcb3450` 只要是它的祖先就能一直获取。 +2. 合并前的最后一个提交执行 `git rm`，删除 `evidence.tar.gz`、`manifest.json`、`archive.json`、`verify.py`；如果证据目录下的 `.gitattributes` 只为 tar.gz 服务，也一起删掉。`portable-handoff/README.md` 改为一份指引，写明： + - 提交 `fcb3450` 和路径； + - 整包 SHA-256（来自 `archive.json`）； + - 获取命令：`git fetch origin pull/65/head && git worktree add /tmp/ev fcb3450`，然后在那里运行 `verify.py`。 +3. 小的文本证据（README、reproduce、引擎提案和补丁，约 50 KB）留在 main。 +4. squash 合并之后核对：`git ls-tree -r origin/main` 中没有 `evidence.tar.gz`；`git rev-list --objects origin/main` 中没有它的 blob。 + +## 5. "全绿"的定义（D4） + +在最终 head（产品字节不再改动的那个提交）上： + +1. **PR CI**：每个 job 都是 success。按条件跳过的 real Zed / release checks，由 W9 中的 release checks 补跑。 +2. **nightly**：**两次**独立的 workflow_dispatch nightly（不在同一个小时触发），每个 job 都是 success，包括 termux 和 two-core。要两次，是因为 U13 在候选版本上的失败率是 2/4，U15 也没有余量，一次通过的证据太弱。每次 nightly 约 1.5 h。 +3. **不改预算，不挑测量起点，不跑到绿为止**。出现红的 job 先归类：环境问题（镜像、限流、runner 故障）→ 修环境，然后两次都重跑；产品问题 → 修复，最终 head 改变，计数从零开始。 +4. 余量要求：U15 busiest minute ≤ 8；U13 的 RSS 增长和 CPU 都要低于预算的 50%（当前 CPU 0.008 离 0.01 太近）。只为 U13/U15 设余量，是因为只有这两项已经证明会在边缘来回摆动；不新增其他门槛。 + +## 6. 还需要你决定的事 + +| # | 问题 | 建议 | +|---|---|---| +| E1 | 到最终门槛时，如果只剩 U13 因为引擎 `.5` 没就绪而红，按 D3 发布，还是按 D4 等？ | **等，但设时间盒**：W7 从第一天开工，补丁和 canary 已有草稿，预计不是瓶颈。其余工作项都绿之后，如果引擎还需要超过 2 天，就按 D3 发布，并在 release notes 和 #59 中写明 U13 RSS 是唯一的例外及其证据 | +| E2 | 如果 W10 查明 two-core 的失败是 clangd 在两核上确实答不过来（mcppls 修不了），two-core 还算不算"全绿"的一部分？ | **算**。到那时把证据拿给你，再决定是否例外，而不是现在就预先放行 | +| E3 | 最终 head 要几次全绿的 nightly？ | **2 次**（理由见 §5 第 2 条）。如果觉得 runner 成本太高，可以改为 1 次完整 nightly + ux 矩阵单独再跑 1 次 | + +## 7. 架构原则（这次收口中统一下来的规则） + +1. **"等用户停下来再做"应当只有一个判定**：本版先让 reload 使用 `editedAt + EDIT_SETTLE`；`BUILD_PROGRAM_QUIET` 和 `DOOM_RETRY_QUIET` 合并到同一判定上，放到 0.0.18 做，并且要逐一说明行为变化。 +2. **隔离要有证据才执行，有证据就交回**：进入隔离需要重复出现或有明确指认；交回由可观察的事实触发（文件变化、闭包身份变化、期满），而不是只靠计时器。 +3. **测试需要哪个版本的外部工具，由 fixture 自己声明**：runner 负责检查并给出清楚的错误；workflow 不再为单个 fixture 硬编码 `case`。 +4. **验收必须基于提交后的干净树**，报告里写明 SHA。 +5. **先归因，后修复，预算不变**：每个性能修复都要有确定性复现，旧产品必须失败、新产品必须通过。一次 nightly 通过不能替代归因。 +6. **跨版本比较时，先固定 runner**：只换 server/payload，才能把产品的变化和测量方法的变化分开。初稿中"0.0.17 是净改善"的结论，就混入了 runner 和 CI 的修复。 +7. **大的二进制证据不进主分支**：留在 PR 历史中，主分支只保留指引和 SHA。 + +## 8. 执行顺序 + +```text +第 1 天 W12 + W1 + W2 + W3(诊断版) ──→ 新 head：PR CI 全绿 + 第一次 nightly（同时验证 W11，并为 W10 拿到日志） + W7 引擎 PR 开工 ──→ engine-candidate CI + U13 ×3 + W6 A/B（payload-from=v0.0.16，ux-mcpp edits 6+6，ux-xlings long 3+3） + +第 2–4 天 W4、W5（确定性 fixture，旧产品失败） + W3 根据数据修复 + W10 根据日志归因并修复 + W6 取证字段 + +收口 最终 head：原九阶段 + 两 profile workspace + 32 组 LSP + 两次 nightly（§5） + ──→ D2 判定、E1 判定 ──→ W9 发布 +``` + +W4/W5 是产品改动，完成之后要重跑完整的两 profile workspace、32 组 LSP、原九阶段和 nightly。W6 的取证字段只增加观察，可以和 W4/W5 一起验证。 + +## 9. 验收清单（合并前） + +- [ ] PR CI 全绿，其中 win32-x64 的 `mcpp-target-windows-real` 实际通过（W2） +- [ ] overflow trace 测试在三个 OS 上通过，失败原因已写入本文（W3） +- [ ] `mcpp-autosave-import-typing`：旧产品失败、新产品通过；70 项 watched-source 不变；nightly U15 ≤ 8（W4） +- [ ] 隔离期的 6 个用例：旧产品在其中 2 个上失败，新产品全部通过（W5） +- [ ] A/B 结果和 D2 判定写入本文；离群值在最终 nightly 中没有再出现，或者已经归因（W6） +- [ ] 引擎 `.5` 已切换，并且 U13 有余量地通过；或者按 E1 写明例外（W7） +- [ ] two-core 通过，或者按 E2 带证据请你决定（W10） +- [ ] 最终 head 两次 nightly 全部 job 通过（§5），termux 和 Zed 默认模式都在内（W11） +- [ ] 原九阶段完整通过；本地两 profile 验收基于干净的提交树（W1） +- [ ] head 不含证据包，README 指向 `fcb3450`；合并后 main 中查不到它的 blob（W12） +- [ ] W9 发布流程完成后再关闭 #63；#59 更新剩余的已知项 + +## 10. 第二版相对初稿的修改（自我 review 记录） + +| # | 初稿 | 问题 | 第二版 | +|---|---|---|---| +| R1 | "0.0.17 相对 0.0.16 是净改善" | self-hosting、startup、U13 CPU、termux、Zed 的转绿主要来自 runner/CI 修复，两个版本不是用同一个 runner 测的 | 结论删除；增加原则 6；A/B 改为固定 runner | +| R2 | "U13 RSS 波动在 0.0.16 就存在" | 0.0.16 的 RSS 8/8 通过；它的测量起点更早，掩盖了增长。现有数据不能直接支持这个说法 | §3.2 用四次数据给出机制推论，并用 A/B 中的 U13 验证；W7 升级 | +| R3 | U15 只看候选版本 | 0.0.16 一直是 10–12，没有余量 | 改为"一直存在的边缘问题"，W4 增加 ≤ 8 的余量验收 | +| R4 | W6 用 6+6 A/B 判定 D2 | 6+6 次分辨不出约 1/12 的离群率；初稿没有用到现有的 18 vs 12 轮数据 | 判定拆成"分布变差"和"离群机制"两部分 | +| R5 | W3 "两边不是同一个集合" | 代码显示两者是同一组请求，诊断不成立 | 改为先打印诊断数据；给出 470 行的候选解释 | +| R6 | W4 新建 `edit_quiet_at`，并合并三个常量 | 已有 `editedAt`；`DOOM_RETRY_QUIET` 是 3 s 且在引擎层，`BUILD_PROGRAM_QUIET` 针对的是磁盘变化，在收口版本里合并会改变现有行为 | 复用 `editedAt + EDIT_SETTLE`，只改 reload；合并推到 0.0.18 | +| R7 | W5 规则 1 用来解释 U7a | U7a 的崩溃有 crash context，规则 1 对它无效；"闭包身份 = 引擎报告的 BMI 哈希"在现有接口里并不存在；"5 次即放弃"实际是退避重启 | 规则 1 对应 U9，规则 2 对应 U7a；身份改用 mcppls 观察到的 `module-built` 序号；表述已更正 | +| R8 | W6 说"取证只改 runner" | 分阶段时间是 server 字段；server 已经有 engine/overhead 统计和 slow-request 记录 | 改为扩展现有的 `slow-request`，并明确这是只增加观察的 server 改动 | +| R9 | W1 把 `verify.py` 加进 allowlist | D5 之后它不在 head 的树里 | 取消该项，只保留干净树规则 | +| R10 | two-core 列为 P3 | D4 下不成立；`38366b9` 证明它可以通过；超时的并不是补全 | 升为 P1（W10），先拿日志归因 | +| R11 | termux、Zed 只出现在验收清单里 | 没有说明它们是在 `fcb3450` 里修的环境问题 | 单列为 W11，规定环境问题的处理方式 | + +## 11. 风险 + +- **W4**：推迟 reload 之后，正在输入 import 的用户最多要等 10 s 才能看到新模块的补全。缓解：`import` 行的模块名补全由 mcppls 自己负责，不依赖模型 reload。 +- **W5 规则 1**：一个没有 crash context 的真实 crash-loop 文件，会多崩溃一次。最坏情况由退避重启约束。 +- **W5 规则 2**：如果 U7a 的崩溃是读取正在重建的 BMI 时发生的竞争，重试可能再崩一次。这种情况期限会翻倍，同一身份也不会再交回，所以最多多一次重启。 +- **W7**：是 D4 的关键路径，但在另一个仓库，CI 和发布有自己的节奏。E1 的时间盒用来防止它无限期拖住 0.0.17。 +- **A/B 与两次 nightly 的成本**：A/B 约 18 个 runner job，最终两次 nightly 约 2 × 1.5 h。 +- **W2**：只给声明了版本需求的 fixture 换上真实 mcpp；默认 pin 2026.9.30.2 与用户实际使用的版本之间的差距，到 0.0.18 处理。 +- **U13 CPU 余量**：`e01e1c1` 的 idle CPU 是 0.008，预算 0.01，余量只有 20%。如果 W7 之后没有改善，就单独归因，不调预算。 diff --git a/.agents/docs/2026-10-24-0.0.17-xmake-sandbox-design.md b/.agents/docs/2026-10-24-0.0.17-xmake-sandbox-design.md new file mode 100644 index 00000000..ff8edb53 --- /dev/null +++ b/.agents/docs/2026-10-24-0.0.17-xmake-sandbox-design.md @@ -0,0 +1,169 @@ +# 0.0.17 增修：xmake 私有运行的隔离与自愈（#68、#69） + +日期：2026-10-24。状态：**已实现**（PR #67；D1 进 0.0.17，D2 全部构建工具，D3 纳入所有 `xmake.lua`）。基线：PR #67 head `e929d12`（引擎已切到 mcppls-clangd 0.0.17 正式版）。 +上游登记：#24 的 [UP-X1](https://github.com/Sunrisepeak/mcpp-language-server/issues/24#issuecomment-6370881870)（mapper 文件共享）、 +[UP-X2](https://github.com/Sunrisepeak/mcpp-language-server/issues/24#issuecomment-6370882444)（describe 依赖只有构建才更新的模块缓存）。 + +## 0. 结论 + +| issue | 用户看到的 | 根因 | 归属 | +|---|---|---|---| +| #69 | mcppls 跑过之后，自己的 `xmake` 增量构建报 `failed to read compiled module`，路径指向 mcppls 的缓存 | mcppls 私有运行写了 xmake 共享临时目录里的 GCC mapper 文件（UP-X1） | **mcppls 的副作用越界**，xmake 的缓存 key 设计是底层原因 | +| #68 | 重新打开项目后，第三方库（gtest）头文件找不到 | mcppls 私有目录里的 xmake 模块缓存过期且不自愈（UP-X2），每次 describe 失败，退回 `inferred` | 坏的是 **mcppls 自己的私有状态**，用户的 `.xmake/`、`build/` 未受影响；xmake 缺陷，mcppls 的用法（只 describe、跨会话复用、原地重配）正好踩中 | + +工程文件（`xmake.lua`）没有问题：同一份工程，guest 实例用全新私有目录每次都成功，用户自己的构建也正常。 + +## 1. 责任划分 + +mcppls 读用户的构建，不替用户构建，也不替 xmake 修状态。由此： + +- **必须做**：mcppls 子进程的一切写入只落在 mcppls 自己的目录里（#69 是这条被违反）。 +- **可以做**：mcppls 自己的私有目录是缓存，坏了就丢掉重来（#68）。 +- **不做**： + - 在私有目录里跑 `xmake build` 来"修"缓存：这是一次真正的编译，有下载包、执行规则脚本等副作用； + - 修改 xmake 的缓存文件； + - 改动用户的 `.xmake/`、`build/`、`~/.xmake`； + - 锁定 xmake 版本。 +- **用户自己**跑 `xmake project` 碰到 UP-X2，或两个 checkout 碰到 UP-X1：mcppls 不处理，只把 xmake 的原话和绕法讲清楚。xmake 的问题在 #24 登记，报给上游。 + +## 2. 证据 + +**log 包**（#68、#69 两个 bundle 共 12 个会话）： + +- owner 实例（私有目录 `workspaces//xmake/`）的 describe **全部失败**：08:48、09:10、13:18、15:07、15:08、15:16。 +- guest 实例（`instances//xmake/`，每次全新）**全部成功**：06:51、08:49、09:11、15:07、15:08、15:18。 +- 失败的会话退回 `inferred`，engine 数据库里没有 `-isystem …/gtest/include`。 +- 15:16 那次是 release 模式下 `xmake f -c` 之后紧接着 describe，失败在 `module/data.cppm`：release 的 dependfile 从未生成，xmake 却认为"已扫描"。 +- 用户点过多次 "reset cache"，都没用：它只删 `model.*.json`，不碰私有 xmake 目录。 +- #69 截图中的 BMI hash `8cd20c490f207114`，和本地复现得到的完全一致。 + +**纯 xmake 复现**（默认布局，不经过 mcppls；xmake 3.1.1、3.1.2，GCC 16.1）： + +- UP-X1（mapper 共享）： + - 同一项目先用第二个 builddir 做 describe，再在自己的 builddir 增量构建：失败。 + - 同一项目的两个副本（例如两个 git worktree）同一天交替构建，删掉第二个后，第一个增量构建：失败。 + - 同一项目 debug、release 交替构建：不受影响。 +- UP-X2（模块缓存过期）：只 describe、从不构建的项目，加一个 target 后删掉 `build/`，之后每次 `xmake project -k compile_commands` 都失败，`xmake f -c` 也恢复不了，构建一次才恢复。3.1.1 上，加 target 后切到 release 也会失败（3.1.2 不会）。 + +**未查明**：报告者私有目录里 dependfile 具体是怎么丢的（报告者的 xmake 版本不在 bundle 里）。下面的方案不依赖这一点。 + +**修复验证**（本地）：加 `XMAKE_TMPDIR` 后，用户的 mapper 文件 md5 不变，用户的增量构建成功；全新私有状态下的 describe 总是成功（guest 实例的每一次运行也证明了这一点）。 + +## 3. 方案 + +### XS-1　隔离临时目录（#69） + +`xmake_environment()` 增加 `XMAKE_TMPDIR=<私有目录>/state/tmp`，和已有的 `XMAKE_CONFIGDIR` 并列。`XMAKE_TMPDIR` 是 xmake 公开的环境变量（`xmake show -l envs` 列出）。 + +### XS-2　私有状态是可丢弃的缓存（#68 的预防） + +- **布局**：`/xmake/state/{config,build,out,tmp}`，加上标记文件 `state/configured`。`xmake-help` 自带 key，仍放在 `/xmake/` 下。 +- **key**（写进 `configured`）在现有的基础上补两项： + - 现有：在线/离线、`xmake.lua` 的戳、用户 `xmake.conf` 的戳； + - 补充：xmake 可执行文件的身份（路径、大小、mtime）； + - 补充：根 `xmake.lua` **以及它 `includes()` 的所有 `xmake.lua`** 中最新的 mtime 和文件数。`xmake-help` 的 key 已经用了 `xmake_manifests()`，这里保持一致。 +- **key 变了**：先 `remove_all(state)`，再从空目录 `xmake f -c …` 配置。不再在旧状态上原地重配。这样"加 target"、"切模式"、"升级 xmake"这几种情况都从全新状态开始，UP-X2 无从触发。 +- **代价**：每次改 `xmake.lua` 多一次全新扫描。log 中 configure 约 1.1 s、describe 约 0.85 s，现状本来就会在 `xmake.lua` 变化时重新配置，增量很小。 +- **磁盘**：只保留一份 state。 + +### XS-3　一次自愈重试（#68 的兜底） + +key 没变、但状态依然过期时（比如目录被外部改动、xmake 自身的边界情况），按 xmake 的原话识别： + +- `dependfile for '…' not found at '…'` +- `module scanner did not run for target(` +- `failed to load moduleinfo for '` + +只在**这次调用开始前 state 就已存在**时才处理。处理方式：丢掉 state，从头配置，再 describe 一次；记一行日志（带 xmake 的原话），两次运行都留在报告的 `toolRuns` 里。还失败，就走现有的失败路径。（实现差异：provider 没有 journal 的接口，所以没有加 journal 事件。） + +工程错误（比如 `missing json dependency for module …`）不在识别范围内，照常报给用户。 + +### XS-4　reset cache 包含构建工具的私有目录 + +`Workspace::reset_cache()` 现在只删 `model.*.json`。增加删除各 provider 的私有目录 `//`（见 D2），使"reset"的含义和用户的预期一致：mcppls 推导出来的东西全部重来。 + +### 不在本次范围（0.0.18） + +- producer 失败时，保留上一次成功的 xmake 模型（标为 stale），而不是退回 `inferred`。 +- 模型问题的提示文字按 UP-X2 给出绕法。 + +## 4. 多角度 + +- **架构**：改动全部在 `src/project/xmake.cpp` 的 `run_private` 及其辅助函数，加上 `reset_cache`。provider 接口不变，其他构建系统不受影响（D2 除外）。 +- **稳定性**： + - 预防（XS-2）和兜底（XS-3）分层：XS-3 最多重试一次，不会循环； + - 私有状态不跨 xmake 版本复用，xmake 升级不会带来旧状态问题。 +- **优雅简洁**：用"按 key 丢弃"代替"原地重配 + 各种修补"，代码路径反而少了 `-c` 原地重配这一支。 +- **用户体验**： + - #68 用户不再丢失第三方 include，不需要手动清缓存； + - #69 用户的构建不再被打断； + - reset 真正有效。 +- **兼容性**： + - 用户的 `.xmake/`、`build/` 继续只读； + - 不支持 `XMAKE_TMPDIR` 的旧 xmake 会忽略这个变量，退回现状（只是没有隔离），不会出错。 +- **跨平台**： + - 路径用 `base::join_path`； + - `XMAKE_TMPDIR` 是 xmake 跨平台的环境变量，但只在 Linux 上实测过，Windows/macOS 要在 CI 上确认； + - mapper 只在 GCC 下出现，其他工具链设置这个变量也无害。 +- **一致性**： + - key 的构成和 `xmake-help` 的 key 一致（都用 `xmake_manifests()`）； + - journal 事件和日志措辞沿用 provider 的现有格式； + - #24 的 UP-X1/X2 行在发布时更新为 "shipped in 0.0.17"。 +- **无感升级**： + - 0.0.16 的旧布局（`/xmake/{config,build,out,configured,left-out}`）在第一次运行时被删掉，并在新布局下从头配置； + - 用户无需任何操作，升级后第一次 describe 多花约 1–2 s； + - 已经处在 #68 状态的用户，升级后自动恢复。 + +## 5. 测试与验收 + +**单元测试**（`tests/test_xmake*.cpp`，不起 xmake）： + +- `xmake_environment()` 包含 `XMAKE_CONFIGDIR`、`XMAKE_TMPDIR`、`XMAKE_THEME`； +- 新增的识别函数 `xmake_state_stale(output)`：三种原话为真，`missing … dependency`、`Invalid option`、包缺失为假； +- `xmake_configuration_key()`：xmake 身份或任意 included 的 `xmake.lua` 变化时，key 也变化； +- 旧布局识别和清理。 + +**conformance**（真实 xmake，CI 固定 3.1.1，GCC 16）： + +- **新 fixture `xmake-shared-tmp`（#69）**： + - 准备阶段：用户自己 `xmake f && xmake` 构建一次（模块库 + 实现单元）； + - mcppls describe 完成（status ready，source xmake）； + - 新 check kind `run`：在工作区里改动实现单元，再跑 `xmake`，要求 exit 0； + - runner 给进程设一个专用的 `XMAKE_TMPDIR`，使用户和 server 共享同一个临时目录（这正是 #69 的条件），同时不污染 CI 机器； + - 0.0.16 必须失败，新版必须通过。 +- **新 fixture `xmake-target-then-mode`（#68）**： + - 初始只有库和 app，用户的配置是 debug； + - 用 `write-file` 加一个依赖库的 `json-test` target，等 reload 完成； + - 再用 `write-file` 把用户的 `xmake.conf` 改成 release，等 reload； + - 断言 source 仍为 xmake，`json-test` 的 engine 命令里有 release 的标志； + - 在 3.1.1 上 0.0.16 预期失败（退回 inferred），新版必须通过。纯 xmake 下验证过的是 `xmake f -m release` 这个序列，mcppls 实际执行的是 `f -c`，所以实现时要先确认 0.0.16 确实失败；如果不失败，就改用 XS-3 的识别路径构造这个 fixture（例如外部删掉私有的 `.d`）。 +- **新 check kind `run`**：在工作区里运行给定程序，断言退出码。现有的 `cli` 只能运行 mcppls 本身。 +- **新 fixture `xmake-stale-state`（XS-1、XS-3、XS-4，实现时增加）**：用 stand-in `xmake` 脚本（POSIX），它的私有状态会像真实 xmake 那样过期：同一份状态里第二次 `xmake project` 报 `dependfile ... not found`，重新配置也修不好。 + - 第一次会话：`XMAKE_TMPDIR` 在 mcppls 的私有目录里； + - 杀掉 server 再启动（等于重新打开编辑器，即 #68 的场景）：过期状态被丢掉并重建一次（这次会话 1 次配置、2 次 describe），模型保持 xmake，没有 stale； + - `mcppls.resetCache` 之后：私有状态已经删掉，下一次加载不会失败。 +- **本地结果**：三个新 fixture 在 0.0.16 上都失败（3、1、6 项），新版全部通过；5 个原有的 xmake fixture 通过。xmake 3.1.2 下 `xmake-shared-tmp` 同样能区分新旧；`xmake-target-then-mode` 的模式切换在 3.1.2 里已经由 xmake 修好，CI 固定 3.1.1，在那里能区分。 +- **回归**：`xmake-basic`、`xmake-late-config`、`xmake-user-mode`、`xmake-no-standard`、`xmake-needs-download` 全部通过；`W1 workspace-unchanged` 保持。 + +**本地**：`/tmp/xm/*.sh` 的纯 xmake 脚本在 3.1.1、3.1.2 上复跑；用报告者布局（模块库、实现单元、`import std`、测试 target）走一遍 #68、#69 的序列。 + +## 6. 文档 + +- `CHANGELOG.md` 0.0.17：新增 "xmake" 小节，写 #68、#69 的修复和用户可见的变化（reset 会清 xmake 私有目录，首次运行多 1–2 s）。 +- `docs/20-projects.md` 及 zh-CN：xmake 小节说明私有运行的隔离范围（config、build、tmp），以及 UP-X1/X2 的用户侧绕法。 +- `docs/50-troubleshooting.md` 及 zh-CN：`dependfile … not found` 的含义和处理。 +- `conformance/README.md`：两个新 fixture 和 `run` check。 +- #24：UP-X1/X2 的 mcppls 列在发布时更新。 + +## 7. 对 0.0.17 发布的影响与需要你决定的事 + +- 只改 mcppls，引擎不动。#67 上多一个提交，再跑一轮 PR CI（约 50 min）；xmake 不在 ux nightly 里，不需要重跑 nightly。 +- **D1**：进 0.0.17（推荐：代码量小，修的是 0.0.16 用户正在遇到的两个问题），还是放 0.0.18。 +- **D2**：reset 清哪些私有目录：只清 xmake，还是所有构建工具（cmake、meson、xmake）？推荐全部清：reset 的语义一致；cmake 重新配置的代价只在用户主动 reset 时发生。 +- **D3**：key 是否纳入所有 `includes()` 的 `xmake.lua`？推荐纳入：target 常常定义在被 include 的文件里，不纳入的话那条路径只能靠 XS-3 兜底。 + +## 8. 风险 + +- **XS-3 的识别依赖 xmake 的报错原话**：xmake 改了措辞就识别不到，退回现状（describe 失败、给出提示），不会更糟；XS-2 不依赖原话。 +- **全新扫描的代价**：每次改 `xmake.lua` 多约 1–2 s，这段时间里规划用的是缓存的模型，用户不需要等待。 +- **D2 选"全部清"时**：大型 cmake 项目在用户 reset 之后，要多等一次私有配置。 diff --git a/.agents/docs/design.md b/.agents/docs/design.md index ceabdb66..312bfabd 100644 --- a/.agents/docs/design.md +++ b/.agents/docs/design.md @@ -168,6 +168,15 @@ nothing (#30). The build tool's hard deadline follows how long it took last time time, then three times the last run, 1 to 10 minutes), and a saved source reruns it only when its module structure changes. +A source's lexical imports can differ from the producer's preprocessed dependency graph. +After a successful, valid answer, the workspace can acknowledge the watched source structure +captured before that run, including its preprocessor directive lines. A body-only save of that +acknowledged structure need not rediscover the same graph. Confirmation is session-local and +belongs to the current model generation and native producer identity; changed file stamps, +pending reloads, failed/rejected answers and unknown preprocessing retain the conservative path. +Only already watched source paths are captured, under a shared 32 MiB read bound. Source +creation/deletion and manifest, target or producer changes keep their existing reload behavior. + **Observability.** One occurrence must be enough to see why. clangd logs at `info` into a ring in memory; crashes, stuck or spinning clangd, files set aside, backed-off restarts and a workaround whose premise is seen broken each write an incident under the workspace's cache (the newest twenty, @@ -306,12 +315,12 @@ before anything is published (`docs/92-release.md`). - mcpp analysis can pass an explicit platform with `mcppls.buildDiscovery.mcppTarget`, or follow the root compilation database's unique explicit target in `auto` mode. This does not recover the full profile/toolchain/features or prove which build last succeeded (2026-10-22 target design). -- The bundled `23.1.0-mcppls.4` engine constructs persistent memo JSON every 30 seconds before - checking whether its counters changed. An isolated session with retained memos and no opened - files reproduces idle RSS growth from those discarded snapshots. Persistence remains enabled - for warm reuse; mcppls does not hide the allocation with a delay or a changed resource budget. - The source attribution and upstream-side fix required are recorded in - [the U13 brief](2026-10-22-issues-59-63-evidence/README.md#memo); no workaround is applied. +- The bundled `23.1.0-mcppls.5` engine no longer serializes unchanged memos (engine patch 0036; + `23.1.0-mcppls.4` built the JSON of every memo each 30 seconds before checking its counters), and + an idle clangd's CPU fell by an order of magnitude. Its resident memory can still grow over the + nightly's ten idle minutes when the allocator gave memory back just before them; that growth was + not caused by the memo writes and is not attributed yet ([0.0.17 release record](mcppls-0.0.17-release.md), + section 10). mcppls does not hide it with a delay or a changed resource budget. - openkal cannot lower a child's scheduling priority, so clangd's cold-start module builds compete with the editor (plan 2026-09-30 R-9, deferred). - Windows gives no CPU reading of clangd yet (plan 2026-09-30 K-1, deferred), so the stuck-clangd diff --git a/.agents/docs/mcppls-0.0.17-release.md b/.agents/docs/mcppls-0.0.17-release.md new file mode 100644 index 00000000..c4d52a8a --- /dev/null +++ b/.agents/docs/mcppls-0.0.17-release.md @@ -0,0 +1,360 @@ +# mcppls 0.0.17 联动发布设计（mcppls 0.0.17 + mcppls-clangd 0.0.17） + +日期：2026-10-23。状态：**已实施，发布中**（§10 记录实施与设计的差异；§8 标出每项的实际结果，`[~]` 为部分达成或带给你决定后放行）。 + +本文取代 [`2026-10-23-0.0.17-part2-design.md`](2026-10-23-0.0.17-part2-design.md)（part2 设计）。那份文档里的数据和分析已并入本文，本文可以独立阅读。 + +## 0. 目标与已定决定 + +**目标**：引擎和 mcppls 一起发布 0.0.17，把 0.0.16 以来已知的问题都在这一版里处理掉。 + +**发布标准**从 CI 的角度定义（§6）： + +- **底线**：不低于 0.0.16。 +- **目标**：尽量全绿。 + +| # | 决定 | +|---|---| +| D1 | W4（autosave reload）和 W5（隔离期）进入 0.0.17 | +| D2 | U5/U6 慢补全是否阻断，看测试内重复覆盖的结果（§5） | +| D3 | 引擎和 mcppls **联动**发布 0.0.17，引擎修复不再推迟到下一版 | +| D4 | CI 底线是不低于 0.0.16，目标是全绿 | +| D5 | 约 1 MB 的证据包只留在 PR 的提交历史里，不进 main | +| E3 | U7a 崩溃的根因尽量复现并修复；如果查不到，靠 W5 把用户感受到的影响降下来，根因在 #59 里记为未解决 | +| W6 | 偶发慢补全不靠多跑 12 次构建，而是在测试内覆盖 12 次；同时整体降低整条链路的构建成本 | +| PR | 新开 0.0.17 release PR，#65 的提交并入其中 | + +## 1. 现状 + +### 1.1 mcppls-clangd（引擎） + +- 已发布 v0.0.16（`23.1.0-mcppls.4`，10-13 发布）。之后没有新的 PR，也没有新的发布。补丁序列到 `0035-UP-40`。 +- mcppls 0.0.16 和 0.0.17 候选版本锁定的是**同一份引擎**，所以 mcppls 侧的所有对比都不涉及引擎的变化。 +- **引擎自己的定时 CI 从 10-07 起每天都红**，0.0.16 发布时就是红的： + - `e2e`：gate 中 `product_completion_both` 和 `product_diagnostics_both` 两项 FAIL，其余三项 PASS（`fork_version_detected`、`fork_memo_fired`、`no_latency_regression`）。 + - `soak`：sanitizer 构建（`build-san`）中，`modules.test`、`modules-stale-lock.test`、`modules-validation-cache.test` 三项 lit 测试失败，日志里有 "Failed to build module prerequisites"。冷构建大约要 3 小时。 +- 已知的引擎问题（0.0.16 上同样存在）： + - memo 每 30 s 构造一次完整的 JSON，空闲时 RSS 会上涨。 + - U7a 场景里，AST 构建时发生崩溃，崩溃栈中有 realloc。Linux/macOS 的 release 是 stripped 的，这个崩溃无法符号化。 + +### 1.2 mcppls + +- main 在 `fd869bd`：#64 已 squash 合入，版本号已是 0.0.17，还没打 tag。 +- PR #65（draft）在 main 之上有 4 个提交： + +| 提交 | 内容 | 是否改 server | +|---|---|---| +| `072c230` | 子进程消息在 writer 上编码 | 是（`src/lsp/connection.*`） | +| `1341938` | 计划应用后发布已提交的 workspace 快照 | 是（`workspace.cpp` 19 行） | +| `e01e1c1` | 真实 mcpp Windows target 的测试 | **否** | +| `fcb3450` | watched source、editor roots；CI 环境修复（termux 官方源、Zed 预装 clangd、保存失败阶段日志）；证据包 | 是 | + +- `fcb3450` 的 PR CI 有 3 类失败，没有一类来自 server： + 1. `test_check`：`verify.py` 不在 allowlist 中（×6）。 + 2. Windows 上 overflow trace 的断言失败（合并树中的第 466、467、470 行）。 + 3. `mcpp-target-windows-real` 用了 PATH 上的旧版 mcpp `2026.9.30.2`（`\"` 引号缺陷，上游 2026.10.5.2 已修复）。 +- `fcb3450` 还没跑过 nightly。 + +### 1.3 nightly 对比（0.0.16 定时 8 晚 vs 候选 4 次 workflow_dispatch） + +| job | 0.0.16 | 候选 | 说明 | +|---|---|---|---| +| ux-xlings 1（U13） | 8/8 红（CPU） | 2/4 红（RSS） | 见 §1.4 | +| ux-mcpp 1/2/3 | 1/24 红（U9 121 s） | 3/12 红（U15 13 ×2，U5/U6 离群 ×1） | U15 在 0.0.16 上也一直是 10–12，预算 12 | +| ux-mcpp two-core | 8/8 红 | 0/4 红 | 变好 | +| ux-xlings two-core | 0/8 红 | 2/4 红 | U5 有 1–2 次 server 超时；U7a p95 1.53/1.71 s，预算 1.5 | +| self-hosting、startup | 8/8、4/8 红 | 0/4 红 | 主要来自 CI/runner 修复 | +| termux | 8/8 红 | `e01e1c1` 因镜像 404 红 | `fcb3450` 已修，待验证 | +| Zed 默认模式 | 绿 | `e01e1c1` 因 GitHub API 403 红 | `fcb3450` 已修，待验证 | + +**关键事实**：`e01e1c1` 的 server 代码和 `1341938` **完全相同**，但前者的 nightly 中 ux 全绿，后者红了 4 个 job。也就是说，同一份产品代码在不同的运行中结果不同，坏结果是按"那一次运行"聚在一起的。所以单次 nightly 不足以判断产品好坏，必须做重复测量和同机对照（§5）。 + +### 1.4 U13 RSS：失败和通过分成两组 + +| run | 结果 | idle 开始 → 结束 | idle CPU | +|---|---|---|---| +| `38366b9` | 通过 | 1391.8 → 660.3 MB（−52.6%） | 0.0059 | +| `e01e1c1` | 通过 | 1494.3 → 675.4 MB（−54.8%） | 0.0080 | +| `072c230` | 失败 | 624.7 → 1086.6 MB（+73.9%） | 0.0060 | +| `1341938` | 失败 | 948.1 → 1188.3 MB（+25.3%） | 0.0061 | + +- 通过的两次：测量期间发生了 trim。 +- 失败的两次：trim 发生在测量开始之前，之后 10 分钟内 RSS 一直涨，没有再 trim。 +- 每次的 idle CPU 合计约 3.6 CPU 秒。这和"memo 每 30 s 构造一次约 30 MB 的 JSON，共 20 次"的量级相符。 +- 0.0.16 的测量起点更早，正好落在高位，所以这个增长被掩盖了。 +- mcppls 侧唯一能让它变绿的办法是"挑测量起点"，而 #59 的 review 已经禁止这样做。所以根本的修复在引擎里（E1）。 + +## 2. 问题清单（按仓库） + +### 2.1 mcppls-clangd 0.0.17（`23.1.0-mcppls.5`） + +| # | 问题 | 方案 | 验收 | +|---|---|---|---| +| E1 | memo 周期性构造 JSON（U13 RSS） | 新补丁 `0036`：先比较两个缓存廉价的 changeCount，没有变化就不构造 JSON；最后一次 flush 仍然强制写 snapshot。不改 30 s 周期，不改 JSON 格式 | 原生 canary（旧实现必须失败）；mcppls nightly 中 U13 RSS 增长 < 5%（预算 10%）。runner 在 idle 两端各取一次 `$/memoryUsage`（M6）：如果 MemoryTree 不变而 RSS 仍在涨，再考虑第二步，在 flush 之后调用 `malloc_trim` | +| E2 | 崩溃不可符号化 | release 为 linux-x64/arm64 发布 `objcopy --only-keep-debug` 生成的 debug 文件（按 build-id 命名），为 macOS 发布 dSYM。用户下载的包大小不变 | 拿一个已知崩溃，用发布出来的符号文件离线符号化成功 | +| E3 | U7a：AST 构建时崩溃（栈中有 realloc） | 用 E5 修好的 sanitizer 构建，在本地跑 U7a 序列（U5→U6→U16→U7a）来复现。查到原因就修，并补 lit 回归测试；查不到，就在 #59 中记为未解决，由 W5 降低影响 | 复现成功时：修复前崩溃，修复后不崩溃 | +| E4 | 定时 `e2e` 的 product gate 红（从 10-07 起） | 先归类：是 gate 引用的 mcppls 产品或接口过时，还是引擎行为确实变了。前者改 e2e，后者修引擎 | 定时 `e2e` 变绿 | +| E5 | 定时 `soak` 的 sanitizer 构建中 3 项 modules lit 失败 | 先修 sanitizer 构建下的 module prerequisites 环境（很可能是 resource-dir/标准库路径问题，需要读日志确认），然后给 `build-san` 加上缓存，让它不再每次都冷构建 3 小时 | 定时 `soak` 变绿，热构建时间 < 1 h | +| E6 | 如果 W6 取证指向引擎（例如补全要等 preamble 或模块前置重建） | 根据取证结果决定是否在引擎侧让补全使用旧 preamble | 取证之后再定 | + +### 2.2 mcppls 0.0.17 + +| # | 问题 | 方案 | 验收 | +|---|---|---|---| +| M1（W4） | autosave 下正在输入的文件，每次保存都触发 reload（U15） | 复用 `editedAt + EDIT_SETTLE`：文件打开、并且还在输入时，把结构 reload 推迟到 `edit_quiet_at`，最多推迟 10 s。不改 status 语义，不改预算。`BUILD_PROGRAM_QUIET` 和 `DOOM_RETRY_QUIET` 的合并留到 0.0.18 | 确定性 fixture `mcpp-autosave-import-typing`：旧产品 ≥ 8 次，新产品 ≤ 2 次；nightly 中 U15 ≤ 8 | +| M2（W5） | crash/exit 之后一律隔离 2 min（U9 121 s，U7a 107 s） | ① 没有 crash context 的退出，第一次只记 strike，10 min 内第二次才隔离（针对 U9）。② 有 crash context 的隔离：等 import 闭包里的模块全部构建成功、并且身份变化后（身份取 mcppls 观察到的 `module-built` 序号），交回重试一次（针对 U7a）。其余规则不变；"5 min 内 5 次退出"仍然走退避重启 | 6 个确定性用例，旧产品必须在 2 个上失败 | +| M3（W6） | U5/U6 偶发慢补全（8.26 s，1/12） | 取证：扩展现有的 `slow-request` 记录，加入 `queueMs`、`engineMs`、`answerLayerMs` 和发送时的 clangd fileStatus；completion 的阈值降到 1 s，最多保留 32 条。重复覆盖见 §5。修复方案在取证之后再定（mcppls 侧或 E6） | §5 的验收 | +| M4（W10） | ux-xlings two-core：server 超时，U7a p95 略超 | 从保存下来的失败阶段日志中取出超时的 method 和原因（是 `waited too long for …` 还是 `did not answer … in time`）。如果是 mcppls 的等待上限在两核机器上太紧，就修 mcppls；如果是 clangd 在两核上确实答不过来，带着证据来找你 | two-core 在 §5 的重复测量中通过 | +| M5（W3） | Windows overflow trace 断言 | 先让测试在失败时打印诊断信息。候选解释是：mock 计数的 completion 里包含打字开始之前的那些请求，所以慢请求在客户端的 index 低于 256。在证据出来之前，不放宽断言，也不按平台跳过 | 三个 OS 都通过，原因写入本文 | +| M6 | U13 的归因手段 | runner 在 idle 两端各发一次 `$/memoryUsage`，记录 MemoryTree 和 RSS | U13 的 measure 中有这两个字段 | + +### 2.3 CI 与仓库 + +| # | 问题 | 方案 | +|---|---|---| +| C1 | **nightly 用不了候选引擎**：在 reusable 调用中 `github.workflow` 是 `nightly`，不满足 Candidate engine 步骤的放行条件，会直接报错。结果是 E1 的效果（U13）只能等引擎发布后才能验证，违反"先在产品 CI 中验证引擎"的规则 | 放行**非 main 分支上手动触发的** nightly。定时 nightly、main 和 release 照旧拒绝 | +| C2 | nightly 每次都要重建一遍 ci（约 46 min） | 见 §4 R1 | +| C3 | 没有办法定向、重复地跑 | 见 §4 R2 和 §5 | +| C4（W2） | Windows 真实 fixture 用了旧版 mcpp | fixture 在 `scenario.json` 中声明 `"requires-mcpp": ">=2026.10.5.2"`，由 runner 检查；CI 只要矩阵项里有这类声明，就安装 release 版 mcpp。两个环境变量合并为 `MCPPLS_REAL_MCPP`。默认 pin 本版不动。验收：win32 conformance 上 i686→x64→i686 实际跑通（到目前为止从未通过） | +| C5（W11） | termux 和 Zed 默认模式 | `fcb3450` 已经修了，在 nightly 上实际验证。如果再遇到环境故障，修环境后重跑，不对单个 job 豁免 | +| C6（W12、W1） | 证据包与本地验收 | 见 §3.3；本地验收必须在提交后的干净树上跑，报告里写明 SHA | + +## 3. 联动流程与 PR 组织 + +### 3.1 分支与 PR + +- **mcppls**： + - 从 `fcb3450` 拉出 `release/0.0.17`，后续工作作为新提交加在上面，开一个新 PR，标题写"0.0.17: … (mcppls-clangd 0.0.17)"。 + - #65 关闭，并附上新 PR 的链接。`fcb3450` 同时是新 PR 和 `refs/pull/65/head` 的祖先，这满足 D5 的前提。 +- **mcppls-clangd**：从 main（`92a4c63`）拉出 `engine-0.0.17`，开 PR。 + +### 3.2 顺序（沿用 `docs/92-release.md` 的 "A new engine"） + +1. 引擎 PR 的 CI 构建 `clangd--portable`。 +2. mcppls release PR 提交 `packaging/engine-candidate.json`，指向这次引擎构建。PR CI 和手动 nightly（C1 修好之后）都用这个候选引擎来跑。 +3. 两边发现的问题各自修复；候选文件始终指向最新的引擎构建，直到两边都绿。 +4. 合并并发布引擎（mcppls-clangd v0.0.17）。然后在同一个提交里，把 `payload.lock.json` 切换到已发布的 archive，并删除候选文件。 +5. 用已发布的引擎字节跑一遍 PR CI。如果发布出来的 clangd sha256 和候选版本**相同**，就不再跑 nightly；**不同**，就再跑一次 nightly（§4 R5）。 +6. squash 合并 mcppls，按发布流程（§7 第 6 步）发布。 + +### 3.3 证据包（D5） + +1. 不 rebase，也不 force-push 越过 `fcb3450`。 +2. 合并前的最后一个提交，用 `git rm` 删除 `portable-handoff/` 下的 `evidence.tar.gz`、`manifest.json`、`archive.json`、`verify.py`。`README.md` 改写成一份指引，写明： + - 提交号 `fcb3450`； + - 整包的 SHA-256； + - 获取方法：`git fetch origin pull/65/head && git worktree add /tmp/ev fcb3450`。 +3. `verify.py` 不在最终树里，所以 `test_check` 和 allowlist 的问题也就自然消失了。 +4. squash 合并后核对：main 的树里和 `git rev-list --objects` 里都找不到这个 blob。 + +## 4. 降低整条链路的构建成本 + +当前每轮迭代的成本（实测）： + +| 环节 | 耗时 | +|---|---| +| mcppls PR CI | 约 46 min（最长的是 ux-xlings 31 min、conformance 22 min、build 12 min） | +| mcppls nightly | 约 78 min，其中 ci 约 46 min，ux 约 20–35 min | +| 引擎 PR CI（build-test） | 31–37 min（缓存命中时）；174 min（未命中） | +| 引擎 release | 87–135 min | +| 引擎 soak | 约 3 h（每次冷构建 sanitizer） | + +改进： + +- **R1：nightly 复用已有的 payload**。workflow_dispatch 新增输入 `payload-run`（某次 PR CI 的 run id）。设置后跳过 `ci` job，下游 job 用 `download-artifact` 的 `run-id` 取 `payload-`。 + - 效果：每次 nightly 从约 78 min 降到约 35 min。更重要的是，nightly 测的就是 PR CI 测过的**同一份字节**。 + - 定时 nightly 不受影响，仍然完整构建。 +- **R2：定向运行**。workflow_dispatch 新增 `fixtures`、`stages`、`repeat` 和 `payload-from` 四个输入（§5）。只跑需要的部分： + - 验 U13：只跑 ux-xlings 的 long 阶段，约 15 min； + - 验 U5/U6：只跑 edits 阶段的重复测量。 +- **R3：引擎在本地迭代**。补丁在本地构建 clangd，换进 payload 来跑 fixture（做法见 local-engine-payload-swap 记录：修 `payload.json` 中的 identity 和 sha256，并用 `payload --verify` 检查）。`CLANGD_TRACE` 和 `--log-level debug` 用来看 span 与 memo 计数。引擎 PR **只在里程碑时推送**（补丁就绪、修复完成），不每改一处就推。 +- **R4：mcppls 先本地、后批量推送**。本地在干净树上跑通 32 组 LSP 和相关 fixture 之后，再把几个提交一次推上去，避免每个提交都花 46 min 跑 CI。 +- **R5：引擎发布后的复验**。如果引擎 release 产出的 clangd 字节和候选版本一致，只需要跑一次 PR CI；不一致才再跑 nightly。第一次发布时记下两者的 sha256，以后就知道是否需要这一步。 +- **R6：soak 加缓存**（E5）。`build-san` 改用和 build-test 相同的缓存策略。 + +## 5. 在测试内覆盖 12 次，而不是构建 12 次 + +**原则**：构建和准备只做一次，同一份 payload 在测试里重复跑多个会话。重复的单位是**新起一个 server 会话**，因为离群值看起来是会话级的（同一会话里 U5 和 U6 同时变慢）。 + +**runner 改动**：`mcppls-conformance run` 新增两个参数： + +- `--repeat `：在同一个 workspace、同一份已准备好的缓存上，依次启动 N 个新的 server 会话。 +- `--checks `：只跑指定的检查项，例如 `U5-typing,U6-typing-autosave`。 + +每次会话的 measure 单独保存（`-rep`），另外输出一份汇总：每次的 p95/max、超预算的次数，以及各次 server 证据中的 `slow-request`。预算不变。 + +**分布方式**： + +- 12 次重复分到 4 个并行 job，每个 job 3 次。这样既有同机重复，也覆盖不同的机器。 +- 每次会话约 4.5 min（U5 + U6），每个 job 约 15 min 加准备时间，整轮约 30 min，不需要额外构建。 + +**同机 A/B**： + +- `payload-from=v0.0.16` 时，每个 job 在同一台机器上**交替**跑 0.0.16 的 payload 和候选 payload，顺序为 A B B A。这样可以抵消时段和机器的差异。 +- 1.3 节已经说明，机器和时段的噪声足以把同一份代码判成红。 +- 0.0.16 的 payload 来自 v0.0.16 release 的 `payload-linux-x64.tar.gz`。runner 要能容忍 0.0.16 没有的报告字段（当作 null），并记录缺了哪些字段。 + +**次数与统计（说明）**： + +- 如果离群率确实是 1/12，12 次重复能看到它的概率约为 65%；24 次约 88%；36 次约 96%。 +- 所以分两个阶段： + - **取证阶段**：跑 3 轮 × 12 次，每轮约 30 min，用来抓到离群值和它的日志。 + - **验收阶段**：最终 head 跑 12 次，并且必须与 0.0.16 做同机 A/B。 + +**D2 判定**： + +1. 最终 head 的 12 次全部在预算内，并且在同机 A/B 中，候选版本 max 的中位数不比 0.0.16 高出两组合并标准差的 2 倍以上，才算通过。 +2. 取证阶段抓到离群值时，必须先归因并修复（在 mcppls 或 E6），然后重新验收。 + +U13 和 two-core 也走同样的方式：U13 每边 3 次（只跑 long 阶段）；two-core 用 `taskset -c 0-1` 跑 6 次。 + +## 6. CI 发布标准 + +### 6.1 底线（不低于 0.0.16） + +- **mcppls PR CI**：每个 job 都通过，包括新增的 fixture（0.0.16 发布时 PR CI 是全绿的）。 +- **mcppls nightly**： + - 0.0.16 上一直是绿的 job，必须是绿的。 + - 0.0.16 上就是红的 job，原因不能变，结果也不能变差。 +- **引擎 PR CI**：build-test、verify-series、win-historical-crash-capture 都通过，和 0.0.16 的 PR 相同。 + +### 6.2 目标（尽量全绿） + +- mcppls nightly 的每个 job 都通过，包括 continue-on-error 的 termux 和 two-core。0.0.16 上红的每个 job，这一版都有对应的修复：U13 → E1，two-core → M4，termux → C5，U9 → M2，U15 → M1。 +- 引擎的定时 `e2e` 和 `soak` 变绿（E4、E5）。 +- 有余量：U15 ≤ 8；U13 的 RSS 增长和 CPU 都低于预算的 50%。 + +### 6.3 最终判定 + +- 在最终 head 上（引擎已发布、候选文件已删除），用 R1 复用 PR CI 的 payload，**跑两次 nightly**，加上 §5 的验收重复测量。 +- 某个 job 红了先归类： + - **环境问题**（镜像、限流、runner 故障）：修好环境后重跑。 + - **产品问题**：修复之后，最终 head 改变，计数从零开始。 +- 不改预算，不挑测量起点，也不靠重跑直到变绿。 +- 如果只达到底线、没达到目标：把没变绿的项目和证据带给你决定，不自动放行。 + +## 7. 执行顺序 + +1. **第一批，两边同时开工**： + - CI：C1（nightly 可以用候选引擎）、R1（复用 payload）、R2 和 §5 runner 的 `--repeat/--checks`。这些是后面所有验证的前提。 + - 仓库：C6、C4、M5（先上诊断版）。推一次 PR CI，跑第一次 nightly。这一次 nightly 同时完成三件事：验证 C5，为 M4 拿到日志，并作为 0.0.17 head 的第一个基线。 + - 引擎：E5（修 sanitizer 构建并加缓存），E1 在本地开发。 +2. **第二批**： + - mcppls：M1、M2、M3 取证字段、M6。 + - 引擎：E1 推到引擎 PR；E3 用 sanitizer 构建在本地复现；E4 归类；E2 符号发布。 + - mcppls PR 加上 `engine-candidate.json`。 +3. **取证与修复**： + - §5 取证阶段跑 3 轮 × 12 次，抓 U5/U6 离群值和 two-core 的日志。 + - 按结果修复：M3/M4，或者 E6。 + - U13 用候选引擎做 3 + 3 次 A/B。 +4. **收口**： + - 引擎 PR 全绿，合并并发布 v0.0.17。 + - mcppls 切换 lock、删除候选文件。 + - 跑原九阶段、两个 profile 的 workspace、32 组 LSP、两次 nightly，以及 §5 的验收。 +5. **D5**：最后一个提交删除证据包。 +6. **发布**：squash 合并，然后依次完成： + - main CI； + - draft release； + - 逐一下载 12 个资产并校验； + - 用两个原 0.0.16 profile 实际升级； + - 正式发布，并再次下载回验； + - 核对 main 中没有证据包的 blob。 + +## 8. 验收清单 + +`[x]` 达成；`[~]` 部分达成，或按 §6.3 带给你决定后放行；`[ ]` 未做。 + +**引擎** + +- [~] E1：原生 canary 中旧实现失败、新实现通过（`UnchangedMemosAreNotSerializedAgain`）；空闲 CPU 降了一个数量级。**U13 RSS 增长 < 5% 未达成**：ux-xlings +58%，原因不是 memo，你决定按已知问题发布（CHANGELOG Known issues） +- [~] E2：已发布未 strip 的 clangd（Linux、macOS 的 `*-symbols.tar.gz`）；没有用它实际离线符号化一个已知崩溃 +- [~] E3：未复现；复现过程和未解决的原因记在 §10（按你的指示，没有在 #59 评论） +- [x] E4、E5：分支上的 `e2e`、`soak` 变绿（引擎 PR #9） +- [x] 引擎 PR CI 全绿；v0.0.17 已发布，四个平台的包各下载两次并和 `.sha256` 一致；mcppls 已切换 lock + +**mcppls** + +- [x] M1：`mcpp-autosave-import-typing` 在旧产品上失败、新产品通过；同机 A/B 中 U15 最忙一分钟为 6–7.5（0.0.16 为 12） +- [x] M2：`clangd-exit-strike`、`clangd-crash-closure` 在旧产品上失败、新产品通过；`clangd-crash-context` 等回归通过 +- [~] M3：取证字段已发布；U6 的补全 max 比 0.0.16 高 0.15–0.25 s（p95 不变），没有归因，你决定按已知问题发布 +- [x] M4：two-core 两轮在同机 A/B（每轮 edits 重复 4 次）中通过 +- [x] M5：三个 OS 都通过，原因已写入 §10 +- [x] C4：win32 上 `mcpp-target-windows-real` 实际通过（PR CI） +- [x] C5：完整 nightly `40009683587`（`e278a58`，复用 PR CI 的 payload）中 termux + proot-distro、Zed 默认与推荐设置都通过 +- [~] PR CI 全绿（引擎候选、正式引擎、xmake 增修各一轮）。nightly 两次：ux A/B（`39889041291`）中 ux-xlings 第 1 轮 U13 失败（与 0.0.16 同一原因），按 §6.3 带给你决定；完整 nightly `40009683587` **全部通过**（U13：ux-xlings −8.5%、ux-mcpp −22.2%，窗口内都有 trim，所以已知问题仍然成立） +- [ ] 原九阶段的本地完整重跑没有做；本地验证改为：发布资产校验、真实会话、0.0.16 缓存的升级接管 +- [x] D5：最终树里没有证据包（`fcb3450` 之后已删除） +- [ ] 发布流程完成；关闭 #63、更新 #59（issue 状态按你的指示不改） +- [x] 增修 #68、#69：见 [2026-10-24-0.0.17-xmake-sandbox-design.md](2026-10-24-0.0.17-xmake-sandbox-design.md)，三个新 fixture 在 0.0.16 上失败、新版通过，并已在 CI 中运行 + +## 9. 风险 + +- **E3 可能复现不出来**：崩溃可能依赖 CI 机器的时序。缓解：sanitizer 构建会把很多时序相关的问题变成确定性的报错；即使根因没查到，M2 也能把用户感受到的影响降到几秒。 +- **E4、E5 可能牵出引擎测试体系里更多的问题**：它们从 10-07 起就是红的，原因还没读过。如果工作量超出预期，按 §6.3 带来请你决定，不阻塞底线。 +- **M3 离群值可能在取证阶段抓不到**：36 次都没出现时，95% 置信度下离群率 ≤ 约 8%。到那时，按"验收的 12 次全部通过"放行，并把取证字段随 0.0.17 一起发布，用户导出的诊断包里也会带上这些字段。 +- **C1 放宽了候选引擎的使用范围**：只放行非 main 分支上的手动 nightly；release workflow 中"拒绝候选文件"的检查保持不变，所以候选引擎仍然到不了发布版。 +- **M1**：正在输入 import 的用户最多要等 10 s 才能看到模型更新。`import` 行的模块名补全由 mcppls 自己提供，不受影响。 +- **M2 规则 ①**：一个没有 crash context 的真实崩溃循环，会多崩溃一次；最坏情况由退避重启约束。 +- **成本**：用了 R1–R6 之后，每次迭代的主要成本是 PR CI 约 46 min，加上定向 nightly 约 15–35 min。引擎 PR CI 只在里程碑时触发。 + +## 10. 实施记录（与设计的差异） + +- **W4 的机制被本地复现证实，上限改为 30 s**：新 fixture `mcpp-autosave-import-typing` 按 U6 的方式交替输入 + import 行和函数体（10 Hz，每秒自动保存，40 s）。0.0.16 在该 fixture 上跑出 6 次 producer、13 次状态变化/分钟， + 复现了 U15 的失败；只输入 import 的连续输入不能复现（防抖窗口一直被推后）。设计里的 10 s 上限在不停顿的输入下 + 恰好是 6 次/分钟、12 次状态变化，正好卡在预算上，没有余量；改为 30 s 后为 3 次、4 次状态变化。停顿两秒仍会触发 + 运行，所以只有完全不停顿的输入才会碰到这个上限。 +- **W5**：`clangd-exit-strike`（无 crash context 的 kill）和 `clangd-crash-closure`（指认文件的崩溃 + 闭包重建） + 在 0.0.16 上失败，在新产品上通过；`clangd-crash-context` 等回归 fixture 不变。 +- **E2 改为发布未 strip 的 clangd**，不加 `-g`：给每个目标文件加调试信息会让 ccache 超过 GitHub 的 10 GB 缓存上限， + 每次引擎 CI 都变成约 3 h 的冷构建。未 strip 的二进制保留了符号表，没有额外构建成本，足以给崩溃地址补上函数名。 +- **E4 根因**：`lsp_driver.py` 要求 `publishDiagnostics.version == doc_version`，而 mcppls 发布的合并诊断不带这个 + 可选字段，所以产品层的两个 gate 自 10-07 起一直失败。现在接受不带版本、但落在只存在于最新文本中的那一行的诊断； + 本地 5 个 gate 全部通过。 +- **E5 根因**：module worker 用 `RLIMIT_AS` 限制地址空间，而 sanitizer 运行时启动时就已映射了数 TB 的 shadow 内存， + 所以 worker 的第一次 mmap 就失败（"module worker failed: ERROR: Failed to mmap"）。新补丁 0037 在 sanitizer 构建下 + 让上限只约束 worker 在已有映射之外新增的部分。本地 ASan+UBSan 构建下三个 lit 测试和 soak 的 kill/restart 循环都通过。 +- **E3 未复现**：用本地 ASan+UBSan 构建的 clangd（E5 修好之后才可用），在真实 ux-mcpp 上完整跑了一遍 + cold、warm、edits（U5、U6、U16、U7a、U8）：没有任何 sanitizer 报告，没有 crash context；两次重启都是计划内的 + （引擎数据库变化）。几个预算检查因为 ASan 变慢而超时，与崩溃无关。U7a 的根因按约定记为未解决，影响由 W5 规则 ② + 兜住（`clangd-crash-closure`）。 +- **M5（W3）根因**：新加的诊断输出显示 Windows 上只发出了 65 个补全请求。Windows 的 sleep 精度是 15.6 ms， + "1000 Hz 打一秒"实际只有约 64 次按键，mock 故意放慢的第 270 个请求根本没有发出。所以"统计和 trace 不是同一组请求" + 这个最初的解释不成立，"最慢的那个落在 256 之前"也只是表象。测试改为 100 Hz 打 6 秒，三个 OS 上都至少有 300 个请求。 +- **C4（W2）的真正阻碍不是 mcpp 版本**：换成 2026.10.11.1 之后 prepare 能通过了，但 clangd 构建 MSVC 的 `std.ixx` + 失败（`: unknown type name 'suspend_always'`）。原因是 Clang 对 32 位 x86 MSVC 目标不定义 + `__cpp_impl_coroutine`（见 clang 的 `InitPreprocessor.cpp`："Support for coroutines on 32-bit x86 Microsoft + platforms is incomplete"），MSVC STL 的 `` 因此为空。也就是说，任何 Clang 都无法为 i686 构建 MSVC 的 + `std` 模块，mcppls 退回语义工具包是正确的做法。`fcb3450` 给这个 fixture 加的 `import std` 已撤掉；64 位 MSVC 的 + `std` 仍由 `mcpp-llvm-msvc`、`mcpp-split-msvc` 覆盖。这一限制已写进 docs/20-projects.md。 +- **CI 权限**：本仓库默认的 token 只有 contents/packages 读权限。`zed-smoke.yml` 声明 `actions: read` 会让 + `ci.yml` 一启动就失败（startup_failure），所以改为继承调用方的权限，只有 nightly 授予 `actions: read`。 +- **M6 未实施**：clangd 的 `$/memoryUsage` 要经过 mcppls 转发，涉及协议面。U13 的效果改由同机 A/B 直接衡量 + （`ux-baseline`、`measure compare`）。 +- **同机 A/B（nightly `39889041291`，payload 来自 PR CI `39887785592`，`ux-repeat=4`，`ux-baseline=v0.0.16`）**： + - U15 最忙一分钟的状态变化：ux-mcpp 第 2 轮 12 → 7.5，第 3 轮 12 → 6（W4 的目标达成）。 + - U5/U16/U7a 在噪声范围内。U6 补全 **max** 两轮都偏高：0.73 / 0.88 s，对比 0.58 / 0.64 s；p95 不变 + （0.46 / 0.53 s，对比 0.48 / 0.52 s）。按 §5 的 2σ 规则，第 2 轮通过、第 3 轮不通过。慢样本固定落在 U6 脚本中输入 + import 行的前后，并且都是 1000 项的补全，耗时在 clangd 内（mcppls 开销 1–3 ms）。猜测是 W4 推迟的 reload + 落到了大补全上，未证实：本地 3×3 二分（新版 / W4 关闭 / 0.0.16，`taskset -c 0-3`）去掉前 10 s 后 max 的中位数为 + 0.62 / 0.61 / 0.59 s，没有复现差异。 + - ux-xlings 第 1 轮 U13：+57.9%（687 → 1085 MB），空闲 CPU 0.0006 核（0.0.16 约 0.006）。ux-mcpp 第 1 轮 U13 −24.6% + （窗口内有 trim）。与 §1.4 的两组现象相同：E1 去掉了空闲时的 memo 序列化（CPU 降一个数量级），但 RSS 增长 + 不是它造成的。本地 cold → long 同一 payload 为 +0.0%，复现不了；空闲窗口的 server 日志在 CI 上已被轮转掉。 + E1 验收（RSS 增长 < 5%）**未达成**，记为已知问题，0.0.18 用 M6（`$/memoryUsage`）取证。 + - 同一轮 edits-2 的 U5 "diagnostics after typing" 6.54 s（预算 6.0），基线在相同位置是 6.38 s，判为机器问题。 + - 基线的 U8-config 和 U13 都等满了 900 s：0.0.16 不报告后台工作状态，runner 一直等到超时。`e9482fa` 改为 + 看到这种情况立即判定，结论不变。 + - 以上结果已带给你决定（§6.3），你选择按现状发布；U13 和 U6 max 写进了 CHANGELOG 的 Known issues。 +- **xmake 增修（#68、#69）**：按 [2026-10-24-0.0.17-xmake-sandbox-design.md](2026-10-24-0.0.17-xmake-sandbox-design.md) 并入本 PR，只改 mcppls。 + - XS-1：私有运行加 `XMAKE_TMPDIR`； + - XS-2：私有状态按 key 丢弃重建； + - XS-3：xmake 报状态过期时重建一次； + - XS-4：reset 清构建工具的私有状态。 + + 三个新 fixture 在 0.0.16 上失败、新版通过。xmake 侧的问题登记在 #24 的 UP-X1、UP-X2。 +- **完整 nightly（`40009683587`，`e278a58`，`scope=all`，复用 PR CI `40002265045` 的 payload）全部通过**： + - ux 8 轮全部通过：U15 最忙一分钟 ux-mcpp 7、ux-xlings 2；U13 ux-xlings −8.5%、ux-mcpp −22.2%； + - termux、Zed（默认、推荐设置）、self-hosting 4 项、三个平台的 startup timing、最新版 mcpp 的 fixture 都通过。 + + 这次 U13 能过，是因为窗口内发生了 trim；分组现象没有变，所以 Known issues 保持不变。 diff --git a/.github/actions/payload/action.yml b/.github/actions/payload/action.yml new file mode 100644 index 00000000..62fb0364 --- /dev/null +++ b/.github/actions/payload/action.yml @@ -0,0 +1,28 @@ +name: The payload +description: >- + Download a platform's payload artifact and unpack it (payload/ and stage/): the one this run built, + or the one another run of this repository built (nightly's `payload-run`, which tests a pull + request's own bytes instead of building them again). +inputs: + platform: + description: The payload's platform (linux-x64, linux-arm64, darwin-arm64, win32-x64) + required: true + run-id: + description: The run that built it; empty for this run + default: '' +runs: + using: composite + steps: + - if: inputs.run-id == '' + uses: actions/download-artifact@v8 + with: + name: payload-${{ inputs.platform }} + - if: inputs.run-id != '' + uses: actions/download-artifact@v8 + with: + name: payload-${{ inputs.platform }} + run-id: ${{ inputs.run-id }} + github-token: ${{ github.token }} + - name: Unpack the payload + shell: bash + run: tar -xzf payload-${{ inputs.platform }}.tar.gz diff --git a/.github/versions.env b/.github/versions.env index df90e888..0a4eb6c3 100644 --- a/.github/versions.env +++ b/.github/versions.env @@ -7,9 +7,10 @@ # 2026.9.30.2: the first mcpp that reads an exact `=x.y.z` dependency pin, which the openkal set # needs (see the root mcpp.toml: 0.15.4/0.20.1 float past the vendored openkal-linux). MCPP_VERSION=2026.9.30.2 -# The mcpp the mcpp-build-programs fixture describes its project with, installed beside MCPP_VERSION, which stays the -# default: the first whose build database describes build programs (build.mcpp, mcpp-community/mcpp#797). -MCPP_BUILD_PROGRAMS_VERSION=2026.10.11.1 +# The mcpp a fixture that declares "requires-mcpp" runs with (MCPPLS_REAL_MCPP), installed beside MCPP_VERSION, which +# stays the default: the first whose build database describes build programs (build.mcpp, mcpp-community/mcpp#797), +# and later than the one that quotes a Windows argument ending in a backslash (2026.10.5.2, mcpp-target-windows-real). +MCPP_FIXTURE_VERSION=2026.10.11.1 LLVM_VERSION=22.1.8 XLINGS_VERSION=v2026.8.17.2 # The xmake the xmake conformance fixtures run (installed from xlings; 0.0.8 part 2 X-7). diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index ddb6491b..cb0db5aa 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -245,14 +245,20 @@ jobs: chmod +x host-devtools cross/* # A candidate engine (docs/92-release.md, "A new engine"): packaging/engine-candidate.json names # a mcppls-clangd PR's CI run, whose packaged part replaces the lock's clangd, so the two are - # tested together before the engine is released. Only a pull request (or a manual CI run) may. + # tested together before the engine is released. Only a pull request, or a manual CI or nightly + # run of a branch other than main, may: the nightly's ux rounds are where an engine change to + # idle memory or typing latency shows, and they have to see the engine before it is released. - name: Candidate engine id: candidate run: | set -euo pipefail file=packaging/engine-candidate.json [ -f "$file" ] || exit 0 - if [ '${{ github.event_name }}' != pull_request ] && { [ '${{ github.event_name }}' != workflow_dispatch ] || [ '${{ github.workflow }}' != CI ]; }; then + manual=no + if [ '${{ github.event_name }}' = workflow_dispatch ] && [ '${{ github.ref }}' != refs/heads/main ]; then + case '${{ github.workflow }}' in CI|nightly) manual=yes ;; esac + fi + if [ '${{ github.event_name }}' != pull_request ] && [ "$manual" = no ]; then echo "::error::$file names an unreleased engine; it is for a pull request's CI, not ${{ github.workflow }} (${{ github.event_name }}). Release the engine, switch packaging/payload.lock.json to it and remove the file." exit 1 fi @@ -321,30 +327,30 @@ jobs: - platform: linux-x64 part: 1 of 2 os: ubuntu-24.04 - fixtures: mcpp-split mcpp-partition-definition mcpp-all-cppm verify-changes mingw mcpp-split-gcc mcpp-watch multi-root mcpp-llvm mcpp-build-programs mcpp-watch@polling mcpp-emit mcpp-rules-generated s1-two-sets compdb-lto-msvc failure-at-base@vscode generated-module-negotiated typing-import typing-import-spin workaround-canaries typing-autosave module-edit-autosave partial-scan-standins compdb-rejected-command compdb-mixed-standards mcpp-emit-partial mcpp-emit-partial-download compdb-midwrite xmake-basic xmake-late-config xmake-user-mode xmake-no-standard xmake-needs-download ux-modules-qt mcpp-target-selection mcpp-target-explicit mcpp-target-project mcpp-target-invalid mcpp-target-windows mcpp-producer-identity mcpp-target-race mcpp-target-live mcpp-target-mismatch mcpp-target-mismatch-cold mcpp-target-build-tool-off mcpp-target-provider-excluded mcpp-target-discovery-off mcpp-target-auto-fallback mcpp-producer-file-identity mcpp-producer-relative-path mcpp-target-selection@polling mcpp-producer-identity@polling mcpp-polling-watch-inputs@polling mcpp-target-fallback-mismatch mcpp-target-fallback-preserve + fixtures: mcpp-split mcpp-partition-definition mcpp-all-cppm verify-changes mingw mcpp-split-gcc mcpp-watch multi-root mcpp-llvm mcpp-build-programs mcpp-watch@polling mcpp-emit mcpp-rules-generated s1-two-sets compdb-lto-msvc failure-at-base@vscode generated-module-negotiated typing-import typing-import-spin workaround-canaries typing-autosave module-edit-autosave partial-scan-standins compdb-rejected-command compdb-mixed-standards mcpp-emit-partial mcpp-emit-partial-download compdb-midwrite xmake-basic xmake-late-config xmake-user-mode xmake-no-standard xmake-needs-download xmake-shared-tmp xmake-target-then-mode xmake-stale-state ux-modules-qt mcpp-target-selection mcpp-target-explicit mcpp-target-project mcpp-target-invalid mcpp-target-windows mcpp-producer-identity mcpp-target-race mcpp-target-live mcpp-target-mismatch mcpp-target-mismatch-cold mcpp-target-build-tool-off mcpp-target-provider-excluded mcpp-target-discovery-off mcpp-target-auto-fallback mcpp-producer-file-identity mcpp-producer-relative-path mcpp-target-selection@polling mcpp-producer-identity@polling mcpp-polling-watch-inputs@polling mcpp-target-fallback-mismatch mcpp-target-fallback-preserve mcpp-watched-source-structure mcpp-autosave-import-typing - platform: linux-x64 part: 2 of 2 os: ubuntu-24.04 extras: true - fixtures: inferred inferred-bom typing-pause module-fails-in-import save-plan build-discovery-off engine-none module-faults untrusted mcpp-gcc mcpp-gcc@plain mcpp-emit-package-std mcpp-emit-broken mcpp-emit-unavailable mcpp-emit-hang mcpp-emit-needs-download mcpp-emit-provisioned mcpp-emit-watch mcpp-emit-watch@polling mcpp-emit-edits mcpp-build-programs-cross cmake-clang cmake-clang-bdb cmake-fetchcontent-offline watch-polling payload-corrupt clangd-cannot-load generated-module generated-module@vscode generated-module-old-mcpp generated-module-old-mcpp@neovim module-faults@zed failure-at-base failure-at-base@neovim completion-keywords diagnostic-bundle clangd-crash-context mcpp-emit-wait provisional-no-prime inferred-cxx26 reset-cache cache-budget include-cleaner-modules no-modules tidy-const-views format-explicit preparation-after-idle build-program-broken build-program-gated build-program-manual + fixtures: inferred inferred-bom typing-pause module-fails-in-import save-plan build-discovery-off engine-none module-faults untrusted mcpp-gcc mcpp-gcc@plain mcpp-emit-package-std mcpp-emit-broken mcpp-emit-unavailable mcpp-emit-hang mcpp-emit-needs-download mcpp-emit-provisioned mcpp-emit-watch mcpp-emit-watch@polling mcpp-emit-edits mcpp-build-programs-cross cmake-clang cmake-clang-bdb cmake-fetchcontent-offline watch-polling payload-corrupt clangd-cannot-load generated-module generated-module@vscode generated-module-old-mcpp generated-module-old-mcpp@neovim module-faults@zed failure-at-base failure-at-base@neovim completion-keywords diagnostic-bundle clangd-crash-context clangd-exit-strike clangd-crash-closure mcpp-emit-wait provisional-no-prime inferred-cxx26 reset-cache cache-budget include-cleaner-modules no-modules tidy-const-views format-explicit preparation-after-idle build-program-broken build-program-gated build-program-manual # generated-module{,-old-mcpp,-negotiated}'s mcpp-mock.json bakes in a POSIX driver path # (${env:HOME}/.mcpp/registry/..., no {exe}); it resolves the same way here as on Linux, so # these run on macOS but are left off win32-x64 below rather than fixed unverified. - platform: darwin-arm64 os: macos-14 extras: true - fixtures: inferred inferred-bom typing-pause module-fails-in-import save-plan build-discovery-off engine-none module-faults untrusted mcpp-llvm mcpp-build-programs mcpp-split mcpp-partition-definition mcpp-all-cppm verify-changes mcpp-watch multi-root failure-at-base clangd-cannot-load failure-at-base@zed generated-module generated-module-old-mcpp generated-module-negotiated typing-import typing-import-spin workaround-canaries completion-keywords diagnostic-bundle typing-autosave module-edit-autosave partial-scan-standins clangd-crash-context mcpp-emit-wait provisional-no-prime inferred-cxx26 reset-cache cache-budget include-cleaner-modules no-modules tidy-const-views format-explicit build-program-broken build-program-gated build-program-manual mcpp-target-selection mcpp-target-explicit mcpp-target-project mcpp-target-invalid mcpp-target-windows mcpp-producer-identity mcpp-target-race mcpp-target-live mcpp-target-mismatch mcpp-target-mismatch-cold mcpp-target-build-tool-off mcpp-target-provider-excluded mcpp-target-discovery-off mcpp-target-auto-fallback mcpp-producer-file-identity mcpp-producer-relative-path mcpp-polling-watch-inputs@polling mcpp-target-fallback-mismatch mcpp-target-fallback-preserve + fixtures: inferred inferred-bom typing-pause module-fails-in-import save-plan build-discovery-off engine-none module-faults untrusted mcpp-llvm mcpp-build-programs mcpp-split mcpp-partition-definition mcpp-all-cppm verify-changes mcpp-watch multi-root failure-at-base clangd-cannot-load failure-at-base@zed generated-module generated-module-old-mcpp generated-module-negotiated typing-import typing-import-spin workaround-canaries completion-keywords diagnostic-bundle typing-autosave module-edit-autosave partial-scan-standins clangd-crash-context clangd-exit-strike clangd-crash-closure mcpp-emit-wait provisional-no-prime inferred-cxx26 reset-cache cache-budget include-cleaner-modules no-modules tidy-const-views format-explicit build-program-broken build-program-gated build-program-manual mcpp-target-selection mcpp-target-explicit mcpp-target-project mcpp-target-invalid mcpp-target-windows mcpp-producer-identity mcpp-target-race mcpp-target-live mcpp-target-mismatch mcpp-target-mismatch-cold mcpp-target-build-tool-off mcpp-target-provider-excluded mcpp-target-discovery-off mcpp-target-auto-fallback mcpp-producer-file-identity mcpp-producer-relative-path mcpp-polling-watch-inputs@polling mcpp-target-fallback-mismatch mcpp-target-fallback-preserve mcpp-watched-source-structure mcpp-autosave-import-typing # No mcpp on the arm64 runner: the server and mock producers are cross-built. Most # fixtures use the semantic kit; the SDK-free target fixtures probe distribution clang, # installed below. Module faults, a corrupt payload and polling are included. - platform: linux-arm64 os: ubuntu-24.04-arm cross-tools: true - fixtures: inferred inferred-bom typing-pause module-fails-in-import build-discovery-off engine-none module-faults untrusted payload-corrupt clangd-cannot-load failure-at-base watch-polling typing-import typing-import-spin workaround-canaries completion-keywords diagnostic-bundle typing-autosave module-edit-autosave partial-scan-standins clangd-crash-context inferred-cxx26 reset-cache cache-budget include-cleaner-modules no-modules tidy-const-views mcpp-target-selection mcpp-target-explicit mcpp-target-project mcpp-target-invalid mcpp-target-windows mcpp-producer-identity mcpp-target-race mcpp-target-live mcpp-target-mismatch mcpp-target-mismatch-cold mcpp-target-build-tool-off mcpp-target-provider-excluded mcpp-target-discovery-off mcpp-target-auto-fallback mcpp-producer-file-identity mcpp-producer-relative-path mcpp-polling-watch-inputs@polling mcpp-target-fallback-mismatch mcpp-target-fallback-preserve + fixtures: inferred inferred-bom typing-pause module-fails-in-import build-discovery-off engine-none module-faults untrusted payload-corrupt clangd-cannot-load failure-at-base watch-polling typing-import typing-import-spin workaround-canaries completion-keywords diagnostic-bundle typing-autosave module-edit-autosave partial-scan-standins clangd-crash-context clangd-exit-strike clangd-crash-closure inferred-cxx26 reset-cache cache-budget include-cleaner-modules no-modules tidy-const-views mcpp-target-selection mcpp-target-explicit mcpp-target-project mcpp-target-invalid mcpp-target-windows mcpp-producer-identity mcpp-target-race mcpp-target-live mcpp-target-mismatch mcpp-target-mismatch-cold mcpp-target-build-tool-off mcpp-target-provider-excluded mcpp-target-discovery-off mcpp-target-auto-fallback mcpp-producer-file-identity mcpp-producer-relative-path mcpp-polling-watch-inputs@polling mcpp-target-fallback-mismatch mcpp-target-fallback-preserve mcpp-watched-source-structure mcpp-autosave-import-typing - platform: win32-x64 part: 1 of 2 os: windows-2022 - fixtures: mcpp-split mcpp-partition-definition mcpp-all-cppm verify-changes mcpp-split-msvc cmake-msvc-std compdb-clangxx-msvc-std multi-root compdb-clang-cl-std mcpp-llvm-msvc mcpp-build-programs failure-at-base@vscode typing-import typing-import-spin workaround-canaries typing-autosave module-edit-autosave partial-scan-standins ux-modules-windows mcpp-target-selection mcpp-target-explicit mcpp-target-project mcpp-target-invalid mcpp-target-windows mcpp-producer-identity mcpp-target-race mcpp-target-live mcpp-target-mismatch mcpp-target-mismatch-cold mcpp-target-build-tool-off mcpp-target-provider-excluded mcpp-target-discovery-off mcpp-target-auto-fallback mcpp-producer-file-identity mcpp-producer-relative-path mcpp-polling-watch-inputs@polling mcpp-target-fallback-mismatch mcpp-target-fallback-preserve + fixtures: mcpp-split mcpp-partition-definition mcpp-all-cppm verify-changes mcpp-split-msvc cmake-msvc-std compdb-clangxx-msvc-std multi-root compdb-clang-cl-std mcpp-llvm-msvc mcpp-build-programs failure-at-base@vscode typing-import typing-import-spin workaround-canaries typing-autosave module-edit-autosave partial-scan-standins ux-modules-windows mcpp-target-selection mcpp-target-explicit mcpp-target-project mcpp-target-invalid mcpp-target-windows mcpp-target-windows-real mcpp-producer-identity mcpp-target-race mcpp-target-live mcpp-target-mismatch mcpp-target-mismatch-cold mcpp-target-build-tool-off mcpp-target-provider-excluded mcpp-target-discovery-off mcpp-target-auto-fallback mcpp-producer-file-identity mcpp-producer-relative-path mcpp-polling-watch-inputs@polling mcpp-target-fallback-mismatch mcpp-target-fallback-preserve mcpp-watched-source-structure mcpp-autosave-import-typing - platform: win32-x64 part: 2 of 2 os: windows-2022 @@ -382,13 +388,18 @@ jobs: done xmake --version | head -1 g++ --version | head -1 - # mcpp-community/mcpp#797: mcpp-build-programs needs an mcpp whose build database describes build programs. It is - # installed beside the pinned one, which stays the default, and named to the fixture by MCPPLS_BUILD_PROGRAMS_MCPP. - - name: The mcpp that describes build programs + # A fixture that needs a newer mcpp than the pinned one says so in its scenario.json ("requires-mcpp"; the runner + # checks the version and says what it found). That release is installed beside the pinned one, which stays the + # default, and named to the fixtures by MCPPLS_REAL_MCPP. + - name: The mcpp the fixtures that require one run with if: '!matrix.cross-tools' run: | set -euo pipefail - case " ${{ matrix.fixtures }} " in *" mcpp-build-programs "*) ;; *) exit 0 ;; esac + wanted=no + for fixture in ${{ matrix.fixtures }}; do + grep -q '"requires-mcpp"' "conformance/fixtures/${fixture%@*}/scenario.json" 2>/dev/null && wanted=yes + done + [ "$wanted" = yes ] || exit 0 # From mcpp's own release, its checksum checked: the index snapshot the pinned xlings syncs can lag a new mcpp # by days (2026.10.11.1 was in xim-pkgindex, not yet in the snapshot), and the pinned mcpp stays the default. case "$RUNNER_OS-$(uname -m)" in @@ -398,8 +409,8 @@ jobs: Windows-*) platform=windows-x86_64; ext=zip ;; *) echo "::error::no mcpp release for $RUNNER_OS-$(uname -m)"; exit 1 ;; esac - archive="mcpp-$MCPP_BUILD_PROGRAMS_VERSION-$platform.$ext" - base="https://github.com/mcpp-community/mcpp/releases/download/v$MCPP_BUILD_PROGRAMS_VERSION" + archive="mcpp-$MCPP_FIXTURE_VERSION-$platform.$ext" + base="https://github.com/mcpp-community/mcpp/releases/download/v$MCPP_FIXTURE_VERSION" temp="$RUNNER_TEMP"; python=python3 if [ "$RUNNER_OS" = Windows ]; then temp=$(cygpath -u "$RUNNER_TEMP"); python=python; fi curl -fsSL --retry 5 --retry-all-errors --retry-delay 5 -o "$temp/$archive" "$base/$archive" @@ -407,15 +418,15 @@ jobs: expected=$(cut -d' ' -f1 "$temp/$archive.sha256") actual=$("$python" -c "import hashlib,sys; print(hashlib.sha256(open(sys.argv[1],'rb').read()).hexdigest())" "$temp/$archive") [ "$expected" = "$actual" ] || { echo "::error::$archive: sha256 $actual, the release says $expected"; exit 1; } - rm -rf "$temp/mcpp-build-programs" && mkdir -p "$temp/mcpp-build-programs" - "$python" -c "import shutil,sys; shutil.unpack_archive(sys.argv[1], sys.argv[2])" "$temp/$archive" "$temp/mcpp-build-programs" + rm -rf "$temp/mcpp-fixture" && mkdir -p "$temp/mcpp-fixture" + "$python" -c "import shutil,sys; shutil.unpack_archive(sys.argv[1], sys.argv[2])" "$temp/$archive" "$temp/mcpp-fixture" exe=''; [ "$RUNNER_OS" = Windows ] && exe='.exe' - mcpp="$temp/mcpp-build-programs/mcpp-$MCPP_BUILD_PROGRAMS_VERSION-$platform/bin/mcpp$exe" + mcpp="$temp/mcpp-fixture/mcpp-$MCPP_FIXTURE_VERSION-$platform/bin/mcpp$exe" chmod +x "$mcpp" 2>/dev/null || true "$mcpp" --version mcpp --version if [ "$RUNNER_OS" = Windows ]; then mcpp=$(cygpath -m "$mcpp"); fi - echo "MCPPLS_BUILD_PROGRAMS_MCPP=$mcpp" >> "$GITHUB_ENV" + echo "MCPPLS_REAL_MCPP=$mcpp" >> "$GITHUB_ENV" # usable plan W3, closing the loop with mcpp itself: what `mcpp emit build-database` prints for # each of this host's mcpp fixtures, against the S1 and S2 schemas, S1's semantic rules and the # contract of mcpp-community/mcpp#636 (docs/specs/tools/validate.py). The fixtures below then have the @@ -439,7 +450,7 @@ jobs: [ -f "$source/mcpp.toml" ] && [ ! -f "$source/mcpp-mock.json" ] || continue [ -e "$temp/mcpp-emit/$fixture.json" ] && continue mkdir -p "$temp/mcpp-emit" && cp -R "$source" "$temp/mcpp-emit/$fixture" - emit=mcpp; [ "$fixture" = mcpp-build-programs ] && emit="$MCPPLS_BUILD_PROGRAMS_MCPP" + emit=mcpp; grep -q '"requires-mcpp"' "$source/scenario.json" && emit="${MCPPLS_REAL_MCPP:-mcpp}" (cd "$temp/mcpp-emit/$fixture" && "$emit" emit build-database --format json > "$temp/mcpp-emit/$fixture.json") envelopes+=("$temp/mcpp-emit/$fixture.json") done @@ -534,6 +545,12 @@ jobs: case "$fixture" in cmake-clang-bdb) path_prefix="$RUNNER_TEMP/cmake-4.4.2/bin:" ;; cmake-msvc-bdb|cmake-clang-cl) fixture_extra=(--msvc-env "$RUNNER_TEMP/msvc-bdb.env") ;; + # Preserve the real producer's first CDBs and server log independently + # of the diagnostic rerun: no reconstructed or rewritten target entries. + mcpp-target-windows-real) + local evidence="$RUNNER_TEMP/mcpp-target-windows-real-first" + case " $* " in *" --verbose "*) evidence="$RUNNER_TEMP/mcpp-target-windows-real-diagnostic" ;; esac + fixture_extra+=(--workspace-dir "$evidence/workspace" --cache-dir "$evidence/cache") ;; # usable plan W9.3: the fallback runs only when the client never declares # didChangeWatchedFiles.dynamicRegistration. NAME@polling runs fixture NAME that way. watch-polling|*@polling) fixture_extra+=(--no-dynamic-watch) ;; @@ -571,6 +588,15 @@ jobs: exit 1 fi + # Preserve the first producer envelopes even when schema validation stops + # the job before any LSP fixture starts. + - uses: actions/upload-artifact@v7 + if: always() && !matrix.cross-tools + with: + name: producer-envelopes-${{ matrix.platform }}-${{ matrix.part || 'full' }} + path: ${{ runner.temp }}/mcpp-emit/*.json + if-no-files-found: ignore + - uses: actions/upload-artifact@v7 if: always() with: @@ -578,6 +604,17 @@ jobs: path: ${{ runner.temp }}/conformance-measures/*.json if-no-files-found: ignore + - uses: actions/upload-artifact@v7 + if: always() && contains(matrix.fixtures, 'mcpp-target-windows-real') + with: + name: real-mcpp-target-cdb-${{ matrix.platform }} + path: | + ${{ runner.temp }}/mcpp-target-windows-real-*/workspace/mcpp-target-windows-real/recorded-*.json + ${{ runner.temp }}/mcpp-target-windows-real-*/workspace/mcpp-target-windows-real/compile_commands.json + ${{ runner.temp }}/mcpp-target-windows-real-*/cache/logs/*.log + ${{ runner.temp }}/mcpp-target-windows-real-*/cache/workspaces/**/model.mcpp.json + if-no-files-found: ignore + # Issue #23 fix plan F18: the bundles diagnostic-bundle exported on this runner's own account, whose # name is in every path of it (runneradmin and RUNNER~1 on Windows). The fixture already held each # to its manifest and found no user name and no home directory in it; here another program's zip diff --git a/.github/workflows/nightly.yml b/.github/workflows/nightly.yml index 3d7df733..b129612e 100644 --- a/.github/workflows/nightly.yml +++ b/.github/workflows/nightly.yml @@ -8,10 +8,29 @@ name: nightly on: schedule: - cron: '30 18 * * *' + # A manual run can test what a pull request's CI already built, and only the ux rounds, as many + # times as asked, beside a released server: a change is measured without being built again + # (docs/92-release.md, "Measuring a change"). workflow_dispatch: + inputs: + payload-run: + description: A CI run of this branch whose payloads to test (its run id); empty builds them here + default: '' + scope: + description: 'all: every job; ux: only the ux rounds (and their budgets and comparison)' + type: choice + options: [all, ux] + default: all + ux-repeat: + description: How many times each ux round runs its edits stage, each in a new server session + default: '1' + ux-baseline: + description: A release (v0.0.16) whose server runs the same stages beside this one, on the same machine + default: '' permissions: contents: read + actions: read # another run's payload artifacts (payload-run) env: # The build pins live in .github/versions.env and are exported by .github/actions/setup-mcpp, @@ -20,6 +39,7 @@ env: jobs: ci: + if: ${{ github.event_name != 'workflow_dispatch' || inputs.payload-run == '' }} uses: ./.github/workflows/ci.yml # The Zed extension in a real Zed (editors/zed/tests/smoke.sh), on the payload ci.yml just built. @@ -27,14 +47,16 @@ jobs: zed: name: real Zed needs: ci - if: ${{ !cancelled() }} + if: ${{ !cancelled() && inputs.scope != 'ux' }} uses: ./.github/workflows/zed-smoke.yml + with: + payload-run: ${{ inputs.payload-run }} timing: name: startup timing (${{ matrix.platform }}) needs: ci # Timings and self-hosting need only the payloads: a failure elsewhere in CI does not skip them. - if: ${{ !cancelled() }} + if: ${{ !cancelled() && inputs.scope != 'ux' }} runs-on: ${{ matrix.os }} timeout-minutes: 60 strategy: @@ -49,11 +71,10 @@ jobs: shell: bash steps: - uses: actions/checkout@v7 - - uses: actions/download-artifact@v8 + - uses: ./.github/actions/payload with: - name: payload-${{ matrix.platform }} - - name: Unpack the payload - run: tar -xzf payload-${{ matrix.platform }}.tar.gz + platform: ${{ matrix.platform }} + run-id: ${{ inputs.payload-run }} - name: Three cold and warm starts run: | set -euo pipefail @@ -80,7 +101,7 @@ jobs: mcpp-latest: name: mcpp fixtures with the latest mcpp (linux-x64) needs: ci - if: ${{ !cancelled() }} + if: ${{ !cancelled() && inputs.scope != 'ux' }} runs-on: ubuntu-24.04 timeout-minutes: 60 env: @@ -93,11 +114,10 @@ jobs: - uses: ./.github/actions/setup-mcpp - name: GCC 16, which mcpp-gcc builds with run: mcpp toolchain install gcc 16.1.0 - - uses: actions/download-artifact@v8 + - uses: ./.github/actions/payload with: - name: payload-linux-x64 - - name: Unpack the payload - run: tar -xzf payload-linux-x64.tar.gz + platform: linux-x64 + run-id: ${{ inputs.payload-run }} - name: The mcpp fixtures run: | set -euo pipefail @@ -129,7 +149,7 @@ jobs: termux: name: termux + proot-distro Debian (linux-arm64) needs: ci - if: ${{ !cancelled() }} + if: ${{ !cancelled() && inputs.scope != 'ux' }} continue-on-error: true runs-on: ubuntu-24.04-arm timeout-minutes: 60 @@ -138,15 +158,12 @@ jobs: shell: bash steps: - uses: actions/checkout@v7 - - uses: actions/download-artifact@v8 + - uses: ./.github/actions/payload with: - name: payload-linux-arm64 - - name: Unpack the payload - run: | - set -euo pipefail - tar -xzf payload-linux-arm64.tar.gz - # The container's user is not the runner's: what it reads has to be readable. - chmod -R a+rX payload stage conformance + platform: linux-arm64 + run-id: ${{ inputs.payload-run }} + # The container's user is not the runner's: what it reads has to be readable. + - run: chmod -R a+rX payload stage conformance # Runs in the Debian rootfs, as its root, with the checkout bound at /work. The same three # checks as the `proot` job, on a server started directly (there is no wrapper: the whole # session is already under proot). @@ -178,8 +195,12 @@ jobs: EOF cat > "$RUNNER_TEMP/in-termux.sh" <<'EOF' set -eu - yes | pkg update -y || true - pkg install -y proot-distro + # Use the official seed instead of pkg's randomly selected mirrors: + # https://github.com/termux/termux-packages/wiki/Mirrors + # A stale mirror previously advertised a proot-distro package it no longer served. + printf '%s\n' 'deb https://packages.termux.dev/apt/termux-main stable main' > "$PREFIX/etc/apt/sources.list" + apt-get update + apt-get install -y proot-distro proot-distro install debian proot-distro login debian --bind /mnt/work:/work --bind /mnt/scripts:/mnt/scripts -- bash -c 'test -r /mnt/scripts/in-debian.sh && exec bash /mnt/scripts/in-debian.sh' EOF @@ -197,7 +218,7 @@ jobs: self-hosting: name: self-hosting (${{ matrix.fixture }}, ${{ matrix.platform }}) needs: ci - if: ${{ !cancelled() }} + if: ${{ !cancelled() && inputs.scope != 'ux' }} runs-on: ${{ matrix.os }} timeout-minutes: 120 strategy: @@ -238,11 +259,10 @@ jobs: xlings install mcpp@2026.8.8.4 -y -g xlings use mcpp "$MCPP_VERSION" mcpp --version - - uses: actions/download-artifact@v8 + - uses: ./.github/actions/payload with: - name: payload-${{ matrix.platform }} - - name: Unpack the payload - run: tar -xzf payload-${{ matrix.platform }}.tar.gz + platform: ${{ matrix.platform }} + run-id: ${{ inputs.payload-run }} - name: The fixture, timed run: | set -euo pipefail @@ -295,37 +315,82 @@ jobs: xlings install mcpp@2026.9.21.1 -y -g xlings use mcpp "$MCPP_VERSION" mcpp --version - - uses: actions/download-artifact@v8 + - uses: ./.github/actions/payload with: - name: payload-linux-x64 - - name: Unpack the payload - run: tar -xzf payload-linux-x64.tar.gz + platform: linux-x64 + run-id: ${{ inputs.payload-run }} + # ux-baseline: the release's own server and stage tools, beside this run's; its own workspace + # and cache, since the two servers' caches are not each other's. + - name: The baseline release + if: inputs.ux-baseline != '' + env: + GH_TOKEN: ${{ github.token }} + run: | + set -euo pipefail + mkdir -p "$RUNNER_TEMP/baseline" + gh release download '${{ inputs.ux-baseline }}' --repo '${{ github.repository }}' --pattern payload-linux-x64.tar.gz --dir "$RUNNER_TEMP/baseline" + tar -xzf "$RUNNER_TEMP/baseline/payload-linux-x64.tar.gz" -C "$RUNNER_TEMP/baseline" + "$RUNNER_TEMP/baseline/payload/bin/mcppls" --version - name: The stages + env: + REPEAT: ${{ inputs.ux-repeat || '1' }} + BASELINE: ${{ inputs.ux-baseline }} run: | set -euo pipefail nproc cat /proc/loadavg + case "$REPEAT" in ''|*[!0-9]*|0) echo "::error::ux-repeat is a positive number, not '$REPEAT'"; exit 1 ;; esac results="$RUNNER_TEMP/ux-results" work="$RUNNER_TEMP/ux-work" mkdir -p "$results" "$work" # The two-core round pins the runner and, through it, the server and clangd it starts. pin=(); [ -n "${{ matrix.cores }}" ] && pin=(taskset -c "${{ matrix.cores }}") failed='' + # run_stage [runner arguments]: the + # baseline's server runs in a workspace and cache of its own, measured into a directory of its + # own (an artifact the budgets leave out); the runner and the fixture are this run's for both, + # so only the server differs. run_stage() { - local name="$1" cache="$2"; shift 2 - echo "::group::${{ matrix.fixture }} ${{ matrix.round }} $name" - if ! ${pin[@]+"${pin[@]}"} stage/mcppls-conformance run --server payload/bin/mcppls --payload payload \ - --fixture conformance/fixtures/${{ matrix.fixture }} --workspace-dir "$work/workspace" --cache-dir "$work/$cache" \ - --stage "$name" --timeout 900 --measure "$results/${{ matrix.fixture }}-${{ matrix.round }}-$name.json" "$@"; then - failed="$failed $name" + local server="$1" name="$2" suffix="$3" cache="$4"; shift 4 + local payload=payload into="$results" space="$work" label="$name$suffix" + if [ "$server" = baseline ]; then + payload="$RUNNER_TEMP/baseline/payload" into="$RUNNER_TEMP/ux-baseline-results" space="$work/baseline" label="$label (baseline $BASELINE)" + fi + mkdir -p "$into" "$space" + echo "::group::${{ matrix.fixture }} ${{ matrix.round }} $label" + if ! ${pin[@]+"${pin[@]}"} stage/mcppls-conformance run --server "$payload/bin/mcppls" --payload "$payload" \ + --fixture conformance/fixtures/${{ matrix.fixture }} --workspace-dir "$space/workspace" --cache-dir "$space/$cache" \ + --stage "$name" --timeout 900 --measure "$into/${{ matrix.fixture }}-${{ matrix.round }}-$name$suffix.json" "$@"; then + if [ "$server" = this ]; then failed="$failed $name$suffix"; else echo "::warning::the baseline failed $name$suffix"; fi + # Keep the original failed stage's complete server logs before later + # stages rotate them. This runs after the measured process exits. + if [ -d "$space/$cache/logs" ]; then + mkdir -p "$into/failed-$name$suffix/logs" + cp -a "$space/$cache/logs/." "$into/failed-$name$suffix/logs/" || + echo "::warning::could not preserve all $name$suffix server logs" + fi fi echo "::endgroup::" } - run_stage cold cache-a - run_stage warm cache-a --expect-warm - run_stage edits cache-a --expect-warm - run_stage faults cache-b - if [ "${{ matrix.round }}" = 1 ]; then run_stage long cache-a --expect-warm; fi + servers=(this); [ -n "$BASELINE" ] && servers=(this baseline) + for server in "${servers[@]}"; do + run_stage "$server" cold '' cache-a + run_stage "$server" warm '' cache-a --expect-warm + done + # The edits stage REPEAT times, a new server session each, on what cold and warm prepared: the + # typing and save scenarios (U5, U6, U7a, U15) sampled that many times without building again. + # Beside a baseline the two alternate A B B A, so that a slower stretch of the machine is + # shared between them. + for i in $(seq 1 "$REPEAT"); do + suffix=''; [ "$REPEAT" -gt 1 ] && suffix="-$i" + order=("${servers[@]}") + if [ -n "$BASELINE" ] && [ $((i % 2)) -eq 0 ]; then order=(baseline this); fi + for server in "${order[@]}"; do run_stage "$server" edits "$suffix" cache-a --expect-warm; done + done + run_stage this faults '' cache-b + if [ "${{ matrix.round }}" = 1 ]; then + for server in "${servers[@]}"; do run_stage "$server" long '' cache-a --expect-warm; done + fi { echo "### ${{ matrix.fixture }}, round ${{ matrix.round }}" echo @@ -344,6 +409,11 @@ jobs: with: name: nightly-ux-${{ matrix.fixture }}-${{ matrix.round }} path: ${{ runner.temp }}/ux-results + - uses: actions/upload-artifact@v7 + if: always() && inputs.ux-baseline != '' + with: + name: nightly-ux-${{ matrix.fixture }}-${{ matrix.round }}-baseline + path: ${{ runner.temp }}/ux-baseline-results # U-* (0.0.8 plan §3.4): every scenario's numbers over the night's rounds, for setting each budget from what was # measured (1.3 x the 95th percentile over two weeks of nights) instead of widening it after a slow runner. The @@ -355,11 +425,11 @@ jobs: runs-on: ubuntu-24.04 timeout-minutes: 20 steps: - - uses: actions/download-artifact@v8 + - uses: actions/checkout@v7 + - uses: ./.github/actions/payload with: - name: payload-linux-x64 - - name: Unpack the payload - run: tar -xzf payload-linux-x64.tar.gz + platform: linux-x64 + run-id: ${{ inputs.payload-run }} - uses: actions/download-artifact@v8 with: pattern: nightly-ux-ux-*-[123] @@ -369,7 +439,19 @@ jobs: set -euo pipefail stage/mcppls-devtools measure budgets ux-rounds | tee ux-budgets.md { echo '### ux budgets over the night'"'"'s rounds'; echo; cat ux-budgets.md; } >> "$GITHUB_STEP_SUMMARY" + # ux-baseline: each scenario's numbers beside the release's, from the same machines and stages. + - uses: actions/download-artifact@v8 + if: inputs.ux-baseline != '' + with: + pattern: nightly-ux-ux-*-[123]-baseline + path: ux-baseline + - name: Beside the baseline + if: inputs.ux-baseline != '' + run: | + set -euo pipefail + stage/mcppls-devtools measure compare ux-rounds ux-baseline | tee ux-compare.md + { echo '### ux beside ${{ inputs.ux-baseline }}'; echo; cat ux-compare.md; } >> "$GITHUB_STEP_SUMMARY" - uses: actions/upload-artifact@v7 with: name: nightly-ux-budgets - path: ux-budgets.md + path: ux-*.md diff --git a/.github/workflows/zed-smoke.yml b/.github/workflows/zed-smoke.yml index 1ebb827f..8233466a 100644 --- a/.github/workflows/zed-smoke.yml +++ b/.github/workflows/zed-smoke.yml @@ -12,13 +12,18 @@ name: Zed smoke # nightly.yml every night, after ci.yml # release-checks every release candidate (and a pull request that changes release packaging) # -# It needs the `payload-linux-x64` artifact of the calling run. +# It needs the `payload-linux-x64` artifact of the calling run, or of the run `payload-run` names. on: workflow_call: + inputs: + payload-run: + description: The run whose payload-linux-x64 to test; empty for the calling run + type: string + default: '' -permissions: - contents: read +# No permissions of its own: it has the caller's. CI's default (contents) is all it needs for its own run's +# payload; nightly's `payload-run` reads another run's artifacts, and nightly grants `actions: read` for it. jobs: zed: @@ -38,11 +43,10 @@ jobs: case "$name" in ''|\#*) continue ;; esac echo "$name=$value" >> "$GITHUB_ENV" done < .github/versions.env - - uses: actions/download-artifact@v8 + - uses: ./.github/actions/payload with: - name: payload-linux-x64 - - name: Unpack the payload - run: tar -xzf payload-linux-x64.tar.gz + platform: linux-x64 + run-id: ${{ inputs.payload-run }} # Zed needs a display and a Vulkan driver: xvfb for the first, lavapipe (software Vulkan) # for the second. The libraries are what a desktop has and a runner image may not. - name: Display, Vulkan and the libraries Zed loads @@ -50,7 +54,11 @@ jobs: set -euo pipefail sudo apt-get update sudo apt-get install -y --no-install-recommends xvfb mesa-vulkan-drivers libvulkan1 libegl1 libgl1 libgbm1 \ - libxkbcommon0 libxkbcommon-x11-0 libwayland-client0 libx11-xcb1 libxcb1 libfontconfig1 libasound2t64 + libxkbcommon0 libxkbcommon-x11-0 libwayland-client0 libx11-xcb1 libxcb1 libfontconfig1 libasound2t64 clangd + # Zed's default lookup tries PATH before downloading from GitHub. Keep its default + # server selection and require the independently started process, without testing downloads. + command -v clangd + clangd --version - name: Zed ${{ env.ZED_VERSION }} id: zed-cache uses: actions/cache@v6 @@ -81,7 +89,7 @@ jobs: MCPPLS_PAYLOAD: ${{ github.workspace }}/payload MCPPLS_DEVTOOLS: ${{ github.workspace }}/stage/mcppls-devtools ZED_SMOKE_DIR: ${{ runner.temp }}/zs - # Zed downloads its clangd from GitHub on first use; here that has to work. + # The preinstalled PATH clangd must be started by Zed beside mcppls. ZED_SMOKE_REQUIRE_CLANGD: '1' run: xvfb-run -a -s '-screen 0 1280x1024x24' editors/zed/tests/smoke.sh ${{ matrix.mode }} - uses: actions/upload-artifact@v7 @@ -92,6 +100,7 @@ jobs: ${{ runner.temp }}/zs/*.log ${{ runner.temp }}/zs/zed.out ${{ runner.temp }}/zs/cache/logs + ${{ runner.temp }}/zs/cache/bundles ${{ runner.temp }}/zs/home/.local/share/zed/logs if-no-files-found: ignore retention-days: 7 diff --git a/CHANGELOG.md b/CHANGELOG.md index 7b9b109e..86bc2455 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,10 +7,57 @@ release's notes are that section. Versions are three-part semantic versions, `MAJOR.MINOR.PATCH`, and every editor plugin carries the product version unchanged. -## [0.0.17] — 2026-10-22 +## [0.0.17] — 2026-10-24 -The bundled engine remains [mcppls-clangd 0.0.16](https://github.com/Sunrisepeak/mcppls-clangd/releases/tag/v0.0.16) -(`23.1.0-mcppls.4`). +The bundled engine is [mcppls-clangd 0.0.17](https://github.com/Sunrisepeak/mcppls-clangd/releases/tag/v0.0.17) +(`23.1.0-mcppls.5`), released together with this version. This release is about following an mcpp build target, +what an idle or a crashed clangd costs you, and typing imports with autosave on. + +### The engine + +- **An idle clangd does next to nothing.** The engine kept the memos of a project's module checks on disk by building + the JSON of all of them every 30 seconds, and only then finding nothing had changed. It now writes only when a memo + changed (engine patch 0036): over ten idle minutes clangd uses about a tenth of the CPU it did. Its memory can still + grow while idle; see Known issues. +- The engine's module workers work in sanitizer builds again (0037), which is how the engine's own nightly tests the + module paths for memory errors; and its releases publish the unstripped clangd for Linux and macOS beside the + stripped one, so a crash's stack can be named. + +### Typing and saving + +- **Typing an import with autosave on no longer asks the build tool at every pause.** A save that changes a file's + imports asks for the project model again; while that file is still being typed in, the request now waits until it + has been quiet for two seconds (never more than thirty), so a typed import is one run of mcpp, not one per save, + and the status no longer turns `loading` and back every couple of seconds (#59 U15). + +### When clangd exits + +- **A clangd that is killed (or stops with no crash report) no longer takes the file you are typing in away from it + for two minutes.** An exit that names no file is a strike against the files clangd was working on, not a verdict: + only a second such exit on the same file within ten minutes sets it aside (#59 U9). +- **A file clangd crashed on goes back to it once the modules it imports are built again**, instead of after its + whole two-minute term: what it crashed on may have been one of those modules, half rebuilt. It goes back once for + each such build; another crash sets it aside for a doubled term, as before (#59 U7a). +- A slow completion (two seconds and more) is recorded with where its time went -- waiting before the engine was + asked, the engine, and what came after -- in the report's events and in diagnostic bundles. + +### xmake + +- **mcppls no longer breaks your own `xmake` build.** With GCC, xmake keeps each unit's module mapper file in its + temporary directory under a name only the source's relative path decides, and your builds read it. mcppls's + private run rewrote those files to name its own build directory, where nothing is built, and your next + incremental build failed with `failed to read compiled module` until you ran `xmake` again (#69). Its xmake now + runs with a temporary directory of its own (`XMAKE_TMPDIR`), beside the configuration and build directories it + already had. +- **A project whose xmake description failed once no longer stays inferred, without its packages' include paths, + in every later session** (#68). xmake's module scan cache is updated only by a build; mcppls describes without + building and configured its state again in place, so after a target was added (or, with xmake 3.1.1, after you + switched mode) every `xmake project -k compile_commands` failed with `dependfile ... not found`. The private + state is now a cache: made again from nothing whenever xmake itself, any `xmake.lua` or your `xmake.conf` + changes, and once more when xmake says it is out of date. Resetting the workspace cache removes the private + states of build tools too (xmake, CMake, Meson), so a reset is a fresh start. The first description after the + upgrade takes a second or two longer. +- Both are xmake's own behaviour as well, recorded with reproductions in #24 (UP-X1, UP-X2). ### Build targets @@ -38,6 +85,22 @@ The bundled engine remains [mcppls-clangd 0.0.16](https://github.com/Sunrisepeak ### Reliability and measurements +- A successful producer description can acknowledge a watched source's lexical imports even + when preprocessing excludes some of them from the active graph. Body-only saves then avoid + repeatedly describing the same structure; changed directives, failures and in-flight edits + retain conservative rediscovery. +- An existing source file named as an LSP workspace folder is ignored in favor of the client's + valid directory roots. Zed file worktrees no longer make clangd start with a source file as + its working directory. +- Child-connection messages are encoded by the writer, after an owned snapshot is queued. + Large JSON serialization no longer runs on the caller's event loop; FIFO order, framing and + UTF-8 replacement are preserved. A writer encoding failure ends the connection without terminating the server. +- A workspace publishes its status after every participating engine finishes applying a plan. + Synchronous engine callbacks no longer expose the new profile before that application completes. + Timeline measurements retain compact transition timestamps and the busiest minute's evidence. +- Typing measurements retain bounded request timestamps and the slowest successful reply, + together with the existing before/after server reports. Sampling, timeout and latency budgets + are unchanged; truncated or recent evidence is identified explicitly. - Renaming a symbol owned by mcpp's read-only engine interface is refused even from project references when the compiler returns only project edits. Definition and declaration ownership are checked before edits; unresolved or stale ownership returns an error without partial changes. @@ -53,8 +116,6 @@ The bundled engine remains [mcppls-clangd 0.0.16](https://github.com/Sunrisepeak files. A navigation answered from the previous session's cache can otherwise expose a partially written BMI read copy. Missing work state or an expired wait fails; navigation budgets and the zero-rebuild assertion remain unchanged. -- Unchanged persistent memo snapshots in the bundled engine can still allocate memory during - idle periods. The engine remains unchanged; this release does not claim to fix that allocation. - Module builds completed inside a preparation closure count as real activity for stall detection, without incrementing the completed-prime counter. Resource and configuration measurements keep the engine activity and rebuild evidence needed to distinguish ongoing work from a stall. @@ -66,7 +127,28 @@ The bundled engine remains [mcppls-clangd 0.0.16](https://github.com/Sunrisepeak - After a cache write failure, fixing the disk and restarting clangd restores analysis directly; changing the project again is no longer needed to recover the failed plan. - The Termux nightly binds its script directory into the inner Debian rootfs, so the job can reach - the server checks. The existing best-effort policy for Termux and two-core rounds is unchanged. + the server checks, and uses the official package source instead of a randomly picked mirror. The Zed + smoke test uses a preinstalled clangd through Zed's own lookup instead of a GitHub download. +- On Windows with MSVC, an mcpp project that uses `import std` needs mcpp 2026.10.5.2 or later; for a 32-bit target it + is not available with Clang at all (Clang has no coroutines on 32-bit x86 MSVC, so the MSVC STL's `std` module + cannot be built), and such a project is read with the semantic kit (docs/20-projects.md). A conformance fixture that + needs a newer mcpp than CI's pinned one declares it (`requires-mcpp`). +- **A change can be measured without being built again.** A manual nightly can test the payloads a pull request's CI + already built (`payload-run`), run only the ux rounds (`scope`), run their edits stage several times in new server + sessions (`ux-repeat`), and run a released server beside them on the same machines (`ux-baseline`), compared by + `mcppls-devtools measure compare`; a manual nightly of a branch can test a candidate engine (docs/92-release.md). + A released server that does not report its background work fails the checks that wait for it at once, instead of + after their 15-minute bound. + +### Known issues + +- **clangd's memory can grow while idle** (#59 U13). On one of the nightly's two real projects, when the allocator + had just given memory back before the idle ten minutes, clangd's resident memory rose by about half over them with + almost no CPU; 0.0.16 did the same. The memo writes were not the cause (above), and a local run of the same + sequence does not reproduce it. 0.0.18 looks at it with the engine's own memory accounting. +- **The slowest completion while typing with autosave is about 0.2 s slower than in 0.0.16** on the nightly's + machines (one request in about 125, still well inside the 3-second budget; the 95th percentile is unchanged). It + is probably where a deferred project reload now lands; a local comparison did not reproduce the difference. ## [0.0.16] — 2026-10-14 diff --git a/conformance/README.md b/conformance/README.md index 824d47f1..95d800a4 100644 --- a/conformance/README.md +++ b/conformance/README.md @@ -28,6 +28,9 @@ checks fail at once with that reason instead of each waiting out its timeout. | `xmake-needs-download` | Plan 0.0.9 D-1, D-2, D-3, D-6, against a stand-in `xmake` script (`fake-bin/`, first on the server's PATH by the scenario's `server-path-prepend`; POSIX only, so in the Linux fixture lists). Offline, missing packages: `producer-needs-download` naming them, that build tools are included, that the run stayed offline and that the system package manager works too, `askOnline`, the project untouched. `mcppls.describeOnline` and the install fails: `producer-install-failed` with xmake's error lines and its install log, no ask. Missing at the `xmake project` stage: needs-download, not `xmake-configure-failed`. `.xmake/*/*/xmake.conf` holds `proxy`, `dotnet` and `dotnet_sdkver` (which `xmake f` refuses) and a project option: `xmake f` runs once per load, with no `xmake-options-left-out` | | `xmake-user-mode` | X-2 (E5): `.xmake/linux/x86_64/xmake.conf` says `mode = "debug"` and `fancy = true`; the engine commands have `-O0 -g` and the project's `-DFANCY_PART`, and no `-DNDEBUG`; the project directory, `.xmake/` included, is unchanged | | `xmake-no-standard` | X-3, X-7 (E4): no `set_languages`, GCC 16, `main.cpp` alone imports std (no module file, so xmake names no standard); the module units are read as C++23 -- the engine command carries `-std=gnu++23`, std is prepared and `std::println` is found -- and the report says the standard was mcppls's choice | +| `xmake-shared-tmp` | 0.0.17 XS-1 (#69, #24 UP-X1): the user built a module library with GCC (`prepare`); mcppls describes it privately; the user changes the module's implementation unit and builds again (`run`): the build passes. In 0.0.16 the private run had rewritten the module mapper files in xmake's shared temporary directory to name its own build directory, and the build failed with `failed to read compiled module`. Linux, needs `xmake` on PATH | +| `xmake-target-then-mode` | 0.0.17 XS-2 (#68, #24 UP-X2): a module library with `import std`, described and never built; a test target that depends on it is added, then the user's `xmake.conf` turns to release: the model stays xmake's, follows release (`-DNDEBUG` in the test's command), and nothing was kept as stale. With xmake 3.1.1 (CI's), 0.0.16 configured its state again in place and every describe failed with `dependfile ... not found`. Linux, needs `xmake` on PATH | +| `xmake-stale-state` | 0.0.17 XS-1, XS-3, XS-4 (#68, #24 UP-X2), against a stand-in `xmake` (`fake-bin/`, POSIX only) whose private state goes out of date as real xmake's does (a second `xmake project` in one state fails with `dependfile ... not found`; configuring again does not repair it): `XMAKE_TMPDIR` is inside mcppls's private directory; after the server is killed and started again on the same cache, the out-of-date state is discarded and made again once (one configure, two describes in that session) and the model stays xmake's, never stale; after `mcppls.resetCache` the private state is gone and the next load does not fail at all | | `compdb-midwrite` | X-5: a `compile_commands.json` cut to half and written complete 300 ms later (and again with a writer that takes 2.5 s): no `compdb-invalid`, no `model-stale`, the model follows the complete file, the slow writer's half is read again a second later (`model-reread` event) | | `inferred` | Loose module sources, no build system and no compiler: the semantic kit provides libc++ semantics (design 13.5) | | `build-discovery-off` | Plan 2026-09-27 B-7: `mcppls.buildDiscovery = off` on an mcpp project whose producer would answer: nothing is detected, read or run, the model is scanned sources (L4), the report says discovery is off, and module features still work | @@ -36,7 +39,7 @@ checks fail at once with that reason instead of each waiting out its timeout. | `untrusted` | An mcpp package in an untrusted workspace: nothing is executed, the kit answers, the status is `degraded` with the reason | | `mcpp-gcc` | mcpp with GCC 16, described by mcpp's own `emit build-database` (mcpp 2026.9.15.1): level 3 once the S1 library has structured mcpp's level 2 document, GCC arguments translated for clangd (P1), libstdc++'s `std` from its manifest, a test that imports the package's module across sets, and the workspace unchanged | | `mcpp-llvm` | The same with LLVM 22: libc++ selected through include paths, BMI arguments removed (P3) | -| `mcpp-build-programs` | A package whose `build.mcpp` imports mcpp's engine interface and a library of `[build-dependencies]`, and which has a package in both tables, described by mcpp 2026.10.11.1 (mcpp-community/mcpp#797; `MCPPLS_BUILD_PROGRAMS_MCPP`, which CI installs beside the pinned mcpp). `build.mcpp` is read as C++ with the arguments of its own set; navigation, hover and completion reach its libraries and the engine interface; the target reads the shared package as it builds it; an import across the two configurations is diagnosed; a rename that would edit the engine interface is refused | +| `mcpp-build-programs` | A package whose `build.mcpp` imports mcpp's engine interface and a library of `[build-dependencies]`, and which has a package in both tables, described by mcpp 2026.10.11.1 (mcpp-community/mcpp#797; `MCPPLS_REAL_MCPP`, which CI installs beside the pinned mcpp). `build.mcpp` is read as C++ with the arguments of its own set; navigation, hover and completion reach its libraries and the engine interface; the target reads the shared package as it builds it; an import across the two configurations is diagnosed; a rename that would edit the engine interface is refused | | `mcpp-build-programs-cross` | The same project's build programs given a toolchain of their own, as in a cross build, simulated by `mcppls-mock-mcpp` from what mcpp 2026.10.11.1 printed, with stand-ins for the engine interface inside the fixture. The default context says where `build.mcpp` is read; the context `app:build` reads it; formatting, code actions and renames do not edit the engine interface | | `mcpp-split` | The shape of a project that separates interfaces from implementations: interface units and an interface partition in `.cppm`, two implementation units (`module hello.greet;`) in `.cpp`, and an implementation partition; from mcpp's own build database. Declarations and definitions are reached from importers, definitions in implementation units before any of them is open, implementation units navigate into partitions and complete module-internal names, edits reach importers, and a file outside the build (`apps/gui/main.cpp`, importing a module nothing provides) is answered by clangd. `mcpp-split-gcc` (Linux) and `mcpp-split-msvc` (Windows) are the same project with GCC 16 and with `msvc@system` | | `mcpp-partition-definition` | Plan 2026-09-27 N-7 (WA-CLANGD-008): a module split into partitions, functions and members declared in a partition with class-type parameters and defined in other units of the module. clangd's background index builds no module a unit imports, so these definitions stayed on their declarations (0.0.5 fails N1-N5); the server builds the module's implementation units through clangd's foreground, and go-to-definition reaches them with only the callers open | @@ -44,6 +47,7 @@ checks fail at once with that reason instead of each waiting out its timeout. | `mcpp-watch` | S2 5 with mcpp itself: a new module interface, a broken `mcpp.toml` and its repair are inputs mcpp names in `watch`; the broken manifest keeps the last model, `degraded` with `model-stale` and mcpp's own `MCPP_BUILD_DATABASE_PLAN_FAILED` (S2-5-9). CI also runs it as `mcpp-watch@polling` on Linux | | `mcpp-target-selection` | A host build-program context listed first cannot replace the project's i686 profile. Root CDB target switches, deletion and recreation reach producer argv and the engine; same-target rewrites do not invoke the producer | | `mcpp-target-windows` | Windows i686/x64 target switching, including conditional module declarations reporting 32-bit semantics, without an SDK or standard-library dependency; runs on every host | +| `mcpp-target-windows-real` | On Windows, real mcpp and LLVM publish complete i686/x64 CDBs. A single session follows the unmodified databases i686 → x64 → i686, overriding the x64 project default; actual engine commands, conditional module constants, navigation and diagnostics are checked | | `mcpp-target-explicit`, `mcpp-target-project` | Explicit workspace target overrides the root CDB; project mode delegates to mcpp defaults; subsequent CDB changes do not invoke either producer again | | `mcpp-target-invalid` | An invalid explicit setting produces a configuration issue before any protocol query or producer invocation | | `mcpp-target-build-tool-off` | A disabled producer keeps the existing i686 CDB and reports the explicit x64 request as pending; no mcpp command runs | @@ -51,6 +55,8 @@ checks fail at once with that reason instead of each waiting out its timeout. | `mcpp-target-auto-fallback` | Conflicting targets and unresolved response files fall back with distinct reasons; a reason change alone does not run the producer; unique targets recover | | `mcpp-producer-identity` | An upgraded producer behind the same launcher reports its actual envelope version while protocol capabilities are cached; a changed launcher pin is watched by the provider and re-queries protocol. Also run with polling on Linux | | `mcpp-polling-watch-inputs@polling` | Watch registration retains existing non-source glob baselines, while external concrete inputs and explicitly declared files under ignored target/ directories still trigger reloads | +| `mcpp-watched-source-structure` | A successful producer answer acknowledges an unchanged lexical import excluded by preprocessing; body-only and duplicate saves avoid repeated emit, while directives, failure/retry, in-flight edits, manifest, target and native producer changes retain rediscovery | +| `mcpp-autosave-import-typing` | 0.0.17 W4 (#59 U15): an import typed one key a second with every key saved. A reload the saves of a file still being typed in ask for waits until the file is quiet (two seconds, at most thirty): imports and bodies typed in turn for forty seconds are 3 producer runs and 4 status changes, against 6 and 13 | | `mcpp-target-fallback-mismatch`, `mcpp-target-fallback-preserve` | Existing-CDB fallback validates the requested target; a disabled producer preserves usable cold CDB data but cannot replace a valid model with mismatched commands | | `mcpp-producer-file-identity` | A copied native producer replaced at the same path triggers the stat-only executable timer and re-queries identity, without a build/source event | | `mcpp-producer-relative-path` | A configured relative producer path resolves against the project root and records the actual absolute invocation | @@ -116,6 +122,8 @@ checks fail at once with that reason instead of each waiting out its timeout. | `preparation-after-idle` | Issue #49: preparation that ended over a minute ago and is given modules again (a file opened that imports one) starts its stall clock afresh. 0.0.11 reported `preparation-stalled` at once, measured from the progress of the preparation before | | `provisional-no-prime` | 0.0.8 plan R-4 (issue #34-1), Linux and macOS: the producer (`prepare delayed-producer 8`: a shell script around `mcppls-mock-mcpp` that waits eight seconds before `emit build-database`) answers late and there is no cached model, so the scanned sources are the provisional model. Six seconds in (`not-before-since-start`), `cxxModules/report` must show at most two units wanted for preparation (std, std.compat), not the project's modules, which are prepared for commands the build tool's model will replace (0.0.7 wanted three); then the model comes (`source: mcpp`) and the importer's hover works | | `clangd-crash-context` | Fix plan 2026-09-26 F3 (issue #23), Linux and macOS: a stand-in clangd (`prepare clangd-crash-context`) prints a crash context naming a file nobody opened while `main.cpp` is typed in, then dies. Only the named file is set aside, the exit is reported with it and the status names it; 0.0.4 set aside `main.cpp` | +| `clangd-exit-strike` | 0.0.17 W5 (#59 U9), Linux and macOS: the same stand-in (`prepare clangd-exit-strike`) is killed with no crash context while `main.cpp` is typed in. The unexplained exit is a strike against the file, not a verdict: nothing is set aside and `main.cpp` keeps clangd once it is back; a second such exit on the same file within ten minutes sets it aside | +| `clangd-crash-closure` | 0.0.17 W5 (#59 U7a), Linux and macOS: the stand-in (`prepare clangd-crash-closure`) crashes naming `main.cpp`, which is set aside; another importer of `hello.greet` is then opened and the module's partition edited, so clangd builds `main.cpp`'s modules again, and `main.cpp` goes back to clangd once, before its two-minute term is up | | `compdb-rejected-command` | Fix plan 2026-09-26 F6: a `compile_commands.json` whose commands carry a value the compiler rejects (`-std=c++99999`, `prepare compdb-rejected-command`), as issue #23's did: every module scan fails, and the status says the command was rejected, as an environment issue in the compiler's words | | `mcpp-emit-wait` | Fix plan 2026-09-26 F4 (D1): the producer hangs past the ten seconds after which scanned sources are planned, to its 20 s bound; mcppls's own engine answers meanwhile and clangd is given no plan until the producer answers or is given up | | `completion-keywords` | Fix plan 2026-09-26 F9 and F15, as VS Code (`client-info`): the space is a completion trigger character; a space after `import ` or `export import ` opens the module list from mcppls's own index, and one typed anywhere else, or after `import ` or `export module `, is answered at once with nothing. The module-syntax keywords are offered where each can begin a declaration and not inside a function body, merged with clangd's answer, and on their own within 1.5 s for a file clangd cannot answer for yet (a new file waiting for clangd to read a database that has it); the report counts what the space trigger cost (S3-6.2-1 to S3-6.2-5) | @@ -133,6 +141,8 @@ checks fail at once with that reason instead of each waiting out its timeout. Before the fixtures run, CI records what `mcpp emit build-database --format json` prints for each of the host's mcpp fixtures that has no `mcpp-mock.json` and passes those envelopes to `docs/specs/tools/validate.py`, which checks them like the simulated data: the S2 envelope and S1 schemas, S1's semantic rules and the contract of mcpp-community/mcpp#636 (level 2, sets per package, every set seeing every other). The producer's side and the consumer's side of the loop are thus both checked against the same mcpp. +A fixture that runs a real mcpp newer than the one CI pins declares it: `"requires-mcpp": {"executable": "{env:MCPPLS_REAL_MCPP|mcpp}", "at-least": "2026.10.5.2"}`. Before its prepare steps the runner asks that mcpp its `--version` and fails with one sentence (`the fixture needs mcpp 2026.10.5.2 or later, and … is 2026.9.30.2`) instead of letting an older mcpp fail in the middle of a build. CI installs `MCPP_FIXTURE_VERSION` (`.github/versions.env`) beside the pinned mcpp for every fixture list that has such a fixture, and names it by `MCPPLS_REAL_MCPP`. + On Windows every server runs without a developer environment, as it does when an editor starts it. Fixtures whose own build needs one declare `"prepare-environment": "msvc"`, and the runner is given the environment with `--msvc-env FILE` (`NAME=value` lines, the output of `set` after `vcvars64.bat`); only the prepare steps see it. `--no-dynamic-watch` makes the runner declare no `workspace.didChangeWatchedFiles.dynamicRegistration` support at all, the way an editor without it would, exercising the server's own polling fallback (usable plan W9.3) instead of dynamic registration. Without it the runner acts as an editor's file watcher: it keeps the watchers the server registers (glob strings and relative patterns alike) and reports its own `write-file` writes that they cover through `workspace/didChangeWatchedFiles`. In CI's fixture lists, `NAME@polling` runs fixture `NAME` with `--no-dynamic-watch`. ## Startup timing @@ -247,6 +257,7 @@ between an adopted target and a pending request while a producer is still runnin | `second-instance` | a second server on the same workspace and cache reports the notice `notice-code` (default `shared-workspace`) in its status (overall design 6.3) | | `mcp` | S5 section 6: `mcppls mcp`, started once per fixture with the fixture's server arguments beside the language server, answers the tool call `"tool"` with `"arguments"` (or, with `"method"` and `"params"`, another request) with a result meeting `"expect"`; `"is-error": true` expects a tool error instead; the call is repeated until the expectations hold or the check's time is up, unless `"retry": false`; with `"via": "daemon"`, through `mcppls mcp --daemon` and the workspace daemon it starts (S5 6.1) | | `execute-command` | `workspace/executeCommand` with `"command"` and `"arguments"` is answered without an error (the editor's review commands, design 7.7); `"{workspace-uri}"` in an argument is the workspace folder's URI as the client sent it, and `"expect"` names fields the answer must carry with those values | +| `run` | 0.0.17 XS-1: `"command"` (a program on PATH and its arguments, `{env:...}` expanded), run in the workspace while the server is up -- a user's own build -- exits with `"exit"` (default 0) within `"timeout"`; a failure shows its error lines | | `cli` | S5 section 7: `mcppls ` with the runner's payload and the fixture's server arguments, run to completion in the workspace, exits with `"exit"` (default 0) and prints one JSON document meeting `"expect"` | | `stress` | real-project stress testing (real-project plan RP0): seeded random use — see below — meets every key present in `"budget"` | | `type-text` | line `line` of `file` takes each of `steps` in turn, `interval-ms` apart (default 120), the whole buffer sent each time; after each, `request` (default `textDocument/documentSymbol`) is answered within `answer-within` seconds (default 5); with `save`, each step is also written to disk and reported as saved and changed, as autosave does; fails when the status turned to a state listed in `states-never` meanwhile (import-hang plan §8) | diff --git a/conformance/fixtures/clangd-crash-closure/scenario.json b/conformance/fixtures/clangd-crash-closure/scenario.json new file mode 100644 index 00000000..b4734b88 --- /dev/null +++ b/conformance/fixtures/clangd-crash-closure/scenario.json @@ -0,0 +1,166 @@ +{ + "name": "clangd-crash-closure", + "description": "0.0.17 W5 (#59 U7a), Linux and macOS: the stand-in clangd (`prepare clangd-crash-closure`) crashes 25 seconds in with a crash context naming src/main.cpp, which the editor is typing in: main.cpp is set aside, as a file clangd crashed on is. Then another importer of hello.greet is edited and the module's partition changes, so clangd builds the modules main.cpp imports again: main.cpp goes back to clangd once, well before its two-minute term is up. Up to 0.0.16 it waited the whole term.", + "prepare": [ + [ + "{conformance}", + "prepare", + "clangd-crash-closure", + "{payload}" + ] + ], + "server-arguments": [ + "--clangd", + "{workspace}/stand-in/clangd", + "--no-discover" + ], + "checks": [ + { + "id": "S1", + "kind": "status", + "state": "ready", + "engine-name": "clangd" + }, + { + "id": "W1-warmed-up", + "kind": "hover-contains", + "file": "src/main.cpp", + "at": [ + 4, + 31 + ], + "expect": "greet", + "timeout": 90 + }, + { + "id": "T1-editing-when-it-crashes", + "kind": "type-text", + "file": "src/main.cpp", + "line": 2, + "steps": [ + "i", + "in", + "int", + "int x", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "" + ], + "interval-ms": 500, + "answer-within": 15 + }, + { + "id": "R1-the-file-it-named", + "kind": "report", + "timeout": 30, + "expect": [ + { + "path": "/roots/0/engines/*/details/lastExit/crashFile", + "contains": "main.cpp" + }, + { + "path": "/roots/0/engines/*/details/filesSetAside", + "min-items": 1 + }, + { + "path": "/roots/0/engines/*/details/filesSetAside/*", + "each-contains": "main.cpp" + } + ] + }, + { + "id": "W2-another-importer", + "kind": "hover-contains", + "file": "src/crasher.cpp", + "at": [ + 3, + 4 + ], + "expect": "crash_here", + "timeout": 90 + }, + { + "id": "E1-its-module-changes", + "kind": "write-file", + "file": "src/greet/detail.cppm", + "text": "export module hello.greet:detail;\nimport std;\nexport namespace hello::detail {\n std::string prefix() { return \"Hi, \"; }\n}\n" + }, + { + "id": "E2-the-importer-is-rebuilt", + "kind": "type-text", + "file": "src/crasher.cpp", + "line": 2, + "steps": [ + "int y;", + "int y;", + "" + ], + "interval-ms": 1000, + "answer-within": 60 + }, + { + "id": "R2-handed-back-once-rebuilt", + "kind": "report", + "timeout": 60, + "expect": [ + { + "path": "/roots/0/events/*/kind", + "equals": "file-handed-back" + }, + { + "path": "/roots/0/engines/*/details/filesSetAside", + "max-items": 0 + } + ] + } + ] +} diff --git a/conformance/fixtures/clangd-crash-closure/src/crasher.cpp b/conformance/fixtures/clangd-crash-closure/src/crasher.cpp new file mode 100644 index 00000000..4e702de9 --- /dev/null +++ b/conformance/fixtures/clangd-crash-closure/src/crasher.cpp @@ -0,0 +1,4 @@ +import std; +import hello.greet; + +int crash_here() { return 1; } diff --git a/conformance/fixtures/clangd-crash-closure/src/greet/detail.cppm b/conformance/fixtures/clangd-crash-closure/src/greet/detail.cppm new file mode 100644 index 00000000..56cb7801 --- /dev/null +++ b/conformance/fixtures/clangd-crash-closure/src/greet/detail.cppm @@ -0,0 +1,5 @@ +export module hello.greet:detail; +import std; +export namespace hello::detail { + std::string prefix() { return "Hello, "; } +} diff --git a/conformance/fixtures/clangd-crash-closure/src/greet/greet.cppm b/conformance/fixtures/clangd-crash-closure/src/greet/greet.cppm new file mode 100644 index 00000000..21a6536d --- /dev/null +++ b/conformance/fixtures/clangd-crash-closure/src/greet/greet.cppm @@ -0,0 +1,7 @@ +export module hello.greet; +export import :detail; +import std; + +export namespace hello { + std::string greet(std::string_view who) { return detail::prefix() + std::string(who); } +} diff --git a/conformance/fixtures/clangd-crash-closure/src/main.cpp b/conformance/fixtures/clangd-crash-closure/src/main.cpp new file mode 100644 index 00000000..71169dbf --- /dev/null +++ b/conformance/fixtures/clangd-crash-closure/src/main.cpp @@ -0,0 +1,7 @@ +import std; +import hello.greet; + +int main(int argc, char* argv[]) { + std::println("{}", hello::greet("mcpp")); + return 0; +} diff --git a/conformance/fixtures/clangd-exit-strike/scenario.json b/conformance/fixtures/clangd-exit-strike/scenario.json new file mode 100644 index 00000000..4805debe --- /dev/null +++ b/conformance/fixtures/clangd-exit-strike/scenario.json @@ -0,0 +1,132 @@ +{ + "name": "clangd-exit-strike", + "description": "0.0.17 W5 (#59 U9), Linux and macOS: the stand-in clangd (`prepare clangd-exit-strike`) is killed 25 seconds in, the way an editor's kill or the OOM killer ends it: no crash context names a file. The editor is typing in src/main.cpp. Up to 0.0.16 that file was set aside from clangd for two minutes; now the exit is a strike against it, the file stays with clangd, and clangd answers for it again as soon as it is back. A second unexplained exit on the same file within ten minutes still sets it aside.", + "prepare": [ + [ + "{conformance}", + "prepare", + "clangd-exit-strike", + "{payload}" + ] + ], + "server-arguments": [ + "--clangd", + "{workspace}/stand-in/clangd", + "--no-discover" + ], + "checks": [ + { + "id": "S1", + "kind": "status", + "state": "ready", + "engine-name": "clangd" + }, + { + "id": "W1-warmed-up", + "kind": "hover-contains", + "file": "src/main.cpp", + "at": [ + 4, + 31 + ], + "expect": "greet", + "timeout": 90 + }, + { + "id": "T1-editing-when-it-crashes", + "kind": "type-text", + "file": "src/main.cpp", + "line": 2, + "steps": [ + "i", + "in", + "int", + "int x", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "int x;", + "" + ], + "interval-ms": 500, + "answer-within": 15 + }, + { + "id": "R1-a-strike-not-a-verdict", + "kind": "report", + "timeout": 30, + "expect": [ + { + "path": "/roots/0/events/*/kind", + "equals": "engine-exit" + }, + { + "path": "/roots/0/events/*/kind", + "equals": "exit-strike" + }, + { + "path": "/roots/0/engines/*/details/filesSetAside", + "max-items": 0 + } + ] + }, + { + "id": "T2-main-keeps-clangd", + "kind": "hover-contains", + "file": "src/main.cpp", + "at": [ + 4, + 31 + ], + "expect": "greet", + "timeout": 25 + } + ] +} diff --git a/conformance/fixtures/clangd-exit-strike/src/crasher.cpp b/conformance/fixtures/clangd-exit-strike/src/crasher.cpp new file mode 100644 index 00000000..4e702de9 --- /dev/null +++ b/conformance/fixtures/clangd-exit-strike/src/crasher.cpp @@ -0,0 +1,4 @@ +import std; +import hello.greet; + +int crash_here() { return 1; } diff --git a/conformance/fixtures/clangd-exit-strike/src/greet/detail.cppm b/conformance/fixtures/clangd-exit-strike/src/greet/detail.cppm new file mode 100644 index 00000000..56cb7801 --- /dev/null +++ b/conformance/fixtures/clangd-exit-strike/src/greet/detail.cppm @@ -0,0 +1,5 @@ +export module hello.greet:detail; +import std; +export namespace hello::detail { + std::string prefix() { return "Hello, "; } +} diff --git a/conformance/fixtures/clangd-exit-strike/src/greet/greet.cppm b/conformance/fixtures/clangd-exit-strike/src/greet/greet.cppm new file mode 100644 index 00000000..21a6536d --- /dev/null +++ b/conformance/fixtures/clangd-exit-strike/src/greet/greet.cppm @@ -0,0 +1,7 @@ +export module hello.greet; +export import :detail; +import std; + +export namespace hello { + std::string greet(std::string_view who) { return detail::prefix() + std::string(who); } +} diff --git a/conformance/fixtures/clangd-exit-strike/src/main.cpp b/conformance/fixtures/clangd-exit-strike/src/main.cpp new file mode 100644 index 00000000..71169dbf --- /dev/null +++ b/conformance/fixtures/clangd-exit-strike/src/main.cpp @@ -0,0 +1,7 @@ +import std; +import hello.greet; + +int main(int argc, char* argv[]) { + std::println("{}", hello::greet("mcpp")); + return 0; +} diff --git a/conformance/fixtures/mcpp-autosave-import-typing/README.md b/conformance/fixtures/mcpp-autosave-import-typing/README.md new file mode 100644 index 00000000..8171aabe --- /dev/null +++ b/conformance/fixtures/mcpp-autosave-import-typing/README.md @@ -0,0 +1,13 @@ +# Typing imports with autosave + +0.0.17 W4 (#59 U15). For forty seconds the editor types into `src/main.cpp` at ten keys a second, +an import line and then a line in a function body, in turn, and saves every second, as ux-mcpp's U6 +does with `files.autoSave`. Some saves change the file's imports and the ones between do not. + +Up to 0.0.16 each pause in the import changes was long enough for the reload they asked for to fire: +against this fixture, 6 producer runs and 13 status changes in the busiest minute (U15's budget is +12). Now a reload that saves of a file still being typed in ask for waits until the file has been +quiet for `EDIT_SETTLE` (two seconds), never more than `EDIT_DEFER_LIMIT` (thirty seconds) after the +first save that waited: 3 runs (at start, at the thirty-second bound, once typing stops) and 4 +status changes. The checks hold the count of `emit` calls in the mock producer's call log to 2-3 +and the status to 6 changes a minute. diff --git a/conformance/fixtures/mcpp-autosave-import-typing/fixture-stdlib/std.compat.cppm b/conformance/fixtures/mcpp-autosave-import-typing/fixture-stdlib/std.compat.cppm new file mode 100644 index 00000000..30fbcb79 --- /dev/null +++ b/conformance/fixtures/mcpp-autosave-import-typing/fixture-stdlib/std.compat.cppm @@ -0,0 +1,2 @@ +// Minimal standard-module stand-in; the target fixture imports neither module. +export module std.compat; diff --git a/conformance/fixtures/mcpp-autosave-import-typing/fixture-stdlib/std.cppm b/conformance/fixtures/mcpp-autosave-import-typing/fixture-stdlib/std.cppm new file mode 100644 index 00000000..4bc8ba95 --- /dev/null +++ b/conformance/fixtures/mcpp-autosave-import-typing/fixture-stdlib/std.cppm @@ -0,0 +1,2 @@ +// Minimal standard-module stand-in; the target fixture imports neither module. +export module std; diff --git a/conformance/fixtures/mcpp-autosave-import-typing/mcpp-mock.json b/conformance/fixtures/mcpp-autosave-import-typing/mcpp-mock.json new file mode 100644 index 00000000..05904c25 --- /dev/null +++ b/conformance/fixtures/mcpp-autosave-import-typing/mcpp-mock.json @@ -0,0 +1,174 @@ +{ + "version": "2026.10.22.1", + "defaultTarget": "x86_64-unknown-linux-gnu", + "callLog": "mcpp-calls.jsonl", + "requireOffline": true, + "watch": [ + "mcpp.toml", + "src/**/*.cppm", + "src/**/*.cpp" + ], + "database": { + "version": 1, + "revision": 0, + "ide": { + "profile-version": "0.2.0", + "generator": { + "name": "mcpp", + "version": "2026.10.22.1" + }, + "toolchains": { + "target": { + "family": "clang", + "version": "22", + "driver": "${compiler}", + "target": "${target}", + "config-files": [] + } + } + }, + "sets": [ + { + "name": "app", + "family-name": "app", + "baseline-arguments": [ + "-std=c++23", + "--target=${target}" + ], + "ide": { + "configuration": "dev", + "kind": "executable", + "toolchain": "target" + }, + "visible-sets": [ + "mcpp:std" + ], + "translation-units": [ + { + "source": "${root}/src/platform.cppm", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "--target=${target}", + "-c", + "${root}/src/platform.cppm" + ], + "local-arguments": [], + "provides": { + "target.platform": "" + }, + "requires": [], + "private": false, + "ide": { + "role": "module-interface" + } + }, + { + "source": "${root}/src/main.cpp", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "--target=${target}", + "-c", + "${root}/src/main.cpp" + ], + "local-arguments": [], + "provides": {}, + "requires": [ + "target.platform" + ], + "private": true, + "ide": { + "role": "non-module" + } + }, + { + "source": "${root}/src/condition.cpp", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "--target=${target}", + "-c", + "${root}/src/condition.cpp" + ], + "local-arguments": [], + "provides": {}, + "requires": [ + "target.platform" + ], + "private": true, + "ide": { + "role": "non-module" + } + } + ] + }, + { + "name": "mcpp:std", + "family-name": "mcpp:std", + "baseline-arguments": [ + "-std=c++23", + "-Wno-reserved-module-identifier", + "--target=${target}" + ], + "ide": { + "toolchain": "target", + "configuration": "dev", + "kind": "library" + }, + "visible-sets": [ + "app" + ], + "translation-units": [ + { + "source": "${root}/fixture-stdlib/std.cppm", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "-Wno-reserved-module-identifier", + "--target=${target}", + "-c", + "${root}/fixture-stdlib/std.cppm" + ], + "local-arguments": [], + "object": "${root}/fixture-stdlib/std.cppm.o", + "provides": { + "std": "" + }, + "requires": [], + "private": false, + "ide": { + "role": "module-interface" + } + }, + { + "source": "${root}/fixture-stdlib/std.compat.cppm", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "-Wno-reserved-module-identifier", + "--target=${target}", + "-c", + "${root}/fixture-stdlib/std.compat.cppm" + ], + "local-arguments": [], + "object": "${root}/fixture-stdlib/std.compat.cppm.o", + "provides": { + "std.compat": "" + }, + "requires": [], + "private": false, + "ide": { + "role": "module-interface" + } + } + ] + } + ] + } +} diff --git a/conformance/fixtures/mcpp-autosave-import-typing/mcpp.toml b/conformance/fixtures/mcpp-autosave-import-typing/mcpp.toml new file mode 100644 index 00000000..3da6496f --- /dev/null +++ b/conformance/fixtures/mcpp-autosave-import-typing/mcpp.toml @@ -0,0 +1,3 @@ +[package] +name = "target-fixture" +version = "0.1.0" diff --git a/conformance/fixtures/mcpp-autosave-import-typing/scenario.json b/conformance/fixtures/mcpp-autosave-import-typing/scenario.json new file mode 100644 index 00000000..2f652e86 --- /dev/null +++ b/conformance/fixtures/mcpp-autosave-import-typing/scenario.json @@ -0,0 +1,102 @@ +{ + "name": "mcpp-autosave-import-typing", + "description": "0.0.17 W4 (#59 U15): imports and function bodies typed in turn at ten keys a second, with autosave every second, for forty seconds, as ux-mcpp's U6 does: some saves change the file's imports and the ones between do not. Up to 0.0.16 a reload fired in every pause of the import changes, a producer run and a loading state each (6 runs, 13 state changes in a minute); now a reload that saves of a file still being typed in ask for waits until the file has been quiet for two seconds, at most thirty, so typing without a pause is one run every thirty seconds and one more once it stops.", + "prepare": [ + [ + "{runner-dir}/mcppls-mock-mcpp{exe}", + "--copy-self", + "producer/mcpp.exe" + ] + ], + "server-arguments": [ + "--mcpp", + "{workspace}/producer/mcpp.exe" + ], + "initialization-options": { + "buildDiscovery.mcppTarget": "project" + }, + "checks": [ + { + "id": "INITIAL", + "kind": "status", + "source": "mcpp", + "state": "ready", + "timeout": 60 + }, + { + "id": "INITIAL-EMIT", + "kind": "producer-call-log", + "file": "mcpp-calls.jsonl", + "command": "emit", + "count": 1, + "settle": 3 + }, + { + "id": "TYPING-WITH-AUTOSAVE", + "kind": "typing", + "file": "src/main.cpp", + "seconds": 40, + "hz": 10, + "completion-every-ms": 1000, + "scripts": [ + { + "line": 1, + "text": "import target.x1;" + }, + { + "line": 4, + "text": "int q1 = 1;" + }, + { + "line": 1, + "text": "import target.x2;" + }, + { + "line": 4, + "text": "int q2 = 2;" + }, + { + "line": 1, + "text": "import target.x3;" + }, + { + "line": 4, + "text": "int q3 = 3;" + }, + { + "line": 1, + "text": "import target.x4;" + }, + { + "line": 4, + "text": "int q4 = 4;" + } + ], + "autosave-ms": 1000 + }, + { + "id": "FEW-STATE-CHANGES", + "kind": "timeline", + "budget": { + "errorStates": 0, + "maxChangesPerMinute": 6 + } + }, + { + "id": "FEW-RUNS-ONCE-QUIET", + "kind": "producer-call-log", + "file": "mcpp-calls.jsonl", + "command": "emit", + "min-count": 2, + "max-count": 3, + "settle": 6 + }, + { + "id": "READY-AGAIN", + "kind": "status", + "source": "mcpp", + "state": "ready", + "timeout": 30 + } + ] +} diff --git a/conformance/fixtures/mcpp-autosave-import-typing/src/condition.cpp b/conformance/fixtures/mcpp-autosave-import-typing/src/condition.cpp new file mode 100644 index 00000000..54831467 --- /dev/null +++ b/conformance/fixtures/mcpp-autosave-import-typing/src/condition.cpp @@ -0,0 +1,4 @@ +#if 1 +import target.platform; +#endif +int condition_user() { return 0; } diff --git a/conformance/fixtures/mcpp-autosave-import-typing/src/main.cpp b/conformance/fixtures/mcpp-autosave-import-typing/src/main.cpp new file mode 100644 index 00000000..d8455a8a --- /dev/null +++ b/conformance/fixtures/mcpp-autosave-import-typing/src/main.cpp @@ -0,0 +1,3 @@ +import target.platform; + +int target_user() { return architecture_bits(); } diff --git a/conformance/fixtures/mcpp-autosave-import-typing/src/platform.cppm b/conformance/fixtures/mcpp-autosave-import-typing/src/platform.cppm new file mode 100644 index 00000000..c85e466f --- /dev/null +++ b/conformance/fixtures/mcpp-autosave-import-typing/src/platform.cppm @@ -0,0 +1,3 @@ +export module target.platform; + +export int architecture_bits() { return sizeof(void*) * 8; } diff --git a/conformance/fixtures/mcpp-build-programs/scenario.json b/conformance/fixtures/mcpp-build-programs/scenario.json index abce8f67..951133b0 100644 --- a/conformance/fixtures/mcpp-build-programs/scenario.json +++ b/conformance/fixtures/mcpp-build-programs/scenario.json @@ -1,9 +1,13 @@ { "name": "mcpp-build-programs", - "description": "A package whose build.mcpp imports mcpp's engine interface and a library of [build-dependencies], gen, and which has a package, both, in both tables, described by mcpp's own `emit build-database` (mcpp 2026.10.11.1, mcpp-community/mcpp#797): build.mcpp is read as C++ with the arguments of its own set, its imports resolve in the build programs' graph, navigation reaches the engine interface, the target reads both as it builds it, and an import across the two configurations is diagnosed. MCPPLS_BUILD_PROGRAMS_MCPP names an mcpp of that version or later, which CI installs beside the pinned one; unset, the mcpp on PATH is asked.", + "description": "A package whose build.mcpp imports mcpp's engine interface and a library of [build-dependencies], gen, and which has a package, both, in both tables, described by mcpp's own `emit build-database` (mcpp 2026.10.11.1, mcpp-community/mcpp#797): build.mcpp is read as C++ with the arguments of its own set, its imports resolve in the build programs' graph, navigation reaches the engine interface, the target reads both as it builds it, and an import across the two configurations is diagnosed. MCPPLS_REAL_MCPP names an mcpp of that version or later, which CI installs beside the pinned one; unset, the mcpp on PATH is asked.", + "requires-mcpp": { + "executable": "{env:MCPPLS_REAL_MCPP|mcpp}", + "at-least": "2026.10.11.1" + }, "server-arguments": [ "--mcpp", - "{env:MCPPLS_BUILD_PROGRAMS_MCPP|}" + "{env:MCPPLS_REAL_MCPP|}" ], "checks": [ { diff --git a/conformance/fixtures/mcpp-target-windows-real/README.md b/conformance/fixtures/mcpp-target-windows-real/README.md new file mode 100644 index 00000000..29d29c36 --- /dev/null +++ b/conformance/fixtures/mcpp-target-windows-real/README.md @@ -0,0 +1,77 @@ +# Real Windows target following + +This fixture requires Windows, Visual Studio C++ tools and Windows SDK, real mcpp, +and `llvm@22.1.8`. Supply the MSVC environment captured by the Windows conformance job: + +```bash +stage/mcppls-conformance.exe run --server payload/bin/mcppls.exe --payload payload \ + --fixture conformance/fixtures/mcpp-target-windows-real \ + --msvc-env "$RUNNER_TEMP/msvc.env" \ + --workspace-dir "$RUNNER_TEMP/target-real/workspace" \ + --cache-dir "$RUNNER_TEMP/target-real/cache" --keep --verbose --timeout 180 +``` + +`MCPPLS_REAL_MCPP` can select a real mcpp executable. The scenario never fixes the +server target. Its preparation runs real configure-only builds for i686 and x64, +copies each complete published CDB unchanged, and configures i686 again. During +one LSP session it publishes these original databases in the order 32 → 64 → 32. +The manifest default is x64, so the initial 32-bit result proves that auto mode +uses the published CDB rather than that default. Checks cover the requested and +effective targets, actual engine command, architecture-dependent module constant, +module navigation, and diagnostics in both source files. The constant also has a +pointer-size assertion, making inconsistent architecture preprocessing visible. +The module does not import `std`: Clang does not support coroutines for 32-bit +x86 MSVC targets and does not define `__cpp_impl_coroutine` there, so the MSVC +STL's `` is empty and its `std` module (``) cannot be built +by any Clang for i686, clangd included. mcppls then reads the project with the +semantic kit (`std-fallback-kit`), which is the right answer for such a project +but would hide the target switch this fixture checks. The standard library module +set of a 64-bit MSVC target is covered by `mcpp-llvm-msvc` and `mcpp-split-msvc`. + +The empty `[target.i686-windows-msvc]` section is a legitimate project custom +target declaration: mcpp's built-in target list does not include this spelling. +It does not imply that every compiler triple is a built-in mcpp target. Real +mcpp versions 2026.9.30.2 and 2026.10.21.1 accepted the canonical +`i686-pc-windows-msvc` spelling supplied to `emit build-database` with this +manifest, resolving it to the declared target before checking SDK availability. + +The strict expected CDB flags are `--target=i686-pc-windows-msvc` and +`--target=x86_64-pc-windows-msvc`. In the pinned mcpp v2026.9.30.2 sources, +`src/build/prepare/toolchain.cpp` derives `crossTargetFlag` using `llvm_triple()` +whenever a target is explicitly selected with a Clang driver; this condition +does not require the selected target to differ from the host; +`modules/toolchain-model/src/triple.cppm` adds `-pc-windows-msvc`; +`src/toolchain/hostflags.cppm` passes that token into the compile flags; and +`src/build/compile_commands.cppm` writes those tokens into `arguments`. +Native execution must still verify the generated databases and effective driver +facts. This fixture contains no build program and does not test host build +context filtering. + +Keep the complete `recorded-i686.json`, `recorded-x86_64.json`, final +`compile_commands.json`, model reports and logs as native CI artifacts. For +replay evidence, compute hashes without altering any rows: + +```bash +cmake -E sha256sum "$RUNNER_TEMP/target-real/workspace/mcpp-target-windows-real/recorded-i686.json" \ + "$RUNNER_TEMP/target-real/workspace/mcpp-target-windows-real/recorded-x86_64.json" +``` + +Local preflight on Linux reached real LLVM target resolution for both targets, +then stopped because Visual Studio tools, MSVC STL and Windows SDK were absent. +The pinned producer also accepted the canonical i686 alias before reporting the +same missing SDK. No CDB was generated locally, so there are no local generated +CDB hashes or full end-to-end pass claims. Windows CI is the native acceptance +gate; a Linux SDK failure does not establish that gate has passed. + +A separate Linux preflight with pinned mcpp 2026.9.30.2 and explicit +`x86_64-linux-gnu` confirmed that this project without `import std` produces +only its project set. With the standard import and type assertion present, real +`emit build-database` produces both the project set and `mcpp:std` (two units), +and the existing specification validator accepts the complete envelope. This +checks the standard-library contract on Linux, not the Windows target chain. + +The same Linux preflight ran real `build --configure-only --target +x86_64-linux-gnu`: both project CDB entries retained the explicit +`--target=x86_64-unknown-linux-gnu` token despite selecting the host architecture. +This supports the explicit-host flag path; native Windows CDB spelling remains +a strict CI assertion. diff --git a/conformance/fixtures/mcpp-target-windows-real/mcpp.toml b/conformance/fixtures/mcpp-target-windows-real/mcpp.toml new file mode 100644 index 00000000..f39b8e7f --- /dev/null +++ b/conformance/fixtures/mcpp-target-windows-real/mcpp.toml @@ -0,0 +1,16 @@ +[package] +name = "target-real" +version = "0.1.0" +description = "Real mcpp Windows analysis target follows the published CDB" +license = "Apache-2.0" + +[toolchain] +default = "llvm@22.1.8" + +# The initial published CDB deliberately overrides this project default with i686. +[build] +target = "x86_64-windows-msvc" + +# mcpp's built-in target table does not list 32-bit Windows; projects may +# declare an explicit custom target that the selected LLVM driver supports. +[target.i686-windows-msvc] diff --git a/conformance/fixtures/mcpp-target-windows-real/scenario.json b/conformance/fixtures/mcpp-target-windows-real/scenario.json new file mode 100644 index 00000000..9e911825 --- /dev/null +++ b/conformance/fixtures/mcpp-target-windows-real/scenario.json @@ -0,0 +1,239 @@ +{ + "name": "mcpp-target-windows-real", + "description": "Real mcpp with LLVM and the Windows MSVC environment publishes unmodified i686 and x64 compile databases. In one LSP session, the root CDB switches i686\u2192x64\u2192i686, overriding the manifest x64 default; actual analysis targets, engine commands, conditional module constants (32/64/32), navigation and diagnostics follow. No mocked producer or rewritten CDB entries.", + "requires-mcpp": { + "executable": "{env:MCPPLS_REAL_MCPP|mcpp}", + "at-least": "2026.10.5.2" + }, + "prepare-environment": "msvc", + "prepare": [ + [ + "{env:MCPPLS_REAL_MCPP|mcpp}", + "build", + "--configure-only", + "--target", + "i686-windows-msvc" + ], + [ + "cmake", + "-E", + "copy", + "compile_commands.json", + "recorded-i686.json" + ], + [ + "{env:MCPPLS_REAL_MCPP|mcpp}", + "build", + "--configure-only", + "--target", + "x86_64-windows-msvc" + ], + [ + "cmake", + "-E", + "copy", + "compile_commands.json", + "recorded-x86_64.json" + ], + [ + "{env:MCPPLS_REAL_MCPP|mcpp}", + "build", + "--configure-only", + "--target", + "i686-windows-msvc" + ] + ], + "server-arguments": [ + "--mcpp", + "{env:MCPPLS_REAL_MCPP|mcpp}" + ], + "checks": [ + { + "id": "TARGET-STATUS-1-i686", + "kind": "status", + "source": "mcpp", + "state": "ready", + "profile-kind": "build-toolchain", + "profile-compiler": "clang", + "profile-target": "i686-pc-windows-msvc", + "target-mode": "auto", + "target-requested": "i686-pc-windows-msvc", + "target-source": "compile-commands", + "target-pending": false, + "timeout": 120 + }, + { + "id": "ENGINE-1-i686", + "kind": "engine-command", + "file": "src/main.cpp", + "contains": [ + "--target=i686-pc-windows-msvc" + ], + "timeout": 60 + }, + { + "id": "ARCH-1-32", + "kind": "hover-contains", + "file": "src/main.cpp", + "at": [ + 2, + 23 + ], + "expect": "32", + "timeout": 120 + }, + { + "id": "DIAGNOSTICS-1-main", + "kind": "diagnostics-empty", + "file": "src/main.cpp", + "timeout": 120 + }, + { + "id": "DIAGNOSTICS-1-module", + "kind": "diagnostics-empty", + "file": "src/platform.cppm", + "timeout": 120 + }, + { + "id": "MODULE-1", + "kind": "definition", + "file": "src/main.cpp", + "at": [ + 0, + 12 + ], + "expect": "src/platform.cppm", + "timeout": 60 + }, + { + "id": "PUBLISH-2-x86_64", + "kind": "write-file", + "file": "compile_commands.json", + "content-from": "recorded-x86_64.json", + "expect-reload": true, + "timeout": 120 + }, + { + "id": "TARGET-STATUS-2-x86_64", + "kind": "status", + "source": "mcpp", + "state": "ready", + "profile-kind": "build-toolchain", + "profile-compiler": "clang", + "profile-target": "x86_64-pc-windows-msvc", + "target-mode": "auto", + "target-requested": "x86_64-pc-windows-msvc", + "target-source": "compile-commands", + "target-pending": false, + "timeout": 120 + }, + { + "id": "ENGINE-2-x86_64", + "kind": "engine-command", + "file": "src/main.cpp", + "contains": [ + "--target=x86_64-pc-windows-msvc" + ], + "timeout": 60 + }, + { + "id": "ARCH-2-64", + "kind": "hover-contains", + "file": "src/main.cpp", + "at": [ + 2, + 23 + ], + "expect": "64", + "timeout": 120 + }, + { + "id": "DIAGNOSTICS-2-main", + "kind": "diagnostics-empty", + "file": "src/main.cpp", + "timeout": 120 + }, + { + "id": "DIAGNOSTICS-2-module", + "kind": "diagnostics-empty", + "file": "src/platform.cppm", + "timeout": 120 + }, + { + "id": "MODULE-2", + "kind": "definition", + "file": "src/main.cpp", + "at": [ + 0, + 12 + ], + "expect": "src/platform.cppm", + "timeout": 60 + }, + { + "id": "PUBLISH-3-i686", + "kind": "write-file", + "file": "compile_commands.json", + "content-from": "recorded-i686.json", + "expect-reload": true, + "timeout": 120 + }, + { + "id": "TARGET-STATUS-3-i686", + "kind": "status", + "source": "mcpp", + "state": "ready", + "profile-kind": "build-toolchain", + "profile-compiler": "clang", + "profile-target": "i686-pc-windows-msvc", + "target-mode": "auto", + "target-requested": "i686-pc-windows-msvc", + "target-source": "compile-commands", + "target-pending": false, + "timeout": 120 + }, + { + "id": "ENGINE-3-i686", + "kind": "engine-command", + "file": "src/main.cpp", + "contains": [ + "--target=i686-pc-windows-msvc" + ], + "timeout": 60 + }, + { + "id": "ARCH-3-32", + "kind": "hover-contains", + "file": "src/main.cpp", + "at": [ + 2, + 23 + ], + "expect": "32", + "timeout": 120 + }, + { + "id": "DIAGNOSTICS-3-main", + "kind": "diagnostics-empty", + "file": "src/main.cpp", + "timeout": 120 + }, + { + "id": "DIAGNOSTICS-3-module", + "kind": "diagnostics-empty", + "file": "src/platform.cppm", + "timeout": 120 + }, + { + "id": "MODULE-3", + "kind": "definition", + "file": "src/main.cpp", + "at": [ + 0, + 12 + ], + "expect": "src/platform.cppm", + "timeout": 60 + } + ] +} diff --git a/conformance/fixtures/mcpp-target-windows-real/src/main.cpp b/conformance/fixtures/mcpp-target-windows-real/src/main.cpp new file mode 100644 index 00000000..fda9c78b --- /dev/null +++ b/conformance/fixtures/mcpp-target-windows-real/src/main.cpp @@ -0,0 +1,3 @@ +import target.real; + +int main() { return register_bits; } diff --git a/conformance/fixtures/mcpp-target-windows-real/src/platform.cppm b/conformance/fixtures/mcpp-target-windows-real/src/platform.cppm new file mode 100644 index 00000000..a07214dd --- /dev/null +++ b/conformance/fixtures/mcpp-target-windows-real/src/platform.cppm @@ -0,0 +1,9 @@ +export module target.real; + +#if defined(_M_IX86) || defined(__i386__) +export constexpr int register_bits = 32; +#else +export constexpr int register_bits = 64; +#endif + +static_assert(register_bits == sizeof(void*) * 8); diff --git a/conformance/fixtures/mcpp-watched-source-structure/README.md b/conformance/fixtures/mcpp-watched-source-structure/README.md new file mode 100644 index 00000000..da4298d6 --- /dev/null +++ b/conformance/fixtures/mcpp-watched-source-structure/README.md @@ -0,0 +1,59 @@ +# Watched C++ source acknowledgement + +Run the fixture with the normal `mcppls-conformance run --fixture` command and the +bundled mock producer beside the runner. It uses real LSP file-watch notifications, +an offline producer call log and a valid, unchanged S1 dependency graph. + +`#if 0` excludes `disabled.alpha` from the compiler's dependency graph, while the +lexical scanner still sees its import. The first structural edit must run the +producer once. Body-only edits and a repeated identical write must then avoid both +another emit and a `loading` status. A second structural edit, a manifest edit, a +live target change and replacement of the native producer must still run it. + +The original **28-check** version failed against the unchanged `1341938` product: +11 actual emits instead of the expected 6, with 14 failed checks (including count +cascades). These results are saved under +`.payload-cache/59-63-watched-source-structure/old-product-*`. They demonstrate a +repeat-scheduling defect; they do not identify the cause of the original U15 +13-transitions-per-minute failure. + +The current **70-check** version keeps those assertions and adds directive-change, +failed-description/retry and in-flight-edit controls. A failed producer response +cannot acknowledge a source. A later valid unchanged response may acknowledge only +the structure captured when that run started. A second edit while a delayed run is +active needs its own producer run. The native workspace unit regression separately +changes disk before the later watch arrives, distinguishing a run-start snapshot +from an incorrect snapshot taken when its answer completes. It also restores the +original bytes after an intervening structure change: the changed file stamp must +prevent that run from acknowledging the original structure. + +The added `src/condition.cpp` begins with an active conditional import that exactly +matches its producer dependencies. Disabling it must cause a new description both +on its first watch and after a successful acknowledgement. The producer recordings +change that unit's `requires` between `target.platform` and empty alongside those +source changes; the test does not pretend an active import has no dependency. + +Acknowledgement is session-local and conservative. Only a non-inferred producer +answer with no model issues confirms it; informational notices and the engine's +source diagnostics do not prevent confirmation. Cache restoration, rejected or +stale answers, unreadable/unknown units and input changes outside that confirmed +generation retain the existing reload path. Snapshot reads cover only previously +watched source paths, with a shared 32 MiB bound and file stamps unchanged through +the valid answer. Stamps use the existing size/time identity; this is not a claim +to detect a writer deliberately restoring those metadata. + +The key includes module/import structure and exact preprocessor logical lines, +including comments and continuations. Directive changes therefore remain inputs +even when import names are unchanged. Alternate directive spellings, pragma +operators, location/counter macros, or `#` characters not accounted for by the +scanner outside comments and literals conservatively disable acknowledgement instead of guessing. Build-program +sources retain their existing dedicated handling. + +An import the lexer cannot fully parse, including `import SELECT(active);`, is not +acknowledged: macro replacement can change its dependency while the lexical names +and macro definition remain unchanged. Header-unit imports also conservatively +disable acknowledgement; the global `required_names` meaning is unchanged. The +scanner's directive-string retention is explicitly opt-in for these two workspace +checks; ordinary index and engine scans retain no directive text. + +Unparsed module declarations (including private fragments not represented by this scanner) also conservatively forgo acknowledgement; the normal global module fragment `module;` remains supported. This is a lexical acknowledgement guard, not a full preprocessor. diff --git a/conformance/fixtures/mcpp-watched-source-structure/fixture-stdlib/std.compat.cppm b/conformance/fixtures/mcpp-watched-source-structure/fixture-stdlib/std.compat.cppm new file mode 100644 index 00000000..30fbcb79 --- /dev/null +++ b/conformance/fixtures/mcpp-watched-source-structure/fixture-stdlib/std.compat.cppm @@ -0,0 +1,2 @@ +// Minimal standard-module stand-in; the target fixture imports neither module. +export module std.compat; diff --git a/conformance/fixtures/mcpp-watched-source-structure/fixture-stdlib/std.cppm b/conformance/fixtures/mcpp-watched-source-structure/fixture-stdlib/std.cppm new file mode 100644 index 00000000..4bc8ba95 --- /dev/null +++ b/conformance/fixtures/mcpp-watched-source-structure/fixture-stdlib/std.cppm @@ -0,0 +1,2 @@ +// Minimal standard-module stand-in; the target fixture imports neither module. +export module std; diff --git a/conformance/fixtures/mcpp-watched-source-structure/mcpp-mock.json b/conformance/fixtures/mcpp-watched-source-structure/mcpp-mock.json new file mode 100644 index 00000000..05904c25 --- /dev/null +++ b/conformance/fixtures/mcpp-watched-source-structure/mcpp-mock.json @@ -0,0 +1,174 @@ +{ + "version": "2026.10.22.1", + "defaultTarget": "x86_64-unknown-linux-gnu", + "callLog": "mcpp-calls.jsonl", + "requireOffline": true, + "watch": [ + "mcpp.toml", + "src/**/*.cppm", + "src/**/*.cpp" + ], + "database": { + "version": 1, + "revision": 0, + "ide": { + "profile-version": "0.2.0", + "generator": { + "name": "mcpp", + "version": "2026.10.22.1" + }, + "toolchains": { + "target": { + "family": "clang", + "version": "22", + "driver": "${compiler}", + "target": "${target}", + "config-files": [] + } + } + }, + "sets": [ + { + "name": "app", + "family-name": "app", + "baseline-arguments": [ + "-std=c++23", + "--target=${target}" + ], + "ide": { + "configuration": "dev", + "kind": "executable", + "toolchain": "target" + }, + "visible-sets": [ + "mcpp:std" + ], + "translation-units": [ + { + "source": "${root}/src/platform.cppm", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "--target=${target}", + "-c", + "${root}/src/platform.cppm" + ], + "local-arguments": [], + "provides": { + "target.platform": "" + }, + "requires": [], + "private": false, + "ide": { + "role": "module-interface" + } + }, + { + "source": "${root}/src/main.cpp", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "--target=${target}", + "-c", + "${root}/src/main.cpp" + ], + "local-arguments": [], + "provides": {}, + "requires": [ + "target.platform" + ], + "private": true, + "ide": { + "role": "non-module" + } + }, + { + "source": "${root}/src/condition.cpp", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "--target=${target}", + "-c", + "${root}/src/condition.cpp" + ], + "local-arguments": [], + "provides": {}, + "requires": [ + "target.platform" + ], + "private": true, + "ide": { + "role": "non-module" + } + } + ] + }, + { + "name": "mcpp:std", + "family-name": "mcpp:std", + "baseline-arguments": [ + "-std=c++23", + "-Wno-reserved-module-identifier", + "--target=${target}" + ], + "ide": { + "toolchain": "target", + "configuration": "dev", + "kind": "library" + }, + "visible-sets": [ + "app" + ], + "translation-units": [ + { + "source": "${root}/fixture-stdlib/std.cppm", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "-Wno-reserved-module-identifier", + "--target=${target}", + "-c", + "${root}/fixture-stdlib/std.cppm" + ], + "local-arguments": [], + "object": "${root}/fixture-stdlib/std.cppm.o", + "provides": { + "std": "" + }, + "requires": [], + "private": false, + "ide": { + "role": "module-interface" + } + }, + { + "source": "${root}/fixture-stdlib/std.compat.cppm", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "-Wno-reserved-module-identifier", + "--target=${target}", + "-c", + "${root}/fixture-stdlib/std.compat.cppm" + ], + "local-arguments": [], + "object": "${root}/fixture-stdlib/std.compat.cppm.o", + "provides": { + "std.compat": "" + }, + "requires": [], + "private": false, + "ide": { + "role": "module-interface" + } + } + ] + } + ] + } +} diff --git a/conformance/fixtures/mcpp-watched-source-structure/mcpp.toml b/conformance/fixtures/mcpp-watched-source-structure/mcpp.toml new file mode 100644 index 00000000..3da6496f --- /dev/null +++ b/conformance/fixtures/mcpp-watched-source-structure/mcpp.toml @@ -0,0 +1,3 @@ +[package] +name = "target-fixture" +version = "0.1.0" diff --git a/conformance/fixtures/mcpp-watched-source-structure/producer-condition-off.json b/conformance/fixtures/mcpp-watched-source-structure/producer-condition-off.json new file mode 100644 index 00000000..db6fa7dc --- /dev/null +++ b/conformance/fixtures/mcpp-watched-source-structure/producer-condition-off.json @@ -0,0 +1,172 @@ +{ + "version": "2026.10.22.3", + "defaultTarget": "x86_64-unknown-linux-gnu", + "callLog": "mcpp-calls.jsonl", + "requireOffline": true, + "watch": [ + "mcpp.toml", + "src/**/*.cppm", + "src/**/*.cpp" + ], + "database": { + "version": 1, + "revision": 0, + "ide": { + "profile-version": "0.2.0", + "generator": { + "name": "mcpp", + "version": "2026.10.22.3" + }, + "toolchains": { + "target": { + "family": "clang", + "version": "22", + "driver": "${compiler}", + "target": "${target}", + "config-files": [] + } + } + }, + "sets": [ + { + "name": "app", + "family-name": "app", + "baseline-arguments": [ + "-std=c++23", + "--target=${target}" + ], + "ide": { + "configuration": "dev", + "kind": "executable", + "toolchain": "target" + }, + "visible-sets": [ + "mcpp:std" + ], + "translation-units": [ + { + "source": "${root}/src/platform.cppm", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "--target=${target}", + "-c", + "${root}/src/platform.cppm" + ], + "local-arguments": [], + "provides": { + "target.platform": "" + }, + "requires": [], + "private": false, + "ide": { + "role": "module-interface" + } + }, + { + "source": "${root}/src/main.cpp", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "--target=${target}", + "-c", + "${root}/src/main.cpp" + ], + "local-arguments": [], + "provides": {}, + "requires": [ + "target.platform" + ], + "private": true, + "ide": { + "role": "non-module" + } + }, + { + "source": "${root}/src/condition.cpp", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "--target=${target}", + "-c", + "${root}/src/condition.cpp" + ], + "local-arguments": [], + "provides": {}, + "requires": [], + "private": true, + "ide": { + "role": "non-module" + } + } + ] + }, + { + "name": "mcpp:std", + "family-name": "mcpp:std", + "baseline-arguments": [ + "-std=c++23", + "-Wno-reserved-module-identifier", + "--target=${target}" + ], + "ide": { + "toolchain": "target", + "configuration": "dev", + "kind": "library" + }, + "visible-sets": [ + "app" + ], + "translation-units": [ + { + "source": "${root}/fixture-stdlib/std.cppm", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "-Wno-reserved-module-identifier", + "--target=${target}", + "-c", + "${root}/fixture-stdlib/std.cppm" + ], + "local-arguments": [], + "object": "${root}/fixture-stdlib/std.cppm.o", + "provides": { + "std": "" + }, + "requires": [], + "private": false, + "ide": { + "role": "module-interface" + } + }, + { + "source": "${root}/fixture-stdlib/std.compat.cppm", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "-Wno-reserved-module-identifier", + "--target=${target}", + "-c", + "${root}/fixture-stdlib/std.compat.cppm" + ], + "local-arguments": [], + "object": "${root}/fixture-stdlib/std.compat.cppm.o", + "provides": { + "std.compat": "" + }, + "requires": [], + "private": false, + "ide": { + "role": "module-interface" + } + } + ] + } + ] + } +} diff --git a/conformance/fixtures/mcpp-watched-source-structure/producer-condition-on.json b/conformance/fixtures/mcpp-watched-source-structure/producer-condition-on.json new file mode 100644 index 00000000..d2cb37f0 --- /dev/null +++ b/conformance/fixtures/mcpp-watched-source-structure/producer-condition-on.json @@ -0,0 +1,174 @@ +{ + "version": "2026.10.22.3", + "defaultTarget": "x86_64-unknown-linux-gnu", + "callLog": "mcpp-calls.jsonl", + "requireOffline": true, + "watch": [ + "mcpp.toml", + "src/**/*.cppm", + "src/**/*.cpp" + ], + "database": { + "version": 1, + "revision": 0, + "ide": { + "profile-version": "0.2.0", + "generator": { + "name": "mcpp", + "version": "2026.10.22.3" + }, + "toolchains": { + "target": { + "family": "clang", + "version": "22", + "driver": "${compiler}", + "target": "${target}", + "config-files": [] + } + } + }, + "sets": [ + { + "name": "app", + "family-name": "app", + "baseline-arguments": [ + "-std=c++23", + "--target=${target}" + ], + "ide": { + "configuration": "dev", + "kind": "executable", + "toolchain": "target" + }, + "visible-sets": [ + "mcpp:std" + ], + "translation-units": [ + { + "source": "${root}/src/platform.cppm", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "--target=${target}", + "-c", + "${root}/src/platform.cppm" + ], + "local-arguments": [], + "provides": { + "target.platform": "" + }, + "requires": [], + "private": false, + "ide": { + "role": "module-interface" + } + }, + { + "source": "${root}/src/main.cpp", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "--target=${target}", + "-c", + "${root}/src/main.cpp" + ], + "local-arguments": [], + "provides": {}, + "requires": [ + "target.platform" + ], + "private": true, + "ide": { + "role": "non-module" + } + }, + { + "source": "${root}/src/condition.cpp", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "--target=${target}", + "-c", + "${root}/src/condition.cpp" + ], + "local-arguments": [], + "provides": {}, + "requires": [ + "target.platform" + ], + "private": true, + "ide": { + "role": "non-module" + } + } + ] + }, + { + "name": "mcpp:std", + "family-name": "mcpp:std", + "baseline-arguments": [ + "-std=c++23", + "-Wno-reserved-module-identifier", + "--target=${target}" + ], + "ide": { + "toolchain": "target", + "configuration": "dev", + "kind": "library" + }, + "visible-sets": [ + "app" + ], + "translation-units": [ + { + "source": "${root}/fixture-stdlib/std.cppm", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "-Wno-reserved-module-identifier", + "--target=${target}", + "-c", + "${root}/fixture-stdlib/std.cppm" + ], + "local-arguments": [], + "object": "${root}/fixture-stdlib/std.cppm.o", + "provides": { + "std": "" + }, + "requires": [], + "private": false, + "ide": { + "role": "module-interface" + } + }, + { + "source": "${root}/fixture-stdlib/std.compat.cppm", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "-Wno-reserved-module-identifier", + "--target=${target}", + "-c", + "${root}/fixture-stdlib/std.compat.cppm" + ], + "local-arguments": [], + "object": "${root}/fixture-stdlib/std.compat.cppm.o", + "provides": { + "std.compat": "" + }, + "requires": [], + "private": false, + "ide": { + "role": "module-interface" + } + } + ] + } + ] + } +} diff --git a/conformance/fixtures/mcpp-watched-source-structure/producer-failure.json b/conformance/fixtures/mcpp-watched-source-structure/producer-failure.json new file mode 100644 index 00000000..4fb145ae --- /dev/null +++ b/conformance/fixtures/mcpp-watched-source-structure/producer-failure.json @@ -0,0 +1,19 @@ +{ + "version": "2026.10.22.3", + "defaultTarget": "x86_64-unknown-linux-gnu", + "callLog": "mcpp-calls.jsonl", + "requireOffline": true, + "watch": [ + "mcpp.toml", + "src/**/*.cppm", + "src/**/*.cpp" + ], + "diagnostics": [ + { + "code": "CONTROLLED_FAILURE", + "severity": "error", + "source": "mcpp", + "message": "no model for this controlled invocation" + } + ] +} diff --git a/conformance/fixtures/mcpp-watched-source-structure/producer-slow.json b/conformance/fixtures/mcpp-watched-source-structure/producer-slow.json new file mode 100644 index 00000000..bb020609 --- /dev/null +++ b/conformance/fixtures/mcpp-watched-source-structure/producer-slow.json @@ -0,0 +1,175 @@ +{ + "version": "2026.10.22.3", + "defaultTarget": "x86_64-unknown-linux-gnu", + "callLog": "mcpp-calls.jsonl", + "requireOffline": true, + "watch": [ + "mcpp.toml", + "src/**/*.cppm", + "src/**/*.cpp" + ], + "database": { + "version": 1, + "revision": 0, + "ide": { + "profile-version": "0.2.0", + "generator": { + "name": "mcpp", + "version": "2026.10.22.3" + }, + "toolchains": { + "target": { + "family": "clang", + "version": "22", + "driver": "${compiler}", + "target": "${target}", + "config-files": [] + } + } + }, + "sets": [ + { + "name": "app", + "family-name": "app", + "baseline-arguments": [ + "-std=c++23", + "--target=${target}" + ], + "ide": { + "configuration": "dev", + "kind": "executable", + "toolchain": "target" + }, + "visible-sets": [ + "mcpp:std" + ], + "translation-units": [ + { + "source": "${root}/src/platform.cppm", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "--target=${target}", + "-c", + "${root}/src/platform.cppm" + ], + "local-arguments": [], + "provides": { + "target.platform": "" + }, + "requires": [], + "private": false, + "ide": { + "role": "module-interface" + } + }, + { + "source": "${root}/src/main.cpp", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "--target=${target}", + "-c", + "${root}/src/main.cpp" + ], + "local-arguments": [], + "provides": {}, + "requires": [ + "target.platform" + ], + "private": true, + "ide": { + "role": "non-module" + } + }, + { + "source": "${root}/src/condition.cpp", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "--target=${target}", + "-c", + "${root}/src/condition.cpp" + ], + "local-arguments": [], + "provides": {}, + "requires": [ + "target.platform" + ], + "private": true, + "ide": { + "role": "non-module" + } + } + ] + }, + { + "name": "mcpp:std", + "family-name": "mcpp:std", + "baseline-arguments": [ + "-std=c++23", + "-Wno-reserved-module-identifier", + "--target=${target}" + ], + "ide": { + "toolchain": "target", + "configuration": "dev", + "kind": "library" + }, + "visible-sets": [ + "app" + ], + "translation-units": [ + { + "source": "${root}/fixture-stdlib/std.cppm", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "-Wno-reserved-module-identifier", + "--target=${target}", + "-c", + "${root}/fixture-stdlib/std.cppm" + ], + "local-arguments": [], + "object": "${root}/fixture-stdlib/std.cppm.o", + "provides": { + "std": "" + }, + "requires": [], + "private": false, + "ide": { + "role": "module-interface" + } + }, + { + "source": "${root}/fixture-stdlib/std.compat.cppm", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "-Wno-reserved-module-identifier", + "--target=${target}", + "-c", + "${root}/fixture-stdlib/std.compat.cppm" + ], + "local-arguments": [], + "object": "${root}/fixture-stdlib/std.compat.cppm.o", + "provides": { + "std.compat": "" + }, + "requires": [], + "private": false, + "ide": { + "role": "module-interface" + } + } + ] + } + ] + }, + "delayMs": 3000 +} diff --git a/conformance/fixtures/mcpp-watched-source-structure/producer-success.json b/conformance/fixtures/mcpp-watched-source-structure/producer-success.json new file mode 100644 index 00000000..d2cb37f0 --- /dev/null +++ b/conformance/fixtures/mcpp-watched-source-structure/producer-success.json @@ -0,0 +1,174 @@ +{ + "version": "2026.10.22.3", + "defaultTarget": "x86_64-unknown-linux-gnu", + "callLog": "mcpp-calls.jsonl", + "requireOffline": true, + "watch": [ + "mcpp.toml", + "src/**/*.cppm", + "src/**/*.cpp" + ], + "database": { + "version": 1, + "revision": 0, + "ide": { + "profile-version": "0.2.0", + "generator": { + "name": "mcpp", + "version": "2026.10.22.3" + }, + "toolchains": { + "target": { + "family": "clang", + "version": "22", + "driver": "${compiler}", + "target": "${target}", + "config-files": [] + } + } + }, + "sets": [ + { + "name": "app", + "family-name": "app", + "baseline-arguments": [ + "-std=c++23", + "--target=${target}" + ], + "ide": { + "configuration": "dev", + "kind": "executable", + "toolchain": "target" + }, + "visible-sets": [ + "mcpp:std" + ], + "translation-units": [ + { + "source": "${root}/src/platform.cppm", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "--target=${target}", + "-c", + "${root}/src/platform.cppm" + ], + "local-arguments": [], + "provides": { + "target.platform": "" + }, + "requires": [], + "private": false, + "ide": { + "role": "module-interface" + } + }, + { + "source": "${root}/src/main.cpp", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "--target=${target}", + "-c", + "${root}/src/main.cpp" + ], + "local-arguments": [], + "provides": {}, + "requires": [ + "target.platform" + ], + "private": true, + "ide": { + "role": "non-module" + } + }, + { + "source": "${root}/src/condition.cpp", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "--target=${target}", + "-c", + "${root}/src/condition.cpp" + ], + "local-arguments": [], + "provides": {}, + "requires": [ + "target.platform" + ], + "private": true, + "ide": { + "role": "non-module" + } + } + ] + }, + { + "name": "mcpp:std", + "family-name": "mcpp:std", + "baseline-arguments": [ + "-std=c++23", + "-Wno-reserved-module-identifier", + "--target=${target}" + ], + "ide": { + "toolchain": "target", + "configuration": "dev", + "kind": "library" + }, + "visible-sets": [ + "app" + ], + "translation-units": [ + { + "source": "${root}/fixture-stdlib/std.cppm", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "-Wno-reserved-module-identifier", + "--target=${target}", + "-c", + "${root}/fixture-stdlib/std.cppm" + ], + "local-arguments": [], + "object": "${root}/fixture-stdlib/std.cppm.o", + "provides": { + "std": "" + }, + "requires": [], + "private": false, + "ide": { + "role": "module-interface" + } + }, + { + "source": "${root}/fixture-stdlib/std.compat.cppm", + "work-directory": "${root}", + "arguments": [ + "${compiler}", + "-std=c++23", + "-Wno-reserved-module-identifier", + "--target=${target}", + "-c", + "${root}/fixture-stdlib/std.compat.cppm" + ], + "local-arguments": [], + "object": "${root}/fixture-stdlib/std.compat.cppm.o", + "provides": { + "std.compat": "" + }, + "requires": [], + "private": false, + "ide": { + "role": "module-interface" + } + } + ] + } + ] + } +} diff --git a/conformance/fixtures/mcpp-watched-source-structure/scenario.json b/conformance/fixtures/mcpp-watched-source-structure/scenario.json new file mode 100644 index 00000000..8240885d --- /dev/null +++ b/conformance/fixtures/mcpp-watched-source-structure/scenario.json @@ -0,0 +1,633 @@ +{ + "name": "mcpp-watched-source-structure", + "description": "G-5: a lexical import inside #if 0 is correctly absent from the producer dependency graph. Describe its first structure change once, then do not re-emit for body-only saves or repeated identical watch events after the unchanged valid model answers. A second structure change, manifest input, target selection and actual producer replacement must still reload. This isolates a repeat-scheduling defect and does not assign the original U15 frequency failure a cause. Preprocessor input changes still emit; failed answers do not acknowledge their structure, and a structure changed during a delayed producer run requires its own subsequent run. A second real source starts with an active conditional import whose dependencies equal the lexical names: disabling it must emit both before and after acknowledgement, and the producer recordings update that unit requires accordingly.", + "prepare": [ + [ + "{runner-dir}/mcppls-mock-mcpp{exe}", + "--copy-self", + "producer/mcpp.exe" + ], + [ + "{runner-dir}/mcppls-mock-mcpp{exe}", + "--copy-self", + "replacement/mcpp.exe" + ] + ], + "server-arguments": [ + "--mcpp", + "{workspace}/producer/mcpp.exe" + ], + "initialization-options": { + "buildDiscovery.mcppTarget": "project" + }, + "checks": [ + { + "id": "INITIAL", + "kind": "status", + "source": "mcpp", + "state": "ready", + "timeout": 60, + "profile-target": "x86_64-unknown-linux-gnu", + "producer-version": "2026.10.22.1" + }, + { + "id": "INITIAL-EMIT", + "kind": "producer-call-log", + "file": "mcpp-calls.jsonl", + "command": "emit", + "count": 1, + "settle": 3 + }, + { + "id": "CONDITIONAL-STRUCTURE-CHANGE", + "kind": "write-file", + "file": "src/main.cpp", + "notify": true, + "timeout": 30, + "content": "import target.platform;\n#if 0\nimport disabled.alpha;\n#endif\n\nint target_user() { return architecture_bits() + 0; }\n", + "expect-reload": true + }, + { + "id": "CONDITIONAL-MODEL-READY", + "kind": "status", + "source": "mcpp", + "state": "ready", + "timeout": 60 + }, + { + "id": "STRUCTURE-DESCRIBED-ONCE", + "kind": "producer-call-log", + "file": "mcpp-calls.jsonl", + "command": "emit", + "count": 2, + "settle": 3 + }, + { + "id": "BODY-ONLY-ONE", + "kind": "write-file", + "file": "src/main.cpp", + "notify": true, + "timeout": 30, + "content": "import target.platform;\n#if 0\nimport disabled.alpha;\n#endif\n\nint target_user() { return architecture_bits() + 1; }\n", + "expect-reload": false, + "settle": 5 + }, + { + "id": "BODY-ONLY-ONE-NO-EMIT", + "kind": "producer-call-log", + "file": "mcpp-calls.jsonl", + "command": "emit", + "count": 2, + "settle": 3 + }, + { + "id": "BODY-ONLY-TWO", + "kind": "write-file", + "file": "src/main.cpp", + "notify": true, + "timeout": 30, + "content": "import target.platform;\n#if 0\nimport disabled.alpha;\n#endif\n\nint target_user() { return architecture_bits() + 2; }\n", + "expect-reload": false, + "settle": 5 + }, + { + "id": "BODY-ONLY-TWO-NO-EMIT", + "kind": "producer-call-log", + "file": "mcpp-calls.jsonl", + "command": "emit", + "count": 2, + "settle": 3 + }, + { + "id": "SAME-BYTES-AGAIN", + "kind": "write-file", + "file": "src/main.cpp", + "notify": true, + "timeout": 30, + "content": "import target.platform;\n#if 0\nimport disabled.alpha;\n#endif\n\nint target_user() { return architecture_bits() + 2; }\n", + "expect-reload": false, + "settle": 5 + }, + { + "id": "SAME-BYTES-AGAIN-NO-EMIT", + "kind": "producer-call-log", + "file": "mcpp-calls.jsonl", + "command": "emit", + "count": 2, + "settle": 3 + }, + { + "id": "NEXT-REAL-LEXICAL-STRUCTURE", + "kind": "write-file", + "file": "src/main.cpp", + "notify": true, + "timeout": 30, + "content": "import target.platform;\n#if 0\nimport disabled.alpha;\nimport disabled.beta;\n#endif\n\nint target_user() { return architecture_bits() + 2; }\n", + "expect-reload": true + }, + { + "id": "NEXT-STRUCTURE-READY", + "kind": "status", + "source": "mcpp", + "state": "ready", + "timeout": 60 + }, + { + "id": "NEXT-STRUCTURE-EMITTED", + "kind": "producer-call-log", + "file": "mcpp-calls.jsonl", + "command": "emit", + "count": 3, + "settle": 3 + }, + { + "id": "MANIFEST-INPUT-CHANGE", + "kind": "write-file", + "file": "mcpp.toml", + "notify": true, + "timeout": 30, + "content": "[package]\nname = \"target-fixture\"\nversion = \"0.1.1\"\n", + "expect-reload": true + }, + { + "id": "MANIFEST-READY", + "kind": "status", + "source": "mcpp", + "state": "ready", + "timeout": 60 + }, + { + "id": "MANIFEST-STILL-RELOADS", + "kind": "producer-call-log", + "file": "mcpp-calls.jsonl", + "command": "emit", + "count": 4, + "settle": 3 + }, + { + "id": "EXPLICIT-TARGET-CHANGE", + "kind": "configuration", + "settings": { + "mcppls": { + "buildDiscovery.mcppTarget": "x86_64-unknown-linux-gnu" + } + } + }, + { + "id": "EXPLICIT-TARGET-READY", + "kind": "status", + "source": "mcpp", + "state": "ready", + "timeout": 60, + "target-mode": "explicit", + "target-requested": "x86_64-unknown-linux-gnu", + "target-source": "configuration", + "target-pending": false + }, + { + "id": "TARGET-STILL-RELOADS", + "kind": "producer-call-log", + "file": "mcpp-calls.jsonl", + "command": "emit", + "count": 5, + "settle": 3, + "arguments-contains": [ + "--target", + "x86_64-unknown-linux-gnu" + ] + }, + { + "id": "NEXT-BODY-AFTER-TARGET", + "kind": "write-file", + "file": "src/main.cpp", + "notify": true, + "timeout": 30, + "content": "import target.platform;\n#if 0\nimport disabled.alpha;\nimport disabled.beta;\n#endif\n\nint target_user() { return architecture_bits() + 3; }\n", + "expect-reload": false, + "settle": 5 + }, + { + "id": "TARGET-BODY-NO-EMIT", + "kind": "producer-call-log", + "file": "mcpp-calls.jsonl", + "command": "emit", + "count": 5, + "settle": 3 + }, + { + "id": "UPGRADE-RECORDING-ONLY", + "kind": "write-file", + "file": "mcpp-mock.json", + "replace": { + "from": "2026.10.22.1", + "with": "2026.10.22.2" + }, + "notify": false, + "expect-reload": false, + "settle": 1 + }, + { + "id": "REPLACE-PRODUCER-IDENTITY", + "kind": "write-file", + "file": "producer/mcpp.exe", + "notify": true, + "timeout": 45, + "content-from": "replacement/mcpp.exe", + "expect-reload": true + }, + { + "id": "NEW-PRODUCER-READY", + "kind": "status", + "source": "mcpp", + "state": "ready", + "timeout": 60, + "producer-version": "2026.10.22.2", + "target-mode": "explicit" + }, + { + "id": "PRODUCER-STILL-RELOADS", + "kind": "producer-call-log", + "file": "mcpp-calls.jsonl", + "command": "emit", + "count": 6, + "settle": 3, + "version": "2026.10.22.2" + }, + { + "id": "NEXT-BODY-AFTER-PRODUCER", + "kind": "write-file", + "file": "src/main.cpp", + "notify": true, + "timeout": 30, + "content": "import target.platform;\n#if 0\nimport disabled.alpha;\nimport disabled.beta;\n#endif\n\nint target_user() { return architecture_bits() + 4; }\n", + "expect-reload": false, + "settle": 5 + }, + { + "id": "PRODUCER-BODY-NO-EMIT", + "kind": "producer-call-log", + "file": "mcpp-calls.jsonl", + "command": "emit", + "count": 6, + "settle": 3 + }, + { + "id": "PREPROCESSOR-INPUT-CHANGES", + "kind": "write-file", + "file": "src/main.cpp", + "notify": true, + "timeout": 30, + "replace": { + "from": "#if 0", + "with": "#if (0)" + }, + "expect-reload": true + }, + { + "id": "PREPROCESSOR-INPUT-READY", + "kind": "status", + "source": "mcpp", + "state": "ready", + "timeout": 60 + }, + { + "id": "DIRECTIVE-CHANGE-STILL-EMITS", + "kind": "producer-call-log", + "file": "mcpp-calls.jsonl", + "command": "emit", + "count": 7, + "settle": 3 + }, + { + "id": "BODY-AFTER-DIRECTIVE", + "kind": "write-file", + "file": "src/main.cpp", + "notify": true, + "timeout": 30, + "content": "import target.platform;\n#if (0)\nimport disabled.alpha;\nimport disabled.beta;\n#endif\n\nint target_user() { return architecture_bits() + 5; }\n", + "expect-reload": false, + "settle": 5 + }, + { + "id": "DIRECTIVE-BODY-NO-EMIT", + "kind": "producer-call-log", + "file": "mcpp-calls.jsonl", + "command": "emit", + "count": 7, + "settle": 3 + }, + { + "id": "CONTROLLED-PRODUCER-FAILURE", + "kind": "write-file", + "file": "mcpp-mock.json", + "content-from": "producer-failure.json", + "notify": false, + "expect-reload": false, + "settle": 0 + }, + { + "id": "STRUCTURE-WHICH-FAILS-TO-DESCRIBE", + "kind": "write-file", + "file": "src/main.cpp", + "notify": true, + "timeout": 30, + "content": "import target.platform;\n#if (0)\nimport disabled.alpha;\nimport disabled.beta;\nimport disabled.gamma;\n#endif\n\nint target_user() { return architecture_bits() + 5; }\n", + "expect-reload": true + }, + { + "id": "FAILED-DESCRIPTION-WAS-ATTEMPTED", + "kind": "producer-call-log", + "file": "mcpp-calls.jsonl", + "command": "emit", + "count": 8, + "settle": 3 + }, + { + "id": "FAILED-DESCRIPTION-KEEPS-MODEL", + "kind": "status", + "source": "mcpp", + "state": "degraded", + "timeout": 60 + }, + { + "id": "PRODUCER-RECOVERS", + "kind": "write-file", + "file": "mcpp-mock.json", + "content-from": "producer-success.json", + "notify": false, + "expect-reload": false, + "settle": 0 + }, + { + "id": "SAME-STRUCTURE-RETRIES-AFTER-FAILURE", + "kind": "write-file", + "file": "src/main.cpp", + "notify": true, + "timeout": 30, + "content": "import target.platform;\n#if (0)\nimport disabled.alpha;\nimport disabled.beta;\nimport disabled.gamma;\n#endif\n\nint target_user() { return architecture_bits() + 6; }\n", + "expect-reload": true + }, + { + "id": "RECOVERED-STRUCTURE-READY", + "kind": "status", + "source": "mcpp", + "state": "ready", + "timeout": 60, + "producer-version": "2026.10.22.3" + }, + { + "id": "FAILED-STRUCTURE-WAS-NOT-ACKNOWLEDGED", + "kind": "producer-call-log", + "file": "mcpp-calls.jsonl", + "command": "emit", + "count": 9, + "settle": 3 + }, + { + "id": "BODY-AFTER-SUCCESSFUL-RETRY", + "kind": "write-file", + "file": "src/main.cpp", + "notify": true, + "timeout": 30, + "content": "import target.platform;\n#if (0)\nimport disabled.alpha;\nimport disabled.beta;\nimport disabled.gamma;\n#endif\n\nint target_user() { return architecture_bits() + 7; }\n", + "expect-reload": false, + "settle": 5 + }, + { + "id": "SUCCESSFUL-RETRY-ACKNOWLEDGED", + "kind": "producer-call-log", + "file": "mcpp-calls.jsonl", + "command": "emit", + "count": 9, + "settle": 3 + }, + { + "id": "DELAY-THE-NEXT-DESCRIPTION", + "kind": "write-file", + "file": "mcpp-mock.json", + "content-from": "producer-slow.json", + "notify": false, + "expect-reload": false, + "settle": 0 + }, + { + "id": "START-INFLIGHT-STRUCTURE", + "kind": "write-file", + "file": "src/main.cpp", + "notify": true, + "timeout": 30, + "content": "import target.platform;\n#if (0)\nimport disabled.alpha;\nimport disabled.beta;\nimport disabled.gamma;\nimport disabled.delta;\n#endif\n\nint target_user() { return architecture_bits() + 7; }\n", + "expect-reload": true + }, + { + "id": "INFLIGHT-RUN-STARTED", + "kind": "producer-call-log", + "file": "mcpp-calls.jsonl", + "command": "emit", + "count": 10, + "settle": 1 + }, + { + "id": "STRUCTURE-CHANGES-WHILE-PRODUCER-RUNS", + "kind": "write-file", + "file": "src/main.cpp", + "notify": true, + "timeout": 30, + "content": "import target.platform;\n#if (0)\nimport disabled.alpha;\nimport disabled.beta;\nimport disabled.gamma;\nimport disabled.delta;\nimport disabled.epsilon;\n#endif\n\nint target_user() { return architecture_bits() + 7; }\n" + }, + { + "id": "NEXT-DESCRIPTION-IS-FAST", + "kind": "write-file", + "file": "mcpp-mock.json", + "content-from": "producer-success.json", + "notify": false, + "expect-reload": false, + "settle": 0 + }, + { + "id": "LATER-STRUCTURE-GETS-ITS-OWN-RUN", + "kind": "producer-call-log", + "file": "mcpp-calls.jsonl", + "command": "emit", + "count": 11, + "settle": 7 + }, + { + "id": "LATER-STRUCTURE-READY", + "kind": "status", + "source": "mcpp", + "state": "ready", + "timeout": 60 + }, + { + "id": "BODY-AFTER-INFLIGHT-STRUCTURE", + "kind": "write-file", + "file": "src/main.cpp", + "notify": true, + "timeout": 30, + "content": "import target.platform;\n#if (0)\nimport disabled.alpha;\nimport disabled.beta;\nimport disabled.gamma;\nimport disabled.delta;\nimport disabled.epsilon;\n#endif\n\nint target_user() { return architecture_bits() + 8; }\n", + "expect-reload": false, + "settle": 5 + }, + { + "id": "LATER-STRUCTURE-ACKNOWLEDGED", + "kind": "producer-call-log", + "file": "mcpp-calls.jsonl", + "command": "emit", + "count": 11, + "settle": 3 + }, + { + "id": "DISABLED-CONDITION-PRODUCER", + "kind": "write-file", + "file": "mcpp-mock.json", + "content-from": "producer-condition-off.json", + "notify": false, + "expect-reload": false, + "settle": 0 + }, + { + "id": "FIRST-WATCH-DISABLES-ACTIVE-IMPORT", + "kind": "write-file", + "file": "src/condition.cpp", + "content": "#if 0\nimport target.platform;\n#endif\nint condition_user() { return 0; }\n", + "notify": true, + "expect-reload": true, + "settle": 5, + "timeout": 30 + }, + { + "id": "FIRST-WATCH-DISABLED-READY", + "kind": "status", + "source": "mcpp", + "state": "ready", + "timeout": 60 + }, + { + "id": "FIRST-WATCH-DISABLED-COUNT", + "kind": "producer-call-log", + "file": "mcpp-calls.jsonl", + "command": "emit", + "count": 12, + "settle": 3 + }, + { + "id": "ENABLED-CONDITION-PRODUCER", + "kind": "write-file", + "file": "mcpp-mock.json", + "content-from": "producer-condition-on.json", + "notify": false, + "expect-reload": false, + "settle": 0 + }, + { + "id": "RESTORE-ACTIVE-IMPORT", + "kind": "write-file", + "file": "src/condition.cpp", + "content": "#if 1\nimport target.platform;\n#endif\nint condition_user() { return 0; }\n", + "notify": true, + "expect-reload": true, + "settle": 5, + "timeout": 30 + }, + { + "id": "ACTIVE-IMPORT-RESTORED-READY", + "kind": "status", + "source": "mcpp", + "state": "ready", + "timeout": 60 + }, + { + "id": "ACTIVE-IMPORT-RESTORED-COUNT", + "kind": "producer-call-log", + "file": "mcpp-calls.jsonl", + "command": "emit", + "count": 13, + "settle": 3 + }, + { + "id": "ACTIVE-CONDITIONAL-BODY-ONLY", + "kind": "write-file", + "file": "src/condition.cpp", + "content": "#if 1\nimport target.platform;\n#endif\nint condition_user() { return 1; }\n", + "notify": true, + "expect-reload": false, + "settle": 5, + "timeout": 30 + }, + { + "id": "ACTIVE-CONDITIONAL-BODY-READY", + "kind": "status", + "source": "mcpp", + "state": "ready", + "timeout": 60 + }, + { + "id": "ACTIVE-CONDITIONAL-BODY-COUNT", + "kind": "producer-call-log", + "file": "mcpp-calls.jsonl", + "command": "emit", + "count": 13, + "settle": 3 + }, + { + "id": "DISABLED-CONDITION-PRODUCER-AGAIN", + "kind": "write-file", + "file": "mcpp-mock.json", + "content-from": "producer-condition-off.json", + "notify": false, + "expect-reload": false, + "settle": 0 + }, + { + "id": "BASELINED-ACTIVE-IMPORT-DISABLED", + "kind": "write-file", + "file": "src/condition.cpp", + "content": "#if 0\nimport target.platform;\n#endif\nint condition_user() { return 1; }\n", + "notify": true, + "expect-reload": true, + "settle": 5, + "timeout": 30 + }, + { + "id": "BASELINED-CONDITION-DISABLED-READY", + "kind": "status", + "source": "mcpp", + "state": "ready", + "timeout": 60 + }, + { + "id": "BASELINED-CONDITION-DISABLED-COUNT", + "kind": "producer-call-log", + "file": "mcpp-calls.jsonl", + "command": "emit", + "count": 14, + "settle": 3 + }, + { + "id": "INACTIVE-CONDITIONAL-BODY-ONLY", + "kind": "write-file", + "file": "src/condition.cpp", + "content": "#if 0\nimport target.platform;\n#endif\nint condition_user() { return 2; }\n", + "notify": true, + "expect-reload": false, + "settle": 5, + "timeout": 30 + }, + { + "id": "INACTIVE-CONDITIONAL-BODY-READY", + "kind": "status", + "source": "mcpp", + "state": "ready", + "timeout": 60 + }, + { + "id": "INACTIVE-CONDITIONAL-BODY-COUNT", + "kind": "producer-call-log", + "file": "mcpp-calls.jsonl", + "command": "emit", + "count": 14, + "settle": 3 + } + ] +} diff --git a/conformance/fixtures/mcpp-watched-source-structure/src/condition.cpp b/conformance/fixtures/mcpp-watched-source-structure/src/condition.cpp new file mode 100644 index 00000000..54831467 --- /dev/null +++ b/conformance/fixtures/mcpp-watched-source-structure/src/condition.cpp @@ -0,0 +1,4 @@ +#if 1 +import target.platform; +#endif +int condition_user() { return 0; } diff --git a/conformance/fixtures/mcpp-watched-source-structure/src/main.cpp b/conformance/fixtures/mcpp-watched-source-structure/src/main.cpp new file mode 100644 index 00000000..d8455a8a --- /dev/null +++ b/conformance/fixtures/mcpp-watched-source-structure/src/main.cpp @@ -0,0 +1,3 @@ +import target.platform; + +int target_user() { return architecture_bits(); } diff --git a/conformance/fixtures/mcpp-watched-source-structure/src/platform.cppm b/conformance/fixtures/mcpp-watched-source-structure/src/platform.cppm new file mode 100644 index 00000000..c85e466f --- /dev/null +++ b/conformance/fixtures/mcpp-watched-source-structure/src/platform.cppm @@ -0,0 +1,3 @@ +export module target.platform; + +export int architecture_bits() { return sizeof(void*) * 8; } diff --git a/conformance/fixtures/xmake-shared-tmp/scenario.json b/conformance/fixtures/xmake-shared-tmp/scenario.json new file mode 100644 index 00000000..9a1018c5 --- /dev/null +++ b/conformance/fixtures/xmake-shared-tmp/scenario.json @@ -0,0 +1,64 @@ +{ + "name": "xmake-shared-tmp", + "description": "0.0.17 XS-1 (#69, #24 UP-X1): with GCC, xmake keeps each unit's module mapper file in its temporary directory under a name only the source's relative path decides, and the user's own builds read it. The user built the project (prepare); mcppls describes it privately; then the user changes the module's implementation unit and builds again. In 0.0.16 the private run had rewritten the mapper files to name its own build directory, where nothing was ever built, and that build failed (`failed to read compiled module`); its xmake now runs with XMAKE_TMPDIR in its own directory, so the user's build is untouched.", + "prepare": [ + [ + "xmake", + "f", + "-y", + "-m", + "debug" + ], + [ + "xmake", + "-y" + ] + ], + "checks": [ + { + "id": "S1", + "kind": "status", + "source": "xmake", + "tier": 3, + "state": "ready", + "timeout": 120 + }, + { + "id": "E1-the-module-is-described", + "kind": "engine-command", + "file": "src/greet.cpp", + "contains": [ + "-std=c++23" + ], + "timeout": 30 + }, + { + "id": "W1-the-project-directory-is-as-it-was", + "kind": "workspace-unchanged" + }, + { + "id": "U1-the-user-changes-the-implementation", + "kind": "write-file", + "file": "src/greet.cpp", + "replace": { + "from": "return 42;", + "with": "return 40 + 2;" + } + }, + { + "id": "B1-and-builds-again", + "kind": "run", + "command": [ + "xmake", + "-y" + ], + "timeout": 300 + }, + { + "id": "U2-put-back", + "kind": "write-file", + "file": "src/greet.cpp", + "restore": true + } + ] +} diff --git a/conformance/fixtures/xmake-shared-tmp/src/greet.cpp b/conformance/fixtures/xmake-shared-tmp/src/greet.cpp new file mode 100644 index 00000000..ef68b13d --- /dev/null +++ b/conformance/fixtures/xmake-shared-tmp/src/greet.cpp @@ -0,0 +1,3 @@ +module greet; + +int answer() { return 42; } diff --git a/conformance/fixtures/xmake-shared-tmp/src/greet.cppm b/conformance/fixtures/xmake-shared-tmp/src/greet.cppm new file mode 100644 index 00000000..1fa3139f --- /dev/null +++ b/conformance/fixtures/xmake-shared-tmp/src/greet.cppm @@ -0,0 +1,3 @@ +export module greet; + +export int answer(); diff --git a/conformance/fixtures/xmake-shared-tmp/src/main.cpp b/conformance/fixtures/xmake-shared-tmp/src/main.cpp new file mode 100644 index 00000000..abb8e237 --- /dev/null +++ b/conformance/fixtures/xmake-shared-tmp/src/main.cpp @@ -0,0 +1,3 @@ +import greet; + +int main() { return answer() == 42 ? 0 : 1; } diff --git a/conformance/fixtures/xmake-shared-tmp/xmake.lua b/conformance/fixtures/xmake-shared-tmp/xmake.lua new file mode 100644 index 00000000..f3d02ba3 --- /dev/null +++ b/conformance/fixtures/xmake-shared-tmp/xmake.lua @@ -0,0 +1,12 @@ +add_rules("mode.debug", "mode.release") +set_languages("c++23") + +target("greet") + set_kind("static") + add_files("src/greet.cppm", {public = true}) + add_files("src/greet.cpp") + +target("hello") + set_kind("binary") + add_deps("greet") + add_files("src/main.cpp") diff --git a/conformance/fixtures/xmake-stale-state/fake-bin/xmake b/conformance/fixtures/xmake-stale-state/fake-bin/xmake new file mode 100755 index 00000000..1b1ab510 --- /dev/null +++ b/conformance/fixtures/xmake-stale-state/fake-bin/xmake @@ -0,0 +1,28 @@ +#!/bin/sh +# A stand-in for xmake (0.0.17 XS-1, XS-3, XS-4; #68, #24 UP-X2). It keeps what real xmake 3.1 keeps where real xmake does: +# xmake f ... configures; configuring again does not repair a state (xmake's module scan cache outlives `xmake f -c`). +# xmake project the first run in a state succeeds; every later one in the same state fails in xmake's own words, as a +# state whose module dependfiles are gone does. A state made again from nothing succeeds once more. +# The commands say whether XMAKE_TMPDIR was given inside the private directory (-DXMAKE_TMPDIR_PRIVATE). +case "$1" in +f) + : > "$XMAKE_CONFIGDIR/fake-configured" + exit 0 + ;; +project) + if [ -f "$XMAKE_CONFIGDIR/fake-scanned" ]; then + echo "error: target(hello): dependfile for 'src/main.cpp' not found at 'build/.deps/hello/linux/x86_64/release/src/main.cpp.d'" >&2 + exit 255 + fi + : > "$XMAKE_CONFIGDIR/fake-scanned" + compiler=$(command -v g++) + define=-DXMAKE_TMPDIR_SHARED + case "$XMAKE_TMPDIR" in "$(dirname "$XMAKE_CONFIGDIR")"/*) define=-DXMAKE_TMPDIR_PRIVATE ;; esac + cat > "$4/compile_commands.json" <&2 +exit 2 diff --git a/conformance/fixtures/xmake-stale-state/scenario.json b/conformance/fixtures/xmake-stale-state/scenario.json new file mode 100644 index 00000000..9eaf1bad --- /dev/null +++ b/conformance/fixtures/xmake-stale-state/scenario.json @@ -0,0 +1,130 @@ +{ + "name": "xmake-stale-state", + "description": "0.0.17 XS-1, XS-3, XS-4 (#68, #24 UP-X2), against a stand-in xmake (fake-bin/xmake, first on the server's PATH; POSIX only) whose private state goes out of date the way real xmake's does: the first `xmake project` in a state succeeds, every later one in it fails with `dependfile ... not found`, and configuring again does not repair it. (a) The first session describes the project, with XMAKE_TMPDIR inside mcppls's private directory. (b) The editor is closed and opened again (the server is killed and started on the same cache): the next session meets the out-of-date state, discards it and makes it again from nothing once, and the model stays xmake's -- 0.0.16 failed there in every later session, kept the last model as stale, and a reset did not help. (c) Resetting the cache removes the private state too, so the load after it starts from nothing and does not fail at all. Tool runs are counted per server session.", + "server-path-prepend": "{workspace}/fake-bin", + "prepare": [ + [ + "chmod", + "+x", + "{workspace}/fake-bin/xmake" + ] + ], + "checks": [ + { + "id": "A1", + "kind": "status", + "source": "xmake", + "state": "ready", + "timeout": 60 + }, + { + "id": "A2-the-temporary-directory-is-mcppls-s", + "kind": "engine-command", + "file": "src/main.cpp", + "contains": [ + "-DXMAKE_TMPDIR_PRIVATE" + ], + "absent": [ + "-DXMAKE_TMPDIR_SHARED" + ], + "timeout": 30 + }, + { + "id": "B1-the-editor-opens-the-project-again", + "kind": "fault", + "action": "kill-server", + "recover-probe": { + "kind": "definition", + "file": "src/main.cpp", + "at": [ + 0, + 4 + ], + "expect": "src/main.cpp" + }, + "wait-seconds": 60, + "budget": { + "usesPrimaryCache": true, + "orphans": 0 + }, + "open": [ + "src/main.cpp" + ] + }, + { + "id": "B2-the-state-was-made-again-once", + "kind": "report", + "timeout": 30, + "expect": [ + { + "path": "/roots/0/toolRuns/*/purpose", + "equals": "producer", + "min-matches": 2, + "max-matches": 2 + }, + { + "path": "/roots/0/toolRuns/*/purpose", + "equals": "configure", + "min-matches": 1, + "max-matches": 1 + }, + { + "path": "/roots/0/events/*/kind", + "equals": "model-stale", + "max-matches": 0 + }, + { + "path": "/roots/0/project/source", + "equals": "xmake" + } + ] + }, + { + "id": "B3-ready", + "kind": "status", + "source": "xmake", + "state": "ready", + "timeout": 30 + }, + { + "id": "C1-reset", + "kind": "execute-command", + "command": "mcppls.resetCache" + }, + { + "id": "C2-the-load-after-it-starts-from-nothing", + "kind": "report", + "timeout": 30, + "expect": [ + { + "path": "/roots/0/toolRuns/*/purpose", + "equals": "producer", + "min-matches": 3, + "max-matches": 3 + }, + { + "path": "/roots/0/toolRuns/*/purpose", + "equals": "configure", + "min-matches": 2, + "max-matches": 2 + }, + { + "path": "/roots/0/events/*/kind", + "equals": "model-stale", + "max-matches": 0 + } + ] + }, + { + "id": "C3-ready", + "kind": "status", + "source": "xmake", + "state": "ready", + "timeout": 30 + }, + { + "id": "U1-the-project-directory-is-as-it-was", + "kind": "workspace-unchanged" + } + ] +} diff --git a/conformance/fixtures/xmake-stale-state/src/main.cpp b/conformance/fixtures/xmake-stale-state/src/main.cpp new file mode 100644 index 00000000..76e81970 --- /dev/null +++ b/conformance/fixtures/xmake-stale-state/src/main.cpp @@ -0,0 +1 @@ +int main() { return 0; } diff --git a/conformance/fixtures/xmake-stale-state/xmake.lua b/conformance/fixtures/xmake-stale-state/xmake.lua new file mode 100644 index 00000000..61dc9e48 --- /dev/null +++ b/conformance/fixtures/xmake-stale-state/xmake.lua @@ -0,0 +1,3 @@ +target("hello") + set_kind("binary") + add_files("src/main.cpp") diff --git a/conformance/fixtures/xmake-target-then-mode/.xmake/linux/x86_64/xmake.conf b/conformance/fixtures/xmake-target-then-mode/.xmake/linux/x86_64/xmake.conf new file mode 100644 index 00000000..7f709141 --- /dev/null +++ b/conformance/fixtures/xmake-target-then-mode/.xmake/linux/x86_64/xmake.conf @@ -0,0 +1,6 @@ +{ + arch = "x86_64", + host = "linux", + mode = "debug", + plat = "linux" +} diff --git a/conformance/fixtures/xmake-target-then-mode/app/main.cpp b/conformance/fixtures/xmake-target-then-mode/app/main.cpp new file mode 100644 index 00000000..e4dadbfa --- /dev/null +++ b/conformance/fixtures/xmake-target-then-mode/app/main.cpp @@ -0,0 +1,3 @@ +import json; +import std; +int main() { std::println("{}", json::decode("x").text); } diff --git a/conformance/fixtures/xmake-target-then-mode/json.cppm b/conformance/fixtures/xmake-target-then-mode/json.cppm new file mode 100644 index 00000000..0bd3892e --- /dev/null +++ b/conformance/fixtures/xmake-target-then-mode/json.cppm @@ -0,0 +1,4 @@ +export module json; +export import :data; +export import :error; +export import :decode; diff --git a/conformance/fixtures/xmake-target-then-mode/module/data.cppm b/conformance/fixtures/xmake-target-then-mode/module/data.cppm new file mode 100644 index 00000000..481efa44 --- /dev/null +++ b/conformance/fixtures/xmake-target-then-mode/module/data.cppm @@ -0,0 +1,3 @@ +export module json:data; +import std; +export namespace json { struct Value { std::string text; }; } diff --git a/conformance/fixtures/xmake-target-then-mode/module/decode.cppm b/conformance/fixtures/xmake-target-then-mode/module/decode.cppm new file mode 100644 index 00000000..edc60d72 --- /dev/null +++ b/conformance/fixtures/xmake-target-then-mode/module/decode.cppm @@ -0,0 +1,5 @@ +export module json:decode; +import std; +import :data; +import :error; +export namespace json { Value decode(std::string_view s); } diff --git a/conformance/fixtures/xmake-target-then-mode/module/error.cppm b/conformance/fixtures/xmake-target-then-mode/module/error.cppm new file mode 100644 index 00000000..956727b6 --- /dev/null +++ b/conformance/fixtures/xmake-target-then-mode/module/error.cppm @@ -0,0 +1,3 @@ +export module json:error; +import std; +export namespace json { struct Error { std::string what; }; } diff --git a/conformance/fixtures/xmake-target-then-mode/scenario.json b/conformance/fixtures/xmake-target-then-mode/scenario.json new file mode 100644 index 00000000..46d6f3b9 --- /dev/null +++ b/conformance/fixtures/xmake-target-then-mode/scenario.json @@ -0,0 +1,86 @@ +{ + "name": "xmake-target-then-mode", + "description": "0.0.17 XS-2 (#68, #24 UP-X2): a module library with `import std`, described by mcppls and never built. The user adds a test target that depends on the library, then switches their own configuration to release. xmake's module scan cache, which only a build updates, still counts the library as scanned, and with xmake 3.1.1 every `xmake project -k compile_commands` in a state configured again in place then fails (`dependfile ... not found`): 0.0.16 fell back to inferring the project and kept failing until its cache was removed by hand. The private state is now made again from nothing whenever what it was made from changes, so the model stays xmake's and follows release.", + "checks": [ + { + "id": "S1", + "kind": "status", + "source": "xmake", + "tier": 3, + "state": "ready", + "timeout": 180 + }, + { + "id": "W1-a-test", + "kind": "write-file", + "file": "test/basic.cpp", + "content": "import json;\nimport std;\n\nint main() { return json::decode(\"y\").text.size() == 1 ? 0 : 1; }\n" + }, + { + "id": "W2-its-target", + "kind": "write-file", + "file": "xmake.lua", + "replace": { + "from": "target(\"app\")", + "with": "target(\"json-test\")\n set_kind(\"binary\")\n add_deps(\"json\")\n add_files(\"test/*.cpp\")\n\ntarget(\"app\")" + }, + "expect-reload": true, + "timeout": 60 + }, + { + "id": "S2-ready-with-the-test", + "kind": "status", + "source": "xmake", + "state": "ready", + "timeout": 180 + }, + { + "id": "W3-release", + "kind": "write-file", + "file": ".xmake/linux/x86_64/xmake.conf", + "replace": { + "from": "mode = \"debug\"", + "with": "mode = \"release\"" + }, + "expect-reload": true, + "timeout": 60 + }, + { + "id": "S3-ready-in-release", + "kind": "status", + "source": "xmake", + "state": "ready", + "timeout": 180 + }, + { + "id": "E1-the-test-in-release", + "kind": "engine-command", + "file": "test/basic.cpp", + "contains": [ + "-DNDEBUG" + ], + "timeout": 60 + }, + { + "id": "R1-xmake-answered", + "kind": "report", + "timeout": 20, + "expect": [ + { + "path": "/roots/0/project/source", + "equals": "xmake" + }, + { + "path": "/roots/0/project/issues/*/code", + "equals": "xmake-configure-failed", + "max-matches": 0 + }, + { + "path": "/roots/0/events/*/kind", + "equals": "model-stale", + "max-matches": 0 + } + ] + } + ] +} diff --git a/conformance/fixtures/xmake-target-then-mode/src/decode.cpp b/conformance/fixtures/xmake-target-then-mode/src/decode.cpp new file mode 100644 index 00000000..dd3b2834 --- /dev/null +++ b/conformance/fixtures/xmake-target-then-mode/src/decode.cpp @@ -0,0 +1,3 @@ +module json; +import std; +namespace json { Value decode(std::string_view s) { return Value { std::string { s } }; } } diff --git a/conformance/fixtures/xmake-target-then-mode/xmake.lua b/conformance/fixtures/xmake-target-then-mode/xmake.lua new file mode 100644 index 00000000..9bd41b2b --- /dev/null +++ b/conformance/fixtures/xmake-target-then-mode/xmake.lua @@ -0,0 +1,11 @@ +add_rules("mode.debug", "mode.release") +set_languages("c++23") +set_policy("build.c++.modules.std", true) +target("json") + set_kind("static") + add_files("json.cppm", "module/*.cppm", {public = true}) + add_files("src/*.cpp") +target("app") + set_kind("binary") + add_deps("json") + add_files("app/*.cpp") diff --git a/conformance/traceability.json b/conformance/traceability.json index 82c459f2..2d308d03 100644 --- a/conformance/traceability.json +++ b/conformance/traceability.json @@ -894,6 +894,12 @@ } ], "S2-5-1": [ + { + "check": "mcpp-watched-source-structure/DIRECTIVE-CHANGE-STILL-EMITS" + }, + { + "check": "mcpp-watched-source-structure/LATER-STRUCTURE-GETS-ITS-OWN-RUN" + }, { "check": "mcpp-emit-watch/W1-named-input" }, @@ -914,7 +920,7 @@ "S2-5-2": [ { "script": "src/orchestrator/workspace.cpp", - "contains": "void schedule_reload(bool fromEdits = false) {" + "contains": "void schedule_reload(bool fromEdits = false, const std::vector& typing = {}) {" } ], "S2-5-9": [ diff --git a/docs/20-projects.md b/docs/20-projects.md index 48ea5287..4439e2e0 100644 --- a/docs/20-projects.md +++ b/docs/20-projects.md @@ -33,9 +33,14 @@ A few things are worth knowing: first time, then three times what its last run took, between 1 and 10 minutes; the project's cached model serves meanwhile, or its sources when there is none (`mcppls.producerTimeout` sets a fixed limit instead). Saving a source does not run it: a saved source whose module declaration and - imports are unchanged only updates the index. The build tool is asked again when a build file, a + imports are unchanged only updates the index. When conditional compilation excludes an import, + a successful description can still confirm that source structure, so later body-only saves do + not repeatedly ask for the same graph. Preprocessing changes, failed descriptions and files + changed while discovery runs retain conservative rechecks. The build tool is asked again when a build file, a non-source input or a source's module structure changes, and edits that keep coming are one run — - the wait before it grows with how long the build tool takes, up to a minute. + the wait before it grows with how long the build tool takes, up to a minute. A save of a file you + are still typing in (autosave while you type an import) waits until that file has been quiet for + two seconds, at most thirty, so a typed import is one run, made once you pause. - **The run is offline.** A build description is a question about the project, not an errand, so the server asks it with `MCPP_OFFLINE` set. If the project's dependencies are not on the machine yet, mcpp says so, and nothing waits for you to decide anything: @@ -93,7 +98,13 @@ after `mcpp build --target i686-windows-msvc` publishes that database, analysis another editor setting. This follows the latest published compilation configuration, which can also come from a configure-only command or a failed compilation; it does not certify a successful build. Project ownership recognizes Windows 8.3 and long directory spellings as the same location; -the compilation database's original paths and arguments are preserved. +the compilation database's original paths and arguments are preserved. On Windows with MSVC, an +mcpp project that uses `import std` needs mcpp 2026.10.5.2 or later: earlier releases pass the +Windows SDK path with a trailing backslash that swallows the next argument, the standard library +modules fail to build, and mcppls shows mcpp's own error as `producer-failed`. For a 32-bit +target (`i686-windows-msvc`), `import std` is not available with Clang at all: Clang does not +support coroutines on 32-bit x86 MSVC, so the MSVC STL's `std` module cannot be built, and mcppls +reads such a project with the semantic kit and says so (`std-fallback-kit`). To fix this workspace's analysis target, put this in `.vscode/settings.json` (or use the corresponding LSP initialization setting in another editor): @@ -148,9 +159,14 @@ Continue** configures once with the network, in that private directory). `xmake.lua` makes an xmake project. mcppls asks xmake itself, with its own command, `xmake project -k compile_commands`, which compiles nothing — but it configures and scans modules, so -mcppls points xmake's configuration and build directories at its cache (`XMAKE_CONFIGDIR`, -`--builddir`) and your project stays untouched: nothing is generated, changed or deleted in it, your -own `compile_commands.json` included. It runs offline (`--policies=package.fetch_only,network.mode:private`): +mcppls points xmake's configuration, build and temporary directories at its cache (`XMAKE_CONFIGDIR`, +`--builddir`, `XMAKE_TMPDIR`) and your project stays untouched: nothing is generated, changed or deleted in it, your +own `compile_commands.json` included, and your own builds do not read anything it wrote (with GCC, xmake keeps +module mapper files in its temporary directory under names that only the source's relative path decides; 0.0.16 +wrote them there, and your next incremental build failed with `failed to read compiled module`, #69). That private +state is a cache of mcppls's: it is made again from nothing whenever xmake itself, any `xmake.lua` or your +`xmake.conf` changes, once more when xmake says it is out of date (`dependfile ... not found`, a scan cache that +only a build updates; #68), and resetting the workspace cache removes it. It runs offline (`--policies=package.fetch_only,network.mode:private`): a package that is not installed stops it with the same offer as above. The first description takes a few seconds (about 6–8 s measured, most of it xmake detecting the toolchain); the project is served from its sources meanwhile. Module roles come from scanning, so an xmake project is L3. diff --git a/docs/50-troubleshooting.md b/docs/50-troubleshooting.md index dcc3ea0a..8ae2029d 100644 --- a/docs/50-troubleshooting.md +++ b/docs/50-troubleshooting.md @@ -102,6 +102,15 @@ finds it. If the status says `producer-install-failed` instead, the network was failed; the message has xmake's error lines and the path of its `install.txt` log, and the usual cause is a tool the source build needs (autotools, a compiler) that is not installed. +**xmake: `failed to read compiled module` in your own build, or `dependfile ... not found` in the status.** +Both are xmake's own state going out of step with what is on disk (#24 UP-X1, UP-X2). The first: with GCC, xmake +keeps module mapper files in its temporary directory under names that only the source's relative path decides, so +another build of the same paths (a second build directory, another checkout of the project) leaves names of its +modules there; running `xmake` again repairs it. mcppls 0.0.16 caused it with its private run; 0.0.17 runs xmake with a +temporary directory of its own. The second: `xmake project -k compile_commands` trusts a module scan cache that only a +build updates; a project described but not built since a target was added fails that way once a dependfile is gone. +mcppls makes its own xmake state again on its own; for your own `xmake project`, building once (`xmake`) repairs it. + **Go-to-definition works, completion does not, or the standard library is missing.** Look at `profile`. A semantic kit means no usable compiler was found; `import std` still resolves, but diagnostics come from libc++ rather than from your toolchain. @@ -280,8 +289,12 @@ argument 3: -O0 -> -O2)`), so a burst names its cause. **"clangd crashed while building NormalJsonTranslator.Core.cpp".** clangd names the file it crashed on (its crash context), and that file is set aside: answered by mcppls's own engine while clangd is -restarted without it. `engines[].details.lastExit` in the report has the exit code, the file, what -clangd was doing and, on Windows, the exception code. After five exits in five +restarted without it. It goes back to clangd when it changes, when its term (two minutes, doubling +if it crashes clangd again) is up, or once, as soon as the modules it imports have been built again: +it may have crashed on one of them half rebuilt. A clangd that exits without naming a file (killed, +or out of memory) costs the files it was working on a strike, not their place: only a second such +exit on the same file within ten minutes sets it aside. `engines[].details.lastExit` in the report +has the exit code, the file, what clangd was doing and, on Windows, the exception code. After five exits in five minutes clangd is started again after 1, 2, 4, then 8 minutes, and the status says so (`engine-crash-loop`, with **Restart clangd**); it is not given up for the session. The crash and timeout issues clear by themselves once clangd has stayed up for a minute. The last resort is @@ -384,8 +397,9 @@ the one workspace instead of deleting directories by hand: - **Neovim**: `:McpplsResetCache` - **Other clients**: `workspace/executeCommand` `mcppls.resetCache` -It stops the root's engines, removes its cache — the models, the engine database, and clangd's -module cache and locks — and starts again. The logs stay. The first session afterwards is a cold +It stops the root's engines, removes its cache — the models, the engine database, clangd's +module cache and locks, and the private states of the build tools mcppls runs (xmake, CMake, Meson; removed +before the next description starts, so never under one that is running) — and starts again. The logs stay. The first session afterwards is a cold one. Old modules are pruned by themselves: each unit keeps the built modules (BMIs) of its two newest diff --git a/docs/92-release.md b/docs/92-release.md index cae6a803..b3972181 100644 --- a/docs/92-release.md +++ b/docs/92-release.md @@ -142,8 +142,25 @@ releases has already passed this repository's CI with it, and the release brings 4. Merge this PR and release. A candidate file never reaches a release: CI refuses it on `main` and in a release's run, and the -Release workflow stops at once if it is there. Artifacts are kept 14 days; an older run is rebuilt -by re-running the engine PR's CI. +Release workflow stops at once if it is there. A manual run of CI or of the nightly on a branch other +than `main` may use it: the nightly's ux rounds are where an engine change to idle memory or typing +latency shows, so they see the engine before it is released. Artifacts are kept 14 days; an older run +is rebuilt by re-running the engine PR's CI. + +## Measuring a change + +What the nightly measures varies from run to run on the same bytes: a slower runner, a busier hour. +One red round is not a regression, and one green round is not a fix. A manual nightly +(`gh workflow run nightly.yml --ref -f …`) measures a change without building it again: + +| Input | What it does | +|---|---| +| `payload-run` | Tests the payloads a CI run of the branch built (its run id) instead of building them again: about 35 minutes instead of 80, on the bytes the pull request's CI tested | +| `scope` | `all` (every job) or `ux` (only the ux rounds, their budgets and the comparison) | +| `ux-repeat` | Each round runs its edits stage (typing, saving, U5, U6, U7a, U15) this many times, each in a new server session on what the cold and warm stages prepared: 4 over the three rounds is 12 samples of each, built once | +| `ux-baseline` | A release (`v0.0.16`) whose server runs the same stages on the same machines, in its own workspace and cache, alternating with this one (A B B A); `ux budgets` then prints `mcppls-devtools measure compare`, the typing, save, state-change, idle and timeout numbers side by side, with those whose median is worse than the baseline's beyond their spread marked | + +The runner, the fixtures and the budgets are the branch's for both servers, so only the server differs. ## Verifying a release diff --git a/docs/specs/tools/validate.py b/docs/specs/tools/validate.py index d0b10f35..98dd16cb 100644 --- a/docs/specs/tools/validate.py +++ b/docs/specs/tools/validate.py @@ -329,8 +329,12 @@ def mcpp_contract(name, doc): all(sorted(s["visible-sets"]) == sorted(n for n in names if n != s["name"] and (n in programs) == (s["name"] in programs)) for s in sets)) std_sets = {s["name"] for s in sets for tu in s["translation-units"] if {"std", "std.compat"} & set(tu.get("provides", {}))} + # The standard library modules the build compiles (S1 section 7): none when no unit imports them -- a 32-bit MSVC + # target cannot build MSVC's std with Clang, so mcpp-target-windows-real imports nothing of it. + std_expected = {"mcpp:std"} | ({"mcpp:std:host"} if programs else set()) + std_imported = any({"std", "std.compat"} & set(tu.get("requires", [])) for s in sets for tu in s["translation-units"]) check(f"mcpp {name}: the standard library modules are units of mcpp:std, and of mcpp:std:host for build programs", - std_sets == {"mcpp:std"} | ({"mcpp:std:host"} if programs else set())) + std_sets == std_expected if std_imported else std_sets <= std_expected) check(f"mcpp {name}: :test sets hold tests", all(s["ide"].get("kind") == "test" for s in sets if s["name"].endswith(":test"))) check(f"mcpp {name}: :build sets are executables, the build programs", all(s["ide"].get("kind") == "executable" for s in sets if s["name"].endswith(":build"))) diff --git a/docs/zh-CN/20-projects.md b/docs/zh-CN/20-projects.md index 570ca88b..9d21cf07 100644 --- a/docs/zh-CN/20-projects.md +++ b/docs/zh-CN/20-projects.md @@ -18,7 +18,7 @@ mcpp 给出的文档里列出了每个翻译单元、它的模块角色、它的 有几点要注意： -- **它最多可以跑多久，什么时候会再跑。** 构建工具的硬性时限是：第一次 5 分钟，之后是上一次运行用时的三倍，最短 1 分钟、最长 10 分钟；这期间由工程缓存的模型提供服务，没有缓存时按源码提供服务（`mcppls.producerTimeout` 可以改成固定时限）。保存源文件不会重新运行它：一个保存后模块声明和 import 都没变的源文件，只更新索引。只有构建文件、非源码输入或某个源文件的模块结构变化时才会再次询问构建工具，而且持续不断的编辑只算一次运行——运行前的等待会随构建工具的耗时变长，最长一分钟。 +- **它最多可以跑多久，什么时候会再跑。** 构建工具的硬性时限是：第一次 5 分钟，之后是上一次运行用时的三倍，最短 1 分钟、最长 10 分钟；这期间由工程缓存的模型提供服务，没有缓存时按源码提供服务（`mcppls.producerTimeout` 可以改成固定时限）。保存源文件不会重新运行它：一个保存后模块声明和 import 都没变的源文件，只更新索引。只有构建文件、非源码输入或某个源文件的模块结构变化时才会再次询问构建工具，而且持续不断的编辑只算一次运行——运行前的等待会随构建工具的耗时变长，最长一分钟。正在输入的文件被保存时（比如边输入 import 边自动保存），这次运行会等到该文件停止输入两秒之后再进行，最多等三十秒，所以输入一个 import 只运行一次，在你停下来之后。 - **这一步是离线的。** 获取构建描述是查询工程信息，而不是替用户执行任务，所以服务端以 `MCPP_OFFLINE` 发起查询。如果项目依赖的东西机器上还没有，mcpp 会说明这一点，而且不需要等你做任何决定： - 工程立即按源码提供服务（L4），mcpp 能描述的部分照常使用； - 右下角的通知提供 **Download and Continue**（这一次允许构建工具联网）、**Run in Terminal**（你的代理和凭证在那里）和 **Don't Ask Again**。它可以一直不点；每个工作区、每组缺失的东西只问一次； @@ -36,6 +36,9 @@ mcpp 给出的文档里列出了每个翻译单元、它的模块角色、它的 `compile_commands.json`；项目源码明确使用同一目标平台时，mcppls 将该目标传给 mcpp，重新获取模块构建描述。 例如执行 `mcpp build --target i686-windows-msvc` 发布编译数据库后，编辑器跟随 i686，无需再设置一次。 这里跟随的是最近发布的编译配置；仅配置命令或随后编译失败的命令也可能发布它，不代表最近一次成功构建。 +在 Windows MSVC 下，使用 `import std` 的 mcpp 工程需要 mcpp 2026.10.5.2 或更新版本：更早的版本传给编译器的 +Windows SDK 路径以反斜杠结尾，会吞掉下一个参数，标准库模块无法构建，mcppls 以 `producer-failed` 原样显示 mcpp 的错误。32 位目标（`i686-windows-msvc`）则完全不能用 Clang `import std`： +Clang 不支持 32 位 x86 MSVC 上的协程，MSVC STL 的 `std` 模块因此无法构建，mcppls 会改用语义工具包读取这类工程并说明原因（`std-fallback-kit`）。 要固定当前工程的分析平台，可在 `.vscode/settings.json` 中设置： @@ -73,7 +76,7 @@ mcpp 给出的文档里列出了每个翻译单元、它的模块角色、它的 ## xmake -有 `xmake.lua` 就是 xmake 工程。mcppls 用 xmake 自己的命令 `xmake project -k compile_commands` 向 xmake 要一份，这个命令不编译任何东西——但它会配置并扫描模块，所以 mcppls 把 xmake 的配置目录和构建目录指向自己的缓存（`XMAKE_CONFIGDIR`、`--builddir`），你的工程保持不变：不生成、不修改、不删除里面的任何文件，包括你自己的 `compile_commands.json`。它离线运行（`--policies=package.fetch_only,network.mode:private`）：没有安装的包会让它停下，并给出和上面一样的询问。第一次描述要几秒钟（实测约 6–8 秒，大部分是 xmake 在探测工具链）；这期间工程按源码提供服务。模块角色靠扫描得到，所以 xmake 工程是 L3。 +有 `xmake.lua` 就是 xmake 工程。mcppls 用 xmake 自己的命令 `xmake project -k compile_commands` 向 xmake 要一份，这个命令不编译任何东西——但它会配置并扫描模块，所以 mcppls 把 xmake 的配置目录、构建目录和临时目录指向自己的缓存（`XMAKE_CONFIGDIR`、`--builddir`、`XMAKE_TMPDIR`），你的工程保持不变：不生成、不修改、不删除里面的任何文件，包括你自己的 `compile_commands.json`；你自己的构建也不会读到它写的任何东西（用 GCC 时，xmake 把模块 mapper 文件放在临时目录里，文件名只由源文件的相对路径决定；0.0.16 写在了那里，你下一次增量构建就会报 `failed to read compiled module`，#69）。这份私有状态是 mcppls 的缓存：xmake 本身、任何一个 `xmake.lua` 或你的 `xmake.conf` 变化时，它会从头重建；xmake 报告它已过期时（`dependfile ... not found`，一份只有构建才会更新的扫描缓存；#68）再从头重建一次；重置工作区缓存也会把它删掉。它离线运行（`--policies=package.fetch_only,network.mode:private`）：没有安装的包会让它停下，并给出和上面一样的询问。第一次描述要几秒钟（实测约 6–8 秒，大部分是 xmake 在探测工具链）；这期间工程按源码提供服务。模块角色靠扫描得到，所以 xmake 工程是 L3。 模型会跟着你的操作走，不需要手动做任何事：任何一个 `xmake.lua` 变了，就重新描述工程；你自己运行 `xmake f` 也一样——mcppls 读取它留在 `.xmake///xmake.conf` 里的内容（只读；有多个时取最新的那个），并让自己的私有运行用同样的方式配置：平台、架构、模式（`-m debug` 得到 `-O0 -g`，而不是 release 的参数）、工具链、SDK、运行库、kind，以及你的 `xmake.lua` 声明的选项。文件里哪些键算选项，由 xmake 自己回答：mcppls 在工程里运行一次 `xmake f --help`（结果缓存在自己的缓存里，直到 `xmake` 或 `xmake.lua` 变化），只传这份列表里有的键。其余的是 xmake 写给自己的（`proxy`、`dotnet`、`dotnet_sdkver`、工具链探测结果）：`xmake f` 一律拒绝它们，所以直接跳过，也不会提示。只有 xmake 给不出可用的帮助、又拒绝了某个选项时，mcppls 才会只带标准选项再配置一次，状态栏会说明哪些没有带上。 diff --git a/docs/zh-CN/50-troubleshooting.md b/docs/zh-CN/50-troubleshooting.md index 0df4db88..665a131f 100644 --- a/docs/zh-CN/50-troubleshooting.md +++ b/docs/zh-CN/50-troubleshooting.md @@ -76,6 +76,8 @@ mcppls report --bundle problem.zip --root path/to/project # 可加 --hide-proj **“xmake needs libtool, libpthread-stubs downloaded”。** 这些名字不是你的代码缺的库：它们是 xmake 为你的 `xmake.lua` 所要求的包而要下载或构建的东西，包括构建工具（设成从源码构建的包会带进自己的构建工具，比如 `libtool`、`meson`），而 mcppls 保持离线，不会自己去下载。有三条路：在通知里选 **Download and Continue**；在终端里运行 `xmake`（做完后会重新读取描述）；或者用系统包管理器安装（`apt install libtool libpthread-stubs0-dev`、`pacman -S libtool`、`brew install libtool`）——xmake 的包定义里写了对应的系统包，找到系统里的就直接用。如果状态里写的是 `producer-install-failed`，说明已允许联网、安装本身失败了；消息里有 xmake 的 `error:` 行和 `install.txt` 日志的路径，常见原因是源码构建需要的工具（autotools、编译器）没有安装。 +**xmake：你自己的构建报 `failed to read compiled module`，或者状态里出现 `dependfile ... not found`。** 两者都是 xmake 自己的状态和磁盘上的内容对不上（#24 UP-X1、UP-X2）。前者：用 GCC 时，xmake 把模块 mapper 文件放在临时目录里，文件名只由源文件的相对路径决定，所以同样路径的另一次构建（另一个构建目录、工程的另一份 checkout）会把它自己模块的路径留在那里；再运行一次 `xmake` 即可恢复。mcppls 0.0.16 的私有运行会引起这个问题；0.0.17 让 xmake 使用自己的临时目录。后者：`xmake project -k compile_commands` 依赖一份只有构建才会更新的模块扫描缓存；加了 target 之后只描述、从没构建过的工程，一旦某个 dependfile 不在了就会这样失败。mcppls 会自己重建它的 xmake 状态；如果是你自己运行的 `xmake project`，构建一次（`xmake`）即可恢复。 + **所有功能失效，任何位置都无法跳转到定义。** 看报告里的 `project.source`。如果一个用了构建系统的项目里它是 `inferred`，说明构建工具没有给出答复；原因在 `project.issues` 和 `toolRuns` 的最后一条里。常见原因是构建工具需要下载东西：这时状态栏会提议在你的终端里运行它。 **跳转到定义能用，补全不能用，或者标准库缺失。** 看 `profile`。语义工具包意味着没找到可用的编译器；`import std` 仍然能解析，但诊断来自 libc++，不是来自你的工具链。 @@ -122,7 +124,7 @@ mcppls report --bundle problem.zip --root path/to/project # 可加 --hide-proj **clangd 反复重启。** 看 `engines[].restarts`、`engines[].details.restartBudget` 和 `events` 日志。每种原因各有十分钟三次的重启额度：引擎数据库变化（`plan`）、恢复停止应答或空转的 clangd（`recovery`）、clangd 退出（`crash`）。用完之后，同类的下一次重启依次等待一、二、四、八分钟——是退避而不是拒绝，所以卡住的 clangd 总能恢复——状态会说明（`engine-restart-capped`）并提供 **Restart clangd** 按钮（其他编辑器用 `workspace/executeCommand` `mcppls.restartEngine`），它立即重启且从不计入额度。切换工具链、profile 或 context 也从不计入，模块编译不过从来不是重启的理由。引擎数据库的每次变化都会记入日志并写明改了什么（`engine database changed: … compiled otherwise (main.cpp: argument 3: -O0 -> -O2)`），重启密集时能直接看到原因。 -**“clangd crashed while building NormalJsonTranslator.Core.cpp”。** clangd 会说明它在哪个文件上崩溃（崩溃上下文），隔离的就是这个文件：它改由 mcppls 自己的引擎应答，同时重启一个不带它的 clangd。报告里的 `engines[].details.lastExit` 有退出码、文件、clangd 当时在做什么，Windows 上还有异常码。五分钟内退出五次之后，clangd 会在 1、2、4、8 分钟后重新启动，状态会说明（`engine-crash-loop`，带 **Restart clangd**）；它不会在本次会话里被放弃。clangd 稳定运行满一分钟后，崩溃和超时这两类 issue 会自行清除。最后的办法见[下文](#mcppls-无法自行恢复时)。 +**“clangd crashed while building NormalJsonTranslator.Core.cpp”。** clangd 会说明它在哪个文件上崩溃（崩溃上下文），隔离的就是这个文件：它改由 mcppls 自己的引擎应答，同时重启一个不带它的 clangd。它在以下情况交回 clangd：文件本身有改动；隔离期满（两分钟，再次让 clangd 崩溃则翻倍）；或者它导入的模块重新构建完成（只交回一次，因为它可能正是在某个重建到一半的模块上崩溃的）。clangd 退出时没有指明文件（被杀掉或内存不足），它当时在处理的文件只记一次警告，不会被隔离；同一个文件在十分钟内第二次碰到这种退出，才会被隔离。报告里的 `engines[].details.lastExit` 有退出码、文件、clangd 当时在做什么，Windows 上还有异常码。五分钟内退出五次之后，clangd 会在 1、2、4、8 分钟后重新启动，状态会说明（`engine-crash-loop`，带 **Restart clangd**）；它不会在本次会话里被放弃。clangd 稳定运行满一分钟后，崩溃和超时这两类 issue 会自行清除。最后的办法见[下文](#mcppls-无法自行恢复时)。 **“mcpp could not describe tools/updater/mcpp.toml”。** 构建工具描述了工作区的其余部分，并说明了它没能描述的那一部分（例如某个成员的构建程序失败）；那部分的文件按其余部分提供的信息来读，其余部分照常工作（`producer-partial`，S2 0.3.0）。修好消息里指出的问题，下次重新加载就会一并描述它。 @@ -166,7 +168,7 @@ Windows 上补全还慢在哪里、这一版做了什么，看日志里的 `Comp - **Neovim**：`:McpplsResetCache` - **其他客户端**：`workspace/executeCommand` `mcppls.resetCache` -它会停掉这个根目录的引擎，删除它的缓存——模型、引擎数据库，以及 clangd 的模块缓存和锁——然后重新开始。日志保留。之后的第一次会话是冷启动。 +它会停掉这个根目录的引擎，删除它的缓存——模型、引擎数据库、clangd 的模块缓存和锁，以及 mcppls 运行的构建工具（xmake、CMake、Meson）的私有状态（在下一次描述开始前删除，不会删在正在运行的描述底下）——然后重新开始。日志保留。之后的第一次会话是冷启动。 旧的模块会自动清理：每个单元保留它最新两条编译命令构建出的模块（BMI），更旧的在 clangd 启动时于后台删除。`mcppls cache --prune` 会对所有没有实例打开的工作区做同样的清理。 diff --git a/docs/zh-CN/92-release.md b/docs/zh-CN/92-release.md index 76c6d6c3..d2d55c2d 100644 --- a/docs/zh-CN/92-release.md +++ b/docs/zh-CN/92-release.md @@ -105,7 +105,20 @@ nightly 运行还多一个 `termux` job：termux 自己的用户空间和 PRoot 3. **合入并发布引擎**，再把锁文件切换到发布的归档（大小和 SHA-256，各自用两次下载和发布的 `.sha256` 核对），并在同一个提交里**删除候选文件**。CI 在发布的字节上再跑一次。 4. 合入本 PR 并发布。 -候选文件永远不会进入发布：CI 在 `main` 和发布的运行中拒绝它，Release 工作流发现它会立即停止。构建产物保留 14 天；更早的运行可以重新运行引擎 PR 的 CI 重建。 +候选文件永远不会进入发布：CI 在 `main` 和发布的运行中拒绝它，Release 工作流发现它会立即停止。在 `main` 以外的分支上手动运行 CI 或 nightly 时可以使用它：引擎对空闲内存或输入延迟的影响要在 nightly 的 ux 轮次里才看得出来，所以这些轮次要在引擎发布之前就能测到它。构建产物保留 14 天；更早的运行可以重新运行引擎 PR 的 CI 重建。 + +## 衡量一次改动 + +同样的字节，nightly 每次测出来的结果都会有起伏：runner 慢一些、时段忙一些都会影响。一轮红不等于回归，一轮绿也不等于修好了。手动运行 nightly（`gh workflow run nightly.yml --ref <分支> -f …`）可以在不重新构建的情况下衡量一次改动： + +| 输入 | 作用 | +|---|---| +| `payload-run` | 测试该分支某次 CI 运行（填它的 run id）构建好的 payload，不再重新构建：约 35 分钟而不是 80 分钟，测的就是 PR 的 CI 测过的那份字节 | +| `scope` | `all`（所有任务）或 `ux`（只跑 ux 轮次、预算汇总和对比） | +| `ux-repeat` | 每一轮把 edits 阶段（输入、保存、U5、U6、U7a、U15）跑这么多次，每次一个新的 server 会话，复用 cold 和 warm 阶段准备好的东西：填 4，三轮合计每项 12 个样本，只构建一次 | +| `ux-baseline` | 一个已发布的版本（`v0.0.16`），它的 server 在同一批机器上跑同样的阶段，用自己的工作区和缓存，与本版本按 A B B A 交替；`ux budgets` 随后输出 `mcppls-devtools measure compare`：输入、保存、状态变化、空闲和超时这几项数字并排列出，中位数比基线差、且超出两者波动范围的会被标出 | + +两个 server 用的 runner、fixture 和预算都是该分支的，所以唯一的差别就是 server。 ## 核实一次发布 diff --git a/editors/zed/tests/lsp_frames.py b/editors/zed/tests/lsp_frames.py index e6fbf241..21369ab2 100755 --- a/editors/zed/tests/lsp_frames.py +++ b/editors/zed/tests/lsp_frames.py @@ -11,11 +11,16 @@ didopen from `textDocument/didOpen` diagnostics from `textDocument/publishDiagnostics` +With `--directory-workspace`, print no facts and return success only when a complete initialize +names existing directories in workspaceFolders (or rootUri if folders are empty). + A frame still being written at the end of the file is ignored; it shows up on the next read. """ import json +import pathlib import re import sys +import urllib.parse def frames(data): @@ -38,6 +43,21 @@ def frames(data): def main(): with open(sys.argv[1], "rb") as handle: data = handle.read() + if len(sys.argv) > 2 and sys.argv[2] == "--directory-workspace": + for message in frames(data): + if message.get("method") != "initialize": + continue + params = message.get("params") or {} + uris = [entry.get("uri", "") for entry in params.get("workspaceFolders") or []] + if not uris: + uris = [params.get("rootUri", "")] + + def directory(uri): + parsed = urllib.parse.urlparse(uri) + return parsed.scheme == "file" and pathlib.Path(urllib.parse.unquote(parsed.path)).is_dir() + + return 0 if uris and all(directory(uri) for uri in uris) else 1 + return 1 for message in frames(data): method = message.get("method") params = message.get("params") or {} @@ -47,6 +67,8 @@ def main(): print("didopen", (params.get("textDocument") or {}).get("uri", "")) elif method == "textDocument/publishDiagnostics": print("diagnostics", params.get("uri", ""), len(params.get("diagnostics") or [])) + return 0 -main() +if __name__ == "__main__": + sys.exit(main()) diff --git a/editors/zed/tests/smoke.sh b/editors/zed/tests/smoke.sh index 43a9badf..e8d628b3 100755 --- a/editors/zed/tests/smoke.sh +++ b/editors/zed/tests/smoke.sh @@ -21,7 +21,8 @@ # # recommended `"language_servers": ["mcppls", "!clangd"]` in the user's settings (Z-2): within # 60 s mcppls has been initialized by "Zed", been sent the file and published -# diagnostics for it, and Zed started no clangd of its own +# diagnostics for it, its own clangd completed initialization and remains alive, +# and Zed started no clangd of its own (this does not await every module BMI) # default Zed's own settings (Z-3): mcppls answers all the same, and Zed's clangd is started # beside it; that both ran is recorded, because it is what the README tells users # to turn off @@ -32,8 +33,8 @@ # ZED_EDITOR, MCPPLS_PAYLOAD, MCPPLS_DEVTOOLS required # ZED_SMOKE_DIR scratch directory (default: a new temporary one, removed on success) # ZED_SMOKE_BUDGET seconds for mcppls to answer (default 60) -# ZED_SMOKE_REQUIRE_CLANGD=1 `default` needs Zed's clangd running, not just started: Zed -# downloads it from GitHub, which a machine without network cannot do +# ZED_SMOKE_REQUIRE_CLANGD=1 `default` needs Zed's clangd running, not just attempted; +# CI preinstalls it on PATH using Zed's default binary lookup set -euo pipefail mode=${1:-recommended} @@ -151,6 +152,25 @@ has_client() { zed_said | grep -qx 'client Zed'; } has_didopen() { zed_said | grep -q "^didopen .*/$file\$"; } has_diagnostics() { mcppls_said | grep -q "^diagnostics .*/$file "; } +# Inspect the actual initialize, not just the command line that was meant to open a directory. +# Zed can otherwise make the first file a single-file worktree before opening its parent folder. +has_directory_workspace() { + python3 "$here/lsp_frames.py" "$root/lsp-in.log" --directory-workspace +} + +# Initialization is proved by the engine's own protocol log and a live direct child of mcppls, +# not by module-layer diagnostics or Zed's separate clangd. Module preparation may still continue. +has_mcppls_engine() { + local pid + grep -qsE 'clangd \(.*\): .*<-- initialized' "$root"/cache/logs/server-*.log || return 1 + for pid in $(server_pids); do + if ps -eo ppid=,comm= | awk -v parent="$pid" '$1 == parent && $2 ~ /^clangd/ { found=1 } END { exit !found }'; then + return 0 + fi + done + return 1 +} + # --------------------------------------------------------------------------------------------- # The extension, installed the way a release is checked: linked into Zed's data directory, the # payload put where the extension looks for it. @@ -190,11 +210,12 @@ say "starting Zed ($mode settings) on $workspace" XDG_CACHE_HOME="$root/home/.cache" XDG_RUNTIME_DIR="$root/run" SHELL=/bin/bash \ ZED_ALLOW_EMULATED_GPU=1 MCPPLS_CACHE_DIR="$root/cache" MCPPLS_LOG_LEVEL=debug \ PATH="$root/bin:$PATH" - exec "$editor" "$workspace/$file" "$workspace" + exec "$editor" "$workspace" "$workspace/$file" ) > "$root/zed.out" 2>&1 & zed_pid=$! wait_for 'mcppls is initialized by a client named "Zed"' has_client +wait_for 'the actual LSP workspace folders name directories' has_directory_workspace wait_for "mcppls is sent $file (didOpen)" has_didopen wait_for "mcppls publishes diagnostics for $file" has_diagnostics @@ -204,6 +225,7 @@ has_log() { ls "$root"/cache/logs/server-*.log >/dev/null 2>&1; } has_debug_log() { grep -qs '\[debug\]' "$root"/cache/logs/server-*.log; } wait_for "mcppls wrote its log under MCPPLS_CACHE_DIR" has_log wait_for "the log is at debug level (--log-level debug reached the server)" has_debug_log +wait_for "mcppls's own clangd completed initialization and is alive" has_mcppls_engine mcppls_pid=$(server_pids | head -1) test -n "$mcppls_pid" || fail "no mcppls process while Zed is showing its answers" @@ -218,7 +240,7 @@ case "$mode" in summary "Zed ($mode settings): mcppls (pid $mcppls_pid) answered within $((SECONDS - started)) s; Zed started no clangd" ;; default) - # Zed downloads clangd on first use (tens of megabytes from GitHub) before it starts it. + # Zed first checks PATH (preinstalled in CI), then downloads if none is usable. clangd_deadline=$((SECONDS + 240)) clangd='' until clangd=$(zed_clangd_pid); do diff --git a/modules/base/src/version.cppm b/modules/base/src/version.cppm index b3cccf9a..b3a3db66 100644 --- a/modules/base/src/version.cppm +++ b/modules/base/src/version.cppm @@ -11,7 +11,7 @@ export namespace mcppls::base { // the S1 profile version is spec::PROFILE_VERSION, the kit manifest version is spec::KIT_VERSION. inline constexpr std::string_view VERSION { "0.0.17" }; // The clangd the payload ships (mcppls-clangd). Checked against packaging/payload.lock.json by the same command. -inline constexpr std::string_view CLANGD_VERSION { "23.1.0-mcppls.4" }; +inline constexpr std::string_view CLANGD_VERSION { "23.1.0-mcppls.5" }; // Maintained-engine identity stays intact in logs and caches. Its LLVM base // decides libc++ kit compatibility; unknown vendor suffixes remain unchanged. inline std::string_view llvm_base_version(std::string_view engineVersion) { diff --git a/packaging/README.md b/packaging/README.md index 5abfaf81..671dc2a2 100644 --- a/packaging/README.md +++ b/packaging/README.md @@ -55,7 +55,7 @@ and the kit installed by xlings. | Entry | What | Used by | |---|---|---| -| `mcppls-clangd-` | the maintained engine, mcppls-clangd 0.0.16's `clangd-23.1.0-mcppls.4-.tar.gz`: clangd, its builtin headers, licenses and `engine.json` (source and binary identity), taken byte for byte | `mcppls.pack.clangd` | +| `mcppls-clangd-` | the maintained engine, mcppls-clangd 0.0.17's `clangd-23.1.0-mcppls.5-.tar.gz`: clangd, its builtin headers, licenses and `engine.json` (source and binary identity), taken byte for byte | `mcppls.pack.clangd` | | `llvm-project-src` | llvm-project 23.1.0 source archive | `mcppls.pack.kit`, recipe `libcxx-source` | | `llvm-mingw` | llvm-mingw 20260826 (LLVM 23.1.0), UCRT, Linux x86_64 host | `mcppls.pack.kit`, recipe `llvm-mingw` | diff --git a/packaging/payload.lock.json b/packaging/payload.lock.json index e89fcd91..67a50c3e 100644 --- a/packaging/payload.lock.json +++ b/packaging/payload.lock.json @@ -1,35 +1,35 @@ { "lock-version": 1, "comment": "Every upstream input of the bundled payload, and the one table of platforms. Change versions here only; CI fetches, verifies and assembles. Each sha256 was checked against two independent downloads and against the digest the release publishes beside the asset. The server version is read from mcpp.toml. A platform is named - as VS Code names extension targets; the VS Code extension, the release manifest and CI's matrices list the same platforms, which `mcppls-devtools check platforms` holds them to.", - "clangd-version": "23.1.0-mcppls.4", + "clangd-version": "23.1.0-mcppls.5", "libcxx-version": "23.1.0", "entries": { "mcppls-clangd-linux-x64": { - "url": "https://github.com/Sunrisepeak/mcppls-clangd/releases/download/v0.0.16/clangd-23.1.0-mcppls.4-linux-x64.tar.gz", - "file": "clangd-23.1.0-mcppls.4-linux-x64.tar.gz", - "size": 20957975, - "sha256": "56960d9316d413cbfe116b9a902021bbfd564d7178b6d165eacf2daeeb9eb65f", + "url": "https://github.com/Sunrisepeak/mcppls-clangd/releases/download/v0.0.17/clangd-23.1.0-mcppls.5-linux-x64.tar.gz", + "file": "clangd-23.1.0-mcppls.5-linux-x64.tar.gz", + "size": 20958477, + "sha256": "b2625e139b049fe7f05abefe013eaafc8e02c4000c5fc9b10ca96cbed48c3641", "license": "Apache-2.0 WITH LLVM-exception" }, "mcppls-clangd-linux-arm64": { - "url": "https://github.com/Sunrisepeak/mcppls-clangd/releases/download/v0.0.16/clangd-23.1.0-mcppls.4-linux-arm64.tar.gz", - "file": "clangd-23.1.0-mcppls.4-linux-arm64.tar.gz", - "size": 19905490, - "sha256": "c5814260956e4c1e232b42bdca736da5738d88a709472fb9e97baf0fd4faedf8", + "url": "https://github.com/Sunrisepeak/mcppls-clangd/releases/download/v0.0.17/clangd-23.1.0-mcppls.5-linux-arm64.tar.gz", + "file": "clangd-23.1.0-mcppls.5-linux-arm64.tar.gz", + "size": 19905303, + "sha256": "86bdb8e9da8b7793c2b1b8ce4b0f6088068d98ad2399c9644ec5492eed66e1a8", "license": "Apache-2.0 WITH LLVM-exception" }, "mcppls-clangd-darwin-arm64": { - "url": "https://github.com/Sunrisepeak/mcppls-clangd/releases/download/v0.0.16/clangd-23.1.0-mcppls.4-darwin-arm64.tar.gz", - "file": "clangd-23.1.0-mcppls.4-darwin-arm64.tar.gz", - "size": 19860020, - "sha256": "aacbef6d7cbc8f12adfeeb6c1834bc5ba62361461e546f6c45c2cb80e619a839", + "url": "https://github.com/Sunrisepeak/mcppls-clangd/releases/download/v0.0.17/clangd-23.1.0-mcppls.5-darwin-arm64.tar.gz", + "file": "clangd-23.1.0-mcppls.5-darwin-arm64.tar.gz", + "size": 19859900, + "sha256": "b074da48002393e8f288b61cd0aaf357dccc6d5c84ea11c253777e7ae2877dee", "license": "Apache-2.0 WITH LLVM-exception" }, "mcppls-clangd-win32-x64": { - "url": "https://github.com/Sunrisepeak/mcppls-clangd/releases/download/v0.0.16/clangd-23.1.0-mcppls.4-win32-x64.tar.gz", - "file": "clangd-23.1.0-mcppls.4-win32-x64.tar.gz", - "size": 20240575, - "sha256": "701ec54edf0b8edaa58e7e68c887025714e90c0b21197a1b62c40c4b3d90a78a", + "url": "https://github.com/Sunrisepeak/mcppls-clangd/releases/download/v0.0.17/clangd-23.1.0-mcppls.5-win32-x64.tar.gz", + "file": "clangd-23.1.0-mcppls.5-win32-x64.tar.gz", + "size": 20242765, + "sha256": "db89c00263be56d23fee0b1efd343073f6ed6dd3bacad9263e3cd71ad00dc5cd", "license": "Apache-2.0 WITH LLVM-exception" }, "llvm-project-src": { diff --git a/src/bin/conformance.cpp b/src/bin/conformance.cpp index 1ccc8aa1..b1378a00 100644 --- a/src/bin/conformance.cpp +++ b/src/bin/conformance.cpp @@ -264,6 +264,48 @@ bool run_prepare(const Json& command, const std::string& workspace, bool verbose return true; } +// A dotted version's numbers ("2026.10.5.2"), compared number by number; empty when it has none. +std::vector version_numbers(std::string_view text) { + std::vector numbers; + for (const auto part : std::views::split(text, '.')) { + long value { 0 }; + const std::string_view piece { part.begin(), part.end() }; + const auto [end, error] = std::from_chars(piece.data(), piece.data() + piece.size(), value); + if (error != std::errc {} || end != piece.data() + piece.size()) return {}; + numbers.push_back(value); + } + return numbers; +} + +// "requires-mcpp": {"executable": "{env:MCPPLS_REAL_MCPP|mcpp}", "at-least": "2026.10.5.2"}: a fixture that runs a real +// mcpp says which release it needs, and gets that sentence instead of the compiler errors an older mcpp gives in the +// middle of its prepare steps. Empty when the mcpp asked is recent enough, else why the fixture cannot run. +std::string missing_mcpp(const Json& requirement, const Expansion& expansion) { + if (!requirement.is_object()) return {}; + const std::string wanted { requirement.value("at-least", std::string {}) }; + std::string program { expand(requirement.value("executable", std::string { "mcpp" }), expansion) }; + if (program.empty()) program = "mcpp"; + if (!base::is_absolute_path(program)) { + auto found = mcppls::platform::env::find_executable(program); + if (!found) return std::format("the fixture needs mcpp {} or later, and {} is not on PATH", wanted, program); + program = *found; + } + mcppls::platform::SpawnOptions options; + options.program = program; + options.arguments = { "--version" }; + auto result = mcppls::platform::run(std::move(options), std::chrono::seconds { 60 }); + if (!result || result->exitCode != 0 || result->timedOut) return std::format("the fixture needs mcpp {} or later, and {} --version failed", wanted, program); + // "mcpp 2026.10.11.1": the last word of the first line. + std::string_view line { result->output }; + line = line.substr(0, line.find_first_of("\r\n")); + const std::string_view version { line.substr(line.find_last_of(' ') == std::string_view::npos ? 0 : line.find_last_of(' ') + 1) }; + const auto have = version_numbers(version), need = version_numbers(wanted); + if (have.empty() || need.empty() || std::ranges::lexicographical_compare(have, need)) { + return std::format("the fixture needs mcpp {} or later, and {} is {}", wanted, program, version.empty() ? std::string_view { "of no version" } : version); + } + return {}; +} + // A file's size and FNV-1a digest, read a block at a time: a build tree holds files far larger than a check needs to keep. std::optional digest(const std::string& path) { std::ifstream stream { std::filesystem::path { path }, std::ios::binary }; @@ -2791,6 +2833,49 @@ class Scenario { } } + // Freeze the same response population used above before a diagnostic probe or post-check + // report can deliver late answers. This observes existing requests, without pumping the + // client, asking anything, or reading another clock. JSON is built only after sampling. + constexpr std::size_t COMPLETION_TRACE_LIMIT { 256 }; + struct CompletionTraceEntry { + std::size_t index; + std::int64_t id; + Clock::time_point sent; + std::optional answered; + bool error; + std::optional items; + std::optional incomplete; + }; + struct CompletionTrace { + std::size_t total; + std::vector entries; + std::optional slowestSuccessful; + std::optional slowestAnswered; + }; + const auto capture_trace = [&](const std::vector& ids) { + CompletionTrace trace { ids.size(), {}, std::nullopt, std::nullopt }; + trace.entries.reserve(std::min(ids.size(), COMPLETION_TRACE_LIMIT)); + for (std::size_t index { 0 }; index < ids.size(); ++index) { + const auto& request { client_.asyncRequests.at(ids[index]) }; + const Json& result { request.result }; + const Json* items { result.is_array() ? &result : result.is_object() ? lsp::find(result, "items") : nullptr }; + const Json* incomplete { result.is_object() ? lsp::find(result, "isIncomplete") : nullptr }; + CompletionTraceEntry entry { index, ids[index], request.sent, request.answered, request.isError, + items != nullptr && items->is_array() ? std::optional { items->size() } : std::nullopt, + incomplete != nullptr && incomplete->is_boolean() ? std::optional { incomplete->get() } : std::nullopt }; + if (trace.entries.size() < COMPLETION_TRACE_LIMIT) trace.entries.push_back(entry); + if (!entry.answered) continue; + const auto slower = [&](const std::optional& previous) { + return !previous || *entry.answered - entry.sent > *previous->answered - previous->sent; + }; + if (slower(trace.slowestAnswered)) trace.slowestAnswered = entry; + if (!entry.error && slower(trace.slowestSuccessful)) trace.slowestSuccessful = entry; + } + return trace; + }; + const CompletionTrace completionTrace { capture_trace(completions) }; + const CompletionTrace importerTrace { capture_trace(importerCompletions) }; + // Diagnostics after the typing stops: a real mistake is typed at once and how long its diagnostic takes is timed. std::optional refresh; if (const auto probe = check.find("diagnostic-probe"); probe != check.end() && probe->is_object()) { @@ -2896,6 +2981,97 @@ class Scenario { { "restarts", restarts }, { "filesSetAside", setAside }, { "diagnosticsRefresh", refresh ? Json(*refresh) : Json(nullptr) }, { "statesSeen", statesSeen }, { "doomed", doomed }, { "moduleFailedDiagnostics", moduleFailedDiagnostics }, { "stalledSamples", stalledSamples }, { "firstOpenSeconds", firstOpen ? Json(*firstOpen) : Json(nullptr) } }; + const auto trace_json = [&](const CompletionTrace& trace) { + const auto nanoseconds_since = [&](Clock::time_point at) { + return std::chrono::duration_cast(at - began).count(); + }; + const auto entry_json = [&](const CompletionTraceEntry& entry) { + return Json { { "index", entry.index }, { "id", entry.id }, { "sentNanoseconds", nanoseconds_since(entry.sent) }, + { "answeredNanoseconds", entry.answered ? Json(nanoseconds_since(*entry.answered)) : Json(nullptr) }, + { "elapsedNanoseconds", entry.answered ? Json(std::chrono::duration_cast(*entry.answered - entry.sent).count()) : Json(nullptr) }, + { "elapsedSeconds", entry.answered ? Json(seconds_between(entry.sent, *entry.answered)) : Json(nullptr) }, + { "error", entry.error }, { "items", entry.items ? Json(*entry.items) : Json(nullptr) }, + { "isIncomplete", entry.incomplete ? Json(*entry.incomplete) : Json(nullptr) } }; + }; + Json entries = Json::array(); + for (const auto& entry : trace.entries) entries.push_back(entry_json(entry)); + return Json { { "origin", "typing-start" }, { "population", "original-completion-statistics" }, + { "limit", COMPLETION_TRACE_LIMIT }, { "total", trace.total }, { "dropped", trace.total - trace.entries.size() }, + { "entries", std::move(entries) }, + // These are separate from the ordered prefix: a truncated trace never hides its + // slowest answer, and the successful one reproduces the original latency max. + { "slowestSuccessful", trace.slowestSuccessful ? entry_json(*trace.slowestSuccessful) : Json(nullptr) }, + { "slowestAnswered", trace.slowestAnswered ? entry_json(*trace.slowestAnswered) : Json(nullptr) } }; + }; + summary["completionTrace"] = trace_json(completionTrace); + // These are the reports already taken by the check. Keep the method's public + // counters/distributions at both boundaries: its recent engine samples can roll out + // before a later timeline report. They are cumulative/recent, not per-check percentiles. + const auto completion_stats = [](const Json& root) -> Json { + const Json* stats { lsp::find_path(root, { "requests", "textDocument/completion" }) }; + if (stats == nullptr || !stats->is_object()) return nullptr; + Json snapshot = Json::object(); + for (const std::string_view key : { "count", "empty", "errors", "cancelled", "p50Ms", "p95Ms", "maxMs", "engineAnswered", + "engineP50Ms", "engineP95Ms", "overheadP50Ms", "overheadP95Ms" }) { + const Json* value { lsp::find(*stats, key) }; + snapshot[std::string { key }] = value != nullptr && value->is_number() ? *value : Json(nullptr); + } + const Json* lastAt { lsp::find(*stats, "lastAt") }; + snapshot["lastAt"] = lastAt != nullptr && lastAt->is_string() ? *lastAt : Json(nullptr); + const Json* by { lsp::find(*stats, "answeredBy") }; + snapshot["answeredBy"] = nullptr; + if (by != nullptr && by->is_object()) { + snapshot["answeredBy"] = Json::object(); + for (const auto& item : by->items()) if (item.value().is_number()) snapshot["answeredBy"][item.key()] = item.value(); + } + return snapshot; + }; + const auto slow_total = [](const Json& root) -> std::optional { + const Json* value { lsp::find_path(root, { "eventTotals", "slow-request" }) }; + if (value == nullptr || !value->is_number_integer() || value->get() < 0) return std::nullopt; + return value->get(); + }; + const auto slowBefore { slow_total(before) }, slowAfter { slow_total(after) }; + constexpr std::size_t SLOW_TRACE_LIMIT { 256 }; + const Json* events { lsp::find(after, "events") }; + const bool eventsAvailable { events != nullptr && events->is_array() }; + std::size_t recentSlowCount { 0 }; + Json slowEvents = Json::array(); + if (eventsAvailable) { + for (const auto& event : *events) { + if (!event.is_object() || event.value("kind", std::string {}) != "slow-request") continue; + ++recentSlowCount; + Json entry = Json::object(); + const Json* at { lsp::find(event, "at") }; + entry["at"] = at != nullptr && at->is_string() ? *at : Json(nullptr); + for (const std::string_view field : { "method", "ms", "outcome", "answeredBy", "queueMs", "engineMs", "answerMs" }) { + const Json* value { lsp::find_path(event, { "detail", field }) }; + const bool number { field.ends_with("ms") || field.ends_with("Ms") }; + entry[std::string { field }] = value != nullptr && (number ? value->is_number() : value->is_string()) ? *value : Json(nullptr); + } + if (slowEvents.size() == SLOW_TRACE_LIMIT) slowEvents.erase(slowEvents.begin()); + slowEvents.push_back(std::move(entry)); + } + } + const bool totalsComparable { slowBefore && slowAfter && *slowAfter >= *slowBefore }; + const auto added { totalsComparable ? std::optional { *slowAfter - *slowBefore } : std::nullopt }; + // Recent events are chronological suffixes of the cumulative journal. Counters expose + // omitted new events; timestamps alone cannot establish that every slow request survived. + const bool coverageKnown { added && eventsAvailable && recentSlowCount <= *slowAfter }; + const std::uint64_t retainedNew { coverageKnown ? std::min(*added, static_cast(slowEvents.size())) : 0 }; + summary["serverCompletionEvidence"] = Json { + { "requestsBefore", completion_stats(before) }, { "requestsAfter", completion_stats(after) }, + { "recentSampleLimit", 256 }, { "statisticsScope", "cumulative counters and recent samples; not per-check percentile deltas" }, + { "slowRequests", Json { + { "totalBefore", slowBefore ? Json(*slowBefore) : Json(nullptr) }, { "totalAfter", slowAfter ? Json(*slowAfter) : Json(nullptr) }, + { "added", added ? Json(*added) : Json(nullptr) }, { "limit", SLOW_TRACE_LIMIT }, + { "afterRecentCount", eventsAvailable ? Json(recentSlowCount) : Json(nullptr) }, + { "droppedByTraceLimit", eventsAvailable ? Json(recentSlowCount - slowEvents.size()) : Json(nullptr) }, + { "newRetained", coverageKnown ? Json(retainedNew) : Json(nullptr) }, + { "newNotRetained", coverageKnown ? Json(*added - retainedNew) : Json(nullptr) }, + { "eventsAfter", eventsAvailable ? std::move(slowEvents) : Json(nullptr) } + } } + }; if (!reask.empty()) { summary["wordTrace"] = std::move(wordTrace); summary["wordTraceLimit"] = WORD_TRACE_LIMIT; @@ -2904,7 +3080,8 @@ class Scenario { std::string importerBrief; if (withImporter) { summary["importer"] = Json { { "file", importerFile }, { "completions", importerCompletions.size() }, { "answered", importerAnswered.size() }, - { "empty", importerEmpty }, { "completion", importerLatency }, { "moduleFailedDiagnostics", importerModuleFailed } }; + { "empty", importerEmpty }, { "completion", importerLatency }, { "completionTrace", trace_json(importerTrace) }, + { "moduleFailedDiagnostics", importerModuleFailed } }; importerBrief = std::format(", importer {} completions p95 {:.2f}s {} empty {} module-failed", importerCompletions.size(), importerLatency["p95"].get(), importerEmpty, importerModuleFailed); } @@ -3434,22 +3611,56 @@ class Scenario { // State changes in the busiest minute, and the times the status was `error`. std::vector changes; + std::vector changeTraceIndices; + Json transitionTrace = Json::array(); + const auto analyzedSamples { samples.size() }; // root_report below may pump later statuses; these alone were counted std::string previous; int errors { 0 }; - for (const auto& sample : samples) { + for (std::size_t i { 0 }; i < samples.size(); ++i) { + const auto& sample = samples[i]; + const bool changed { !previous.empty() && sample.state != previous && sample.state != "restart" }; + // Only initial states, reset boundaries and counted changes: no full status or paths. + // Integer timestamps preserve the strict <60s boundary without JSON-double rounding. + if (previous.empty() || changed || sample.state == "restart") { + transitionTrace.push_back(Json { + { "sampleIndex", i }, { "seconds", seconds_between(begin_, sample.at) }, + { "sinceStartNanoseconds", std::chrono::duration_cast(sample.at - begin_).count() }, + { "kind", sample.state == "restart" ? "restart" : changed ? "change" : "initial" }, + { "from", previous.empty() ? Json(nullptr) : Json(previous) }, { "to", sample.state }, { "counted", changed }, + { "done", sample.done }, { "total", sample.total }, { "issues", sample.issues } + }); + } if (sample.state == "restart") { previous.clear(); continue; } if (sample.state == "error") ++errors; - if (!previous.empty() && sample.state != previous) changes.push_back(sample.at); + if (changed) { + changes.push_back(sample.at); + changeTraceIndices.push_back(transitionTrace.size() - 1); + } previous = sample.state; } std::size_t busiest { 0 }; + std::size_t busiestBegin { 0 }; for (std::size_t i { 0 }; i < changes.size(); ++i) { std::size_t inWindow { 0 }; for (std::size_t j { i }; j < changes.size() && changes[j] - changes[i] < std::chrono::seconds { 60 }; ++j) ++inWindow; - busiest = std::max(busiest, inWindow); + if (inWindow > busiest) { busiest = inWindow; busiestBegin = i; } + } + Json busiestWindow = nullptr; + if (busiest > 0) { + Json support = Json::array(); + for (std::size_t i { busiestBegin }; i < busiestBegin + busiest; ++i) support.push_back(changeTraceIndices[i]); + const auto start { changes[busiestBegin] }; + busiestWindow = Json { + { "startSeconds", seconds_between(begin_, start) }, + { "endSecondsExclusive", seconds_between(begin_, start + std::chrono::seconds { 60 }) }, + { "startNanoseconds", std::chrono::duration_cast(start - begin_).count() }, + { "endNanosecondsExclusive", std::chrono::duration_cast(start + std::chrono::seconds { 60 } - begin_).count() }, + { "lastChangeSeconds", seconds_between(begin_, changes[busiestBegin + busiest - 1]) }, + { "count", busiest }, { "traceIndices", std::move(support) } + }; } // Ready, and prepared: the first `ready` that held, and the last one after a preparation, from the start of the run. @@ -3491,6 +3702,8 @@ class Scenario { else enforce_max(check, "preparedSeconds", prepared, "preparation finished at", "s", failures); } Json summary { { "statuses", samples.size() }, { "longestStallSeconds", longestStall }, { "stateChanges", changes.size() }, { "busiestMinuteChanges", busiest }, + { "stateTransitionTrace", std::move(transitionTrace) }, { "stateTransitionStatusSamples", analyzedSamples }, + { "stateTransitionObservedThroughSeconds", seconds_between(begin_, now) }, { "busiestMinuteWindow", std::move(busiestWindow) }, { "errorStates", errors }, { "clangdRestarts", root.is_object() ? Json(clangdRestarts) : Json(nullptr) }, { "firstReadySeconds", firstReady ? Json(*firstReady) : Json(nullptr) }, { "preparedSeconds", prepared ? Json(*prepared) : Json(nullptr) }, { "serverRestarts", client_.restartMarks.size() }, @@ -3536,6 +3749,10 @@ class Scenario { // ---- resources ---- + // A server that does not report its background work (one before 0.0.17, run beside this one as a + // baseline) never will: waiting out the bound for it only delays the same verdict. + static constexpr std::string_view WORK_NOT_REPORTED { "the engine does not report its background work state" }; + // Eligibility is task state, never a low CPU sample chosen to pass the idle CPU budget. // In particular an empty implementation queue before its delayed full sweep is not idle. static std::string background_work(const Json& root) { @@ -3555,7 +3772,7 @@ class Scenario { || !implementations.contains("sweepComplete") || !implementations.contains("sweepScheduled") || !implementations.contains("queued") || !implementations.contains("running") || !index.contains("active") || !details.contains("pendingRequests") || !details.contains("filesAwaitingDiagnostics") - || !details.contains("deferredRequests") || !details.contains("heldRequests")) return "the engine does not report its background work state"; + || !details.contains("deferredRequests") || !details.contains("heldRequests")) return std::string { WORK_NOT_REPORTED }; if (!details.value("handshakeDone", false) || !details.value("accepting", false) || details.value("restartScheduled", false)) return "the engine is starting or waiting for a restart"; if (preparation.value("running", 0L) > 0 || preparation.value("done", 0L) < preparation.value("wanted", 0L)) return "module preparation is unfinished"; @@ -3584,7 +3801,7 @@ class Scenario { do { result.report = root_report(true, std::chrono::seconds { 2 }); result.unfinished = background_work(result.report); - if (result.unfinished.empty() || Clock::now() >= deadline) break; + if (result.unfinished.empty() || result.unfinished == WORK_NOT_REPORTED || Clock::now() >= deadline) break; client_.pump_until(std::min(deadline, Clock::now() + std::chrono::seconds { 2 })); } while (Clock::now() < deadline); const auto until { Clock::now() }; @@ -4017,6 +4234,41 @@ class Scenario { } while (Clock::now() < deadline); return { false, why }; } + if (kind == "run") { + // 0.0.17 XS-1 (#69): a program the user runs in the workspace while the server is up -- their own build -- exits + // with "exit" (0 unless given). What the server ran on the machine must not have broken it. + std::vector argv; + for (const auto& word : check.value("command", Json::array())) argv.push_back(expand(word.get())); + if (argv.empty()) return { false, "a run check names its \"command\"" }; + std::string program { argv.front() }; + if (!base::is_absolute_path(program)) { + const auto found = mcppls::platform::env::find_executable(program); + if (!found) return { false, std::format("{} is not on PATH", program) }; + program = *found; + } + mcppls::platform::SpawnOptions spawn; + spawn.program = program; + spawn.arguments.assign(argv.begin() + 1, argv.end()); + spawn.workDirectory = workspace_; + const auto bound = std::chrono::seconds { check.value("timeout", static_cast(timeout_.count())) }; + auto running = std::async(std::launch::async, [spawn, bound]() mutable { return mcppls::platform::run(std::move(spawn), bound); }); + while (running.wait_for(std::chrono::milliseconds { 200 }) != std::future_status::ready) client_.drain(std::chrono::milliseconds { 0 }); + auto result = running.get(); + if (!result) return { false, result.error().message }; + if (result->timedOut) return { false, std::format("{} did not finish in time", base::join(argv, " ")) }; + const int wantedExit { check.value("exit", 0) }; + if (result->exitCode != wantedExit) { + // What it said went wrong: its error lines, colours taken out, or else the end of what it printed. + const std::string printed { std::regex_replace(result->output + "\n" + result->error, std::regex { "\x1b\\[[0-9;]*m" }, "") }; + std::vector errors; + for (const auto line : base::split_lines(printed)) { + if (line.contains("error") && errors.size() < 4) errors.emplace_back(base::trim(line)); + } + return { false, std::format("{} exited {} instead of {}: {}", base::join(argv, " "), result->exitCode, wantedExit, + errors.empty() ? printed.substr(printed.size() - std::min(printed.size(), 600)) : base::join(errors, " | ")) }; + } + return { true, std::format("{} exited {}", base::join(argv, " "), result->exitCode) }; + } if (kind == "cli") { // S5 7: a command of the query entries, run to completion in the workspace; its standard output is // one JSON document meeting the expectations, and it exits with "exit" (0 unless given). @@ -4322,6 +4574,11 @@ class Scenario { } if (check.contains("count") && calls.size() != check["count"].get()) return { false, std::format("{} calls instead of {}", calls.size(), check["count"].get()) }; + // "min-count" / "max-count": a bound instead of an exact number, for runs whose count is timing's (0.0.17 W4). + if (check.contains("min-count") && calls.size() < check["min-count"].get()) + return { false, std::format("{} calls, fewer than {}", calls.size(), check["min-count"].get()) }; + if (check.contains("max-count") && calls.size() > check["max-count"].get()) + return { false, std::format("{} calls, more than {}", calls.size(), check["max-count"].get()) }; if (calls.empty()) return { check.contains("count") && check["count"].get() == 0, "no matching producer calls" }; if (const auto expected = check.find("expect"); expected != check.end()) { const auto [held, why] = expectations_hold(calls, *expected); @@ -5046,6 +5303,10 @@ int run(Options options) { } prepareEnvironment = prepare_environment(options.msvcEnvironment); } + if (const auto missing = missing_mcpp(scenario.value("requires-mcpp", Json {}), expansion); !missing.empty()) { + say("FAIL requires-mcpp: {}", missing); + return 1; + } if (!alreadyPrepared) { for (const auto& command : scenario.value("prepare", Json::array())) { if (!run_prepare(command, workspace, options.verbose, expansion, prepareEnvironment)) return 1; @@ -5799,21 +6060,31 @@ int prepare_clangd_cannot_load() { // Fix plan F3: a clangd that crashes the way clangd 23.1 did on Windows in issue #23 -- its crash context on standard // error, naming a file that is not the one being edited, and then gone -- once, 25 seconds after it starts; every // later start is the payload's real clangd. POSIX only: the stand-in is a shell script around the real one. -int prepare_clangd_crash_context(const std::string& payload) { +// `crashFile` (relative to the workspace) is the file the context names; empty, the stand-in prints no context and is +// gone the way a clangd killed from outside is (0.0.17 W5: clangd-exit-strike). +int prepare_crashing_clangd(std::string_view kind, const std::string& payload, std::string_view crashFile) { if constexpr (mcppls::os::FAMILY == mcppls::os::Family::windows) { - say("clangd-crash-context: POSIX only"); + say("{}: POSIX only", kind); return 2; } const std::string workspace { fs::current_directory() }; const std::string real { base::join_path(payload, "clangd/bin/clangd") }; if (!fs::exists(real)) { - say("clangd-crash-context: no clangd at {}", real); + say("{}: no clangd at {}", kind, real); return 1; } const std::string directory { base::join_path(workspace, "stand-in") }; (void)fs::create_directories(directory); const std::string crashed { base::join_path(directory, "crashed-once") }; - const std::string crasher { base::join_path(workspace, "src/crasher.cpp") }; + const std::string crasher { crashFile.empty() ? std::string {} : base::join_path(workspace, crashFile) }; + const std::string context { crasher.empty() ? std::string {} : std::format( + "echo 'PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.' >&2\n" + "echo 'Signalled during AST worker action: Build AST' >&2\n" + "echo ' Filename: {}' >&2\n" + "echo ' Directory: {}' >&2\n" + "echo ' Command Line: clang++ -std=c++23 -c -- {}' >&2\n" + "echo ' Version: 1' >&2\n", + crasher, workspace, crasher) }; const std::string script { std::format( "#!/bin/sh\n" "real='{}'\n" @@ -5829,22 +6100,17 @@ int prepare_clangd_crash_context(const std::string& payload) { // on the slowest Intel macOS runners, which killed the stand-in before its designed crash). "end=$(( $(date +%s) + 25 ))\n" "while [ $(date +%s) -lt $end ]; do i=0; while [ $i -lt 5000 ]; do i=$((i+1)); done; done\n" - "echo 'PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.' >&2\n" - "echo 'Signalled during AST worker action: Build AST' >&2\n" - "echo ' Filename: {}' >&2\n" - "echo ' Directory: {}' >&2\n" - "echo ' Command Line: clang++ -std=c++23 -c -- {}' >&2\n" - "echo ' Version: 1' >&2\n" + "{}" "kill -KILL $child\n" - "exit 139\n", - real, crashed, crashed, crasher, workspace, crasher) }; + "exit {}\n", + real, crashed, crashed, context, crasher.empty() ? 137 : 139) }; const std::string clangd { base::join_path(directory, "clangd") }; if (auto written = fs::write_file(clangd, script); !written) { - say("clangd-crash-context: {}", written.error().message); + say("{}: {}", kind, written.error().message); return 1; } if (auto marked = fs::make_executable(std::vector { clangd }); !marked) { - say("clangd-crash-context: {}", marked.error().message); + say("{}: {}", kind, marked.error().message); return 1; } return 0; @@ -5926,13 +6192,15 @@ int prepare(const std::string& kind, const std::string& argument) { if (kind == "compdb-clangxx-msvc-std") return prepare_compdb_msvc_std(false); if (kind == "generated-module-old-mcpp") return prepare_generated_module_compdb(argument); if (kind == "clangd-cannot-load") return prepare_clangd_cannot_load(); - if (kind == "clangd-crash-context") return prepare_clangd_crash_context(argument); + if (kind == "clangd-crash-context") return prepare_crashing_clangd(kind, argument, "src/crasher.cpp"); + if (kind == "clangd-crash-closure") return prepare_crashing_clangd(kind, argument, "src/main.cpp"); + if (kind == "clangd-exit-strike") return prepare_crashing_clangd(kind, argument, ""); if (kind == "compdb-rejected-command") return prepare_compdb_rejected_command(argument); if (kind == "compdb-mixed-standards") return prepare_compdb_mixed_standards(argument); if (kind == "xmake-stale-compdb") return prepare_marked_compdb(kind, argument.empty() ? std::string { "g++" } : argument, "-DSTALE_COMPDB", "-std=c++17"); if (kind == "compdb-midwrite") return prepare_marked_compdb(kind, argument.empty() ? std::string { "clang++" } : argument, "-DVERSION_ONE", "-std=c++23"); if (kind == "compdb-lto-msvc") return prepare_compdb_lto_msvc(argument); - say("prepare: unknown fixture kind {} (heavy-headers, s1-two-sets, payload-corrupt, producer-candidate, delayed-producer, failure-at-base, compdb-clang-cl-std, compdb-clangxx-msvc-std, generated-module-old-mcpp, clangd-cannot-load, compdb-lto-msvc, clangd-crash-context, compdb-rejected-command, compdb-mixed-standards, xmake-stale-compdb, compdb-midwrite)", kind); + say("prepare: unknown fixture kind {} (heavy-headers, s1-two-sets, payload-corrupt, producer-candidate, delayed-producer, failure-at-base, compdb-clang-cl-std, compdb-clangxx-msvc-std, generated-module-old-mcpp, clangd-cannot-load, compdb-lto-msvc, clangd-crash-context, clangd-crash-closure, clangd-exit-strike, compdb-rejected-command, compdb-mixed-standards, xmake-stale-compdb, compdb-midwrite)", kind); return 2; } diff --git a/src/engine/clangd.cpp b/src/engine/clangd.cpp index a6ed0d12..3407baf0 100644 --- a/src/engine/clangd.cpp +++ b/src/engine/clangd.cpp @@ -481,8 +481,20 @@ class ClangdEngine final : public Engine { // Fix plan F16: set aside for what is on disk, not in the editor: it goes back when the disk is fixed. bool onDisk { false }; std::string modelOrigin; // the model's source then (fix plan F4: a provisional model's verdicts do not outlive it) + // 0.0.17 W5 (#59 U7a): set aside for an exit of clangd, the modules it imports as they were built then + // (closure_identity_). Once they are built again, and differently, it goes back to clangd once. + std::optional crashClosure; }; std::map> aside_; // path key + // W5: an exit of clangd that names no file (a kill, the OOM killer) is not yet a file's fault: the files it was busy + // with get a strike, and a file that gets a second within EXIT_STRIKE_WINDOW is set aside, as a crash naming it is. + static constexpr std::chrono::minutes EXIT_STRIKE_WINDOW { 10 }; + std::map> exitStrikes_; // path key -> its strike + // W5: how many times each module has been built (clangd's "Built module"), for the closure identity of a file set + // aside after an exit; and the identities each such file already went back to clangd with. + std::map> moduleBuilds_; + std::map, std::less<>> crashRetried_; // path key -> closure identities tried + bool closureRetryCheck_ { false }; std::map> fileStatus_; // client URI -> clangd's last textDocument/clangd.fileStatus state // robustness design C10: a definition that clangd finds only as a declaration in a module's interface is looked for in that // module's other units, which are opened in clangd without the editor (background units) and closed again when unused. @@ -1387,6 +1399,8 @@ class ClangdEngine final : public Engine { } else if (kind == "module-built") { const std::string module { event.value("module", std::string {}) }; ++moduleBuildCompletions_; + ++moduleBuilds_[module]; + closureRetryCheck_ = closureRetryCheck_ || !aside_.empty(); lastCompletedModule_ = module; lastModuleBuiltAt_ = Clock::now(); // #59 U15: clangd can build a whole closure before its prime TU completes. These @@ -1514,6 +1528,8 @@ class ClangdEngine final : public Engine { // So the status settles into ready or degraded on its own, not only when something else wakes // the event loop (real-project plan RP1.4, design P7). if (primer_.busy() && lastPrimeProgressAt_) consider(*lastPrimeProgressAt_ + options_.preparationStallTimeout); + // W5: a file set aside after an exit is looked at again once preparation is done with its rebuilt modules. + if (closureRetryCheck_) consider(Clock::now() + std::chrono::seconds { 1 }); for (const auto& [key, held] : held_) { if (const auto joined = joinedAt_.find(key); joined != joinedAt_.end()) consider(joined->second + DATABASE_REREAD); else if (!held.planned) consider(held.since + PLAN_PATIENCE); @@ -1637,6 +1653,7 @@ class ClangdEngine final : public Engine { } else if (restart) { request_restart_("clangd did not answer initialize"); } + if (closureRetryCheck_ && !primer_.busy()) retry_after_closure_rebuilt_(); for (const auto& key : quarantine_.due(now)) { // Nor is a file whose text on disk would stall clangd (fix plan F16): it waits for the disk, or the database. if (const auto aside = aside_.find(key); aside != aside_.end() && aside->second.onDisk) { @@ -2925,12 +2942,26 @@ class ClangdEngine final : public Engine { log::warning("clangd exited with {} ({})", *code, host_->root_directory()); } incident_("engine-crash", lastExit_, std::vector { suspects.begin(), suspects.end() }); + const auto now = Clock::now(); + std::erase_if(exitStrikes_, [&](const auto& item) { return now - item.second > EXIT_STRIKE_WINDOW; }); for (const auto& path : suspects) { + const std::string key { base::path_key(path) }; // A file whose text on disk is known to stall clangd (fix plan F16) is what crashed it on Windows, where // UP-01 crashes instead of spinning: it goes back when the disk is fixed, not when a crash's term is up. - const bool onDisk { diskHazards_.contains(base::path_key(path)) }; + const bool onDisk { diskHazards_.contains(key) }; + // W5 (#59 U9): an exit nothing names a file for was set aside for two minutes on the first time: a clangd + // killed while a file was being typed in took that file from clangd. Now the first is a strike. + if (!known && !onDisk && !exitStrikes_.contains(key)) { + exitStrikes_.emplace(key, now); + log::info("clangd exited while working on {} ({}); it stays with clangd, and a second exit within {} minutes sets it aside", + path, host_->root_directory(), EXIT_STRIKE_WINDOW.count()); + host_->record_event("exit-strike", Json { { "file", path } }); + continue; + } + exitStrikes_.erase(key); set_aside_(path, known ? std::format("clangd crashed while building it ({})", context->second.action) : std::string { "clangd exited while working on it" }, Reclaim::no, false, std::nullopt, onDisk); + if (const auto aside = aside_.find(key); aside != aside_.end() && !onDisk) aside->second.crashClosure = closure_identity_(key); } host_->status_changed(); } @@ -3473,6 +3504,60 @@ class ClangdEngine final : public Engine { // it is let finish for DISK_SETTLE, and clangd is restarted without the file if it has not by then. enum class Reclaim { no, if_busy, now, after_build }; + // W5: the modules a file imports, transitively (the plan's), each with how many times clangd has built it: what it + // was compiled against. Empty for a file that imports nothing the plan knows. + std::set closure_of_(std::string_view key) const { + std::set closure; + std::vector pending; + if (const auto imports = fileImports_.find(key); imports != fileImports_.end()) pending = imports->second; + while (!pending.empty()) { + std::string module { std::move(pending.back()) }; + pending.pop_back(); + if (!closure.insert(module).second) continue; + if (const auto requires_ = moduleRequires_.find(module); requires_ != moduleRequires_.end()) { + pending.insert(pending.end(), requires_->second.begin(), requires_->second.end()); + } + } + return closure; + } + + std::string closure_identity_(std::string_view key) const { + std::string identity; + for (const auto& module : closure_of_(key)) { + const auto built = moduleBuilds_.find(module); + identity += std::format("{}={};", module, built == moduleBuilds_.end() ? 0 : built->second); + } + return identity; + } + + // W5 (#59 U7a): a file set aside after clangd exited on it goes back once its modules have been built again and + // differently -- what it crashed on may well have been one of them, in the middle of being rebuilt -- and only once + // for each such closure: an exit on it after that sets it aside for a doubled term, as before. Asked when modules + // were built, once preparation is done; a closure with a module that failed to build stays as it is. + void retry_after_closure_rebuilt_() { + closureRetryCheck_ = false; + std::vector retried; + for (const auto& [key, aside] : aside_) { + if (!aside.crashClosure || aside.crashClosure->empty() || !quarantine_.contains(key)) continue; + const std::string identity { closure_identity_(key) }; + if (identity == *aside.crashClosure || crashRetried_[key].contains(identity)) continue; + if (std::ranges::any_of(closure_of_(key), [&](const std::string& module) { return modulesFailedAt_.contains(module); })) continue; + crashRetried_[key].insert(identity); + retried.push_back(key); + } + for (const auto& key : retried) { + quarantine_.release(key); + aside_.erase(key); + for (const auto& document : host_->documents()) { + if (document.path.empty() || base::path_key(document.path) != key) continue; + log::info("handing {} back to clangd ({}): the modules it imports were built again since clangd exited on it", document.path, host_->root_directory()); + host_->record_event("file-handed-back", Json { { "file", document.path }, { "why", "the modules it imports were built again" } }); + if (accepting_ && !excluded_path_(document.path)) open_or_hold_(document, true); + } + } + if (!retried.empty()) update_quarantine_issue_(); + } + void set_aside_(const std::string& path, std::string_view why, Reclaim reclaim, bool moduleFailed = false, std::optional spunOn = std::nullopt, bool onDisk = false) { const std::string key { base::path_key(path) }; @@ -3857,6 +3942,7 @@ class ClangdEngine final : public Engine { // wrong with that one -- exits, restarts, files set aside -- says nothing about this one. void forget_provisional_verdicts_() { crashes_.clear(); + exitStrikes_.clear(); earlyExits_ = 0; restartGate_.reset(); std::erase_if(issues_, [](const Issue& issue) { return issue.code == "engine-crashed" || issue.code == "engine-restart-capped"; }); @@ -4172,6 +4258,7 @@ class ClangdEngine final : public Engine { stuckAtCap_ = false; std::erase_if(issues_, [](const Issue& issue) { return issue.code == "engine-crashed" || issue.code == "engine-timeout" || issue.code == "engine-crash-loop"; }); crashes_.clear(); + exitStrikes_.clear(); crashLoops_ = 0; if (unavailable_ && !incompatible_) unavailable_ = false; host_->record_event("engine-restart-requested", Json { { "filesHandedBack", released } }); diff --git a/src/lsp/connection.cpp b/src/lsp/connection.cpp index 1480746c..7626ed2a 100644 --- a/src/lsp/connection.cpp +++ b/src/lsp/connection.cpp @@ -21,7 +21,6 @@ base::Result> Connection::start(platform::SpawnOptio auto connection = std::make_unique(); connection->process_ = std::move(*process); - connection->framing_ = framing; Connection* self { connection.get() }; connection->reader_ = std::jthread { [self, framing, onMessage = std::move(onMessage), onClosed = std::move(onClosed)] { FrameReader reader; @@ -55,21 +54,35 @@ base::Result> Connection::start(platform::SpawnOptio self->closed_.store(true); if (onClosed) onClosed(); } }; - connection->writer_ = std::jthread { [self] { + connection->writer_ = std::jthread { [self, framing] { while (true) { - std::string frame; + Json message; { std::unique_lock lock { self->outMutex_ }; self->outReady_.wait(lock, [&] { return !self->outgoing_.empty() || self->outClosing_; }); - if (self->outgoing_.empty()) break; // closing, and everything queued is out - frame = std::move(self->outgoing_.front()); + if (self->closed_.load() || self->outgoing_.empty()) break; + message = std::move(self->outgoing_.front()); self->outgoing_.pop_front(); } - if (!self->process_.write(frame)) { - self->writeFailed_.store(true); - break; + // Encoding is also peer work: large JSON must not occupy the event-loop thread. + // Keep the snapshot and all framing work outside the queue lock. + try { + const std::string frame { framing == Framing::lines ? dump(message) + "\n" : encode_frame(message) }; + if (self->process_.write(frame)) continue; + } catch (...) { + // An exception escaping a writer thread would terminate the whole server. + // The failed connection closes its input just as it does after a pipe error. } + self->writeFailed_.store(true); + break; } + { + const std::lock_guard lock { self->outMutex_ }; + self->outClosing_ = true; + } + // No sender can append after observing outClosing_ under the lock. Dispose of + // any unsent snapshots without making a concurrent sender wait for their destructors. + self->outgoing_.clear(); // The input is closed here, by the one thread that writes to it: the peer reads its end and may exit by itself. self->process_.close_input(); } }; @@ -94,14 +107,13 @@ base::Result> Connection::start(platform::SpawnOptio return connection; } -base::Result Connection::send(const Json& message) { +base::Result Connection::send(Json message) { if (closed_.load()) return base::fail("connection-closed", "the peer has exited"); - if (writeFailed_.load()) return base::fail("connection-write", "the peer stopped reading its input"); - std::string frame { framing_ == Framing::lines ? dump(message) + "\n" : encode_frame(message) }; + if (writeFailed_.load()) return base::fail("connection-write", "the connection could not encode or write its queued input"); { const std::lock_guard lock { outMutex_ }; if (outClosing_) return base::fail("connection-closed", "the peer is being stopped"); - outgoing_.push_back(std::move(frame)); + outgoing_.push_back(std::move(message)); } outReady_.notify_one(); return {}; diff --git a/src/lsp/connection.cppm b/src/lsp/connection.cppm index 6fe36bfc..5797a258 100644 --- a/src/lsp/connection.cppm +++ b/src/lsp/connection.cppm @@ -30,11 +30,10 @@ private: std::jthread writer_; std::mutex outMutex_; std::condition_variable outReady_; - std::deque outgoing_; + std::deque outgoing_; bool outClosing_ { false }; // stop(): no more sends; the writer closes the input once the queue is out std::atomic writeFailed_ { false }; std::atomic closed_ { false }; - Framing framing_ { Framing::content_length }; public: Connection() = default; @@ -48,8 +47,9 @@ public: static base::Result> start(platform::SpawnOptions options, MessageHandler onMessage, ClosedHandler onClosed, ErrorLineHandler onErrorLine = {}, Framing framing = Framing::content_length); - // Queues the message for the writer thread; fails only once the peer is gone or a write failed. - base::Result send(const Json& message); + // Queues an owned snapshot; the writer encodes it and writes it in order. Success means + // accepted, not delivered. A later encoding/write failure is visible to subsequent sends. + base::Result send(Json message); // Ends the peer: closes its input, waits up to `grace`, then terminates it. void stop(std::chrono::milliseconds grace); bool closed() const { return closed_.load(); } diff --git a/src/orchestrator/workspace.cpp b/src/orchestrator/workspace.cpp index 1eec5d7c..4989ce4f 100644 --- a/src/orchestrator/workspace.cpp +++ b/src/orchestrator/workspace.cpp @@ -105,6 +105,15 @@ std::string structure_of(const project::ScanResult& scan) { return key; } +std::optional described_source_structure(std::string_view text, const project::ScanResult& scan) { + // The scanner recognizes ordinary # directives. Alternate directive tokens are + // deliberately unacknowledged instead of guessing how the compiler processes them. + if (text.contains("%:") || text.contains("?" "?=") || text.contains("_Pragma") || text.contains("__pragma") + || text.contains("__LINE__") || text.contains("__COUNTER__") || !scan.preprocessorComplete) return std::nullopt; + if (!scan.importsComplete || !scan.declarationsComplete || std::ranges::any_of(scan.imports, [](const auto& import) { return import.isHeaderUnit; })) return std::nullopt; + return structure_of(scan) + "\npreprocessor-inputs\n" + scan.preprocessorDirectives; +} + // The inputs a producer named in its database's watch list (S2 5), shared with the polling worker. struct WatchPatterns { std::atomic stopped { false }; @@ -478,7 +487,18 @@ struct Workspace::Impl final : engine::Host { int modelGeneration { 0 }; std::string modelDescription; // describe_model of `model`, to recognize an unchanged reload bool loading { false }; + bool applyingPlan { false }; bool reloadAfterLoad { false }; + bool producerStatesResetPending { false }; // XS-4: a reset asked for them to go before the next load + // A lexical source structure can legitimately differ from the producer's active + // dependency graph (#if 0, for example). Remember only inputs a valid answer covered. + std::map producerSourcePaths; + std::map loadingSourceStructures; + std::map loadingSourceStamps; + std::map describedSourceStructures; + int describedSourceGeneration { -1 }; + std::string loadingSourceProducerIdentity; + std::string describedSourceProducerIdentity; project::TargetSelection targetSelection; bool targetReadPending { false }; int targetReadRetries { 0 }; @@ -518,6 +538,13 @@ struct Workspace::Impl final : engine::Host { // Fix plan F13: a change to a file's imports or module declaration made in the editor is planned once the file // has been quiet this long, so a name being typed is not planned letter by letter. static constexpr std::chrono::milliseconds EDIT_SETTLE { 2000 }; + // 0.0.17 W4 (#59 U15): a structure reload that saves of a file still being typed in asked for waits until the file + // is quiet (EDIT_SETTLE after its last change). With autosave, every pause while an import is typed is a save that + // changes the file's imports; each was a run of the producer and a loading state of its own. Edits that keep coming + // put it off no longer than EDIT_DEFER_LIMIT after the first save that waited, so the model is never left behind. + static constexpr std::chrono::seconds EDIT_DEFER_LIMIT { 30 }; + std::set> reloadWaitsFor; // path keys of the files being typed in + std::optional reloadDeferredSince; std::optional loadGiveUpAt; // the producer has not answered: take what there is (design 4.1) std::optional lastResortAt; // nothing at all came: serve without a database rather than nothing std::optional sdkCheckAt; @@ -1926,9 +1953,20 @@ struct Workspace::Impl final : engine::Host { recent.push_back(ms); if (recent.size() > 64) recent.pop_front(); } - if (ms >= 5000) { - journal.add("slow-request", Json { { "method", job.method }, { "file", job.path }, { "ms", static_cast(ms) }, - { "outcome", std::string { outcome } }, { "answeredBy", job.answeredBy } }); + // A completion a person notices is slow from two seconds on (0.0.17 M3, #59 U5/U6: a typing session's slowest answers were + // 6-8 s, and nothing said where the time went); anything else from five. Where it went: waiting before an engine was asked + // (queueMs), the engine (engineMs), and what came after its answer -- merging, the answer layer, the reply (answerMs). + const bool completion { job.method == lsp::method::TEXT_DOCUMENT_COMPLETION }; + if (ms >= (completion ? 2000 : 5000)) { + Json detail { { "method", job.method }, { "file", job.path }, { "ms", static_cast(ms) }, + { "outcome", std::string { outcome } }, { "answeredBy", job.answeredBy } }; + const auto milliseconds = [](Clock::duration span) { return static_cast(std::chrono::duration(span).count()); }; + if (job.engineSentAt) detail["queueMs"] = milliseconds(*job.engineSentAt - job.started); + if (job.engineSentAt && job.engineRepliedAt) { + detail["engineMs"] = milliseconds(*job.engineRepliedAt - *job.engineSentAt); + detail["answerMs"] = milliseconds(Clock::now() - *job.engineRepliedAt); + } + journal.add("slow-request", std::move(detail)); } } @@ -2527,6 +2565,63 @@ struct Workspace::Impl final : engine::Host { } } + void capture_source_structures() { + loadingSourceStructures.clear(); + loadingSourceStamps.clear(); + loadingSourceProducerIdentity = producer_file_identity(); + // Only sources reported by the producer's watcher, never a recursive scan. + // Capture before dispatch: a later disk edit is not evidence this run read it. + std::size_t remaining { 32 * 1024 * 1024 }; + for (const auto& [key, path] : producerSourcePaths) { + if (remaining == 0) break; + const auto before = platform::fs::stamp(path); + if (!before || before->size > remaining) continue; + const auto text = platform::fs::read_file(path, remaining); + if (!text) break; // a failed bounded read may already have consumed the remaining budget + remaining -= text->size(); + if (platform::fs::stamp(path) == before) { + if (auto structure = described_source_structure(*text, project::scan_source(*text, true))) { + loadingSourceStructures.emplace(key, std::move(*structure)); + loadingSourceStamps.emplace(key, *before); + } + } + } + } + + void confirm_source_structures(const project::ProjectModel& loaded) { + if (loaded.source == project::SourceKind::inferred || !loaded.issues.empty() || reloadAfterLoad + || loadingSourceProducerIdentity != producer_file_identity() + || (!loaded.producer.empty() && !producerPath.empty() && !base::same_path(loaded.producer, producerPath))) return; + std::set units; + for (const auto& set : loaded.database.sets) { + for (const auto& unit : set.units) units.insert(base::path_key(spec::absolute_source(unit))); + } + describedSourceStructures.clear(); + for (const auto& [key, structure] : loadingSourceStructures) { + const auto path = producerSourcePaths.find(key); + const auto stamp = loadingSourceStamps.find(key); + if (units.contains(key) && path != producerSourcePaths.end() && stamp != loadingSourceStamps.end() + && platform::fs::stamp(path->second) == std::optional { stamp->second } + && matches_watch_entries(loaded.watch, root, path->second)) { + describedSourceStructures.emplace(key, structure); + } + } + describedSourceGeneration = modelGeneration; + describedSourceProducerIdentity = loadingSourceProducerIdentity; + } + + // 0.0.17 XS-4 (#68): what a build tool made for itself in this cache (a provider's private configure directory, its id under + // the cache) goes with a reset too, or a state xmake no longer recovers from outlives the reset that was to clear it. Removed + // only while no load runs in it: a reset during a load leaves it to the start of the next one. + void remove_producer_states() { + for (const auto* provider : project::registered_providers()) { + const std::string directory { base::join_path(cacheDirectory, std::string { provider->id() }) }; + if (!platform::fs::is_directory(directory)) continue; + platform::fs::remove_all(directory); + log::info("removed {}'s private state for {} ({})", provider->id(), root, directory); + } + } + void start_model_load() { if (loading) { reloadAfterLoad = true; @@ -2537,9 +2632,11 @@ struct Workspace::Impl final : engine::Host { update_status(); return; } + if (std::exchange(producerStatesResetPending, false)) remove_producer_states(); loading = true; remember_described_inputs(true); const int generation { ++modelGeneration }; + capture_source_structures(); project::LoadOptions load; load.trusted = options.trusted; load.cacheDirectory = cacheDirectory; @@ -2791,6 +2888,7 @@ struct Workspace::Impl final : engine::Host { } keptModelNotice.clear(); staleModelReason.clear(); + confirm_source_structures(*loadedModel); adopt_model(std::move(loadedModel), "producer"); } @@ -3073,11 +3171,20 @@ struct Workspace::Impl final : engine::Host { core_wait_limit().count()); journal.add("engine-waits-for-producer", Json { { "detected", std::string { project::to_string(detectedSource) } } }); } - for (const auto& engine : engines) { - if (coreWaits && engine.get() == coreEngine) continue; - engine->apply(&plan); + { + // apply may synchronously report status before its new plan is installed. + // Publish only after every engine has applied this same workspace plan. + struct StatusBatch { + bool& active; + bool previous; + ~StatusBatch() { active = previous; } + } batch { applyingPlan, std::exchange(applyingPlan, true) }; + for (const auto& engine : engines) { + if (coreWaits && engine.get() == coreEngine) continue; + engine->apply(&plan); + } + for (const Document* document : documents_.all()) publish_diagnostics(document->uri); } - for (const Document* document : documents_.all()) publish_diagnostics(document->uri); update_status(); } @@ -3107,12 +3214,34 @@ struct Workspace::Impl final : engine::Host { // `fromEdits`: asked for by changes under the workspace rather than by a build description. Such a reload waits as // long as the producer takes to answer (at most a minute), so edits that keep coming are one run, not a queue of them. // A reload a build description asked for is not put off by edits that follow it: it reads them too. - void schedule_reload(bool fromEdits = false) { + // `typing`: of the files whose changes asked for it, those still being typed in (W4); the reload waits for them. + void schedule_reload(bool fromEdits = false, const std::vector& typing = {}) { using namespace std::chrono_literals; if (fromEdits && reloadAt && !reloadMovable) return; + const auto now { Clock::now() }; const auto wait = fromEdits ? std::clamp(std::chrono::milliseconds { lastProducerMs }, 1500ms, 60s) : 1500ms; - reloadAt = Clock::now() + wait; + reloadAt = now + wait; reloadMovable = fromEdits; + if (!fromEdits) { + reloadWaitsFor.clear(); + reloadDeferredSince.reset(); + return; + } + reloadWaitsFor.insert(typing.begin(), typing.end()); + defer_reload_while_typing(now); + } + + // W4: the pending reload put off until the files it waits for are quiet, within EDIT_DEFER_LIMIT of the first wait. + void defer_reload_while_typing(Clock::time_point now) { + if (!reloadAt || !reloadMovable) return; + std::optional quiet; + for (const auto& key : reloadWaitsFor) { + const auto edited = editedAt.find(key); + if (edited != editedAt.end() && edited->second + EDIT_SETTLE > now) quiet = std::max(quiet.value_or(now), edited->second + EDIT_SETTLE); + } + if (!quiet) return; + if (!reloadDeferredSince) reloadDeferredSince = now; + reloadAt = std::max(*reloadAt, std::min(*quiet, *reloadDeferredSince + EDIT_DEFER_LIMIT)); } // K-7: the issues nothing here recovers from by itself. @@ -3467,6 +3596,7 @@ struct Workspace::Impl final : engine::Host { void update_status() { if (!initializeAnswered) return; // see the field's own comment + if (applyingPlan) return; // replan publishes the committed snapshot after all synchronous apply calls request_auto_bundles(); const State computed { compute_state() }; State state { computed }; @@ -3902,6 +4032,8 @@ struct Workspace::Impl final : engine::Host { phase("the workspace lease and dead instances"); if (reloadAt && *reloadAt <= now) { reloadAt.reset(); + reloadWaitsFor.clear(); + reloadDeferredSince.reset(); start_model_load(); } if (targetRetryAt && *targetRetryAt <= now) { @@ -4129,6 +4261,7 @@ void Workspace::did_change(const Json& message, const Json& params) { if (!document->path.empty()) { if (impl_->answers_) impl_->answers_->changed(document->path); impl_->editedAt[base::path_key(document->path)] = Clock::now(); + if (impl_->reloadWaitsFor.contains(base::path_key(document->path))) impl_->defer_reload_while_typing(Clock::now()); impl_->note_recent_file(document->path); impl_->index.update(document->path, document->text); impl_->note_structure_change(document->path, true); @@ -4195,6 +4328,7 @@ void Workspace::handle_watched_files(const Json& changes) { bool reload { false }; bool describedAgain { false }; // a build file or another input of the producer's, not a source: reloaded without the edit delay bool replan { false }; + std::vector typing; // W4: changed sources whose structure differs and that are being typed in for (const auto& change : changes) { const std::string path { impl_->path_of_uri(change.value("uri", std::string {})) }; if (path.empty()) continue; @@ -4250,7 +4384,39 @@ void Workspace::handle_watched_files(const Json& changes) { // pause in typing: on GalTranslPP a minute of CPU each time, beside clangd. An edit inside functions leaves // the model as it is; the index and the plan already have it. // What is on disk is what the producer reads: an open file changed under the editor (a checkout) counts too. - if (producerInput && (type != 2 || !diskText || impl_->model_structure_differs(path, project::scan_source(*diskText)))) reload = true; + if (producerInput) { + const std::string key { base::path_key(path) }; + if (type == 3) { + impl_->producerSourcePaths.erase(key); + impl_->describedSourceStructures.erase(key); + impl_->loadingSourceStructures.erase(key); + impl_->loadingSourceStamps.erase(key); + } else { + impl_->producerSourcePaths.insert_or_assign(key, path); + } + bool differs { true }; + if (type == 2 && diskText) { + const auto scan { project::scan_source(*diskText, true) }; + const auto structure { described_source_structure(*diskText, scan) }; + const auto known = impl_->describedSourceStructures.find(key); + if (impl_->describedSourceGeneration == impl_->modelGeneration && known != impl_->describedSourceStructures.end()) { + differs = !structure || known->second != *structure + || impl_->describedSourceProducerIdentity != impl_->producer_file_identity(); + } else { + // Conditional imports need one actual answer even when the old model + // happens to list the same lexical names (for example #if 1 -> #if 0). + differs = !structure || scan.uncertain || impl_->model_structure_differs(path, scan); + const auto started = impl_->loadingSourceStructures.find(key); + if (started != impl_->loadingSourceStructures.end() + && (!structure || started->second != *structure || !impl_->loading || !impl_->staleModelReason.empty())) differs = true; + } + } + if (differs) { + reload = true; + const auto edited = impl_->editedAt.find(key); + if (open && edited != impl_->editedAt.end() && Clock::now() - edited->second < Impl::EDIT_SETTLE) typing.push_back(key); + } + } if (open) continue; if (impl_->model && impl_->model->source == project::SourceKind::inferred && type != 2) reload = true; else replan = true; @@ -4258,7 +4424,7 @@ void Workspace::handle_watched_files(const Json& changes) { if (reload || replan) { for (const auto& engine : impl_->engines) engine->sources_changed(); } - if (reload) impl_->schedule_reload(!describedAgain); + if (reload) impl_->schedule_reload(!describedAgain, typing); else if (replan) impl_->schedule_replan(); // 0.0.13 W10 (#53): clangd refuses the whole notification for one change that is not a file ("clangd only supports // 'file' URI scheme for workspace files"), and the files beside it went unseen; only file: URIs go on. @@ -4485,6 +4651,7 @@ std::uint64_t Workspace::reset_cache() { if (const auto stamp = platform::fs::stamp(entry)) freed += stamp->size; platform::fs::remove_all(entry); } + impl.producerStatesResetPending = true; // removed as the load below starts, or the next one if one runs (XS-4) log::info("the cache of {} was reset on request ({} bytes freed)", root_, freed); impl.journal.add("cache-reset", Json { { "bytes", freed } }); { diff --git a/src/project/scan.cpp b/src/project/scan.cpp index 25d28db1..87a1998f 100644 --- a/src/project/scan.cpp +++ b/src/project/scan.cpp @@ -35,11 +35,17 @@ class Lexer { std::size_t at_ { 0 }; bool lineStart_ { true }; int conditionalDepth_ { 0 }; + std::string directives_; + bool retainDirectives_ { false }; + bool directivesComplete_ { true }; public: - explicit Lexer(std::string_view text) : text_ { text }, at_ { base::byte_order_mark_size(text) } {} + explicit Lexer(std::string_view text, bool retainDirectives = false) + : text_ { text }, at_ { base::byte_order_mark_size(text) }, retainDirectives_ { retainDirectives } {} int conditional_depth() const { return conditionalDepth_; } + std::string take_directives() { return std::move(directives_); } + bool directives_complete() const { return directivesComplete_; } // The next token outside comments, literals' interiors and preprocessor directives. std::optional next(bool wantHeaderName) { @@ -71,6 +77,7 @@ class Lexer { directive_(); continue; } + if (c == '#') directivesComplete_ = false; const bool startsLine { lineStart_ }; lineStart_ = false; const std::size_t begin { at_ }; @@ -164,6 +171,7 @@ class Lexer { } void directive_() { + const auto begin { at_ }; std::size_t cursor { at_ + 1 }; while (cursor < text_.size() && is_space(text_[cursor])) ++cursor; std::size_t nameEnd { cursor }; @@ -184,6 +192,10 @@ class Lexer { } ++at_; } + if (retainDirectives_) { + directives_.append(text_.substr(begin, at_ - begin)); + directives_.push_back('\n'); + } lineStart_ = true; } }; @@ -359,9 +371,9 @@ std::vector scan_syntax_tokens(std::string_view text) { return tokens; } -ScanResult scan_source(std::string_view text) { +ScanResult scan_source(std::string_view text, bool retainPreprocessor) { ScanResult result; - Lexer lexer { text }; + Lexer lexer { text, retainPreprocessor }; int braceDepth { 0 }; std::optional token { lexer.next(false) }; while (token) { @@ -387,7 +399,11 @@ ScanResult scan_source(std::string_view text) { const bool isImport { token->text == "import" }; const bool conditional { lexer.conditional_depth() > 0 }; token = lexer.next(isImport); - if (!token) break; + if (!token) { + if (isImport) result.importsComplete = false; + else result.declarationsComplete = false; + break; + } if (isImport && (token->kind == TokenKind::header_name || token->kind == TokenKind::string)) { ImportDeclaration import; @@ -402,16 +418,26 @@ ScanResult scan_source(std::string_view text) { result.imports.push_back(std::move(import)); if (conditional) result.uncertain = true; token = lexer.next(false); - } + } else result.importsComplete = false; continue; } // A contextual keyword is a declaration only when a name or partition follows. const bool nameFollows { token->kind == TokenKind::identifier || (isImport && token->kind == TokenKind::punctuation && token->text == ":") }; - if (!nameFollows) continue; + if (!nameFollows) { + if (isImport) result.importsComplete = false; + else if (token->text != ";") result.declarationsComplete = false; + continue; + } NameParse name { parse_name(lexer, token, isImport) }; - if (!name.ok) continue; + if (!name.ok) { + if (isImport) result.importsComplete = false; + else result.declarationsComplete = false; + continue; + } skip_attributes(lexer, token); if (!token || token->kind != TokenKind::punctuation || token->text != ";") { + if (isImport) result.importsComplete = false; + else result.declarationsComplete = false; // The directive's line ended first: `import hello.greet` with its `;` not typed yet. if (isImport && (!token || token->startsLine)) { const base::Range range { position_in(text, name.begin), position_in(text, name.end) }; @@ -431,6 +457,8 @@ ScanResult scan_source(std::string_view text) { result.uncertain = true; } } + result.preprocessorComplete = lexer.directives_complete(); + result.preprocessorDirectives = lexer.take_directives(); return result; } diff --git a/src/project/scan.cppm b/src/project/scan.cppm index 2625a0d6..c727de95 100644 --- a/src/project/scan.cppm +++ b/src/project/scan.cppm @@ -36,9 +36,15 @@ struct ScanResult { // diagnostics; a module a file needs all the same (required_names). std::vector unterminatedImports; bool uncertain { false }; + // Exact directive logical lines, including continuations and comments. A producer + // may preprocess imports differently even when their lexical names stay the same. + std::string preprocessorDirectives; + bool preprocessorComplete { true }; // false when a code-level # could not be classified as a directive + bool importsComplete { true }; // false when an import needs preprocessing or is not fully parsed + bool declarationsComplete { true }; // false when a module declaration is not fully parsed }; -ScanResult scan_source(std::string_view text); +ScanResult scan_source(std::string_view text, bool retainPreprocessor = false); // A minimal token for syntax highlighting a module declaration or import (design doc 2026-09-25 // K/§7): produced from the text alone, complete or not, so a person still typing `import hello.` diff --git a/src/project/xmake.cpp b/src/project/xmake.cpp index d6fff90b..402f503a 100644 --- a/src/project/xmake.cpp +++ b/src/project/xmake.cpp @@ -169,37 +169,35 @@ std::optional missing_packages_answer(const std::string& output, bool of return Answer { .code = std::string { spec::INSTALL_FAILED }, .reason = xmake_install_failed_reason(missing, output), .missing = std::move(missing) }; } -// Runs xmake privately: two commands, in a directory of ours, never the project's (see the comment in the body). -Answer run_private(const Claim& claim, const ProviderContext& context, const std::string& xmake) { - context.producerUsed = xmake; - +// What one run in the private state came to. `stale` (XS-3): xmake failed saying that state is out of date. +struct Attempt { + Answer answer; + bool stale { false }; +}; + +// Runs xmake privately in `state`: two commands, in a directory of ours, never the project's (see the comment in the body). +Attempt run_in_state(const Claim& claim, const ProviderContext& context, const std::string& xmake, const std::string& state, + const std::string& key, const std::optional& user) { // Isolation (plan §3.5): `xmake project -k compile_commands` alone still writes `build/` into // the project (its implicit configuration and module dependency scan, both measured side // effects of that command) unless it is given a private `--builddir` -- which only `xmake f` // can set, so a first run is two commands; `XMAKE_CONFIGDIR` keeps the configuration itself out - // of the project's own `.xmake/` too. - const std::string configDirectory { base::join_path(context.privateDirectory, "config") }; - const std::string buildDirectory { base::join_path(context.privateDirectory, "build") }; - const std::string outDirectory { base::join_path(context.privateDirectory, "out") }; + // of the project's own `.xmake/` too, and `XMAKE_TMPDIR` its module mapper files out of the + // temporary directory the user's own builds read (XS-1, #69). + const std::string configDirectory { base::join_path(state, "config") }; + const std::string buildDirectory { base::join_path(state, "build") }; + const std::string outDirectory { base::join_path(state, "out") }; + const std::string tmpDirectory { base::join_path(state, "tmp") }; (void)fs::create_directories(configDirectory); (void)fs::create_directories(outDirectory); - const auto environment { environment_with(xmake_environment(configDirectory), context.environmentWait) }; + (void)fs::create_directories(tmpDirectory); + const auto environment { environment_with(xmake_environment(configDirectory, tmpDirectory), context.environmentWait) }; - // Keep it simple (B-1 task 3): configure again whenever there is no private configuration yet, - // or it was made for another mode, another xmake.lua or another `xmake f` of the user's (the key recorded after the last success). - const std::string configuredMarker { base::join_path(context.privateDirectory, "configured") }; - const std::string leftOutMarker { base::join_path(context.privateDirectory, "left-out") }; - const auto manifestStamp { fs::stamp(claim.manifest) }; - // X-2: the user's own `xmake f` is read, never run or written; `xmake f` resets every option it is not given (E5). - const auto user { read_xmake_user_config(claim.root) }; - const std::string key { xmake_configuration_key( - context.offline, manifestStamp ? std::optional { std::pair { manifestStamp->size, manifestStamp->modified } } : std::nullopt, - user ? std::optional { std::pair { user->size, user->modified } } : std::nullopt) }; - const bool hasConfig { fs::is_directory(configDirectory) && !fs::list_directory(configDirectory).empty() }; + // A state is configured once, from nothing: the caller removes it when what it was made from changed (XS-2). + const std::string configuredMarker { base::join_path(state, "configured") }; + const std::string leftOutMarker { base::join_path(state, "left-out") }; const auto recorded { fs::read_file(configuredMarker) }; - const bool stale { !recorded || *recorded != key }; - if (!hasConfig || stale) { - fs::remove_all(configuredMarker); // a failed `xmake f -c` leaves no configuration worth keeping + if (!recorded || *recorded != key) { fs::remove_all(leftOutMarker); // D-1: asked only when xmake.conf holds something that is neither internal nor standard; the answer is kept. std::vector accepted; @@ -235,16 +233,16 @@ Answer run_private(const Claim& claim, const ProviderContext& context, const std configured = configure(true); if (configured && configured->exitCode == 0 && !configured->timedOut) (void)fs::write_file(leftOutMarker, base::join(leftOut, ", ")); } - if (!configured) return Answer { .code = configured.error().code, .reason = configured.error().message }; + if (!configured) return { Answer { .code = configured.error().code, .reason = configured.error().message } }; if (configured->timedOut) { - return Answer { .outcome = Outcome::timed_out, .code = "xmake-configure-failed", .reason = "xmake f did not finish in time" }; + return { Answer { .outcome = Outcome::timed_out, .code = "xmake-configure-failed", .reason = "xmake f did not finish in time" } }; } if (configured->exitCode != 0) { const std::string combined { configured->output + "\n" + configured->error }; - if (auto packages = missing_packages_answer(combined, context.offline)) return std::move(*packages); - return Answer { .code = "xmake-configure-failed", - .reason = std::format("xmake f failed ({}): {}", configured->exitCode, - base::trim(configured->error.empty() ? configured->output : configured->error)) }; + if (auto packages = missing_packages_answer(combined, context.offline)) return { std::move(*packages) }; + return { Answer { .code = "xmake-configure-failed", + .reason = std::format("xmake f failed ({}): {}", configured->exitCode, + base::trim(configured->error.empty() ? configured->output : configured->error)) } }; } (void)fs::write_file(configuredMarker, key); } @@ -262,20 +260,22 @@ Answer run_private(const Claim& claim, const ProviderContext& context, const std .environmentWait = context.environmentWait, .onSoftDeadline = context.onSlow, }); - if (!described) return Answer { .code = described.error().code, .reason = described.error().message }; + if (!described) return { Answer { .code = described.error().code, .reason = described.error().message } }; if (described->timedOut) { - return Answer { .outcome = Outcome::timed_out, .code = "xmake-configure-failed", - .reason = "xmake project -k compile_commands did not finish in time" }; + return { Answer { .outcome = Outcome::timed_out, .code = "xmake-configure-failed", + .reason = "xmake project -k compile_commands did not finish in time" } }; } if (described->exitCode != 0) { // D-6: the project stage reads the packages too; they can be missing here when `xmake f` did not need them. - if (auto packages = missing_packages_answer(described->output + "\n" + described->error, context.offline)) return std::move(*packages); - return Answer { .code = "xmake-configure-failed", - .reason = std::format("xmake project -k compile_commands failed ({}): {}", described->exitCode, - base::trim(described->error.empty() ? described->output : described->error)) }; + const std::string combined { described->output + "\n" + described->error }; + if (auto packages = missing_packages_answer(combined, context.offline)) return { std::move(*packages) }; + return { Answer { .code = "xmake-configure-failed", + .reason = std::format("xmake project -k compile_commands failed ({}): {}", described->exitCode, + base::trim(described->error.empty() ? described->output : described->error)) }, + xmake_state_stale(combined) }; } const std::string commands { base::join_path(outDirectory, "compile_commands.json") }; - if (!fs::is_regular_file(commands)) return Answer { .code = "xmake-no-database", .reason = "xmake did not write compile_commands.json" }; + if (!fs::is_regular_file(commands)) return { Answer { .code = "xmake-no-database", .reason = "xmake did not write compile_commands.json" } }; Answer answer { from_result(database_from_compile_commands_file(commands, claim.root, "xmake", context.scanner, context.prober)) }; if (answer.database) { if (const auto leftOut = fs::read_file(leftOutMarker); leftOut && !leftOut->empty()) { @@ -284,7 +284,41 @@ Answer run_private(const Claim& claim, const ProviderContext& context, const std "the commands may differ from your build's; run `xmake f -c` in the project to renew its configuration", *leftOut)); } } - return answer; + return { std::move(answer) }; +} + +// XS-2, XS-3 (#68): the private state is a cache of this provider's, never a fact. It is kept while what it was made from is the +// same, and otherwise removed and made again from nothing; once more, from nothing, when xmake says it is out of date. +Answer run_private(const Claim& claim, const ProviderContext& context, const std::string& xmake) { + context.producerUsed = xmake; + // 0.0.16 and earlier kept the state in the private directory itself, configured again in place: removed, made again in `state`. + for (const std::string_view legacy : { "config", "build", "out", "configured", "left-out" }) { + fs::remove_all(base::join_path(context.privateDirectory, legacy)); + } + const std::string state { base::join_path(context.privateDirectory, "state") }; + const auto manifestStamp { fs::stamp(claim.manifest) }; + // X-2: the user's own `xmake f` is read, never run or written; `xmake f` resets every option it is not given (E5). + const auto user { read_xmake_user_config(claim.root) }; + std::int64_t manifestsModified { 0 }; + const auto manifests { xmake_manifests(claim) }; + for (const auto& manifest : manifests) { + if (const auto stamp = fs::stamp(manifest)) manifestsModified = std::max(manifestsModified, stamp->modified); + } + const auto toolStamp { fs::stamp(xmake) }; + const std::string key { xmake_state_key( + xmake_configuration_key(context.offline, manifestStamp ? std::optional { std::pair { manifestStamp->size, manifestStamp->modified } } : std::nullopt, + user ? std::optional { std::pair { user->size, user->modified } } : std::nullopt), + xmake, toolStamp ? std::optional { std::pair { toolStamp->size, toolStamp->modified } } : std::nullopt, manifestsModified, manifests.size()) }; + const auto recorded { fs::read_file(base::join_path(state, "configured")) }; + const bool kept { recorded && *recorded == key }; + if (!kept) fs::remove_all(state); + auto attempt { run_in_state(claim, context, xmake, state, key, user) }; + if (attempt.stale && kept) { + base::log::info("xmake's private state for {} is out of date ({}); making it again from nothing", claim.root, attempt.answer.reason); + fs::remove_all(state); + attempt = run_in_state(claim, context, xmake, state, key, user); + } + return std::move(attempt.answer); } } // namespace @@ -466,8 +500,22 @@ bool xmake_commands_out_of_date(std::int64_t commandsModified, std::span commandsModified; }); } -std::vector> xmake_environment(std::string_view configDirectory) { - return { { "XMAKE_CONFIGDIR", std::string { configDirectory } }, { "XMAKE_THEME", "plain" } }; +std::vector> xmake_environment(std::string_view configDirectory, std::string_view tmpDirectory) { + std::vector> environment { { "XMAKE_CONFIGDIR", std::string { configDirectory } }, { "XMAKE_THEME", "plain" } }; + if (!tmpDirectory.empty()) environment.emplace_back("XMAKE_TMPDIR", std::string { tmpDirectory }); + return environment; +} + +std::string xmake_state_key(std::string_view configurationKey, std::string_view xmake, + std::optional> tool, std::int64_t manifestsModified, std::size_t manifests) { + return std::format("{}\nxmake {} {} {}\nmanifests {} {}", configurationKey, xmake, tool ? tool->first : 0, tool ? tool->second : 0, + manifestsModified, manifests); +} + +bool xmake_state_stale(std::string_view output) { + // xmake 3.1's own words (rules/c++/modules/support.lua, load_moduleinfo). + return (output.contains("dependfile for '") && output.contains("' not found at '")) || output.contains("module scanner did not run for target(") + || output.contains("failed to load moduleinfo for '"); } std::string xmake_configuration_key(bool offline, std::optional> manifest, diff --git a/src/project/xmake.cppm b/src/project/xmake.cppm index 8b405ced..23e9959c 100644 --- a/src/project/xmake.cppm +++ b/src/project/xmake.cppm @@ -12,9 +12,23 @@ export namespace mcppls::project { // `XMAKE_CONFIGDIR=/config`, `XMAKE_THEME=plain` (plan §3.5): the configuration directory // xmake would otherwise write into the project's own `.xmake/`, and a theme with no interactive -// prompt or color codes to strip out of a captured log. A free function so tests can check it -// without spawning xmake. -std::vector> xmake_environment(std::string_view configDirectory); +// prompt or color codes to strip out of a captured log. `XMAKE_TMPDIR=/tmp` (0.0.17 XS-1, #69, +// #24 UP-X1): with GCC, xmake keeps each unit's module mapper file in its temporary directory under a +// name only the source's relative path decides, and the user's own builds read it; a private run that +// wrote it there pointed the user's next incremental build at modules that were never built. A free +// function so tests can check it without spawning xmake. +std::vector> xmake_environment(std::string_view configDirectory, std::string_view tmpDirectory = {}); + +// 0.0.17 XS-2 (#68, #24 UP-X2): what the private xmake state was made from. The configuration key, and the xmake that made it +// (its path, size and time) and every xmake.lua of the project (the newest time, and how many: a target added in an included +// file is a change too). The state is kept only while this is the same; otherwise it is removed and made again from nothing, +// never configured again in place: xmake's module scan cache, which only a build updates, outlives `xmake f -c`. +std::string xmake_state_key(std::string_view configurationKey, std::string_view xmake, + std::optional> tool, std::int64_t manifestsModified, std::size_t manifests); + +// 0.0.17 XS-3: xmake said its private state is out of date -- a module unit's dependfile its scan cache counts on is gone, or +// the scan it expects did not run -- rather than anything about the project. Such a state is discarded and made again once. +bool xmake_state_stale(std::string_view output); // One scalar the user's own `xmake f` left in `.xmake///xmake.conf` (X-2, plan 0.0.8 part 2): // a string, or a true/false (`flag`), which xmake takes back as `--name=y|n`. diff --git a/src/server/session.cpp b/src/server/session.cpp index 2d15006d..19b1af63 100644 --- a/src/server/session.cpp +++ b/src/server/session.cpp @@ -29,6 +29,32 @@ import mcppls.bundle.writer; namespace mcppls::server { +std::vector workspace_folders(const nlohmann::json& params, std::string_view fallbackDirectory) { + auto folder = [](const std::string& uri) -> std::optional { + auto path = base::uri_to_path(uri); + if (!path || platform::fs::is_regular_file(*path)) return std::nullopt; + return WorkspaceFolder { *path, uri }; + }; + std::vector found; + if (const auto* folders = lsp::find(params, "workspaceFolders"); folders != nullptr && folders->is_array()) { + for (const auto& entry : *folders) { + if (auto uri = lsp::string_at(entry, "uri")) { + if (auto named = folder(*uri)) found.push_back(std::move(*named)); + } + } + if (!found.empty()) return found; + } + if (auto uri = lsp::string_at(params, "rootUri")) { + if (auto named = folder(*uri)) return { std::move(*named) }; + } + if (auto path = lsp::string_at(params, "rootPath"); path && !path->empty() && !platform::fs::is_regular_file(*path)) { + const std::string normalized { base::normalize_path(*path) }; + return { WorkspaceFolder { normalized, base::path_to_uri(normalized) } }; + } + const std::string current { fallbackDirectory }; + return { WorkspaceFolder { current, base::path_to_uri(current) } }; +} + namespace { using Json = nlohmann::json; @@ -517,38 +543,18 @@ class Session { } // A workspace folder: the path it names, and the URI the client named it by. - struct Folder { - std::string path; - std::string uri; - }; + using Folder = WorkspaceFolder; static std::optional folder_of_uri_(const std::string& uri) { auto path = base::uri_to_path(uri); - if (!path) return std::nullopt; + if (!path || platform::fs::is_regular_file(*path)) return std::nullopt; return Folder { *path, uri }; } // Every folder `initialize` names, or the single root a pre-3.6 client implies (design 13.1 // predates multi-root; usable plan W9.1 is additive over it). std::vector workspace_roots_(const Json& params) const { - std::vector found; - if (const Json* folders = lsp::find(params, "workspaceFolders"); folders != nullptr && folders->is_array()) { - for (const auto& folder : *folders) { - if (auto uri = lsp::string_at(folder, "uri")) { - if (auto named = folder_of_uri_(*uri)) found.push_back(std::move(*named)); - } - } - if (!found.empty()) return found; - } - if (auto uri = lsp::string_at(params, "rootUri")) { - if (auto named = folder_of_uri_(*uri)) return { std::move(*named) }; - } - if (auto path = lsp::string_at(params, "rootPath"); path && !path->empty()) { - const std::string normalized { base::normalize_path(*path) }; - return { Folder { normalized, base::path_to_uri(normalized) } }; - } - const std::string current { platform::fs::current_directory() }; - return { Folder { current, base::path_to_uri(current) } }; + return workspace_folders(params, platform::fs::current_directory()); } // `isFirst` wires the session's own `initialize` response to this root's engine settling diff --git a/src/server/session.cppm b/src/server/session.cppm index abe63b86..c0696e1c 100644 --- a/src/server/session.cppm +++ b/src/server/session.cppm @@ -5,12 +5,22 @@ export module mcppls.server.session; import std; +import nlohmann.json; export import mcppls.orchestrator.workspace; export namespace mcppls::server { using orchestrator::SessionOptions; +struct WorkspaceFolder { + std::string path; + std::string uri; +}; + +// Resolves initialize's workspace folders, retaining the client's URI spelling. +// Existing regular files are not workspace directories; absent directories remain valid. +std::vector workspace_folders(const nlohmann::json& params, std::string_view fallbackDirectory); + // Serves the Language Server Protocol on standard input and output until the client exits. // Returns the process exit code. int run_session(const SessionOptions& options); diff --git a/tests/test_answer_routing.cpp b/tests/test_answer_routing.cpp index f08fda85..0ffa1bea 100644 --- a/tests/test_answer_routing.cpp +++ b/tests/test_answer_routing.cpp @@ -145,14 +145,17 @@ struct Fixture { std::map versions; int nextId { 1 }; - explicit Fixture(std::string at = canonical_root(), std::string cache = "on", bool ready = true, std::string agent = "cached") : root { std::move(at) } { + explicit Fixture(std::string at = canonical_root(), std::string cache = "on", bool ready = true, std::string agent = "cached", + std::function(std::shared_ptr)> coreFactory = {}) : root { std::move(at) } { o::SessionOptions options; options.answerCache = cache; options.answerAgent = agent; options.answerPrefetch = false; - options.engineFactories = [held = held](const auto&, const auto&, bool) { + options.engineFactories = [held = held, coreFactory](const auto&, const auto&, bool) { o::EngineFactories factories; - factories.core = [held] { return std::make_unique(held); }; + factories.core = [held, coreFactory]() -> std::unique_ptr { + return coreFactory ? coreFactory(held) : std::make_unique(held); + }; return factories; }; workspace = std::make_unique(root, root, options, e::PayloadPaths {}, false, false, "", events, sink); @@ -161,15 +164,19 @@ struct Fixture { } workspace->start(Json { { "clientInfo", Json { { "name", "Visual Studio Code" } } }, { "capabilities", Json { { "experimental", Json { { "cxxModules", Json { { "version", 1 }, { "completionReady", ready } } } } } } } }, - false, false); + static_cast(coreFactory), false); load_model(); } // The project: a.cpp and b.cpp, b.cpp compiled with `extra` too. - void load_model(const std::vector& extra = {}, bool engineInterface = false) { + void load_model(const std::vector& extra = {}, bool engineInterface = false, std::string target = {}, int generation = 1) { auto model = std::make_shared(); model->root = root; model->source = mcppls::project::SourceKind::build_database; + if (!target.empty()) { + model->profile.kind = "build-toolchain"; + model->profile.target = std::move(target); + } mcppls::spec::Set set; set.name = "app"; set.hasIde = true; @@ -206,7 +213,7 @@ struct Fixture { model->database.sets.push_back(std::move(engineSet)); } // As the build tool's answer to the load start() began (generation 1); a later one replaces it. - workspace->handle_model_loaded(1, std::move(model)); + workspace->handle_model_loaded(generation, std::move(model)); } ~Fixture() { @@ -280,6 +287,7 @@ constexpr std::string_view A { "#include \nvoid f() {\n doc.size();\n s int main() { using namespace mcppls::testing; + const auto arguments = mcppls::platform::env::arguments(); if (arguments.size() == 1) { // The workspaces keep their answers under the cache: isolated, so a user's cache is never touched. @@ -304,6 +312,174 @@ int main() { std::print(std::cerr, "{}", result->error); return result->timedOut ? 1 : result->exitCode; } + "watched source acknowledgement covers successful run-start inputs, not failures or later disk edits"_test = [] { + for (const int scenario : { 0, 1, 2, 3, 4, 5 }) { + const bool failed { scenario == 1 }, restored { scenario == 2 }; + Fixture f; + const auto source = [](std::string_view hidden, int body) { + return std::format("import active;\n#if 0\nimport {};\n#endif\nint value() {{ return {}; }}\n", hidden, body); + }; + const std::string path { mcppls::base::join_path(f.root, "a.cpp") }; + const std::string initial { scenario == 5 ? "#define SELECT(x) x\nimport SELECT(active);\nint value() { return 0; }\n" + : scenario >= 3 ? "#if 1\nimport active;\n#endif\nint value() { return 0; }\n" + : "import active;\nint value() { return 0; }\n" }; + expect(fs::write_file(path, initial).has_value()); + expect(fs::write_file(mcppls::base::join_path(f.root, "b.cpp"), "export module active;\n").has_value()); + const auto loaded = [&](int generation, bool success) { + auto model = std::make_shared(); + model->root = f.root; + model->detected = mcppls::project::SourceKind::build_database; + if (success) { + model->source = mcppls::project::SourceKind::build_database; + model->watch = { "a.cpp" }; + mcppls::spec::Set set; + set.name = "app"; + for (const char* name : { "a.cpp", "b.cpp" }) { + mcppls::spec::TranslationUnit unit; + unit.source = mcppls::base::join_path(f.root, name); + unit.workDirectory = f.root; + unit.arguments = { "clang++", "-std=c++23", "-c", unit.source }; + if (std::string_view { name } == "a.cpp") unit.requiredModules = { "active" }; + else unit.providedModules = { { "active", "" } }; + set.units.push_back(std::move(unit)); + } + model->database.sets.push_back(std::move(set)); + } else model->issues.emplace_back("producer-failed", "the controlled producer returned no model"); + f.workspace->handle_model_loaded(generation, std::move(model)); + }; + const auto watched = [&] { + f.workspace->handle_watched_files(Json::array({ Json { + { "uri", f.uri("a.cpp") }, { "type", 2 } + } })); + }; + loaded(1, true); + if (scenario == 5) { + watched(); + f.tick(1600ms); + expect(fatal(f.workspace->report()["state"] == "loading")); + loaded(2, true); // SELECT(active) really imports active; this answer is valid. + expect(fs::write_file(path, "#define SELECT(x) x\nimport SELECT(other);\nint value() { return 0; }\n").has_value()); + watched(); + f.tick(1600ms); + expect(f.workspace->report()["state"] == "loading") + << "the same macro definition with another import argument needs a new producer answer"; + continue; + } + if (scenario >= 3) { + if (scenario == 4) { + // The source's active dependency exactly matches the producer's model. + // A successful refresh establishes a baseline for that same structure. + watched(); + f.tick(1600ms); + expect(fatal(f.workspace->report()["state"] == "loading")); + loaded(2, true); + } + expect(fs::write_file(path, "#if 0\nimport active;\n#endif\nint value() { return 0; }\n").has_value()); + watched(); + f.tick(1600ms); + expect(f.workspace->report()["state"] == "loading") + << "disabling the import changes effective dependencies even though lexical names still match the old model; baseline=" << (scenario == 4); + continue; + } + expect(fs::write_file(path, source("hidden.alpha", 1)).has_value()); + watched(); + const auto beforeRun = fs::stamp(path); + f.tick(1600ms); // The existing producer debounce, not a status hold. + expect(fatal(f.workspace->report()["state"] == "loading")); + if (!failed) { + // A checkout changes disk after the run began, before its watch reaches us. + expect(fs::write_file(path, source("hidden.beta", 2)).has_value()); + if (restored) { + expect(fs::write_file(path, source("hidden.alpha", 1)).has_value()); + expect(fatal(fs::stamp(path) != beforeRun)) << "the A -> B -> A control changed the original file stamp"; + } + } + loaded(2, !failed); + expect(fs::write_file(path, source(failed || restored ? "hidden.alpha" : "hidden.beta", 3)).has_value()); + watched(); + f.tick(1600ms); + expect(f.workspace->report()["state"] == "loading") + << (failed ? "a failed answer cannot acknowledge this structure: retry on the next body save" + : restored ? "a changed stamp cannot acknowledge A -> B -> A while the producer ran" + : "a run cannot acknowledge the later disk structure it did not start with"); + loaded(3, true); + expect(fs::write_file(path, source(failed || restored ? "hidden.alpha" : "hidden.beta", 4)).has_value()); + watched(); + f.tick(1600ms); + expect(f.workspace->report()["state"] != "loading") << "a valid unchanged answer acknowledges the run-start structure"; + } + }; + + "a model's status is published only after its synchronous engine apply commits"_test = [] { + struct Probe { + Sink* sink { nullptr }; + bool notifyDuringApply { false }; + unsigned callbacks { 0 }; + unsigned commits { 0 }; + std::vector intermediate; + }; + // clangd's apply calls update_quarantine_issue_ -> host.status_changed() + // before updating the primer and scheduling the new plan's restart. Model that + // permitted reentrant callback, without inventing a timer or an engine failure. + class ApplyingCore : public Core { + public: + ApplyingCore(std::shared_ptr held, std::shared_ptr probe) + : Core { std::move(held) }, probe_ { std::move(probe) } {} + void start(e::Host& host) override { + host_ = &host; + Core::start(host); + } + void apply(const mcppls::normalize::EnginePlan* plan) override { + if (plan == nullptr || probe_->sink == nullptr) return; + if (probe_->notifyDuringApply) { + const auto before = probe_->sink->sent.size(); + ++probe_->callbacks; + host_->status_changed(); + for (std::size_t i { before }; i < probe_->sink->sent.size(); ++i) { + const auto& message = probe_->sink->sent[i]; + if (message.value("method", std::string {}) == "cxxModules/status") { + probe_->intermediate.push_back(message["params"]); + } + } + } + ++probe_->commits; + } + private: + std::shared_ptr probe_; + e::Host* host_ { nullptr }; + }; + for (const bool reentrant : { false, true }) { + auto probe = std::make_shared(); + probe->notifyDuringApply = reentrant; + Fixture f { canonical_root(), "off", true, "cached", + [probe](std::shared_ptr held) -> std::unique_ptr { + return std::make_unique(std::move(held), probe); + } }; + f.workspace->allow_status_notifications(); + // Use the existing explicit refresh API: its asynchronous response is delivered + // below as generation 2. No sleep or status coalescing timeout is involved. + f.workspace->apply_build_program(); + expect(f.workspace->report()["state"] == "loading") << "the replacement model is being described"; + f.sink.sent.clear(); + probe->sink = &f.sink; + f.load_model({}, false, "aarch64-unknown-linux-gnu", 2); + expect(probe->commits == 1u) << "the model actually reached its engine"; + expect(probe->callbacks == (reentrant ? 1u : 0u)) << "the callback control was exercised"; + expect(probe->intermediate.empty()) + << "a synchronous apply callback exposed the new profile before engine commit: " << Json(probe->intermediate).dump(); + const auto statuses = f.sink.notifications("cxxModules/status"); + expect(statuses > 0u) << "the committed model still needs a status notification"; + const auto last = std::ranges::find_if(f.sink.sent.rbegin(), f.sink.sent.rend(), [](const Json& message) { + return message.value("method", std::string {}) == "cxxModules/status"; + }); + if (last != f.sink.sent.rend()) { + expect((*last)["params"]["profile"]["target"] == "aarch64-unknown-linux-gnu"); + } + // No async work or elapsed-time hold makes this assertion pass: both cases + // use the same completed apply, with only the real callback pattern toggled. + probe->sink = nullptr; + } + }; "rename refuses an engine-interface owner even when the compiler edits only project references"_test = [] { for (int shape { 0 }; shape < 3; ++shape) { Fixture f; diff --git a/tests/test_conformance.cpp b/tests/test_conformance.cpp index eda968d9..8f5e5a33 100644 --- a/tests/test_conformance.cpp +++ b/tests/test_conformance.cpp @@ -53,6 +53,11 @@ int serve() { const bool never { std::ranges::find(arguments, "--drain-never") != arguments.end() }; const bool missing { std::ranges::find(arguments, "--drain-missing") != arguments.end() }; const bool restarting { std::ranges::find(arguments, "--drain-restarting") != arguments.end() }; + const bool timelineStates { std::ranges::find(arguments, "--timeline-states") != arguments.end() }; + const bool completionTrace { std::ranges::find(arguments, "--completion-trace") != arguments.end() }; + const bool completionOverflow { std::ranges::find(arguments, "--completion-trace-overflow") != arguments.end() }; + int traceReports { 0 }; + std::optional lateCompletion; int reports { 0 }, changes { 0 }, completions { 0 }; const auto send = [](const Json& message) { return platform::stdio::write_output(lsp::encode_frame(message)); }; const auto status = [&](std::string state) { return send(lsp::make_notification("cxxModules/status", Json { { "state", std::move(state) } })); }; @@ -66,6 +71,28 @@ int serve() { const Json& request { **frame }; const std::string method { request.value("method", std::string {}) }; if (method == "exit") return 0; + if (timelineStates && method == "initialized") { + const auto sample = [&](std::string state, int done, int total, std::string issue = {}) { + Json issues = Json::array(); + if (!issue.empty()) issues.push_back(Json { { "code", issue }, { "message", "private-status-text" } }); + return send(lsp::make_notification("cxxModules/status", Json { + { "state", std::move(state) }, { "progress", Json { { "done", done }, { "total", total } } }, { "issues", std::move(issues) } + })); + }; + (void)sample("loading", 0, 3); + (void)sample("loading", 1, 3); // Progress changes are not state transitions. + (void)sample("preparing", 1, 3, "preparation-stalled"); + (void)sample("preparing", 2, 3); + (void)sample("ready", 3, 3); + (void)sample("error", 3, 3, "mock-state-error"); + (void)sample("ready", 3, 3); + // Exercise the exact sentinel recorded by Client::reset without a slow OS kill. + (void)sample("restart", 0, 0); + (void)sample("ready", 0, 2); + (void)sample("loading", 0, 2); + (void)sample("ready", 2, 2); + (void)sample("ready", 2, 2); + } if (drain && method == "initialized") { if (!cacheMode.empty()) { (void)fs::create_directories(base::parent_path(readCopy)); @@ -89,6 +116,16 @@ int serve() { if (method == "initialize") result = Json { { "capabilities", Json { { "experimental", Json { { "cxxModules", Json::object() } } }, { "hoverProvider", true } } } }; if (method == "textDocument/completion") { ++completions; + if (completionTrace && completions == 1) std::this_thread::sleep_for(std::chrono::milliseconds { 30 }); + if (completionOverflow && completions == 270) std::this_thread::sleep_for(std::chrono::milliseconds { 500 }); + if (completionTrace && completions == 2) { + (void)send(lsp::make_error(request["id"], -32001, "private-completion-error")); + continue; + } + if (completionTrace && completions == 3) { + lateCompletion = request["id"]; + continue; // It arrives only with the post-sampling report, after statistics are frozen. + } result = Json { { "isIncomplete", completions == 1 }, { "items", Json::array({ Json { { "label", "private-completion-text" } } }) } }; if (completions == 2) result["items"].push_back(Json { { "label", "second-private-item" } }); @@ -101,6 +138,30 @@ int serve() { { "details", Json { { "preparation", Json { { "runningUnits", Json::array({ Json { { "module", "sample" }, { "seconds", 12 } } }) } } } } } } }) } } }) } }; + if ((completionTrace || completionOverflow) && method == "cxxModules/report") { + ++traceReports; + if (traceReports == 2 && lateCompletion) { + (void)send(lsp::make_result(*lateCompletion, Json { { "items", Json::array({ Json { { "label", "private-late-item" } } }) } })); + lateCompletion.reset(); + } + result["traceObservation"] = Json { { "completionRequests", completions }, { "reportRequests", traceReports } }; + if (completionTrace) { + result["roots"][0]["requests"] = Json { { "textDocument/completion", Json { + { "count", 10 + completions }, { "empty", 0 }, { "errors", completions > 0 ? 1 : 0 }, { "cancelled", 0 }, + { "p50Ms", 12 }, { "p95Ms", 34 }, { "maxMs", 8765 }, { "engineAnswered", 7 }, + { "engineP50Ms", 10 }, { "engineP95Ms", 30 }, { "overheadP50Ms", 2 }, { "overheadP95Ms", 4 }, + { "answeredBy", Json { { "clangd", 7 } } }, { "lastAt", "fake-at" }, { "file", "private-method-file" } + } } }; + result["roots"][0]["eventTotals"] = Json { { "slow-request", traceReports == 1 ? 5 : 8 } }; + Json slowEvents = Json::array(); + for (int index { 0 }; index < 4; ++index) slowEvents.push_back(Json { + { "at", std::format("fake-at-{}", index) }, { "kind", "slow-request" }, + { "detail", Json { { "method", "textDocument/completion" }, { "ms", 8765 + index }, + { "outcome", "result" }, { "answeredBy", "clangd" }, { "file", "private-slow-request-file" } } } + }); + result["roots"][0]["events"] = std::move(slowEvents); + } + } if (drain && method == "cxxModules/report") { const bool busy { !cacheMode.empty() ? (++reports < 2 || cacheMode == "--cache-never") : !restarting && (never || ++reports < 3) }; @@ -226,6 +287,54 @@ int main() { expect(result->output.find("engine work evidence unavailable") != std::string::npos); } }; + "a failed timeline preserves exact counted transitions and restart boundaries"_test = [&] { + const Json sample { { "name", "timing" }, { "server-arguments", Json::array({ "--timeline-states" }) }, + { "checks", Json::array({ + Json { { "id", "PUMP" }, { "kind", "hover-contains" }, { "file", "main.cpp" }, { "at", Json::array({ 0, 0 }) }, { "expect", "中文" } }, + Json { { "id", "STATES" }, { "kind", "timeline" }, { "budget", Json { { "maxChangesPerMinute", 5 } } } } + }) } + }; + expect(fs::write_file(base::join_path(fixture, "scenario.json"), sample.dump()).has_value()); + const auto result = invoke(measure, evidence); + expect(fatal(result.has_value())); + expect(result->exitCode == 1) << result->output << result->error; + const auto document = fs::read_file(measure); + expect(fatal(document.has_value())); + const Json summary = Json::parse(*document); + expect(summary["checks"][0]["ok"] == true); + expect(summary["checks"][1]["ok"] == false); + const Json& timeline = summary["checks"][1]["measure"]; + expect(timeline["stateChanges"] == 6 && timeline["busiestMinuteChanges"] == 6); + expect(fatal(timeline.contains("stateTransitionTrace") && timeline.contains("stateTransitionStatusSamples") + && timeline.contains("busiestMinuteWindow"))) << "the failing gate must retain its transition evidence"; + expect(timeline["stateTransitionStatusSamples"] == 12 && timeline["errorStates"] == 1); + const Json& trace = timeline["stateTransitionTrace"]; + expect(fatal(trace.is_array() && trace.size() == 9)); + expect(trace[0]["kind"] == "initial" && trace[0]["from"].is_null() && trace[0]["counted"] == false); + expect(trace[1]["from"] == "loading" && trace[1]["to"] == "preparing"); + expect(trace[1]["done"] == 1 && trace[1]["total"] == 3 && trace[1]["issues"] == Json::array({ "preparation-stalled" })); + expect(trace[5]["kind"] == "restart" && trace[5]["to"] == "restart" && trace[5]["counted"] == false); + expect(trace[6]["kind"] == "initial" && trace[6]["from"].is_null() && trace[6]["counted"] == false); + Json counted = Json::array(); + std::int64_t previous { -1 }; + for (std::size_t i { 0 }; i < trace.size(); ++i) { + const auto at { trace[i]["sinceStartNanoseconds"].get() }; + expect(at >= previous && trace[i]["seconds"].get() >= 0.0); + previous = at; + if (trace[i]["counted"].get()) counted.push_back(i); + } + expect(counted.size() == timeline["stateChanges"].get()); + const Json& window = timeline["busiestMinuteWindow"]; + expect(window["traceIndices"] == counted && window["count"] == counted.size()); + const auto start { window["startNanoseconds"].get() }; + const auto end { window["endNanosecondsExclusive"].get() }; + expect(end - start == 60'000'000'000LL); + for (const auto& index : window["traceIndices"]) { + const auto at { trace[index.get()]["sinceStartNanoseconds"].get() }; + expect(at >= start && at < end); + } + expect(trace.dump().find("private-status-text") == std::string::npos) << "keep only issue codes, no full status"; + }; "typing trace preserves per-word requests and records the original completion metadata"_test = [&] { const Json sample { { "name", "timing" }, { "checks", Json::array({ Json { { "id", "WORDS" }, { "kind", "typing" }, { "file", "main.cpp" }, { "seconds", 1 }, { "hz", 5 }, @@ -261,6 +370,126 @@ int main() { } expect(trace.dump().find("private") == std::string::npos) << "trace keeps counts, never completion text"; }; + "failed periodic typing keeps exact completion evidence at the original statistics boundary"_test = [&] { + const Json sample { { "name", "timing" }, { "server-arguments", Json::array({ "--completion-trace" }) }, + { "checks", Json::array({ Json { { "id", "PERIODIC" }, { "kind", "typing" }, { "file", "main.cpp" }, + { "seconds", 1 }, { "hz", 5 }, { "completion-every-ms", 100 }, { "requestTimeout", 0 }, + { "scripts", Json::array({ Json { { "line", 0 }, { "text", "abc" } } }) }, + { "budget", Json { { "max", 0 } } } + } }) } }; + expect(fs::write_file(base::join_path(fixture, "scenario.json"), sample.dump()).has_value()); + const auto result = invoke(measure, evidence); + expect(fatal(result.has_value())); + expect(result->exitCode == 1) << result->output << result->error; + const auto text = fs::read_file(measure); + expect(fatal(text.has_value())); + const Json summary = Json::parse(*text); + expect(summary["checks"][0]["ok"] == false); + const Json& typed = summary["checks"][0]["measure"]; + expect(typed["reask"] == "" && !typed.contains("wordTrace")); + expect(fatal(typed.contains("completionTrace"))); + const Json& trace = typed["completionTrace"]; + const Json& entries = trace["entries"]; + expect(trace["origin"] == "typing-start" && trace["population"] == "original-completion-statistics"); + expect(trace["limit"] == 256 && trace["dropped"] == 0); + expect(fatal(entries.size() >= 3 && entries.size() == typed["completions"].get())); + expect(trace["total"] == entries.size()); + expect(entries[0]["items"] == 1 && entries[0]["isIncomplete"] == true); + expect(entries[1]["error"] == true && entries[1]["items"].is_null()); + expect(entries[2]["answeredNanoseconds"].is_null() && entries[2]["elapsedNanoseconds"].is_null()); + expect(entries[2]["elapsedSeconds"].is_null()) << "the late response must not rewrite the original timeout"; + std::vector answered; + std::size_t errors { 0 }, unanswered { 0 }; + std::int64_t previousId { -1 }, previousSent { -1 }; + for (std::size_t index { 0 }; index < entries.size(); ++index) { + const auto& entry = entries[index]; + expect(entry["index"] == index); + expect(entry["id"].is_number_integer() && entry["sentNanoseconds"].is_number_integer()); + const auto id = entry["id"].get(); + const auto sent = entry["sentNanoseconds"].get(); + expect(id > previousId && sent >= previousSent && sent >= 0); + previousId = id; + previousSent = sent; + if (entry["answeredNanoseconds"].is_null()) { ++unanswered; continue; } + expect(entry["answeredNanoseconds"].is_number_integer() && entry["elapsedNanoseconds"].is_number_integer()); + expect(entry["answeredNanoseconds"].get() - sent == entry["elapsedNanoseconds"].get()); + expect(entry["elapsedNanoseconds"].get() >= 0); + if (entry["error"].get()) ++errors; + else answered.push_back(entry["elapsedSeconds"].get()); + } + expect(unanswered == 1 && typed["clientTimeouts"] == unanswered); + expect(errors == 1 && typed["errors"] == errors); + std::ranges::sort(answered); + expect(fatal(!answered.empty())); + expect(typed["answered"] == answered.size() && typed["completion"]["n"] == answered.size()); + for (const auto& [field, fraction] : std::array, 2> {{ { "p50", 0.5 }, { "p95", 0.95 } }}) { + const auto rank = static_cast(std::ceil(fraction * static_cast(answered.size()))); + expect(typed["completion"][std::string { field }] == answered[rank - 1]); + } + expect(typed["completion"]["max"] == answered.back()); + expect(trace["slowestSuccessful"]["elapsedSeconds"] == answered.back()); + expect(trace.dump().find("private") == std::string::npos && trace.dump().find("main.cpp") == std::string::npos); + const Json& server = typed["serverCompletionEvidence"]; + expect(server["requestsBefore"]["count"] == 10); + expect(server["requestsAfter"]["count"] == 10 + entries.size()); + expect(server["requestsAfter"]["engineP95Ms"] == 30 && server["requestsAfter"]["overheadP95Ms"] == 4); + expect(server["recentSampleLimit"] == 256); + const Json& slow = server["slowRequests"]; + expect(slow["totalBefore"] == 5 && slow["totalAfter"] == 8 && slow["added"] == 3); + expect(slow["afterRecentCount"] == 4 && slow["droppedByTraceLimit"] == 0); + expect(slow["newRetained"] == 3 && slow["newNotRetained"] == 0); + expect(slow["eventsAfter"].size() == 4 && slow["eventsAfter"][0].size() == 8) << "at, the four names and the three spans (0.0.17 M3)"; + expect(slow["eventsAfter"][0]["at"] == "fake-at-0" && slow["eventsAfter"][0]["ms"] == 8765); + expect(server.dump().find("private") == std::string::npos) << "only public method statistics and whitelisted event fields"; + const auto artifact = fs::read_file(evidence); + expect(fatal(artifact.has_value())); + const Json observed = Json::parse(*artifact)["report"]["traceObservation"]; + expect(observed["completionRequests"] == entries.size() && observed["reportRequests"] == 3) + << "tracing must add neither completion requests nor reports"; + }; + "truncated completion traces still identify the original slowest response"_test = [&] { + const Json sample { { "name", "timing" }, { "server-arguments", Json::array({ "--completion-trace-overflow" }) }, + { "checks", Json::array({ Json { { "id", "OVERFLOW" }, { "kind", "typing" }, { "file", "main.cpp" }, + // At least 300 requests on every OS (the mock delays the 270th): Windows sleeps in 15.6 ms steps, so a + // second at "1000 Hz" there was 65 keystrokes, and the slow request was never asked (0.0.17 M5). + { "seconds", 6 }, { "hz", 100 }, { "completion-every-ms", 0 }, { "reask", "always" }, { "requestTimeout", 5 }, + { "scripts", Json::array({ Json { { "line", 0 }, { "text", std::string(300, 'a') } } }) } + } }) } }; + expect(fs::write_file(base::join_path(fixture, "scenario.json"), sample.dump()).has_value()); + const auto result = invoke(measure, evidence); + expect(fatal(result.has_value())); + expect(result->exitCode == 0) << result->output << result->error; + const auto text = fs::read_file(measure); + expect(fatal(text.has_value())); + const Json typed = Json::parse(*text)["checks"][0]["measure"]; + expect(fatal(typed.contains("completionTrace"))); + const Json& trace = typed["completionTrace"]; + const auto total = typed["completions"].get(); + expect(total >= 300 && trace["entries"].size() == 256); + expect(trace["total"] == total && trace["dropped"] == total - 256); + // What a failure here needs to be read (0.0.17 M5, seen on Windows only): both sides of each comparison, and the + // slowest entries of the retained prefix with their send and answer times. + std::string observed { std::format("max {} n {} total {}; slowestSuccessful {}; slowestAnswered {}; prefix slowest:", + typed["completion"]["max"].dump(), typed["completion"]["n"].dump(), total, + trace["slowestSuccessful"].dump(), trace["slowestAnswered"].dump()) }; + { + std::vector prefix { trace["entries"].begin(), trace["entries"].end() }; + std::ranges::sort(prefix, std::greater {}, [](const Json& entry) { + const Json& elapsed { entry["elapsedNanoseconds"] }; + return elapsed.is_number_integer() ? elapsed.get() : std::int64_t { 0 }; + }); + for (std::size_t i { 0 }; i < std::min(3, prefix.size()); ++i) observed += " " + prefix[i].dump(); + } + expect(trace["slowestSuccessful"]["elapsedSeconds"] == typed["completion"]["max"]) << observed; + expect(trace["slowestAnswered"]["elapsedSeconds"] == typed["completion"]["max"]) << observed; + expect(trace["slowestSuccessful"]["index"].get() >= 256) + << "the deliberately slow response beyond the prefix must survive truncation: " << observed; + expect(typed["answered"] == total && typed["completion"]["n"] == total); + expect(trace.dump().find("private") == std::string::npos); + const Json& server = typed["serverCompletionEvidence"]; + expect(server["requestsBefore"].is_null() && server["requestsAfter"].is_null()); + expect(server["slowRequests"]["eventsAfter"].is_null() && server["slowRequests"]["newRetained"].is_null()); + }; "module reuse drains partial read copies and preserves cold, rebuild and missing-state checks"_test = [&] { // Owned payloads are Linux's cache representation; its source scanner uses POSIX paths. if constexpr (mcppls::os::FAMILY != mcppls::os::Family::linux) return; diff --git a/tests/test_process.cpp b/tests/test_process.cpp index 20754d63..1c9f0996 100644 --- a/tests/test_process.cpp +++ b/tests/test_process.cpp @@ -229,10 +229,128 @@ int main() { sent = sent && (*connection)->send(nlohmann::json { { "jsonrpc", "2.0" }, { "method", "m" }, { "params", big } }).has_value(); } expect(sent); - expect(std::chrono::steady_clock::now() - started < 2s) << "4 MB to a peer that reads none of it"; + const auto sendElapsed = std::chrono::steady_clock::now() - started; + expect(sendElapsed < 2s) << "4 MB to a peer that reads none of it; actual " + << std::chrono::duration(sendElapsed).count() << " s"; const auto stopping = std::chrono::steady_clock::now(); (*connection)->stop(200ms); - expect(std::chrono::steady_clock::now() - stopping < 10s) << "the writer, blocked in a write, lets go once the peer is gone"; + const auto stopElapsed = std::chrono::steady_clock::now() - stopping; + std::println("connection nonreader evidence: 64 x 64 KiB send={:.9f}s; stop={:.9f}s", + std::chrono::duration(sendElapsed).count(), std::chrono::duration(stopElapsed).count()); + expect(stopElapsed < 10s) << "the writer, blocked in a write, lets go once the peer is gone; actual " + << std::chrono::duration(stopElapsed).count() << " s"; + }; + + "connection writer drains owned snapshots in FIFO order through both framings"_test = [&] { + using namespace std::chrono_literals; + using Json = nlohmann::json; + for (const auto framing : { mcppls::lsp::Framing::content_length, mcppls::lsp::Framing::lines }) { + const std::string_view framingName { framing == mcppls::lsp::Framing::lines ? "lines" : "content-length" }; + struct Received { + std::mutex mutex; + std::condition_variable ready; + std::vector messages; + bool release { false }; + bool gateTimedOut { false }; + bool closed { false }; + } received; + auto connection = mcppls::lsp::Connection::start( + { .program = self, .arguments = { "--echo" } }, + [&](Json message) { + std::unique_lock lock { received.mutex }; + received.messages.push_back(std::move(message)); + received.ready.notify_all(); + // Hold the first reply while later messages queue. A large following frame + // also supplies backpressure, so the writer must own queued nested values. + if (received.messages.size() == 1 + && !received.ready.wait_for(lock, 20s, [&] { return received.release; })) received.gateTimedOut = true; + }, + [&] { + const std::lock_guard lock { received.mutex }; + received.closed = true; + received.ready.notify_all(); + }, {}, framing); + expect(fatal(connection.has_value())) << framingName; + // On any assertion/exception unwind, release the callback before Connection joins it. + struct Release { + Received& received; + void open() { + { const std::lock_guard lock { received.mutex }; received.release = true; } + received.ready.notify_all(); + } + ~Release() { open(); } + } release { received }; + std::vector expected; + const auto send = [&](Json message) { + expected.push_back(message); + return (*connection)->send(std::move(message)).has_value(); + }; + bool sent { send(Json { { "jsonrpc", "2.0" }, { "id", 0 }, { "result", "first" } }) }; + bool firstArrived { false }; + { + std::unique_lock lock { received.mutex }; + firstArrived = received.ready.wait_for(lock, 20s, [&] { return !received.messages.empty() || received.closed; }) + && !received.messages.empty(); + } + expect(firstArrived) << framingName << " first echo did not arrive"; + sent = send(Json { { "jsonrpc", "2.0" }, { "id", 1 }, { "result", std::string(4 * 1024 * 1024, 'b') } }) && sent; + Json original { { "jsonrpc", "2.0" }, { "id", 2 }, + { "result", { { "nested", { { "text", "h\xC3\xA9llo \xF0\x9F\x98\x80\n" }, + { "values", Json::array({ 1, 2, 3 }) } } } } } }; + expected.push_back(original); + sent = (*connection)->send(original).has_value() && sent; + original["result"]["nested"]["text"] = "changed after send"; + original["result"]["nested"]["values"][0] = 99; + original.clear(); + { + Json temporary { { "jsonrpc", "2.0" }, { "id", 3 }, { "result", Json::array({ "temporary", 7 }) } }; + expected.push_back(temporary); + sent = (*connection)->send(std::move(temporary)).has_value() && sent; + } // The moved-from source is destroyed before the writer is released. + Json invalid { { "jsonrpc", "2.0" }, { "id", 4 }, { "result", std::string { "A\xFF\xFE" "Z" } } }; + expected.push_back(Json { { "jsonrpc", "2.0" }, { "id", 4 }, { "result", "A\xEF\xBF\xBD\xEF\xBF\xBD" "Z" } }); + sent = (*connection)->send(std::move(invalid)).has_value() && sent; + sent = send(Json { { "jsonrpc", "2.0" }, { "id", 5 }, { "result", "last" } }) && sent; + release.open(); + // Stop without first waiting for all replies: normal shutdown must drain the queue, + // close stdin, then allow --echo to finish and the reader to deliver every reply. + (*connection)->stop(20s); + expect(sent) << framingName; + expect(!received.gateTimedOut) << framingName << " first-reply gate timed out"; + expect(received.closed && (*connection)->closed()) << framingName; + expect(received.messages.size() == expected.size()) << framingName << " expected " << expected.size() + << " replies, received " << received.messages.size(); + for (std::size_t i { 0 }; i < std::min(received.messages.size(), expected.size()); ++i) { + expect(received.messages[i] == expected[i]) << framingName << " reply " << i << " differs from its sent snapshot"; + } + expect(!(*connection)->send(Json { { "after", "stop" } }).has_value()) << framingName; + (*connection)->stop(200ms); // Repeated cleanup must remain harmless after a normal drain. + } + }; + + "connection reports a departed peer and stopping it cleans up"_test = [&] { + using namespace std::chrono_literals; + std::mutex mutex; + std::condition_variable ready; + bool closed { false }; + auto connection = mcppls::lsp::Connection::start( + { .program = self, .arguments = { "--exit", "0" } }, [](nlohmann::json) {}, + [&] { const std::lock_guard lock { mutex }; closed = true; ready.notify_all(); }); + expect(fatal(connection.has_value())); + bool observedClosed { false }; + { + std::unique_lock lock { mutex }; + observedClosed = ready.wait_for(lock, 20s, [&] { return closed; }); + } + expect(observedClosed) << "the real child exited without closing the connection"; + const auto sent = (*connection)->send(nlohmann::json { { "jsonrpc", "2.0" }, { "method", "after-exit" } }); + expect(!sent.has_value()); + if (!sent) expect(sent.error().code == "connection-closed" || sent.error().code == "connection-write") << sent.error().code; + const auto stopping = std::chrono::steady_clock::now(); + (*connection)->stop(200ms); + (*connection)->stop(200ms); + expect(std::chrono::steady_clock::now() - stopping < 10s) << "departed peer cleanup"; + expect((*connection)->closed()); }; "writing after the child exits returns an error and preserves the parent"_test = [&] { diff --git a/tests/test_project.cpp b/tests/test_project.cpp index 50260820..2c993326 100644 --- a/tests/test_project.cpp +++ b/tests/test_project.cpp @@ -382,9 +382,12 @@ int main() { }; "B-5: xmake's environment, arguments and classification"_test = [] { - const auto environment = p::xmake_environment("/cache/xmake/config"); - expect(std::ranges::find(environment, std::pair { "XMAKE_CONFIGDIR", "/cache/xmake/config" }) != environment.end()); + const auto environment = p::xmake_environment("/cache/xmake/state/config", "/cache/xmake/state/tmp"); + expect(std::ranges::find(environment, std::pair { "XMAKE_CONFIGDIR", "/cache/xmake/state/config" }) != environment.end()); expect(std::ranges::find(environment, std::pair { "XMAKE_THEME", "plain" }) != environment.end()); + // XS-1 (#69): the module mapper files of the private run stay out of the temporary directory the user's builds read. + expect(std::ranges::find(environment, std::pair { "XMAKE_TMPDIR", "/cache/xmake/state/tmp" }) != environment.end()); + expect(std::ranges::none_of(p::xmake_environment("/c"), [](const auto& entry) { return entry.first == "XMAKE_TMPDIR"; })); const auto offline = p::xmake_configure_arguments("/cache/xmake/build", true); expect(offline == (std::vector { "f", "-c", "--confirm=no", "--policies=package.fetch_only,network.mode:private", @@ -410,6 +413,30 @@ int main() { expect(p::xmake_configuration_key(true, manifest) == p::xmake_configuration_key(true, manifest)); }; + "XS-2, XS-3 (#68): the private state's key, and xmake saying that state is out of date"_test = [] { + const std::string configuration { p::xmake_configuration_key(true, std::pair { 64, 1000 }) }; + const std::pair tool { 4096, 7000 }; + const std::string key { p::xmake_state_key(configuration, "/usr/bin/xmake", tool, 1000, 1) }; + expect(key == p::xmake_state_key(configuration, "/usr/bin/xmake", tool, 1000, 1)); + expect(key != p::xmake_state_key(configuration, "/opt/xmake", tool, 1000, 1)) << "another xmake"; + expect(key != p::xmake_state_key(configuration, "/usr/bin/xmake", std::pair { 4096, 8000 }, 1000, 1)) << "xmake upgraded in place"; + expect(key != p::xmake_state_key(configuration, "/usr/bin/xmake", tool, 1500, 1)) << "an included xmake.lua changed"; + expect(key != p::xmake_state_key(configuration, "/usr/bin/xmake", tool, 1000, 2)) << "an xmake.lua was added"; + expect(key != p::xmake_state_key(p::xmake_configuration_key(false, std::pair { 64, 1000 }), "/usr/bin/xmake", tool, 1000, 1)); + + // Recorded from xmake 3.1.1 and 3.1.2 (#68's bundle and a local reproduction). + expect(p::xmake_state_stale("error: target(json): dependfile for '/usr/include/c++/16/bits/std.compat.cc' not found at " + "'../../../.cache/mcppls/workspaces/json-parser-ddc093863fd8c441/xmake/build/.deps/json/linux/x86_64/debug/usr/include/c++/16/bits/std.compat.cc.d'")); + expect(p::xmake_state_stale("error: target(json): dependfile for 'module/data.cppm' not found at 'build/.deps/json/linux/x86_64/release/module/data.cppm.d'")); + expect(p::xmake_state_stale("error: module scanner did not run for target(json), maybe a custom `on_prepare()` script overrides the modules one")); + expect(p::xmake_state_stale("error: target(json): failed to load moduleinfo for 'module/data.cppm'")); + // The project's own errors are not the state's: they are said to the user as they are. + expect(!p::xmake_state_stale("error: missing json dependency for module test/basic.cpp")); + expect(!p::xmake_state_stale("error: Invalid option: --fancy=y")); + expect(!p::xmake_state_stale("The packages(xxhash, fmt) not found, please install them first!")); + expect(!p::xmake_state_stale("error: json: error: failed to read compiled module: No such file or directory")); + }; + "X-2: an xmake.conf is parsed as text, top-level scalars only"_test = [] { // Recorded 2026-10-01 from `xmake f -m debug --fancy=y --tag=zz` (xmake v3.1.1) in a project that declares two options. const std::string conf { diff --git a/tests/test_scan.cpp b/tests/test_scan.cpp index d009c9df..79b01b49 100644 --- a/tests/test_scan.cpp +++ b/tests/test_scan.cpp @@ -12,6 +12,45 @@ using mcppls::spec::Role; int main() { using namespace mcppls::testing; + "source acknowledgement preserves preprocessor inputs independently of lexical import names"_test = [] { + const auto disabled = scan_source("/* prefix */ #if /* condition\n continued */ 0\nimport hidden;\n#endif\nint value() { return 1; }\n", true); + const auto enabled = scan_source("/* prefix */ #if /* condition\n continued */ 1\nimport hidden;\n#endif\nint value() { return 1; }\n", true); + expect(required_names(disabled) == required_names(enabled)); + expect(disabled.preprocessorDirectives != enabled.preprocessorDirectives) + << "the same import names can have different effective dependency graphs"; + expect(disabled.preprocessorDirectives == "#if /* condition\n continued */ 0\n#endif\n"); + const auto body = scan_source("/* prefix */ #if /* condition\n continued */ 0\nimport hidden;\n#endif\nint value() { return 2; }\n", true); + expect(disabled.preprocessorDirectives == body.preprocessorDirectives); + const auto macro = scan_source("#define FLAG \\\n 0\n#if FLAG\nimport hidden;\n#endif\n", true); + expect(macro.preprocessorDirectives == "#define FLAG \\\n 0\n#if FLAG\n#endif\n"); + const auto literal = scan_source("// #define COMMENT 1\nconst char* text = \"#if 1\";\n", true); + expect(literal.preprocessorDirectives.empty() && literal.preprocessorComplete); + expect(!scan_source("int x; /* across\n lines */ #if 0\nimport hidden;\n#endif\n").preprocessorComplete) + << "an unclassified code-level # must not acknowledge a source"; + const std::string_view text { "#define FEATURE 1\nexport module sample;\n#if FEATURE\nimport hidden;\n#endif\n" }; + const auto ordinary = scan_source(text), retained = scan_source(text, true); + expect(ordinary.preprocessorDirectives.empty() && !retained.preprocessorDirectives.empty()); + expect(fatal(ordinary.declaration.has_value() && retained.declaration.has_value())); + expect(ordinary.declaration->module == retained.declaration->module + && ordinary.declaration->isExported == retained.declaration->isExported + && ordinary.declaration->nameRange == retained.declaration->nameRange); + expect(required_names(ordinary) == required_names(retained) && role_of(ordinary) == role_of(retained)); + const auto active = scan_source("#define SELECT(x) x\nimport SELECT(active);\n", true); + const auto other = scan_source("#define SELECT(x) x\nimport SELECT(other);\n", true); + expect(active.preprocessorDirectives == other.preprocessorDirectives && required_names(active) == required_names(other)); + expect(!active.importsComplete && !other.importsComplete) + << "macro arguments after import can change real dependencies without changing lexical names"; + expect(!scan_source("export module unfinished").declarationsComplete); + expect(!scan_source("export module SELECT(active);\n", true).declarationsComplete); + expect(scan_source("module;\nexport module complete;\n").declarationsComplete); + const auto oneHeader = scan_source("import \"one.hpp\";\n", true); + const auto twoHeaders = scan_source("import \"two.hpp\";\n", true); + expect(required_names(oneHeader) == required_names(twoHeaders)); + expect(fatal(oneHeader.imports.size() == 1 && twoHeaders.imports.size() == 1)); + expect(oneHeader.imports[0].isHeaderUnit && twoHeaders.imports[0].isHeaderUnit + && oneHeader.imports[0].header != twoHeaders.imports[0].header); + }; + "the fixture's primary interface"_test = [] { const auto result = scan_source("export module hello.greet;\nexport import :detail;\nimport std;\n\n" "export namespace hello {\n std::string greet(std::string_view who) { return detail::prefix() + std::string(who); }\n}\n"); diff --git a/tests/test_session.cpp b/tests/test_session.cpp new file mode 100644 index 00000000..98f40e97 --- /dev/null +++ b/tests/test_session.cpp @@ -0,0 +1,61 @@ +import std; +import nlohmann.json; +import mcppls.testing; +import mcppls.base.path; +import mcppls.base.uri; +import mcppls.platform.fs; +import mcppls.platform.dirs; +import mcppls.server.session; + +using Json = nlohmann::json; +namespace fs = mcppls::platform::fs; +namespace b = mcppls::base; + +int main() { + using namespace mcppls::testing; + const std::string root { b::join_path(mcppls::platform::dirs::temp_directory(), + std::format("mcppls-test-session-{}", std::chrono::steady_clock::now().time_since_epoch().count())) }; + const std::string directory { b::join_path(root, "src/greet") }; + const std::string source { b::join_path(directory, "greet.cppm") }; + (void)fs::create_directories(directory); + (void)fs::write_file(source, "export module hello.greet;\n"); + auto folders = [](const std::string& path) { + return Json::array({ Json { { "uri", b::path_to_uri(path) }, { "name", b::file_name(path) } } }); + }; + + "Zed file worktree falls back to its directory rootUri"_test = [&] { + const Json params { { "rootUri", b::path_to_uri(directory) }, { "rootPath", directory }, + { "workspaceFolders", folders(source) } }; + const auto chosen = mcppls::server::workspace_folders(params, root); + expect(fatal(chosen.size() == 1)); + expect(chosen[0].path == directory); + expect(chosen[0].uri == b::path_to_uri(directory)); + }; + + "a file-only initialize never makes the source its working directory"_test = [&] { + Json params { { "rootUri", b::path_to_uri(source) }, { "rootPath", source }, { "workspaceFolders", folders(source) } }; + auto chosen = mcppls::server::workspace_folders(params, root); + expect(fatal(chosen.size() == 1)); + expect(chosen[0].path == root) << "use the existing cwd fallback, without guessing a parent project"; + params["rootPath"] = directory; + chosen = mcppls::server::workspace_folders(params, root); + expect(fatal(chosen.size() == 1)); + expect(chosen[0].path == directory); + }; + + "valid and not-yet-created folders retain precedence and spelling"_test = [&] { + const std::string absent { b::join_path(root, "not-created") }; + Json params { { "rootUri", b::path_to_uri(root) }, { "workspaceFolders", folders(source) } }; + params["workspaceFolders"].push_back(folders(directory)[0]); + params["workspaceFolders"].push_back(folders(absent)[0]); + const auto chosen = mcppls::server::workspace_folders(params, root); + expect(fatal(chosen.size() == 2)); + expect(chosen[0].path == directory); + expect(chosen[0].uri == params["workspaceFolders"][1]["uri"].get()); + expect(chosen[1].path == absent); + expect(chosen[1].uri == params["workspaceFolders"][2]["uri"].get()); + }; + + fs::remove_all(root); + return report(); +} diff --git a/tools/devtools/src/measure.cpp b/tools/devtools/src/measure.cpp index e8433403..498787af 100644 --- a/tools/devtools/src/measure.cpp +++ b/tools/devtools/src/measure.cpp @@ -163,11 +163,16 @@ double percentile_of(std::vector values, double fraction) { } // namespace -base::Result budgets(const std::string& directory) { +namespace { + +using Samples = std::map, std::vector>; // (check, number) -> one per run + +// Every number of every check's `measure` in the --measure files under `directory`, at any depth. +base::Result collect(const std::string& directory) { if (!fs::is_directory(directory)) return base::fail("measure-dir", std::format("{} is not a directory", directory)); std::vector files; measure_files(directory, files); - std::map, std::vector> values; // (check, number) -> one per run + Samples values; for (const auto& path : files) { auto text = fs::read_file(path); if (!text) return std::unexpected { text.error() }; @@ -185,8 +190,31 @@ base::Result budgets(const std::string& directory) { for (const auto& [name, number] : numbers) values[{ check.value("id", std::string {}), name }].push_back(number); } } + return values; +} + +double deviation_of(const std::vector& values) { + if (values.size() < 2) return 0; + double mean { 0 }; + for (const double value : values) mean += value; + mean /= static_cast(values.size()); + double squares { 0 }; + for (const double value : values) squares += (value - mean) * (value - mean); + return std::sqrt(squares / static_cast(values.size() - 1)); +} + +// The numbers a release is compared on: what a person waits for while typing and saving, how often the state +// changes, what an idle server costs, and requests that were never answered. Every other number is in `budgets`. +constexpr std::array COMPARED { "completion.p95", "completion.max", "importer.completion.max", "busiestMinuteChanges", + "rssGrowth", "idleCpuCores", "serverTimeouts", "clientTimeouts", "republishSeconds" }; + +} // namespace + +base::Result budgets(const std::string& directory) { + auto values = collect(directory); + if (!values) return std::unexpected { values.error() }; std::string out { "| check | measure | runs | p50 | p95 | max | p95 x 1.3 |\n|---|---|---|---|---|---|---|\n" }; - for (const auto& [key, measured] : values) { + for (const auto& [key, measured] : *values) { const double p95 { percentile_of(measured, 0.95) }; out += std::format("| {} | {} | {} | {:.3g} | {:.3g} | {:.3g} | {:.3g} |\n", key.first, key.second, measured.size(), median_of(measured), p95, std::ranges::max(measured), p95 * 1.3); @@ -194,6 +222,30 @@ base::Result budgets(const std::string& directory) { return out; } +base::Result compare(const std::string& candidate, const std::string& baseline) { + auto ours = collect(candidate); + if (!ours) return std::unexpected { ours.error() }; + auto theirs = collect(baseline); + if (!theirs) return std::unexpected { theirs.error() }; + Comparison result { "| check | measure | runs | median | max | baseline runs | baseline median | baseline max | |\n" + "|---|---|---|---|---|---|---|---|---|\n", {} }; + for (const auto& [key, measured] : *ours) { + if (std::ranges::find(COMPARED, key.second) == COMPARED.end()) continue; + const auto other = theirs->find(key); + if (other == theirs->end()) continue; + const double median { median_of(measured) }, otherMedian { median_of(other->second) }; + // Worse: a median above the baseline's by more than twice their pooled deviation, and by more than a tenth of it, so + // that two counts that never vary (0 and 0) are the same and a noise-free 0 against 1 is not called noise. + const double pooled { std::sqrt((std::pow(deviation_of(measured), 2) + std::pow(deviation_of(other->second), 2)) / 2) }; + const bool worse { median > otherMedian + std::max(2 * pooled, 0.1 * std::abs(otherMedian)) }; + if (worse) result.worse.push_back(std::format("{} {}: median {:.3g} against {:.3g}", key.first, key.second, median, otherMedian)); + result.table += std::format("| {} | {} | {} | {:.3g} | {:.3g} | {} | {:.3g} | {:.3g} | {} |\n", key.first, key.second, measured.size(), median, + std::ranges::max(measured), other->second.size(), otherMedian, std::ranges::max(other->second), + worse ? "worse" : ""); + } + return result; +} + } // namespace mcppls::devtools::measure namespace mcppls::devtools { @@ -257,11 +309,28 @@ int command_measure_budgets(const cmdline::ParsedArgs& arguments) { return 0; } +int command_measure_compare(const cmdline::ParsedArgs& arguments) { + const std::string candidate { arguments.positional_or(0, "") }, baseline { arguments.positional_or(1, "") }; + if (candidate.empty() || baseline.empty()) { + std::println(std::cerr, "mcppls-devtools: measure compare needs two directories"); + return 2; + } + auto compared = measure::compare(candidate, baseline); + if (!compared) { + std::println(std::cerr, "mcppls-devtools: {}", compared.error().message); + return 1; + } + std::print("{}", compared->table); + for (const auto& worse : compared->worse) std::println(std::cerr, "mcppls-devtools: worse than the baseline: {}", worse); + return arguments.is_flag_set("fail-worse") && !compared->worse.empty() ? 1 : 0; +} + int dispatch(bool& handled, int& status, std::string_view verb, const cmdline::ParsedArgs& inner) { handled = true; if (verb == "summary") return command_measure_summary(inner); if (verb == "budgets") return command_measure_budgets(inner); - std::println(std::cerr, "mcppls-devtools: measure needs a verb: summary or budgets"); + if (verb == "compare") return command_measure_compare(inner); + std::println(std::cerr, "mcppls-devtools: measure needs a verb: summary, budgets or compare"); return 2; } @@ -278,6 +347,11 @@ cmdline::App measure_command(bool& handled, int& status) { (void) command.subcommand("budgets") .description("Each check's measured numbers over every --measure file under a directory: runs, p50, p95, max, p95 x 1.3") .arg("directory").required().help("Directory of --measure files, at any depth (nightly's ux artifacts)"); + (void) command.subcommand("compare") + .description("The typing, save, state-change, idle and timeout numbers of two directories of --measure files, side by side") + .option("fail-worse").help("Fail when a median is worse than the baseline's beyond their spread") + .arg("candidate").required().help("Directory of the candidate's --measure files") + .arg("baseline").required().help("Directory of the baseline's, measured on the same machines (nightly's ux-baseline)"); (void) command.action([&handled, &status](const cmdline::ParsedArgs& arguments) { const auto sub = arguments.subcommand(); static const cmdline::ParsedArgs EMPTY {}; diff --git a/tools/devtools/src/measure.cppm b/tools/devtools/src/measure.cppm index a674b46c..eac97091 100644 --- a/tools/devtools/src/measure.cppm +++ b/tools/devtools/src/measure.cppm @@ -24,6 +24,15 @@ base::Result summarize(const std::string& directory); // from the measured distribution would be. A Markdown table, sorted by check and number. base::Result budgets(const std::string& directory); +// The same files of a candidate and of a baseline measured beside it (nightly's ux-baseline): for the numbers a release +// is compared on, each side's runs, median and largest value, as a Markdown table, and one sentence per number whose +// median is worse than the baseline's beyond their spread. +struct Comparison { + std::string table; + std::vector worse; +}; +base::Result compare(const std::string& candidate, const std::string& baseline); + // What a pre-release gate needs from the same files: the median `first-navigation` of each kind // against a ceiling in seconds, and every run's `failures` at zero. One sentence per breach; an // empty answer passes. A budget for a kind with no files is a breach too: a gate that measured diff --git a/tools/devtools/tests/test_measure.cpp b/tools/devtools/tests/test_measure.cpp index 8f254b11..716860c3 100644 --- a/tools/devtools/tests/test_measure.cpp +++ b/tools/devtools/tests/test_measure.cpp @@ -48,6 +48,31 @@ int main() { expect(!measure::budgets("/definitely/not/a/real/directory-mcppls-test").has_value()); }; + "compare: the typing and idle numbers beside a baseline's, and only a difference beyond their spread is worse"_test = [&] { + const std::string candidate { scratch("compare-candidate") }, baseline { scratch("compare-baseline") }; + const auto write_run = [&](const std::string& directory, int index, double max, double growth, int timeouts) { + const std::string run { std::format(R"({{"fixture":"ux","checks":[)" + R"({{"id":"U5","measure":{{"completion":{{"max":{},"n":10}},"serverTimeouts":{}}}}},)" + R"({{"id":"U13","measure":{{"rssGrowth":{},"seconds":600}}}}]}})", max, timeouts, growth) }; + (void) fs::write_file(base::join_path(directory, std::format("ux-edits-{}.json", index)), run); + }; + for (int i { 0 }; i != 4; ++i) { + write_run(candidate, i, 0.80 + 0.01 * i, 0.30 + 0.01 * i, 0); + write_run(baseline, i, 0.78 + 0.01 * i, -0.10 + 0.01 * i, 0); + } + auto compared = measure::compare(candidate, baseline); + expect(fatal(compared.has_value())); + expect(compared->table.contains("| U5 | completion.max | 4 |")) << compared->table; + expect(compared->table.contains("| U13 | rssGrowth | 4 |")) << compared->table; + expect(!compared->table.contains("completion.n") && !compared->table.contains("| seconds |")) << "only the compared numbers"; + expect(compared->worse.size() == 1u) << "0.02 s on a 0.8 s max is noise; idle growth +30% against -10% is not"; + expect(!compared->worse.empty() && compared->worse.front().starts_with("U13 rssGrowth")) << (compared->worse.empty() ? "" : compared->worse.front()); + expect(compared->table.contains("| U5 | serverTimeouts | 4 | 0 |")) << "two counts that never vary are the same"; + expect(!measure::compare(candidate, "/definitely/not/a/real/directory-mcppls-test").has_value()); + std::filesystem::remove_all(candidate); + std::filesystem::remove_all(baseline); + }; + "an empty directory renders the header alone"_test = [&] { const std::string directory { scratch("empty") }; auto text = measure::summarize(directory);