diff --git a/.agents/docs/2026-10-26-control-contract-runtime-and-ide-ecosystem-design.md b/.agents/docs/2026-10-26-control-contract-runtime-and-ide-ecosystem-design.md new file mode 100644 index 000000000..c67618a2b --- /dev/null +++ b/.agents/docs/2026-10-26-control-contract-runtime-and-ide-ecosystem-design.md @@ -0,0 +1,886 @@ +--- +subject: ecosystem +status: active +--- + +# 控制契约、并发与性能、单一 C++ 运行时，以及 IDE 生态（生态级，#838、#839） + +日期：2026-10-26。分析基线：mcpp `c4f8b011`（2026.10.24.1 之后的 main）；xlings `f437e780`； +mcpp-vscode `432325a`（0.9.0）；mcpp-language-server `0.0.17`；mcpp-plugins `0.19.1`； +mcpp-index（mcpplibs）本机快照。 + +**修订记录** + +- rev 1：由一次讨论收敛而来（问答与结论见附录 A），评审前的完整稿。写作中实测了 O7（E19、E20）：SYCL 单元跨运行时按类型捕获 `sycl::exception`，孤岛不成立，§6.3 增加条件 6，sycl 改为只走 U。 +- rev 2（第一轮评审）： + - O1 采纳 `--deny`。 + - O2：孤岛声明**不放在 `[package]`**。改为复用已有的 `requires` 与依赖的 `visibility` 词汇：`requires` 增加对象形式 + `{ layer = "…", visibility = "private" }`，并增加按目标的写法 `[target.] requires`（对称于 2026.9.12.3 的 `requires_abi`）。理由见 §6.3。 + - O3：**新建仓库 mcpp-ide，不改 mcpp-vscode 的仓库名**；README 参照 mcppls；插件名沿用 `mcpp-xxx` 模式（VS Code 仍发布为 `mcpp-vscode`，JetBrains 为 `mcpp-clion`）。 + - O4 采纳（webui 只渲染；后端经 xlings 索引分发）。 + - O6 按索引习惯：`gcc.libstdcxx`（对称于 `llvm.libcxx`），版本为"gcc 版本.包修订号"。 + - mcppls：本批不提 PR，改为 issue [Sunrisepeak/mcpp-language-server#71](https://github.com/Sunrisepeak/mcpp-language-server/issues/71)。 + - 新增 §23 多视角自审，自审结论已并入正文（链接产物原子替换、修改段内复核计划、外来运行时按 DT_NEEDED 识别、退出码 5 与 130/143、启动描述只含 mcpp 注入的变量、`--home` 组合规则、各格式下孤岛条件的差异等）。 +- rev 3（实测收口与实施调整）： + - **IDE 一线本批不实施**：mcpp-vscode 不动，计划记录于 [mcpp-vscode#23](https://github.com/mcpp-community/mcpp-vscode/issues/23)（含 O9）；mcppls 记录于 #71。mcpp 的控制契约照常实施（agent、CI 同样需要）。 + - 开放问题全部关闭（§22）：O5、O8、O10 由临时 PR [mcpp#840](https://github.com/mcpp-community/mcpp/pull/840) 实测（run 40221149103、40222782650）；O7 由分析与本机测量关闭（E21）。 + - D22 按平台定形：POSIX 先写临时名再改名；PE 先把旧产物改名挪开再原地链接（E23）。 + - 持久计划（D6）落到已有机制：配置记录 + `build.ninja`，有效性由 `admit_recorded_build`（#757 起的唯一闸门）判断；本批把记录改为每配置一个文件，并让查询读取它（§3.2）。依据：X1 之后 `toolchain list` 64 ms，`emit build-database` 的剩余开销是规划本身（E22）。 + - 锁的范围从"指纹确定"开始：规划的 0–11 阶段（读清单、解析、联网、工具链、扫描）不持锁，收尾阶段与 ninja 持锁（§4.1）。 + - X2 不需要单独的迁移步骤：认领规则在每次投影变更时生效，home 在第一次变更时自行完成迁移（§10.2）。 + - §20 增加任务依赖图。 +- rev 4（实施收口，2026-10-26）：以下各项按实现与实测修订正文，正文中与之冲突的旧表述以本条为准。 + - **持久计划**：不新增计划格式。计划仍是配置记录 + `build.ninja` + 配置级编译数据库，有效性只由 `admit_recorded_build` 判断； + 快速路径保留，作为读取记录的一方（它们在接纳记录之前取构建目录锁）。`target/.build_cache` **不拆分**：读改写在 + `target/.build_cache.lock` 下进行并原子替换，格式不变，因此 §17 的"旧格式迁移"一行取消。 + - **查询不读记录**：编译器探测缓存（`/cache/probe`，键为命令与引号内文件的 stat）与提供者握手缓存 + （`/cache/provider-handshake`）之后，`emit build-database` 71 ms、`toolchain list` 44 ms，已达 §21.3 目标（E28）。 + - **编译数据库**：根目录副本改为按需后，完整构建中的 `compile-commands` 阶段为 14 ms（mcpp 自身），不再单独跳过。 + - **锁**：`mcpp test` 持锁直到测试结束（测试在同一构建目录运行刚链接的程序）；`run` 在 exec 前释放。 + - **命令形状**：`mcpp run --print-launch`（`run` 的 `--format` 表示可分发物）；`mcpp pack --list-formats` 只输出 JSON； + `index list --format json` 给出注册表与工程的 `[indices]`，包目录由 `search --format json` 回答（不带关键字时列出全部）。 + - **pack 不加事件流**：`mcpp pack --message-format json` 已有单文档输出（docs/50），加事件行会破坏按单个 JSON 解析的使用者； + 需要构建事件的调用方先 `mcpp build --message-format json`。事件流不含 `result` 事件，结果即信封。build 与 pack 的 + 信封把 mcpp 自身的错误作为诊断携带。 + - **取消**：`cancelled` 记录在修改段开始时预先生成，信号处理器在结束受管进程组之后用 `open`/`write`/`rename` 落盘， + 然后按默认动作结束（POSIX 状态 `128 + N`）。ninja 进程组仍以 SIGKILL 结束（既有理由：ninja 只在等待子进程时检查信号）。 + Windows 的状态是系统的 `STATUS_CONTROL_C_EXIT`，不改写为 130（D23 的 Windows 部分据此修订）。 + - **compiler-runtime 规则不变**：图提供的 libstdc++ 在 clang 下链接为 `-stdlib=libstdc++ -fuse-ld=lld + --rtlib=compiler-rt --unwindlib=libgcc`，提供包不声明 `compiler-runtime`；§6.4 第二条取消。 + - **构建程序环境**：`MCPP_CXX_STDLIB` 取依赖图的 `c++-abi` 供给（构建程序在目标侧解析之前运行，因此按声明读取）， + 新增 `MCPP_CXX_ABI_ROOT`（供给包根目录），规则包据此让设备编译器使用同一个库。 + - **条件 4** 检查孤岛包的 role=Object 动作产出（规则包产出的目标文件），不检查普通编译单元。 + - **拒绝信息不点名实现包**：SPEC-002 §1.1 禁止引擎持有实现名，因此 `requires` 不满足时的补救写为 + `help: depend on a package that declares provides = ["mcpp:c++-abi=libstdc++"]`；`gcc.libstdcxx` 由 docs/22、docs/42 与 + index 的 sycl-runtime 描述给出。 + - **`mcpp.lock` 乐观写入**：不一致时不在进程内重新规划，以 `[lockfile-concurrent]` 失败并给出 `help: run the command again` + （重跑即重新规划）。写入改为原子替换，符号链接写到其目标，只读文件照旧拒绝。 + - **清单诊断的位置**：清单拒绝按消息点名的 `[表].键` 定位到该值（或表头）所在行；信封诊断从消息开头的位置得到 + `path` 与 `range`（docs/50 §2）。 + - **补救行统一为 `help:`**（原 `hint:`），写入 SPEC-012。 + - **D13 的 index 测量（E27）**：本机 registry 中 `DT_NEEDED` 含 libstdc++ 的共享库属于 dpcpp、glslang、mesa（驱动、 + xatracker）、slang、libllvm、qt；index 描述文件按名链接的只有 `compat.sycl-runtime` 的 `libsycl`（本批声明 `requires`）； + mesa 的驱动在运行期 dlopen，不在链接行上；其余不被 index 包按名链接。因此 D13 对 index 的新增拒绝只有 llvm 下的 + SYCL，新增警告为 0。 + - **X4（xlings，新增）**：项目 store（mcpp 为工程索引维护的 `.mcpp/.xlings`）的 subos 只有已安装包的 shim，没有 `xlings` + 入口；安装 `gcc.libstdcxx` 时 `xim:gcc` 的 config hook 以 `system.bindir()/xlings script` 重写 specs，报 + `sh: …/subos/_/bin/xlings: not found`。xlings 在为 hook 配置 bin 目录时补上指向 home 入口的链接（与其旁 shim 指向同一 + 二进制）；`tests/e2e/project_hook_runs_xlings_test.sh` 在 2026.10.24.1 上失败。 + - **`gcc.libstdcxx` 只有一个版本**：描述文件的依赖边按平台而不按版本，一个描述文件只能跟随一个 GCC 发布，取 mcpp 默认 + gcc 工具链所用的 16.1.0（§12.1 的"按 `xim:gcc` 已有版本各一个"据此修订）。sycl-consumer 与 index 的 + `sycl-runtime` 成员按条件声明依赖：`--no-accel` 的 CPU 变体仍用 libc++；index 成员在 Linux 上陈述 gcc，llvm 组合由新成员 + `libstdcxx-llvm` 覆盖。 + - **mcpp-plugins 的 sycl 规则读环境变量** `MCPP_CXX_ABI_ROOT` 而非 `mcpp::cxx_abi_root()`：后者在旧引擎上不存在，用 gcc 的 + 旧引擎用户因此不受影响；整个插件包的引擎下限不因一个规则提高。 + - **计划等价**（§21.1 第 10 条）：不新增计划格式后，等价性即既有快速路径的正确性，由 #544、#716、#755 与 riscv 的 e2e + 覆盖，不另设语料测试。 + +关联：**mcpp#838**（sycl lane 下 `std::exception` 匹配不到）、**mcpp#839**（同配置并发构建在同一构建目录竞争、双方失败）、 +mcpp#596（unwinder 被拆成两半）、mcpp#514（两份 libc++ 头文件）、mcpp#544 / #716 / #755（快速路径的拒绝清单）； +[SPEC-002](../../docs/specs/target-side.md)、[SPEC-003](../../docs/specs/exit-codes.md)、 +[SPEC-010](../../docs/specs/home-initialization.md)、[docs/50](../../docs/50-machine-output.md)、 +[docs/22](../../docs/22-target-side.md)。 + +复现：[2026-10-26-control-contract-runtime-and-ide-ecosystem-repro.sh](2026-10-26-control-contract-runtime-and-ide-ecosystem-repro.sh) +（场景 A–I，只用 mcpp home 中已有的载荷；`FRESH=1` 时另建一个全新 home 复现 X2；场景 I 需要 `xim:dpcpp` 7.1.0）。 + +读者：评审与实现者。涉及六个仓库：mcpp、openxlings/xlings、mcpp-community/mcpp-plugins、 +mcpplibs/mcpp-index、Sunrisepeak/mcpp-language-server，以及新建的 mcpp-community/mcpp-ide +（由 mcpp-vscode 演进）。**每个仓库一个 PR，PR 内用 commit 作检查点**（§20）。 + +**本设计的边界**： + +- **mcpp 是基础设施**。只放三类东西：构建语义、对 mcpp 自身行为的标准化契约、通用基础设施原语 + （锁、取消、原子发布、记录）。和构建系统没有直接关系的东西不进 mcpp。判据：**不打开 IDE 的场景 + （脚本、CI、agent）也需要它，才进 mcpp**。优先扩展已有机制，不新增概念。 +- **IDE 的通用逻辑在独立的 C++ 仓库 `mcpp-ide`**，用 `mcpp.toml` 构建（形态参照 mcppls），各 IDE 插件只做接入与 UI。 +- **mcpp 本身必须很好操作**：终端里的人、只有 shell 的 agent，不经过 IDE 或 MCP，也能完成全部事情。 +- 清单优先（`mcpp.toml` 为主，`config.toml` 只给默认值），核心知识判据沿用 2026-10-23 设计的 D18。 +- 一次做完，不分期；兼容靠显式的 COMPAT 读法和移除下限，不靠分阶段。 + +## 0. 结论 + +| # | 决定 | +|---|---| +| D1 | **调用链模型**：人、agent、IDE、CI 的每条调用链都终止于 mcpp，mcpp 是唯一公共节点。必须在所有链上成立的东西放进 mcpp；只有一条链需要的放在那条链自己的层（§2） | +| D2 | **跨链观察靠 mcpp 落盘的规范化记录**，不靠常驻进程转发。agent 在 shell 里构建，IDE 读记录就能看到（§5.2） | +| D3 | **mcpp-ide 只做 IDE 后端，不提供 MCP**；agent 一律直接调用 mcpp（§14） | +| D4 | **不设单独的 agent 模式**。现有机器协议升格为统一的**控制契约**，IDE 后端与 agent 共用；范围包括非 TTY 下的人类输出（§5） | +| D5 | **构建流水线分三段：规划（只读，可联网，不加锁）→ 修改（加构建目录锁）→ 执行（释放锁后运行程序或测试）**（§3） | +| D6 | **规划的产出持久化为"计划"**：每个配置一份记录（取代共享的 `target/.build_cache`）加 `build.ninja` 与配置级编译数据库；有效性只由 `admit_recorded_build` 判断（#757 起已是唯一闸门）；build、run 与查询（`emit build-database`）在记录有效时不再规划（§3，rev 3） | +| D7 | **锁的粒度最小**：每个构建目录一把，只覆盖修改阶段；读操作永不等锁；`mcpp.lock` 用乐观并发；锁描述符 CLOEXEC，`run` 在 exec 前释放（§4） | +| D8 | **副作用约束是 `--offline` 的推广**，不是新概念：`--deny `，`--offline` ≡ `--deny network`（§5.5） | +| D9 | **一个进程一套 C++ 运行时（U，默认）**；孤岛（I）只在组件显式声明且引擎能机械验证时成立；宿主决定（H）由 target base 陈述（§6） | +| D10 | **工程层面不新增"标准库"配置项**。`c++-abi` 是目标侧的独立一层，按已有模型由依赖选择；llvm 加 libstdc++ 通过依赖一个 libstdc++ 提供包实现（与 openkal-llvm-runtime 对称）（§6.2） | +| D11 | **`requires` 不满足时拒绝，并给出一行修复**；解析不自动补提供方（§6.3） | +| D12 | **`abi:` 能力并入 `requires = ["mcpp:<层>=<实现>"]`**，旧写法作 COMPAT 读法；发布前清点 index 中把 `c++-abi` 当编译器代名词的用法（§6.7） | +| D13 | **删除 ldflags 中搜 "stdc++" 子串的推断**；未声明的外来运行时按孤岛规则校验：可靠的放行并提示补声明，不可靠的拒绝（§6.6） | +| D14 | **工程根目录的 `compile_commands.json` 默认不写**；`mcpp.toml` 配置输出路径，`config.toml` 可给全局默认；已有副本停止更新，提示一次（§7） | +| D15 | **颜色**：识别 `CLICOLOR_FORCE` / `FORCE_COLOR`；`NO_COLOR` / `MCPP_NO_COLOR` 优先（§5.9） | +| D16 | **性能**：xlings interface 的固定 0.5 s 等待消除；缓存条目在发布时记录大小；编译器探测不经 shell 并缓存；查询命令走持久计划（§8） | +| D17 | **xlings 升级缺陷（X2）**：认领能证明是自己创建的未登记条目，`self init` 完成迁移，报错可操作；mcpp 透传 xlings 的诊断（§9.2、§10.2） | +| D18 | **home 选择规范升为生态级**：xlings 补上 `self init --home` 与持久选择；mcpp 与 xlings 都增加单次调用的全局 `--home`（§9.1、§10.3） | +| D19 | **mcpp-ide 是新仓库**（不改 mcpp-vscode 的仓库名）：C++ 后端（mcpp.toml），stdio 上的 JSON-RPC；共享 spec 生成两端声明；共享 webui（只渲染）；VS Code 适配层仍发布为 `mcpp-vscode`，JetBrains 为 `mcpp-clion`；依赖 mcppls 提供语言功能；README 参照 mcppls（§14） | +| D20 | **孤岛声明复用 `requires` + `visibility`**：`requires = [{ layer = "mcpp:c++-abi=libstdc++", visibility = "private" }]`，可写在 `[target.]` 上；不新增键名（§6.3） | +| D21 | **libstdc++ 提供包名 `gcc.libstdcxx`**，版本"gcc 版本.修订号"，内容借用 `xim:gcc` 载荷（§12.1） | +| D22 | **链接产物原子发布**：POSIX 先写 `.tmp` 再改名；PE 先把旧产物改名挪开（`.old-`）再原地链接，正在运行的程序保留其文件。执行段因此无需持锁（§4.1、§4.3、E23） | +| D23 | **退出码**：副作用被拒绝为 `5`；被信号取消为 `128 + 信号`（130、143）；写入 SPEC-003（§5.4、§5.5） | + +## 1. 问题与证据 + +### 1.1 实测（Linux x86_64，mcpp 2026.10.24.1，除注明外均在本机） + +| # | 现象 | 数据 | +|---|---|---| +| E1 | **#838 最小复现，无需 sycl**：llvm@22.1.8，`[build] ldflags = ["-L/lib64", "-l:libstdc++.so.6"]`（sycl 规则追加的就是这个），测试调用一次 `std::make_exception_ptr`，再 `catch (const std::exception&)` 库里抛出的 `runtime_error` | 输出 `std::exception missed -> (...)`，退出 1。`__cxa_throw`、`__cxa_begin_catch`、`typeinfo for std::exception` 来自 `libc++abi.a`（本地、被 `--exclude-libs` 隐藏）；`__gxx_personality_v0`、`typeinfo for std::runtime_error` 来自 `libstdc++.so.6` | +| E2 | 同上，把 `libc++.a libc++abi.a` 手动移到 `-l:libstdc++.so.6` 之前重链 | 正确捕获，退出 0；对 libstdc++ 的 `GLIBCXX` 导入为 0 | +| E3 | `mcpp build` 产出的 `libxrt.so`（ToolchainCoupled） | NEEDED 顺序 `libstdc++.so.6` 在 `libc++.so.1` 之前；`__cxa_throw`、`__gxx_personality_v0`、typeinfo 绑到 `@CXXABI_1.3` / `@GLIBCXX_3.4`；同时 NEEDED `libunwind.so.1`，进程内两套 unwinder | +| E4 | **#839**：24 个模块的工程，两个冷构建间隔 0.1–0.5 s 启动，5 轮 | **3 轮双方都失败**（`failed to write/read compiled module: No such file or directory`）；2 轮成功但都完整编译了一遍 | +| E5 | dev 与 `--release` 并发 | 都成功（指纹不同，构建目录天然分开） | +| E6 | `mcpp run` 的程序运行中修改源码再构建 | 成功，1.3 s；旧进程输出旧结果，新二进制输出新结果 | +| E7 | 构建过程中执行 `emit build-database` | 成功，未被阻塞 | +| E8 | 命令耗时（hyperfine，热缓存） | `--version` 0.45 ms；`--protocol-version` 1.8 ms；`self env` 2.2 ms；`build`（无改动）49.6 ms；`run`（无改动）52.4 ms；**`emit build-database` 604 ms**；**`toolchain list` 554 ms**；**`cache list` 376 ms** | +| E9 | `emit build-database` / `toolchain list` 的时间分解（strace） | xlings `interface payload_status` 约 590 ms；`interface --version` 6 ms。`payload_status` CPU 只有 40 ms，墙钟 510 ms：心跳线程一次 `nanosleep(0.500 s)`，主线程 futex 等待 0.299 s。查 1 个还是 8 个目标都是 517 ms | +| E10 | `cache list`：5.0 GB、61,265 个文件、580 个条目 | 61,742 次 `stat`、约 39,000 对 mmap/munmap、9,315 次 openat，全是 CPU 开销 | +| E11 | 完整构建的阶段计时（`MCPP_LOG_LEVEL=info`） | 规划 64–84 ms；**`compile-commands` 119–126 ms**；emit-ninja 19–22 ms | +| E12 | X2：全新 home 中先装 gcc 16.1.0，再在 default subos 里切到 15.1.0 再切回 | 正常 | +| E13 | 同一全新 home，只删掉 `subos/default/.xlings.json` 中 `libatomic.so` 的两条登记，链接保留，再切换 | `user-owned or unobserved entry preserved`，与本机旧 home 的错误逐字相同。按文件登记从 xlings 2026.10.9.1（#641）起引入 | +| E14 | clang 22 + libstdc++ 15.1.0 / 16.1.0 | `bits/std.cc` 预编译成功（需要 `-isystem /backward`）；`import std` 编译、链接、运行（`println`、`format`、`ranges`、异常、`jthread`）；产物只 NEEDED `libstdc++.so.6`、`libgcc_s.so.1` | +| E15 | clang 22 + libstdc++ 16.1.0，`.so` 抛 `runtime_error`，可执行文件 `catch (const std::exception&)` | 跨 `.so` 正确捕获 | +| E16 | `mcpp run` 的退出码 | 程序返回 0/3/255 → 同值；256 → 0；异常 134；segv 139（POSIX 上 mcpp `execve` 替换自身） | +| E17 | 用 clangd 22 检查 gcc lane 的 `import std` | `module 'std' not found`；llvm lane 无错误；C++20 + `#include` 对照组无错误 | +| E18 | 管道 vs 伪终端下的 `Running` | 管道：无颜色码（CLion 按 stderr 涂红）；PTY：`ESC[1mESC[92m`（绿） | +| E19 | 冲突集 ∩ SYCL 单元的未定义符号（dpcpp 7.1.0，`-fsycl -std=c++17 -O2 -fPIC --gcc-install-dir=`） | `saxpy.o` 21 个、Tensorvia `runtime` 10 个、`arithmetic` 13 个、`matmul` 13 个，全部是 ABI 层与 `std::exception` / `std::bad_alloc`；布局相关成员 0 个 | +| E20 | 同上单元引用的、冲突集之外的 typeinfo | `saxpy.o`、`arithmetic`、`matmul`：`typeinfo for sycl::_V1::exception`（定义在 `libsycl.so`） | +| E21 | HIP（O7） | 本生态的 HIP 只有 NVIDIA 平台：`mcpp.rules.hip` 用工程自己的 clang 编译 `.hip`（`-x cuda`），AMD 平台被规则拒绝（无 ROCm 载荷）；`libcudart_static.a` 未定义的 C++ ABI 符号只有 `__cxa_atexit`（glibc 提供）。没有第二套 C++ 运行时，HIP 无需声明 | +| E22 | X1 后的查询耗时（本机，xlings 发布构建替换进 home 的符号链接副本） | `xlings interface payload_status` 525 → 23–31 ms；`mcpp toolchain list --format json` 569 → 64 ms；`mcpp emit build-database`（mcpp 仓库自身）912 → 417 ms，剩余为规划时间 | +| E23 | 并发重链与启动（4 个启动线程，25 次重链；mcpp#840） | Linux bfd 原地写：4 896 次 EACCES；gold：589 次 ETXTBSY；macOS ld64 原地写：允许覆盖运行中的程序，出现 ENOEXEC；先写临时名再改名：Linux、macOS 均 0 次错误。Windows：运行中的 exe 不能被替换（错误 5）但可以改名；改名挪开后原地链接，启动只会失败于"文件不存在 / 共享冲突"，不会执行半个文件 | +| E24 | 锁语义（mcpp#840） | Linux、macOS：`O_CLOEXEC` 的 flock 在 exec 时释放，非 CLOEXEC 被程序继承；SIGKILL 释放。Windows：`LockFileEx` 立即失败返回 33；阻塞获取在持有者被终止时立即返回；句柄不被继承 | +| E25 | PE 上的孤岛（O10；MinGW g++ 16.2 libstdc++ DLL + llvm-mingw 20261020 libc++；mcpp#840） | C 接口：正确；C++ 接口按类型捕获：只有 `catch(...)` 接住（条件 6 成立）；我们一侧两个 libc++ 映像 + libstdc++ DLL：跨我们映像按类型捕获正确（条件 1 放宽）；libstdc++ 导入库在 libc++ 之前：exe 从 `libstdc++-6.dll` 导入 `__cxa_throw`、personality、`logic_error`，完全不导入 `libc++.dll`（#838 的 PE 形态，条件 3、5 成立）；在其后：正确 | +| E26 | llvm + libstdc++ 的其他组合（O8；本机与 mcpp#840） | x86_64：`import std.compat`、ASan / UBSan / TSan、跨 `.so` 按类型捕获全部正确；aarch64 Linux：index 没有 llvm 载荷（`E_UNSUPPORTED_TARGET`），组合不存在 | +| E27 | D13 的 index 测量（rev 4） | 本机 registry 中 `DT_NEEDED` 含 libstdc++ 的共享库：dpcpp 13 个、qt 175 个、qt-base 65 个、slang 6 个、glslang 4 个、mesa 3 个（`libgallium`、`libvulkan_radeon`、`libxatracker`）、libllvm 1 个；index 描述文件按名链接的只有 `compat.sycl-runtime`（`libsycl.so`） | +| E28 | 查询耗时（rev 4，本机，xlings 2026.10.26.1） | `mcpp emit build-database` 71 ms；`mcpp toolchain list --format json` 44 ms；`mcpp build`（无改动）50 ms；`cache list` 117 ms（旧条目，迁移后读记录）；完整构建的 `compile-commands` 阶段 14 ms | + +### 1.2 缺陷、设计缺口与根因 + +| 编号 | 问题 | 根因 | +|---|---|---| +| B1 | #839 并发构建冲突 | 工程构建目录是唯一没有跨进程保护的共享可变资源（全局侧：索引刷新非阻塞锁、BMI 缓存按条目锁、工具链按版本目录，已经细粒度） | +| B2 | #838 异常 ABI 被拆成两半 | 链接规则是 `$cxx … $ldflags $unit_ldflags`（`ninja_backend.cppm:2196–2202`），外来运行时在 `$ldflags` 里先于自身运行时归档；更深一层是两套 Itanium 运行时共用 ABI 符号名，多映像时没有可靠的链接写法 | +| B3 | 外来运行时靠字符串推断 | `flags.cppm:1452` 在 ldflags 里搜 `stdc++` 子串；规则包用 `link_lib(":libstdc++.so.6")` 写字符串，绕开了已有的有类型通道（`requires = ["mcpp:c++-abi=…"]`） | +| B4 | 快速路径的正确性靠清单 | `try_fast_build` / `try_fast_run` 各自重放、各自维护拒绝清单，#544、#716、#755 与 riscv 那次都是漏项；查询命令没有快速路径 | +| B5 | 查询慢 | xlings `heartbeat_loop_` 的 `sleep_for(500ms)` 不可中断（`interface.cpp:456`）；stdin 线程 100 ms poll；`cache list` 现算大小；探测经 `sh -c env …` | +| B6 | X2 升级缺陷 | 2026.10.9.1 之前投影的链接只在 home 顶层别名表里，没有迁入 subos 的按文件登记 | +| B7 | IDE 标红（gcc lane） | gcc 的 `.gcm` 不能被 clangd 读；根目录 `compile_commands.json` 让 CLion 以 Compilation Database 接管工程，mcppls 按设计让出 | +| B8 | 外部工具无法强制颜色 | `detect_color` 只看 TTY 与 `NO_COLOR`（`ui.cppm:435`） | +| B9 | 契约缺口 | build/pack 无事件流；无启动描述；无结果记录；取消语义、并发语义未规范；索引与搜索无 JSON；打包格式不可查询 | + +## 2. 调用链模型 + +参与方：人、agent、IDE（适配层 + mcpp-ide 后端）、CI、mcpp；旁及 mcppls 与被构建的程序。 + +``` +A 人 ─────────────────────────────→ mcpp 终端 +B 人 → IDE（适配层 → mcpp-ide）───→ mcpp 按钮、运行配置 +C agent ─(shell)──────────────────→ mcpp 今天 agent 的主路径 +D 人 ─委托→ agent ────────────────→ mcpp 权限属于人 +E 人 → IDE 内的 agent ─(终端)─────→ mcpp +F CI ─────────────────────────────→ mcpp 非交互、可复现 +G mcpp → build.mcpp、规则包、工具链、程序本身 +``` + +**多重关系**：同一工程同时被多条链触碰，共享 `target/`、`MCPP_HOME`、`mcpp.lock`、`mcpp.toml`； +观察关系有"人看 agent 做了什么""IDE 看终端里构建出了什么""agent 读 IDE 的诊断"。 + +**放置原则**： + +1. 所有链都需要的 → mcpp。 +2. 只有一条链需要的 → 那条链自己的层（会话 → mcpp-ide；原生 UI → 适配层）。 +3. 直连链 A、C 必须完整；IDE 与 mcpp-ide 永远不是某个能力的前提。 +4. 跨链观察靠 mcpp 落盘的记录（D2）；仲裁（并发）只能由 mcpp 做，因为终端里的人不经过任何中间层。 + +**能力 × 层**（每项的细节在后续各节）： + +| 能力 | 需要它的链 | 层 | +|---|---|---| +| 构建、运行、测试、打包语义 | 全部 | mcpp | +| 输出信封、事件流、退出码、副作用表 | B C D F | mcpp | +| 启动描述 | B（调试）C F | mcpp | +| 结果记录落盘 | 跨链观察 | mcpp | +| 构建目录锁、取消、原子发布 | 多链并发 | mcpp | +| 副作用约束 | B（不受信任工作区）C D | mcpp | +| manifest 诊断带位置、索引与搜索 JSON、打包格式查询 | B C | mcpp | +| 非 TTY 的人类输出规范、颜色约定 | A C D | mcpp | +| 会话：监听、缓存、编排、视图模型、多工程 | B | mcpp-ide | +| 原生 UI：运行配置、测试树、调试器、面板 | B | 适配层 | +| agent 使用说明 | C D E | mcpp 文档；针对具体 agent 的封装 → mcpp-ide 仓库 `agents/` | +| 语义查询（符号、引用、验证） | B C | mcppls（已有） | + +## 3. mcpp：三段式流水线与持久计划 + +### 3.1 三段 + +| 段 | 做什么 | 写入 | 锁 | +|---|---|---|---| +| **规划** | 读 manifest、解析依赖与工具链（可联网）、目标侧、扫描，直到配置指纹确定（准备阶段 0–11） | 不写构建目录 | 无 | +| **修改** | 准备的收尾阶段（13：动作、BMI 缓存、`resolution.json`、`build.ninja`、配置级编译数据库）、运行 ninja、校验产物、写配置记录与结果记录 | 构建目录 | 构建目录锁 | +| **执行** | `run` exec 程序；`test` 运行测试；`pack` 打包（读构建产物，写 `-o` 目标） | 无（pack 写其输出） | 无 | + +实现（rev 3）：配置目录由指纹决定，而指纹在扫描阶段末尾才确定（`prepare/scan.cpp`）；所有写构建目录的步骤都在其后的收尾阶段 +（`phase13_finish`）与 ninja 中。因此锁在收尾阶段开始时获取、随 `BuildContext` 持有到 ninja 结束，规划不需要改写成"先算后写"。 + +### 3.2 持久计划 + +rev 3：计划就是已有的**配置记录**加 `build.ninja` 与配置级编译数据库。记录写于每次完整构建之后；有效性只由 +`admit_recorded_build`（`execute.cppm`）判断——#757 起它已是 `try_fast_build`、`try_fast_workspace_build`、`try_fast_run` +共用的唯一闸门：引擎身份、运行时绑定、`build.ninja` 的模式与请求、清单与源码树的时间、路径依赖、载荷目录、按路径命名的工具链、产物快照。 +各路径只保留属于自己的判断（工作区选择、要运行的程序、runner、运行层工具）。 + +本批的改动： + +| 改动 | 原因 | +|---|---| +| `target/.build_cache` 改为每配置一个文件 `target/.build_cache.d/-`，原子替换 | 共享文件的读改写在不同配置并发构建时互相覆盖（§4） | +| `emit build-database` 在记录有效时直接读取配置级数据库 | E22：X1 之后剩余的 417 ms 是规划本身；读记录不规划 | +| 读取旧的 `target/.build_cache`（COMPAT），首次写入时迁移 | 升级后第一次构建仍走快速路径之外的完整路径一次即可 | + +**并发规划**：两个进程可能同时规划同一配置；后到者在修改段拿到锁之后才写，写入内容由同一组输入决定，结果相同。 + +### 3.3 编译数据库从计划派生 + +配置级 `compile_commands.json` 是计划的函数：计划不变时不重写（E11：今天每次完整构建 119–126 ms）。 + +## 4. mcpp：并发（修复 #839） + +### 4.1 资源与保护 + +| 资源 | 方案 | +|---|---| +| 构建目录 `target///` | 每目录一把锁 `.mcpp-build.lock`，**只在修改段持有**（准备收尾阶段 + ninja；快速路径只在 ninja 期间）。后到者等待，拿到锁后 ninja 基本无事可做，等价于合并两次构建 | +| `run` / `test` 的执行段 | 不持锁。`run` 在 `execve` 前释放；锁描述符 CLOEXEC（POSIX 的 `FileLock` 已用 `O_CLOEXEC`，`fs.cppm:373`），保证程序不继承锁。执行段不持锁之所以安全，靠下一行的原子替换 | +| 链接产物（可执行文件、共享库、测试二进制） | **原子发布**（D22，E23）。POSIX：链接写 `.tmp`，再 `mv -f` 改名（同目录 rename 原子）。PE：链接前把旧产物改名为 `.old-`，再原地链接；运行中的程序保留旧文件，下次链接清理不再被占用的 `.old-*`。由 ninja 链接边的命令实现，与链接器无关 | +| 查询（`emit`、`why`、`cache list`、读记录） | 不加锁；所有记录写临时文件后原子改名 | +| 根目录单文件（根 `compile_commands.json`、`target/.last-result.json`） | 原子替换 | +| `mcpp.lock` | 乐观并发：规划时记哈希，写入前比对；不一致则重新规划一次，仍冲突则报错，点名"并发修改"。**不在网络或解析期间持锁** | +| 全局 `MCPP_HOME` | 沿用现有细粒度模式（索引刷新非阻塞、BMI 缓存按条目、工具链按版本目录）；实现时逐个核对其余写入点 | + +### 4.2 锁的实现 + +- `FileLock` 增加可等待的获取：非阻塞尝试加退避（50 ms 起），可被取消（§5.4）。Windows 用 `LockFileEx`（已有）。 +- 锁文件内容写持有者：pid、开始时间、命令行、调用方标签（`MCPP_CLIENT`，见 §5.8）。等待超过 300 ms 时： + 人看到 `Waiting for build lock held by pid 1234 (mcpp build, from mcpp-ide, 8s)`； + 机器读到 `{"kind":"mcpp.event","event":"waiting","holder":{…}}`。 +- 进程崩溃时操作系统释放锁，不存在僵死锁。 +- 不新增"等待或失败"参数：默认等待；调用方取消即可。IDE 的"保存时构建，新的取代旧的"由 mcpp-ide 取消自己上一次构建实现。 + +### 4.3 Windows + +实测（E23、E24）：正在运行的 exe 与已加载的 DLL 不能被替换（错误 5），但可以改名；改名后的文件在程序退出前不能删除。 +"先写临时名再改名"在 Windows 上失败于第二步（替换运行中的文件），因此 PE 采用"先改名挪开、再原地链接"：今天正在运行时重链会以 +LNK1104 失败，改后成功；与链接同时发生的启动只会失败于"文件不存在 / 共享冲突"，不会执行半个文件（Windows 不允许执行正被写入的映像）。 +`LockFileEx` 的语义与 POSIX flock 一致：立即失败返回 33，持有者终止时等待者立即获得，句柄不被子进程继承。 + +## 5. mcpp：控制契约 + +### 5.1 事件流 + +- `mcpp build` 与 `mcpp pack` 接受 `--message-format json`：stdout 上是 NDJSON，记录 `{"schemaVersion":1,"kind":"mcpp.event","event":…}`， + `event` ∈ `plan`、`waiting`、`progress`、`diagnostic`、`artifact`、`result`；最后一条是结果信封（`mcpp.build` / 已有的 `mcpp.pack`）。 +- `mcpp test` 的测试流早于信封存在，**保持不变**（docs/50 的稳定性承诺）。 +- `mcpp run` 的 stdout 属于程序，**不加事件流**。需要结构化构建的调用方先 `mcpp build --message-format json`，再按启动描述（§5.3）执行。 + +### 5.2 结果记录（D2） + +- 修改段结束时写 `target///result.json`（信封 kind `mcpp.result`）：命令、请求、调用方标签、开始与结束时间、 + 状态（`ok` / `failed` / `cancelled`）、诊断（带 path 与 range）、产物（含启动描述）、阶段耗时、配置级编译数据库路径。 +- 工程级指针 `target/.last-result.json`：最近一次结果所在的配置。 +- 读者：mcpp-ide（监听后更新问题面板）、mcppls（找编译数据库的回退路径，§13）、CI、`mcpp why`。 +- 与 `resolution.json` 的关系：后者记"做了什么决定"，结果记录记"结果如何"。两者并列，都在修改段原子写入。 + +### 5.3 启动描述 + +每个可运行产物：`argv`（runner 前缀 + 程序 + 参数占位）、`env`（mcpp run 注入的 loader 环境，即今天的 `childEnv`）、`cwd`、 +`runner`（名字与来源）。出现在结果记录的产物条目里；单次查询用 `mcpp run --print-launch --format json`（构建但不执行）。 +调试器、IDE、CI 按它执行，与 `mcpp run` 的环境逐项一致。`env` 只含 mcpp 注入或改写的变量（loader 路径、runner 需要的变量），不含调用方的其余环境，避免把机密写进结果记录。 + +### 5.4 取消 + +- SIGINT / SIGTERM（Windows：Ctrl-C / CTRL_BREAK）：规划段直接退出（没有写入）；修改段转发给 ninja 进程组、等待、 + 写结果记录（`cancelled`）、释放锁。退出码按 `128 + 信号`（SIGINT 130、SIGTERM 143；Windows 的 Ctrl-C 同样返回 130），写入 SPEC-003（D23）。 +- 构建目录在取消后保持一致（ninja 的日志语义），下一次构建从断点继续。 + +### 5.5 副作用约束（`--offline` 的推广，D8） + +- 全局 `--deny ` 与环境变量 `MCPP_DENY_EFFECTS`；词汇沿用 docs/50 §4：`init-mcpp-home`、`write-project`、 + `write-global-cache`、`network`、`exec-build-script`（`read-project` 不可拒绝）。 +- `--offline` 保留，等价于 `--deny network`。 +- 运行到需要被拒绝的副作用之前停下，诊断写明副作用与原因（例：`network: install xim:gcc@15.1.0`），**退出码 5**（SPEC-003 新增，D23）。只有 shell 的 agent 靠退出码就能区分"被我的约束挡住"与"构建失败"。 +- 计划有效时不会运行 `build.mcpp`，因此 `--deny exec-build-script` 下也能构建；约束针对的是本次运行实际要做的事。 + 调用方（agent 的宿主、IDE 的不受信任工作区）审批后去掉约束重跑。 +- 信封里的 `effects` 继续报告实际发生的副作用。 + +### 5.6 查询的补全 + +| 查询 | 形式 | +|---|---| +| manifest 诊断 | 所有 manifest 相关诊断在信封里带 `path` 与 `range`（TOML 语法、未知键、类型、指向某个键的语义拒绝）。不新增命令 | +| 搜索 | `mcpp search --format json`（kind `mcpp.search`） | +| 索引目录 | `mcpp index list --format json`（kind `mcpp.index`：namespace、name、description、licenses、repo、versions、镜像），**一次调用**取代 mcpp-vscode 对 Lua 文本的解析 | +| 打包格式 | `mcpp pack --list-formats --format json`（kind `mcpp.pack.formats`，来自已解析的依赖图） | +| 文件归属 | 已有：`emit build-database` | + +### 5.7 非 TTY 的人类输出规范 + +新规范（`docs/specs/human-output.md`）： + +1. 非 TTY 下不输出进度区域；叙述只有动词行（`Compiling `、`Finished`、`Running`），不逐单元输出。 +2. 每个错误在有可执行的下一步时，以一行 `help:` 结尾，给出命令或要改的键。 +3. 编译器诊断原样完整输出（它就是内容）；mcpp 自己的叙述有行数上限。 +4. 动词词汇固定，写入规范。 + +docs/50 仍声明人类输出没有兼容承诺；本规范约束的是形态，不是逐字内容。 + +### 5.8 调用方标签 + +`MCPP_CLIENT=`（IDE、agent 宿主自行设置）：写入锁持有者与结果记录，便于"谁在构建"的回答。只记录，不改变行为。 + +### 5.9 颜色（D15） + +优先级：`MCPP_NO_COLOR=1` / `NO_COLOR` 非空 → 关闭；`CLICOLOR_FORCE` 非 `0` 或 `FORCE_COLOR` 非空 → 开启；否则看 TTY。 +Windows 强制开启时同样打开 VT 处理。 + +### 5.10 agent 使用说明 + +mcpp 文档新增一章（docs/52）：调用链、非 TTY 输出、副作用与审批、等待锁、结果记录、启动描述、退出码。 +面向具体 agent 的封装（如 Claude Code 的 skill）放在 mcpp-ide 仓库 `agents/`。 + +## 6. mcpp：单一 C++ 运行时（修复 #838） + +### 6.1 原理 + +两套 Itanium C++ ABI 运行时共用同样的 ABI 符号名：`__cxa_*`、`__gxx_personality_v0`、`std::exception` 一族 +（不在 libc++ 的 `__1` 内联命名空间里）的类型信息、虚表与成员、`operator new/delete`。同一进程的全局符号空间里， +它们要么像 #838 那样各用一半，要么像 #596 那样互相串用。`std::runtime_error` 等类型在两边的**内存布局不同** +（libstdc++ 是 COW 字符串，libc++ 是 refstring），构造用一边、析构用另一边就是内存错误。 +因此**没有任何链接写法能让"多映像 + 两套运行时 + 异常跨映像"可靠**。 + +### 6.2 模型：沿用目标侧五层（D10） + +`c++-abi` 是独立的一层（docs/22）：至少两个可互换实现、可独立替换（E14 实测 clang 可换 libstdc++）、与 `c-abi` 有"为其配置"关系。 +工程只用三种方式表达目标侧：triple、`[toolchain]`（只选 `compiler`）、依赖（其余各层）。因此： + +- **不在 `[toolchain]` 或 `[build]` 新增标准库键**。前者会把 `compiler` 与 `c++-abi` 合成一层，后者把解析期事实当成编译参数。 +- **llvm + libstdc++** = 依赖一个 libstdc++ 提供包（§12.1），与 openkal 的写法对称： + +```toml +[dependencies] +gcc.libstdcxx = "16.1.0" # provides mcpp:c++-abi=libstdc++（D21） + +[toolchain] +default = "llvm@22.1.8" # 只选编译器 +``` + +### 6.3 三种模式 + +**U：统一（默认）**。一个进程一套运行时，由"每层一个提供方"（已有）决定：载荷、预编译、或依赖图。 +包用 `requires = ["mcpp:c++-abi=…"]` 声明需求；不满足时**拒绝并给出一行修复**（D11）： + +``` +error: compat.sycl-runtime requires c++-abi = libstdc++, but this build's c++-abi is libc++ (llvm@22.1.8 payload) + help: add a libstdc++ supplier: gcc.libstdcxx = "16.1.0" under [target.'cfg(accelerator = "sycl")'.dependencies] + help: or build with a gcc toolchain: --toolchain gcc@16.1.0 +``` + +**I：孤岛（显式声明，可验证）**。组件声明自己的 C++ 运行时只在 C 接口之后（D20）： + +```toml +[target.'cfg(os = "linux")'] +requires = [{ layer = "mcpp:c++-abi=libstdc++", visibility = "private" }] +``` + +为什么这样写，而不是 `[package]` 下的新键： + +- **`[package]` 是包的身份与元数据**；孤岛描述的是包的代码与目标侧 `c++-abi` 层的关系，属于目标侧事实。 +- **它随目标变化**：孤岛只在两套运行时可能共处的目标上有意义（ELF 上的 libc++ 与 libstdc++）；在 MSVC 或 macOS 上没有对应物。 + `[package]` 不能按目标条件化，2026.9.15 把 `std-module` 从 `[package]` 挪到 `[build]` 也是同一个原因。 +- **词汇已经存在**：`requires` 已经表达"某层必须解析为某实现"；依赖的 `visibility` 已经定义为"用途是否越过本包自身的边界"（docs/05）。 + `private` 的含义恰好是：这个需求只对本包自己的代码成立，不越过本包的边界，不约束映像的 `c++-abi`。字符串形式保持原义（等价于 `visibility = "public"`）。 + 行业对照：Conan 2 的 requirement trait `visible=False` 表达同样的"私有需求"。 +- **按目标的 `requires` 本身就缺**：index 的 Lua 描述文件可以按平台写 `requires`，TOML 却只有 `[package] requires`。 + `[target.] requires` 对称于已有的 `[target.] requires_abi`（2026.9.12.3），合并规则相同：`[package]` 的与所有匹配选择器的取并集。 +解析优先级：**能统一就统一**（图的 `c++-abi` 恰为所需实现时，孤岛声明不生效）；否则在下列条件**全部**可验证时成立： + +1. **我们这一侧只有一个 C++ 映像**：按根包**声明的全部目标**判定（不按本次请求的目标，保证各命令结论一致）：没有含 C++ 的 + `kind = "shared"` 目标，图中没有含 C++ 的共享依赖。自身运行时静态嵌入并隐藏（已有 `hide_static_cxx_runtime`）。 +2. **一套 unwinder**：`--unwindlib=libgcc`（已有）。 +3. **链接顺序**：自身运行时先于外来运行时（修 B2，E2 已验证）。 +4. **冲突集判据**：链进我们映像的孤岛目标文件（规则包 action 的产出），其未定义符号不得引用"布局相关冲突集" + （`std::exception` 一族中除 `std::exception` 自身外的构造、析构、`what`，`std::bad_alloc` 等）；ABI 层冲突集 + （`__cxa_*`、personality、`operator new/delete`）必须**整体**绑定到自身运行时。冲突集是一张随运行时版本维护的数据表。 +5. **链接后校验**：我们的映像不得以 `GLIBCXX_*` / `CXXABI_*` 版本导入 ABI 层冲突集中的任何符号（E1 的坏产物正有此特征，E2 的好产物为 0）。 +6. **不跨运行时按类型捕获**：孤岛目标文件不得引用由外来运行时映像（如 `libsycl.so`）定义的 typeinfo。原因：那个映像用外来运行时的 + `__cxa_throw` 抛出（异常类 `GNUCC++`），我们映像里的 personality 属于自身运行时（libc++abi 只认 `CLNGC++`），会把它当外来异常， + 只有 `catch (...)` 能接住，带类型的 `catch` 失效。 + +不满足 → 拒绝，给出 U 的两条修复。 + +**实测（E19、E20）**：冲突集按实际运行时计算（`libstdc++.so.6` 导出名 ∩ `libc++.a`/`libc++abi.a` 定义名，381 个，其中 ABI 层 56 个、 +`std` 非 `__1` 名 132 个）。sycl-consumer 的 `saxpy.o` 与 Tensorvia 的 `runtime`、`arithmetic`、`matmul` 三个 `.sycl` 单元（dpcpp 7.1.0， +按 `mcpp.rules.sycl` 的参数编译）引用的冲突集符号只有 ABI 层与布局不变的 `std::exception`、`std::bad_alloc`，**条件 4 成立**； +但 `saxpy.o`、`arithmetic`、`matmul` 都引用 `typeinfo for sycl::_V1::exception`（源码 `catch (const sycl::exception&)`），**条件 6 不成立**。 +结论：**SYCL 只能走 U**；孤岛机制保留给 C 接口、内部不让异常外泄的组件（例如只暴露 C API 的预编译 C++ 库）。 + +**各目标格式下的条件**（E25）：上述 6 条按 ELF 的全局符号作用域推导。PE 按 DLL 导入表绑定，没有全局作用域： +条件 1 放宽为"我们一侧的所有映像共用同一个运行时映像"（多个 libc++ 映像共用 `libc++.dll` 时跨映像按类型捕获正确）； +条件 3、5 照常成立（导入库的顺序决定 ABI 层从哪个 DLL 导入，链接后校验读导入表）；条件 4、6 与格式无关。 +Mach-O：Apple 平台只有 libc++，不存在孤岛。MSVC ABI：只有一个 C++ 运行时家族，不存在孤岛。 + +**H：宿主决定**。Python 扩展、JNI、宿主插件：`c++-abi` 由 target base 陈述（2026-10-23 的目标基座模型，来源 `prebuilt`），包只能服从。 + +### 6.4 引擎改动 + +- **图提供的 libstdc++ 在 clang 下的接线**（对称于已有的图提供 libc++）：`-nostdinc++`（已有，`hostflags.cppm:502` 附近） + 加提供包的头文件目录（含 `backward/`）；链接 `-stdlib=libstdc++`；std 模块取提供包的 `[build] std-module`； + `hostflags` 中硬编码的 `-stdlib=libc++` 改为按已解析的 `c++-abi` 选择。 +- **compiler-runtime 层的规则修订**：docs/22 现写"compiler runtime 必须属于编译器自身的家族"。GNU/Linux 发行版的 clang + 默认就是 `--rtlib=libgcc --unwindlib=libgcc`，E14 也按此链接成功。规则改为"必须是编译器支持的实现" + （clang：compiler-rt 或 libgcc；gcc：libgcc），libstdc++ 提供包同时 `provides mcpp:compiler-runtime=libgcc`。修订 SPEC-002。 +- **构建程序环境**暴露已解析的 `c++-abi` 实现与提供方根目录，规则包据此决定 `--gcc-install-dir` 等，使设备半边与宿主半边用同一份 libstdc++。 +- **链接顺序**：外来运行时（孤岛声明的）由引擎放在自身运行时之后，不再由规则包写进 `$ldflags`。 +- **冲突集检查与链接后校验**并入 `symbol_provision`（已有的同类检查所在）。 + +### 6.5 解析时序 + +运行时需求是**静态元数据**（包的 manifest），解析阶段即可知道；规则包运行在解析之后，只负责编译与链接机制。 +这回答了"构建扩展系统是否最优"的问题：会影响解析的信息必须静态声明（与 Cargo 的 `links`、Conan 的 settings、 +CPS 的 `cpp_runtime_vendor` 一致），构建程序不得用字符串表达它。 + +### 6.6 删除子串推断（D13） + +`flags.cppm:1452` 的 `stdc++` 子串推断删除。替代：识别**未声明的外来运行时**，来源有两处：链接线上直接点名的运行时库（`-l:libstdc++.so.6` 等），以及链接输入中的共享库其 `DT_NEEDED` 含外来运行时的 SONAME（数据表：`libstdc++.so.*`、`libc++.so.*`、`libc++abi.so.*`；只看直接链接输入，不递归）。这类库就是条件 6 所说的"外来运行时映像"，它们定义的 typeinfo 构成条件 6 的检查集合。识别结果视为**未声明的孤岛**，按 §6.3 的 I 条件校验： + +- 全部满足 → 构建，并 `warning` 提示规则包作者补声明（COMPAT 读法，移除下限见 §17）。 +- 不满足 → 拒绝，给出 U 的修复。今天能正常工作的工程照常构建；今天悄悄出错的工程得到明确的错误。 + +### 6.7 合并 `abi:` 能力（D12） + +- `abi:=`（`abi.cppm`）与 `requires = ["mcpp:="]` 表达同一事实。本机 index：`abi:cxxstdlib` 0 个包， + `abi:glibc` 2 个（compat.cpptrace、compat.glfw），`mcpp:c++-abi` 3 处。 +- `abi:` 作为 COMPAT 读法映射到 `requires`（`abi:glibc` → `mcpp:c-abi=glibc`，`abi:cxxstdlib=X` → `mcpp:c++-abi=X`， + `arch`、`os` 映射到目标 cfg 检查），`step13_abi_enforcement` 并入目标侧的 requires 检查。 +- **语义漂移**：`mpusz.mp-units` 在 Linux 上声明 `requires mcpp:c++-abi=libc++`，注释的真实原因是 GCC 16.1 的模块缺陷， + 即约束的是编译器。`c++-abi` 可独立选择后，应改为 `requires = ["mcpp:compiler=llvm"]`。发布前清点 index 中全部 `mcpp:c++-abi` 用法（§12）。 + +## 7. mcpp：工程根目录的 `compile_commands.json`（D14） + +- 配置级数据库（`target///compile_commands.json`）保持，内部产物，按 §3.3 只在计划变化时重写。 +- 根目录副本**默认不写**。`mcpp.toml`： + +```toml +[build] +compile_commands = "compile_commands.json" # 相对工程根目录的输出路径；不写则不输出 +``` + +- `~/.mcpp/config.toml` 可给同名键的全局默认（想保留旧行为的人设一次）。 +- 一次性导出已有：`mcpp emit build-database --spec compile-commands -o PATH`。 +- 迁移：根目录已有副本、且未配置该键时，停止更新，提示一次（`diag::note`，按工程记一次，记录在 `target/` 下），说明如何开启。 +- `${mcpp.compile_db}` 替换（`prepare/plan.cpp:1021`）在未配置时指向配置级数据库。 + +行业对照：CMake `CMAKE_EXPORT_COMPILE_COMMANDS` 默认关闭且写在构建目录；Meson 写在构建目录；xmake 显式命令或插件。 + +## 8. mcpp：性能（D16） + +| 项 | 改动 | 依据 | +|---|---|---| +| 查询命令 | 走持久计划（§3.2） | E8：604 / 554 ms | +| xlings 固定等待 | xlings 侧修复（§10.1），mcpp 提高内置 xlings 下限 | E9 | +| `cache list` | 条目发布时把大小与文件数写进 `entry.json`；`list` 只读 580 个小文件；旧条目缺字段时现算一次并回写（原子替换） | E10 | +| 编译器探测 | 直接 exec（不经 `sh -c env`），结果按（编译器路径、大小、mtime、参数）缓存在 `MCPP_HOME` | E9 | +| 编译数据库 | 计划不变不重写；根目录副本默认关闭 | E11 | +| 锁开销 | 无竞争时一次非阻塞 flock | §4 | + +验收数见 §21.3。 + +## 9. mcpp：其他 + +### 9.1 单次调用的 `--home`（D18） + +全局 `--home `：等同于只在本进程设置 `MCPP_HOME`，并按 SPEC-010 的规则解析（持久选择、定位器照常读取）。 +与 `self init --home` 的组合：前者选择本进程的原始 home，后者是持久选择的目标；两者同时出现且值不同时拒绝（用法错误，退出码 2），值相同时等同于只写一个。 +SPEC-010 升为生态级规范，xlings 引用同一份条文（§10.3）。 + +### 9.2 provisioning 失败时透传诊断（D17） + +今天失败时 mcpp 建议 `xlings install …` 手动安装，而这条命令会碰到同一个错误。改为原样透传 xlings 的诊断（含其 `help`）， +不再给出照做无效的命令。 + +## 10. xlings + +### 10.1 X1：消除固定等待 + +`heartbeat_loop_` 改为在条件变量上带超时等待、对 `stop_token` 立即唤醒（`std::condition_variable_any::wait_for` 的 stop_token 重载）； +`stdin_loop_` 的 poll 同样可被唤醒（自管道或 eventfd）。验收：`interface payload_status` ≤ 30 ms（今天 517 ms）。 + +### 10.2 X2：认领可证明的条目 + +- **认领规则**：未登记的符号链接，若指向本 home 自己的包存储，且恰为某个已登记包版本中同名的文件，则必为 xlings 所建，补登记后继续。 + 普通文件、指向存储之外的链接，仍按"用户文件"保留，安全边界不变。 +- 规则在每次投影变更时生效（`prepare_projection_change` 与单项放置共用），变更成功后 subos 的登记记录新状态，home 在第一次变更时自行完成迁移，不需要单独的迁移步骤（rev 3）。 +- 报错写明条目、原因、处理方式（例如 `xlings self doctor` 或手工移除）。 +- 影响范围：2026.10.9.1 之前初始化、装过 gcc 的 home，切换 gcc 版本时失败（E13）。 + +### 10.3 X3：home 选择 + +- `xlings self init --home ` 与持久选择（定位器文件，语义与 SPEC-010 一致）。 +- 全局 `--home `：本次调用有效；导出给子进程的 `XLINGS_HOME` 与之一致。 +- shim 的"按自身所在 home 定位"不变；mcpp 调用 xlings 继续经环境变量传 `XLINGS_HOME`。 + +## 11. mcpp-plugins + +- `mcpp.rules.sycl`：删除 `link_lib(":libstdc++.so.6")`；`--gcc-install-dir` 取自构建程序环境中已解析的 libstdc++ 提供方， + 使设备半边与宿主半边用同一份 libstdc++。保留 `-lsycl` 与搜索目录。SYCL 只走 U（§6.3 实测，条件 6）。 +- `mcpp.rules.hip`：同样处理；HIP 单元是否捕获运行时库抛出的带类型异常，按 E20 的方法实测后定为 U 或可孤岛（O7）。 +- 文件头关于"接缝"的说明改写：异常不能从运行时库跨到我们的映像按类型捕获，因此需要统一运行时。 +- fixture：`tests/sycl-consumer` 在 U 下跑两种（gcc；llvm + libstdc++ 提供包）；在 llvm 且无提供包时断言被拒绝且 `help` 给出一行修复； + 新增一个 `kind = "shared"` 的 fixture，断言 U 下跨 `.so` 捕获成功。 + +## 12. mcpp-index（mcpplibs） + +### 12.1 libstdc++ 提供包 `gcc.libstdcxx`（D21） + +```toml +provides = ["mcpp:c++-abi=libstdc++", "mcpp:compiler-runtime=libgcc"] +requires = ["mcpp:compiler=llvm", "mcpp:c-abi=glibc"] + +[build] +std-module = "/bits/std.cc" +std-module-flags = ["-isystem", "/backward"] +``` + +命名按索引习惯：上游项目作命名空间、组件作名字，对称于 `llvm.libcxx`、`llvm.compiler-rt-builtins`。版本为"gcc 版本.包修订号"（`16.1.0.1`、`15.1.0.1`），跟随 `xim:gcc` 已有的版本。 +内容借用 `xim:gcc` 载荷（不复制、无源码仓库）：头文件、`bits/std.cc`、`libstdc++.so.6`、`libstdc++.a`、`libsupc++.a`。只提供 `linux`（`requires mcpp:c-abi=glibc`）；musl 目标若需要，另由 `xim:musl-gcc` 以同样方式提供，不在本批。 + +### 12.2 声明与清点 + +- `compat.sycl-runtime`：声明 `requires = ["mcpp:c++-abi=libstdc++"]`（U；E20 表明不能孤岛）。HIP 对应包若存在，按 O7 的实测结果声明。 +- `mpusz.mp-units`：`requires mcpp:c++-abi=libc++` → `requires mcpp:compiler=llvm`。 +- `compat.cpptrace`、`compat.glfw`：`abi:glibc` → `requires mcpp:c-abi=glibc`。 +- `compat.mysql-connector-cpp`：保持（Linux 上的预编译静态库确实绑定 libstdc++）。 +- 发布前清点全部 `mcpp:c++-abi` 与 `abi:` 用法，逐条判定"约束的是哪一层"。 + +## 13. mcpp-language-server + +**本批不提 PR**：mcpp 的主路径（`emit build-database`）不变，影响有限。已提 issue [Sunrisepeak/mcpp-language-server#71](https://github.com/Sunrisepeak/mcpp-language-server/issues/71)，记录 mcpp 发布后要做的三件事： + +- 回退路径：mcpp 无法询问时，今天读根目录残留的 `compile_commands.json`（`src/project/mcpp.cppm:41`）。根目录副本默认关闭后， + 回退改为读 `target/.last-result.json` 指向的配置级数据库。 +- 构建后刷新：监听结果记录（§5.2），取代由 mcpp-vscode 转发的刷新。 +- CLion 插件：根目录副本不再默认存在，CLion 不再以 Compilation Database 接管 mcpp 工程，mcppls 自然启用；无需改动，补一条 e2e 断言。 + +## 14. mcpp-ide（新仓库，由 mcpp-vscode 演进） + +**rev 3：本批不实施**。mcpp-vscode 不动；本节的计划（仓库、命名、三层边界、能力表、O9）记录于 +[mcpp-vscode#23](https://github.com/mcpp-community/mcpp-vscode/issues/23)。本批交付的是 IDE 一线所依赖的 mcpp 契约（§5）， +它们同样服务于 agent 与 CI（D1），不依赖 IDE 一线的进度。 + +**仓库与命名（D19）**：新建 `mcpp-community/mcpp-ide`，**不改 mcpp-vscode 的仓库名**。插件名沿用 `mcpp-xxx` 模式： +VS Code 适配层仍以 `mcpp-vscode`（publisher `mcpp-community`，扩展 ID 不变）发布，Marketplace 上的用户无感升级； +JetBrains 适配层发布为 `mcpp-clion`。mcpp-vscode 仓库在 mcpp-ide 首个发布之后的去向见 O9。 + +**README 参照 mcppls**：标题与一句话定位；`**English** | 简体中文`；文档、安装、设置、For agents、规范、发布的链接行； +两段说明（问题是什么、mcpp-ide 怎么解决）；演示图；`Features`；`Install`（From an agent / From a release / From source）； +`Documentation`；`Contributing`；`License`。中英双语（`README.md`、`README.zh-CN.md`）。 + +### 14.1 结构 + +``` +mcpp-ide/ + mcpp.toml C++ 后端（mcpp 构建），二进制 mcpp-ide，`mcpp-ide serve`：stdio 上的 JSON-RPC + protocol/ 协议 JSON Schema + 来自 mcpp 真实输出的 golden fixture（两端适配层共同验收） + spec/ 命令表、设置注册表（沿用 config-registry.json）、菜单表、i18n → 生成 VS Code package.json 声明部分与 plugin.xml、设置类、Bundle + webui/ 共享面板：依赖图、库、缓存、设置、打包（VS Code webview / JetBrains JCEF 加载同一份） + vscode/ TS 适配层 + jetbrains/ Kotlin 适配层（CLion 2025.2+） + agents/ 面向具体 agent 的封装（skill 等） + tools/ 生成器、lint +``` + +### 14.2 三层边界 + +| 层 | 内容 | 不放 | +|---|---|---| +| mcpp | §5 的契约 | 任何 IDE 概念 | +| mcpp-ide 后端 | 会话与工作区模型（打开的文件属于哪个 mcpp.toml）、监听（结果记录、manifest）、编排（同工程串行、取消、新取代旧、构建后刷新）、视图模型（工程树、缓存、库目录、依赖图投影与折叠）、策略（清理的危险等级与确认）、设置校验与迁移、自检报告、运行时文案 i18n、与 mcpp 的能力协商（`--protocol-version`） | 不解析 mcpp.toml、不做依赖解析，只消费 mcpp 契约（同 mcppls 只读 build-database 的约束） | +| 适配层 | 生成的静态声明；协议到原生 UI 的映射；承载 webui 与消息桥；后端进程管理；找到或安装 mcpp 与后端；工作区信任 | 任何业务判断 | + +### 14.3 能力（两端同一张表） + +| 能力 | VS Code | JetBrains（CLion） | 依赖的 mcpp 契约 | +|---|---|---|---| +| 运行、构建、测试 | Task（PTY，颜色正常） | "mcpp" 运行配置类型，PTY 控制台，`main()` 旁 ▶ | 事件流、结果记录 | +| 测试树、单测重跑 | Testing API | IntelliJ 测试运行器 | `test --message-format json`（已有） | +| 调试 | DebugConfigurationProvider（cppdbg / CodeLLDB / lldb-dap） | 内置 GDB/LLDB | 启动描述 | +| 诊断 | Diagnostics（不再用正则 problem matcher） | Problems + console filter | 事件流的 diagnostic 记录 | +| manifest 编辑 | 诊断来自 mcpp（删除 TS 的 TOML 校验层） | 同左，解析与高亮由平台 TOML 插件提供 | manifest 诊断带位置 | +| 库浏览 | webui | webui | `index list` / `search` 的 JSON | +| 打包 | webui 打包面板 | 同左 | `pack --list-formats`、`pack --message-format json` | +| 语言功能 | 依赖 mcppls 扩展 | 依赖 mcppls CLion 插件 | — | + +### 14.4 分发 + +后端作为预编译包经 xlings 索引分发（O4），插件不分平台、只出一个包；版本：后端与适配层同仓同步发版，彼此握手校验。 + +## 15. 诊断分级 + +| 情形 | 等级 | 码 | +|---|---|---| +| 两个依赖对 `c++-abi` 的需求冲突 | 规划时拒绝（已有） | — | +| `requires mcpp:c++-abi` 不满足 | 规划时拒绝，附两条 `help` | 已有 requires 拒绝 | +| 孤岛声明存在但 I 条件 1–3 不满足 | 规划时拒绝 | 新：`cxx-runtime-island` | +| 孤岛目标文件跨运行时按类型捕获（条件 6） | 构建时错误（目标文件编译后才可检查） | 同 `cxx-runtime-collision` | +| 冲突集判据不满足（孤岛目标文件引用布局相关符号） | 构建时错误（目标文件在规划后才产生） | 新：`cxx-runtime-collision` | +| 链接后校验：映像以 GLIBCXX/CXXABI 导入 ABI 层冲突集 | 构建时错误 | 同上 | +| 未声明的外来运行时，I 条件满足 | `warning`（作者向），提示补声明 | — | +| 未声明的外来运行时，I 条件不满足 | 规划或构建时错误（同上两码） | — | +| `abi:` 旧写法 | `warning`（COMPAT 读法） | — | +| 根目录 `compile_commands.json` 不再更新 | `note`，每工程一次 | — | +| 等待构建锁 | 状态行与事件，不是诊断 | — | +| 副作用被拒绝 | 拒绝，列出副作用与原因；退出码 5 | 新：`effect-denied` | +| `requires` 的 `visibility = "private"` 用在 `c++-abi` 以外的层 | 规划时拒绝（一个映像只能有一个 C 运行时，openkal 边界一节） | 新：`requires-private-layer` | +| `mcpp.lock` 并发修改，重试后仍冲突 | 错误 | 新：`lockfile-concurrent` | +| provisioning 失败 | 错误，透传 xlings 诊断 | — | + +## 16. 规范与文档修订 + +| 文档 | 修订 | +|---|---| +| docs/50 + zh | 事件流、`mcpp.result`、`mcpp.search`、`mcpp.index`、`mcpp.pack.formats`、启动描述、`--deny`、`MCPP_CLIENT` | +| SPEC-003 | 新增退出码 `5`（副作用被约束拒绝）与 `128 + 信号`（130、143）；诊断码 `effect-denied`、`lockfile-concurrent` | +| docs/04、docs/06、docs/22 | `requires` 的对象形式与 `visibility`；`[target.] requires` | +| SPEC-002 + docs/22 | 孤岛与宿主模式、compiler-runtime 规则修订、图提供的 libstdc++、冲突集 | +| 新 `docs/specs/concurrency.md` | 三段、锁粒度、持有者、等待、原子发布、乐观并发 | +| 新 `docs/specs/human-output.md` | §5.7 | +| SPEC-010 → 生态级 | `--home`；xlings 引用 | +| docs/42（异构构建） | sycl / hip 的运行时与孤岛 | +| docs/04（mcpp.toml） | `[build] compile_commands` | +| 新 docs/52 + zh | 从程序与 agent 驱动 mcpp | +| CHANGELOG | 全部 | + +## 17. 兼容 + +| 项 | 读法 | 移除下限 | +|---|---|---| +| `abi:` 能力 | 映射到 `requires`，`warning` | 两个发布周期之后，且 index 中已无使用 | +| 未声明的外来运行时 | 按孤岛校验，可靠放行 + `warning` | 同上，且 mcpp-plugins 已发布声明版本 | +| 根目录 `compile_commands.json` | 停止更新 + `note` | 不移除（只是默认值变化） | +| `target/.build_cache` 旧格式 | 读取后迁移为每配置文件 | 一个发布周期 | +| `--offline` | 永久保留，等价 `--deny network` | — | +| sycl 在 llvm 下（含 mcpp-plugins 0.19.x 的旧规则） | 今天能构建、但运行时库抛出的 `sycl::exception` 接不住（E20）；新版拒绝并给出一行修复（加 libstdc++ 提供包，或 gcc） | 行为变化写入发布说明；mcpp-plugins 的 fixture 同批更新 | +| 缓存 | `c++-abi` 已是 ABI 维度，切换后重建 | — | +| 外来运行时按 `DT_NEEDED` 识别 | 比子串推断覆盖更多：链接了"内部用 libstdc++、对外只有 C 接口"的预编译库（例如系统 ICU）的 llvm 工程，单映像时得到 `warning`，含 C++ 共享库目标时被拒绝 | 合并前用 llvm 跑一遍 index 冒烟，统计新增的 warning 与拒绝，写入设计修订（§20.3） | +| `requires` 的对象形式、`[target.] requires` | 旧 mcpp 读到会报未知形式 | 使用它们的 index 描述文件声明 mcpp 版本下限 | + +## 18. 行业对照 + +| 体系 | 做法 | 本设计 | +|---|---|---| +| Conan | `compiler.libcxx` 是全图设置，计入 package_id；纯 C 包去掉该设置；允许按包覆盖，但不校验可靠性 | 全图一个提供方 + 声明 + 链接后校验 | +| CPS（CMake 4.3 正式支持） | `platform.cpp_runtime_vendor`（gnu / llvm / microsoft），有类型的声明 | 对应 `requires mcpp:c++-abi` | +| DPC++ | 预编译 `libsycl` 依赖 libstdc++；libc++ 需自建运行时且为实验性 | U 默认用 libstdc++；I 仅在可验证时 | +| Cargo | `links` 在清单中静态声明"同一原生库全图只出现一次" | 影响解析的事实静态声明 | +| CMake / Meson / xmake | 编译数据库默认不写源码根目录 | D14 | + +## 19. 不采纳的方案 + +| 方案 | 原因 | +|---|---| +| 在 `[toolchain]` 加 `stdlib` 键 | 合并 `compiler` 与 `c++-abi` 两层，违背层的划分条件（E14 证明可独立替换） | +| 解析时自动补 libstdc++ 提供包 | 多一个看不见的输入，可复现性变差；拒绝 + 一行修复已足够 | +| 只修链接顺序 | 只覆盖单映像；多映像与布局冲突仍不可靠 | +| 常驻服务放进 mcpp（`mcpp ide serve`） | 会话状态不属于构建契约；mcpp 保持一次性、无状态 | +| mcpp-ide 提供 MCP | 跨链观察已由结果记录解决，排队由 mcpp 的锁解决；两条 agent 路径会分歧 | +| 每个命令各自的快速路径 | 正确性靠清单，已多次漏项；查询无法受益 | +| 工程级的粗粒度锁 | 不同配置本来互不影响（E5）；锁住整个工程会让 IDE 与终端互相阻塞 | +| 新增"白名单"概念 | `--offline` 已是副作用约束，推广即可 | + +## 20. 交付：每仓库一个 PR，commit 作检查点 + +### 20.1 顺序（发布依赖） + +xlings → mcpp（提高内置 xlings 下限）→ mcpp-index（libstdc++ 提供包与 `requires` 新形式需要 mcpp 下限）→ mcpp-plugins。 +mcpp-language-server 与 IDE 一线本批不提 PR，跟踪于 mcppls#71、mcpp-vscode#23。 + +**任务与依赖（rev 3）** + +``` +X1 xlings 等待 ─┐ +X2 xlings 认领 ─┼─→ xlings 2026.10.26.1 发布 ─→ M15 内置 xlings 下限 +X3 xlings home ─┘ │ +M1 FileLock 可等待 ─→ M4 构建目录锁 ─→ M6 结果记录/启动描述 ─→ M13 编译数据库 +M3 配置记录每配置一文件 ─┘ │ │ +D22 链接产物原子发布 ────────┘（与 M4 同批：执行段不持锁的前提） │ +M5 取消 ─────────────────────→ M6 │ +M7 --deny（依赖 M6 的信封与 effects 报告） │ +M8 查询 JSON、M9 输出与颜色（独立） │ +M10 图提供的 libstdc++ ─→ M11 运行时模式 U/I/H ─→ M12 abi: 并入 requires + │ │ + └─→ index gcc.libstdcxx ─→ plugins sycl 规则 ─→ Tensorvia 冒烟 +M14 性能（独立）；M16 文档与规范（最后）；M17 测量（合并前） +``` + +每个节点一个检查点 commit；每个检查点在本地构建并运行受影响的单元与 e2e。依赖边表示"后者的测试需要前者"： +锁（M4）与原子发布（D22）同批，因为执行段不持锁正是以原子发布为前提；运行时模式（M11）的 e2e 需要图提供的 libstdc++（M10）与 index 的提供包。 +mcpp PR 合并前按惯例对 mcpp-plugins 跑 `ci.yml -f mcpp_source_ref=`，并用 Tensorvia 做外部冒烟（gcc、llvm + libstdc++、sycl）。 + +### 20.2 openxlings/xlings（一个 PR） + +1. `interface`：心跳与 stdin 线程可中断等待；测试断言 `payload_status` 耗时。 +2. `xvm/materialize`：可证明条目的认领规则；报错可操作。 +3. `self init`：升级时认领迁移；e2e 用"删登记保留链接"的 home 复现 E13。 +4. home：`self init --home`、定位器、全局 `--home`；引用生态 home 规范。 +5. 文档与 CHANGELOG。 + +### 20.3 mcpp-community/mcpp（一个 release PR，`release/`） + +1. `platform`：`FileLock` 可等待获取（退避、可取消、持有者内容）；`MCPP_CLIENT`。 +2. 流水线三段：规划段不写构建目录；修改段集中落盘；记录原子替换。 +3. 持久计划：指纹输入表、有效性检查；`.build_cache` 改为每配置文件；删除 `try_fast_build` / `try_fast_run` 与拒绝清单。 +4. 构建目录锁：只覆盖修改段；`run` exec 前释放；等待事件；`mcpp.lock` 乐观并发。修 #839（e2e：E4 复现在上个发布失败）。 +5. 取消语义；SPEC-003。 +6. 事件流（build、pack）、结果记录、`.last-result.json`、启动描述与 `run --print-launch`。 +7. `--deny` / `MCPP_DENY_EFFECTS`；`--offline` 改为别名。 +8. 查询补全：manifest 诊断位置、`search` / `index list` JSON、`pack --list-formats`。 +9. 非 TTY 输出规范、`help:` 行、颜色约定。 +10. 目标侧：图提供的 libstdc++ 在 clang 下的接线（头文件、std 模块、`-stdlib`）；compiler-runtime 规则修订；构建程序环境暴露 `c++-abi`。 +11. 运行时模式 U / I / H：孤岛声明、I 条件、冲突集数据表、链接顺序、链接后校验；删除子串推断，未声明外来运行时的 COMPAT 校验。修 #838（e2e：E1 在上个发布失败）。 +12. `abi:` 并入 `requires`（COMPAT 读法）。 +13. 根目录 `compile_commands.json` 默认关闭、`[build] compile_commands`、config 默认值、迁移提示；编译数据库按计划派生。 +14. 性能：`cache list` 条目大小、探测直接 exec 与缓存。 +15. 全局 `--home`；provisioning 诊断透传；内置 xlings 下限。 +16. 规范、双语文档、CHANGELOG、版本号；`check_*` 脚本全过。 +17. 合并前测量：用 llvm 对 index 全量冒烟，统计 D13（`DT_NEEDED` 识别）新增的 warning 与拒绝；计划等价性测试的语料结果；写回本设计。 + +### 20.4 mcpplibs/mcpp-index（一个 PR） + +1. libstdc++ 提供包（§12.1）。 +2. `compat.sycl-runtime` 声明 `requires mcpp:c++-abi=libstdc++`；HIP 对应包按 O7 实测结果声明。 +3. mp-units、cpptrace、glfw 的声明修正；全量清点记录。 +4. 一个用 llvm + libstdc++ 的冒烟包测试。 +5. 使用 `requires` 新形式或依赖 `gcc.libstdcxx` 的描述文件声明 mcpp 版本下限。 + +### 20.5 mcpp-community/mcpp-plugins（一个 PR） + +1. sycl / hip 规则：删除 `:libstdc++.so.6`，`--gcc-install-dir` 跟随已解析的提供方。 +2. fixture：U（gcc；llvm + 提供包）；llvm 无提供包时被拒绝且给出一行修复；`kind = "shared"` 在 U 下跨 `.so` 捕获成功。 +3. 文档与版本。 + +### 20.6 Sunrisepeak/mcpp-language-server（本批不提 PR） + +已提 issue [#71](https://github.com/Sunrisepeak/mcpp-language-server/issues/71)：回退路径读 `.last-result.json`；监听结果记录刷新；CLion e2e 断言 mcpp 工程不被 Compilation Database 接管。mcpp 发布后由 mcppls 自行排期。 + +### 20.7 mcpp-community/mcpp-ide（rev 3：本批不实施，见 mcpp-vscode#23） + +1. 仓库骨架、`mcpp.toml`、CI（含两端适配层）；README 与 README.zh-CN 按 mcppls 的结构（§14）。 +2. `protocol/`：schema 与 golden fixture（取自 §20.3 的 mcpp 分支）。 +3. 后端：会话、监听、编排、视图模型、能力协商。 +4. `spec/` 与生成器。 +5. `webui/`：依赖图、库、缓存、设置、打包。 +6. `vscode/`：由 mcpp-vscode 迁移，改为消费后端；Testing API、调试、诊断；仍以扩展 ID `mcpp-vscode` 发布。 +7. `jetbrains/`：运行配置、测试树、调试、console filter、面板；以 `mcpp-clion` 发布。 +8. `agents/`：面向具体 agent 的封装。 +9. 分发与发布流程。 + +## 21. 验收 + +### 21.1 不变量 + +1. 同配置并发构建：双方都成功，第二个等待后不重复编译（E4 的复现 20 轮全过）。 +2. 不同配置并发：互不等待。 +3. `mcpp run` 的程序运行期间，同配置构建不被阻塞（锁已在 exec 前释放）。 +4. 查询命令在构建进行中不阻塞。 +5. 任何组合下进程内只有一套 C++ ABI 运行时被我们的代码使用；不可靠的组合在规划或构建时被拒绝，不存在"能运行但 catch 失败"的产物。 +6. 跨 `.so` 的 `catch (const std::exception&)` 在 U 模式下成功（E15）。 +7. 计划有效性判断只在一处；删除快速路径后，#544、#716、#755 与 riscv 那次的 e2e 全部通过。 +8. 启动描述执行的环境与 `mcpp run` 一致（逐项比较）。 +9. 结果记录在构建结束（成功、失败、取消）时总是存在且完整。 +10. **计划等价**：对一组工程（mcpp 自身、examples/、mcpp-plugins 的 fixture、index 冒烟包），读取持久计划与重新规划得到的 `plan.json` 逐字节相同；只改动一个指纹输入时计划失效。 +11. 链接产物在任何时刻要么是旧文件、要么是完整的新文件（并发构建期间反复 exec 测试二进制不出现 ENOEXEC）。 + +### 21.2 新 e2e（必须在上一个发布上失败） + +并发冷构建；#838 最小复现；孤岛拒绝；llvm + libstdc++ 提供包的 `import std` 与跨 `.so` 捕获；exec 前释放锁；等待事件； +取消与结果记录；`--deny network` 在需要安装时拒绝；根目录 `compile_commands.json` 默认不写与迁移提示；`CLICOLOR_FORCE`； +`run --print-launch`；`index list --format json`；`--home`。 + +### 21.3 性能（本机同条件，hyperfine） + +| 命令 | 今天 | 目标 | +|---|---|---| +| `xlings interface payload_status` | 517 ms | ≤ 30 ms | +| `emit build-database`（计划有效） | 604 ms | ≤ 80 ms | +| `toolchain list --format json` | 554 ms | ≤ 80 ms | +| `cache list --format json`（580 条目） | 376 ms | ≤ 40 ms | +| `build`（无改动） | 49.6 ms | ≤ 55 ms（不回退） | +| 完整构建中的编译数据库 | 119–126 ms | 计划不变时 0 | + +## 22. 开放问题 + +| # | 问题 | 处理 | +|---|---|---| +| O1 | ~~`--deny` 的名字~~ **已定**：`--deny`；与 `MCPP_DENY_EFFECTS` 取并集，manifest 不能放宽 | — | +| O2 | ~~孤岛声明的位置~~ **已定**（D20）：`requires` 对象形式 + `visibility = "private"`，可写在 `[target.]` | — | +| O3 | ~~仓库~~ **已定**（D19）：新仓库 mcpp-ide，不改 mcpp-vscode 仓库名，插件名沿用 `mcpp-xxx` | — | +| O4 | ~~webui 与分发~~ **已定**：webui 只渲染；后端经 xlings 索引分发 | — | +| O5 | ~~Windows 与 macOS 的锁和运行中程序的替换~~ **已实测**（E23、E24）：PE 先改名挪开再原地链接；锁语义三平台一致 | mcpp#840 | +| O6 | ~~提供包名~~ **已定**（D21）：`gcc.libstdcxx`，按 `xim:gcc` 已有版本各一个 | — | +| O7 | ~~冲突集判据对 SYCL / HIP 是否成立~~ **已关闭**：SYCL 走 U（E19、E20）；HIP 没有第二套运行时（E21） | — | +| O8 | ~~llvm + libstdc++ 的其他组合~~ **已实测**（E26）：x86_64 全部正确；aarch64 无 llvm 载荷 | mcpp#840 | +| O9 | ~~mcpp-vscode 仓库的去向~~ **已定**：本批不动，记录于 mcpp-vscode#23 | — | +| O10 | ~~PE 下孤岛条件的形式~~ **已实测**（E25）：条件 1 放宽为"共用一个运行时映像"，条件 3、5 照常成立（§6.3） | mcpp#840 | + +## 23. 多视角自审 + +每条：发现 → 处理。已并入正文的标注所在章节。 + +| 视角 | 发现 | 处理 | +|---|---|---| +| 架构 | 三段式与持久计划触及每个命令，是全批最大的风险点 | 在 mcpp PR 中拆成 commit 2、3 两个检查点；验收增加"计划等价"（§21.1 第 10 条）；把 #544、#716、#755、riscv 那次的 e2e 全部保留并在删除快速路径后通过 | +| 架构 | 规划段不持锁，两个进程可能交错写同一配置的 `plan.json` | 修改段拿锁后复核计划指纹，不一致则重规划一次（§3.2） | +| 架构 | 计划若包含模块图，源码改动（新增 import）会让计划过期 | 工程内模块边由 ninja dyndep 在构建时扫描，源码内容不是计划输入；源文件的增删经 glob 目录列表进入指纹（§3.2） | +| 正确性 | 执行段不持锁时，`mcpp test` 可能 exec 一个正被并发构建重写的测试二进制（GNU ld 先删后写） | 链接产物一律"临时名 + 原子改名"，由 ninja 链接边包装实现，与链接器无关（D22，§4.1）；验收第 11 条 | +| 正确性 | 构建被 SIGKILL 时没有结果记录，`.last-result.json` 指向更早的结果 | 结果记录带开始时间与调用方标签；读者把"比当前锁持有者更早"的记录视为陈旧；锁随进程消失，不会僵死 | +| 正确性 | 持锁方卡死时等待者无限等待 | 不加超时参数；等待状态显示持有者与已用时间，调用方可以取消（§4.2）。IDE 由 mcpp-ide 取消自己的旧构建 | +| 语义 | `visibility = "private"` 若用于 `c-abi`，等于在一个映像里放第二个 C 运行时，违背 openkal 边界一节"一个映像一个 C 运行时" | 只接受 `mcpp:c++-abi`；其他层规划时拒绝（`requires-private-layer`，§15） | +| 语义 | compiler-runtime 规则放宽后，`__udivti3` 的那条依据是否失效 | 依据的真实不变量是"一个程序只有一个 compiler-runtime"，由"每层一个提供方"保证；放宽的只是"必须同族"，提供包同时提供 `compiler-runtime=libgcc`，整个程序仍只有一个（§6.4） | +| 语义 | `--deny` 能否被工程放宽 | 不能：约束来自调用方（命令行与环境变量取并集），manifest 没有对应键（§5.5、O1） | +| 体验 | sycl 在 llvm 默认工具链下被拒绝，Tensorvia 这类工程第一次构建就报错 | 拒绝信息给出两条可直接粘贴的修复（加 `gcc.libstdcxx`，或 `--toolchain gcc@16.1.0`）；docs/42 写明原因（E20）；Tensorvia 自己的 manifest 注释已建议 gcc | +| 体验 | 根目录 `compile_commands.json` 默认关闭后，只用 clangd、不用 mcppls 的新用户没有编译数据库 | 接受这个代价（与 CMake、Meson 一致；CLion 接管问题随之消失）；docs/01 入门写明 `[build] compile_commands`；`mcpp new` 模板不默认开启 | +| 体验 | 只有 shell 的 agent 需要区分"被约束挡住"与"构建失败" | 新增退出码 5（D23） | +| 兼容 | `DT_NEEDED` 识别比子串推断覆盖更广，可能新拒绝一些今天能构建的 llvm 工程 | 合并前对 index 全量冒烟测量影响面并写回设计（§17、§20.3 第 17 条）；单映像时只是 `warning` | +| 兼容 | 旧 mcpp 读不懂 `requires` 对象形式与 `gcc.libstdcxx` 的接线 | index 描述文件声明 mcpp 版本下限（§20.4 第 5 条） | +| 兼容 | `target/.build_cache` 格式变化 | 读取旧格式后迁移，一个发布周期后移除读取（§17） | +| 跨平台 | 孤岛的 6 个条件是按 ELF 的全局符号作用域推导的 | Mach-O：Apple 平台只有 libc++，不存在孤岛；PE：按 DLL 导入表绑定，没有全局作用域，条件 1、3 可能可以放宽，条件 4、5、6 仍成立（链接期在同一映像内、按类型捕获跨运行时都与格式无关）。列为 O10，probe PR 实测后写入 SPEC-002 | +| 跨平台 | Windows 上正在运行的 exe 不能被覆盖 | 原子替换在 Windows 上先把旧文件改名挪开（§4.3）；O5 实测 | +| 性能 | 计划有效性检查本身的开销 | 预算：清单哈希 + glob 目录列表 + 载荷目录 `stat`，典型工程 ≤ 10 ms；验收时用 mcpp 仓库自身（大 glob）测一次 | +| 性能 | `DT_NEEDED` 识别要读每个链接输入的 ELF 头 | 结果按（路径、大小、mtime）缓存在计划里 | +| 安全 | 启动描述与结果记录可能把调用方环境（含机密）写进 `target/` | `env` 只含 mcpp 注入或改写的变量（§5.3）；`MCPP_CLIENT` 只做记录，不影响行为 | +| 最小核心 | 逐项核对新增内容是否属于"构建语义、对自身行为的契约、通用基础设施" | 全部满足：锁、取消、原子发布、记录是基础设施；事件流、启动描述、副作用约束、JSON 查询、人类输出规范是对自身行为的契约；运行时模式是构建语义。面向具体 agent 的封装、会话、监听、视图模型都不在 mcpp | +| 最小核心 | 是否新增了概念 | 没有新概念：副作用约束是 `--offline` 的推广；孤岛复用 `requires` 与 `visibility`；结果记录对称于 `resolution.json`；`[target.] requires` 对称于 `requires_abi` | +| 可测性 | 复现脚本里的实验要变成回归测试 | A（#839）、B（#838）、F（llvm + libstdc++）、G（退出码）、I（冲突集）转为 e2e，且必须在上一个发布上失败（§21.2）；D、E 的性能数转为 §21.3 的验收 | +| 生态 | mcpp-ide 依赖尚未发布的 mcpp 契约 | 发布顺序把 mcpp-ide 放在最后；`protocol/` 的 golden fixture 取自 mcpp 发布分支；后端用 `--protocol-version` 做能力协商，旧 mcpp 上关闭相应功能 | + +## 附录 A：讨论收敛记录（摘要） + +| 问题 | 结论 | +|---|---| +| `mcpp run` 的返回值 | 程序自身的退出码（POSIX `execve`），E16 | +| IDE 标红只在 C++23 | `import std` 在 gcc lane 的 BMI 不可被 clangd 读，与退出码无关，E17；CLion 控制台的红色是 stderr 无颜色码，E18 | +| 手动 External Tool 还是插件 | 做 mcpp-ide；语言功能由 mcppls 的插件提供 | +| 合仓与复用 | 共享 spec、webui、协议 fixture；通用逻辑在 C++ 后端 | +| 常驻服务放哪 | 独立工具 mcpp-ide，不进 mcpp | +| agent 模式 | 不单设；统一控制契约；mcpp-ide 不提供 MCP | +| 锁 | 粒度最小；三段式；持久计划 | +| C++ 运行时 | 三种模式；U 默认；按目标侧模型由依赖选择；不新增工程配置项；拒绝 + 一行修复；`abi:` 并入 `requires` | +| 本机无法构建 Tensorvia | xlings 升级缺陷 X2（E12、E13），非 mcpp 缺陷 | +| xlings 的 home | 补齐与 mcpp 一致的机制，规范升为生态级 | +| 根目录编译数据库 | 默认不写，可配置，已有副本停止更新并提示 | diff --git a/.agents/docs/2026-10-26-control-contract-runtime-and-ide-ecosystem-repro.sh b/.agents/docs/2026-10-26-control-contract-runtime-and-ide-ecosystem-repro.sh new file mode 100755 index 000000000..462f7c209 --- /dev/null +++ b/.agents/docs/2026-10-26-control-contract-runtime-and-ide-ecosystem-repro.sh @@ -0,0 +1,202 @@ +#!/usr/bin/env bash +# Reproductions for .agents/docs/2026-10-26-control-contract-runtime-and-ide-ecosystem-design.md +# +# Scenarios (each prints and never asserts; run one with SCENARIO=): +# A #839 concurrent cold builds of one configuration (E4) and of two configurations (E5) +# B #838 split exception ABI without sycl (E1), and the link-order fix (E2) +# C query/build latency baseline (E8) -- needs hyperfine +# D xlings interface fixed wait (E9) -- needs strace +# E cache list cost (E10) -- needs strace +# F clang + libstdc++ import std, cross-.so catch (E14, E15) +# G mcpp run exit status (E16) +# H X2 unrecorded projection entry refused (E12, E13) -- FRESH=1 builds a fresh home (downloads gcc twice) +# I SYCL units vs the runtime collision set and foreign typeinfo (E19, E20) -- needs xim:dpcpp 7.1.0; +# TENSORVIA= adds Tensorvia's device units +# +# Uses only payloads already in the mcpp home, except H with FRESH=1. +set -u +M=${MCPP:-$HOME/.xlings/data/xpkgs/xim-x-mcpp/2026.10.24.1/bin/mcpp} +R=${MCPP_HOME:-$HOME/.mcpp}/registry/data/xpkgs +W=${WORK:-/tmp/mcpp-ecosystem-repro} +export MCPP_JOBS=${MCPP_JOBS:-8} +mkdir -p "$W" +want() { [ -z "${SCENARIO:-}" ] || [ "$SCENARIO" = "$1" ]; } + +modules_project() { # $1 = dir + rm -rf "$1"; mkdir -p "$1/src" + printf '[package]\nname = "lk"\nversion = "0.1.0"\nstandard = "c++23"\n[toolchain]\ndefault = "gcc@16.1.0"\n' > "$1/mcpp.toml" + for i in $(seq 1 24); do + printf 'export module m%d;\nimport std;\nexport int f%d(){ std::vector v(%d); return (int)v.size(); }\n' $i $i $i > "$1/src/m$i.cppm" + done + { echo 'import std;'; for i in $(seq 1 24); do echo "import m$i;"; done + echo 'int main(){ std::println("{}", f1()+f24()); }'; } > "$1/src/main.cpp" +} + +if want A; then + echo "===== A: concurrent builds (#839)" + modules_project "$W/a"; cd "$W/a" + for r in 1 2 3 4 5; do + rm -rf target + ( nice -n 10 "$M" build >"$W/a1.log" 2>&1; echo $? >"$W/a1.rc" ) & + ( sleep 0.$r; nice -n 10 "$M" build >"$W/a2.log" 2>&1; echo $? >"$W/a2.rc" ) & + wait + echo "round $r: rc=$(cat "$W/a1.rc"),$(cat "$W/a2.rc") $(grep -h -m1 'error:' "$W/a1.log" "$W/a2.log" | head -1)" + done + ( "$M" build >/dev/null 2>&1; echo "dev rc=$?" ) & ( "$M" build --release >/dev/null 2>&1; echo "release rc=$?" ) & wait +fi + +if want B; then + echo "===== B: split exception ABI (#838)" + G=$R/xim-x-gcc/16.1.0/lib64 + rm -rf "$W/b"; mkdir -p "$W/b/src" "$W/b/tests"; cd "$W/b" + cat > mcpp.toml < src/xrt.cppm + cat > tests/catch.cpp <<'EOF' +import std; +import xrt; +int main() { + auto ep = std::make_exception_ptr(std::logic_error("x")); // a libc++abi-only entry point + try { xrt::checked(-1); } + catch (const std::exception& e) { std::println("caught as std::exception: {}", e.what()); return 0; } + catch (...) { std::println("std::exception missed -> (...)"); return 1; } +} +EOF + nice -n 10 "$M" test 2>&1 | tail -3 + D=$(dirname "$(find target -name build.ninja | head -1)"); LL=$R/xim-x-llvm/22.1.8/bin + "$LL/llvm-nm" -D --undefined-only "$D/bin/catch" | grep -E "__cxa_throw|__gxx_personality|_ZTISt13runtime_error" + echo "-- relink with the toolchain's own runtime first (E2)" + cd "$D" + NJ=$(command -v ninja || find "$R" -name ninja -type f -perm -u+x | head -1) + CMD=$("$NJ" -t commands bin/catch | tail -1) + LIBS="$R/xim-x-llvm/22.1.8/lib/x86_64-unknown-linux-gnu/libc++.a $R/xim-x-llvm/22.1.8/lib/x86_64-unknown-linux-gnu/libc++abi.a" + FIXED=$(echo "$CMD" | sed "s# -l:libstdc++.so.6# $LIBS -l:libstdc++.so.6#; s#-o bin/catch#-o bin/catch.fixed#; s#bin/catch.rsp#bin/catch.fixed.rsp#") + printf 'obj/xrt.m.o\nobj/catch.o\nobj/std.o\n' > bin/catch.fixed.rsp + eval "$FIXED" && ./bin/catch.fixed; echo "rc=$? GLIBCXX imports: $("$LL/llvm-nm" -D --undefined-only bin/catch.fixed | grep -c GLIBCXX)" +fi + +if want C; then + echo "===== C: latency baseline" + modules_project "$W/c"; cd "$W/c"; "$M" build >/dev/null 2>&1 + hyperfine -N --warmup 2 --runs 8 "$M --version" "$M --protocol-version" "$M build" "$M run" \ + "$M emit build-database --format json" "$M self env --format json" \ + "$M cache list --format json" "$M toolchain list --format json" 2>&1 | grep -E "^Benchmark|Time \(mean" +fi + +if want D; then + echo "===== D: xlings interface fixed wait" + X=${MCPP_HOME:-$HOME/.mcpp}/registry/bin/xlings + cd "${MCPP_HOME:-$HOME/.mcpp}/registry" + for a in '--version' 'payload_status --args {"targets":["xim:gcc@16.1.0"]}'; do + s=$(date +%s%N); env -u XLINGS_PROJECT_DIR -u XLINGS_ACTIVE_SUBOS XLINGS_HOME="$PWD" "$X" interface $a >/dev/null 2>&1 + echo "interface ${a%% *}: $(( ($(date +%s%N)-s)/1000000 )) ms" + done + strace -f -tt -T -e trace=nanosleep,clock_nanosleep,futex,poll -o "$W/d.txt" \ + env -u XLINGS_PROJECT_DIR -u XLINGS_ACTIVE_SUBOS XLINGS_HOME="$PWD" "$X" interface payload_status --args '{"targets":["xim:gcc@16.1.0"]}' >/dev/null 2>&1 + awk '{ if (match($0, /<([0-9.]+)>$/, m) && m[1] > 0.05) print }' "$W/d.txt" +fi + +if want E; then + echo "===== E: cache list" + C=${MCPP_HOME:-$HOME/.mcpp}/build-cache + echo "files: $(find "$C" -type f | wc -l)" + strace -c -f -o "$W/e.txt" "$M" cache list --format json >/dev/null 2>&1; head -8 "$W/e.txt" +fi + +if want F; then + echo "===== F: clang + libstdc++" + CXX=$R/xim-x-llvm/22.1.8/bin/clang++; GLIBC=$R/xim-x-glibc/2.44.3; UAPI=$R/xim-x-linux-headers/5.11.1 + for V in 15.1.0 16.1.0; do + G=$R/xim-x-gcc/$V; GID=$(ls -d "$G"/lib/gcc/x86_64-linux-gnu/* | head -1) + rm -rf "$W/f-$V"; mkdir -p "$W/f-$V"; cd "$W/f-$V" + INC="-nostdinc++ -isystem $G/include/c++/$V -isystem $G/include/c++/$V/x86_64-linux-gnu -isystem $G/include/c++/$V/backward -nostdlibinc -isystem $GLIBC/include -isystem $UAPI/include" + C="--no-default-config -std=c++23 --gcc-install-dir=$GID $INC" + L="--no-default-config -stdlib=libstdc++ --gcc-install-dir=$GID -fuse-ld=lld --rtlib=libgcc --unwindlib=libgcc -L$G/lib64 -Wl,-rpath,$G/lib64 -B$GLIBC/lib64 -L$GLIBC/lib64 -Wl,-rpath,$GLIBC/lib64 -Wl,--dynamic-linker=$GLIBC/lib64/ld-linux-x86-64.so.2" + "$CXX" $C -Wno-reserved-module-identifier --precompile -x c++-module "$G/include/c++/$V/bits/std.cc" -o std.pcm 2>/dev/null; echo "[$V] std.pcm rc=$?" + "$CXX" $C -c std.pcm -o std.o 2>/dev/null + printf 'import std;\nint main(){ std::vector v{3,1,2}; std::ranges::sort(v); std::println("{} {}", v, std::format("{:.2f}", 3.14159));\n try { throw std::runtime_error("boom"); } catch (const std::exception& e) { std::println("caught: {}", e.what()); }\n std::jthread t([]{ std::println("thread ok"); }); }\n' > main.cpp + "$CXX" $C -fmodule-file=std=std.pcm -c main.cpp -o main.o && "$CXX" $L main.o std.o -o app && ./app; echo "[$V] run rc=$?" + printf '#include \nint checked(int v){ if (v<0) throw std::runtime_error("negative (from .so)"); return v*2; }\n' > lib.cpp + printf 'import std;\nint checked(int);\nint main(){ try { checked(-1); } catch (const std::exception& e) { std::println("caught across .so: {}", e.what()); return 0; } catch (...) { return 1; } }\n' > use.cpp + "$CXX" $C -fPIC -c lib.cpp -o lib.o && "$CXX" $L -shared lib.o -o libthrow.so \ + && "$CXX" $C -fmodule-file=std=std.pcm -c use.cpp -o use.o && "$CXX" $L use.o std.o -L. -lthrow -Wl,-rpath,"$PWD" -o use && ./use + echo "[$V] cross-.so rc=$?" + done +fi + +if want G; then + echo "===== G: mcpp run exit status" + rm -rf "$W/g"; mkdir -p "$W/g/src"; cd "$W/g" + printf '[package]\nname = "rc"\nversion = "0.1.0"\nstandard = "c++23"\n' > mcpp.toml + printf 'import std;\nint main(int argc, char* argv[]) {\n std::string_view m = argc > 1 ? argv[1] : "ok";\n if (m == "exit") return std::atoi(argv[2]);\n if (m == "throw") throw std::runtime_error("boom");\n if (m == "abort") std::abort();\n if (m == "segv") std::raise(11);\n return 0; }\n' > src/main.cpp + "$M" build >/dev/null 2>&1 + for a in "ok" "exit 3" "exit 255" "exit 256" "throw" "abort" "segv"; do + "$M" run -- $a >/dev/null 2>&1; echo "mcpp run -- $a => $?" + done +fi + +if want H; then + echo "===== H: X2 unrecorded projection entry" + if [ "${FRESH:-0}" = 1 ]; then + H=$W/h-home; rm -rf "$H" "$W/h1"; mkdir -p "$W/h1/src" + printf '[package]\nname = "p1"\nversion = "0.1.0"\nstandard = "c++23"\n[toolchain]\ndefault = "gcc@16.1.0"\n[xlings.workspace]\n"xim:gcc" = "15.1.0"\n' > "$W/h1/mcpp.toml" + printf 'import std;\nint main(){ std::println("p1"); }\n' > "$W/h1/src/main.cpp" + (cd "$W/h1" && MCPP_HOME=$H nice -n 10 "$M" build 2>&1 | tail -2) + X=$H/registry/bin/xlings; E="env -u XLINGS_PROJECT_DIR -u XLINGS_ACTIVE_SUBOS XLINGS_HOME=$H/registry" + cd "$H/registry" + $E "$X" use gcc 15.1.0 2>&1 | tail -1; $E "$X" use gcc 16.1.0 2>&1 | tail -1 # E12: both succeed + cp subos/default/.xlings.json "$W/h-ledger.bak" + python3 - "$H/registry/subos/default/.xlings.json" <<'EOF' +import json, sys +p = sys.argv[1]; d = json.load(open(p)) +def strip(o): + if isinstance(o, dict): + for k in [k for k in o if k in ("libatomic.so", "libatomic.so.1")]: del o[k] + for v in o.values(): strip(v) + elif isinstance(o, list): + for v in o: strip(v) +strip(d); json.dump(d, open(p, "w"), indent=2) +EOF + $E "$X" use gcc 15.1.0 2>&1 | tail -1 # E13: refused + cp "$W/h-ledger.bak" subos/default/.xlings.json + else + echo "set FRESH=1 to build a fresh home (downloads gcc 16.1.0 and 15.1.0)" + fi +fi + +if want I; then + echo "===== I: SYCL units vs the collision set (E19, E20)" + DX=$R/xim-x-dpcpp/7.1.0/bin/clang++; G=$R/xim-x-gcc/15.1.0; GID=$(ls -d "$G"/lib/gcc/x86_64-linux-gnu/* | head -1) + GLIBC=$R/xim-x-glibc/2.44.3; UAPI=$(ls -d "$R"/xim-x-linux-headers/* | head -1); LL=$R/xim-x-llvm/22.1.8; NM=$LL/bin/llvm-nm + P=$R/mcpp-x-plugins/0.19.1/mcpp-plugins-0.19.1/tests/sycl-consumer + rm -rf "$W/i"; mkdir -p "$W/i"; cd "$W/i" + "$NM" -D --defined-only "$R/xim-x-gcc/16.1.0/lib64/libstdc++.so.6" | awk '{print $3}' | sed 's/@.*//' | sort -u > gnu.txt + "$NM" --defined-only -g "$LL/lib/x86_64-unknown-linux-gnu/libc++.a" "$LL/lib/x86_64-unknown-linux-gnu/libc++abi.a" 2>/dev/null \ + | awk 'NF==3 && $2 ~ /[TDRVWBui]/ {print $3}' | sort -u > llvm.txt + comm -12 gnu.txt llvm.txt > collide.txt; echo "collision set: $(wc -l < collide.txt)" + F="-fsycl -std=c++17 -O2 -fPIC --gcc-install-dir=$GID -isystem$GLIBC/include -isystem$UAPI/include" + nice -n 10 "$DX" $F -I"$P/include" -x c++ -c "$P/src/kernels/saxpy.sycl" -o saxpy.o + if [ -n "${TENSORVIA:-}" ]; then + for f in runtime arithmetic matmul; do + nice -n 10 "$DX" $F -I"$TENSORVIA/include" -I"$TENSORVIA/src/backend/sycl" -x c++ -c "$TENSORVIA/src/backend/sycl/device/$f.sycl" -o "tv_$f.o" + done + fi + for o in *.o; do + "$NM" -u "$o" | awk '{print $2}' | sort -u > u.txt + echo "== $o: $(comm -12 u.txt collide.txt | wc -l) in collision set; layout-sensitive:" \ + "$(comm -12 u.txt collide.txt | c++filt | grep -c -E 'runtime_error|logic_error|out_of_range|invalid_argument|length_error|::what\(\)')" + echo " foreign typeinfo: $(grep '^_ZTI' u.txt | comm -23 - collide.txt | c++filt | tr '\n' ' ')" + done +fi diff --git a/.agents/docs/README.md b/.agents/docs/README.md index 0958a5743..7abf7a5a7 100644 --- a/.agents/docs/README.md +++ b/.agents/docs/README.md @@ -18,7 +18,7 @@ superseded_by: 2026-09-07-....md # when status is superseded --- ``` -336 records. +337 records. ## By subject @@ -64,6 +64,10 @@ Records that declare one. Everything else is listed by date below. - [A curriculum for the examples, a reference for the documentation, and a check with a denominator](2026-09-08-examples-curriculum-and-documentation-plan.md) — superseded by [2026-09-08-documentation-architecture-three-trees.md](2026-09-08-documentation-architecture-three-trees.md) - [Three documentation trees, three audiences, and the rule for citing between them](2026-09-08-documentation-architecture-three-trees.md) — active +### ecosystem + +- [控制契约、并发与性能、单一 C++ 运行时，以及 IDE 生态（生态级，#838、#839）](2026-10-26-control-contract-runtime-and-ide-ecosystem-design.md) — active + ### heterogeneous - [A dlopen surface no closure walks, and a process with two unwinders](2026-09-09-dlopen-surface-and-two-unwinders.md) — landed @@ -139,6 +143,7 @@ Records that declare one. Everything else is listed by date below. ### 2026-10 +- [控制契约、并发与性能、单一 C++ 运行时，以及 IDE 生态（生态级，#838、#839）](2026-10-26-control-contract-runtime-and-ide-ecosystem-design.md) — active - [目标基座与执行宿主：sysroot 的通用模型、运行器契约、自备输入，以及同批的发布产物路径（#832）](2026-10-23-target-base-and-execution-host-design.md) — landed - [依赖 image 的资源归属、模块扫描边界与锁文件稳定写入](2026-10-21-resource-ownership-module-scan-and-lock-write-design.md) — active - [首次初始化、home 重定向与网络诊断优化设计](2026-10-19-first-run-home-and-network-design.md) — landed diff --git a/.github/actions/bootstrap-mcpp/action.yml b/.github/actions/bootstrap-mcpp/action.yml index f50a072a8..5d6b49e1c 100644 --- a/.github/actions/bootstrap-mcpp/action.yml +++ b/.github/actions/bootstrap-mcpp/action.yml @@ -33,7 +33,7 @@ inputs: # `package.name`, so one of the two was simply unreachable — and which one # depended on the machine, which is why CI failed on `compat:lua` on # Windows and `mcpplibs.capi:lua` on Linux. Never pin below that. - default: '2026.10.24.1' + default: '2026.10.26.1' outputs: sandbox-key: diff --git a/.github/actions/setup-macos-llvm/action.yml b/.github/actions/setup-macos-llvm/action.yml index ba8d15ee3..0c6336922 100644 --- a/.github/actions/setup-macos-llvm/action.yml +++ b/.github/actions/setup-macos-llvm/action.yml @@ -15,7 +15,7 @@ inputs: # Floor imposed by the index, not a routine bump — see # .github/actions/bootstrap-mcpp/action.yml for why 0.4.69 is required # (two packages named `lua` in one repo need openxlings/xlings#381). - default: '2026.10.24.1' + default: '2026.10.26.1' image: description: > The runner label the job runs on (macos-15, xcode-27). It is part of the diff --git a/.github/tools/check_function_sizes.sh b/.github/tools/check_function_sizes.sh index a00f1fdfd..a6d18b349 100755 --- a/.github/tools/check_function_sizes.sh +++ b/.github/tools/check_function_sizes.sh @@ -29,7 +29,9 @@ # # A compile database that names BMIs explicitly (-fmodule-file=...), which # only a build actually produces: `mcpp build --toolchain llvm@23.1.3` writes -# compile_commands.json at the project root. This script does not build it: +# the configuration's compile_commands.json, which target/.last-result.json +# names (a root copy only when [build] compile_commands asks for one, mcpp +# 2026.10.26.1+). This script does not build it: # the caller runs that build first. check_file_lengths.sh needs no such # division because it reads the tree. # @@ -57,7 +59,18 @@ cd "$REPO_DIR" || { echo "FAIL: cannot cd to $REPO_DIR" >&2; exit 1; } LINE_THRESHOLD=400 DIR="src/build/prepare" PRIMARY="src/build/prepare.cppm" +# The database the build wrote: the project root's copy when the project asks +# for one (`[build] compile_commands`), else the configuration's own, which the +# result record names (mcpp 2026.10.26.1+, design 2026-10-26 D14). CDB="compile_commands.json" +if [ ! -f "$CDB" ] && [ -f target/.last-result.json ]; then + CDB=$(python3 - <<'PY' +import json +last = json.load(open("target/.last-result.json"))["data"]["result"] +print(json.load(open(last))["data"].get("compileDatabase") or "compile_commands.json") +PY +) +fi [ -d "$DIR" ] || { echo "FAIL: $DIR does not exist -- this guard has gone stale" >&2; exit 1; } @@ -68,8 +81,8 @@ FAIL: $CDB does not exist. compile database that names every imported module's BMI explicitly. Produce one first: mcpp build --toolchain llvm@23.1.3 - (any installed LLVM row works; the database is written at the project - root regardless of the row's exact version). + (any installed LLVM row works; the database is read from the project + root, or from the configuration target/.last-result.json names). EOF exit 1 fi @@ -148,7 +161,7 @@ trap 'rm -f "$OUT"' EXIT "$CLANG_TIDY" \ --checks='-*,readability-function-size' \ --config="{CheckOptions: {readability-function-size.LineThreshold: '$LINE_THRESHOLD'}}" \ - -p "$REPO_DIR" \ + -p "$(dirname "$CDB")" \ "${FILES[@]}" > "$OUT" 2>&1 rc=$? diff --git a/.github/tools/check_unicode_paths.sh b/.github/tools/check_unicode_paths.sh index 574e18ec6..0c97dc68c 100755 --- a/.github/tools/check_unicode_paths.sh +++ b/.github/tools/check_unicode_paths.sh @@ -87,7 +87,10 @@ for name in ascii "$CAFE" "$CJK"; do if [ "$name" != ascii ]; then ninja=$(find target -name build.ninja -newer mcpp.toml | head -1) check_bytes "$ninja" "$name" "$label" - check_bytes compile_commands.json "$name" "$label" + # The configuration's database: the root copy is written only + # when [build] compile_commands asks for one (2026.10.26.1+). + cdb=$(find target -name compile_commands.json -newer mcpp.toml | head -1) + check_bytes "$cdb" "$name" "$label" fi ok "$label" done diff --git a/.github/workflows/bootstrap-macos.yml b/.github/workflows/bootstrap-macos.yml index 689cc2414..2789f1920 100644 --- a/.github/workflows/bootstrap-macos.yml +++ b/.github/workflows/bootstrap-macos.yml @@ -17,7 +17,7 @@ jobs: # Dormant (workflow_dispatch only), but kept in step with the rest — # check_version_pins.sh holds it there. Floor: 0.4.69, below which the # index cannot resolve two packages that share a short name. - XLINGS_VERSION: '2026.10.24.1' + XLINGS_VERSION: '2026.10.26.1' steps: - uses: actions/checkout@v4 diff --git a/.github/workflows/ci-bootstrap.yml b/.github/workflows/ci-bootstrap.yml index 047009df2..e0832a93a 100644 --- a/.github/workflows/ci-bootstrap.yml +++ b/.github/workflows/ci-bootstrap.yml @@ -41,7 +41,7 @@ jobs: os: [ubuntu-24.04, macos-15, windows-2025] env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.10.24.1' + XLINGS_VERSION: '2026.10.26.1' steps: - uses: actions/checkout@v4 @@ -50,7 +50,7 @@ jobs: shell: bash run: | set -euo pipefail - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.10.24.1 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.10.26.1 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" echo "$HOME/.xlings/bin" >> "$GITHUB_PATH" @@ -124,7 +124,7 @@ jobs: shell: bash run: | set -euo pipefail - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.10.24.1 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.10.26.1 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" - name: Install the newest release and build the tree with it (mcpp build --strict) diff --git a/.github/workflows/ci-fresh-install.yml b/.github/workflows/ci-fresh-install.yml index ff16c09c8..eaffb49dc 100644 --- a/.github/workflows/ci-fresh-install.yml +++ b/.github/workflows/ci-fresh-install.yml @@ -161,7 +161,7 @@ jobs: env: XLINGS_NON_INTERACTIVE: '1' run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.10.24.1 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.10.26.1 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" - name: Install mcpp and config mirror @@ -324,7 +324,7 @@ jobs: - name: Install xlings + mcpp run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.10.24.1 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.10.26.1 # Deliberately NOT writing to $GITHUB_PATH here. On container # images that declare no PATH in their config (opensuse/ # tumbleweed), appending a single dir to GITHUB_PATH makes the @@ -473,7 +473,7 @@ jobs: # (older ones carry minos=15 and refuse to start). # v0.4.51+: in-process sha256 — this image has no sha256sum # binary, so pinned fetches failed before it. - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.10.24.1 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.10.26.1 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" - name: Install mcpp and config mirror diff --git a/.github/workflows/ci-linux-e2e.yml b/.github/workflows/ci-linux-e2e.yml index 8615bd0fa..93e1a342c 100644 --- a/.github/workflows/ci-linux-e2e.yml +++ b/.github/workflows/ci-linux-e2e.yml @@ -414,7 +414,7 @@ jobs: - name: Bootstrap xlings + released mcpp run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.10.24.1 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.10.26.1 export PATH="$HOME/.xlings/subos/current/bin:$PATH" xlings update xlings install mcpp -y -g diff --git a/.github/workflows/cross-build-test.yml b/.github/workflows/cross-build-test.yml index bee430e99..ea9f100c1 100644 --- a/.github/workflows/cross-build-test.yml +++ b/.github/workflows/cross-build-test.yml @@ -131,7 +131,7 @@ jobs: # release assets were uploaded in a broken state (records present, # blobs missing → 404 on GET); re-uploaded clean. The stale-INDEX # half is handled by the marker-clear below. - XLINGS_VERSION: '2026.10.24.1' + XLINGS_VERSION: '2026.10.26.1' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" bash "$GITHUB_WORKSPACE/.github/tools/fetch_release.sh" \ @@ -323,7 +323,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.10.24.1' + XLINGS_VERSION: '2026.10.26.1' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" bash "$GITHUB_WORKSPACE/.github/tools/fetch_release.sh" \ diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index f953a03a5..3a1cc10c6 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -104,7 +104,7 @@ jobs: # Pin xlings to a known-good version. The upstream install # script always grabs `latest` (no version override), so we # download + self-install manually to avoid broken releases. - XLINGS_VERSION: '2026.10.24.1' + XLINGS_VERSION: '2026.10.26.1' run: | # BY VERSION, NOT BY PRESENCE. A restored ~/.xlings holds the # xlings of an earlier release, and this binary is the one the @@ -332,7 +332,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.10.24.1' + XLINGS_VERSION: '2026.10.26.1' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" bash "$GITHUB_WORKSPACE/.github/tools/fetch_release.sh" \ @@ -403,7 +403,7 @@ jobs: # below are pinned to the same version as XLINGS_VERSION; they are # NOT interpolated from it, so check_version_pins.sh scans for them # explicitly (they were absent from the old lock-step comment). - XLA="xlings-2026.10.24.1-linux-aarch64.tar.gz" + XLA="xlings-2026.10.26.1-linux-aarch64.tar.gz" # NOT fetch_release.sh: this asset is OPTIONAL and the `if` is the # point — an arch with no prebuilt xlings must fall through quietly, # while the helper retries a 404 five times before giving up. The one @@ -412,9 +412,9 @@ jobs: # cover it. if curl -fsSL --retry 3 --retry-delay 2 --retry-all-errors \ --connect-timeout 20 --max-time 600 -o "/tmp/$XLA" \ - "https://github.com/openxlings/xlings/releases/download/v2026.10.24.1/$XLA"; then + "https://github.com/openxlings/xlings/releases/download/v2026.10.26.1/$XLA"; then tar -xzf "/tmp/$XLA" -C /tmp - XLBIN=$(find /tmp/xlings-2026.10.24.1-linux-aarch64 -path '*/bin/xlings' -type f | head -1) + XLBIN=$(find /tmp/xlings-2026.10.26.1-linux-aarch64 -path '*/bin/xlings' -type f | head -1) if [ -n "$XLBIN" ]; then mkdir -p "$STAGING/$WRAPPER/registry/bin" cp "$XLBIN" "$STAGING/$WRAPPER/registry/bin/xlings" @@ -492,7 +492,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.10.24.1' + XLINGS_VERSION: '2026.10.26.1' run: | # By version, not by presence (see the Linux job). have=$("$HOME/.xlings/subos/default/bin/xlings" --version 2>/dev/null | awk '{print $2}') @@ -682,7 +682,7 @@ jobs: shell: bash env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.10.24.1' + XLINGS_VERSION: '2026.10.26.1' run: | # Captured before the `cd` below, in POSIX form: this step never # returns to the workspace, and GITHUB_WORKSPACE is a backslash diff --git a/CHANGELOG.md b/CHANGELOG.md index 5e18eaa1c..fd383be0e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,95 @@ > Each `## []` section is that release's notes. Entries are written in English > from 2026.9.28.3 on; earlier entries remain as written. +## [2026.10.26.1] - 2026-10-26 + +### Added + +- **The build stream and the result record.** `mcpp build --message-format + json` writes NDJSON events (`plan`, `waiting`, `progress`, `diagnostic`, + `artifact`) and ends with the `mcpp.build` envelope. Every build writes + `target///result.json` (`running`, then `ok`, `failed` or + `cancelled`) and `target/.last-result.json`: the command, the caller + (`MCPP_CLIENT`), the artifacts with how `mcpp run` starts each one, and the + compilers' diagnostics with path and range. `mcpp run --print-launch` prints + the `mcpp.launch` envelope instead of starting the program. +- **`--deny ` and `MCPP_DENY_EFFECTS`.** A run stops before the first + effect its caller denied (`init-mcpp-home`, `write-project`, + `write-global-cache`, `network`, `exec-build-script`), names it, and exits + `5`; a denied `write-global-cache` uses the project's own cache instead. + `--offline` counts as `--deny network`. +- **Queries.** `mcpp search [keyword] --format json` (`mcpp.search`, every + package when no keyword is given), `mcpp index list --format json` + (`mcpp.index`) and `mcpp pack --list-formats` (`mcpp.pack.formats`). +- **`mcpp --home `** runs one command with `` as its home, as + `MCPP_HOME=` does. +- **`[build] compile_commands`** names where the project's compile database is + published, with `MCPP_COMPILE_COMMANDS` for one run and a `config.toml` + default. +- **A C++ runtime island is declared and verified.** `requires` takes a table, + `{ layer = "mcpp:c++-abi=libstdc++", visibility = "private" }` (for the + `c++-abi` layer only), and may be written under `[target.]`. A declared + island is linked with the foreign runtime after the program's own and checked + after the link (`[cxx-runtime-collision]`); a build with a C++ shared library + beside one is refused (`[cxx-runtime-island]`). +- **llvm with a libstdc++ from the graph.** A package that provides + `mcpp:c++-abi=libstdc++` (`gcc.libstdcxx`) gives a clang build its headers, + its `std` module and `-stdlib=libstdc++`. Build programs receive the resolved + `MCPP_CXX_STDLIB` and `MCPP_CXX_ABI_ROOT`. +- Colour on a pipe when `CLICOLOR_FORCE` or `FORCE_COLOR` asks for it; + `NO_COLOR` and `MCPP_NO_COLOR` still win. +- Docs: [52 — Driving mcpp from a Program or an Agent](docs/52-driving-mcpp-from-a-program.md), + [SPEC-011 concurrency](docs/specs/concurrency.md) and + [SPEC-012 human output](docs/specs/human-output.md). + +### Changed + +- **The project root's `compile_commands.json` is written only when + `[build] compile_commands` names it.** Each configuration's database stays in + its build directory. A copy an earlier mcpp left is no longer updated, and the + first build that finds it says so once. +- **A foreign C++ runtime is recognised by type.** A runtime library named on + the link line, or a shared library on it whose `DT_NEEDED` names one, replaces + the search for `stdc++` in the flags. One no package declares is verified as + an island: a build that passes proceeds with a warning (`undeclared-cxx-runtime`), + one that does not is refused. +- `abi:glibc` and `abi:cxxstdlib` read the resolved target side, as + `requires = ["mcpp:c-abi=glibc"]` and `["mcpp:c++-abi=…"]` do (the + compatibility reading `abi-capability`). +- mcpp's next-step lines read `help:` (they read `hint:`). +- A manifest's rejection is located at the key it names + (`mcpp.toml:6:11: error: [build].sources …`), and envelope diagnostics carry + that location as `path` and `range`. +- `cache list` reads each entry's size from its record; compiler probes and the + provider handshake are cached under the home, written by a run that plans + and read by every command (a query such as `self doctor` leaves the home as it + found it). Measured with xlings 2026.10.26.1: `emit build-database` 71 ms, + `toolchain list` 44 ms. +- A provisioning failure passes on xlings' diagnostic and no longer suggests + `xlings install`. +- xlings 2026.10.26.1 is bundled and is the vendored floor: an interface call + ends with its capability, `--home` selects one run's home, and a hook in a + project's store finds `xlings`. + +### Fixed + +- **Two builds of one configuration no longer break each other** (mcpp#839). + A build directory has a lock from the moment its configuration is known; the + second build waits, says whose run it waits for, and finds little left to do. + `mcpp run` releases it before starting the program. +- **A linked file is replaced whole.** A program started while a build links + never runs a partial file (POSIX: link to a temporary name and rename; + Windows: the old file is renamed aside). +- **`catch (const std::exception&)` matches across a second C++ runtime** + (mcpp#838): the foreign runtime is linked after the program's own, so the ABI + layer binds to one runtime. +- `target/.build_cache` and `mcpp.lock` are replaced atomically; a lock another + run wrote meanwhile is not overwritten (`[lockfile-concurrent]`). +- A build ended by SIGINT, SIGTERM or SIGHUP leaves its record `cancelled` and + exits `128 + N`. +- A libstdc++ `std` module source named `std.cc` is compiled as a module + (`-x c++-module`). + ## [2026.10.24.1] - 2026-10-24 ### Added diff --git a/README.md b/README.md index 57f8bd514..f72360789 100644 --- a/README.md +++ b/README.md @@ -272,8 +272,9 @@ import mcpplibs.cmdline; - Three-layer incremental optimization: front-end dirty check + per-file P1689 dyndep + BMI copy-if-different restat - Fingerprinted BMI cache: hashed by compiler/flags/standard library, shared across projects - Ninja backend: auto-generated build.ninja, parallel compilation -- `compile_commands.json` generated automatically (ready for clangd / ccls); use - `mcpp build --configure-only` to refresh it before ordinary sources compile +- `compile_commands.json` generated for each configuration (its location in + `target/.last-result.json`; a project-root copy with `[build] compile_commands`); + `mcpp build --configure-only` refreshes it before ordinary sources compile - First-class C support: `.c` files auto-detected, mixed C/C++ projects - User-defined cflags / cxxflags / ldflags / c_standard @@ -507,7 +508,7 @@ a command in front of a reader, to the chapter that owns it. | `2x` toolchains and targets | [20 Toolchain Management](docs/20-toolchains.md) · [21 The Target Triple](docs/21-the-target-triple.md) · [24 Cross-Compilation Over openkal](docs/24-openkal-cross.md) | | `3x` extending mcpp | [30 Build Programs: `build.mcpp`](docs/30-build-mcpp.md) · [31 Authoring a Rule Package](docs/31-authoring-a-rule-package.md) · [34 Authoring a Board-Support Package](docs/34-authoring-a-bsp.md) | | `4x` devices and accelerators | [40 Bare-Metal and Freestanding Targets](docs/40-baremetal.md) · [41 Reaching a Device](docs/41-devices.md) · [42 Heterogeneous Builds](docs/42-heterogeneous-builds.md) | -| `5x` contracts for programs | [50 Machine-Readable Output](docs/50-machine-output.md) · [51 Supported Versions and Compatibility](docs/51-supported-versions.md) · [the specifications](docs/specs/README.md) | +| `5x` contracts for programs | [50 Machine-Readable Output](docs/50-machine-output.md) · [51 Supported Versions and Compatibility](docs/51-supported-versions.md) · [52 Driving mcpp from a Program or an Agent](docs/52-driving-mcpp-from-a-program.md) · [the specifications](docs/specs/README.md) | | `9x` mcpp itself | [90 Building from Source and Contributing](docs/90-build-from-source.md) · [92 Releasing mcpp](docs/92-release.md) | Every directory under [`examples/`](examples/) is a project that builds, and diff --git a/README.zh-CN.md b/README.zh-CN.md index 93ca76c9f..063c29928 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -246,7 +246,7 @@ import mcpplibs.cmdline; - 三层增量优化：前端脏检查 + 逐文件 P1689 dyndep + BMI copy-if-different restat - 带指纹的 BMI 缓存：按编译器、编译标志与标准库计算哈希，跨项目共享 - Ninja 后端：自动生成 build.ninja，并行编译 -- 自动生成 `compile_commands.json`（可直接供 clangd / ccls 使用）；`mcpp build --configure-only` 可在编译普通源码之前刷新它 +- 为每个配置生成 `compile_commands.json`（位置记录在 `target/.last-result.json`；`[build] compile_commands` 可在工程根目录放一份副本）；`mcpp build --configure-only` 可在编译普通源码之前刷新它 - C 语言一等支持：自动识别 `.c` 文件，支持 C/C++ 混合项目 - 用户自定义 cflags / cxxflags / ldflags / c_standard @@ -432,7 +432,7 @@ mcpp 的身份模型有两条正交的轴：**工具链**是 `family@version`（ | `2x` 工具链与目标 | [20 工具链管理](docs/zh/20-toolchains.md) · [21 目标三元组](docs/zh/21-the-target-triple.md) · [24 基于 openkal 的交叉构建](docs/zh/24-openkal-cross.md) | | `3x` 扩展 mcpp | [30 构建程序：`build.mcpp`](docs/zh/30-build-mcpp.md) · [31 编写规则包](docs/zh/31-authoring-a-rule-package.md) · [34 编写板级支持包](docs/zh/34-authoring-a-bsp.md) | | `4x` 设备与加速器 | [40 裸机与 freestanding 目标](docs/zh/40-baremetal.md) · [41 在设备上运行](docs/zh/41-devices.md) · [42 异构硬件构建](docs/zh/42-heterogeneous-builds.md) | -| `5x` 面向程序的契约 | [50 机器可读输出](docs/zh/50-machine-output.md) · [51 受支持的版本与兼容性](docs/zh/51-supported-versions.md) · [规范](docs/specs/README.md) | +| `5x` 面向程序的契约 | [50 机器可读输出](docs/zh/50-machine-output.md) · [51 受支持的版本与兼容性](docs/zh/51-supported-versions.md) · [52 从程序或 agent 驱动 mcpp](docs/zh/52-driving-mcpp-from-a-program.md) · [规范](docs/specs/README.md) | | `9x` mcpp 自身 | [90 从源码构建与参与贡献](docs/zh/90-build-from-source.md) · [92 发布 mcpp](docs/zh/92-release.md) | [`examples/`](examples/) 下的每个目录都是一个可以构建的项目，[03 —— 示例项目](docs/zh/03-examples.md) 说明每个示例演示什么。任意命令的完整选项可通过 `mcpp --help` 查看。 diff --git a/docs/01-getting-started.md b/docs/01-getting-started.md index 6b549dd90..cdf0c62d0 100644 --- a/docs/01-getting-started.md +++ b/docs/01-getting-started.md @@ -163,14 +163,18 @@ Run it only in a trusted workspace. The process exit code and the resulting `compile_commands.json` are the stable integration contract; stdout remains human-readable. -`compile_commands.json` at the project root is a copy of the current -configuration's own database, written at +Each configuration's database is written at `target///compile_commands.json` — the directory the toolchain, target, profile and features together name, and the directory the compiler actually runs in. Every command that plans in that configuration (`build`, `test`, `run`, `--configure-only`) merges its fresh plan into that one file, so a unit `mcpp test` added stays listed after a plain `mcpp -build`. The root copy is never merged: it is replaced whole with the +build`. Its location is in the result record (`compileDatabase` in +`target/.last-result.json`'s record, [50](50-machine-output.md) §9), which is +where an editor finds it. A copy at the project root is written only when +`[build] compile_commands` names one (2026.10.26.1+; +[04](04-mcpp-toml.md#compile_commands--where-the-projects-compile-database-is-published-mcpp-202610261)). +That copy is never merged: it is replaced whole with the configuration's file, and left untouched when the two already agree, so switching `--toolchain` or `--profile` switches the whole root file and switching back restores that configuration's entries, test units included. A diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index e14cef9c8..ea3ade7de 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -887,6 +887,31 @@ machine running mcpp: `mcpp build --target aarch64-macos` honors this field (and the environment variable) the same way on Linux or Windows as it does on a Mac, and a non-macOS target never sees it. +### `compile_commands` — where the project's compile database is published (mcpp 2026.10.26.1+) + +```toml +[build] +compile_commands = "compile_commands.json" # a path relative to the project root +``` + +Every build writes its configuration's database, +`target///compile_commands.json`; its location is in the +result record (`compileDatabase`, [50](50-machine-output.md) §9) and in +`mcpp emit build-database`. A copy at the project root is written only when this +key names one. Until 2026.10.26.1 every build wrote `compile_commands.json` at +the root, and an IDE that found it there took the project over as a compilation +database, which is mcppls's job for an mcpp project. + +| source | value | +|---|---| +| `MCPP_COMPILE_COMMANDS` | a path, or `off` / `none`: this run only | +| `[build] compile_commands` | a path; `""` asks for no copy | +| `config.toml` `[build] compile_commands` | the machine's default for projects that state nothing | +| (nothing) | no copy | + +The first that is set decides. A copy an earlier mcpp left at the root is no +longer updated; the first build that finds it says so once, naming this key. + ### `[build]` keys described in other chapters | Key | Chapter | @@ -1091,7 +1116,7 @@ spells only part of Unicode. On either, some paths have no UTF-8 spelling: ```text warning: '/home/user/pkg/test/data' contains names that have no UTF-8 spelling impact: those files take no part in the build - hint: The name's bytes are not UTF-8, and build.ninja and compile_commands.json hold UTF-8 text. ... + help: The name's bytes are not UTF-8, and build.ninja and compile_commands.json hold UTF-8 text. ... ``` The reported path is the nearest ancestor that has a UTF-8 spelling. Names of diff --git a/docs/05-dependencies.md b/docs/05-dependencies.md index 698e34275..f2f6e0f40 100644 --- a/docs/05-dependencies.md +++ b/docs/05-dependencies.md @@ -219,7 +219,7 @@ comp = { path = "libs/comp" } ``` warning: mcpp.toml names 2 dependencies in namespace acme, and the manifests they reach declare huxdemo; the declared identity is used: acme.fw, acme.comp - hint: write them in a [dependencies.huxdemo] table, for example `fw = { path = "libs/fw" }` + help: write them in a [dependencies.huxdemo] table, for example `fw = { path = "libs/fw" }` ``` Two edges written `fw` and `huxdemo.fw` over one directory are therefore one diff --git a/docs/07-workspace.md b/docs/07-workspace.md index be3eac545..828d18893 100644 --- a/docs/07-workspace.md +++ b/docs/07-workspace.md @@ -408,7 +408,7 @@ warning: dependency `render` declares standard = "c++26", and this graph is built at c++23 impact: a C++ module graph has one standard, so the dependency's declaration is not applied and its sources are compiled at the graph's level - hint: raise the consumer's standard to "c++26", or declare it once for + help: raise the consumer's standard to "c++26", or declare it once for every member: [workspace.package] @@ -644,7 +644,8 @@ member that several members use is compiled once. build-database` plan as the build does, one plan per configuration with each member's tests, so a package the members share is described once per configuration. A command that planned several configurations writes the root - `compile_commands.json` once, as the union of their databases (2026.9.29.5+). + copy (when `[build] compile_commands` names one) once, as the union of their + databases (2026.9.29.5+). - **No-op builds.** A command repeated with nothing changed is answered by one check per configuration, without planning. - **Module names.** A module name is unique within one program, not within one @@ -701,8 +702,8 @@ myproject/ - A shared library placed in several product directories is one file with several names where the file system supports hard links; elsewhere it is copied. -- `compile_commands.json` at the workspace root covers every member that has - been built or configured. +- The workspace root's compile database, when `[build] compile_commands` names + one, covers every member that has been built or configured. - `mcpp.lock` at the workspace root records the resolution of every member. `mcpp build --workspace` writes the whole record; `mcpp build -p X` updates the entries of X's graph. diff --git a/docs/20-toolchains.md b/docs/20-toolchains.md index 3c4c6a954..db84919e4 100644 --- a/docs/20-toolchains.md +++ b/docs/20-toolchains.md @@ -1086,7 +1086,7 @@ build says so: ``` warning: the assembler for this build is the host's ('/usr/bin/nasm'), not the one this engine pins impact: two machines can assemble the same source with different assemblers, and the build records only this line - hint: run `xlings install nasm` so the pinned copy is used + help: run `xlings install nasm` so the pinned copy is used ``` **A host tool that reaches a build is not by itself the defect. A host tool diff --git a/docs/22-target-side.md b/docs/22-target-side.md index ceb917ae8..52525b8e9 100644 --- a/docs/22-target-side.md +++ b/docs/22-target-side.md @@ -311,6 +311,96 @@ toolchain values ([32 — Authoring a Payload](32-authoring-a-payload.md)). It must not build a different variant into the same store directory, because the first consumer would then decide the variant for every later one. +### `requires` as a table, and on a target (mcpp 2026.10.26.1+) + +An entry of `requires` may be a table: + +```toml +requires = [{ layer = "mcpp:c++-abi=libstdc++", visibility = "private" }] +``` + +A string is the table with `visibility = "public"`. `private` states that the +requirement holds for this package's own code and does not cross its boundary: +the package's C++ runtime lives behind a C interface (an island, below). It is +accepted for the `c++-abi` layer only; on any other layer the manifest is +refused with `requires-private-layer`, because a second C library in one image +is two C libraries, not an island. + +`requires` may also be written under a target selector, and the requirement set +is the union of `[package] requires` and every matching selector's, as for +`requires_abi`: + +```toml +[target.'cfg(os = "linux")'] +requires = [{ layer = "mcpp:c++-abi=libstdc++", visibility = "private" }] +``` + +An engine older than 2026.10.26.1 refuses the table form; a package that uses it +states the floor (`[build-dependencies.mcpp] version = ">= 2026.10.26.1"`, or +`mcpp = ">= 2026.10.26.1"` in an index descriptor). + +### One C++ runtime per process (mcpp 2026.10.26.1+) + +Two Itanium C++ runtimes — libstdc++, and libc++ with libc++abi — define the +same ABI names: `__cxa_*`, `__gxx_personality_v0`, the type information and +virtual tables of `std::exception` and its family, `operator new` and +`delete`. In one process's global symbol space they end up each serving half of +an exception (mcpp#838: thrown by one, caught by the other, and only +`catch (...)` matches), and `std::runtime_error` has a different layout in each. +No link line makes several images, two runtimes and exceptions that cross the +images reliable. So the engine holds a build to one of three modes: + +| mode | when | the engine's obligation | +|---|---|---| +| **U, unified** (default) | the process has one C++ runtime, the one supplier of `c++-abi` (the compiler's payload, a target base, or a package of the graph) | a `requires` the build does not satisfy is refused before compilation, with the remedy | +| **I, island** | a component's C++ runtime lives only behind its C interface, and a package declares it with `visibility = "private"` | every condition below is verified; a build that fails one is refused | +| **H, host decides** | a Python extension, a JNI library, a host's plug-in: the target base states the `c++-abi` | the package follows the base | + +An island stands when all of these hold: + +1. the build has one C++ image: no C++ shared library among the root's targets, + or the plan is refused with `[cxx-runtime-island]`; +2. one unwinder, `--unwindlib=libgcc`; +3. the foreign runtime comes after the build's own runtime on the link line — + the engine places it there; a rule package does not write it into `ldflags`; +4. the island's objects reference no layout-dependent member of the collision + set (the constructors, destructors and `what` of the `std::exception` + family other than `std::exception` itself); +5. the linked image imports no ABI-layer symbol at a `GLIBCXX_*` or `CXXABI_*` + version; +6. no type information the image references or defines is also exported by a + shared library built on the foreign runtime — a typed `catch` across two + runtimes never matches. + +Conditions 4 to 6 are checked after the link and reported as +`[cxx-runtime-collision]`, naming the symbols. A foreign runtime is recognised +by type, never by a substring of the link flags: a runtime library named on the +link line (`-l:libstdc++.so.6`), or a shared library on it whose `DT_NEEDED` +names one (`libstdc++.so.*`, `libc++.so.*`, `libc++abi.so.*`). One that no +package declares is verified as an island; when it stands, the build proceeds +with a warning (the compatibility reading `undeclared-cxx-runtime`, +[51](51-supported-versions.md)), and when it does not, the build is refused. + +A SYCL program is never an island: its units catch `sycl::exception`, which +`libsycl.so` defines and throws on libstdc++ (condition 6). It builds in mode U — +with gcc, or with llvm and a libstdc++ supplied by the graph: + +```toml +[dependencies] +gcc.libstdcxx = "16.1.0" # provides mcpp:c++-abi=libstdc++ + +[toolchain] +default = "llvm@22.1.8" # selects the compiler only +``` + +With a graph-supplied libstdc++ under clang, the engine compiles with +`-nostdinc++` and the supplier's headers, precompiles `std` from its +`std-module`, links with `-stdlib=libstdc++` (the compiler runtime stays +`compiler-rt`, the unwinder `libgcc`), and gives build programs +`MCPP_CXX_STDLIB=libstdc++` and `MCPP_CXX_ABI_ROOT`, the supplier's root, so a +rule package points a second compiler at the same library. The contract is +[SPEC-002 §7](specs/target-side.md). + ### The C Environment A `c-abi` Package Presents (mcpp 2026.9.18+) A traditional stack never has to say this: the compiler payload's target diff --git a/docs/30-build-mcpp.md b/docs/30-build-mcpp.md index 6dd899883..60007333d 100644 --- a/docs/30-build-mcpp.md +++ b/docs/30-build-mcpp.md @@ -1181,7 +1181,7 @@ more after an upgrade to the release that introduced them. ### Structured diagnostics: `mcpp::report` (protocol 14) A diagnostic stated with `mcpp::report` is rendered in the engine's form: the -message, then an `impact:` line and a `hint:` line. It reaches +message, then an `impact:` line and a `help:` line. It reaches `--message-format json` with the same fields, a `degraded` one fails the build under `--strict`, and a cached run reports it again. `mcpp::warning(text)` stays, and equals a diagnostic with a message only. @@ -1228,7 +1228,7 @@ feature, the error names the package and the feature: ```text error: build.mcpp imports 'mcpp.rules.qt' provided by: mcpp.plugins, feature "rules-qt" (not enabled) - hint: enable it on the dependency edge: mcpp.plugins = { ..., features = ["rules-qt"] } + help: enable it on the dependency edge: mcpp.plugins = { ..., features = ["rules-qt"] } ``` ### `import mcpp;` is the surface that evolves (mcpp 2026.8.5.1+) diff --git a/docs/42-heterogeneous-builds.md b/docs/42-heterogeneous-builds.md index f9ba54f27..599129c2b 100644 --- a/docs/42-heterogeneous-builds.md +++ b/docs/42-heterogeneous-builds.md @@ -676,28 +676,39 @@ Which shape to choose is a property of the program, not of mcpp: a seam that swaps an implementation wants link-time selection, and a program that ships to machines it has not seen wants run-time selection. -## One unwinder, when the link carries a second C++ runtime - -A lane whose device compiler is configured against libstdc++ puts libstdc++ on -the link line while the artifact links libc++ statically. Both are then in the -image, and mcpp's duplicate-symbol check reports what they share. - -For most of those symbols the consequence is that one implementation is called -instead of an interchangeable other. For the unwinder it is not. A static -archive contributes only the members something references, so the interposition -is partial by construction: measured on the SYCL lane, ten of libgcc's eighteen -`_Unwind_*` entry points came from the artifact and eight stayed in libgcc_s, -including the accessors a personality routine uses. libstdc++'s personality -then read an LLVM libunwind context through libgcc's accessors, found no -landing pad, and called `std::terminate` past a handler three frames up. The -program was correct until something threw. - -So when the link line names libstdc++ and the toolchain's own library is -libc++, mcpp links the unwinder from libgcc (`--unwindlib=libgcc`) instead of -the payload's `libunwind.a`, and hides the static archives' symbols with -`--exclude-libs`. libgcc_s is already in the process — libstdc++ needs it — so -this names a library rather than adding one, and the C++ runtime stays -embedded. A build with no second runtime on its line is unchanged. +## One C++ runtime, and the island that may differ (2026.10.26.1+) + +A lane whose device runtime is built on libstdc++ (`libsycl.so`) meets a program +whose own C++ library is the toolchain's. Two Itanium runtimes in one process +share the ABI names — `__cxa_throw`, the personality routine, the type +information of `std::exception` — and a typed `catch` across them does not +match (mcpp#838: `catch (const std::exception&)` missed an exception the +library threw, and only `catch (...)` caught it). So one process carries one +C++ runtime ([22 — Target Side](22-target-side.md#one-c-runtime-per-process-mcpp-202610261)): + +- **SYCL builds with one runtime, libstdc++.** `compat.sycl-runtime` declares + `requires = ["mcpp:c++-abi=libstdc++"]`. A gcc toolchain satisfies it, and so + does llvm with `gcc.libstdcxx` in the graph; the rule reads + `MCPP_CXX_ABI_ROOT` and points the device compiler at the same library. An + llvm build without a libstdc++ supplier is refused before anything compiles, + with both remedies. Measured: the device units catch `sycl::exception`, which + `libsycl.so` defines and throws on libstdc++, so a SYCL unit can never be an + island. +- **An island is declared and verified.** A component whose C++ runtime stays + behind its C interface declares `requires = [{ layer = + "mcpp:c++-abi=libstdc++", visibility = "private" }]`. The engine places the + foreign runtime after the program's own on the link line, links the unwinder + from libgcc (`--unwindlib=libgcc`), hides the static runtime's symbols with + `--exclude-libs`, and checks after the link that the ABI layer is bound to one + runtime and that no type is caught across the two. A static archive + contributes only the members something references, so a second unwinder is + partial by construction: measured on the SYCL lane before this rule, ten of + libgcc's eighteen `_Unwind_*` entry points came from the artefact and eight + stayed in libgcc_s, and a throw three frames below its handler ended in + `std::terminate`. +- **HIP has no second runtime.** The NVIDIA platform compiles `.hip` with the + project's own clang, and `libcudart_static.a` references no C++ ABI symbol + but `__cxa_atexit`. ## Two boundaries worth stating @@ -751,12 +762,11 @@ therefore contains no binaries. The AMD platform needs a ROCm runtime this ecosystem does not publish yet, and the rule refuses it by name rather than producing an object nothing can link. -**A SYCL build carries two C++ runtimes, and the seam is what makes that safe.** -`libsycl.so` is compiled against libstdc++ while an mcpp artifact links libc++, -so both are in the image and mcpp's duplicate-symbol check reports the -unwinder symbols they share. Nothing may cross the seam: a SYCL exception is -caught in the device translation unit and returned as a code, because the -runtime that threw it is not the one the caller would unwind with. +**A SYCL build carries one C++ runtime, libstdc++.** `libsycl.so` is compiled +against libstdc++, so the program is built on it too: with gcc, or with llvm and +`gcc.libstdcxx` in the graph. Until 2026.10.26.1 an llvm SYCL build linked libc++ +beside it, and a `sycl::exception` the runtime threw reached only `catch (...)`; +such a build is now refused, naming the two remedies. ## The platforms each lane reaches diff --git a/docs/50-machine-output.md b/docs/50-machine-output.md index 232df17ed..734179a26 100644 --- a/docs/50-machine-output.md +++ b/docs/50-machine-output.md @@ -83,6 +83,13 @@ file mcpp read. A diagnostic without a location omits `path` and `range` rather than sending zeros — `line: 0` would point at a position that does not exist. +A diagnostic whose message begins with a source location — `::: `, +`:: ` or `: error: `, the form of every located message mcpp +writes — carries that location as `path` and `range` (2026.10.26.1+). A +manifest's rejection is located at the key or the table it names: `[build].sources +must be …` points at the line `sources` is written on. A command that already +names the file in `path` keeps its spelling and gains the `range`. + `code` is always present. Parse `code`; never parse `message`. ## 3. Asking for machine output @@ -100,7 +107,9 @@ output with `--message-format json` instead, as `mcpp test` does. `mcpp pack --format` names the package format (`tar`, `dir`, `msi`), so its report is `mcpp pack --message-format json` (2026.9.16.1+). The shape follows the kind: `mcpp test` streams a record per test because tests finish over time, and -`mcpp pack` prints one envelope because a pack has one result. +`mcpp pack` prints one envelope because a pack has one result. `mcpp build +--message-format json` (2026.10.26.1+) streams events while it builds and ends +with one envelope (§9). ### Unsupported values and unknown options @@ -128,6 +137,7 @@ cannot produce. The full mapping across all of mcpp is | 0 | success | | 1 | the command ran and failed — see stderr, and `diagnostics` when stdout carries an envelope | | 2 | usage error — unknown option, unsupported value | +| 5 | the run stopped before an effect its caller denied (§10; 2026.10.26.1+) | | 70 | internal error (uncaught exception) | | 127 | unknown command | @@ -809,6 +819,173 @@ A tree staged by a mode that bundles nothing (`system`, `static`) carries no places libraries itself reads these lines rather than inferring the closure from the files under `lib/`. +## 9. The build stream and the result record *(mcpp 2026.10.26.1+)* + +Every chain that builds a project — a person at a terminal, an agent with a +shell, an IDE, a CI job — ends in a run of mcpp, so what one chain needs to +learn about another's build is written by mcpp, to disk, in one form. + +### `mcpp build --message-format json` — events, then `mcpp.build` + +``` +mcpp build [target] [-p ] [--release] --message-format json +``` + +Standard output is NDJSON. Each line but the last is an event: + +```jsonc +{"schemaVersion":1,"kind":"mcpp.event","event":"plan","configuration":{"triple":"x86_64-linux-gnu","profile":"dev","fingerprint":"c0641c458b2bde43","outputDir":"/…/target/x86_64-linux-gnu/c0641c458b2bde43"}} +{"schemaVersion":1,"kind":"mcpp.event","event":"progress","finished":12,"total":34} +{"schemaVersion":1,"kind":"mcpp.event","event":"artifact","artifact":{"name":"app","kind":"bin","path":"/…/bin/app","launch":{…}}} +``` + +| event | carries | +|---|---| +| `plan` | `configuration`: the configuration whose build directory the build has entered | +| `waiting` | `holder` (`pid`, `command`, `client`, `startedAt`) and `waitedMs`: another build holds this build directory, and this one waits for it (§11) | +| `progress` | `finished`, `total`: ninja's count of steps; a build may run more than one ninja pass, and each restarts the count | +| `diagnostic` | `diagnostic`: one diagnostic of a compiler, from the output of a failed step, in the shape of `diagnostics[]` of the record below | +| `artifact` | `artifact`: one product, as in `artifacts[]` of the record below | + +The last line is the `mcpp.build` envelope, on one line. Its `data` is the +result record this run wrote (below); a run that failed before it entered a +build directory omits `data`. Its `diagnostics` are mcpp's own errors, each +located where its message says (§2), with the code `MCPP_BUILD_FAILED`, or +`MCPP_EFFECT_DENIED` for a run that stopped before a denied effect. The +narration goes to standard error, as for every machine-readable mode. + +### The result record: `result.json` and `target/.last-result.json` + +A build writes `/result.json` (kind `mcpp.result`) when it starts +writing its build directory, with `"status": "running"`, and again when it +ends. `target/.last-result.json` (kind `mcpp.last-result`) names the record the +project's most recent build wrote. Both are replaced atomically. A build in a +terminal is therefore visible to an IDE that reads the record, and the reverse. + +| field of `data` | | +|---|---| +| `command` | the command line of the run | +| `client` | the value of `MCPP_CLIENT` the caller set (§12), or `""` | +| `pid` | the process that wrote the record | +| `startedAt`, `finishedAt` | milliseconds since the epoch; `finishedAt` is `null` while `running` and after `cancelled` | +| `status` | `running`, `ok`, `failed`, or `cancelled` | +| `configuration` | `triple`, `profile`, `fingerprint`, `outputDir` | +| `compileDatabase` | the configuration's `compile_commands.json`, or `null` | +| `artifacts` | `name`, `kind` (`bin`, `test`, `shared`, `static`), `path`, and for a runnable product `launch` | +| `diagnostics` | the compilers' diagnostics of a failed build: `severity`, `message`, `path`, `range` | + +`launch` is how `mcpp run` starts the product: `argv`, `cwd`, `env` and +`via`. `env` holds only the variables mcpp sets or rewrites for the program +(its loader's search path, the variables a runtime payload needs), never the +caller's other variables, so the record holds no secret of the caller's. +`via` is `null`, `"runner"` (a `[target.].runner` wraps the program) or +`"run_under"` (an execution host wraps it). A debugger, an IDE or a CI job that +starts the program from `launch` starts it as `mcpp run` would. + +A run ended by SIGINT, SIGTERM or SIGHUP leaves `cancelled` in both files +([exit codes](specs/exit-codes.md) §3.7). A record still `running` whose +`pid` no longer exists belongs to a run that was killed. + +`.last-result.json`: + +```json +{"schemaVersion":1,"kind":"mcpp.last-result","kindVersion":1, + "data":{"result":"/…/target/x86_64-linux-gnu/c0641c458b2bde43/result.json","status":"ok","finishedAt":1792964065954}} +``` + +### `mcpp.launch` — the way `mcpp run` starts the program + +``` +mcpp run [target] --print-launch +``` + +Builds the program as `mcpp run` does and prints the envelope instead of +starting it. `data` is `program`, `argv`, `cwd`, `env` and `via`, as +`launch` above. `--print-launch` takes no `--format`: on `mcpp run`, `--format` +names a distributable. + +## 10. Denying effects — `--deny` *(mcpp 2026.10.26.1+)* + +``` +mcpp --deny network,exec-build-script build +MCPP_DENY_EFFECTS=write-project mcpp build +``` + +`--deny` takes effects of §4 separated by commas; `MCPP_DENY_EFFECTS` takes +the same list, and a run denies the union of the two. `read-project` cannot be +denied; an unknown name is a usage error (exit 2). `--offline` denies `network` +and is kept as its spelling. + +A run stops before the first effect it would perform and its caller denied, +names the effect and what it was for on standard error (`effect denied: +write-project -- the build writes …`; a download a denied `network` would make +is reported as the offline refusal it always was), ends with `note: stopped +before a denied effect (…); exit status 5`, and exits **5**. A denied +`write-global-cache` does not stop a build: it uses the project's own cache +instead. A denied effect the run does not need is no obstacle — a build whose +recorded plan is valid runs no build program, so it succeeds under `--deny +exec-build-script`. The `effects` of an envelope keep reporting what the run +did. + +## 11. Concurrent runs + +One configuration's build directory is written by one run at a time. A second +build of the same configuration waits for the first, reports the holder on +standard error (`Waiting for the build directory, held by pid 1234 (mcpp +build, from vscode, 8s)`) and as a `waiting` event, and then finds little left +to do. Builds of different configurations, queries (`emit build-database`, +`cache list`, reading the records) and the programs `mcpp run` and `mcpp test` +start never wait. A linked product is replaced whole, so a program started +while a build links never runs a partial file. `mcpp.lock` is written +optimistically: a lock another run wrote while this one resolved is not +overwritten, and the run stops with `[lockfile-concurrent]`. The contract is +[docs/specs/concurrency.md](specs/concurrency.md). + +## 12. The caller's name, colour, and the home of one run *(mcpp 2026.10.26.1+)* + +- `MCPP_CLIENT=` names the caller (an IDE's backend, an agent's host) in + the build directory's lock holder and in the result record. It changes + nothing else. +- Colour: `MCPP_NO_COLOR=1` or a non-empty `NO_COLOR` turns colour off; else a + non-empty `CLICOLOR_FORCE` other than `0`, or a non-empty `FORCE_COLOR` other + than `0` and `false`, turns it on for a pipe; else a terminal has colour and a pipe has none. +- `mcpp --home ` runs one command with `` as its mcpp home, + as `MCPP_HOME=` does ([SPEC-010](specs/home-initialization.md)). + +## 13. Queries *(mcpp 2026.10.26.1+)* + +### `mcpp.search` — the packages the indexes offer + +``` +mcpp search [keyword] --format json +``` + +`data` is `keyword` (`""` when none was given, which lists every package) and +`packages`: one record per package, with `id` (`namespace:name`), `namespace`, +`name`, `index`, `description`, `licenses`, `repo` and `versions` (newest +first). Each index is read once; the command needs no project. + +### `mcpp.index` — the registries and the project's indexes + +``` +mcpp index list --format json +``` + +`data` is `registries` (`name`, `url`, `default`) and `indices` (the +project's `[indices]` entries, when run in a project). The packages are +`mcpp search`'s answer. + +### `mcpp.pack.formats` — the formats `mcpp pack --format` accepts + +``` +mcpp pack --list-formats +``` + +`data.formats` lists the engine's own formats (`tar`, `dir`; `builtin: true`) +and each format a package of the resolved graph provides (`builtin: false`, +with `providers`). The graph is planned in a directory of the mcpp home, so +nothing is written into the project. + ## Current limitations - **The exit-code table is scoped to the commands it names.** A code another diff --git a/docs/51-supported-versions.md b/docs/51-supported-versions.md index cf1ebf4d1..28ecf4f27 100644 --- a/docs/51-supported-versions.md +++ b/docs/51-supported-versions.md @@ -77,6 +77,8 @@ the rows of this table (mcpp 2026.10.11.1+). |---|---|---|---| | `workspace-position`: a workspace root's own `[toolchain]`, `[indices]`, `[profile.]`, `[target.]` scalars or `[xlings]` reaching members by position | 2026.10.10.1 | 2027.1 | `[workspace.toolchain]`, `[workspace.indices]`, `[workspace.profile.]`, `[workspace.target.]`, `[workspace.xlings]` | | `host-module-in-dependencies`: a `[dependencies]` entry asking for `host-module = true` or for a `rule_module` feature, read as a build dependency | 2026.10.11.1 | 2027.1 | the same entry in `[build-dependencies]` | +| `abi-capability`: an `abi:` runtime capability (`abi:glibc`, `abi:libc=`, `abi:cxxstdlib=`), read as a requirement on a target-side layer | 2026.10.26.1 | 2027.1 | `requires = ["mcpp:c-abi="]` or `["mcpp:c++-abi="]` | +| `undeclared-cxx-runtime`: a C++ runtime other than the build's own, named on its link line or needed by a library the line names, that no package declares | 2026.10.26.1 | 2027.1 | `requires = [{ layer = "mcpp:c++-abi=", visibility = "private" }]` in the package that needs it | | `plugin-imports-from-build-dependencies`: a build plugin's library importing a package it lists in its own `[build-dependencies]` with `host-module = true` | 2026.10.11.1 | 2027.1 | the package in the plugin's `[dependencies]` | ## Reproducing a build diff --git a/docs/52-driving-mcpp-from-a-program.md b/docs/52-driving-mcpp-from-a-program.md new file mode 100644 index 000000000..56c5bb987 --- /dev/null +++ b/docs/52-driving-mcpp-from-a-program.md @@ -0,0 +1,119 @@ +# 52 — Driving mcpp from a Program or an Agent + +**Reader:** the author of an agent's instructions, an editor's backend or a CI +job that runs mcpp in a shell and acts on what it finds. + +**The question this chapter answers:** which commands, outputs and statuses let +a program build, run and inspect an mcpp project with no terminal and no IDE. + +**Not here:** the shape of each document, which is +[50 — Machine-Readable Output](50-machine-output.md); the exit-code contract, +which is [SPEC-003](specs/exit-codes.md); and the editor integrations +themselves, which build on what this chapter describes. + +Every chain that builds a project ends in a run of mcpp: a person at a +terminal, an agent with a shell, an editor's build on save, a CI job. mcpp +therefore writes what one chain needs to know about another's build to disk, +and states its outcome in a status a shell can branch on. A program needs +nothing beyond a shell to use any of it. + +## 1. Build, and read the result + +```sh +mcpp build --message-format json > build.ndjson +``` + +Each line of standard output is one JSON record. The last is the `mcpp.build` +envelope, whose `data` is the result record: `status` (`ok`, `failed`), +`artifacts` (with how to start each runnable one) and `diagnostics` (each with +`path` and `range`). The lines before it are events (`plan`, `waiting`, +`progress`, `diagnostic`, `artifact`) for a reader that reports progress. The +narration — what a person would read — is on standard error. + +A program that did not start the build reads the same record from disk: + +```sh +cat target/.last-result.json # names the record and its status +cat "$(jq -r .data.result target/.last-result.json)" +``` + +A build in a terminal and a build in an editor write the same record, so each +can see the other's result. + +## 2. Run, or start the program yourself + +`mcpp run` builds and then replaces itself with the program, which owns +standard output; its status is the program's (`101` when the build failed). +A debugger or a harness that starts the program itself asks for the launch: + +```sh +mcpp run app --print-launch +``` + +The `mcpp.launch` envelope holds `argv`, `cwd` and `env` — only the variables +mcpp sets for the program — exactly as `mcpp run` would use them. + +## 3. Statuses + +| status | meaning | the program's response | +|---|---|---| +| `0` | done | read the record | +| `1` | the project or the environment failed | read `diagnostics`, fix, run again | +| `2` | usage error | correct the command line | +| `5` | the run stopped before an effect the caller denied | ask the caller for approval, then run again without the denial | +| `101` | `mcpp run` could not build the program | as `1` | +| `130`, `143` | ended by SIGINT, SIGTERM (POSIX) | the record says `cancelled`; the next build resumes | + +## 4. Limit what a run may do + +An agent's host, or an editor in an untrusted workspace, states which effects a +run may not have: + +```sh +mcpp --deny network,exec-build-script build +MCPP_DENY_EFFECTS=network mcpp build # the same, for every command of a session +``` + +The effects are `init-mcpp-home`, `write-project`, `write-global-cache`, +`network` and `exec-build-script`. A run that needs a denied effect stops +before it, names it on standard error and exits `5`; nothing was half done. A +run that does not need it is not affected: a project whose plan is recorded +builds under `--deny exec-build-script`. `mcpp --protocol-version` lists the +effects each command may have, before anything runs. + +## 5. Several runs at once + +Two builds of one configuration do not corrupt each other: the second waits for +the first and says whose run it waits for +(`Waiting for the build directory, held by pid 1234 (mcpp build, from my-agent, 8s)`). +Setting `MCPP_CLIENT` names the caller in that line and in the result record: + +```sh +MCPP_CLIENT=my-agent mcpp build +``` + +A program started by `mcpp run` holds no lock, so rebuilding while it runs is +safe; a linked file is replaced whole. Queries never wait. The contract is +[SPEC-011](specs/concurrency.md). + +## 6. Questions that need no build + +| question | command | +|---|---| +| the packages the indexes offer | `mcpp search [keyword] --format json` | +| the configured registries | `mcpp index list --format json` | +| the package formats `mcpp pack` accepts | `mcpp pack --list-formats` | +| the compile commands of every unit | `mcpp emit build-database` | +| the installed toolchains | `mcpp toolchain list --format json` | +| the locations mcpp uses | `mcpp self env --format json` | +| one run against another home | `mcpp --home ` | + +## 7. Reading the human output + +When a program has only the narration — a CI log, a terminal transcript — the +shape is fixed ([SPEC-012](specs/human-output.md)): a status line starts with a +right-aligned word (`Compiling`, `Finished`); an mcpp message starts with +`error:`, `warning:` or `note:`; a message about a file starts with +`:::`; and a next step is on its own line starting `help:`. +Colour is off in a pipe unless `CLICOLOR_FORCE` or `FORCE_COLOR` asks for it. +The words themselves carry no compatibility promise; the records above do. diff --git a/docs/README.md b/docs/README.md index fd963c6ce..6c74499d3 100644 --- a/docs/README.md +++ b/docs/README.md @@ -102,6 +102,7 @@ Within a part the order is a reading order, not an alphabet. - [50 — Machine-Readable Output](50-machine-output.md) - [51 — Supported Versions and Compatibility](51-supported-versions.md) +- [52 — Driving mcpp from a Program or an Agent](52-driving-mcpp-from-a-program.md) ### 9x — mcpp itself @@ -168,3 +169,5 @@ downstream tooling. - [SPEC-008 — A library's interface: public modules, the published closure, and one interface in both forms](specs/library-interface.md) - [SPEC-009 — The support and maintenance of toolchains: lines, defaults, provenance, moves and retirement](specs/toolchain-maintenance.md) - [SPEC-010 — Home selection and initialization](specs/home-initialization.md) + - [SPEC-011 — Concurrent runs: the build directory lock, whole outputs, optimistic writes, cancellation](specs/concurrency.md) + - [SPEC-012 — Output for people: two streams, two media, the kinds of line, colour](specs/human-output.md) diff --git a/docs/specs/README.md b/docs/specs/README.md index 063af0ad6..fb202045d 100644 --- a/docs/specs/README.md +++ b/docs/specs/README.md @@ -31,15 +31,17 @@ | 编号 | 标题 | 状态 | 最后修改 | 对应实现 | |---|---|---|---|---| | [SPEC-001](package-identity.md) | 包身份(`package.namespace` / `package.name`)、`[dependencies]` 选择器与匹配机制 | 评审中 v1.1 | 2026-08-03 | mcpp >= 0.0.106 | -| [SPEC-002](target-side.md) | 目标侧模型与能力声明(`mcpp:` 保留命名空间、五层、三条规则、目标侧执行) | 评审中 v1.2 | 2026-10-23 | mcpp >= 2026.8.24.2;§2.4 与 §6 mcpp >= 2026.10.24.1 | -| [SPEC-003](exit-codes.md) | 退出码契约(分类、语义、稳定性承诺) | 评审中 v1.0 | 2026-09-01 | mcpp >= 2026.9.1.1 | +| [SPEC-002](target-side.md) | 目标侧模型与能力声明(`mcpp:` 保留命名空间、五层、三条规则、目标侧执行、一个进程一套 C++ 运行时) | 评审中 v1.3 | 2026-10-26 | mcpp >= 2026.8.24.2;§2.4 与 §6 mcpp >= 2026.10.24.1;§2.2 的对象形式与 §7 mcpp >= 2026.10.26.1 | +| [SPEC-003](exit-codes.md) | 退出码契约(分类、语义、稳定性承诺) | 评审中 v1.2 | 2026-10-26 | mcpp >= 2026.9.1.1;`5` 与信号 mcpp >= 2026.10.26.1 | | [SPEC-004](manifest-semantics.md) | `mcpp.toml` 的平面划分、条件化形状、解析轴与命名规约 | 草案 v1.17 | 2026-10-23 | 条件化形状 mcpp >= 2026.8.29.1;目标轴 mcpp >= 2026.9.6.4;`linkage` 默认值 mcpp >= 2026.9.15.2;链接 flag 的词读法 mcpp >= 2026.9.26.2;条件化的 `dialect_cxxflags` 与 `-p` 的包身份 mcpp >= 2026.9.28.1;条件表按具体程度生效 mcpp >= 2026.9.28.2;平台前缀规约 mcpp >= 2026.10.5.1;根包的继承、根位置的 profile、链接 flag 的作用域、链接期优化与导出发现 mcpp >= 2026.10.5.2;架构段的 `x86` 即 `i686` mcpp >= 2026.10.5.3;前缀即语义与工作空间层 mcpp >= 2026.10.10.1;依赖的库与配置(§12) mcpp >= 2026.10.11.1;值的四档层次与 `run_under` mcpp >= 2026.10.24.1 | | [SPEC-005](build-database.md) | 构建数据库:`mcpp emit build-database` 的内容、取值规则与不写工程目录的保证 | 评审中 v1.8 | 2026-10-11 | mcpp >= 2026.9.15.1;v1.3 条款 mcpp >= 2026.9.26.2;v1.4 条款 mcpp >= 2026.9.27.1;v1.5 条款 mcpp >= 2026.9.28.1;v1.6 条款 mcpp >= 2026.9.29.5;v1.7 条款 mcpp >= 2026.9.30.2;v1.8 条款 mcpp >= 2026.10.11.1 | | [SPEC-006](toolchain-management.md) | 工具链管理:身份、来源、选择与载荷契约 | 草案 v0.10 | 2026-10-23 | 逐条标注;已实现条款 mcpp >= 2026.9.24.1;§3.7 mcpp >= 2026.9.28.1;§3.7.1 mcpp >= 2026.9.28.2;§2.2.1 与 §3.3 的非缺省来源 mcpp >= 2026.10.1.3;§3.8 mcpp >= 2026.10.8.1;§3.9 至 §3.11 mcpp >= 2026.10.24.1 | | [SPEC-007](build-plugins.md) | 构建插件:配置、施工与校验的分工,运行时与规划期的义务 | 草案 v0.9 | 2026-10-11 | 逐条标注;mcpp >= 2026.9.26.2;v0.3 条款 mcpp >= 2026.9.27.1;v0.4 条款 mcpp >= 2026.9.28.1;v0.5 条款 mcpp >= 2026.9.28.2;v0.6(§9)mcpp >= 2026.9.28.3;R3.9、R3.10 mcpp >= 2026.10.10.1;§10 mcpp >= 2026.10.11.1 | | [SPEC-008](library-interface.md) | 库的接口:公开模块、发布闭包与两种形态的一致 | 草案 v0.2 | 2026-10-11 | 第一阶段(只警告)mcpp >= 2026.9.28.3;v0.2 mcpp >= 2026.10.11.1 | | [SPEC-009](toolchain-maintenance.md) | 工具链的支持与维护:版本线、默认值、来源、移动与退役 | 草案 v0.6 | 2026-10-09 | 逐条标注;本版只有规范,多数条款未实现;§6.4 mcpp >= 2026.10.8.1 | -| [SPEC-010](home-initialization.md) | home 选择、单跳入口、初始化与网络边界 | 草案 v0.1 | 2026-10-19 | mcpp >= 2026.10.19.1；网络能力逐条标注 | +| [SPEC-010](home-initialization.md) | home 选择、单跳入口、初始化、网络边界与单次运行的 `--home` | 草案 v0.2 | 2026-10-26 | mcpp >= 2026.10.19.1；网络能力逐条标注；§6 mcpp >= 2026.10.26.1、xlings >= 2026.10.26.1 | +| [SPEC-011](concurrency.md) | 并发运行：三段、构建目录锁、原子发布、`mcpp.lock` 的乐观写入、取消 | 评审中 v1.0 | 2026-10-26 | mcpp >= 2026.10.26.1 | +| [SPEC-012](human-output.md) | 面向人的输出：两条流、两种介质、行的种类、颜色 | 评审中 v1.0 | 2026-10-26 | mcpp >= 2026.10.26.1 | ## 文档约定 diff --git a/docs/specs/concurrency.md b/docs/specs/concurrency.md new file mode 100644 index 000000000..59b3c65ef --- /dev/null +++ b/docs/specs/concurrency.md @@ -0,0 +1,101 @@ +# SPEC-011: 并发运行 + +| 项 | 值 | +|---|---| +| 规范编号 | SPEC-011 | +| 标题 | 同一工程上的并发运行:构建目录锁、原子发布、乐观写入与取消 | +| 状态 | 评审中 v1.0 | +| 版本 | 1.0 | +| 最后修改 | 2026-10-26 | +| 对应实现 | mcpp >= 2026.10.26.1 | +| 相关设计文档 | `.agents/docs/2026-10-26-control-contract-runtime-and-ide-ecosystem-design.md` §3、§4、§5.4 | +| 相关 issue | #839 | +| 使用文档 | [50 — 机器可读输出](../zh/50-machine-output.md) §9、§11 | + +用语按 RFC 2119:**必须 / 禁止**为强制,**应当**为建议,**可以**为可选。 + +## 0. 这份规范存在的原因 + +一个工程同时被多条调用链驱动:终端里的人、只有 shell 的 agent、IDE 的保存时构建、 +CI。在 2026.10.24.1 上,同一配置的两次冷构建相隔 0.1–0.5 s 启动,5 轮中有 3 轮双方 +都失败(`failed to write/read compiled module`,#839)。全局一侧(索引刷新、BMI 缓存、 +工具链目录)已有细粒度保护,工程的构建目录是唯一没有跨进程保护的共享可变资源。本规范 +规定每一种共享资源的保护方式,以及读者可以依赖的保证。 + +## 1. 三段 + +一次构建分为三段: + +| 段 | 内容 | 写入 | 锁 | +|---|---|---|---| +| 规划 | 读清单、解析依赖与工具链(可联网)、目标侧、扫描,直到配置指纹确定 | 不写构建目录 | 无 | +| 修改 | 准备的收尾(动作、`resolution.json`、`build.ninja`、配置级编译数据库)、ninja、产物校验、配置记录与结果记录 | 构建目录 | 构建目录锁 | +| 执行 | `mcpp run` 启动程序;`mcpp test` 之后的程序运行由测试命令持锁(§2.3) | 无 | `run`:无 | + +配置目录 `target///` 由指纹决定,指纹在规划末尾才确定。规划段 +**禁止**写入构建目录。 + +## 2. 构建目录锁 已实现 + +1. 每个构建目录一把锁,文件为 `/.mcpp-build.lock`,旁边的 `.owner` 记录持有者: + `pid`、开始时间、命令行与调用方标签(`MCPP_CLIENT`)。 +2. 锁**必须**在修改段开始时获取(完整规划时在准备的收尾阶段之前,快速路径在接纳记录的 + 计划之前),持有到修改段结束。规划段**禁止**持有它。 +3. 后到者**必须**等待而不是失败。等待超过 300 ms 时,标准错误上出现一行 + `Waiting for the build directory, held by pid N (, from , Ns)`, + `--message-format json` 下另有 `waiting` 事件。取得锁后,ninja 只做前者未做完的事。 +4. 不同配置的构建使用不同的目录与不同的锁,**禁止**相互等待。 +5. 锁描述符**必须**带 close-on-exec;`mcpp run` **必须**在 exec 程序之前释放锁,程序 + 运行期间同配置的构建不被阻塞。`mcpp test` 持锁直到测试结束,因为它在同一构建目录中 + 运行刚链接的测试程序。 +6. 进程崩溃或被 SIGKILL 时由操作系统释放锁,不存在僵死锁。Windows 使用 `LockFileEx`, + 语义相同(实测见设计 E24)。 + +## 3. 原子发布 已实现 + +| 资源 | 规则 | +|---|---| +| 链接产物(可执行文件、共享库、测试程序) | POSIX 非 PE 目标:链接写 `.tmp`,再 `mv -f` 改名。Windows 宿主:`mcpp link-output` 先把旧产物改名为 `.old-`,再原地链接;运行中的程序保留旧文件,以后的链接清理不再被占用的 `.old-*`。POSIX 宿主上的 PE 目标:原地链接 | +| `result.json`、`target/.last-result.json`、`resolution.json`、配置级与根目录的 `compile_commands.json` | 写临时文件后改名替换 | +| `target/.build_cache`(配置记录) | 读改写在 `target/.build_cache.lock` 下进行,写临时文件后改名替换;格式不变 | +| `mcpp.lock` | 写临时文件后改名替换;乐观并发(§4) | + +读者(查询命令、IDE、`mcpp emit build-database`、读取记录的程序)**禁止**加锁,并且 +**必须**只会看到某一次完整写入的内容。链接产物在任何时刻要么是旧文件、要么是完整的 +新文件;在链接进行时反复启动程序,不会出现 `ENOEXEC`、`ETXTBSY` 或执行半个文件 +(设计 E23)。 + +## 4. `mcpp.lock` 的乐观写入 已实现 + +解析依赖可能联网,持锁会把一次慢的下载变成对整个工程的阻塞。因此: + +1. 规划开始读取 `mcpp.lock` 时,记录读到的内容(换行统一为 LF;文件不存在也是一种内容)。 +2. 写入前**必须**重新读取。当前内容与记录相同,或与本次将要写入的内容相同,则写入 + (后一种情况下无需改写)。 +3. 否则**禁止**覆盖,运行以 `[lockfile-concurrent]` 失败(退出 1),并以 + `help: run the command again` 给出补救:重跑即按当前的锁文件重新解析。 + +## 5. 取消 已实现 + +1. SIGINT、SIGTERM、SIGHUP 结束修改段中的运行时,mcpp **必须**先结束它启动的进程组, + 再写下 `cancelled` 的结果记录,然后按信号的默认动作结束(退出状态见 + [SPEC-003](exit-codes.md) §3.7)。记录的这一形态在修改段开始时预先生成,信号处理器 + 只调用异步信号安全的 `open`、`write`、`close`、`rename`。 +2. 锁由进程结束释放。构建目录保持一致:ninja 的日志只记录完成的步骤,被中断的步骤在 + 下一次构建中重做。 +3. SIGKILL 不经过处理器;`status` 为 `running` 而 `pid` 已不存在的记录,读者**应当** + 读作"被强制结束"。 + +## 6. 全局一侧 + +全局 `MCPP_HOME` 保持既有的细粒度保护:索引刷新使用非阻塞锁,BMI 缓存按条目加锁, +工具链按版本目录安装。编译器探测缓存(`/cache/probe`)与提供者握手缓存 +(`/cache/provider-handshake`)的条目以写临时文件后改名的方式发布,并发的写入者 +写出相同的内容。它们只由声明了 `write-global-cache` 的运行(规划或构建)写入,其他命令 +只读取:`self doctor`、`self env`、`toolchain list` 不改变 home。 + +## 7. 变更记录 + +| 版本 | 日期 | 变更 | +|---|---|---| +| 1.0 | 2026-10-26 | 首版:三段、构建目录锁、原子发布、`mcpp.lock` 的乐观写入、取消(#839) | diff --git a/docs/specs/exit-codes.md b/docs/specs/exit-codes.md index 4c7168c93..53b5edd4e 100644 --- a/docs/specs/exit-codes.md +++ b/docs/specs/exit-codes.md @@ -4,12 +4,12 @@ |---|---| | 规范编号 | SPEC-003 | | 标题 | mcpp 的进程退出码:分类、语义与稳定性承诺 | -| 状态 | 评审中 v1.1 | -| 版本 | 1.1 | -| 最后修改 | 2026-10-01 | -| 对应实现 | mcpp >= 2026.9.1.1 | -| 相关设计文档 | `.agents/docs/2026-08-08-machine-readable-output-protocol-design.md` §R4、`.agents/docs/2026-08-31-issue540-seven-audit-findings.md` §4 | -| 相关 issue | #379、#540 | +| 状态 | 评审中 v1.2 | +| 版本 | 1.2 | +| 最后修改 | 2026-10-26 | +| 对应实现 | mcpp >= 2026.9.1.1;`5` 与信号(§3.6、§3.7)mcpp >= 2026.10.26.1 | +| 相关设计文档 | `.agents/docs/2026-08-08-machine-readable-output-protocol-design.md` §R4、`.agents/docs/2026-08-31-issue540-seven-audit-findings.md` §4、`.agents/docs/2026-10-26-control-contract-runtime-and-ide-ecosystem-design.md` §5.4、§5.5(D23) | +| 相关 issue | #379、#540、#839 | ## 0. 这份规范存在的原因 @@ -48,9 +48,11 @@ usage / internal 一半(`2`、`70`、`127`),runtime 的一半 —— 也就是 | `1` | 运行期失败 | 命令跑了、请求合法、结果是失败 | 人类可读的原因走 stderr | | `2` | 用法错误 | 未知选项、不支持的选项值、缺少必需参数 | stderr | | `4` | 环境未就绪 | 全局配置加载 / 首次初始化失败(`$MCPP_HOME` 不可写、config.toml 损坏、引导 xlings 失败) | stderr | +| `5` | 副作用被拒绝 | 运行在调用方拒绝的副作用(`--deny`、`MCPP_DENY_EFFECTS`、`--offline`)之前停下;项目本身没有失败 | stderr 写明副作用与原因 | | `70` | 内部错误 | 未捕获异常。`EX_SOFTWARE` | stderr | | `101` | 构建失败(仅 `mcpp run`) | `mcpp run` 的规划或构建失败,程序没有启动。Cargo 对 `cargo run` 构建失败给出同一个值 | stderr | | `127` | 未知命令 | 第一个位置参数不是一个子命令 | stderr | +| `128 + N` | 被信号结束(POSIX) | SIGINT(`130`)、SIGTERM(`143`)、SIGHUP(`129`)结束了这次运行;状态由进程死于该信号得到,不是 mcpp 返回的值 | 结果记录写 `cancelled`(§3.7) | `4` 与 `1` 的分界是**谁需要被修**:`4` 说明 mcpp 自己的家还没有准备好,任何命令都会 撞上同一堵墙;`1` 说明这一次请求失败了,而 mcpp 是可用的。把二者合并会让「我的工程 @@ -95,9 +97,42 @@ issue;`1` 通常不是。 任何非零退出**必须**在 stderr 上留下至少一行说明。空 stderr 加非零退出是缺陷。 +### 3.6 `5`:调用方的约束,不是项目的失败 已实现 + +调用方用 `--deny <副作用,…>`、`MCPP_DENY_EFFECTS` 或 `--offline`(等价于拒绝 +`network`)约束一次运行可以产生的副作用。副作用的词汇与信封的 `effects` 相同 +(`docs/50-machine-output.md` §4):`init-mcpp-home`、`write-project`、 +`write-global-cache`、`network`、`exec-build-script`;`read-project` 不可拒绝。未知的 +副作用名是用法错误,返回 `2`。 + +运行在第一个被拒绝的副作用**之前**停下,stderr 写明副作用与它要做的事(例如 +`network: install xim:gcc@16.1.0`),退出码为 `5`。被拒绝的 `write-global-cache` 不 +停下运行:构建改用工程本地的缓存,运行照常完成。 + +`5` 与 `1` 的分界是**谁的决定挡住了运行**:`5` 说明调用方的约束挡住了一件运行需要做 +的事,去掉约束(或由调用方的宿主审批)后重跑即可;`1` 说明项目或环境本身失败。只有 +shell 的 agent 依靠这一个数就能区分两者。 + +### 3.7 信号:状态由信号得到,记录由 mcpp 留下 已实现 + +POSIX 上,SIGINT、SIGTERM、SIGHUP 结束 mcpp 时,mcpp 先结束它启动的进程组(ninja、 +钩子),再按信号的默认动作死去,因此调用方读到的状态是 `128 + N`:`130`、`143`、 +`129`。这些值不是 mcpp 返回的,§3.1 的稳定性承诺对它们的含义是"被该信号结束"。 + +正在写构建目录的运行(从配置目录确定到构建结束)被信号结束时,结果记录 +`result.json` 与 `target/.last-result.json` 的 `status` 为 `cancelled` +(`docs/50-machine-output.md` §9)。记录的这一形态在运行开始时预先写好,由信号处理器用 +异步信号安全的调用落盘。SIGKILL 不经过处理器:一份 `status` 仍为 `running`、而 `pid` +已不存在的记录,说明那次运行被强制结束。构建目录在两种情况下都保持一致,下一次构建 +从中断处继续。 + +Windows 上,Ctrl-C 与 Ctrl-Break 由控制台处理器结束受管的作业、写下同样的 +`cancelled` 记录,然后交给系统的默认处理;进程的状态是系统给出的 +`STATUS_CONTROL_C_EXIT`(`0xC000013A`)。 + ## 4. 当前实现与本规范的差异 -无。§2 的码是 `src/` 中出现的全部进程退出码(2026-09-01 核对时是六个;1.1 起增加 `101`,见 `execute.cppm` 的 `kRunBuildFailed`)。 +无。§2 的码是 `src/` 中出现的全部进程退出码(2026-09-01 核对时是六个;1.1 起增加 `101`,见 `execute.cppm` 的 `kRunBuildFailed`;1.2 起增加 `5`,见 `cli.cppm` 末尾按 `env::effect_refused()` 的映射)。`128 + N` 不是 mcpp 返回的值,不在这张核对表中。 2026-09-01 穷举核对,记录方法与读数,因为「我数了一遍」不是判据: @@ -124,5 +159,6 @@ issue;`1` 通常不是。 | 版本 | 日期 | 变更 | |---|---|---| +| 1.2 | 2026-10-26 | 新增 `5`:运行在调用方拒绝的副作用之前停下(§3.6)。新增 `128 + N`:被信号结束,结果记录为 `cancelled`;Windows 为 `STATUS_CONTROL_C_EXIT`(§3.7)。 | | 1.1 | 2026-10-01 | 新增 `101`:`mcpp run` 的规划或构建失败(此前为 `1` 或 `2`,与返回同一个值的程序无法区分)。 | | 1.0 | 2026-09-01 | 首版。落地 2026-08-08 协议设计文档 §R4 指派而未写的契约;补上 `1` 与 `4`,并说明为什么 `4` 不属于 `docs/11` 那张按信封命令划定的表(#540)。 | diff --git a/docs/specs/home-initialization.md b/docs/specs/home-initialization.md index 621773f21..9e3b440f3 100644 --- a/docs/specs/home-initialization.md +++ b/docs/specs/home-initialization.md @@ -4,11 +4,11 @@ |---|---| | 规范编号 | SPEC-010 | | 标题 | home 的入口、实际目录、初始化阶段与网络边界 | -| 状态 | 草案 v0.1 | -| 最后修改 | 2026-10-19 | -| 对应实现 | mcpp >= 2026.10.19.1；网络预算依赖提供者能力，逐条标注 | -| 相关设计文档 | `.agents/docs/2026-10-19-first-run-home-and-network-design.md` | -| 相关 issue | mcpp#822；openxlings/xlings#683 | +| 状态 | 草案 v0.2 | +| 最后修改 | 2026-10-26 | +| 对应实现 | mcpp >= 2026.10.19.1；网络预算依赖提供者能力，逐条标注；§6 mcpp >= 2026.10.26.1、xlings >= 2026.10.26.1 | +| 相关设计文档 | `.agents/docs/2026-10-19-first-run-home-and-network-design.md`、`.agents/docs/2026-10-26-control-contract-runtime-and-ide-ecosystem-design.md` §9.1、§10.3（D18） | +| 相关 issue | mcpp#822；openxlings/xlings#683、openxlings/xlings#707 | | 使用文档 | [25 — home、索引与存储](../zh/25-home-indices-and-store.md) | 本规范定义普通用户 home 的选择、保存与初始化边界。系统 home 的管理员权限和共享策略继续按使用文档执行。 @@ -95,8 +95,25 @@ heartbeat 或重复字节值**禁止**更新下载进展时间。小型索引请 提供者缺失或不支持诊断时**必须**报告 unavailable/unsupported，继续本地检查， **禁止**为了 doctor 下载提供者。旧提供者不能满足新下载预算时，应按能力握手选择兼容的随附版本。 +## 6. 单次运行的 `--home` 与生态一致性 已实现 + +1. `mcpp --home ` **必须**与 `MCPP_HOME= mcpp ` 等价：`` + 是这一次运行的原始 home，其 `home.toml` 按 §1 第 2 条读取一跳。选择只对这一次运行生效， + **禁止**写入任何 locator。 +2. `--home` **可以**写在任何子命令之前或之后。与 `self init --home` 同时出现时，两个值 + 指向同一目录则照常执行；指向不同目录**必须**以用法错误（退出 2）拒绝，**禁止**取其一。 +3. xlings 采用相同的模型：全局 `--home ` 是单次运行的 `XLINGS_HOME`；`self init + --home ` 初始化 `` 并在原始 xlings home 中写入入口文件 `home.json` + （`{"home": ""}`），之后的运行跟随它一跳；`self init --force --home ` 授权改写 + 已有入口；以原始目录为目标时删除入口。入口无法读取或目标不可访问时**必须**显式报错并给出修复命令，**禁止**静默 + 改用原始 home。 +4. 两个 home 互相独立：mcpp 的 home 选择**禁止**改变 xlings 的 home，反之亦然。mcpp 通过 + 提供者调用 xlings 时，按 mcpp home 中记录的提供者位置调用，与用户自己的 xlings home + 选择无关。 + ## 变更记录 | 版本 | 日期 | 变更 | |---|---|---| +| v0.2 | 2026-10-26 | 新增 §6：mcpp 与 xlings 的单次运行 `--home`、与 `self init --home` 的组合规则、xlings 的入口文件 | | v0.1 | 2026-10-19 | 定义两阶段单跳 home、非删除切换、Windows 选择与初始化网络边界 | diff --git a/docs/specs/human-output.md b/docs/specs/human-output.md new file mode 100644 index 000000000..1480a9ec8 --- /dev/null +++ b/docs/specs/human-output.md @@ -0,0 +1,73 @@ +# SPEC-012: 面向人的输出 + +| 项 | 值 | +|---|---| +| 规范编号 | SPEC-012 | +| 标题 | 面向人的输出的形态:两条流、两种介质、行的种类、颜色 | +| 状态 | 评审中 v1.0 | +| 版本 | 1.0 | +| 最后修改 | 2026-10-26 | +| 对应实现 | mcpp >= 2026.10.26.1 | +| 相关设计文档 | `.agents/docs/2026-09-29-build-progress-display-design.md`、`.agents/docs/2026-09-30-build-output-refinement-design.md`、`.agents/docs/2026-10-26-control-contract-runtime-and-ide-ecosystem-design.md` §5.7、§5.9 | +| 使用文档 | [50 — 机器可读输出](../zh/50-machine-output.md) §3 | + +用语按 RFC 2119:**必须 / 禁止**为强制,**应当**为建议,**可以**为可选。 + +## 0. 范围 + +本规范约束面向人的输出的**形态**:哪一条流、什么介质、哪几种行、何时有颜色。它不约束 +逐字内容;面向人的输出没有兼容承诺,程序读取的是 +[docs/50](../zh/50-machine-output.md) 的信封与记录。agent 与 CI 读到的是管道中的输出, +因此管道中的形态与终端中的形态同等重要。 + +## 1. 两条流 已实现 + +标准输出携带命令的**结果**(程序的输出、文档、列表、`mcpp test` 的报告);标准错误携带 +**叙述**(状态行、进度、`warning:`、`error:`、`note:`、`tip:`、编译器诊断)。划分规则与 +例外见 docs/50 §3。`--quiet` 去掉叙述,保留警告、错误与诊断。 + +## 2. 两种介质 已实现 + +1. **终端介质**:标准错误是能移动光标的终端时,构建在输出下方保留一行状态行,只有这一行 + 被原地重绘;已经写出的行**禁止**被改写、重新着色或移动。 +2. **日志介质**:管道、文件、`TERM=dumb`、不支持虚拟终端序列的控制台。**禁止**写出任何 + 光标移动或回车重绘序列;行在内容确定后写出一次。构建静默 60 s 后追加一行心跳 + (`Building <完成>/<总数> · <时钟>`),之后每 60 s 至多一行。 + +## 3. 行的种类 已实现 + +| 种类 | 形式 | 用途 | +|---|---|---| +| 状态行 | 右对齐到第 12 列的一个词,后接主体:` Compiling app v0.1.0 (.)` | 命令正在做或已经做完的事 | +| 消息行 | `error: …`、`warning: …`、`note: …`、`tip: …` | mcpp 自己的判断 | +| 续行 | 缩进到消息正文的列 | 消息的细节、列表 | +| 补救行 | `help: …` | 读者下一步可以执行的命令或要改的键 | +| 诊断 | 编译器、链接器的原文 | 内容本身 | + +1. 状态行的词:进行中的工作用现在分词(`Resolving`、`Compiling`、`Running`、`Waiting`、 + `Packing`、`Installing`),完成的工作用过去分词(`Finished`、`Packed`、`Installed`), + 报告的事实用名词标签(`Home`、`Default`)。构建的主线固定为 + `Resolving` → `Compiling` → `Finished`,`run` 另有 `Running`。 +2. 一条错误存在可执行的下一步时,**必须**以一行 `help:` 给出;下一步是一条命令时,写出 + 完整的命令。外部工具(xlings)给出的补救按原样转述。 +3. 带位置的消息**必须**以 `::: ` 或 `: ` 开头,与编译器诊断同形, + 编辑器与终端可以直接跳转;信封中的诊断由此得到 `path` 与 `range`(docs/50 §2)。 +4. 编译器诊断**必须**原样完整输出,mcpp **禁止**截断或改写;mcpp 自己的列表**应当** + 有上限,并说明省略了多少(`… and N more`)。 + +## 4. 颜色 已实现 + +优先级从高到低: + +1. `MCPP_NO_COLOR=1` 或非空的 `NO_COLOR`:关闭。 +2. `CLICOLOR_FORCE` 非空且不为 `0`,或 `FORCE_COLOR` 非空且不为 `0`、`false`:开启, + 管道同样开启。状态行仍只在终端介质中绘制。 +3. 否则:终端开启,管道关闭。 + +颜色只修饰状态行的词、消息行的前缀与强调,**禁止**承载其他地方没有的信息。 + +## 5. 变更记录 + +| 版本 | 日期 | 变更 | +|---|---|---| +| 1.0 | 2026-10-26 | 首版:两条流、两种介质、行的种类(`help:` 统一补救行)、颜色的优先级 | diff --git a/docs/specs/target-side.md b/docs/specs/target-side.md index 7b3933055..e6f57a160 100644 --- a/docs/specs/target-side.md +++ b/docs/specs/target-side.md @@ -4,10 +4,10 @@ |---|---| | 规范编号 | SPEC-002 | | 标题 | 目标侧模型与能力声明(`mcpp:` 保留命名空间) | -| 状态 | 评审中 v1.2 | -| 最后修改 | 2026-10-23 | -| 对应实现 | mcpp >= 2026.8.24.2 | -| 相关设计文档 | `.agents/docs/2026-08-24-target-side-design.md`、`.agents/docs/2026-10-23-target-base-and-execution-host-design.md` | +| 状态 | 评审中 v1.3 | +| 最后修改 | 2026-10-26 | +| 对应实现 | mcpp >= 2026.8.24.2;§2.2 的对象形式与 §7 mcpp >= 2026.10.26.1 | +| 相关设计文档 | `.agents/docs/2026-08-24-target-side-design.md`、`.agents/docs/2026-10-23-target-base-and-execution-host-design.md`、`.agents/docs/2026-10-26-control-contract-runtime-and-ide-ecosystem-design.md` §6 | | 使用文档 | [docs/22 - 目标侧](../zh/22-target-side.md) | 本规范定义一次构建的目标侧由哪些层构成、每一层可以由谁供给、 @@ -74,6 +74,14 @@ mcpp:<层名>[=<实现名>] 省略 `=<实现名>` 时,语义为该层**必须**有供给者。 +`requires` 的条目**可以**是表 `{ layer = "mcpp:<层名>=<实现名>", visibility = "public" | "private" }`。 +字符串等价于 `visibility = "public"`。`private` 声明该需求只对本包自己的代码成立、 +不越过本包的边界(C++ 运行时孤岛,§7.2);它**只**允许用于 `c++-abi` 层,用于其他层时 +清单**必须**以 `requires-private-layer` 报错 —— 一个映像中的第二个 C 库不是孤岛,而是两个 C 库。 + +`requires` **可以**写在 `[target.<选择器>]` 下。生效的需求是 `[package] requires` 与所有匹配 +选择器的 `requires` 的并集,合并规则与 `requires_abi` 相同。 + ### 2.3 标准库模块源 已实现 供给 `c++-abi` 的包**可以**在 `[build]` 下声明: @@ -259,6 +267,65 @@ mcpp:<层名>[=<实现名>] 本机没有任何路径能执行的产物,**必须**在执行前被拒绝,并给出 `run_under` 的写法。当前的判据: 非 macOS 宿主上的 Apple 产物,且没有 `run_under`。`mcpp test` 把这种情形报告为未运行。 +## 7. 一个进程一套 C++ 运行时 已实现 + +### 7.1 原理 + +两套 Itanium C++ ABI 运行时(libstdc++、libc++/libc++abi)共用同样的 ABI 符号名:`__cxa_*`、 +`__gxx_personality_v0`、`std::exception` 一族的类型信息与虚表、`operator new/delete`。同一进程的 +全局符号空间里,它们要么各用一半(mcpp#838:抛出用一边、捕获用另一边,带类型的 `catch` 失效), +要么互相串用。`std::runtime_error` 等类型在两边的内存布局不同。因此**不存在**让"多个映像 + 两套 +运行时 + 异常跨映像"可靠的链接写法。 + +### 7.2 三种模式 + +| 模式 | 成立条件 | 引擎的义务 | +|---|---|---| +| **U 统一**(默认) | 一个进程只有一套 C++ 运行时,由 §3.1 的唯一供给者决定:编译器载荷、目标基座或依赖图 | `requires` 不满足时**必须**在编译前拒绝,并给出补救 | +| **I 孤岛** | 某个组件的 C++ 运行时只存在于它的 C 接口之后,且由包以 `visibility = "private"` 声明 | **必须**机械验证 §7.3 的全部条件,不满足即拒绝 | +| **H 宿主决定** | Python 扩展、JNI、宿主插件:`c++-abi` 由目标基座陈述 | 包服从基座 | + +能统一就统一:依赖图的 `c++-abi` 恰为所需实现时,孤岛声明不生效。 + +### 7.3 孤岛的条件 + +1. **我们一侧只有一个 C++ 映像**:按根包声明的全部目标判定,没有含 C++ 的共享库目标。不满足时 + **必须**在规划期以 `[cxx-runtime-island]` 拒绝。自身运行时静态嵌入并隐藏。 +2. **一套展开器**:`--unwindlib=libgcc`。 +3. **链接顺序**:外来运行时由引擎放在自身运行时之后(链接行的 `foreignCxxRuntime` 位置),规则包 + **禁止**把它写进 `$ldflags` 之前的位置。 +4. **布局相关的冲突集**:链进映像的孤岛目标文件(孤岛包的 role=Object 动作产出)**禁止**以未定义 + 符号引用布局相关的冲突集(`std::exception` 一族中除 `std::exception` 自身外的构造、析构与 `what`)。 +5. **ABI 层整体绑定到自身运行时**:链接后的映像**禁止**以 `GLIBCXX_*`、`CXXABI_*` 版本导入 ABI 层 + 冲突集中的任何符号。 +6. **不跨运行时按类型捕获**:映像引用或定义的 typeinfo,**禁止**同时由一个建立在外来运行时上的 + 共享库导出。 + +条件 4–6 在链接后检查,违反时**必须**以 `[cxx-runtime-collision]` 拒绝并列出符号。 + +**各目标格式**:上述条件按 ELF 的全局符号作用域推导。PE 按导入表绑定:条件 1 放宽为"我们一侧的 +所有映像共用同一个运行时映像",条件 3、5 照常成立。Apple 平台只有 libc++,MSVC ABI 只有一个运行时 +家族,两者都不存在孤岛。 + +### 7.4 外来运行时的识别 + +外来运行时**必须**按类型识别,**禁止**在链接参数中搜索子串: + +- 链接行上被点名的运行时库(`-l:libstdc++.so.6`、`-lstdc++` 等,按词读取); +- 链接输入中的共享库,其 `DT_NEEDED` 含外来运行时的 SONAME(`libstdc++.so.*`、`libc++.so.*`、 + `libc++abi.so.*`;只看直接输入,不递归)。 + +识别出、但没有任何包以 `private` 声明的外来运行时,按孤岛条件校验:全部满足时构建并警告 +(兼容读法 `undeclared-cxx-runtime`,docs/51),不满足时拒绝。 + +### 7.5 依赖图供给的 libstdc++ + +`c++-abi` 来自依赖图中的 libstdc++ 供给包、而编译器为 clang 时,引擎**必须**:以 `-nostdinc++` +加供给包的头文件目录编译;以供给包的 `std-module` 预编译 `std`(源文件不是模块扩展名时以 +`-x c++-module` 编译);链接带 `-stdlib=libstdc++`,运行库与展开器仍为 `--rtlib=compiler-rt +--unwindlib=libgcc`(`compiler-runtime` 层不变,§3.2);向构建程序暴露 `MCPP_CXX_STDLIB` 与 +`MCPP_CXX_ABI_ROOT`。 + --- ## 变更记录 @@ -267,4 +334,5 @@ mcpp:<层名>[=<实现名>] |---|---|---| | v1.0 | 2026-08-24 | 初版。五层闭集、`provides`/`requires` 语法、三条规则、报告与兼容性条款。 | | v1.1 | 2026-09-26 | 规则三增补:`c-abi` 来自图时,链接不搜索宿主库目录(ELF 已实现,mcpp#696)。 | +| v1.3 | 2026-10-26 | 随 mcpp 2026.10.26.1:§2.2 增加 `requires` 的对象形式与 `visibility`、`[target.<选择器>] requires`;新增 §7 一个进程一套 C++ 运行时(U/I/H、孤岛条件、外来运行时按类型识别、依赖图供给的 libstdc++)(mcpp#838)。 | | v1.2 | 2026-10-23 | 随 mcpp 2026.10.24.1:§1.3 `prebuilt` 放宽为被点名、按内容识别的预制目录;§2.4 改为由测量回答、`provides` 声明或核对,标为已实现;规则一增加基座是一个供给者与 c-abi 独占律;规则二增加基座上的编译器运行库与可测量的下限;规则三增加可叠加层;新增 §6 目标侧执行(组合、运行器契约、物理边界)。 | diff --git a/docs/zh/01-getting-started.md b/docs/zh/01-getting-started.md index 36fcf1a2d..5772115c9 100644 --- a/docs/zh/01-getting-started.md +++ b/docs/zh/01-getting-started.md @@ -156,13 +156,16 @@ toolchain、lock/resolution 元数据以及构建目录元数据都可能被更 可信的 workspace 中运行。进程退出码与生成的 `compile_commands.json` 是 稳定的集成契约；标准输出仍是面向人的文本。 -工程根目录下的 `compile_commands.json` 是当前配置自己那份数据库的副本， -后者写在 `target///compile_commands.json`——这个目录 +每个配置的数据库写在 +`target///compile_commands.json`——这个目录 由 toolchain、target、profile 与 feature 共同命名，也正是编译器实际运行 所在的目录。每条在这个配置下做规划的命令(`build`、`test`、`run`、 `--configure-only`)都把新鲜的计划合并进这一份文件，因此 `mcpp test` 加入 -的条目在之后一次普通 `mcpp build` 后仍然留在里面。根目录的这份副本从不 -合并：它整体替换为配置数据库的内容，两者已经一致时保持不变；因此切换 +的条目在之后一次普通 `mcpp build` 后仍然留在里面。它的位置记录在结果记录中 +(`target/.last-result.json` 所指记录的 `compileDatabase`，见 +[50](50-machine-output.md) §9)，编辑器从那里找到它。工程根目录的副本只在 +`[build] compile_commands` 指定时写出(2026.10.26.1+，见 [04](04-mcpp-toml.md))。 +这份副本从不合并：它整体替换为配置数据库的内容，两者已经一致时保持不变；因此切换 `--toolchain` 或 `--profile` 会整体切换根文件，切回去也会恢复那个配置的 条目，包括它的测试单元。根目录若是一个符号链接，写入会跟随到链接指向的 位置。被替换掉的根文件如果持有 mcpp 未写入的条目，会输出一条警告说明 diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index 889ed7727..818ceb5bd 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -830,6 +830,29 @@ minos`）里记录的最低系统版本，也就是这个二进制能运行的 aarch64-macos` 在 Linux 或 Windows 上和在 Mac 上一样遵从这个字段 （以及环境变量），非 macOS 目标则永远不会看到它。 +### `compile_commands` —— 工程编译数据库的发布位置（mcpp 2026.10.26.1+） + +```toml +[build] +compile_commands = "compile_commands.json" # 相对工程根目录的路径 +``` + +每次构建都写下所在配置的数据库 `target///compile_commands.json`； +它的位置记录在结果记录中（`compileDatabase`，[50](50-machine-output.md) §9），也由 +`mcpp emit build-database` 给出。工程根目录的副本只在本键指定时写出。2026.10.26.1 之前， +每次构建都在根目录写 `compile_commands.json`，IDE 发现它后会把工程作为编译数据库工程接管， +而对 mcpp 工程这是 mcppls 的职责。 + +| 来源 | 取值 | +|---|---| +| `MCPP_COMPILE_COMMANDS` | 一个路径，或 `off` / `none`：只对本次运行 | +| `[build] compile_commands` | 一个路径；`""` 表示不要副本 | +| `config.toml` 的 `[build] compile_commands` | 未作陈述的工程在这台机器上的默认值 | +| （无） | 不写副本 | + +第一个被设置的来源决定取值。更早的 mcpp 留在根目录的副本不再更新；第一次发现它的构建 +提示一次，并点名本键。 + ### 在其他章节说明的 `[build]` 键 | 键 | 章节 | @@ -1018,7 +1041,7 @@ Ninja 1.11 及以后的版本在同样的声明下以 UTF-8 读取 `build.ninja` ```text warning: '/home/user/pkg/test/data' contains names that have no UTF-8 spelling impact: those files take no part in the build - hint: The name's bytes are not UTF-8, and build.ninja and compile_commands.json hold UTF-8 text. ... + help: The name's bytes are not UTF-8, and build.ninja and compile_commands.json hold UTF-8 text. ... ``` 报出的路径是最近的、有 UTF-8 拼法的祖先目录。测试数据或文档的名字是无害的； diff --git a/docs/zh/05-dependencies.md b/docs/zh/05-dependencies.md index 0c4939446..a1a87c111 100644 --- a/docs/zh/05-dependencies.md +++ b/docs/zh/05-dependencies.md @@ -197,7 +197,7 @@ comp = { path = "libs/comp" } ``` warning: mcpp.toml names 2 dependencies in namespace acme, and the manifests they reach declare huxdemo; the declared identity is used: acme.fw, acme.comp - hint: write them in a [dependencies.huxdemo] table, for example `fw = { path = "libs/fw" }` + help: write them in a [dependencies.huxdemo] table, for example `fw = { path = "libs/fw" }` ``` 因此，同一目录上分别写作 `fw` 与 `huxdemo.fw` 的两条边是同一个包，只编译 diff --git a/docs/zh/07-workspace.md b/docs/zh/07-workspace.md index c80761dbb..8d3d401e3 100644 --- a/docs/zh/07-workspace.md +++ b/docs/zh/07-workspace.md @@ -369,7 +369,7 @@ warning: dependency `render` declares standard = "c++26", and this graph is built at c++23 impact: a C++ module graph has one standard, so the dependency's declaration is not applied and its sources are compiled at the graph's level - hint: raise the consumer's standard to "c++26", or declare it once for + help: raise the consumer's standard to "c++26", or declare it once for every member: [workspace.package] @@ -554,7 +554,7 @@ mcpp test --workspace --workspace-timeout 1800 # whole fan-out (default 0 = no 对一个成员的 `mcpp test` 是该成员的一次规划，一如既往。 - **编译数据库。** `mcpp build --configure-only` 与 `mcpp emit build-database` 按构建的方式 规划，每个配置一次规划并包含各成员的测试，因此成员共用的包在每个配置中只描述一次。规划了 - 多个配置的命令只写一次根目录的 `compile_commands.json`，内容为各配置数据库的并集 + 多个配置的命令只写一次根目录的副本(在 `[build] compile_commands` 指定时)，内容为各配置数据库的并集 （2026.9.29.5+）。 - **无事可做的构建。** 在没有任何改动时重复执行的命令，每个配置只做一次检查，不重新规划。 - **模块名。** 模块名在一个程序之内唯一，而不是在一张图之内唯一（2026.9.30.2+）。不共享任何 @@ -605,7 +605,7 @@ myproject/ 放在旁边。两个成员包名相同时使用 `bin/./`。带根包的工作空间中，根包仍使用 `bin/`。 - 放入多个产物目录的共享库，在支持硬链接的文件系统上是同一个文件的多个名字，其他情况下被复制。 -- 工作空间根的 `compile_commands.json` 覆盖所有已构建或已配置的成员。 +- 工作空间根的编译数据库(在 `[build] compile_commands` 指定时)覆盖所有已构建或已配置的成员。 - 工作空间根的 `mcpp.lock` 记录全体成员的解析结果。`mcpp build --workspace` 写入完整记录； `mcpp build -p X` 更新 X 所在图中的条目。 - 成员的构建程序写入 `/target/.build-mcpp/`。各成员的构建程序在 workspace 的构建程序图 diff --git a/docs/zh/20-toolchains.md b/docs/zh/20-toolchains.md index c8c030617..b494ea157 100644 --- a/docs/zh/20-toolchains.md +++ b/docs/zh/20-toolchains.md @@ -976,7 +976,7 @@ mcpp 不从那里读任何东西。 ``` warning: the assembler for this build is the host's ('/usr/bin/nasm'), not the one this engine pins impact: two machines can assemble the same source with different assemblers, and the build records only this line - hint: run `xlings install nasm` so the pinned copy is used + help: run `xlings install nasm` so the pinned copy is used ``` **一个宿主工具到达构建，本身不是缺陷；一个宿主工具静默地到达构建才是。** 这 diff --git a/docs/zh/22-target-side.md b/docs/zh/22-target-side.md index 026b394d8..b4337f3b4 100644 --- a/docs/zh/22-target-side.md +++ b/docs/zh/22-target-side.md @@ -268,6 +268,84 @@ requires = ["mcpp:c++-abi=libstdc++"] 另一种变体构建进同一个存储目录，否则第一个消费者就会替之后所有消费者决定 变体。 +### 表形式与按目标的 `requires`（mcpp 2026.10.26.1+） + +`requires` 的条目可以是一张表： + +```toml +requires = [{ layer = "mcpp:c++-abi=libstdc++", visibility = "private" }] +``` + +字符串等价于 `visibility = "public"` 的表。`private` 陈述该需求只对本包自己的代码 +成立，不越过本包的边界：本包的 C++ 运行时位于一个 C 接口之后（孤岛，见下文）。它 +只用于 `c++-abi` 层；用于其他层时清单以 `requires-private-layer` 被拒绝，因为一个 +映像中的第二个 C 库是两个 C 库，而不是孤岛。 + +`requires` 也可以写在目标选择器下，需求集合是 `[package] requires` 与所有匹配 +选择器的 `requires` 的并集，与 `requires_abi` 相同： + +```toml +[target.'cfg(os = "linux")'] +requires = [{ layer = "mcpp:c++-abi=libstdc++", visibility = "private" }] +``` + +早于 2026.10.26.1 的引擎拒绝表形式；使用它的包声明版本下限 +（`[build-dependencies.mcpp] version = ">= 2026.10.26.1"`，或 index 描述文件中的 +`mcpp = ">= 2026.10.26.1"`）。 + +### 一个进程一套 C++ 运行时（mcpp 2026.10.26.1+） + +两套 Itanium C++ 运行时 —— libstdc++，以及 libc++ 加 libc++abi —— 定义同样的 ABI +名字：`__cxa_*`、`__gxx_personality_v0`、`std::exception` 一族的类型信息与虚表、 +`operator new` 与 `delete`。在一个进程的全局符号空间里，它们最终各自承担一次异常的 +一半（mcpp#838：一边抛出、另一边捕获，只有 `catch (...)` 能匹配），而 +`std::runtime_error` 在两边的布局不同。没有任何链接写法能让"多个映像、两套运行时、 +异常跨映像"可靠。因此引擎让一次构建处于三种模式之一： + +| 模式 | 适用情形 | 引擎的义务 | +|---|---|---| +| **U，统一**（默认） | 进程只有一套 C++ 运行时，即 `c++-abi` 的唯一供给者（编译器载荷、目标基座或依赖图中的包） | 构建不满足的 `requires` 在编译前被拒绝，并给出补救 | +| **I，孤岛** | 某个组件的 C++ 运行时只位于它的 C 接口之后，由一个包以 `visibility = "private"` 声明 | 逐条验证下列条件；不满足任一条的构建被拒绝 | +| **H，宿主决定** | Python 扩展、JNI 库、宿主插件：目标基座陈述 `c++-abi` | 包服从基座 | + +孤岛在以下条件全部成立时成立： + +1. 构建只有一个 C++ 映像：根包的目标中没有含 C++ 的共享库，否则规划以 + `[cxx-runtime-island]` 被拒绝； +2. 一套展开器，`--unwindlib=libgcc`； +3. 外来运行时在链接行上位于构建自身的运行时之后 —— 由引擎放置，规则包不把它写进 + `ldflags`； +4. 孤岛的目标文件不引用冲突集中与布局相关的成员（`std::exception` 一族中除 + `std::exception` 本身以外的构造、析构与 `what`）； +5. 链接后的映像不以 `GLIBCXX_*` 或 `CXXABI_*` 版本导入任何 ABI 层符号； +6. 映像引用或定义的类型信息，没有同时由一个建立在外来运行时上的共享库导出 —— + 跨两套运行时的带类型 `catch` 永远不会匹配。 + +条件 4 至 6 在链接后检查，以 `[cxx-runtime-collision]` 报告并列出符号。外来运行时 +按类型识别，从不在链接参数中搜索子串：链接行上点名的运行时库（`-l:libstdc++.so.6`）， +或链接行上 `DT_NEEDED` 含外来运行时的共享库（`libstdc++.so.*`、`libc++.so.*`、 +`libc++abi.so.*`）。没有任何包声明的外来运行时按孤岛验证：成立时构建继续并给出警告 +（兼容读法 `undeclared-cxx-runtime`，见 [51](51-supported-versions.md)），不成立时 +构建被拒绝。 + +SYCL 程序永远不是孤岛：它的单元捕获 `sycl::exception`，该类型由 `libsycl.so` 定义并在 +libstdc++ 上抛出（条件 6）。它在 U 模式下构建 —— 使用 gcc，或使用 llvm 加依赖图供给的 +libstdc++： + +```toml +[dependencies] +gcc.libstdcxx = "16.1.0" # provides mcpp:c++-abi=libstdc++ + +[toolchain] +default = "llvm@22.1.8" # 只选择编译器 +``` + +clang 下由依赖图供给 libstdc++ 时，引擎以 `-nostdinc++` 加供给包的头文件编译，以它的 +`std-module` 预编译 `std`，链接带 `-stdlib=libstdc++`（编译器运行库仍为 `compiler-rt`， +展开器为 `libgcc`），并向构建程序提供 `MCPP_CXX_STDLIB=libstdc++` 与 +`MCPP_CXX_ABI_ROOT`（供给包的根目录），使规则包让第二个编译器使用同一个库。契约见 +[SPEC-002 §7](../specs/target-side.md)。 + ### `c-abi` 包陈述它呈现的 C 环境（mcpp 2026.9.18+） 传统技术栈不需要陈述这件事：编译器载荷的目标三元组已经蕴含了 C 程序所编译 diff --git a/docs/zh/30-build-mcpp.md b/docs/zh/30-build-mcpp.md index 0e580bab7..6757adb4c 100644 --- a/docs/zh/30-build-mcpp.md +++ b/docs/zh/30-build-mcpp.md @@ -1001,7 +1001,7 @@ import mcpp.core; // 与 `import mcpp;` 导出相同的符号 ### 结构化诊断：`mcpp::report`（协议 14） -用 `mcpp::report` 发出的诊断按引擎自己的形式呈现：先是消息，然后是 `impact:` 行与 `hint:` +用 `mcpp::report` 发出的诊断按引擎自己的形式呈现：先是消息，然后是 `impact:` 行与 `help:` 行。它以相同的字段进入 `--message-format json`；`degraded` 级别在 `--strict` 下使构建失败； 缓存命中的运行会再次报告它。`mcpp::warning(text)` 保留，等价于只有消息的诊断。 @@ -1043,7 +1043,7 @@ data/b.txt bin/data/b.txt ```text error: build.mcpp imports 'mcpp.rules.qt' provided by: mcpp.plugins, feature "rules-qt" (not enabled) - hint: enable it on the dependency edge: mcpp.plugins = { ..., features = ["rules-qt"] } + help: enable it on the dependency edge: mcpp.plugins = { ..., features = ["rules-qt"] } ``` ### `import mcpp;` 才是会演进的那一面（mcpp 2026.8.5.1+） diff --git a/docs/zh/42-heterogeneous-builds.md b/docs/zh/42-heterogeneous-builds.md index 33bfe47a4..d8168a6e1 100644 --- a/docs/zh/42-heterogeneous-builds.md +++ b/docs/zh/42-heterogeneous-builds.md @@ -591,25 +591,29 @@ CPU 实现，要么是上面那条链接错误。 选哪个形态是**程序自己的性质**，不是 mcpp 的性质：一个用接缝替换实现的程序， 想要链接期选择；一个要发到没见过的机器上的程序，想要运行期选择。 -## 链接行上出现第二个 C++ 运行时时，只保留一个 unwinder - -一条 lane 的设备编译器，如果是按 libstdc++ 配置的，链接行上就会出现 -libstdc++，而产物本身静态链接 libc++。两者同时出现在镜像里，mcpp 的重复符号 -检查会报出它们共有的那些符号。 - -对其中大多数符号，后果只是「调用了另一份可互换的实现」。对 unwinder 不是。 -静态归档只贡献被引用到的成员，所以这种抢占**按构造是部分的**：在 SYCL lane -上实测，libgcc 的 18 个 `_Unwind_*` 入口点里，有 10 个来自产物，8 个仍留在 -libgcc_s 里，其中包括 personality 例程要用的那几个访问器。于是 libstdc++ 的 -personality 拿 libgcc 的访问器去读一个 LLVM libunwind 的 context，找不到 -landing pad，越过三帧之上一个本应命中的 handler，直接调用了 -`std::terminate`。在有东西抛出之前，这个程序一直是正确的。 - -因此，当链接行上出现 libstdc++、而工具链自带的标准库是 libc++ 时，mcpp 改从 -libgcc 取 unwinder（`--unwindlib=libgcc`），不再链载荷的 `libunwind.a`，并用 -`--exclude-libs` 把静态归档的符号挡在动态符号表之外。libgcc_s 本来就在 -进程里——libstdc++ 需要它——所以这一步只是点名一个已有的库，不是新增一个， -C++ 运行时仍然是内嵌的。链接行上没有第二个运行时的构建，一字节不变。 +## 一套 C++ 运行时，以及可以不同的孤岛（2026.10.26.1+） + +设备运行时建立在 libstdc++ 上的 lane（`libsycl.so`），会遇到自身 C++ 库来自工具链的 +程序。一个进程中的两套 Itanium 运行时共用 ABI 名字 —— `__cxa_throw`、personality +例程、`std::exception` 的类型信息 —— 跨越两者的带类型 `catch` 不会匹配（mcpp#838： +`catch (const std::exception&)` 没有接住库抛出的异常，只有 `catch (...)` 接住）。因此 +一个进程只有一套 C++ 运行时（[22 —— 目标侧](22-target-side.md)）： + +- **SYCL 只用一套运行时：libstdc++。** `compat.sycl-runtime` 声明 + `requires = ["mcpp:c++-abi=libstdc++"]`。gcc 工具链满足它，llvm 加依赖图中的 + `gcc.libstdcxx` 同样满足；规则读取 `MCPP_CXX_ABI_ROOT`，让设备编译器使用同一个库。 + 没有 libstdc++ 供给者的 llvm 构建在任何编译之前被拒绝，并给出两种补救。实测：设备 + 编译单元捕获 `sycl::exception`，该类型由 `libsycl.so` 定义并在 libstdc++ 上抛出，因此 + SYCL 单元不可能是孤岛。 +- **孤岛需要声明，并被验证。** C++ 运行时只位于 C 接口之后的组件声明 + `requires = [{ layer = "mcpp:c++-abi=libstdc++", visibility = "private" }]`。引擎把 + 外来运行时放在程序自身运行时之后，从 libgcc 取展开器（`--unwindlib=libgcc`），用 + `--exclude-libs` 隐藏静态运行时的符号，并在链接后检查 ABI 层只绑定到一套运行时、 + 没有跨两套运行时的按类型捕获。静态归档只贡献被引用的成员，所以第二个展开器按构造 + 是部分的：此规则之前在 SYCL lane 上实测，libgcc 的 18 个 `_Unwind_*` 入口中 10 个 + 来自产物、8 个留在 libgcc_s，一次在处理器之下三帧的抛出以 `std::terminate` 结束。 +- **HIP 没有第二套运行时。** NVIDIA 平台用工程自己的 clang 编译 `.hip`， + `libcudart_static.a` 引用的 C++ ABI 符号只有 `__cxa_atexit`。 ## 两条值得写明的边界 @@ -655,11 +659,10 @@ HIP 与 SYCL 两行的 `accel` 值是**两段**，不是一段。第一段命名 ROCm。`xim:hip-nvidia` 因此不含任何二进制。AMD 平台需要一个本生态尚未发布的 ROCm 运行时，规则会点名拒绝，而不是产出一个没有东西能链接的目标文件。 -**一次 SYCL 构建携带两个 C++ 运行时，而接缝是让这件事安全的机制。** -`libsycl.so` 是对着 libstdc++ 编译的，而一个 mcpp 产物链接 libc++，于是两者 -都在同一个镜像里，mcpp 的重复符号检查会报出它们共有的那些 unwinder 符号。 -任何东西都不得穿过接缝：一个 SYCL 异常在设备编译单元里被捕获，转成一个返回码， -因为抛出它的那个运行时，不是调用方会用来展开的那一个。 +**一次 SYCL 构建只携带一套 C++ 运行时：libstdc++。** `libsycl.so` 是对着 libstdc++ +编译的，所以程序也建立在它之上：使用 gcc，或使用 llvm 加依赖图中的 `gcc.libstdcxx`。 +2026.10.26.1 之前，llvm 的 SYCL 构建在它旁边链接 libc++，运行时抛出的 `sycl::exception` +只有 `catch (...)` 能接住；这样的构建现在被拒绝，并给出两种补救。 ## 每条 lane 能到达的平台 diff --git a/docs/zh/50-machine-output.md b/docs/zh/50-machine-output.md index 8490823e2..8d9729d59 100644 --- a/docs/zh/50-machine-output.md +++ b/docs/zh/50-machine-output.md @@ -74,6 +74,12 @@ CI 脚本，以及任何解析 mcpp 输出的程序，都可以依赖本章陈 一条没有位置的诊断会省略 `path` 与 `range`，而不是填 0 —— `line: 0` 会把 读者指向一个不存在的位置。 +消息以源码位置开头的诊断 —— `::: `、`:: ` 或 +`: error: `，即 mcpp 写出的每一条带位置的消息的形式 —— 把这个位置作为 +`path` 与 `range` 携带（2026.10.26.1+）。清单的拒绝定位到它点名的键或表： +`[build].sources must be …` 指向 `sources` 所在的那一行。已经在 `path` 中写明文件 +的命令保留自己的写法，并补上 `range`。 + `code` 永远存在。解析 `code`，永远不要解析 `message`。 ## 3. 请求机器输出 @@ -90,7 +96,8 @@ mcpp --format json （`tar`、`dir`、`msi`），所以它的报告方式是 `mcpp pack --message-format json`（2026.9.16.1+）。具体形状随 kind 而定： `mcpp test` 的测试是随时间陆续完成的，所以它按测试逐条输出流式记录；一次 -`mcpp pack` 只有一个结果，所以它输出一个信封。 +`mcpp pack` 只有一个结果，所以它输出一个信封。`mcpp build --message-format json` +（2026.10.26.1+）在构建过程中输出事件流，以一个信封结束（§9）。 ### 不支持的值与未知选项 @@ -117,6 +124,7 @@ $ echo $? | 0 | 成功 | | 1 | 命令运行了并且失败 —— 见 stderr；当 stdout 携带信封时见 `diagnostics` | | 2 | 用法错误 —— 未知选项、不支持的取值 | +| 5 | 运行在调用方拒绝的副作用之前停下（§10；2026.10.26.1+） | | 70 | 内部错误（未捕获的异常） | | 127 | 未知命令 | @@ -726,6 +734,156 @@ mcpp test [pattern] [--workspace] --message-format json 出的树不带 `needs` 行，由更早版本的 mcpp 暂存出的树同样不带；自行放置库的 读者应当读取这些行，而不是从 `lib/` 下的文件推断闭包。 +## 9. 构建事件流与结果记录 *（mcpp 2026.10.26.1+）* + +构建一个工程的每条调用链 —— 终端里的人、只有 shell 的 agent、IDE、CI 任务 —— +都终止于一次 mcpp 运行，因此一条链需要了解另一条链的构建时，由 mcpp 以同一种 +形式写到磁盘上。 + +### `mcpp build --message-format json` —— 事件，然后是 `mcpp.build` + +``` +mcpp build [target] [-p ] [--release] --message-format json +``` + +标准输出是 NDJSON。除最后一行外，每一行是一个事件： + +```jsonc +{"schemaVersion":1,"kind":"mcpp.event","event":"plan","configuration":{"triple":"x86_64-linux-gnu","profile":"dev","fingerprint":"c0641c458b2bde43","outputDir":"/…/target/x86_64-linux-gnu/c0641c458b2bde43"}} +{"schemaVersion":1,"kind":"mcpp.event","event":"progress","finished":12,"total":34} +{"schemaVersion":1,"kind":"mcpp.event","event":"artifact","artifact":{"name":"app","kind":"bin","path":"/…/bin/app","launch":{…}}} +``` + +| 事件 | 携带 | +|---|---| +| `plan` | `configuration`：构建已进入其构建目录的那个配置 | +| `waiting` | `holder`（`pid`、`command`、`client`、`startedAt`）与 `waitedMs`：另一次构建持有这个构建目录，本次在等待（§11） | +| `progress` | `finished`、`total`：ninja 的步骤计数；一次构建可以运行不止一轮 ninja，每轮重新计数 | +| `diagnostic` | `diagnostic`：失败步骤输出中的一条编译器诊断，形状同下文记录的 `diagnostics[]` | +| `artifact` | `artifact`：一个产物，形状同下文记录的 `artifacts[]` | + +最后一行是 `mcpp.build` 信封，占一行。它的 `data` 是本次运行写下的结果记录（见 +下文）；在进入构建目录之前就失败的运行省略 `data`。它的 `diagnostics` 是 mcpp +自己的错误，每条按其消息定位（§2），代码为 `MCPP_BUILD_FAILED`；在被拒绝的副作 +用之前停下的运行为 `MCPP_EFFECT_DENIED`。叙述写到标准错误，与每一种机器可读模式 +相同。 + +### 结果记录：`result.json` 与 `target/.last-result.json` + +构建在开始写构建目录时写下 `/result.json`（kind `mcpp.result`），其中 +`"status": "running"`，结束时再写一次。`target/.last-result.json`（kind +`mcpp.last-result`）指向工程最近一次构建写下的记录。两者都原子替换。因此终端里的 +一次构建对读取记录的 IDE 可见，反之亦然。 + +| `data` 的字段 | | +|---|---| +| `command` | 本次运行的命令行 | +| `client` | 调用方设置的 `MCPP_CLIENT`（§12），没有时为 `""` | +| `pid` | 写下记录的进程 | +| `startedAt`、`finishedAt` | 自纪元起的毫秒数；`running` 期间与 `cancelled` 之后 `finishedAt` 为 `null` | +| `status` | `running`、`ok`、`failed` 或 `cancelled` | +| `configuration` | `triple`、`profile`、`fingerprint`、`outputDir` | +| `compileDatabase` | 该配置的 `compile_commands.json`，或 `null` | +| `artifacts` | `name`、`kind`（`bin`、`test`、`shared`、`static`）、`path`；可运行的产物另有 `launch` | +| `diagnostics` | 失败构建的编译器诊断：`severity`、`message`、`path`、`range` | + +`launch` 是 `mcpp run` 启动该产物的方式：`argv`、`cwd`、`env` 与 `via`。`env` 只含 +mcpp 为程序设置或改写的变量（加载器的搜索路径、运行时载荷需要的变量），从不包含 +调用方的其余变量，因此记录中不含调用方的机密。`via` 为 `null`、`"runner"`（由 +`[target.].runner` 包装）或 `"run_under"`（由执行宿主包装）。调试器、IDE 或 +CI 按 `launch` 启动程序，与 `mcpp run` 的启动一致。 + +被 SIGINT、SIGTERM 或 SIGHUP 结束的运行在两个文件中都留下 `cancelled` +（[退出码](../specs/exit-codes.md) §3.7）。`status` 仍为 `running`、而 `pid` 已不存在 +的记录属于一次被强制结束的运行。 + +`.last-result.json`： + +```json +{"schemaVersion":1,"kind":"mcpp.last-result","kindVersion":1, + "data":{"result":"/…/target/x86_64-linux-gnu/c0641c458b2bde43/result.json","status":"ok","finishedAt":1792964065954}} +``` + +### `mcpp.launch` —— `mcpp run` 启动程序的方式 + +``` +mcpp run [target] --print-launch +``` + +按 `mcpp run` 的方式构建程序，然后打印信封而不启动它。`data` 为 `program`、 +`argv`、`cwd`、`env` 与 `via`，同上文的 `launch`。`--print-launch` 不接受 +`--format`：在 `mcpp run` 上，`--format` 表示可分发物的格式。 + +## 10. 拒绝副作用 —— `--deny` *（mcpp 2026.10.26.1+）* + +``` +mcpp --deny network,exec-build-script build +MCPP_DENY_EFFECTS=write-project mcpp build +``` + +`--deny` 接受 §4 中以逗号分隔的副作用；`MCPP_DENY_EFFECTS` 接受同样的列表，一次 +运行拒绝两者的并集。`read-project` 不可拒绝；未知的名字是用法错误（退出 2）。 +`--offline` 拒绝 `network`，作为它的写法保留。 + +运行在它将要执行、且被调用方拒绝的第一个副作用之前停下，在标准错误上写明副作用 +及其用途（`effect denied: write-project -- the build writes …`；被拒绝的 `network` +所需的下载照旧报告为离线拒绝），以 `note: stopped before a denied effect (…); +exit status 5` 结束，退出 **5**。被拒绝的 `write-global-cache` 不会让构建停下：构建 +改用工程自己的缓存。运行不需要的副作用即使被拒绝也不构成阻碍 —— 记录的计划有效 +的构建不运行构建程序，因此在 `--deny exec-build-script` 下也能成功。信封的 +`effects` 继续报告运行实际做了什么。 + +## 11. 并发运行 + +一个配置的构建目录同一时刻只由一次运行写入。同一配置的第二次构建等待第一次，在 +标准错误上报告持有者（`Waiting for the build directory, held by pid 1234 (mcpp +build, from vscode, 8s)`），并发出 `waiting` 事件，之后几乎无事可做。不同配置的 +构建、查询（`emit build-database`、`cache list`、读取记录），以及 `mcpp run` 与 +`mcpp test` 启动的程序从不等待。链接产物整体替换，因此在链接期间启动的程序不会 +执行半个文件。`mcpp.lock` 以乐观方式写入：本次解析期间被另一次运行写入的锁文件 +不会被覆盖，运行以 `[lockfile-concurrent]` 停下。契约见 +[docs/specs/concurrency.md](../specs/concurrency.md)。 + +## 12. 调用方的名字、颜色与单次运行的 home *（mcpp 2026.10.26.1+）* + +- `MCPP_CLIENT=` 在构建目录锁的持有者与结果记录中写明调用方（IDE 的后端、 + agent 的宿主），不改变其他任何行为。 +- 颜色：`MCPP_NO_COLOR=1` 或非空的 `NO_COLOR` 关闭颜色；否则 `CLICOLOR_FORCE` 非空且 + 不为 `0`、或 `FORCE_COLOR` 非空且不为 `0`、`false` 时，管道也有颜色；否则终端有颜色，管道没有。 +- `mcpp --home ` 以 `` 作为一条命令的 mcpp home 运行，与 + `MCPP_HOME=` 相同（[SPEC-010](../specs/home-initialization.md)）。 + +## 13. 查询 *（mcpp 2026.10.26.1+）* + +### `mcpp.search` —— 索引提供的包 + +``` +mcpp search [keyword] --format json +``` + +`data` 为 `keyword`（未给出时为 `""`，此时列出全部包）与 `packages`：每个包一条 +记录，含 `id`（`namespace:name`）、`namespace`、`name`、`index`、`description`、 +`licenses`、`repo` 与 `versions`（最新在前）。每个索引只读一次；命令不需要工程。 + +### `mcpp.index` —— 注册表与工程的索引 + +``` +mcpp index list --format json +``` + +`data` 为 `registries`（`name`、`url`、`default`）与 `indices`（在工程中运行时， +工程的 `[indices]` 条目）。包的目录由 `mcpp search` 回答。 + +### `mcpp.pack.formats` —— `mcpp pack --format` 接受的格式 + +``` +mcpp pack --list-formats +``` + +`data.formats` 列出引擎自己的格式（`tar`、`dir`；`builtin: true`）以及已解析依赖 +图中某个包提供的每种格式（`builtin: false`，带 `providers`）。依赖图在 mcpp home 的 +一个目录中规划，因此不向工程写入任何东西。 + ## 当前边界 - **退出码表的作用域仅限于它点名的那些命令。** 别的命令返回的退出码不在表 diff --git a/docs/zh/51-supported-versions.md b/docs/zh/51-supported-versions.md index 26055ad87..2ba476499 100644 --- a/docs/zh/51-supported-versions.md +++ b/docs/zh/51-supported-versions.md @@ -64,6 +64,8 @@ schema，那是 [50 —— 机器可读输出](50-machine-output.md)。 |---|---|---|---| | `workspace-position`：工作空间根自己的 `[toolchain]`、`[indices]`、`[profile.]`、`[target.]` 标量或 `[xlings]` 按位置到达成员 | 2026.10.10.1 | 2027.1 | `[workspace.toolchain]`、`[workspace.indices]`、`[workspace.profile.]`、`[workspace.target.]`、`[workspace.xlings]` | | `host-module-in-dependencies`：`[dependencies]` 中请求 `host-module = true` 或 `rule_module` 特性的条目，按构建期依赖读取 | 2026.10.11.1 | 2027.1 | `[build-dependencies]` 中的同一条目 | +| `abi-capability`：`abi:` 运行时能力（`abi:glibc`、`abi:libc=`、`abi:cxxstdlib=`），按目标侧某一层的需求读取 | 2026.10.26.1 | 2027.1 | `requires = ["mcpp:c-abi="]` 或 `["mcpp:c++-abi="]` | +| `undeclared-cxx-runtime`：链接线点名、或链接线上的库需要的、本构建之外的另一套 C++ 运行时，且没有包声明 | 2026.10.26.1 | 2027.1 | 在需要它的包中写 `requires = [{ layer = "mcpp:c++-abi=", visibility = "private" }]` | | `plugin-imports-from-build-dependencies`：构建插件的库导入其自身 `[build-dependencies]` 中以 `host-module = true` 写出的包 | 2026.10.11.1 | 2027.1 | 插件 `[dependencies]` 中的该包 | ## 复现一次构建 diff --git a/docs/zh/52-driving-mcpp-from-a-program.md b/docs/zh/52-driving-mcpp-from-a-program.md new file mode 100644 index 000000000..8add8fd44 --- /dev/null +++ b/docs/zh/52-driving-mcpp-from-a-program.md @@ -0,0 +1,106 @@ +# 52 —— 从程序或 agent 驱动 mcpp + +**读者：** 编写 agent 指令、编辑器后端或 CI 任务的作者，这些程序在 shell 中运行 mcpp， +并按得到的结果行动。 + +**本章回答的那一个问题：** 哪些命令、输出与退出状态，使一个程序不经终端、不经 IDE +就能构建、运行并检查一个 mcpp 工程。 + +**不在这里：** 各份文档的形状，见 [50 —— 机器可读输出](50-machine-output.md)；退出码 +契约，见 [SPEC-003](../specs/exit-codes.md)；以及建立在本章内容之上的编辑器集成本身。 + +构建一个工程的每条调用链都终止于一次 mcpp 运行：终端里的人、只有 shell 的 agent、 +编辑器的保存时构建、CI 任务。因此 mcpp 把一条链需要了解的另一条链的构建写到磁盘上， +并以 shell 可以分支的退出状态陈述结果。程序使用其中任何一项，都只需要一个 shell。 + +## 1. 构建并读取结果 + +```sh +mcpp build --message-format json > build.ndjson +``` + +标准输出的每一行是一条 JSON 记录。最后一行是 `mcpp.build` 信封，它的 `data` 是结果 +记录：`status`（`ok`、`failed`）、`artifacts`（每个可运行产物附带启动方式）与 +`diagnostics`（每条带 `path` 与 `range`）。之前的各行是事件（`plan`、`waiting`、 +`progress`、`diagnostic`、`artifact`），供报告进度的读者使用。叙述 —— 人读的内容 —— +在标准错误上。 + +没有发起构建的程序从磁盘读取同一份记录： + +```sh +cat target/.last-result.json # 指向记录及其状态 +cat "$(jq -r .data.result target/.last-result.json)" +``` + +终端里的构建与编辑器里的构建写下同一种记录，因此双方都能看到对方的结果。 + +## 2. 运行，或自行启动程序 + +`mcpp run` 先构建，然后以程序替换自身，标准输出属于程序；退出状态是程序的状态（构建 +失败时为 `101`）。自行启动程序的调试器或测试框架请求启动描述： + +```sh +mcpp run app --print-launch +``` + +`mcpp.launch` 信封含 `argv`、`cwd` 与 `env` —— 只有 mcpp 为程序设置的变量 —— 与 +`mcpp run` 使用的完全一致。 + +## 3. 退出状态 + +| 状态 | 含义 | 程序的处理 | +|---|---|---| +| `0` | 完成 | 读取记录 | +| `1` | 工程或环境失败 | 读取 `diagnostics`，修复后重跑 | +| `2` | 用法错误 | 修正命令行 | +| `5` | 运行在调用方拒绝的副作用之前停下 | 请调用方审批，然后去掉拒绝重跑 | +| `101` | `mcpp run` 无法构建程序 | 同 `1` | +| `130`、`143` | 被 SIGINT、SIGTERM 结束（POSIX） | 记录为 `cancelled`；下一次构建从中断处继续 | + +## 4. 限制一次运行可以做的事 + +agent 的宿主，或处于不受信任工作区的编辑器，陈述一次运行不得产生的副作用： + +```sh +mcpp --deny network,exec-build-script build +MCPP_DENY_EFFECTS=network mcpp build # 同上，作用于一个会话的每条命令 +``` + +副作用为 `init-mcpp-home`、`write-project`、`write-global-cache`、`network` 与 +`exec-build-script`。需要被拒绝副作用的运行在它之前停下，在标准错误上点名，退出 `5`； +没有任何事做了一半。不需要它的运行不受影响：计划已记录的工程在 +`--deny exec-build-script` 下照常构建。`mcpp --protocol-version` 在任何运行之前列出每条 +命令可能产生的副作用。 + +## 5. 同时进行的多次运行 + +同一配置的两次构建不会互相破坏：第二次等待第一次，并说明等待的是谁 +（`Waiting for the build directory, held by pid 1234 (mcpp build, from my-agent, 8s)`）。 +设置 `MCPP_CLIENT` 会在这一行与结果记录中写明调用方： + +```sh +MCPP_CLIENT=my-agent mcpp build +``` + +`mcpp run` 启动的程序不持有锁，因此在它运行期间重新构建是安全的；链接产物整体替换。 +查询从不等待。契约见 [SPEC-011](../specs/concurrency.md)。 + +## 6. 不需要构建的查询 + +| 问题 | 命令 | +|---|---| +| 索引提供的包 | `mcpp search [keyword] --format json` | +| 已配置的注册表 | `mcpp index list --format json` | +| `mcpp pack` 接受的包格式 | `mcpp pack --list-formats` | +| 每个编译单元的编译命令 | `mcpp emit build-database` | +| 已安装的工具链 | `mcpp toolchain list --format json` | +| mcpp 使用的各个路径 | `mcpp self env --format json` | +| 以另一个 home 运行一次 | `mcpp --home ` | + +## 7. 读取面向人的输出 + +程序只有叙述可读时 —— CI 日志、终端记录 —— 形态是固定的 +（[SPEC-012](../specs/human-output.md)）：状态行以一个右对齐的词开头（`Compiling`、 +`Finished`）；mcpp 的消息以 `error:`、`warning:` 或 `note:` 开头；关于某个文件的消息以 +`:::` 开头；下一步单独成行，以 `help:` 开头。管道中没有颜色，除非 +`CLICOLOR_FORCE` 或 `FORCE_COLOR` 要求。这些词本身没有兼容承诺；上文的记录有。 diff --git a/docs/zh/README.md b/docs/zh/README.md index 69855f076..5a300e6aa 100644 --- a/docs/zh/README.md +++ b/docs/zh/README.md @@ -98,6 +98,7 @@ - [50 —— 机器可读输出](50-machine-output.md) - [51 —— 受支持的版本与兼容性](51-supported-versions.md) +- [52 —— 从程序或 agent 驱动 mcpp](52-driving-mcpp-from-a-program.md) ### 9x —— mcpp 自身 @@ -163,3 +164,5 @@ - [SPEC-008 —— 库的接口：公开模块、发布闭包与两种形态的一致](../specs/library-interface.md) - [SPEC-009 —— 工具链的支持与维护：版本线、默认值、来源、移动与退役](../specs/toolchain-maintenance.md) - [SPEC-010 —— home 选择与初始化](../specs/home-initialization.md) + - [SPEC-011 —— 并发运行：构建目录锁、整体发布、乐观写入与取消](../specs/concurrency.md) + - [SPEC-012 —— 面向人的输出：两条流、两种介质、行的种类与颜色](../specs/human-output.md) diff --git a/mcpp.toml b/mcpp.toml index c9b732f42..fe5c96036 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.10.24.1" +version = "2026.10.26.1" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/modules/libs/src/toml.cppm b/modules/libs/src/toml.cppm index 435110455..e102293c6 100644 --- a/modules/libs/src/toml.cppm +++ b/modules/libs/src/toml.cppm @@ -69,8 +69,10 @@ private: class Document { public: - explicit Document(Table root, std::set> explicitTables = {}) - : root_(std::move(root)), explicitTables_(std::move(explicitTables)) {} + explicit Document(Table root, std::set> explicitTables = {}, + std::map> headers = {}) + : root_(std::move(root)), explicitTables_(std::move(explicitTables)), + headers_(std::move(headers)) {} const Table& root() const { return root_; } Table& root() { return root_; } @@ -97,9 +99,19 @@ public: return explicitTables_; } + // Where `dotted_path` is written: a value's own position, or the first + // `[header]` / `[[header]]` of a table. nullopt for a table that only a + // dotted key or a deeper header created, and for a path that is absent. + std::optional position_of(std::string_view dotted_path) const { + if (auto* v = get(dotted_path); v && v->position.line > 0) return v->position; + if (auto it = headers_.find(dotted_path); it != headers_.end()) return it->second; + return std::nullopt; + } + private: Table root_; std::set> explicitTables_; + std::map> headers_; }; std::expected parse(std::string_view src); @@ -404,6 +416,7 @@ inline Table* dive(Table& root, const std::vector& path, bool creat if (it == cur->end()) { if (!create_intermediate) return nullptr; (*cur)[seg] = Value{Table{}}; + (*cur)[seg].position = {0, 0}; // written nowhere itself cur = &(*cur)[seg].as_table(); } else if (!it->second.is_table()) { return nullptr; // collision: existing non-table @@ -455,6 +468,7 @@ inline std::expected open_array_of_tables( } Array& arr = it->second.as_array(); arr.push_back(Value{Table{}}); + arr.back().position = pos; return &arr.back().as_table(); } @@ -535,6 +549,7 @@ std::expected parse(std::string_view src) { Lexer L { src }; Table root; std::set> explicitTables; + std::map> headers; Table* current_table = &root; while (true) { @@ -542,6 +557,7 @@ std::expected parse(std::string_view src) { if (L.eof()) break; if (L.peek() == '[') { + const auto headerAt = L.position(); L.advance(); if (L.peek() == '[') { // [[dotted.path]] — array-of-tables (#227). Every occurrence @@ -557,9 +573,10 @@ std::expected parse(std::string_view src) { } L.advance(); L.advance(); - auto t = open_array_of_tables(root, *path, L.position()); + auto t = open_array_of_tables(root, *path, headerAt); if (!t) return std::unexpected(t.error()); explicitTables.insert(join_path(*path)); + headers.emplace(join_path(*path), headerAt); current_table = *t; continue; } @@ -576,6 +593,7 @@ std::expected parse(std::string_view src) { join_path(*path)), L.position()}); explicitTables.insert(join_path(*path)); + headers.emplace(join_path(*path), headerAt); current_table = t; } else { // key = value @@ -600,7 +618,7 @@ std::expected parse(std::string_view src) { } } - return Document{std::move(root), std::move(explicitTables)}; + return Document{std::move(root), std::move(explicitTables), std::move(headers)}; } std::expected parse_file(const std::filesystem::path& p) { diff --git a/modules/manifest/src/compat/registry.cppm b/modules/manifest/src/compat/registry.cppm index 66d52d232..aaf0f31ef 100644 --- a/modules/manifest/src/compat/registry.cppm +++ b/modules/manifest/src/compat/registry.cppm @@ -66,6 +66,21 @@ inline constexpr Deprecation kDeprecations[] = { "the package in the plugin's [dependencies]", "manifest/deprecated", "docs/31-authoring-a-rule-package.md", }, + { + "abi-capability", "2026.10.26.1", kRemovalFloor, + "an `abi:` runtime capability (`abi:glibc`, `abi:libc=`, " + "`abi:cxxstdlib=`), read as a requirement on a target-side layer", + "requires = [\"mcpp:c-abi=\"] or [\"mcpp:c++-abi=\"]", + "manifest/deprecated", "docs/22-target-side.md", + }, + { + "undeclared-cxx-runtime", "2026.10.26.1", kRemovalFloor, + "a C++ runtime other than the build's own, named on its link line or needed " + "by a library the line names, that no package declares", + "requires = [{ layer = \"mcpp:c++-abi=\", visibility = \"private\" }] " + "in the package that needs it", + "manifest/deprecated", "docs/22-target-side.md", + }, }; // The row named `id`, or null. diff --git a/modules/manifest/src/key_registry.cppm b/modules/manifest/src/key_registry.cppm index e137af2c4..df89c41e1 100644 --- a/modules/manifest/src/key_registry.cppm +++ b/modules/manifest/src/key_registry.cppm @@ -54,6 +54,7 @@ inline constexpr KeyClass kKeyRegistry[] = { {"build", "c_standard", KeyScope::Package, KeyMerge::Scalar, true, false}, {"build", "cache", KeyScope::Configuration, KeyMerge::Scalar, false, false}, {"build", "cflags", KeyScope::Package, KeyMerge::List, true, true }, + {"build", "compile_commands", KeyScope::Metadata, KeyMerge::Scalar, false, false}, {"build", "cxxflags", KeyScope::Package, KeyMerge::List, true, true }, {"build", "cxx_runtime", KeyScope::Configuration, KeyMerge::Scalar, true, false}, {"build", "default-profile", KeyScope::Configuration, KeyMerge::Scalar, false, false}, diff --git a/modules/manifest/src/targetside_model.cppm b/modules/manifest/src/targetside_model.cppm index 12004348f..b448dc171 100644 --- a/modules/manifest/src/targetside_model.cppm +++ b/modules/manifest/src/targetside_model.cppm @@ -488,6 +488,18 @@ struct Requirement { std::string requiredBy; // package id that stated it, for the diagnostic CapLayer layer; std::string interfaceName; // what that layer must resolve to + // `visibility = "private"` (design 2026-10-26 §6.3): the requirement + // holds for the package's own code and stops at its boundary, so a graph + // that resolves the layer otherwise does not refuse it -- the package is + // then an island (`cxx_islands`), verified separately. + bool privateScope = false; +}; + +// A package whose private C++ requirement the graph's C++ layer does not +// meet: its code runs on another C++ runtime, behind its C interface. +struct Island { + std::string requiredBy; + std::string interfaceName; }; // ── Interface enumeration: the resolution-time half of the capability model ── @@ -830,6 +842,8 @@ check_requirements(const TargetSide& ts, std::span reqs, std::string_view compilerStatedBy = {}) { constexpr std::string_view kPad = " "; for (auto const& r : reqs) { + // A private requirement does not bind the graph (see `cxx_islands`). + if (r.privateScope) continue; // An entry with no `=` asks only that the layer be // supplied by someone, which the absence check below still answers. auto const& have = layer_of(ts, r.layer); @@ -845,7 +859,7 @@ check_requirements(const TargetSide& ts, std::span reqs, r.layer == CapLayer::Compiler ? (compilerStatedBy.empty() ? std::format( - " Select that compiler for this project:\n" + " help: select that compiler for this project:\n" " [toolchain]\n" " default = \"{}\"\n" " or, for one target only:\n" @@ -862,9 +876,9 @@ check_requirements(const TargetSide& ts, std::span reqs, "configuration to do it.", compilerStatedBy, r.interfaceName)) : std::format( - " Depend on a package that declares `provides = " - "[\"mcpp:{}={}\"]`,\n" - " or remove the package that requires it.", + " help: depend on a package that declares `provides = " + "[\"mcpp:{}={}\"]`\n" + " help: or remove the package that requires it", name, r.interfaceName); if (have.absent()) @@ -1127,4 +1141,17 @@ inline std::string format_report(const TargetSide& ts, std::string_view targetNa return head + body; } +// The islands of a resolved target side: each private C++ requirement the +// graph's C++ layer does not meet. One it meets is no island -- the graph +// unifies where it can (U before I, design 2026-10-26 §6.3). +inline std::vector cxx_islands(const TargetSide& ts, std::span reqs) { + std::vector out; + for (auto const& r : reqs) { + if (!r.privateScope || r.layer != CapLayer::CxxAbi) continue; + if (!r.interfaceName.empty() && ts.cxx.interfaceName == r.interfaceName) continue; + out.push_back({r.requiredBy, r.interfaceName}); + } + return out; +} + } // namespace mcpp::targetside diff --git a/modules/manifest/src/toml.cppm b/modules/manifest/src/toml.cppm index df44c6ed5..dc48cfc0c 100644 --- a/modules/manifest/src/toml.cppm +++ b/modules/manifest/src/toml.cppm @@ -618,6 +618,82 @@ ManifestError error(const std::filesystem::path& origin, return ManifestError{msg, origin, pos.line, pos.column}; } +// A rejection that names its key -- `[build].sources must be …`, `[package] +// needs …`, the convention of these messages -- is located where that key (or +// that table's header) is written, so a reader of the message, or of the +// envelope, is taken to the line (design 2026-10-26 §5.6). The first +// `[table]` or `[table].key` in the message is the one it is about; a table +// whose name the message elides (`[target.'cfg(...)']`) is not located. +void locate_named_key(ManifestError& e, const t::Document& doc) { + const std::string_view msg = e.message; + const auto open = msg.find('['); + if (open == std::string_view::npos) return; + const auto close = msg.find(']', open); + if (close == std::string_view::npos || close == open + 1) return; + std::string dotted(msg.substr(open + 1, close - open - 1)); + std::string table = dotted; + std::size_t i = close + 1; + if (i + 1 < msg.size() && msg[i] == '.') { + std::size_t j = i + 1; + while (j < msg.size() && (std::isalnum(static_cast(msg[j])) + || msg[j] == '_' || msg[j] == '-')) + ++j; + if (j > i + 1) dotted += "." + std::string(msg.substr(i + 1, j - i - 1)); + } + auto at = doc.position_of(dotted); + if (!at && dotted != table) at = doc.position_of(table); + if (!at) return; + e.line = at->line; + e.column = at->column; +} + +// One `requires` array, of `[package]` or of a `[target.]` (design +// 2026-10-26 §6.3, D20): each entry a string, or a table +// `{ layer = "", visibility = "public" | "private" }`. A string +// is a public requirement, as it always was. `private` is accepted for the +// C++ layer alone -- a second C library in one image is not an island, it is +// two C libraries (openkal's boundary) -- and refused otherwise, naming +// `requires-private-layer`. `where` names the table for the messages. +std::expected +read_requires(const t::Value& v, std::string_view where, + std::vector& publicOut, std::vector& privateOut) { + if (!v.is_array()) + return std::unexpected(std::format( + "{} requires must be an array of capabilities, such as " + "[\"mcpp:compiler=llvm\"] or [{{ layer = \"mcpp:c++-abi=libstdc++\", " + "visibility = \"private\" }}]", where)); + for (auto const& e : v.as_array()) { + if (e.is_string()) { publicOut.push_back(e.as_string()); continue; } + if (!e.is_table()) + return std::unexpected(std::format( + "{} requires: an entry is a string or a table {{ layer, visibility }}", where)); + std::string layer, visibility = "public"; + for (auto const& [k, kv] : e.as_table()) { + if (k == "layer" && kv.is_string()) layer = kv.as_string(); + else if (k == "visibility" && kv.is_string()) visibility = kv.as_string(); + else + return std::unexpected(std::format( + "{} requires: `{}` is not a member of a requirement; the members are " + "`layer` (a string) and `visibility` (\"public\" or \"private\")", where, k)); + } + if (layer.empty()) + return std::unexpected(std::format("{} requires: a table entry needs `layer`", where)); + if (visibility == "public") { publicOut.push_back(layer); continue; } + if (visibility != "private") + return std::unexpected(std::format( + "{} requires: visibility = \"{}\"; the values are \"public\" and \"private\"", + where, visibility)); + if (!layer.starts_with("mcpp:c++-abi=")) + return std::unexpected(std::format( + "{} requires: visibility = \"private\" is for `mcpp:c++-abi=` only, " + "and `{}` is another layer [requires-private-layer].\n" + " A private C++ runtime stays behind its package's C interface; " + "one image has one C library and one compiler runtime.", where, layer)); + privateOut.push_back(layer); + } + return {}; +} + // #227 follow-up: libs/toml.cppm's `[[dotted.path]]` support is intentionally // schema-agnostic — it accepts an array-of-tables at ANY dotted path, so a // doubled-bracket typo like `[[dependencies]]` (single brackets meant) or @@ -919,7 +995,9 @@ std::expected parse_string(std::string_view content, if (!doc) { return std::unexpected(error(origin, doc.error().message, doc.error().where)); } - return parse_document(*doc, origin, ctx); + auto m = parse_document(*doc, origin, ctx); + if (!m && m.error().line == 0) locate_named_key(m.error(), *doc); + return m; } namespace { @@ -958,6 +1036,12 @@ std::expected parse_document(const t::Document& documen // reader below type-checks every entry, so nothing is silently // dropped on this path either way. "xlings.deps", + // `requires = [{ layer = "...", visibility = "private" }]` (design + // 2026-10-26 §6.3): an inline array whose entries are tables, the same + // Value shape as `[[package.requires]]`; `read_requires` type-checks + // every entry. Mixed arrays (strings and tables) never reach here. + "package.requires", + "target.*.requires", }; if (auto badPath = find_disallowed_array_of_tables(doc->root(), "", kAllowedArraysOfTables)) { return std::unexpected(error(origin, std::format( @@ -1413,11 +1497,12 @@ std::expected parse_document(const t::Document& documen } // [package] requires — validated exactly like `provides`: names under the // reserved prefix are a closed set, everything else passes through. - if (auto v = doc->get_string_array("package.requires")) { - for (auto const& entry : *v) + if (auto* rv = doc->get("package.requires")) { + if (auto r = read_requires(*rv, "[package]", m.requires_, m.privateRequires); !r) + return std::unexpected(error(origin, r.error())); + for (auto const& entry : m.requires_) if (auto cap = mcpp::targetside::parse_capability(entry); !cap) m.unknownCapabilities.push_back(entry); - m.requires_ = *v; } // [package] requires_abi -- see Manifest::requiresAbiThreads / // Manifest::requiresAbiExceptions. @@ -2789,6 +2874,7 @@ std::expected parse_document(const t::Document& documen if (auto v = doc->get_string("build.default-profile")) m.buildConfig.defaultProfile = *v; else if (auto v = doc->get_string("build.profile")) m.buildConfig.defaultProfile = *v; // accepted alias if (auto v = doc->get_string("build.cache")) m.buildConfig.cacheMode = *v; + if (auto v = doc->get_string("build.compile_commands")) m.buildConfig.compileCommands = *v; // [build] platform-dependencies — see Manifest::platformDependencies. // Validated HERE (unlike `cache`) because the value set is one word and // there is nothing build-mode-specific about it to keep out of the @@ -3836,7 +3922,7 @@ std::expected parse_document(const t::Document& documen static constexpr std::string_view kKnownTargetScalars[] = { "cxx_runtime", "linkage", "min_api_level", "sysroot", "toolchain", }; - static constexpr std::string_view kKnownTargetArrays[] = { "run_under", "runner" }; + static constexpr std::string_view kKnownTargetArrays[] = { "requires", "run_under", "runner" }; static constexpr std::string_view kKnownTargetTables[] = { "abi", "build", "build-dependencies", "dependencies", "dev-dependencies", "feature-deps", "feature-requires-abi", @@ -3876,6 +3962,22 @@ std::expected parse_document(const t::Document& documen "Per-role contracts go in [build].cxx_runtime's table form.", triple, key)); } + // `[target.] requires = [...]` (design 2026-10-26 §6.3): the + // target-axis form of `[package] requires`, symmetric with + // `requires_abi`; handed to this selector's conditional config + // below and unioned in when the predicate matches. + std::vector targetRequires, targetPrivateRequires; + if (auto rq = body.find("requires"); rq != body.end()) { + if (!rq->second.is_array()) + return std::unexpected(error(origin, std::format( + "[target.{}] requires must be an array of capabilities", triple))); + if (auto r = read_requires(rq->second, std::format("[target.{}]", triple), + targetRequires, targetPrivateRequires); !r) + return std::unexpected(error(origin, r.error())); + for (auto const& entry : targetRequires) + if (auto cap = mcpp::targetside::parse_capability(entry); !cap) + m.unknownCapabilities.push_back(entry); + } m.targetOverrides[canon_triple(triple)] = std::move(e); // [target..{build,dependencies,...}] — platform-conditional @@ -3884,6 +3986,8 @@ std::expected parse_document(const t::Document& documen // prepare_build. ConditionalConfig cc; cc.predicate = triple; + cc.requires_ = std::move(targetRequires); + cc.privateRequires = std::move(targetPrivateRequires); // `[target..runtime]` — the dialect-neutral link intent. Two // keys only, and the same two `[runtime]` already has at the top // level: this makes them per-target, it does not invent a vocabulary. diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index 432e80135..05c75d622 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -1137,6 +1137,13 @@ struct BuildConfig : BuildInputs { // error under --strict) rather than here, so parsing a manifest never // depends on the build-mode vocabulary. std::string cacheMode; + // `[build] compile_commands` -- where the project's compile database is + // published, relative to the project root (design 2026-10-26 D14). Absent: + // no copy outside target/ (the configuration's own database is always in + // its build directory). `~/.mcpp/config.toml` [build] compile_commands is + // the default and MCPP_COMPILE_COMMANDS overrides both. `""` states "none" + // over a machine default, which is why absent and empty are told apart. + std::optional compileCommands; // `[build] platform-dependencies` — "" (default, allowed) | "refuse". // design 2026-09-18 §6: the machine-checkable form of "this build is a // closure entirely on openkal (or whatever kernel-abi it names) and @@ -1628,6 +1635,11 @@ struct ConditionalConfig { // renders and where `false` overrides an outer `true`. bool requiresAbiThreads = false; bool requiresAbiExceptions = false; + // `[target.] requires = [...]` (design 2026-10-26 §6.3): the + // target-axis form of `[package] requires`, its public and private + // entries apart; unioned in by `merge_conditional_config`. + std::vector requires_; + std::vector privateRequires; // `[target..feature-requires-abi] = { ... }` -- the // per-feature form of the same requirement, on the target axis. Named // after `feature-deps`/`feature-xlings` below, the two existing @@ -1713,6 +1725,7 @@ inline bool is_empty(const ConditionalConfig& c) { && c.xlings.empty() && c.xlings.overrides.empty() && !c.abiThreadsDeclared && !c.abiExceptionsDeclared && !c.requiresAbiThreads && !c.requiresAbiExceptions + && c.requires_.empty() && c.privateRequires.empty() && c.featureRequiresAbiThreads.empty() && c.featureRequiresAbiExceptions.empty() // #717: `dialect_cxxflags` is a member of ConditionalConfig, not of // `c.inputs` (BuildInputs), so `is_empty(c.inputs)` above does not see @@ -2138,6 +2151,15 @@ struct Manifest { // through untouched, exactly as they do in `provides`. // The spelling is `requires_` because `requires` is a keyword. std::vector requires_; + // `requires = [{ layer = "mcpp:c++-abi=", visibility = "private" }]` + // (design 2026-10-26 §6.3, D20): a requirement that holds for THIS + // package's own code and stops at its boundary. Where the graph's C++ + // layer is another implementation, the package's runtime is an island + // behind a C interface -- accepted only where the engine can verify the + // island conditions (SPEC-002). `private` is accepted for `mcpp:c++-abi` + // alone: one image has one C library (openkal's boundary). Unioned from + // `[package]` and every matching `[target.]`, as `requires_` is. + std::vector privateRequires; // `requires_abi = { threads = true, exceptions = true }` at package // level, and per feature. A statement that the ARTEFACT's ABI has a // switch on, compared at resolution with the root's diff --git a/modules/manifest/src/xpkg.cppm b/modules/manifest/src/xpkg.cppm index c6c1139e7..6a1523471 100644 --- a/modules/manifest/src/xpkg.cppm +++ b/modules/manifest/src/xpkg.cppm @@ -72,6 +72,17 @@ std::vector merge_xpkg_versions_desc( // find the call sites that only knew three. std::vector all_xpkg_versions_desc(std::string_view luaContent); +// THE CATALOGUE FIELDS OF A DESCRIPTOR, for `mcpp search --format json` +// (design 2026-10-26 §5.6): what an editor's library browser shows next to the +// versions. Read from the text of the `package` table's own keys, as the +// version reader above reads its tables: a descriptor that states neither +// yields empty fields, never an error. +struct XpkgCatalogFields { + std::vector licenses; + std::string repo; +}; +XpkgCatalogFields xpkg_catalog_fields(std::string_view luaContent); + // How many versions a display line shows before truncating (#487). The full // list is one `mcpp search --all-versions` away; a hint only has to prove the // package is alive. @@ -1184,6 +1195,45 @@ std::string join_versions_desc(const std::vector& versions, return joined; } +XpkgCatalogFields xpkg_catalog_fields(std::string_view lua) { + XpkgCatalogFields out; + // The first ` =` at the start of a line (after indentation): the + // package table's own key, not a word inside a string or a nested table + // further down, which descriptors indent deeper and place later. + const auto key_at = [&](std::string_view key) -> std::size_t { + std::size_t pos = 0; + while ((pos = lua.find(key, pos)) != std::string_view::npos) { + std::size_t b = pos; + while (b > 0 && (lua[b - 1] == ' ' || lua[b - 1] == '\t')) --b; + std::size_t e = pos + key.size(); + while (e < lua.size() && (lua[e] == ' ' || lua[e] == '\t')) ++e; + if ((b == 0 || lua[b - 1] == '\n') && e < lua.size() && lua[e] == '=') return e + 1; + pos += key.size(); + } + return std::string_view::npos; + }; + const auto quoted = [&](std::size_t from, std::size_t until, std::vector& into) { + for (std::size_t p = from; p < until && p < lua.size(); ++p) { + if (lua[p] != '"') continue; + auto q = lua.find('"', p + 1); + if (q == std::string_view::npos || q > until) break; + into.emplace_back(lua.substr(p + 1, q - p - 1)); + p = q; + } + }; + if (auto at = key_at("licenses"); at != std::string_view::npos) { + auto open = lua.find('{', at), close = lua.find('}', at); + if (open != std::string_view::npos && close != std::string_view::npos && open < close) + quoted(open, close, out.licenses); + } + if (auto at = key_at("repo"); at != std::string_view::npos) { + std::vector one; + quoted(at, lua.find('\n', at), one); + if (!one.empty()) out.repo = one.front(); + } + return out; +} + std::vector all_xpkg_versions_desc(std::string_view luaContent) { std::vector> perPlatform; perPlatform.reserve(std::size(mcpp::platform::xpkg_platforms)); @@ -1454,8 +1504,47 @@ synthesize_from_xpkg_lua(std::string_view luaContent, } cur.skip_ws_and_comments(); while (!cur.eof() && cur.peek() != '}') { - auto s = cur.read_string(); - if (!s.empty()) dst->push_back(std::move(s)); + // A `requires` entry may be a table, `{ layer = "...", + // visibility = "private" }` (design 2026-10-26 §6.3): the same + // two members the TOML form takes, read the same way. + if (key == "requires" && cur.peek() == '{') { + cur.consume('{'); + std::string layer, visibility = "public"; + cur.skip_ws_and_comments(); + while (!cur.eof() && cur.peek() != '}') { + auto k = cur.read_key(); + cur.skip_ws_and_comments(); + if (!cur.consume('=')) + return std::unexpected(ManifestError{ + "expected '=' in a `requires` entry", m.sourcePath, 0, 0}); + auto v = cur.read_string(); + if (k == "layer") layer = std::move(v); + else if (k == "visibility") visibility = std::move(v); + else + return std::unexpected(ManifestError{std::format( + "`requires`: `{}` is not a member of a requirement; the members " + "are `layer` and `visibility`", k), m.sourcePath, 0, 0}); + cur.skip_ws_and_comments(); + cur.consume(','); + cur.skip_ws_and_comments(); + } + cur.consume('}'); + if (visibility == "private") { + if (!layer.starts_with("mcpp:c++-abi=")) + return std::unexpected(ManifestError{std::format( + "`requires`: visibility = \"private\" is for `mcpp:c++-abi=` " + "only, and `{}` is another layer [requires-private-layer]", layer), + m.sourcePath, 0, 0}); + m.privateRequires.push_back(std::move(layer)); + } else if (!layer.empty()) { + m.requires_.push_back(std::move(layer)); + } + } else { + auto s = cur.read_string(); + if (!s.empty()) dst->push_back(std::move(s)); + } + cur.skip_ws_and_comments(); + cur.consume(','); cur.skip_ws_and_comments(); } cur.consume('}'); diff --git a/modules/platform/src/env.cppm b/modules/platform/src/env.cppm index 8a47c70c3..daea25695 100644 --- a/modules/platform/src/env.cppm +++ b/modules/platform/src/env.cppm @@ -51,6 +51,32 @@ std::vector with_prefix(std::string_view prefix); // MCPP_OFFLINE. "0" and the empty string mean off, matching MCPP_VERBOSE. bool offline_mode(); +// THE EFFECTS THIS RUN MAY NOT HAVE (design 2026-10-26 §5.5, D8). +// +// `--deny ` and MCPP_DENY_EFFECTS name effects of docs/50 §4 the +// caller does not allow: `network`, `write-project`, `write-global-cache`, +// `exec-build-script`, `init-mcpp-home` (`read-project` cannot be denied). +// The two are a union, and nothing a project declares widens them: the +// constraint is the caller's. `--offline` is `--deny network`, and both +// spellings answer `offline_mode()`. A run that stops before a denied effect +// records it here, and the process exits 5 (SPEC-003): a caller tells "my +// constraint stopped it" from "it failed" by the status alone. +bool effect_denied(std::string_view effect); +// The deniable effects, for the CLI's validation of `--deny`. +const std::vector& deniable_effects(); +void note_effect_refused(std::string_view effect, std::string_view what); +// "network: install xim:gcc@15.1.0" -- the first refusal of this run, if any. +std::optional effect_refused(); + +// WHETHER THIS RUN WRITES THE GLOBAL CACHE. Answers kept under the home (a +// compiler's probe, the provider's handshake) are read by every command and +// written only by a run whose effects include `write-global-cache`: one that +// plans or builds (it says so with `note_writes_global_cache`) and was not +// denied it. A query -- `self doctor`, `self env`, `toolchain list` -- declares +// no such effect, so it leaves the home exactly as it found it. +void note_writes_global_cache(); +bool writes_global_cache(); + // The older, narrower spelling: MCPP_NO_AUTO_INSTALL gates ONLY the toolchain // auto-install, nothing else. Kept working (CI and existing setups export it) // and kept next to offline_mode() so the two network knobs cannot drift apart. @@ -140,7 +166,55 @@ bool offline_mode() { // Not cached: `mcpp test` runs nested mcpp invocations in-process in some // paths, and a test that flips the var must see the change. auto* v = std::getenv("MCPP_OFFLINE"); - return v && *v && std::string_view(v) != "0"; + return (v && *v && std::string_view(v) != "0") || effect_denied("network"); +} + +const std::vector& deniable_effects() { + static const std::vector names{ + "init-mcpp-home", "write-project", "write-global-cache", "network", "exec-build-script"}; + return names; +} + +bool effect_denied(std::string_view effect) { + auto* v = std::getenv("MCPP_DENY_EFFECTS"); + if (!v || !*v) return false; + std::string_view list(v); + while (!list.empty()) { + auto comma = list.find(','); + auto item = list.substr(0, comma); + while (!item.empty() && item.front() == ' ') item.remove_prefix(1); + while (!item.empty() && item.back() == ' ') item.remove_suffix(1); + if (item == effect) return true; + if (comma == std::string_view::npos) break; + list.remove_prefix(comma + 1); + } + return false; +} + +namespace { +std::mutex g_refusedMutex; +std::optional g_refused; +} + +void note_effect_refused(std::string_view effect, std::string_view what) { + std::lock_guard lock(g_refusedMutex); + if (!g_refused) g_refused = what.empty() ? std::string(effect) + : std::format("{}: {}", effect, what); +} + +std::optional effect_refused() { + std::lock_guard lock(g_refusedMutex); + return g_refused; +} + +namespace { +std::atomic g_writesGlobalCache{false}; +} + +void note_writes_global_cache() { g_writesGlobalCache.store(true); } + +bool writes_global_cache() { + return g_writesGlobalCache.load() && !effect_denied("write-global-cache"); } bool no_auto_install() { diff --git a/modules/platform/src/fs.cppm b/modules/platform/src/fs.cppm index abcee2e1f..ed9dbd2e2 100644 --- a/modules/platform/src/fs.cppm +++ b/modules/platform/src/fs.cppm @@ -96,9 +96,32 @@ bool append_atomically(const std::filesystem::path& file, std::string_view bytes // POSIX: flock(2) on a file descriptor class FileLock { public: - // Try to acquire an exclusive lock on /.lock. + // Try to acquire an exclusive lock on /.lock (or /). // Returns nullopt if another process holds the lock. - static std::optional try_acquire(const std::filesystem::path& dir); + static std::optional try_acquire(const std::filesystem::path& dir, + std::string_view name = ".lock"); + + // WAITS for the lock on /, which `try_acquire` never does. + // + // Once held, `owner` is published beside the lock as /.owner + // (replaced atomically): who holds it, for a waiter to name. A sidecar + // rather than the lock file's own bytes, because a Windows byte-range lock + // is mandatory and would refuse that read. While the lock is taken by + // another process, `waiting` is called once, after `notice` has passed, + // with what the holder published (empty when it published nothing). + // + // The wait polls with a backoff from 20 ms to 200 ms rather than blocking + // in the kernel: a blocking flock/LockFileEx cannot report the wait, and a + // waiter killed by a signal holds nothing either way. nullopt only when + // the lock file cannot be created at all (an unwritable directory, whose + // build fails on its own terms). + static std::optional acquire( + const std::filesystem::path& dir, std::string_view name, std::string_view owner, + std::chrono::milliseconds notice, + const std::function& waiting); + + // What the current holder of / published (see acquire). + static std::string holder(const std::filesystem::path& dir, std::string_view name); ~FileLock(); FileLock(FileLock&& o) noexcept; @@ -320,12 +343,54 @@ bool append_atomically(const std::filesystem::path& file, std::string_view bytes // ── FileLock ────────────────────────────────────────────────────────────── +std::optional FileLock::acquire( + const std::filesystem::path& dir, std::string_view name, std::string_view owner, + std::chrono::milliseconds notice, + const std::function& waiting) { + namespace fs = std::filesystem; + const auto start = std::chrono::steady_clock::now(); + auto pause = std::chrono::milliseconds(20); + bool told = false; + for (;;) { + if (auto lock = try_acquire(dir, name)) { + if (!owner.empty()) { + const auto record = dir / (std::string(name) + ".owner"); + auto tmp = record; + tmp += std::format(".{:x}", std::random_device{}()); + { + std::ofstream out(tmp, std::ios::binary | std::ios::trunc); + out << owner; + } + std::error_code ec; + fs::rename(tmp, record, ec); + if (ec) fs::remove(tmp, ec); + } + return lock; + } + // try_acquire could not even create the file: nothing to wait for. + std::error_code ec; + if (!fs::exists(dir / std::string(name), ec)) return std::nullopt; + if (!told && std::chrono::steady_clock::now() - start >= notice) { + told = true; + if (waiting) waiting(holder(dir, name)); + } + std::this_thread::sleep_for(pause); + pause = (std::min)(pause * 2, std::chrono::milliseconds(200)); // parenthesised: defines min + } +} + +std::string FileLock::holder(const std::filesystem::path& dir, std::string_view name) { + std::ifstream in(dir / (std::string(name) + ".owner"), std::ios::binary); + return in ? std::string(std::istreambuf_iterator(in), {}) : std::string{}; +} + #if defined(_WIN32) -std::optional FileLock::try_acquire(const std::filesystem::path& dir) { +std::optional FileLock::try_acquire(const std::filesystem::path& dir, + std::string_view name) { std::error_code ec; std::filesystem::create_directories(dir, ec); - auto lockPath = dir / ".lock"; + auto lockPath = dir / std::string(name); HANDLE h = CreateFileW(lockPath.wstring().c_str(), GENERIC_READ | GENERIC_WRITE, FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL); @@ -366,10 +431,11 @@ FileLock& FileLock::operator=(FileLock&& o) noexcept { #else // POSIX -std::optional FileLock::try_acquire(const std::filesystem::path& dir) { +std::optional FileLock::try_acquire(const std::filesystem::path& dir, + std::string_view name) { std::error_code ec; std::filesystem::create_directories(dir, ec); - auto lockPath = dir / ".lock"; + auto lockPath = dir / std::string(name); int fd = ::open(lockPath.c_str(), O_CREAT | O_RDWR | O_CLOEXEC, 0644); if (fd < 0) return std::nullopt; if (::flock(fd, LOCK_EX | LOCK_NB) != 0) { diff --git a/modules/platform/src/process.cppm b/modules/platform/src/process.cppm index 7e38ea43f..e2b3d7efa 100644 --- a/modules/platform/src/process.cppm +++ b/modules/platform/src/process.cppm @@ -393,6 +393,14 @@ std::string windows_wrap_for_cmd_c(std::string_view cmd); // AutoRun so a user's registry-installed shell hook cannot alter it. std::string windows_shell_command_line(std::string_view command); +// The files an interrupt leaves (design 2026-10-26 §5.4): when SIGINT, SIGTERM +// or SIGHUP (Windows: a console control event) ends this process, each +// `(path, content)` replaces its file -- through a temporary name beside it +// and a rename -- after the guarded children are stopped. Setting replaces the +// previous set; at most four files. +void set_interrupt_files(const std::vector>& files); +void clear_interrupt_files(); + } // namespace mcpp::platform::process // ─── Implementation ────────────────────────────────────────────────────── @@ -420,6 +428,28 @@ std::string windows_shell_command_line(std::string_view command) { return "cmd.exe /d /s /c " + windows_wrap_for_cmd_c(command); } +void set_interrupt_files(const std::vector>& files) { + std::vector paths, temps, contents; + for (auto const& [path, content] : files) { + paths.push_back(path.string()); + temps.push_back(path.string() + ".interrupted"); + contents.push_back(content); + } +#if defined(_WIN32) + mcpp::platform::winproc::set_interrupt_files(paths, temps, contents); +#else + mcpp::platform::unixproc::set_interrupt_files(paths, temps, contents); +#endif +} + +void clear_interrupt_files() { +#if defined(_WIN32) + mcpp::platform::winproc::clear_interrupt_files(); +#else + mcpp::platform::unixproc::clear_interrupt_files(); +#endif +} + namespace { // The section every child start holds from the creation of its pipe to the diff --git a/modules/platform/src/terminal.cppm b/modules/platform/src/terminal.cppm index 659dcdd87..fcdc7060f 100644 --- a/modules/platform/src/terminal.cppm +++ b/modules/platform/src/terminal.cppm @@ -169,6 +169,12 @@ private: // `emit build-database` held the caller's pipe as its descriptor 3. The copy // is therefore close-on-exec on POSIX and not inheritable on Windows, so a // child receives only descriptors 0 to 2, and 1 is standard error here. +// Writes `text` to the process's standard output as it was before any +// StdoutToStderr took it: a machine-readable stream (the build's NDJSON +// events) keeps the channel the protocol owns while the narration around it is +// redirected. +void write_protocol_stdout(std::string_view text); + class StdoutToStderr { public: StdoutToStderr(); @@ -511,6 +517,29 @@ std::vector KeyInput::read() { return keys; } +namespace { +std::atomic g_protocolStdout{-1}; +} + +void write_protocol_stdout(std::string_view text) { + const int fd = g_protocolStdout.load(); + if (fd < 0) { + std::fwrite(text.data(), 1, text.size(), stdout); + std::fflush(stdout); + return; + } + std::size_t done = 0; + while (done < text.size()) { +#if defined(_WIN32) + const int n = ::_write(fd, text.data() + done, static_cast(text.size() - done)); +#else + const auto n = ::write(fd, text.data() + done, text.size() - done); +#endif + if (n <= 0) return; + done += static_cast(n); + } +} + StdoutToStderr::StdoutToStderr() { std::fflush(stdout); #if defined(_WIN32) @@ -529,11 +558,17 @@ StdoutToStderr::StdoutToStderr() { saved_ = ::fcntl(1, F_DUPFD_CLOEXEC, 3); if (saved_ >= 0) ::dup2(2, 1); #endif + if (saved_ >= 0) { + int expected = -1; + g_protocolStdout.compare_exchange_strong(expected, saved_); + } } StdoutToStderr::~StdoutToStderr() { std::fflush(stdout); if (saved_ < 0) return; + int expected = saved_; + g_protocolStdout.compare_exchange_strong(expected, -1); #if defined(_WIN32) ::_dup2(saved_, 1); ::_close(saved_); diff --git a/modules/platform/src/unix/bounded_process.cppm b/modules/platform/src/unix/bounded_process.cppm index abf28ad93..1177015e0 100644 --- a/modules/platform/src/unix/bounded_process.cppm +++ b/modules/platform/src/unix/bounded_process.cppm @@ -238,6 +238,18 @@ void guard_terminal_mode(int fd); // Restores the saved mode and stops guarding it. void unguard_terminal_mode(); +// THE FILES A SIGNAL LEAVES (design 2026-10-26 §5.4). A build interrupted while +// it writes its directory says so in its result record: the interrupted form is +// rendered beforehand, and the handler -- after it has stopped the guarded +// groups -- writes each `contents[i]` to `temps[i]` and renames it over +// `paths[i]` (open, write, close and rename are async-signal-safe). While files +// are set the handler is installed. Setting replaces the previous set; at most +// four files. +void set_interrupt_files(const std::vector& paths, + const std::vector& temps, + const std::vector& contents); +void clear_interrupt_files(); + } // namespace mcpp::platform::unixproc namespace mcpp::platform::unixproc { @@ -430,6 +442,39 @@ volatile sig_atomic_t g_guardedGroups[kMaxGuardedGroups] = {}; // handler never takes it. std::mutex g_guardMutex; +// The files a signal leaves. A set is never freed while it may be read: the +// handler can run on another thread between the read of the pointer and its +// writes, so every set lives until exit, and `g_interrupt` only moves. +struct InterruptFiles { + std::vector paths, temps, contents; +}; +std::atomic g_interrupt{nullptr}; +std::vector> g_interruptSets; + +void write_interrupt_files() { + auto* set = g_interrupt.load(); + if (!set) return; + for (std::size_t i = 0; i < set->paths.size(); ++i) { + const int fd = ::open(set->temps[i].c_str(), O_WRONLY | O_CREAT | O_TRUNC | O_CLOEXEC, 0644); + if (fd < 0) continue; + const char* p = set->contents[i].data(); + std::size_t left = set->contents[i].size(); + while (left > 0) { + const auto n = ::write(fd, p, left); + if (n <= 0) break; + p += n; + left -= static_cast(n); + } + ::close(fd); + if (left == 0) ::rename(set->temps[i].c_str(), set->paths[i].c_str()); + else ::unlink(set->temps[i].c_str()); + } +} + +// Whether anything still needs the handler: a guarded group, the terminal's +// mode, or files to leave. +bool handler_needed(); + // The platforms without `pipe2` share one section across every child start. std::mutex g_launchMutex; @@ -438,6 +483,7 @@ volatile sig_atomic_t g_terminalFd = -1; struct termios g_terminalMode{}; void install_handlers(); +void uninstall_handlers(); extern "C" void background_signal_handler(int sig) { // killpg is async-signal-safe. SIGKILL rather than SIGTERM: this is the @@ -453,6 +499,7 @@ extern "C" void background_signal_handler(int sig) { const auto group = g_guardedGroups[i]; if (group > 0) ::killpg(static_cast(group), SIGKILL); } + write_interrupt_files(); if (g_terminalFd >= 0) ::tcsetattr(g_terminalFd, TCSANOW, &g_terminalMode); // Die of the signal we were sent, so the exit status is the one the shell // and any outer script expect from a Ctrl-C. @@ -612,27 +659,16 @@ void guard_group_on_signal(long long group) { void unguard_group(long long group) { if (group <= 0) return; std::lock_guard lock(g_guardMutex); - bool any = false; - for (int i = 0; i < kMaxGuardedGroups; ++i) { + for (int i = 0; i < kMaxGuardedGroups; ++i) if (g_guardedGroups[i] == static_cast(group)) g_guardedGroups[i] = 0; - else if (g_guardedGroups[i] != 0) - any = true; - } - if (!any && g_terminalFd < 0) { - ::signal(SIGINT, SIG_DFL); - ::signal(SIGTERM, SIG_DFL); - ::signal(SIGHUP, SIG_DFL); - } + if (!handler_needed()) uninstall_handlers(); } void clear_group_guard() { std::lock_guard lock(g_guardMutex); for (int i = 0; i < kMaxGuardedGroups; ++i) g_guardedGroups[i] = 0; - if (g_terminalFd >= 0) return; // the terminal's guard still needs the handler - ::signal(SIGINT, SIG_DFL); - ::signal(SIGTERM, SIG_DFL); - ::signal(SIGHUP, SIG_DFL); + if (!handler_needed()) uninstall_handlers(); } bool group_guard_idle() { @@ -654,13 +690,25 @@ void unguard_terminal_mode() { if (g_terminalFd < 0) return; ::tcsetattr(g_terminalFd, TCSANOW, &g_terminalMode); g_terminalFd = -1; - bool any = false; - for (int i = 0; i < kMaxGuardedGroups; ++i) any = any || g_guardedGroups[i] != 0; - if (!any) { - ::signal(SIGINT, SIG_DFL); - ::signal(SIGTERM, SIG_DFL); - ::signal(SIGHUP, SIG_DFL); - } + if (!handler_needed()) uninstall_handlers(); +} + +void set_interrupt_files(const std::vector& paths, + const std::vector& temps, + const std::vector& contents) { + if (paths.empty() || paths.size() > 4 || temps.size() != paths.size() + || contents.size() != paths.size()) + return; + std::lock_guard lock(g_guardMutex); + g_interruptSets.push_back(std::make_unique(InterruptFiles{paths, temps, contents})); + g_interrupt.store(g_interruptSets.back().get()); + install_handlers(); +} + +void clear_interrupt_files() { + std::lock_guard lock(g_guardMutex); + g_interrupt.store(nullptr); + if (!handler_needed()) uninstall_handlers(); } namespace { @@ -669,6 +717,17 @@ void install_handlers() { ::signal(SIGTERM, background_signal_handler); ::signal(SIGHUP, background_signal_handler); } +void uninstall_handlers() { + ::signal(SIGINT, SIG_DFL); + ::signal(SIGTERM, SIG_DFL); + ::signal(SIGHUP, SIG_DFL); +} +bool handler_needed() { + if (g_terminalFd >= 0 || g_interrupt.load()) return true; + for (int i = 0; i < kMaxGuardedGroups; ++i) + if (g_guardedGroups[i] != 0) return true; + return false; +} } // namespace #else @@ -697,6 +756,9 @@ void clear_group_guard() {} bool group_guard_idle() { return true; } void guard_terminal_mode(int) {} void unguard_terminal_mode() {} +void set_interrupt_files(const std::vector&, const std::vector&, + const std::vector&) {} +void clear_interrupt_files() {} #endif diff --git a/modules/platform/src/windows/bounded_process.cppm b/modules/platform/src/windows/bounded_process.cppm index 6ab1594f7..f061ed6f9 100644 --- a/modules/platform/src/windows/bounded_process.cppm +++ b/modules/platform/src/windows/bounded_process.cppm @@ -209,6 +209,15 @@ void guard_job_on_signal(unsigned long long job); void unguard_job(unsigned long long job); void clear_job_guard(); +// The files a Ctrl-C or a Ctrl-Break leaves (the POSIX peer states why): the +// console handler, after it has ended the guarded jobs, writes each +// `contents[i]` to `temps[i]` and moves it over `paths[i]`. While files are set +// the handler is installed. At most four files. +void set_interrupt_files(const std::vector& paths, + const std::vector& temps, + const std::vector& contents); +void clear_interrupt_files(); + // Wait for a child started by `spawn_background` and return its exit code. // Blocking, so an owned run costs no polling latency; `background_running` stays // for the supervisor that must not block. @@ -495,6 +504,39 @@ volatile unsigned long long g_guardedJobs[kMaxGuardedJobs] = {}; // console handler never takes it. std::mutex g_guardMutex; +// The files an interrupt leaves; every set lives until exit (the POSIX peer +// states why). +struct InterruptFiles { + std::vector paths, temps, contents; +}; +std::atomic g_interrupt{nullptr}; +std::vector> g_interruptSets; + +void write_interrupt_files() { + auto* set = g_interrupt.load(); + if (!set) return; + for (std::size_t i = 0; i < set->paths.size(); ++i) { + HANDLE f = ::CreateFileA(set->temps[i].c_str(), GENERIC_WRITE, 0, nullptr, + CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, nullptr); + if (f == INVALID_HANDLE_VALUE) continue; + DWORD written = 0; + const BOOL ok = ::WriteFile(f, set->contents[i].data(), + static_cast(set->contents[i].size()), &written, nullptr); + ::CloseHandle(f); + if (ok && written == set->contents[i].size()) + ::MoveFileExA(set->temps[i].c_str(), set->paths[i].c_str(), MOVEFILE_REPLACE_EXISTING); + else + ::DeleteFileA(set->temps[i].c_str()); + } +} + +bool handler_needed() { + if (g_interrupt.load()) return true; + for (int i = 0; i < kMaxGuardedJobs; ++i) + if (g_guardedJobs[i] != 0) return true; + return false; +} + BOOL WINAPI background_console_handler(DWORD) { // TerminateJobObject, not CloseHandle: this handler races `background_stop` // on the normal path, and terminating is idempotent while closing the same @@ -505,6 +547,7 @@ BOOL WINAPI background_console_handler(DWORD) { ::TerminateJobObject( reinterpret_cast(static_cast(job)), 1); } + write_interrupt_files(); return FALSE; // FALSE = also run the default handler, i.e. still exit } @@ -657,18 +700,33 @@ void guard_job_on_signal(unsigned long long job) { void unguard_job(unsigned long long job) { if (!job) return; std::lock_guard lock(g_guardMutex); - bool any = false; - for (int i = 0; i < kMaxGuardedJobs; ++i) { + for (int i = 0; i < kMaxGuardedJobs; ++i) if (g_guardedJobs[i] == job) g_guardedJobs[i] = 0; - else if (g_guardedJobs[i] != 0) any = true; - } - if (!any) ::SetConsoleCtrlHandler(background_console_handler, FALSE); + if (!handler_needed()) ::SetConsoleCtrlHandler(background_console_handler, FALSE); } void clear_job_guard() { std::lock_guard lock(g_guardMutex); for (int i = 0; i < kMaxGuardedJobs; ++i) g_guardedJobs[i] = 0; - ::SetConsoleCtrlHandler(background_console_handler, FALSE); + if (!handler_needed()) ::SetConsoleCtrlHandler(background_console_handler, FALSE); +} + +void set_interrupt_files(const std::vector& paths, + const std::vector& temps, + const std::vector& contents) { + if (paths.empty() || paths.size() > 4 || temps.size() != paths.size() + || contents.size() != paths.size()) + return; + std::lock_guard lock(g_guardMutex); + g_interruptSets.push_back(std::make_unique(InterruptFiles{paths, temps, contents})); + g_interrupt.store(g_interruptSets.back().get()); + ::SetConsoleCtrlHandler(background_console_handler, TRUE); +} + +void clear_interrupt_files() { + std::lock_guard lock(g_guardMutex); + g_interrupt.store(nullptr); + if (!handler_needed()) ::SetConsoleCtrlHandler(background_console_handler, FALSE); } namespace { @@ -755,6 +813,9 @@ void background_stop(unsigned long long, unsigned long long, long long) {} void guard_job_on_signal(unsigned long long) {} void unguard_job(unsigned long long) {} void clear_job_guard() {} +void set_interrupt_files(const std::vector&, const std::vector&, + const std::vector&) {} +void clear_interrupt_files() {} int wait_background(unsigned long long, int*) { return -1; } int run_foreground(const char*, unsigned long*) { return -1; } diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index 09ea24dc9..ff99a01c2 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.10.24.1"; +inline constexpr std::string_view MCPP_VERSION = "2026.10.26.1"; } // namespace mcpp diff --git a/src/bmi_cache.cppm b/src/bmi_cache.cppm index a96f4fec8..2aace2001 100644 --- a/src/bmi_cache.cppm +++ b/src/bmi_cache.cppm @@ -286,6 +286,27 @@ std::string now_iso8601() { return std::to_string(secs); } +// THE ENTRY'S FOOTPRINT, RECORDED WITH IT (design 2026-10-26 §8, E10): its +// bytes and its file count, the two numbers `mcpp cache list` reports. +// Computed when the entry is written, because the directory is complete then +// and already in the page cache; `cache list` used to walk every file of every +// entry -- 61 265 files for 580 entries, 376 ms -- to say the same thing. +// The entry.json being written is not counted; it is a few hundred bytes. +void record_footprint(nlohmann::json& j, const std::filesystem::path& dir) { + std::uintmax_t bytes = 0, files = 0; + std::error_code ec; + for (auto it = std::filesystem::recursive_directory_iterator(dir, ec); + !ec && it != std::filesystem::recursive_directory_iterator(); it.increment(ec)) { + std::error_code fe; + if (!it->is_regular_file(fe) || fe) continue; + if (it->path().filename() == "entry.json") continue; + bytes += it->file_size(fe); + ++files; + } + j["size"] = bytes; + j["files"] = files; +} + std::expected write_entry(const std::filesystem::path& path, const nlohmann::json& j) { auto tmp = path; @@ -361,6 +382,9 @@ void touch_accessed(const CacheKey& key) { auto j = read_entry(key.entryFile()); if (!j) return; (*j)["accessed"] = now_iso8601(); + // An entry written before the footprint was recorded gains it on its + // first use, so `cache list` -- which writes nothing -- stops measuring it. + if (!j->contains("size") || !j->contains("files")) record_footprint(*j, key.dir()); (void)write_entry(key.entryFile(), *j); } @@ -438,6 +462,7 @@ populate_from(const CacheKey& key, j["obj"] = std::move(objs); } j["accessed"] = now_iso8601(); + record_footprint(j, cacheDir); return write_entry(key.entryFile(), j); } @@ -517,6 +542,7 @@ publish_staged(const CacheKey& key, j["obj"] = std::move(objs); } j["accessed"] = now_iso8601(); + record_footprint(j, staged); // entry.json LAST: it is the sentinel, and it is written into the directory // before the directory has an address, so the entry is never seen without it. if (auto w = write_entry(staged / "entry.json", j); !w) { diff --git a/src/bmi_cache/maintenance.cppm b/src/bmi_cache/maintenance.cppm index d96f03a2a..8abb62efc 100644 --- a/src/bmi_cache/maintenance.cppm +++ b/src/bmi_cache/maintenance.cppm @@ -148,12 +148,15 @@ std::int64_t dir_mtime_seconds(const std::filesystem::path& p) { return (nowSys - age).count(); } +// Bytes and regular files under the entry, entry.json excepted -- the same +// two numbers bmi_cache's record_footprint stores when it writes an entry. void measure(Entry& e) { - e.size = dir_size(e.dir); std::error_code ec; - for (auto& _ : std::filesystem::recursive_directory_iterator(e.dir, ec)) { - (void)_; - if (ec) break; + for (auto it = std::filesystem::recursive_directory_iterator(e.dir, ec); + !ec && it != std::filesystem::recursive_directory_iterator(); it.increment(ec)) { + std::error_code fe; + if (!it->is_regular_file(fe) || fe || it->path().filename() == "entry.json") continue; + e.size += it->file_size(fe); ++e.fileCount; } } @@ -227,7 +230,17 @@ std::vector walk_pkg_entries(bool verify = false) { if (verify) check_pkg_files(e, *j); } if (e.accessed == 0) e.accessed = dir_mtime_seconds(e.dir); - measure(e); + // The footprint the entry recorded when it was written (see + // record_footprint); measured only for an entry from before + // the field. This command writes nothing, so the answer is not + // stored back here -- the entry's next use stores it. + if (j && j->contains("size") && (*j)["size"].is_number_unsigned() + && j->contains("files") && (*j)["files"].is_number_unsigned()) { + e.size = (*j)["size"].get(); + e.fileCount = static_cast((*j)["files"].get()); + } else { + measure(e); + } out.push_back(std::move(e)); } } diff --git a/src/build/backend.cppm b/src/build/backend.cppm index be859fbfa..3e4d3041c 100644 --- a/src/build/backend.cppm +++ b/src/build/backend.cppm @@ -78,6 +78,10 @@ struct BuildError { // failed; `diagnosticOutput` holds only what follows them (the advice // that reads the whole output). bool reported = false; + // Everything ninja wrote, the compilers' diagnostics included, whether or + // not they were reported as they failed: what the result record parses + // its diagnostics from (design 2026-10-26 §5.2). + std::string rawOutput; }; struct Backend { diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index 6c811364c..f263620a4 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -151,6 +151,12 @@ struct BuildProgramEnv { // key, the ABI tag, the toolchain fingerprint and `resolution.json`. It was // simply never handed to the layer that had to decide on it. std::string cxxStdlib; + // The root of the package of the graph that supplies the `c++-abi` layer + // (`gcc.libstdcxx` beside an llvm compiler), empty when the compiler's own + // payload supplies it (design 2026-10-26 §6.4). A rule package that drives + // a second compiler -- a SYCL device compiler -- points it at the same C++ + // library through this, so one process holds one C++ runtime. + std::string cxxAbiRoot; // Three more answers a board-support package would otherwise hardcode. // // THE COUPLING THESE REMOVE IS INVISIBLE IN A MANIFEST. `riscv-virt-rt` @@ -677,6 +683,7 @@ contract_env(const fs::path& root, const fs::path& outDir, const BuildProgramEnv e.emplace_back("MCPP_TOOLCHAIN_BINUTILS_DIR", env.toolchainBinutilsDir); e.emplace_back("MCPP_COMPILER", shared_value("MCPP_COMPILER")); e.emplace_back("MCPP_CXX_STDLIB", shared_value("MCPP_CXX_STDLIB")); + e.emplace_back("MCPP_CXX_ABI_ROOT", env.cxxAbiRoot); e.emplace_back("MCPP_TARGET_SYSROOT", env.targetSysroot); // #734 E2. Every key always, empty when it does not apply, for the reason // the lines above give. @@ -1175,6 +1182,16 @@ std::expected run_build_program_impl( // produces a baffling failure; a build PROGRAM that runs long is usually // stuck — waiting on a network read or spinning — and without a bound the // whole build hangs with no diagnostic at all. + // `--deny exec-build-script` (design 2026-10-26 §5.5): the effect is + // running code from the workspace, so the refusal is here, where a program + // would start -- a program whose answer is current runs nothing and is + // not refused. + if (mcpp::platform::env::effect_denied("exec-build-script")) { + mcpp::platform::env::note_effect_refused("exec-build-script", std::format("{} build program", who)); + return std::unexpected(std::format( + "effect denied: exec-build-script -- {}'s build program would run, and the caller " + "denied running code from the workspace (--deny exec-build-script)", who)); + } mcpp::build::progress::program_running(who, env.requested); programReport.runStart = ProgressClock::now(); bool timedOut = false; diff --git a/src/build/compile_commands.cppm b/src/build/compile_commands.cppm index 6720ec27f..0811f5cb0 100644 --- a/src/build/compile_commands.cppm +++ b/src/build/compile_commands.cppm @@ -47,6 +47,7 @@ import mcpp.platform.fs; import mcpp.platform; import mcpp.manifest.flag_words; import mcpp.toolchain.model; +import mcpp.ui; // the one-time note about a root database no longer asked for export namespace mcpp::build { @@ -195,6 +196,11 @@ write_compile_commands(const BuildPlan& plan, const CompileFlags& flags); // write. One function, so the two paths cannot word it differently. std::string foreign_entries_warning(std::size_t n); +// A root `compile_commands.json` an earlier mcpp published, in a project that +// no longer asks for one (`[build] compile_commands`, design 2026-10-26 D14): +// named once per project, with the key that keeps it. +void note_unrequested_root_compile_commands(const std::filesystem::path& projectRoot); + } // namespace mcpp::build namespace mcpp::build { @@ -752,7 +758,13 @@ write_compile_commands(const BuildPlan& plan, const CompileFlags& flags) { auto configPath = plan.outputDir / "compile_commands.json"; auto configResult = publish_compile_commands(configPath, fresh, fileExists); if (!configResult) return configResult; - if (!plan.publishRootCompileDb) return configResult; + if (!plan.publishRootCompileDb) { + // No copy is asked for (design 2026-10-26 D14): one left by an + // earlier mcpp is named once. A plan of several configurations, whose + // caller publishes the copy, asked for one and is not this case. + if (plan.rootCompileDb.empty()) note_unrequested_root_compile_commands(plan.projectRoot); + return configResult; + } // §3.2 items 2-3: the root is a copy of it, replaced whole. `emit` // (`mcpp emit build-database`) never reaches this function (SPEC-005 @@ -771,6 +783,46 @@ write_compile_commands(const BuildPlan& plan, const CompileFlags& flags) { rootResult->changed, configResult->commandCount, rootResult->foreignEntries}; } +// A ROOT DATABASE THIS PROJECT NO LONGER ASKS FOR (design 2026-10-26 D14). +// Until 2026.10.26.1 every build published `compile_commands.json` at the +// project root; a project that does not name one in `[build] +// compile_commands` keeps the copy it has, which mcpp no longer updates. Said +// once per project -- the mark is under target/ -- with the key that keeps it. +void note_unrequested_root_compile_commands(const std::filesystem::path& projectRoot) { + std::error_code ec; + const auto root = projectRoot / "compile_commands.json"; + const auto mark = projectRoot / "target" / ".compile-commands-noted"; + if (!std::filesystem::exists(root, ec) || std::filesystem::exists(mark, ec)) return; + { + std::ifstream in(root, std::ios::binary); + std::string head(4096, '\0'); + in.read(head.data(), static_cast(head.size())); + head.resize(static_cast(in.gcount())); + // mcpp's entries name the project's target/ as their directory. On + // Windows the file spells it with escaped backslashes, and a drive + // letter in either case, so both sides are compared in one spelling. + auto spelling = [](std::string t) { + std::string out; + for (std::size_t i = 0; i < t.size(); ++i) { + if (t[i] == '\\') { + if (i + 1 < t.size() && t[i + 1] == '\\') ++i; + out.push_back('/'); + } else { + out.push_back(static_cast(std::tolower(static_cast(t[i])))); + } + } + return out; + }; + if (spelling(head).find(spelling((projectRoot / "target").generic_string())) == std::string::npos) + return; + } + mcpp::ui::note("compile_commands.json at the project root is no longer updated: the " + "database is in target///. To keep a copy here, " + "add `compile_commands = \"compile_commands.json\"` under [build] in mcpp.toml"); + std::filesystem::create_directories(mark.parent_path(), ec); + std::ofstream(mark) << ""; +} + std::string foreign_entries_warning(std::size_t n) { return std::format( "compile_commands.json held {} entr{} mcpp did not write; " diff --git a/src/build/cxx_runtime.cppm b/src/build/cxx_runtime.cppm new file mode 100644 index 000000000..957488510 --- /dev/null +++ b/src/build/cxx_runtime.cppm @@ -0,0 +1,270 @@ +// mcpp.build.cxx_runtime — one C++ runtime per process, and the islands that +// are allowed to differ (design 2026-10-26 §6, D9–D13; SPEC-002 §6). +// +// TWO ITANIUM C++ RUNTIMES SHARE THEIR ABI NAMES. `__cxa_throw`, the +// personality routine, `operator new` and the `std::exception` family outside +// libc++'s `__1` namespace are spelled the same by libstdc++ and by +// libc++/libc++abi. In one process's global symbol space the two either split +// the ABI layer between them -- mcpp#838: the throw from one runtime, the +// catch from the other, and `catch (const std::exception&)` missed -- or +// interpose each other (mcpp#596). `std::runtime_error` and its relatives +// even have DIFFERENT LAYOUTS (a COW string in libstdc++, a refstring in +// libc++), so constructing one with one runtime and destroying it with the +// other is memory corruption. No link order makes "several images, two +// runtimes, exceptions crossing" reliable. +// +// So a build has ONE C++ runtime (U, the default), chosen through the target +// side; and a package that needs another declares it privately +// (`requires = [{ layer = "mcpp:c++-abi=", visibility = "private" }]`) +// and becomes an ISLAND, accepted only where the engine verifies: +// +// 1. one C++ image of ours (no C++ shared library in the build); +// 2. one unwinder (`--unwindlib=libgcc`, the runtime table); +// 3. the build's own runtime before the other on the link line; +// 4. island objects reference no layout-dependent member of the collision set; +// 5. the image imports no ABI-layer name at the other runtime's versions; +// 6. no typeinfo crosses: the image imports none from an image built on the +// other runtime (only `catch (...)` could catch its exceptions). +// +// 1 and 3 are decided when the build is planned, 2 by the runtime table, and +// 4–6 here, after the link, from the objects' and the image's symbol tables. +// A runtime named on a link line without a declaration (`-l:libstdc++.so.6`) +// is the same island, undeclared: verified alike, and reported so the package +// that wrote it declares it. +module; +#include + +export module mcpp.build.cxx_runtime; + +import std; +import mcpp.runtime.elf; + +export namespace mcpp::build::cxx_runtime { + +enum class Runtime { None, Libstdcxx, Libcxx }; + +std::string_view name_of(Runtime r); +Runtime runtime_of_stdlib(std::string_view stdlibId); +// The runtime a link word names: `-lstdc++`, `-l:libstdc++.so.6`, a path to +// `libstdc++.so.6`, `-lsupc++`; `-lc++`, `-lc++abi`, `libc++.so.1`, `libc++.a`. +Runtime runtime_named_by(std::string_view word); +// The runtime a DT_NEEDED soname is. +Runtime runtime_of_soname(std::string_view soname); + +// The two classes of the collision set (§6.3): names both runtimes define. +// The ABI layer is bound to ONE runtime as a whole; a layout-dependent name +// differs in object layout between the two and must not be referenced across. +bool abi_layer_symbol(std::string_view mangled); +bool layout_dependent_symbol(std::string_view mangled); + +// Is this version string one of a runtime's own symbol versions? +bool runtime_version(Runtime r, std::string_view version); + +// Conditions 4–6 for one linked image. `islandObjects` are the objects +// compiled against the other runtime; `searchDirs` locate the image's +// DT_NEEDED libraries. nullopt when every condition holds. +struct ImageCheck { + std::filesystem::path image; + Runtime own = Runtime::None; + Runtime foreign = Runtime::None; + std::vector islandObjects; + std::vector searchDirs; +}; +std::optional verify_image(const ImageCheck& in); + +} // namespace mcpp::build::cxx_runtime + +namespace mcpp::build::cxx_runtime { + +std::string_view name_of(Runtime r) { + switch (r) { + case Runtime::Libstdcxx: return "libstdc++"; + case Runtime::Libcxx: return "libc++"; + case Runtime::None: break; + } + return ""; +} + +Runtime runtime_of_stdlib(std::string_view id) { + if (id == "libstdc++") return Runtime::Libstdcxx; + if (id == "libc++") return Runtime::Libcxx; + return Runtime::None; +} + +namespace { +// A library's base name -- `stdc++` for `libstdc++.so.6`, `c++abi` for +// `libc++abi.a` -- or empty when `file` is not a library file name. +std::string_view library_stem(std::string_view file) { + if (!file.starts_with("lib")) return {}; + auto rest = file.substr(3); + for (std::string_view ext : {".so", ".a"}) { + if (auto at = rest.find(ext); at != std::string_view::npos) { + const auto tail = rest.substr(at + ext.size()); + if (tail.empty() || tail.front() == '.') return rest.substr(0, at); + } + } + return {}; +} +Runtime runtime_of_stem(std::string_view stem) { + if (stem == "stdc++" || stem == "supc++") return Runtime::Libstdcxx; + if (stem == "c++" || stem == "c++abi") return Runtime::Libcxx; + return Runtime::None; +} +} // namespace + +Runtime runtime_named_by(std::string_view word) { + if (word.starts_with("-l:")) return runtime_of_stem(library_stem(word.substr(3))); + if (word.starts_with("-l")) return runtime_of_stem(word.substr(2)); + if (word.starts_with("-")) return Runtime::None; + const auto slash = word.find_last_of("/\\"); + return runtime_of_stem(library_stem(slash == std::string_view::npos ? word : word.substr(slash + 1))); +} + +Runtime runtime_of_soname(std::string_view soname) { + return runtime_of_stem(library_stem(soname)); +} + +bool abi_layer_symbol(std::string_view s) { + return s.starts_with("__cxa_") || s.starts_with("__gxx_personality") + || s == "__dynamic_cast" || s.starts_with("_Znw") || s.starts_with("_Zna") + || s.starts_with("_Zdl") || s.starts_with("_Zda") + || s.starts_with("_ZTVN10__cxxabiv") || s.starts_with("_ZTIN10__cxxabiv") + || s.starts_with("_ZNSt9type_info") || s.starts_with("_ZNSt9exception") + || s.starts_with("_ZTISt9exception") || s.starts_with("_ZTVSt9exception") + || s.starts_with("_ZNSt9bad_alloc") || s.starts_with("_ZTISt9bad_alloc") + || s.starts_with("_ZTVSt9bad_alloc") || s.starts_with("_ZSt9terminate") + || s.starts_with("_ZSt13set_terminate") || s.starts_with("_ZSt14get_terminate"); +} + +bool layout_dependent_symbol(std::string_view s) { + // The classes of the `std::exception` family that hold a string, and so + // differ in layout between the two runtimes. Only names outside libc++'s + // `__1` namespace collide at all (`St` is `std::` itself). + static constexpr std::string_view kClasses[] = { + "St11logic_error", "St12domain_error", "St16invalid_argument", + "St12length_error", "St12out_of_range", "St13runtime_error", + "St11range_error", "St14overflow_error", "St15underflow_error", + }; + for (auto c : kClasses) + if (s.find(c) != std::string_view::npos) return true; + return false; +} + +bool runtime_version(Runtime r, std::string_view v) { + if (r == Runtime::Libstdcxx) return v.starts_with("GLIBCXX_") || v.starts_with("CXXABI_"); + return false; // libc++ and libc++abi publish unversioned symbols +} + +namespace { +std::optional locate(std::string_view soname, + const std::vector& dirs) { + std::error_code ec; + for (auto const& d : dirs) { + auto p = d / std::string(soname); + if (std::filesystem::exists(p, ec)) return p; + } + return std::nullopt; +} + +std::string listed(const std::vector& names) { + std::string out; + for (std::size_t i = 0; i < names.size() && i < 5; ++i) + out += std::format("\n {}", names[i]); + if (names.size() > 5) out += std::format("\n … and {} more", names.size() - 5); + return out; +} +} // namespace + +std::optional verify_image(const ImageCheck& in) { + namespace elf = mcpp::platform::elf; + const auto foreign = name_of(in.foreign); + const auto own = name_of(in.own); + + // Condition 4: island objects and the layout-dependent names. + std::vector layout; + for (auto const& o : in.islandObjects) { + auto undef = elf::undefined_object_symbols(o); + if (!undef) continue; + for (auto const& n : *undef) + if (layout_dependent_symbol(n)) layout.push_back(std::format("{} ({})", n, o.filename().string())); + } + if (!layout.empty()) + return std::format( + "{}: an object compiled against {} references a class whose layout differs in {} " + "[cxx-runtime-collision]:{}\n" + " Its C interface must not pass these across. Build the package against {} " + "(remove its private requirement), or build with a {} toolchain.", + in.image.filename().string(), foreign, own, listed(layout), own, foreign); + + auto imports = elf::versioned_dynamic_imports(in.image); + if (!imports) return std::nullopt; // not ELF, or unreadable: nothing to verify here + + // Condition 5: the ABI layer stays whole, on the image's own runtime. + std::vector split; + for (auto const& i : *imports) + if (abi_layer_symbol(i.name) && runtime_version(in.foreign, i.version)) + split.push_back(std::format("{}@{}", i.name, i.version)); + if (!split.empty()) + return std::format( + "{}: the C++ ABI layer is split between {} and {} [cxx-runtime-collision]: these " + "are imported from {}:{}\n" + " A throw from one runtime is then caught by the other, and a typed `catch` " + "misses (mcpp#838). The build's own runtime must answer them; this link names " + "{} before it.", + in.image.filename().string(), own, foreign, foreign, listed(split), foreign); + + // Condition 6: no typeinfo crosses. A typeinfo imported at the other + // runtime's versions is one of its own classes; one from a library that + // is built on the other runtime belongs to that library's exceptions, + // thrown by the other runtime's `__cxa_throw` -- foreign to this image's + // personality, so only `catch (...)` would catch them. + // + // The image's typeinfo is what it imports AND what it defines: a class + // whose type is caught usually has no key function, so the catch clause + // carries its own vague-linkage copy of the typeinfo and imports nothing + // -- matched at run time by name against the thrower's, which the other + // runtime's personality never gets to compare. + std::vector crossing; + std::set ourTypeinfo; + for (auto const& i : *imports) { + if (!i.name.starts_with("_ZTI")) continue; + if (runtime_version(in.foreign, i.version)) + crossing.push_back(std::format("{}@{}", i.name, i.version)); + else + ourTypeinfo.insert(i.name); + } + if (auto defined = elf::defined_object_symbols(in.image)) + for (auto const& n : *defined) + if (n.starts_with("_ZTI")) ourTypeinfo.insert(n); + if (auto facts = elf::inspect_elf_runtime(in.image)) { + auto dirs = in.searchDirs; + for (auto const& r : facts->runpaths) dirs.emplace_back(r); + for (auto const& needed : facts->needed) { + if (runtime_of_soname(needed) != Runtime::None) continue; + auto path = locate(needed, dirs); + if (!path) continue; + auto lib = elf::inspect_elf_runtime(*path); + if (!lib || std::ranges::none_of(lib->needed, [&](const std::string& n) { + return runtime_of_soname(n) == in.foreign; })) + continue; + auto defined = elf::inspect_dynamic_symbols(*path); + if (!defined) continue; + std::set theirs; + for (auto const& d : defined->defined) + if (d.name.starts_with("_ZTI")) theirs.insert(d.name); + for (auto const& t : ourTypeinfo) + if (!abi_layer_symbol(t) && theirs.contains(t)) + crossing.push_back(std::format("{} (also in {}, built on {})", t, needed, foreign)); + } + } + if (!crossing.empty()) + return std::format( + "{}: a type is caught or cast across C++ runtimes [cxx-runtime-collision]:{}\n" + " Exceptions of these types are thrown by {}, and this image's {} cannot match " + "them in a typed `catch`. Build with one runtime: depend on a package that provides " + "`mcpp:c++-abi={}`, or build with a toolchain whose runtime it is.", + in.image.filename().string(), listed(crossing), foreign, own, foreign); + return std::nullopt; +} + +} // namespace mcpp::build::cxx_runtime diff --git a/src/build/dirlock.cppm b/src/build/dirlock.cppm new file mode 100644 index 000000000..d3e6cc7bf --- /dev/null +++ b/src/build/dirlock.cppm @@ -0,0 +1,189 @@ +// mcpp.build.dirlock — the lock a configuration's build directory is written +// under (2026-10-26 design §4, mcpp#839). +// +// ONE LOCK PER BUILD DIRECTORY, HELD ONLY WHILE IT IS WRITTEN. +// +// Two builds of one configuration used to write one directory at once: each +// planned, each wrote build.ninja, and each ran ninja over the other's half +// written module files. Measured on a 24-module project: three rounds of five +// failed on both sides (`failed to write/read compiled module`). Every other +// shared, mutable thing a build touches was already protected (the index +// refresh, each BMI cache entry, each toolchain version directory); the build +// directory was the one that was not. +// +// The lock is taken where the directory starts being written -- the finish +// phase of `prepare_build`, once the fingerprint names the directory -- and is +// held through ninja. Planning before that (the manifest, resolution, the +// network, the toolchain, the scan) holds nothing, a different configuration +// has a different directory, and a query never waits. `mcpp run` releases it +// before the program starts (the descriptor is close-on-exec); `mcpp test` +// releases it before the first test runs. A crashed holder holds nothing: the +// operating system releases the lock with the process. +// +// A waiter is told who holds it once the wait has lasted 300 ms, which is what +// the owner record exists for: the process, the command, the caller's label +// (`MCPP_CLIENT`), and how long ago it started. +module; +#if defined(_WIN32) +#include +#else +#include +#endif + +export module mcpp.build.dirlock; + +import std; +import mcpp.platform; +import mcpp.ui; + +export namespace mcpp::build::dirlock { + +inline constexpr std::string_view kLockName = ".mcpp-build.lock"; + +// The holder a waiter is shown, parsed from the owner record. Fields absent +// from the record are empty. +struct Holder { + std::string pid; + std::string command; + std::string client; + std::int64_t startedAt = 0; // seconds since the epoch, 0 when unknown +}; + +Holder parse_holder(std::string_view record); + +// "pid 1234 (mcpp build, from mcpp-ide, 8s)" -- or "another mcpp" when the +// holder published nothing. +std::string describe(const Holder& h, std::int64_t now); + +// This process's owner record. +std::string owner_record(); + +// This process, as the owner record and the result record name it. +long process_id(); +const std::string& command(); + +// The command this process runs, as the owner record names it: set once from +// argv by the CLI, "mcpp" otherwise. +void set_command(std::string command); + +// Called once when a wait has lasted long enough to report (the event stream +// listens here; the human line is printed by `acquire` itself). +void set_waiting_observer(std::function observer); + +// Waits for the build directory's lock. Shared, because a BuildContext is +// copied, and the last copy to go releases it; `release` drops it early. +std::shared_ptr acquire(const std::filesystem::path& outputDir); + +} // namespace mcpp::build::dirlock + +namespace mcpp::build::dirlock { + +namespace { +std::string& command_slot() { + static std::string command = "mcpp"; + return command; +} +std::function& observer_slot() { + static std::function observer; + return observer; +} +long current_pid() { +#if defined(_WIN32) + return static_cast(::_getpid()); +#else + return static_cast(::getpid()); +#endif +} +std::int64_t now_seconds() { + return std::chrono::duration_cast( + std::chrono::system_clock::now().time_since_epoch()).count(); +} +} // namespace + +void set_command(std::string command) { command_slot() = std::move(command); } +const std::string& command() { return command_slot(); } +long process_id() { return current_pid(); } + +void set_waiting_observer(std::function observer) { + observer_slot() = std::move(observer); +} + +Holder parse_holder(std::string_view record) { + Holder h; + std::size_t pos = 0; + while (pos < record.size()) { + auto end = record.find('\n', pos); + if (end == std::string_view::npos) end = record.size(); + const auto line = record.substr(pos, end - pos); + pos = end + 1; + const auto eq = line.find('='); + if (eq == std::string_view::npos) continue; + const auto key = line.substr(0, eq); + const auto value = std::string(line.substr(eq + 1)); + if (key == "pid") h.pid = value; + else if (key == "command") h.command = value; + else if (key == "client") h.client = value; + else if (key == "started") { + std::int64_t v = 0; + const auto r = std::from_chars(value.data(), value.data() + value.size(), v); + if (r.ec == std::errc{}) h.startedAt = v; + } + } + return h; +} + +std::string describe(const Holder& h, std::int64_t now) { + if (h.pid.empty()) return "another mcpp"; + std::vector parts; + if (!h.command.empty()) parts.push_back(h.command); + if (!h.client.empty()) parts.push_back("from " + h.client); + if (h.startedAt > 0 && now >= h.startedAt) parts.push_back(std::format("{}s", now - h.startedAt)); + std::string out = "pid " + h.pid; + if (!parts.empty()) { + out += " ("; + for (std::size_t i = 0; i < parts.size(); ++i) out += (i ? ", " : "") + parts[i]; + out += ")"; + } + return out; +} + +std::string owner_record() { + std::string client = mcpp::platform::env::get("MCPP_CLIENT").value_or(""); + return std::format("pid={}\nstarted={}\ncommand={}\nclient={}\n", + current_pid(), now_seconds(), + command_slot(), client); +} + +// HELD ONCE PER PROCESS. One run may enter a configuration's directory more +// than once -- `mcpp run --format ` builds the program and then packs it, +// and the pack plans the same configuration -- and a second lock on the same +// file from the same process waits for itself (flock and LockFileEx lock per +// open file, not per process). The lock is shared while any holder keeps it. +std::mutex g_heldMutex; +std::map>& held() { + static std::map> m; + return m; +} + +std::shared_ptr acquire(const std::filesystem::path& outputDir) { + const auto key = outputDir.lexically_normal().generic_string(); + std::lock_guard heldLock(g_heldMutex); + if (auto existing = held()[key].lock()) return existing; + const auto start = std::chrono::steady_clock::now(); + auto lock = mcpp::platform::fs::FileLock::acquire( + outputDir, kLockName, owner_record(), std::chrono::milliseconds(300), + [&](std::string_view record) { + const auto holder = parse_holder(record); + mcpp::ui::status("Waiting", std::format("for the build directory, held by {}", + describe(holder, now_seconds()))); + if (auto& observe = observer_slot()) + observe(holder, std::chrono::duration_cast( + std::chrono::steady_clock::now() - start)); + }); + if (!lock) return nullptr; + auto shared = std::make_shared(std::move(*lock)); + held()[key] = shared; + return shared; +} + +} // namespace mcpp::build::dirlock diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 3b23073e1..4c1692ca2 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -30,6 +30,10 @@ import mcpp.freestanding.linkline; import mcpp.build.graph_shape; // #407: which mode wrote this build.ninja import mcpp.build.backend; import mcpp.build.ninja; +import mcpp.build.dirlock; // the build directory's lock (mcpp#839) +import mcpp.build.result; // the result record and the event stream (design 2026-10-26 §5) +import mcpp.wire; // the mcpp.launch envelope +import mcpp.build.cxx_runtime; // the island, verified after the link (design 2026-10-26 §6.3) import mcpp.build.flags; // profile_descriptor — one answer for the level (#694) import mcpp.build.runtime_validation; import mcpp.bmi_cache; @@ -357,6 +361,14 @@ export struct BuildCacheEntry { // empty identity, because an engine whose version is the empty string does // not exist and "no record" must not be representable as a value. std::optional engine; + // Where the build published the project's compile database (`[build] + // compile_commands` as resolved, design 2026-10-26 D14), relative to the + // project root; empty for none. A fast path publishes to the same place, + // without reading the manifest's key or the machine's default again. + // Recorded apart from the value because an entry written before the field + // must decline once rather than read as "none". + std::string rootCompileDb; + bool rootCompileDbRecorded = false; }; // One list of extensions as a line of the record carries it: joined by a unit @@ -556,6 +568,11 @@ export std::vector read_build_cache(const std::filesystem::path e.group = line.substr(6); haveNextLine = static_cast(std::getline(f, line)); } + if (haveNextLine && line.starts_with("compileDb=")) { + e.rootCompileDb = line.substr(10); + e.rootCompileDbRecorded = true; + haveNextLine = static_cast(std::getline(f, line)); + } // `engine=`, a tab, and the executable's path; see // EngineIdentity. The version holds no tab, so the path is everything // after the first one. Absent, or without the tab, the entry declines @@ -615,13 +632,20 @@ void write_build_cache(const std::filesystem::path& projectRoot, const std::string& toolchainRequest = {}, std::vector xlingsPayloads = {}, const std::string& selection = {}, - const std::string& group = {}) { + const std::string& group = {}, + const std::string& rootCompileDb = {}) { // The configuration groups of one workspace command build at the same // time (workspace design 2026-09-29 §6) and record into one file; the - // read, the edit and the write are one step. + // read, the edit and the write are one step -- within this process under + // the mutex, and across processes under the file's own lock (mcpp#839): + // two configurations built at once each read the file, and the second + // write dropped the first's entry. Held for one read and one write, never + // across a build. static std::mutex cacheWrite; std::lock_guard lock(cacheWrite); auto path = projectRoot / kBuildCacheFile; + auto fileLock = mcpp::platform::fs::FileLock::acquire( + path.parent_path(), ".build_cache.lock", {}, std::chrono::hours(1), {}); auto entries = read_build_cache(projectRoot); // Remove the existing entry for this (target, profile) pair. Keying on the @@ -651,6 +675,8 @@ void write_build_cache(const std::filesystem::path& projectRoot, newEntry.xlingsPayloadsRecorded = true; newEntry.selection = selection; newEntry.group = group; + newEntry.rootCompileDb = rootCompileDb; + newEntry.rootCompileDbRecorded = true; // The engine that wrote the graph is the one writing the record: both // happen in this process, after the graph was emitted. newEntry.engine = running_engine(); @@ -667,7 +693,19 @@ void write_build_cache_entries(const std::filesystem::path& path, const std::vector& entries) { std::error_code ec; std::filesystem::create_directories(path.parent_path(), ec); - std::ofstream f(path, std::ios::trunc); + // Written beside and renamed over, so a fast path reading the record while + // it is replaced sees the old file or the new one, never half of either. + auto tmp = path; + tmp += std::format(".{:x}", std::random_device{}()); + struct Publish { + std::filesystem::path tmp, path; + ~Publish() { + std::error_code e; + std::filesystem::rename(tmp, path, e); + if (e) std::filesystem::remove(tmp, e); + } + } publish{tmp, path}; + std::ofstream f(tmp, std::ios::trunc); if (!f) return; for (auto& e : entries) { f << "[target=" << e.targetTriple << "]\n"; @@ -717,6 +755,7 @@ void write_build_cache_entries(const std::filesystem::path& path, f << "selection=" << e.selection << '\n'; f << "group=" << e.group << '\n'; } + if (e.rootCompileDbRecorded) f << "compileDb=" << e.rootCompileDb << '\n'; if (e.engine) f << "engine=" << e.engine->version << '\t' << e.engine->exe << '\n'; } @@ -1212,6 +1251,157 @@ void report_freestanding_size(const BuildContext& ctx) { } } +// The raw output of the last failed ninja run of this process, for the +// diagnostics of the result record (the fast path's failure is reported when +// it happens, before the record is written). +std::string& failed_output() { + static std::string out; + return out; +} + +// THE ARTIFACTS OF A PLAN, AS THE RESULT RECORD STATES THEM (design 2026-10-26 +// §5.2, §5.3). A program of this package carries its launch: the argv and the +// environment `mcpp run` would give it -- what mcpp injects (the loader path, +// the subos's declarations), not the caller's environment. A program that a +// runner or an execution host wraps carries only `via`, and +// `mcpp run --print-launch --format json` resolves that one. +std::vector result_artifacts(const BuildContext& ctx) { + namespace res = mcpp::build::result; + std::vector out; + const bool wrapped = !choose_runner(ctx).tmpl.empty(); + std::vector> env; + if (auto [k, v] = compute_run_env(ctx.plan); !k.empty() && !v.empty()) env.emplace_back(k, v); + for (auto& kv : compute_subos_env(ctx.plan)) env.push_back(std::move(kv)); + for (auto const& lu : ctx.plan.linkUnits) { + std::string kind; + switch (lu.kind) { + case LinkUnit::Binary: kind = "bin"; break; + case LinkUnit::TestBinary: kind = "test"; break; + case LinkUnit::SharedLibrary: kind = "shared"; break; + case LinkUnit::StaticLibrary: kind = "static"; break; + default: continue; + } + res::Artifact a{lu.targetName, kind, ctx.outputDir / lu.output, std::nullopt}; + if (lu.kind == LinkUnit::Binary && lu.artifactOf.empty()) { + res::Launch l; + l.cwd = ctx.projectRoot.generic_string(); + if (!ctx.runUnder.empty()) l.via = "run_under"; + else if (wrapped) l.via = "runner"; + else { + l.argv = {a.path.string()}; + l.env = env; + } + a.launch = std::move(l); + } + out.push_back(std::move(a)); + } + return out; +} + +mcpp::build::result::Record start_record(const BuildContext& ctx) { + auto r = mcpp::build::result::begin(ctx.projectRoot, ctx.outputDir); + r.command = mcpp::build::dirlock::command(); + r.pid = mcpp::build::dirlock::process_id(); + r.triple = ctx.tc.targetTriple; + r.profile = ctx.profile; + mcpp::build::result::write(r); + mcpp::build::result::emit("plan", {{"configuration", { + {"triple", r.triple}, {"profile", r.profile}, {"fingerprint", r.fingerprint}, + {"outputDir", r.outputDir.generic_string()}}}}); + return r; +} + +// Ends the record: `ok` with the artifacts, or `failed` with the diagnostics +// read from the failed output. Each artifact and diagnostic is also an event. +void finish_record(mcpp::build::result::Record& r, bool ok, std::string_view failure, + std::vector artifacts) { + namespace res = mcpp::build::result; + r.finishedAt = res::now_ms(); + r.status = ok ? "ok" : "failed"; + if (ok) r.artifacts = std::move(artifacts); + else r.diagnostics = res::parse_compiler_diagnostics(failure); + std::error_code ec; + if (std::filesystem::exists(r.outputDir / "compile_commands.json", ec)) + r.compileDatabase = r.outputDir / "compile_commands.json"; + res::write(r); + if (res::events_enabled()) { + auto json = res::to_json(r); + for (auto const& d : json["diagnostics"]) res::emit("diagnostic", {{"diagnostic", d}}); + for (auto const& a : json["artifacts"]) res::emit("artifact", {{"artifact", a}}); + } +} + +// A fast path replays the graph the record's full build planned, so the +// artifacts are that build's; the record is restated with this run's status. +struct FastRecord { + mcpp::build::result::Record record; + std::vector artifacts; +}; +FastRecord fast_path_begin(const std::filesystem::path& projectRoot, + const std::filesystem::path& outputDir) { + namespace res = mcpp::build::result; + FastRecord f; + auto previous = res::read(outputDir); + f.record = res::begin(projectRoot, outputDir); + f.record.command = mcpp::build::dirlock::command(); + f.record.pid = mcpp::build::dirlock::process_id(); + if (previous) { + f.record.triple = previous->triple; + f.record.profile = previous->profile; + f.artifacts = std::move(previous->artifacts); + } + res::write(f.record); + res::emit("plan", {{"configuration", { + {"triple", f.record.triple}, {"profile", f.record.profile}, + {"fingerprint", f.record.fingerprint}, {"outputDir", f.record.outputDir.generic_string()}}}}); + return f; +} +void fast_path_end(FastRecord& f, bool ok) { + finish_record(f.record, ok, ok ? std::string_view{} : std::string_view(failed_output()), + ok ? std::move(f.artifacts) : std::vector{}); +} + +// THE C++ ISLAND, VERIFIED ON WHAT THE LINK PRODUCED (design 2026-10-26 +// §6.3, conditions 4–6; mcpp.build.cxx_runtime). Every C++ image of the plan: +// its objects from the island packages, its dynamic imports, and the +// libraries it needs. A failed condition is a build error: the image links, +// and then catches the wrong exceptions or none (mcpp#838). +std::optional verify_cxx_island(const BuildContext& ctx) { + namespace rt = mcpp::build::cxx_runtime; + const auto& plan = ctx.plan; + if (!plan.cxxIsland) return std::nullopt; + rt::ImageCheck check; + check.own = rt::runtime_of_stdlib(plan.cxxIsland->own); + check.foreign = rt::runtime_of_stdlib(plan.cxxIsland->foreign); + // The island's objects are what its packages' `role = "object"` actions + // produced -- a second compiler's output, built against the other runtime. + // Its packages' own sources were compiled by this build's compiler against + // this build's headers, and are not. + for (auto const& a : plan.actions) { + if (a.role != mcpp::manifest::BuildAction::Role::Object) continue; + if (std::ranges::find(plan.cxxIsland->packages, a.packageName) == plan.cxxIsland->packages.end()) + continue; + for (auto const& o : a.outputs) { + std::filesystem::path p(o); + check.islandObjects.push_back(p.is_absolute() ? p : plan.outputDir / p); + } + } + for (auto const& d : plan.runtimeLibraryDirs) check.searchDirs.push_back(d); + const auto words = mcpp::manifest::flag_words(plan.manifest.buildConfig.ldflags); + for (std::size_t i = 0; i < words.size(); ++i) { + if (words[i] == "-L" && i + 1 < words.size()) check.searchDirs.emplace_back(words[++i]); + else if (words[i].starts_with("-L") && words[i].size() > 2) check.searchDirs.emplace_back(words[i].substr(2)); + } + std::error_code ec; + for (auto const& lu : plan.linkUnits) { + if (lu.kind == LinkUnit::StaticLibrary || !mcpp::build::link_unit_holds_cxx(plan, lu)) continue; + check.image = plan.outputDir / lu.output; + if (!std::filesystem::exists(check.image, ec)) continue; + if (auto why = rt::verify_image(check)) return why; + } + return std::nullopt; +} + export int run_build_plan(BuildContext& ctx, bool verbose, bool no_cache, std::string_view targetOverride = "") { // `--cache=off` means a cold build: no global cache, and target/ cleared — @@ -1266,12 +1456,21 @@ export int run_build_plan(BuildContext& ctx, bool verbose, bool no_cache, // drive takes them from the plan and the open report. mcpp::build::BuildOptions opts; opts.verbose = verbose; + auto record = start_record(ctx); auto r = be->build(ctx.plan, opts); if (!r) { // A failed step was reported when it failed; what follows is the // advice that reads the whole output. if (!r.error().reported) mcpp::ui::error(r.error().message); mcpp::ui::block(r.error().diagnosticOutput); + finish_record(record, false, + r.error().rawOutput + "\n" + r.error().diagnosticOutput, {}); + return 1; + } + + if (auto why = verify_cxx_island(ctx)) { + mcpp::ui::error(*why); + finish_record(record, false, *why, {}); return 1; } @@ -1338,7 +1537,8 @@ export int run_build_plan(BuildContext& ctx, bool verbose, bool no_cache, v.push_back(p.generic_string()); return v; }(), - ctx.workspaceRequest, ctx.workspaceGroup); + ctx.workspaceRequest, ctx.workspaceGroup, + ctx.plan.rootCompileDb); } // The one place the --strict policy is settled. Degradations reported by @@ -1347,7 +1547,11 @@ export int run_build_plan(BuildContext& ctx, bool verbose, bool no_cache, // after the build rather than at the end of prepare_build. Without this // call the whole diag channel would report and then be ignored — the // exact failure mode it exists to prevent. - if (!mcpp::diag::flush(ctx.strict)) return 1; + if (!mcpp::diag::flush(ctx.strict)) { + finish_record(record, false, {}, {}); + return 1; + } + finish_record(record, true, {}, result_artifacts(ctx)); // The descriptor reads the level the compile realised, from the one // function `compute_flags` spells it from (#694), and from the plan's @@ -1627,6 +1831,7 @@ std::optional run_ninja_fast(const std::string& ninjaProgram, if (status != 0) { if (!reported && is_stale_ninja_failure(out)) return std::nullopt; + failed_output() = out; // A failed step was reported when it failed (and every line, under // --verbose); what remains is the advice below. if (!reported) mcpp::ui::error("build failed"); @@ -1813,11 +2018,16 @@ export int list_runners(const std::string& package_filter, // about (`write_compile_commands`), not a reason to fall back to the full // path. void restore_root_compile_commands(const std::filesystem::path& projectRoot, - const std::filesystem::path& outputDir) { + const std::filesystem::path& outputDir, + const std::string& rootCompileDb) { + if (rootCompileDb.empty()) { + mcpp::build::note_unrequested_root_compile_commands(projectRoot); + return; + } auto configPath = outputDir / "compile_commands.json"; std::error_code ec; if (!std::filesystem::exists(configPath, ec) || ec) return; - auto rootPath = projectRoot / "compile_commands.json"; + auto rootPath = (projectRoot / rootCompileDb).lexically_normal(); auto targetRoot = outputDir.parent_path().parent_path(); auto result = mcpp::build::publish_root_compile_commands( configPath, rootPath, targetRoot, mcpp::home::root()); @@ -1835,7 +2045,12 @@ void restore_root_compile_commands(const std::filesystem::path& projectRoot, // group, so the file does not depend on which group's build finished last. export void publish_workspace_compile_commands( const std::filesystem::path& wsRoot, - const std::vector& outputDirs) { + const std::vector& outputDirs, + const std::string& rootCompileDb) { + if (rootCompileDb.empty()) { + mcpp::build::note_unrequested_root_compile_commands(wsRoot); + return; + } std::vector databases; for (auto const& dir : outputDirs) { std::error_code ec; @@ -1844,7 +2059,7 @@ export void publish_workspace_compile_commands( } if (databases.empty()) return; auto result = mcpp::build::publish_root_compile_commands( - databases, wsRoot / "compile_commands.json", + databases, (wsRoot / rootCompileDb).lexically_normal(), outputDirs.front().parent_path().parent_path(), mcpp::home::root()); if (!result) { mcpp::ui::warning(std::format( @@ -2015,6 +2230,8 @@ admit_recorded_build(const BuildCacheEntry& e, const ReplayAsk& ask) { // exists. if (!e.depSourceRootsRecorded) return std::unexpected("the recorded build predates the list of path-dependency roots and their extension tables"); + if (!e.rootCompileDbRecorded) + return std::unexpected("the recorded build predates the location of the project's compile database"); if (dep_sources_newer_than(e.depSourceRoots, ninjaTime)) return std::unexpected(std::format("{} manifest or source is newer than build.ninja", ask.treesName)); if (!xlings_payloads_present(e)) return std::unexpected("a recorded xlings payload is missing"); @@ -2056,6 +2273,9 @@ export std::optional try_fast_build(const std::filesystem::path& projectRoo if (auto v = mcpp::platform::env::get("MCPP_MANAGED_ONLY"); v && *v != "0") return fast_path_declined("build", "MCPP_MANAGED_ONLY asks for the sources to be checked"); + // The full path refuses a denied write and names it (design 2026-10-26 §5.5). + if (mcpp::platform::env::effect_denied("write-project")) + return fast_path_declined("build", "the caller denied write-project"); auto want = fast_path_identity(projectRoot); if (!want) return fast_path_declined("build", "the manifest or the request could not be read"); @@ -2089,6 +2309,12 @@ export std::optional try_fast_build(const std::filesystem::path& projectRoo if (match->runtimeBinding && !match->runtimeBinding->baseIdentity.empty()) return fast_path_declined("build", "a stated target base is measured on the full path"); + // The directory is written by ninja below, under its lock (mcpp#839). + // Taken BEFORE admission: a build of this configuration that held it may + // have rewritten build.ninja (a test graph, another request) while this + // one waited, and the record is judged against what is there now. + auto buildLock = mcpp::build::dirlock::acquire(std::filesystem::path(match->outputDir)); + // The gates every fast path shares (see admit_recorded_build); what is left // here is the project's own tree. auto admitted = admit_recorded_build(*match, { @@ -2111,15 +2337,20 @@ export std::optional try_fast_build(const std::filesystem::path& projectRoo // restored here rather than left to whatever a NEXT full build happens to // do — a project that never edits a source again would otherwise never // see it back. - restore_root_compile_commands(projectRoot, outputDir); + restore_root_compile_commands(projectRoot, outputDir, match->rootCompileDb); std::chrono::milliseconds elapsed{}; + auto record = fast_path_begin(projectRoot, outputDir); auto rc = run_ninja_fast(admitted->ninjaProgram, outputDir, ninjaPath, verbose, runtimeEnvKey, runtimeEnvValue, &elapsed); if (!rc) return fast_path_declined("build", "ninja reported a stale graph"); - if (*rc != 0) return rc; + if (*rc != 0) { + fast_path_end(record, false); + return rc; + } if (!mcpp::build::runtime_validation::artifact_snapshot_unchanged( validatedBefore)) return fast_path_declined("build", "ninja relinked an artifact, whose closure the full path validates"); // relinked: full path reconstructs + validates closure + fast_path_end(record, true); // The descriptor the plan recorded (revision 3, §7.3); empty for a record // written before it was carried. @@ -2141,6 +2372,8 @@ export std::optional try_fast_workspace_build( const std::vector>& groups, bool verbose, bool no_cache) { if (no_cache) return fast_path_declined("workspace", "the build cache is off (--no-cache)"); + if (mcpp::platform::env::effect_denied("write-project")) + return fast_path_declined("workspace", "the caller denied write-project"); if (mcpp::platform::env::get("MCPP_LOCKED").value_or("") == "1") return fast_path_declined("workspace", "MCPP_LOCKED=1 asks for a locked resolution"); // `--managed-only` is checked against the sources prepare decides, which @@ -2159,10 +2392,11 @@ export std::optional try_fast_workspace_build( struct Ready { std::filesystem::path outputDir; - std::string ninjaProgram, runtimeEnvKey, runtimeEnvValue; + std::string ninjaProgram, runtimeEnvKey, runtimeEnvValue, rootCompileDb; mcpp::build::runtime_validation::ArtifactSnapshot validated; }; std::vector ready; + std::vector> locks; std::string profile; std::error_code ec; const auto wsToml = std::filesystem::last_write_time(wsRoot / "mcpp.toml", ec); @@ -2184,6 +2418,10 @@ export std::optional try_fast_workspace_build( && e.toolchainRecorded && e.toolchainRequest == want->toolchainRequest && e.selection == selection && e.group == group) { match = &e; break; } if (!match) return fast_path_declined("workspace", "no recorded build matches this selection"); + // Each group's directory under its own lock, taken before admission + // for the reason try_fast_build gives. Groups are visited in one + // order by every command, so two never wait on each other. + locks.push_back(mcpp::build::dirlock::acquire(std::filesystem::path(match->outputDir))); // The gates every fast path shares (see admit_recorded_build). A // workspace has no project tree of its own to sweep: its members are // recorded roots, each classified by its own package. @@ -2195,19 +2433,19 @@ export std::optional try_fast_workspace_build( }); if (!admitted) return fast_path_declined("workspace", admitted.error()); ready.push_back({admitted->outputDir, admitted->ninjaProgram, match->runtimeEnvKey, - match->runtimeEnvValue, std::move(admitted->validated)}); + match->runtimeEnvValue, match->rootCompileDb, std::move(admitted->validated)}); profile = want->profile; } if (ready.size() > 1) { std::vector dirs; for (auto const& r : ready) dirs.push_back(r.outputDir); - publish_workspace_compile_commands(wsRoot, dirs); + publish_workspace_compile_commands(wsRoot, dirs, ready.front().rootCompileDb); } // With more than one configuration, a package's line names its own. mcpp::build::progress::configurations(ready.size()); for (auto& r : ready) { - if (ready.size() == 1) restore_root_compile_commands(wsRoot, r.outputDir); + if (ready.size() == 1) restore_root_compile_commands(wsRoot, r.outputDir, r.rootCompileDb); std::chrono::milliseconds elapsed{}; auto rc = run_ninja_fast(r.ninjaProgram, r.outputDir, r.outputDir / "build.ninja", verbose, r.runtimeEnvKey, r.runtimeEnvValue, &elapsed); @@ -2257,9 +2495,21 @@ void yield_terminal() { // Returns nullopt when there's no usable cache entry (build_run_target // falls back to the full prepare_build path, which also refreshes the // cache for next time), an exit code otherwise. +// `mcpp run --print-launch` (design 2026-10-26 §5.3): build, then print how +// the program would be started -- the `mcpp.launch` envelope -- instead of +// starting it. Read at the one point where `run` composes the spawn, so the +// description is the run's own composition, not a second derivation of it. +bool& print_launch_flag() { + static bool on = false; + return on; +} +export void set_print_launch(bool on) { print_launch_flag() = on; } + std::optional try_fast_run(const std::filesystem::path& projectRoot, const std::optional& targetName, std::span passthrough) { + if (print_launch_flag()) + return fast_path_declined("run", "--print-launch describes the launch the full path composes"); // Same reason as try_fast_build's: this path skips resolution, and // `--locked` is an assertion about resolution. if (mcpp::platform::env::get("MCPP_LOCKED").value_or("") == "1") @@ -2268,6 +2518,8 @@ std::optional try_fast_run(const std::filesystem::path& projectRoot, // a fast path does not run (mcpp#755). if (auto v = mcpp::platform::env::get("MCPP_MANAGED_ONLY"); v && *v != "0") return fast_path_declined("run", "MCPP_MANAGED_ONLY asks for the sources to be checked"); + if (mcpp::platform::env::effect_denied("write-project")) + return fast_path_declined("run", "the caller denied write-project"); auto want = fast_path_identity(projectRoot); if (!want) return fast_path_declined("run", "the manifest or the request could not be read"); @@ -2330,6 +2582,11 @@ std::optional try_fast_run(const std::filesystem::path& projectRoot, } if (!chosen) return fast_path_declined("run", "the requested program is not among the recorded ones"); + // Under the directory's lock while ninja writes it, released before the + // program starts: a program that runs for an hour holds nothing, and a + // build of the same configuration meanwhile proceeds (mcpp#839). + auto buildLock = mcpp::build::dirlock::acquire(std::filesystem::path(match->outputDir)); + // The gates every fast path shares (see admit_recorded_build), for the // reason `mcpp run` is among the paths that need them: it execs the // artifact itself, so an entry that cannot say which engine wrote it, or @@ -2352,14 +2609,20 @@ std::optional try_fast_run(const std::filesystem::path& projectRoot, // Fresh → run ninja (picks up any incremental object/link work) then // exec the cached exe path directly. C1, same reason as try_fast_build's. - restore_root_compile_commands(projectRoot, outputDir); + restore_root_compile_commands(projectRoot, outputDir, match->rootCompileDb); + auto record = fast_path_begin(projectRoot, outputDir); auto rc = run_ninja_fast(admitted->ninjaProgram, outputDir, ninjaPath, /*verbose=*/false, match->runtimeEnvKey, match->runtimeEnvValue); if (!rc) return fast_path_declined("run", "ninja reported a stale graph"); - if (*rc != 0) return kRunBuildFailed; + if (*rc != 0) { + fast_path_end(record, false); + return kRunBuildFailed; + } if (!mcpp::build::runtime_validation::artifact_snapshot_unchanged( validatedBefore)) return fast_path_declined("run", "ninja relinked an artifact, whose closure the full path validates"); // never execute an artifact not validated for this binding + fast_path_end(record, true); + buildLock.reset(); auto exe = outputDir / chosen->second; auto pathCtx = mcpp::fetcher::make_path_ctx(/*cfg=*/nullptr, projectRoot); @@ -2619,6 +2882,22 @@ int run_artifact_via_runner(mcpp::build::BuildContext& ctx, // ENOEXEC is the kernel saying this host cannot load the artifact, and the // message carries the key that would change that. Anything else is reported // as itself — EACCES is a permission problem, not an absence. + // The build directory's lock goes before the program starts (mcpp#839): + // exec closes the descriptor on POSIX, but Windows waits for the child. + ctx.buildLock.reset(); + if (print_launch_flag()) { + mcpp::build::result::Launch launch{argv, childEnv, + std::filesystem::current_path().generic_string(), + !host.prefix.empty() ? "run_under" + : !choice.tmpl.empty() ? "runner" : ""}; + mcpp::wire::Envelope env{.kind = "mcpp.launch", + .effects = {mcpp::wire::Effect::ReadProject, + mcpp::wire::Effect::WriteProject}}; + env.data = mcpp::build::result::to_json(launch); + env.data["program"] = exe.generic_string(); + mcpp::wire::emit(env); + return 0; + } int spawnErr = 0; const int rc = mcpp::platform::process::run_foreground(argv, childEnv, &spawnErr); if (spawnErr != 0) { @@ -4231,7 +4510,7 @@ export void run_workspace_tests(std::span passthrough, if (sessions.size() > 1) { std::vector dirs; for (auto& s : sessions) dirs.push_back(s->ctx->outputDir); - publish_workspace_compile_commands(wsRoot, dirs); + publish_workspace_compile_commands(wsRoot, dirs, sessions.front()->ctx->plan.rootCompileDb); } // One record or line per group, before its first member's tests: the diff --git a/src/build/flags.cppm b/src/build/flags.cppm index 5744a22be..345bff092 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -65,6 +65,9 @@ struct CompileFlags { // artifact loaded a different build of a library than it linked against. // It reaches the line through `link_line::UnitTail::runtimeFallback`. std::string ldRuntimeFallback; + // The link words that name the island's C++ runtime (design 2026-10-26 + // §6.3), placed after the build's own runtime in each C++ unit's tail. + std::string ldForeignRuntime; // A compile-command prefix stated with a toolchain named by path // (`launcher = "ccache"`, mcpp#755): written in front of the compiler in // the build graph, and not into the compile database, whose consumers @@ -685,6 +688,17 @@ CompileFlags compute_flags(const BuildPlan& plan) { // traits — BMI mechanics + module-flag spellings auto caps = mcpp::toolchain::capabilities_for(plan.toolchain); const auto& d = mcpp::toolchain::dialect_for(plan.toolchain); + // THE C++ STANDARD LIBRARY THIS BUILD LINKS, which is the payload's unless + // a package supplies the C++ layer (design 2026-10-26 §6.2, D10). A + // package that supplies libstdc++ to an llvm build (`provides = + // ["mcpp:c++-abi=libstdc++"]`) brings a prebuilt library, so the link takes + // the libstdc++ cells of the runtime table with the package's library + // directory; a package that supplies libc++ is built from source and its + // objects are already on the line (`graphCxxRuntime`). `caps.stdlib_id` + // keeps describing the payload; this is what the link is about. + const bool graphLibstdcxx = plan.targetSide.cxx.fromGraph() + && plan.targetSide.cxx.interfaceName == "libstdc++"; + const std::string linkedStdlib = graphLibstdcxx ? std::string("libstdc++") : caps.stdlib_id; // macOS minimum supported OS version for produced binaries. // Precedence: MACOSX_DEPLOYMENT_TARGET env (explicit per-invocation @@ -974,8 +988,15 @@ CompileFlags compute_flags(const BuildPlan& plan) { link_toolchain_flags += lm.link_flags(ninjaEsc); link_toolchain_flags_c = link_toolchain_flags + std::string(mcpp::toolchain::ClangDriverModel::kLinkDriverFlagsC); - link_toolchain_flags += - mcpp::toolchain::ClangDriverModel::kLinkDriverFlags; + // A graph-supplied libstdc++ under the payload's clang: the driver + // links it (`-lstdc++` from the package's library directory) and + // libgcc's unwinder, which libstdc++.so is built against -- one + // unwinder per process (mcpp#596). The compiler runtime stays the + // payload's compiler-rt, crt objects included; no GCC installation is + // consulted (measured: NEEDED = libstdc++, libm, libgcc_s, libc). + link_toolchain_flags += graphLibstdcxx + ? std::string(" -stdlib=libstdc++ -fuse-ld=lld --rtlib=compiler-rt --unwindlib=libgcc") + : std::string(mcpp::toolchain::ClangDriverModel::kLinkDriverFlags); } f.sysroot = link_toolchain_flags; } else if (lm.mode != mcpp::toolchain::CLibMode::None) { @@ -1192,10 +1213,24 @@ CompileFlags compute_flags(const BuildPlan& plan) { // host and then escaped for ninja. A `$ORIGIN` therefore reaches the // linker as written, and an element that packs several tokens is several // words on every host. + // + // A word that names the OTHER C++ runtime (the island's, design 2026-10-26 + // §6.3) is taken out of this string and placed after the build's own + // runtime in each C++ unit's tail (`ldForeignRuntime`): condition 3. Left + // here it preceded the runtime archives, the shared library answered + // `__cxa_throw` and the personality first, and libc++abi answered the rest + // (mcpp#838, measured: `catch (const std::exception&)` missed). std::string user_ldflags; for (auto const& word : mcpp::manifest::flag_words(plan.manifest.buildConfig.ldflags)) { + const auto rendered = ninja_command_word(normalize_ldflag(plan.projectRoot, word)); + if (plan.cxxIsland && std::ranges::find(plan.cxxIsland->namedWords, word) + != plan.cxxIsland->namedWords.end()) { + f.ldForeignRuntime += ' '; + f.ldForeignRuntime += rendered; + continue; + } user_ldflags += ' '; - user_ldflags += ninja_command_word(normalize_ldflag(plan.projectRoot, word)); + user_ldflags += rendered; } // C standard. The file-level `$cflags` carries the engine default, a @@ -1418,7 +1453,7 @@ CompileFlags compute_flags(const BuildPlan& plan) { }; dist::MechanismInput mi; - mi.stdlibId = caps.stdlib_id; + mi.stdlibId = linkedStdlib; mi.hostIsWindows = mcpp::platform::is_windows; mi.fullStaticLibc = (f.linkage == "static"); // The CRT model this WHOLE project is being compiled with. Per-role @@ -1436,23 +1471,13 @@ CompileFlags compute_flags(const BuildPlan& plan) { // already settled where the machine's Xcode was consulted. mi.appleCrossTarget = !plan.toolchain.base.apple.root.empty(); mi.format = format; - // A SECOND C++ RUNTIME ON THIS LINE. - // - // Read off the link flags rather than inferred, because the only way - // one gets there is that something wrote it: a rule package whose - // second compiler is configured against libstdc++ emits - // `-l:libstdc++.so.6` through `mcpp::link_lib`, which lands in - // `bc.ldflags`. `mcpp.rules.sycl` and `mcpp.rules.hip` are the two - // that do today. - // - // The substring is `stdc++`, which covers `-lstdc++`, - // `-l:libstdc++.so.6` and a spelled-out path. It cannot match this - // toolchain's own runtime: the predicate requires libc++, and a - // libstdc++ toolchain never reaches the branch that reads this. - mi.foreignCxxRuntime = - caps.stdlib_id == "libc++" - && std::ranges::any_of(bc.ldflags, [](std::string_view flag) { - return flag.find("stdc++") != std::string_view::npos; }); + // A SECOND C++ RUNTIME BEHIND THIS LINE -- the plan's island (design + // 2026-10-26 §6.3), decided once where the graph and the link words + // are known: a package's private requirement, a word naming the other + // runtime, or a library on the line that needs it. It used to be a + // substring search for `stdc++` in these flags, which saw only the + // second of the three and nothing of the first or the third. + mi.foreignCxxRuntime = plan.cxxIsland.has_value(); // Bare metal short-circuits the whole table: the archives found below // are the HOST's, and linking them into a riscv64 image fails with // "incompatible with elf64lriscv". See MechanismInput::freestanding. @@ -1488,7 +1513,7 @@ CompileFlags compute_flags(const BuildPlan& plan) { // (`llvm.libcxx` on the iOS rows, mcpp#630), and asking the C library // there answered "payload" and linked the SDK's `-lc++` under objects // compiled against the package's headers. - mi.graphCxxRuntime = plan.targetSide.cxx.fromGraph(); + mi.graphCxxRuntime = plan.targetSide.cxx.fromGraph() && !graphLibstdcxx; // AND THE DRIVER'S HALF, WHICH IS ASKED FIRST (#634 A6). // @@ -1531,7 +1556,7 @@ CompileFlags compute_flags(const BuildPlan& plan) { || contracts.tests == dist::Contract::SelfContained || contracts.intermediate == dist::Contract::SelfContained || contracts.shared == dist::Contract::SelfContained) - && caps.stdlib_id == "libc++"; + && linkedStdlib == "libc++"; // THE ARCHIVES A LINKER SCRIPT OPENS, by file name. // // A driver may place an archive name on a linker script rather than on diff --git a/src/build/hostprogram.cppm b/src/build/hostprogram.cppm index c5b91c98a..1e197f123 100644 --- a/src/build/hostprogram.cppm +++ b/src/build/hostprogram.cppm @@ -537,6 +537,13 @@ inline const char* compiler() { return env_or("MCPP_COMPILER // two must not share a word. inline const char* cxx_stdlib() { return env_or("MCPP_CXX_STDLIB"); } +// The root of the package of the graph that supplies the C++ library +// (`gcc.libstdcxx` beside llvm), "" when the compiler's payload supplies it. +// A rule package that drives a second compiler points it here, so the device +// half and the host half of one program use one C++ library +// (design 2026-10-26 §6.4). +inline const char* cxx_abi_root() { return env_or("MCPP_CXX_ABI_ROOT"); } + // Where the TARGET's C library lives, for targets that have one of their own // (today: bare metal). Same argument one line up: the libc is a property of // the target, mcpp resolves it from the target's own row, and a package that diff --git a/src/build/link_line.cppm b/src/build/link_line.cppm index 0a10ae9a4..3600ba61d 100644 --- a/src/build/link_line.cppm +++ b/src/build/link_line.cppm @@ -74,6 +74,14 @@ struct UnitTail { // have always assumed. std::string cxxRuntime; + // 2b. A C++ runtime OTHER than this build's -- an island's (design + // 2026-10-26 §6.3) -- named on the line. After (2), so the build's own + // runtime answers the ABI layer first: an archive member is pulled + // only for a symbol still undefined where the archive appears, and a + // shared library named earlier would answer `__cxa_throw` first + // (mcpp#838). + std::string foreignCxxRuntime; + // 3. LAST-RESORT run-time search — today the SubOS library view ("the // farm"), a flat symlink tree that `xlings install` rewrites. // @@ -98,7 +106,7 @@ struct UnitTail { std::string render() const { std::string out; for (const std::string* slot : - {&dependencies, &cxxRuntime, &runtimeFallback, &loaderTag}) { + {&dependencies, &cxxRuntime, &foreignCxxRuntime, &runtimeFallback, &loaderTag}) { if (slot->empty()) continue; if (!slot->starts_with(' ')) out += ' '; out += *slot; @@ -107,7 +115,7 @@ struct UnitTail { } bool empty() const { - return dependencies.empty() && cxxRuntime.empty() + return dependencies.empty() && cxxRuntime.empty() && foreignCxxRuntime.empty() && runtimeFallback.empty() && loaderTag.empty(); } }; diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index f4ed20faa..6da2340b8 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -2132,10 +2132,47 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements, // choosing an object path. clang/gcc drivers, link.exe, GNU ar and llvm-ar // all accept @rspfile, so there is one rule shape everywhere; the response // file sits next to the output and `cat`ing it beats reading a 160 KB line. + // A LINKED ARTIFACT IS PUBLISHED WHOLE (design 2026-10-26 D22, E23). + // + // A process that starts a program while it is relinked used to see the + // linker's work in progress: GNU ld unlinks and then writes in place, so + // the name briefly holds a file that is not yet executable (EACCES, + // measured 4 896 times in 25 relinks with four launchers), gold writes in + // place (ETXTBSY), and macOS lets a running program's file be overwritten + // (ENOEXEC). That is what `mcpp test` meets when another build of the + // configuration relinks a test binary, and what makes it safe for an + // execution to hold no lock. So, on POSIX, the linker writes + // `.tmp` and a rename -- atomic within one directory -- replaces the + // name: a starter sees the old file or the new one (0 errors measured on + // Linux and macOS). The SONAME and install name are set explicitly from + // the final name (`soname_default`, `soname_flag`), so the temporary name + // reaches no artifact. + // + // A PE image names itself after its output file (a DLL's export + // directory), so a PE target is linked in place: from a POSIX host nothing + // runs it, and on a Windows host `mcpp link-output` first moves the old + // artifact aside -- Windows can rename, but not overwrite, a program that + // runs (see cmd_link_output). + const bool peTarget = [&] { + const auto t = mcpp::toolchain::triple::parse(plan.toolchain.targetTriple); + return t ? t->is_pe() : bool(mcpp::platform::is_windows); + }(); + auto publish_link = [&](std::string cmd) { + if constexpr (mcpp::platform::is_windows) { + return "$mcpp link-output --output $out -- " + cmd; + } else { + if (peTarget) return cmd; + const auto pos = cmd.find("-o $out"); + if (pos == std::string::npos) return cmd; + cmd.replace(pos, 7, "-o $out.tmp"); + return cmd + " && mv -f $out.tmp $out"; + } + }; { constexpr bool useRsp = true; auto link_rule = [&](std::string_view name, std::string cmd, std::string_view desc) { + if (desc != "AR") cmd = publish_link(std::move(cmd)); append(std::format("rule {}\n", name)); if (useRsp) { if (auto pos = cmd.find("$in"); pos != std::string::npos) @@ -3422,6 +3459,9 @@ std::string emit_ninja_string(const BuildPlan& plan, std::string* placements, tail.cxxRuntime = cxxUnit ? uflags.ldStdlibFor(role_of(lu.kind)) : uflags.ldStdlibCFor(role_of(lu.kind)); + // The island's runtime, after the build's own (design 2026-10-26 + // §6.3, condition 3). An archive never reads this tail. + if (lu.kind != LinkUnit::StaticLibrary) tail.foreignCxxRuntime = uflags.ldForeignRuntime; // An archive has no run-time search path of its own: `ar` never // reads `$unit_ldflags`, so rpath flags there would be dead bytes // in every graph that builds a static library. @@ -4877,6 +4917,7 @@ std::expected NinjaBackend::build(const BuildPlan& plan BuildError failure{"build failed", plan.outputDir / "build.ninja", std::move(diagnostics)}; failure.reported = reported; + failure.rawOutput = out; return std::unexpected(std::move(failure)); } return r; diff --git a/src/build/plan.cppm b/src/build/plan.cppm index 86ea1ec0f..fc4968715 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -416,6 +416,21 @@ struct BuildPlan { // `openkal-llvm` family name, which had to be invented to carry a fact // about the dependency graph on an object that describes a compiler. mcpp::targetside::TargetSide targetSide; + // THE C++ ISLAND OF THIS BUILD, when it has one (design 2026-10-26 §6.3): + // a C++ runtime other than the build's own reaches the link -- declared by + // packages' private `mcpp:c++-abi` requirements, named on the link line, + // or needed by a library the line names. The runtime table then embeds + // the build's own runtime hidden with one unwinder, the named words follow + // it on the line, and every C++ image is verified after the link + // (mcpp.build.cxx_runtime). Absent: one runtime, nothing to verify. + struct CxxIsland { + std::string own; // "libc++" / "libstdc++" + std::string foreign; + std::vector packages; // the packages that declared it + std::vector namedWords; // link words naming the foreign runtime + std::vector neededBy; // libraries on the line that NEED it + }; + std::optional cxxIsland; // Which graph this plan will write into build.ninja. The fingerprint does // NOT cover dev-deps or test targets, so `mcpp build` and `mcpp test` // share an output directory and overwrite each other's graph; this is what @@ -486,6 +501,10 @@ struct BuildPlan { // configurations, which publishes the root file itself, once, from all of // them (`publish_root_compile_commands` over a list). bool publishRootCompileDb = true; + // `[build] compile_commands` as resolved (resolve_compile_commands): + // relative to the project root, empty when no copy is published. Recorded + // with the build so a fast path publishes to the same place. + std::string rootCompileDb; // See StdModuleUnit above. Empty when the build does not import `std`. std::vector stdModuleUnits; // GCC only: a specs file that replaces the pristine `*link:`, so the diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 81719eb86..6ccfcb5bb 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -539,6 +539,12 @@ export struct BuildContext { // not, which let a plain build serve a featured artefact. std::string activeFeatureRequest; std::filesystem::path outputDir; + // THE BUILD DIRECTORY'S LOCK (mcpp.build.dirlock, mcpp#839): taken when + // the finish phase starts writing `outputDir`, held through ninja, and + // dropped (`reset()`) before a program or a test runs. Shared because a + // context is copied; the last copy releases it. Null for a planning pass + // that describes (plan_only), which works in its own directory. + std::shared_ptr buildLock; std::filesystem::path stdBmi; std::filesystem::path stdObject; // plan_only: what the std module build WOULD be (sources are on the @@ -720,6 +726,14 @@ void with_member(BuildContext& ctx, std::string_view owner, F&& fn) { fn(); } +// WHERE THE PROJECT'S COMPILE DATABASE IS PUBLISHED (design 2026-10-26 D14), +// relative to the project root; empty when it is not. MCPP_COMPILE_COMMANDS +// ("off" or "none" for no copy), then `[build] compile_commands` of the +// manifest ("" for no copy), then the machine's default from config.toml. +// The configuration's own database is in its build directory either way. +export std::string resolve_compile_commands(const mcpp::manifest::Manifest& m, + std::string_view machineDefault); + export CacheMode resolve_cache_mode(const mcpp::manifest::Manifest& m, std::string_view override_mode); diff --git a/src/build/prepare/config.cpp b/src/build/prepare/config.cpp index 73045c15f..c4705c3aa 100644 --- a/src/build/prepare/config.cpp +++ b/src/build/prepare/config.cpp @@ -365,6 +365,14 @@ void merge_conditional_config(mcpp::manifest::Manifest& m, // text rides along so the unmet-requirement check can name what // asked, the same courtesy `[package] requires_abi` gets by naming // "the package" and a feature's entry by naming the feature. + // `[target.] requires` (design 2026-10-26 §6.3): unioned, as + // `[package] requires` and every other matching selector's entries + // are -- each is a true statement about this target. + for (auto const& r : cc.requires_) + if (std::ranges::find(m.requires_, r) == m.requires_.end()) m.requires_.push_back(r); + for (auto const& r : cc.privateRequires) + if (std::ranges::find(m.privateRequires, r) == m.privateRequires.end()) + m.privateRequires.push_back(r); if (cc.requiresAbiThreads) m.targetRequiresAbiThreads.push_back(cc.predicate); if (cc.requiresAbiExceptions) diff --git a/src/build/prepare/driver.cpp b/src/build/prepare/driver.cpp index d0f59746b..447108744 100644 --- a/src/build/prepare/driver.cpp +++ b/src/build/prepare/driver.cpp @@ -83,6 +83,9 @@ prepare_build(bool print_fingerprint, bool includeDevDeps, std::vector extraTargets, BuildOverrides overrides) { + // A run that plans writes the global cache (its declared effects say so), + // including the answers a compiler probe and the provider's handshake keep. + mcpp::platform::env::note_writes_global_cache(); if (!overrides.toolchain_statement.empty()) return prepare_build_pass(print_fingerprint, includeDevDeps, std::move(extraTargets), std::move(overrides)); diff --git a/src/build/prepare/features.cpp b/src/build/prepare/features.cpp index 3f66c3fdc..fee3e1e3a 100644 --- a/src/build/prepare/features.cpp +++ b/src/build/prepare/features.cpp @@ -1761,6 +1761,7 @@ static std::expected step6_dependency_build_programs(PrepareS // package from naming a toolchain. One call, so a new answer // reaches every build program at once — see fill_target_build_env. fill_target_build_env(bpEnv, *state.m, state.tc ? &*state.tc : nullptr, state.cfg_opt ? &*state.cfg_opt : nullptr); + apply_graph_cxx_abi(bpEnv, state.packages); bpEnv.toolsBin = state.projectSubosBin; bpEnv.profile = state.effectiveProfile; bpEnv.accel = state.resolvedAccel(); diff --git a/src/build/prepare/fetch.cpp b/src/build/prepare/fetch.cpp index 33bf57a7e..eed014cc9 100644 --- a/src/build/prepare/fetch.cpp +++ b/src/build/prepare/fetch.cpp @@ -482,15 +482,18 @@ provision_xlings_addresses(const mcpp::config::GlobalConfig& cfg, // it is the whole of the actionable content. if (called && r->error && !r->error->hint.empty()) why += "\n " + r->error->hint; - // Shaped like the toolchain failure: say what failed and - // hand back a command the user can run themselves. An + // xlings' own diagnostic, with its hint, IS the report + // (design 2026-10-26 D17). This used to end with "you can + // install them manually with: xlings install …" -- the + // same install, which meets the same refusal, so the one + // line of advice was the one that could not help. An // ambiguous bare name ("mesa" matching two repos) lands // here, and xlings' own message names the candidates. return std::unexpected(std::format( "provisioning {} failed: {}\n" - " you can install them manually with:\n" - " xlings install {}", - label, why, join_deps(" "))); + " the whole failure is in ~/.mcpp/log/mcpp.log; `mcpp self doctor` " + "reports the state of mcpp's package home", + label, why)); } // What xlings resolved each address to, where it says so // (protocol 1.1). Recorded per address so every later lookup diff --git a/src/build/prepare/graph_load.cpp b/src/build/prepare/graph_load.cpp index a6505140b..b4e004c19 100644 --- a/src/build/prepare/graph_load.cpp +++ b/src/build/prepare/graph_load.cpp @@ -636,7 +636,7 @@ static void step4a_define_candidate_selection_closures(PrepareState& state) { // answered and how old it is, so the next step is obvious // instead of guessed at. if (auto cfgA = state.get_cfg(true)) { - hint += std::format("\n index: {}\n hint: `mcpp index update` " + hint += std::format("\n index: {}\n help: `mcpp index update` " "if it was published recently", mcpp::pm::staleness_note( mcpp::config::make_xlings_env(**cfgA))); diff --git a/src/build/prepare/host_graph.cpp b/src/build/prepare/host_graph.cpp index 7f8af1c03..51fc852a2 100644 --- a/src/build/prepare/host_graph.cpp +++ b/src/build/prepare/host_graph.cpp @@ -362,7 +362,7 @@ std::expected check_program_imports(PrepareState& state, std: return std::unexpected(std::format( "build.mcpp imports '{}'\n" " provided by: {}, feature \"{}\" (not enabled)\n" - " hint: enable it on the dependency edge: {} = {{ ..., " + " help: enable it on the dependency edge: {} = {{ ..., " "features = [\"{}\"] }}", want, qualified(pm), feature, qualified(pm), feature)); } diff --git a/src/build/prepare/local_toolchain.cpp b/src/build/prepare/local_toolchain.cpp index 305c51858..0ebc841bf 100644 --- a/src/build/prepare/local_toolchain.cpp +++ b/src/build/prepare/local_toolchain.cpp @@ -370,6 +370,7 @@ std::expected step6_toolchain_phase(PrepareState& state) { bpEnv.targetTriple = state.resolvedTargetCanonical; fill_target_build_env(bpEnv, *state.m, state.tc ? &*state.tc : nullptr, state.cfg_opt ? &*state.cfg_opt : nullptr); + apply_graph_cxx_abi(bpEnv, state.packages); bpEnv.toolsBin = state.projectSubosBin; bpEnv.profile = state.effectiveProfile; fill_package_build_env(bpEnv, *state.m); diff --git a/src/build/prepare/manifest.cpp b/src/build/prepare/manifest.cpp index 38e7af873..85869f776 100644 --- a/src/build/prepare/manifest.cpp +++ b/src/build/prepare/manifest.cpp @@ -403,7 +403,7 @@ std::expected check_engine_floors(const PrepareState& state, ? m.package.name : m.package.namespace_ + "." + m.package.name; return std::unexpected(std::format( "package '{}' ({}) requires mcpp >= {}; this is mcpp {}.\n" - " hint: pin \"mcpp\": \"{}\" (or newer) in .xlings.json and run " // provider-boundary-ok: the project file a person edits to pin mcpp + " help: pin \"mcpp\": \"{}\" (or newer) in .xlings.json and run " // provider-boundary-ok: the project file a person edits to pin mcpp "`xlings install`, or run `xlings install mcpp@{}`", who, where, m.package.mcppFloor, mcpp::MCPP_VERSION, m.package.mcppFloor, m.package.mcppFloor)); @@ -667,6 +667,10 @@ std::expected phase0_manifest_and_workspace(PrepareState& sta lockPath = state.runtimeWorkspaceRoot / mp / "mcpp.lock"; break; } + if (auto text = mcpp::pm::read_text(lockPath)) { + state.lockReadPath = lockPath; + state.lockReadText = std::move(*text); + } if (std::filesystem::exists(lockPath)) { if (auto lock = mcpp::pm::load(lockPath); lock) { for (auto const& p : lock->packages) { diff --git a/src/build/prepare/options.cpp b/src/build/prepare/options.cpp index fce6b8c6b..7f67949d3 100644 --- a/src/build/prepare/options.cpp +++ b/src/build/prepare/options.cpp @@ -80,13 +80,31 @@ std::string_view cache_mode_name(CacheMode m) { } } +std::string resolve_compile_commands(const mcpp::manifest::Manifest& m, + std::string_view machineDefault) { + if (const char* e = std::getenv("MCPP_COMPILE_COMMANDS"); e && *e) { + const std::string_view v(e); + return v == "off" || v == "none" ? std::string{} : std::string(v); + } + if (m.buildConfig.compileCommands) return *m.buildConfig.compileCommands; + return std::string(machineDefault); +} + CacheMode resolve_cache_mode(const mcpp::manifest::Manifest& m, std::string_view override_mode) { - if (auto v = parse_cache_mode(override_mode)) return *v; - if (const char* e = std::getenv("MCPP_BUILD_CACHE"); e && *e) - if (auto v = parse_cache_mode(e)) return *v; - if (auto v = parse_cache_mode(m.buildConfig.cacheMode)) return *v; - return CacheMode::Global; + // `--deny write-global-cache` (design 2026-10-26 §5.5): the build goes on + // without the shared dependency cache, as `--cache local` does -- the cache + // is an optimisation, and refusing the build would deny more than asked. + const auto mode = [&] { + if (auto v = parse_cache_mode(override_mode)) return *v; + if (const char* e = std::getenv("MCPP_BUILD_CACHE"); e && *e) + if (auto v = parse_cache_mode(e)) return *v; + if (auto v = parse_cache_mode(m.buildConfig.cacheMode)) return *v; + return CacheMode::Global; + }(); + if (mode == CacheMode::Global && mcpp::platform::env::effect_denied("write-global-cache")) + return CacheMode::Local; + return mode; } std::string resolve_profile_name(const mcpp::manifest::Manifest& m, diff --git a/src/build/prepare/plan.cpp b/src/build/prepare/plan.cpp index 9373cc851..b9ff7cff0 100644 --- a/src/build/prepare/plan.cpp +++ b/src/build/prepare/plan.cpp @@ -10,10 +10,15 @@ import mcpp.build.prepare_inputs; import std; import mcpp.diag; import mcpp.build.stage; +import mcpp.build.dirlock; +import mcpp.build.cxx_runtime; // the C++ island (design 2026-10-26 §6.3) +import mcpp.toolchain.provider; // capabilities_for: the payload's C++ runtime +import mcpp.runtime.elf; // DT_NEEDED of a library the line names import mcpp.build.refusal; import mcpp.build.version_floor; import mcpp.home; import mcpp.platform.axis; +import mcpp.platform.env; // --deny write-project import mcpp.libs.json; import mcpp.log; import mcpp.manifest; @@ -514,6 +519,112 @@ static std::expected step13_make_plan(PrepareState& state, Bu return {}; } +// THE C++ ISLAND (design 2026-10-26 §6.3, D13): whether a C++ runtime other +// than this build's own reaches its link, and from where. Three sources, one +// answer: a package's private `mcpp:c++-abi` requirement the graph does not +// meet (declared), a link word naming the other runtime (`-l:libstdc++.so.6`, +// typed by name rather than found by substring), and a library the line names +// whose DT_NEEDED is the other runtime. The last two are islands no package +// declared -- verified the same way, and reported to the author. +// +// Condition 1 is decided here, where the link units exist: an island is one +// C++ image of ours with the other runtime behind it, so a build that also +// links a C++ shared library has several images on two runtimes, which no +// link order makes reliable -- refused, naming the library. +static std::expected step13_cxx_islands(PrepareState& state, BuildContext& ctx) { + namespace rt = mcpp::build::cxx_runtime; + auto& plan = ctx.plan; + if (!state.tc) return {}; + const auto ownName = plan.targetSide.cxx.fromGraph() && plan.targetSide.cxx.interfaceName == "libstdc++" + ? std::string("libstdc++") + : mcpp::toolchain::capabilities_for(*state.tc).stdlib_id; + const auto own = rt::runtime_of_stdlib(ownName); + if (own == rt::Runtime::None) return {}; + const auto other = own == rt::Runtime::Libcxx ? rt::Runtime::Libstdcxx : rt::Runtime::Libcxx; + + mcpp::build::BuildPlan::CxxIsland island; + island.own = ownName; + island.foreign = std::string(rt::name_of(other)); + // By qualified name, which is how an action names the package it belongs + // to: the island's objects are the outputs of its `role = "object"` + // actions -- a second compiler's, built against the other runtime. + for (auto const& i : state.cxxIslands) { + if (rt::runtime_of_stdlib(i.interfaceName) != other) continue; + for (auto const& pkg : state.packages) { + const auto id = pkg.manifest.package.version.empty() + ? pkg.manifest.package.name + : std::format("{}@{}", pkg.manifest.package.name, pkg.manifest.package.version); + if (id == i.requiredBy) { + island.packages.push_back(mcpp::build::qualified_package_name(pkg.manifest)); + break; + } + } + } + // The link words, and the libraries they name. + std::vector searchDirs; + std::vector libs; + const auto words = mcpp::manifest::flag_words(plan.manifest.buildConfig.ldflags); + for (std::size_t i = 0; i < words.size(); ++i) { + const auto& w = words[i]; + if (rt::runtime_named_by(w) == other) { island.namedWords.push_back(w); continue; } + if (w == "-L" && i + 1 < words.size()) searchDirs.emplace_back(words[++i]); + else if (w.starts_with("-L") && w.size() > 2) searchDirs.emplace_back(w.substr(2)); + else if (w.starts_with("-l:")) libs.push_back(w.substr(3)); + else if (w.starts_with("-l") && w.size() > 2) libs.push_back("lib" + w.substr(2) + ".so"); + else if (!w.starts_with("-") && w.find(".so") != std::string::npos) libs.push_back(w); + } + for (auto const& lib : libs) { + std::filesystem::path path(lib); + std::error_code ec; + if (!path.is_absolute()) { + path.clear(); + for (auto const& d : searchDirs) + if (std::filesystem::exists(d / lib, ec)) { path = d / lib; break; } + } + if (path.empty() || !std::filesystem::exists(path, ec)) continue; + auto facts = mcpp::platform::elf::inspect_elf_runtime(path); + if (!facts) continue; + for (auto const& n : facts->needed) + if (rt::runtime_of_soname(n) == other) { + island.neededBy.push_back(path.filename().string()); + break; + } + } + if (island.packages.empty() && island.namedWords.empty() && island.neededBy.empty()) return {}; + + for (auto const& lu : plan.linkUnits) { + if (lu.kind != mcpp::build::LinkUnit::SharedLibrary || !mcpp::build::link_unit_holds_cxx(plan, lu)) + continue; + mcpp::build::refusal::record(mcpp::build::refusal::Code::LayerRequirement); + std::string from = !island.packages.empty() ? island.packages.front() + : !island.namedWords.empty() ? "`" + island.namedWords.front() + "`" + : island.neededBy.front(); + return std::unexpected(std::format( + "{} brings {} into a build whose C++ runtime is {}, and `{}` is a C++ shared library of " + "this build [cxx-runtime-island].\n" + " Two runtimes are reliable only behind one C++ image of ours: with several " + "images, the two runtimes' ABI names (`__cxa_throw`, the `std::exception` family) are " + "shared across the process and each image may bind the other's (SPEC-002 §6).\n" + " Build with one runtime: depend on a package that provides `mcpp:c++-abi={}`, " + "or build with a toolchain whose runtime is {}; or make `{}` static.", + from, island.foreign, island.own, lu.targetName, island.foreign, island.foreign, + lu.targetName)); + } + // COMPAT(undeclared-cxx-runtime): accepted as the island it is, verified + // alike, and reported so the package that wrote the word declares it. + if (island.packages.empty()) { + std::string what = !island.namedWords.empty() + ? std::format("the link word `{}` names", island.namedWords.front()) + : std::format("{}, named on the link line, needs", island.neededBy.front()); + state.reportCompatUse("undeclared-cxx-runtime", 0, std::format( + "{} {}, a C++ runtime other than this build's ({}), and no package declares it; " + "it is accepted as an island and verified after the link, which", what, + island.foreign, island.own)); + } + plan.cxxIsland = std::move(island); + return {}; +} + static std::expected step13_cxx_private_runtime(PrepareState& state, BuildContext& ctx) { // A DEPENDENCY'S C++ SHARED LIBRARY OVER A C++ RUNTIME THAT IS A PACKAGE // (#641, item 5). @@ -909,7 +1020,21 @@ static void step13_graph_and_schedule(PrepareState& state, BuildContext& ctx) { ? *state.targetRuntimeBinding : state.runtimeBindingSnapshot; ctx.plan.runtimeBinding = artifactRuntime; mcpp::build::merge_runtime_binding_contract(ctx.plan, artifactRuntime); - ctx.plan.compileDbPath = state.workRoot / "compile_commands.json"; + // The project's copy of the compile database, where its manifest (or the + // machine's default) says; none otherwise, and then `${mcpp.compile_db}` + // names the configuration's own (design 2026-10-26 D14). + { + std::string machineDefault; + if (auto c = state.get_cfg(/*requireBootstrap=*/false)) machineDefault = (*c)->compileCommands; + const auto rel = resolve_compile_commands(*state.m, machineDefault); + ctx.plan.rootCompileDb = rel; + if (rel.empty()) { + ctx.plan.compileDbPath = ctx.outputDir / "compile_commands.json"; + ctx.plan.publishRootCompileDb = false; + } else { + ctx.plan.compileDbPath = (state.workRoot / rel).lexically_normal(); + } + } // GCC: a clean `*link:` for this build, so the payload's specs cannot // inject other homes' rpath entries into the artifact. AFTER the plan is // moved in — an earlier assignment was silently overwritten by that move, @@ -1822,15 +1947,39 @@ static std::expected step13_abi_enforcement(PrepareState& sta // before the dep graph, so this enforces/diagnoses rather than reselects — // abi-driven reselection is a resolution-ordering follow-up. { - const auto prof = mcpp::toolchain::abi_profile(ctx.tc); + auto prof = mcpp::toolchain::abi_profile(ctx.tc); + // COMPAT(abi-capability): the C library and the C++ runtime are + // target-side layers (design 2026-10-26 §6.7, D12), so the two + // dimensions are read from the RESOLVED target side -- which a graph + // package can supply -- and not from the payload's profile. An llvm + // build whose C++ layer is a graph libstdc++ satisfies + // `abi:cxxstdlib=libstdc++`; read off the payload it was refused. + if (!ctx.plan.targetSide.cAbi.absent() && !ctx.plan.targetSide.cAbi.interfaceName.empty()) + prof.libc = ctx.plan.targetSide.cAbi.interfaceName; + if (!ctx.plan.targetSide.cxx.absent() && !ctx.plan.targetSide.cxx.interfaceName.empty()) + prof.cxxStdlib = ctx.plan.targetSide.cxx.interfaceName; std::vector constraints; for (auto& cap : ctx.plan.runtimeCapabilities) { std::string provider; for (auto& [c, p] : ctx.plan.runtimeProviders) if (c == cap) { provider = p.canonical(); break; } if (auto con = mcpp::toolchain::parse_abi_capability( - cap, provider.empty() ? std::string_view{"?"} : std::string_view{provider})) + cap, provider.empty() ? std::string_view{"?"} : std::string_view{provider})) { + // Reported where the package that states it can be edited. + if (con->dim == mcpp::toolchain::AbiDim::Libc + || con->dim == mcpp::toolchain::AbiDim::CxxStdlib) { + for (std::size_t i = 0; i < state.packages.size(); ++i) + if (std::ranges::find(state.packages[i].manifest.runtimeConfig.capabilities, cap) + != state.packages[i].manifest.runtimeConfig.capabilities.end()) { + state.reportCompatUse("abi-capability", i, std::format( + "the runtime capability `{}`, read as `requires = [\"mcpp:{}={}\"]`,", + cap, con->dim == mcpp::toolchain::AbiDim::Libc ? "c-abi" : "c++-abi", + con->value)); + break; + } + } constraints.push_back(std::move(*con)); + } } if (auto mismatches = mcpp::toolchain::abi_check(prof, constraints); !mismatches.empty()) { @@ -1925,6 +2074,22 @@ std::expected phase13_finish(PrepareState& state) { ctx.cacheMode = state.cacheMode; ctx.projectRoot= *state.root; ctx.outputDir = target_dir(*state.tc, state.fp, state.workRoot); + // THE CONFIGURATION'S DIRECTORY IS WRITTEN FROM HERE ON (mcpp#839): build + // actions, the dependency BMI cache, resolution.json, build.ninja. The + // fingerprint that names it is the scan's last result, so everything + // before this line -- the manifest, resolution, the network, the + // toolchain, the scan -- ran holding nothing, and a second build of the + // same configuration waits here, then finds ninja with little to do. A + // pass that only describes (plan_only) plans into its own directory. + // `--deny write-project` stops here, before the first write (design + // 2026-10-26 §5.5). + if (!state.overrides.plan_only && mcpp::platform::env::effect_denied("write-project")) { + mcpp::platform::env::note_effect_refused("write-project", ctx.outputDir.generic_string()); + return std::unexpected(std::format( + "effect denied: write-project -- the build writes {}, and the caller denied writing " + "the project (--deny write-project)", ctx.outputDir.generic_string())); + } + if (!state.overrides.plan_only) ctx.buildLock = mcpp::build::dirlock::acquire(ctx.outputDir); // Each step states its duration under `build/stage` in the log file, as // the phases do (build wall-time plan, W9; see prepare_build). @@ -1951,6 +2116,8 @@ std::expected phase13_finish(PrepareState& state) { return std::unexpected(r.error()); if (auto r = timed("make plan", [&] { return step13_make_plan(state, ctx); }); !r) return std::unexpected(r.error()); + if (auto r = timed("cxx islands", [&] { return step13_cxx_islands(state, ctx); }); !r) + return std::unexpected(r.error()); if (auto r = timed("cxx private runtime", [&] { return step13_cxx_private_runtime(state, ctx); }); !r) return std::unexpected(r.error()); if (auto r = timed("cxx process runtime", [&] { return step13_cxx_process_runtime(state, ctx); }); !r) diff --git a/src/build/prepare/records.cpp b/src/build/prepare/records.cpp index 21108006d..1ea35be26 100644 --- a/src/build/prepare/records.cpp +++ b/src/build/prepare/records.cpp @@ -312,6 +312,20 @@ std::expected step13_lockfile(PrepareState& state, BuildConte return std::unexpected(msg); } } + // OPTIMISTIC: no lock is held while the graph resolves (it may + // reach the network). What this run read is compared with what is + // there now; a lock another run wrote meanwhile is not + // overwritten unless it already says what this one would + // (design 2026-10-26 §4.1). + if (!state.lockReadPath.empty() && state.lockReadPath == lockPath) { + auto now = mcpp::pm::read_text(lockPath); + if (now && *now != state.lockReadText && *now != mcpp::lockfile::serialize(lock)) + return std::unexpected(std::format( + "[lockfile-concurrent] {} changed while this run resolved its " + "dependencies (another mcpp run wrote it)\n" + " help: run the command again; it resolves against the lock " + "as it is now", lockPath.string())); + } if (auto written = mcpp::lockfile::write(lock, lockPath); !written) return std::unexpected(written.error().message); } diff --git a/src/build/prepare/state.cppm b/src/build/prepare/state.cppm index 20b4e8436..1bda1fa11 100644 --- a/src/build/prepare/state.cppm +++ b/src/build/prepare/state.cppm @@ -518,6 +518,10 @@ struct PrepareState { std::vector planNotes; std::map gitLockAnchors; std::map packageIdentityLockAnchors; + // mcpp.lock as this run read it (design 2026-10-26 §4.1): the write at the + // end compares, so a lock another run wrote meanwhile is not overwritten. + std::filesystem::path lockReadPath; + std::optional lockReadText; // nullopt: there was no file CacheMode cacheMode{}; // ── P1: toolchain spec resolution, target axis, L1 cfg merge ──────────── @@ -797,6 +801,9 @@ struct PrepareState { std::set programFailedPackages; std::function()> checkVersionFloors; mcpp::targetside::TargetSide resolvedTargetSide; + // The C++ islands of this build (design 2026-10-26 §6.3): packages whose + // private `mcpp:c++-abi` requirement the resolved C++ layer does not meet. + std::vector cxxIslands; std::optional cxxLayerProviderIndex; bool targetSideResolved = false; mcpp::modgraph::UsageRequirements targetSideUsage; @@ -942,6 +949,13 @@ void fill_target_build_env(mcpp::build::BuildProgramEnv& e, const mcpp::manifest::Manifest& m, const mcpp::toolchain::Toolchain* tc, const mcpp::config::GlobalConfig* cfg); +// The `c++-abi` a package of the graph supplies, over the payload's: its +// interface as `cxxStdlib` and its root as `cxxAbiRoot`. Read from the +// declarations, so it is known to the build programs that run before the +// target side is resolved; rule one (one supplier per layer) is enforced +// where the target side is. +void apply_graph_cxx_abi(mcpp::build::BuildProgramEnv& e, + const std::vector& packages); std::string min_platform_version(const mcpp::manifest::Manifest& m, const mcpp::toolchain::triple::Triple& t, const std::filesystem::path& compilerPath); diff --git a/src/build/prepare/target_side.cpp b/src/build/prepare/target_side.cpp index 7bead627f..c6b8fdd75 100644 --- a/src/build/prepare/target_side.cpp +++ b/src/build/prepare/target_side.cpp @@ -226,6 +226,14 @@ step9_gather_target_side_candidates(PrepareState& state) { gather.requirements.push_back({ pkgId, (*parsed)->layer, (*parsed)->interfaceName }); } + // The private ones (design 2026-10-26 §6.3): the manifest parser + // accepted them for `mcpp:c++-abi` only. + for (auto const& entry : pkg.manifest.privateRequires) { + auto parsed = tsd::parse_capability(entry); + if (!parsed || !*parsed) continue; + gather.requirements.push_back({ pkgId, (*parsed)->layer, + (*parsed)->interfaceName, /*privateScope=*/true }); + } } for (auto const& [layerInt, slot] : gather.byLayer) { @@ -998,6 +1006,10 @@ step9_layering_and_requirement_checks(PrepareState& state, TargetSideGather& gat refusal::record(refusal::Code::LayerRequirement); return std::unexpected(*why); } + // The islands: packages whose private C++ requirement the graph's C++ + // layer does not meet. Verified where their objects and the image + // exist (phase 13 and after the link), not here. + state.cxxIslands = tsd::cxx_islands(state.resolvedTargetSide, gather.requirements); // A WARNING, NOT A REFUSAL. The graph decides the C library either // way, so the segment is ignored rather than violated and the artifact // is the same with or without it. Refusing was tried and broke every @@ -1653,6 +1665,7 @@ static std::expected step9_root_build_program(PrepareState& s // and the three answers that keep a board package from naming a // toolchain. One call — see fill_target_build_env. fill_target_build_env(bpEnv, *state.m, state.tc ? &*state.tc : nullptr, state.cfg_opt ? &*state.cfg_opt : nullptr); + apply_graph_cxx_abi(bpEnv, state.packages); bpEnv.toolsBin = state.projectSubosBin; bpEnv.profile = state.effectiveProfile; bpEnv.accel = state.resolvedAccel(); @@ -2064,6 +2077,7 @@ static std::expected step9_member_build_programs(PrepareState bpEnv.targetTriple = state.resolvedTargetCanonical; fill_target_build_env(bpEnv, *state.m, state.tc ? &*state.tc : nullptr, state.cfg_opt ? &*state.cfg_opt : nullptr); + apply_graph_cxx_abi(bpEnv, state.packages); bpEnv.toolsBin = state.projectSubosBin; bpEnv.profile = state.effectiveProfile; bpEnv.accel = state.resolvedAccel(); diff --git a/src/build/prepare/toolchain.cpp b/src/build/prepare/toolchain.cpp index 18aad3ca7..bfbbe11af 100644 --- a/src/build/prepare/toolchain.cpp +++ b/src/build/prepare/toolchain.cpp @@ -143,7 +143,7 @@ static std::expected step1_define_early_toolchain_closures(Pr auto problem = mcpp::config::check_base_init(*state.cfg_opt); if (!problem.empty()) { return std::unexpected(std::format( - "{}\n hint: run `mcpp self init --force` to reset and re-initialize", + "{}\n help: run `mcpp self init --force` to reset and re-initialize", problem)); } } diff --git a/src/build/prepare/toolchain_env.cpp b/src/build/prepare/toolchain_env.cpp index bd2df2fe5..70e9861b5 100644 --- a/src/build/prepare/toolchain_env.cpp +++ b/src/build/prepare/toolchain_env.cpp @@ -31,6 +31,7 @@ import mcpp.freestanding.target; // the target sysroot layout (libdir) import mcpp.freestanding.linkline; // the ISA profile, for the std module command import mcpp.toolchain.post_install; import mcpp.toolchain.abi; +import mcpp.targetside; // the graph's c++-abi supplier, for build programs import mcpp.toolchain.triple; import mcpp.build.plan; import mcpp.build.flags; // compute_flags — the per-role contracts (#418) @@ -459,6 +460,20 @@ build_information(const mcpp::manifest::Manifest& m, const mcpp::toolchain::Tool } // namespace +void apply_graph_cxx_abi(mcpp::build::BuildProgramEnv& e, + const std::vector& packages) { + namespace tsd = mcpp::targetside; + for (auto const& pkg : packages) + for (auto const& entry : pkg.manifest.provides) { + auto parsed = tsd::parse_capability(entry); + if (!parsed || !*parsed || (*parsed)->layer != tsd::CapLayer::CxxAbi) continue; + if ((*parsed)->interfaceName.empty()) continue; + e.cxxStdlib = (*parsed)->interfaceName; + e.cxxAbiRoot = pkg.root.generic_string(); + return; + } +} + void fill_target_build_env(mcpp::build::BuildProgramEnv& e, const mcpp::manifest::Manifest& m, const mcpp::toolchain::Toolchain* tc, diff --git a/src/build/progress.cppm b/src/build/progress.cppm index 57004992b..0875d89f8 100644 --- a/src/build/progress.cppm +++ b/src/build/progress.cppm @@ -222,6 +222,11 @@ void checking(); // `Finished` (design §4.5): the whole command's time, and how it was spent. // Written once per command; a later call does nothing. void finished(std::string_view profile, std::string_view descriptor); + +// Called with ninja's count each time a step of a reported build finishes (the +// `progress` event of `mcpp build --message-format json`). One observer per +// process; a null function removes it. +void set_step_observer(std::function observer); // The sources of a build directory that are not the ecosystem's default // (mcpp#755), read from `/sources.summary`, which prepare writes. // Merged across the configurations of a command and appended to `Finished`; @@ -1550,7 +1555,19 @@ void Build::pass_begin(PassKind kind) { mcpp::ui::touch_region(); } +namespace { +std::function& step_observer() { + static std::function observer; + return observer; +} +} // namespace + +void set_step_observer(std::function observer) { + step_observer() = std::move(observer); +} + void Build::status(const StatusLine& line) { + if (auto& observe = step_observer()) observe(line.finished, line.total); auto& r = report(); std::vector out; { diff --git a/src/build/refusal.cppm b/src/build/refusal.cppm index b9a95261f..d148e8866 100644 --- a/src/build/refusal.cppm +++ b/src/build/refusal.cppm @@ -32,6 +32,7 @@ export module mcpp.build.refusal; import std; +import mcpp.platform.env; // an offline download is a refused `network` effect (exit 5) export namespace mcpp::build::refusal { @@ -263,7 +264,14 @@ namespace { thread_local Code g_last = Code::None; } // namespace -void record(Code c) { g_last = c; } +void record(Code c) { + g_last = c; + // Needing a download offline is the run stopping before a denied effect: + // `--offline` is `--deny network`, and the process exits 5 (design + // 2026-10-26 D23). + if (c == Code::OfflineDownloadRequired) + mcpp::platform::env::note_effect_refused("network", "a download is required"); +} Code take() { Code c = g_last; g_last = Code::None; return c; } diff --git a/src/build/result.cppm b/src/build/result.cppm new file mode 100644 index 000000000..a84b4f839 --- /dev/null +++ b/src/build/result.cppm @@ -0,0 +1,361 @@ +// mcpp.build.result — what a build left behind, for whoever was not watching +// (design 2026-10-26 §5.1–§5.3, D2). +// +// THE RECORD. A build of one configuration writes `result.json` in its build +// directory twice: when it starts writing the directory (`"status": +// "running"`, with its pid) and when it ends (`ok`, `failed`). A build ended by +// SIGINT, SIGTERM or SIGHUP (Windows: a console control event) leaves +// `cancelled`: the interrupted form is rendered when the run starts and the +// handler writes it with async-signal-safe calls. SIGKILL runs no handler, so a +// record still `running` whose process is gone is a build that was killed. `target/.last-result.json` +// names the configuration the most recent build of the project wrote. Both are +// replaced atomically. The record is how an IDE learns what a build in a +// terminal did, and how an agent learns what the IDE's build did: the chains +// meet at mcpp, so mcpp writes the record (D1, D2). +// +// THE EVENT STREAM. `mcpp build --message-format json` writes NDJSON on +// standard output while it works -- `plan`, `waiting`, `progress`, +// `diagnostic`, `artifact` -- and ends with the `mcpp.build` envelope, whose +// `data` is this record. The narration stays on standard error, as for every +// machine-readable mode (docs/50 §3). +// +// THE LAUNCH DESCRIPTOR. A runnable artifact carries how `mcpp run` would +// start it: `argv`, the environment mcpp adds (only what it injects or +// rewrites -- the caller's own environment is not copied into target/), and +// `cwd`. An artifact that a runner or an execution host wraps says so +// (`via`), and `mcpp run --print-launch` answers for it. +export module mcpp.build.result; + +import std; +import mcpp.libs.json; +import mcpp.version; // MCPP_VERSION +import mcpp.platform.terminal; // the protocol's stdout under StdoutToStderr +import mcpp.platform.process; // the files an interrupt leaves + +export namespace mcpp::build::result { + +struct Launch { + std::vector argv; + std::vector> env; + std::string cwd; + std::string via; // "" | "runner" | "run_under" +}; + +struct Artifact { + std::string name; + std::string kind; // bin | test | shared | static + std::filesystem::path path; + std::optional launch; +}; + +// A compiler's diagnostic, from the output of a failed step. +struct Diagnostic { + std::string severity; // error | warning | note + std::string message; + std::string path; + int line = 0; + int column = 0; +}; + +struct Record { + std::string command; + std::string client; + long pid = 0; + std::int64_t startedAt = 0; // ms since the epoch + std::int64_t finishedAt = 0; + std::string status; // running | ok | failed | cancelled + std::string triple; + std::string profile; + std::string fingerprint; + std::filesystem::path outputDir; + std::filesystem::path projectRoot; + std::filesystem::path compileDatabase; + std::vector artifacts; + std::vector diagnostics; +}; + +nlohmann::json to_json(const Launch& l); +nlohmann::json to_json(const Record& r); +std::optional read(const std::filesystem::path& outputDir); + +// Writes `/result.json` and `/target/.last-result.json`. +void write(const Record& r); + +// A record for this process, started now. +Record begin(const std::filesystem::path& projectRoot, const std::filesystem::path& outputDir); + +std::int64_t now_ms(); + +// GCC, Clang and MSVC diagnostic lines in `text`. +std::vector parse_compiler_diagnostics(std::string_view text); + +// ── the event stream ── +void enable_events(bool on); +bool events_enabled(); +// One NDJSON line on stdout: {"schemaVersion":1,"kind":"mcpp.event","event":, ...fields}. +void emit(std::string_view event, nlohmann::json fields = nlohmann::json::object()); + +} // namespace mcpp::build::result + +namespace mcpp::build::result { + +namespace { +std::atomic g_events{false}; +std::mutex g_emit; + +void write_atomically(const std::filesystem::path& file, const std::string& text) { + std::error_code ec; + std::filesystem::create_directories(file.parent_path(), ec); + auto tmp = file; + tmp += std::format(".{:x}", std::random_device{}()); + { + std::ofstream out(tmp, std::ios::binary | std::ios::trunc); + if (!out) return; + out << text; + } + std::filesystem::rename(tmp, file, ec); + if (ec) std::filesystem::remove(tmp, ec); +} +} // namespace + +std::int64_t now_ms() { + return std::chrono::duration_cast( + std::chrono::system_clock::now().time_since_epoch()).count(); +} + +nlohmann::json to_json(const Launch& l) { + nlohmann::json env = nlohmann::json::object(); + for (auto const& [k, v] : l.env) env[k] = v; + nlohmann::json j{{"argv", l.argv}, {"env", std::move(env)}, {"cwd", l.cwd}}; + j["via"] = l.via.empty() ? nlohmann::json(nullptr) : nlohmann::json(l.via); + return j; +} + +nlohmann::json to_json(const Record& r) { + nlohmann::json artifacts = nlohmann::json::array(); + for (auto const& a : r.artifacts) { + nlohmann::json j{{"name", a.name}, {"kind", a.kind}, {"path", a.path.generic_string()}}; + if (a.launch) j["launch"] = to_json(*a.launch); + artifacts.push_back(std::move(j)); + } + nlohmann::json diags = nlohmann::json::array(); + for (auto const& d : r.diagnostics) { + nlohmann::json j{{"severity", d.severity}, {"message", d.message}}; + if (!d.path.empty()) { + j["path"] = d.path; + if (d.line > 0) + j["range"] = {{"start", {{"line", d.line}, {"column", std::max(d.column, 1)}}}, + {"end", {{"line", d.line}, {"column", std::max(d.column, 1)}}}}; + } + diags.push_back(std::move(j)); + } + return nlohmann::json{ + {"command", r.command}, + {"client", r.client}, + {"pid", r.pid}, + {"startedAt", r.startedAt}, + {"finishedAt", r.finishedAt == 0 ? nlohmann::json(nullptr) : nlohmann::json(r.finishedAt)}, + {"status", r.status}, + {"configuration", { + {"triple", r.triple}, + {"profile", r.profile}, + {"fingerprint", r.fingerprint}, + {"outputDir", r.outputDir.generic_string()}, + }}, + {"compileDatabase", r.compileDatabase.empty() ? nlohmann::json(nullptr) + : nlohmann::json(r.compileDatabase.generic_string())}, + {"artifacts", std::move(artifacts)}, + {"diagnostics", std::move(diags)}, + }; +} + +std::optional read(const std::filesystem::path& outputDir) { + std::ifstream in(outputDir / "result.json", std::ios::binary); + if (!in) return std::nullopt; + auto doc = nlohmann::json::parse(in, nullptr, false); + if (doc.is_discarded() || !doc.contains("data") || !doc["data"].is_object()) return std::nullopt; + const auto& d = doc["data"]; + Record r; + r.command = d.value("command", ""); + r.client = d.value("client", ""); + r.pid = d.value("pid", 0L); + r.startedAt = d.value("startedAt", std::int64_t{0}); + r.status = d.value("status", ""); + if (d.contains("configuration") && d["configuration"].is_object()) { + const auto& c = d["configuration"]; + r.triple = c.value("triple", ""); + r.profile = c.value("profile", ""); + r.fingerprint = c.value("fingerprint", ""); + r.outputDir = c.value("outputDir", ""); + } + if (d.contains("compileDatabase") && d["compileDatabase"].is_string()) + r.compileDatabase = d["compileDatabase"].get(); + if (d.contains("artifacts") && d["artifacts"].is_array()) + for (auto const& a : d["artifacts"]) { + Artifact art{a.value("name", ""), a.value("kind", ""), a.value("path", ""), std::nullopt}; + if (a.contains("launch") && a["launch"].is_object()) { + const auto& l = a["launch"]; + Launch launch; + if (l.contains("argv") && l["argv"].is_array()) + for (auto const& v : l["argv"]) if (v.is_string()) launch.argv.push_back(v.get()); + if (l.contains("env") && l["env"].is_object()) + for (auto it = l["env"].begin(); it != l["env"].end(); ++it) + if (it->is_string()) launch.env.emplace_back(it.key(), it->get()); + launch.cwd = l.value("cwd", ""); + if (l.contains("via") && l["via"].is_string()) launch.via = l["via"].get(); + art.launch = std::move(launch); + } + r.artifacts.push_back(std::move(art)); + } + return r; +} + +namespace { +std::string render(const Record& r) { + nlohmann::json doc{ + {"schemaVersion", 1}, + {"kind", "mcpp.result"}, + {"kindVersion", 1}, + {"mcpp", {{"version", std::string(mcpp::MCPP_VERSION)}}}, + {"data", to_json(r)}, + }; + return doc.dump(2) + "\n"; +} +std::string render_last(const Record& r) { + nlohmann::json last{ + {"schemaVersion", 1}, + {"kind", "mcpp.last-result"}, + {"kindVersion", 1}, + {"data", {{"result", (r.outputDir / "result.json").generic_string()}, + {"status", r.status}, + {"finishedAt", r.finishedAt == 0 ? nlohmann::json(nullptr) + : nlohmann::json(r.finishedAt)}}}, + }; + return last.dump(2) + "\n"; +} +} // namespace + +void write(const Record& r) { + write_atomically(r.outputDir / "result.json", render(r)); + const auto last = r.projectRoot / "target" / ".last-result.json"; + if (!r.projectRoot.empty()) write_atomically(last, render_last(r)); + // A running record is armed with its interrupted form: a signal that ends + // the build leaves `cancelled`, not a `running` record whose process is + // gone (§5.4). The end of the build disarms it. + if (r.status == "running") { + auto cancelled = r; + cancelled.status = "cancelled"; + std::vector> files{ + {r.outputDir / "result.json", render(cancelled)}}; + if (!r.projectRoot.empty()) files.emplace_back(last, render_last(cancelled)); + mcpp::platform::process::set_interrupt_files(files); + } else { + mcpp::platform::process::clear_interrupt_files(); + } +} + +std::vector parse_compiler_diagnostics(std::string_view text) { + std::vector out; + const auto severity_at = [](std::string_view s, std::size_t& len) -> std::string { + for (std::string_view sev : {"fatal error", "error", "warning", "note"}) + if (s.starts_with(sev)) { + len = sev.size(); + return std::string(sev == "fatal error" ? "error" : sev); + } + return {}; + }; + const auto number = [](std::string_view s, int& v) { + return std::from_chars(s.data(), s.data() + s.size(), v).ec == std::errc{} && v > 0; + }; + // GCC / Clang: ::: : , or without the + // column, or with a program's name for the path (`cc1plus: error: …`). + // A Windows drive letter's colon belongs to the path. + const auto gnu = [&](std::string_view line) -> std::optional { + for (auto at = line.find(": "); at != std::string_view::npos; at = line.find(": ", at + 2)) { + std::size_t len = 0; + auto severity = severity_at(line.substr(at + 2), len); + if (severity.empty() || line.substr(at + 2 + len).substr(0, 2) != ": ") continue; + auto loc = line.substr(0, at); + Diagnostic d; + d.severity = std::move(severity); + d.message = std::string(line.substr(at + 4 + len)); + int a = 0, b = 0; + auto c2 = loc.rfind(':'); + if (c2 != std::string_view::npos && c2 > 1 && number(loc.substr(c2 + 1), a)) { + auto head = loc.substr(0, c2); + auto c1 = head.rfind(':'); + if (c1 != std::string_view::npos && c1 > 1 && number(head.substr(c1 + 1), b)) { + d.path = std::string(head.substr(0, c1)); + d.line = b; + d.column = a; + } else { + d.path = std::string(head); + d.line = a; + } + } else if (loc.find(' ') == std::string_view::npos) { + d.path = std::string(loc); + } else { + continue; + } + return d; + } + return std::nullopt; + }; + // MSVC: ([,]): : + const auto msvc = [&](std::string_view line) -> std::optional { + auto paren = line.find("): "); + if (paren == std::string_view::npos) return std::nullopt; + auto open = line.rfind('(', paren); + if (open == std::string_view::npos) return std::nullopt; + auto rest = line.substr(paren + 3); + std::size_t len = 0; + auto severity = severity_at(rest, len); + if (severity.empty() || rest.substr(len).substr(0, 1) != " ") return std::nullopt; + auto codeEnd = rest.find(": ", len); + if (codeEnd == std::string_view::npos) return std::nullopt; + Diagnostic d; + d.severity = std::move(severity); + d.message = std::string(rest.substr(codeEnd + 2)); + d.path = std::string(line.substr(0, open)); + auto nums = line.substr(open + 1, paren - open - 1); + auto comma = nums.find(','); + if (!number(nums.substr(0, comma), d.line)) return std::nullopt; + if (comma != std::string_view::npos) number(nums.substr(comma + 1), d.column); + return d; + }; + std::size_t pos = 0; + while (pos < text.size()) { + auto end = text.find('\n', pos); + if (end == std::string_view::npos) end = text.size(); + auto line = text.substr(pos, end - pos); + pos = end + 1; + if (!line.empty() && line.back() == '\r') line.remove_suffix(1); + if (auto d = msvc(line)) out.push_back(std::move(*d)); + else if (auto g = gnu(line)) out.push_back(std::move(*g)); + } + return out; +} + +Record begin(const std::filesystem::path& projectRoot, const std::filesystem::path& outputDir) { + Record r; + r.startedAt = now_ms(); + r.status = "running"; + r.outputDir = outputDir; + r.projectRoot = projectRoot; + r.fingerprint = outputDir.filename().string(); + if (const char* c = std::getenv("MCPP_CLIENT")) r.client = c; + return r; +} + +void enable_events(bool on) { g_events.store(on); } +bool events_enabled() { return g_events.load(); } + +void emit(std::string_view event, nlohmann::json fields) { + if (!events_enabled()) return; + nlohmann::json line{{"schemaVersion", 1}, {"kind", "mcpp.event"}, {"event", std::string(event)}}; + for (auto it = fields.begin(); it != fields.end(); ++it) line[it.key()] = it.value(); + std::lock_guard lock(g_emit); + mcpp::platform::terminal::write_protocol_stdout(line.dump() + "\n"); +} + +} // namespace mcpp::build::result diff --git a/src/build/stage.cppm b/src/build/stage.cppm index aa3fd5e0a..b9b175f3b 100644 --- a/src/build/stage.cppm +++ b/src/build/stage.cppm @@ -229,7 +229,7 @@ std::string failure_message(const std::filesystem::path& src, " file: {}\n" " from: {}\n" " os error: {} ({})\n" - "hint: another process has this file memory-mapped, loaded or open.\n" + "help: another process has this file memory-mapped, loaded or open.\n" " The usual holder is clangd (mcpp writes staged BMI paths into\n" " compile_commands.json so clangd can resolve `import std;`), an\n" " editor/IDE indexer, antivirus, or — for a .dll — a still-running\n" diff --git a/src/cli.cppm b/src/cli.cppm index 66029520c..cae0453c1 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -36,6 +36,7 @@ import mcpp.platform.env; // --offline → MCPP_OFFLINE import mcpp.platform.process; // __action-stamp runs the checked command import mcpp.platform.fs; // __action-stamp writes its stamps import mcpp.build.progress; // __action reports its start +import mcpp.build.dirlock; // the build directory lock names this command import mcpp.platform.runtime_search; // linker-wrapper path-injection opt-out import mcpp.ui; import mcpp.log; @@ -205,6 +206,113 @@ int run(int argc, char** argv) { namespace cl = mcpplibs::cmdline; ReportUnnarrowablePaths reportUnnarrowable_; + // What a build that waits for this one is told it waits for (mcpp#839): + // the command as typed, cut where a long argument list would bury it. + { + std::string command = "mcpp"; + for (int i = 1; i < argc && command.size() < 80; ++i) { + command += ' '; + command += argv[i]; + } + mcpp::build::dirlock::set_command(std::move(command)); + } + + // ─── --home : this run's home (design 2026-10-26 §9.1, SPEC-010) ── + // Exactly MCPP_HOME for this process: the raw home of the two-step + // selection, so its locator is still followed. Taken out of argv before + // anything parses it. After `self init` the flag is that command's own -- + // the home to select persistently -- and stays; a global one naming a + // different directory beside it is a usage error, the same one naming the + // same directory is that command's flag written twice. + std::vector homeFiltered(argv, argv + argc); + { + std::optional global; + std::vector kept{homeFiltered.front()}; + std::vector words; + bool initArgs = false; + for (std::size_t i = 1; i < homeFiltered.size(); ++i) { + const std::string_view a = homeFiltered[i]; + if (a == "--") { kept.insert(kept.end(), homeFiltered.begin() + static_cast(i), homeFiltered.end()); break; } + if (!a.starts_with("-")) { + words.push_back(a); + initArgs = initArgs || (words.size() == 2 && words[0] == "self" && words[1] == "init"); + } + if (!initArgs && (a == "--home" || a.starts_with("--home="))) { + if (a == "--home") { + if (i + 1 >= homeFiltered.size()) { + mcpp::ui::error("--home needs a directory"); + return 2; + } + global = std::string(homeFiltered[++i]); + } else global = std::string(a.substr(7)); + continue; + } + kept.push_back(homeFiltered[i]); + } + if (global) { + std::error_code ec; + const auto absolute = std::filesystem::absolute(*global, ec).lexically_normal(); + for (std::size_t i = 1; initArgs && i < kept.size(); ++i) { + const std::string_view a = kept[i]; + std::string other; + if (a == "--home" && i + 1 < kept.size()) other = kept[i + 1]; + else if (a.starts_with("--home=")) other = std::string(a.substr(7)); + else continue; + if (std::filesystem::absolute(other, ec).lexically_normal() != absolute) { + mcpp::ui::error(std::format("--home names {} and `self init --home` names {}; give one of them", + absolute.string(), other)); + return 2; + } + } + mcpp::platform::env::set("MCPP_HOME", absolute.string()); + homeFiltered = std::move(kept); + argc = static_cast(homeFiltered.size()); + argv = homeFiltered.data(); + } + } + + // ─── --deny (design 2026-10-26 §5.5) ─────────────────────── + // Published as MCPP_DENY_EFFECTS, the union of what the caller's + // environment already denies and what the flag adds, for the reason + // --offline is an env var: the consumers sit in every subsystem. A name + // that is not a deniable effect is a usage error, before anything runs. + { + auto valid = [](std::string_view list) -> std::optional { + std::size_t pos = 0; + while (pos <= list.size()) { + auto comma = list.find(',', pos); + auto item = list.substr(pos, comma == std::string_view::npos ? std::string_view::npos : comma - pos); + while (!item.empty() && item.front() == ' ') item.remove_prefix(1); + while (!item.empty() && item.back() == ' ') item.remove_suffix(1); + const auto& names = mcpp::platform::env::deniable_effects(); + if (!item.empty() && std::ranges::find(names, item) == names.end()) + return std::string(item); + if (comma == std::string_view::npos) break; + pos = comma + 1; + } + return std::nullopt; + }; + std::string denied = mcpp::platform::env::get("MCPP_DENY_EFFECTS").value_or(""); + for (int i = 1; i < argc; ++i) { + const std::string_view a = argv[i]; + if (a == "--") break; + std::string value; + if (a == "--deny") { + if (i + 1 >= argc) { mcpp::ui::error("--deny needs a list of effects"); return 2; } + value = argv[i + 1]; + } else if (a.starts_with("--deny=")) value = std::string(a.substr(7)); + else continue; + denied += (denied.empty() ? "" : ",") + value; + } + if (auto bad = valid(denied)) { + mcpp::ui::error(std::format( + "--deny / MCPP_DENY_EFFECTS: '{}' is not an effect that can be denied; expected " + "init-mcpp-home, write-project, write-global-cache, network, exec-build-script", *bad)); + return 2; + } + if (!denied.empty()) mcpp::platform::env::set("MCPP_DENY_EFFECTS", denied); + } + // ─── --quiet / --no-color: pre-scan ───────────────────────────────── // The cmdline lib propagates global options into nested subcommand // ParsedArgs, but we set ui:: state up-front so that *every* line @@ -416,7 +524,10 @@ int run(int argc, char** argv) { .option(cl::Option("no-color") .help("Disable colored output").global()) .option(cl::Option("offline") - .help("Never touch the network (index refresh, downloads, toolchain install)") + .help("Never touch the network (index refresh, downloads, toolchain install); = --deny network") + .global()) + .option(cl::Option("deny").takes_value().value_name("EFFECTS") + .help("Stop before any of these effects, exit 5: network, write-project, write-global-cache, exec-build-script, init-mcpp-home (also: MCPP_DENY_EFFECTS)") .global()) .option(cl::Option("system") .help("Maintain the machine's system home (self init|doctor|store, toolchain install|remove|default, index update)") @@ -495,6 +606,8 @@ int run(int argc, char** argv) { .help("With --workspace (or at a virtual workspace root), leave the named member out; repeatable, refused with -p")) .option(cl::Option("play-game") .help("Play a game in the status row while it builds: --play-game=snake|stack|runner, or one at random")) + .option(cl::Option("message-format").takes_value().value_name("FMT") + .help("human (default) | json: NDJSON events on stdout, then the mcpp.build envelope")) .action(wrap_rc(cmd_build))) .subcommand(cl::App("run") .description("Build + run a binary target (after `--`, args are passed to it)") @@ -594,6 +707,8 @@ int run(int argc, char** argv) { "--no-runner")) .option(cl::Option("play-game") .help("Play a game in the status row while it builds: --play-game=snake|stack|runner, or one at random")) + .option(cl::Option("print-launch") + .help("Build, then print how the program would be started (argv, the environment mcpp adds, cwd, runner) as the mcpp.launch envelope, instead of starting it")) .action(wrap_rc([&passthrough](const cl::ParsedArgs& p) { return cmd_run(p, std::span(passthrough)); }))) @@ -697,9 +812,11 @@ int run(int argc, char** argv) { .action(wrap_rc(mcpp::pm::commands::cmd_update))) .subcommand(cl::App("search") .description("Search packages in configured registries") - .arg(cl::Arg("keyword").help("Search keyword (substring match)").required()) + .arg(cl::Arg("keyword").help("Search keyword (substring match); optional with --format json, which then lists every package")) .option(cl::Option("all-versions") .help("List every published version instead of the latest few")) + .option(cl::Option("format").takes_value().value_name("FMT") + .help("json: the mcpp.search envelope (every version, licenses, repo)")) .action(wrap_rc(cmd_search))) .subcommand(cl::App("publish") .description("Publish package to default registry") @@ -756,6 +873,8 @@ int run(int argc, char** argv) { .help("Activate root-package features, and a dependency's as /, for every build pass of the pack (comma-separated)")) // `--format` names the PACKAGE format here, so machine output is // asked for the way `mcpp test` asks for it (docs/50 §3). + .option(cl::Option("list-formats") + .help("Print the formats --format accepts here, the engine's and those the resolved graph provides, as the mcpp.pack.formats envelope")) .option(cl::Option("message-format").takes_value().value_name("FMT") .help("Output format: human (default) | json (one mcpp.pack envelope on stdout; narration on stderr)")) .option(cl::Option("no-strip") @@ -944,7 +1063,9 @@ int run(int argc, char** argv) { .subcommand(cl::App("index") .description("Manage configured package registries") .subcommand(cl::App("list") - .description("List configured registries")) + .description("List configured registries") + .option(cl::Option("format").takes_value().value_name("FMT") + .help("json: the mcpp.index envelope"))) .subcommand(cl::App("add") .description("Add a custom registry") .arg(cl::Arg("name").help("Registry name").required()) @@ -1080,6 +1201,12 @@ int run(int argc, char** argv) { .option(cl::Option("list").takes_value().value_name("FILE") .help("Place every `\\t` pair of FILE in one process (#734 E4); several lines naming one destination are its sources")) .action(wrap_rc(cmd_stage))) + .subcommand(cl::App("link-output") + .description("(internal: invoked by ninja on Windows) Move a linked artifact aside, then run the linker command after `--` (a running program keeps its file)") + .option(cl::Option("output").takes_value().value_name("PATH").help("the artifact the command links")) + .action(wrap_rc([&passthrough](const cl::ParsedArgs& p) { + return cmd_link_output(p, std::span(passthrough)); + }))) .subcommand(cl::App("place-dlls") .description("(internal: invoked by ninja) Place beside a Windows program the DLLs it imports from its runtime search directories") .option(cl::Option("output").takes_value().value_name("PATH").help("the stamp to write")) @@ -1502,6 +1629,23 @@ int run(int argc, char** argv) { {"pack", {Effect::InitMcppHome, Effect::ReadProject, Effect::WriteProject, Effect::Network, Effect::WriteGlobalCache, Effect::ExecBuildScript}}, + // `--message-format json` (design 2026-10-26 §5.1): a build, so + // everything a build may do. + {"build", {Effect::InitMcppHome, Effect::ReadProject, + Effect::WriteProject, Effect::Network, + Effect::WriteGlobalCache, Effect::ExecBuildScript}}, + // `--print-launch` builds the program and starts nothing. + {"run --print-launch", {Effect::InitMcppHome, Effect::ReadProject, + Effect::WriteProject, Effect::Network, + Effect::WriteGlobalCache, Effect::ExecBuildScript}}, + // The index is refreshed when older than the search TTL. + {"search", {Effect::InitMcppHome, Effect::Network}}, + {"index list", {Effect::InitMcppHome, Effect::ReadProject}}, + // The same planning as `pack`, which runs the build programs that + // declare formats; nothing is built. + {"pack --list-formats", {Effect::InitMcppHome, Effect::ReadProject, + Effect::Network, Effect::WriteGlobalCache, + Effect::ExecBuildScript}}, }; }; @@ -1532,7 +1676,7 @@ int run(int argc, char** argv) { "update", "search", "publish", "pack", "emit", "xpkg", "toolchain", "cache", "index", "self", "explain", "version", "dyndep", "why", "resolve", "stage", "bmi-equal", "coff-def", - "place-dlls", + "place-dlls", "link-output", "bmi-compile", "bmi-supervise", "bmi-await", }); bool ok = false; @@ -1572,6 +1716,14 @@ int run(int argc, char** argv) { // The run's closing notices come last, after everything the command // printed, whatever its exit status (see mcpp::ui::add_closing_notice). mcpp::ui::print_closing_notices(); + // A run that stopped before an effect its caller denied exits 5 + // (SPEC-003, design 2026-10-26 D23): the caller's constraint, not a + // failure of the project, and a script tells the two apart by the status. + if (action_rc != 0) + if (auto refused = mcpp::platform::env::effect_refused()) { + mcpp::ui::note(std::format("stopped before a denied effect ({}); exit status 5", *refused)); + return 5; + } return action_rc; } diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index cd31cb82b..d52898678 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -26,6 +26,8 @@ import mcpp.cli.selection; import mcpp.build.build_database; import mcpp.build.build_program; import mcpp.build.progress; // the report every building command opens +import mcpp.build.result; // `--message-format json`: events and the record (design 2026-10-26 §5) +import mcpp.build.dirlock; // the `waiting` event import mcpp.build.refusal; // offline-download-required (#648 A1) import mcpp.diag; // a member's own diagnostics, in the envelope (WS3) import mcpp.dyndep; @@ -180,7 +182,85 @@ mcpp::build::BuildOverrides overrides_from_selectors( return ov; } +int cmd_build_body(const mcpplibs::cmdline::ParsedArgs& parsed); + +// `mcpp build --message-format json` (design 2026-10-26 §5.1). +// +// NDJSON ON STANDARD OUTPUT WHILE IT BUILDS, THEN ONE ENVELOPE. Events are +// `{"schemaVersion":1,"kind":"mcpp.event","event":…}`: `plan` (a +// configuration's directory is entered), `waiting` (another build holds it), +// `progress` (ninja's count), `diagnostic` (a compiler's, from a failed step), +// `artifact` (each product, with its launch). The last line is the +// `mcpp.build` envelope; its `data` is the result record the build wrote +// (`target///result.json`). Everything human goes to standard +// error, as for every machine-readable mode. export int cmd_build(const mcpplibs::cmdline::ParsedArgs& parsed) { + std::string messageFormat = parsed.value("message-format").value_or("human"); + if (messageFormat != "human" && messageFormat != "json") { + mcpp::ui::error(std::format("unknown --message-format '{}' (human|json)", messageFormat)); + return 2; + } + if (messageFormat == "human") return cmd_build_body(parsed); + + namespace res = mcpp::build::result; + res::enable_events(true); + mcpp::build::dirlock::set_waiting_observer( + [](const mcpp::build::dirlock::Holder& h, std::chrono::milliseconds waited) { + res::emit("waiting", {{"holder", {{"pid", h.pid}, {"command", h.command}, + {"client", h.client}, {"startedAt", h.startedAt}}}, + {"waitedMs", waited.count()}}); + }); + mcpp::build::progress::set_step_observer([](std::size_t finished, std::size_t total) { + res::emit("progress", {{"finished", finished}, {"total", total}}); + }); + // mcpp's own errors (a manifest's rejection, a refused requirement) are + // the envelope's diagnostics, each located where its message says. + std::vector errors; + mcpp::ui::set_error_observer([&errors](std::string_view m) { errors.emplace_back(m); }); + int rc = 0; + { + mcpp::platform::terminal::StdoutToStderr narration; + rc = cmd_build_body(parsed); + } + mcpp::ui::set_error_observer({}); + res::enable_events(false); + + using mcpp::wire::Effect; + mcpp::wire::Envelope env{.kind = "mcpp.build", + .effects = {Effect::ReadProject, Effect::WriteProject}}; + env.data = nullptr; + std::error_code ec; + if (auto root = mcpp::project::find_manifest_root(std::filesystem::current_path(ec))) { + std::ifstream in(*root / "target" / ".last-result.json", std::ios::binary); + auto last = nlohmann::json::parse(in, nullptr, false); + if (!last.is_discarded() && last.contains("data") && last["data"].contains("result") + && last["data"]["result"].is_string()) { + const std::filesystem::path recordPath = last["data"]["result"].get(); + if (auto record = res::read(recordPath.parent_path()); + record && record->pid == mcpp::build::dirlock::process_id()) { + std::ifstream rin(recordPath, std::ios::binary); + auto doc = nlohmann::json::parse(rin, nullptr, false); + if (!doc.is_discarded() && doc.contains("data")) env.data = doc["data"]; + } + } + } + if (rc != 0) { + const bool denied = mcpp::platform::env::effect_refused().has_value(); + const std::string code = denied ? "MCPP_EFFECT_DENIED" : "MCPP_BUILD_FAILED"; + for (auto& e : errors) + env.diagnostics.push_back({code, mcpp::wire::Severity::Error, std::move(e)}); + if (env.diagnostics.empty()) + env.diagnostics.push_back({code, mcpp::wire::Severity::Error, + "mcpp build failed; the compiler diagnostics are in data.diagnostics " + "and the narration on standard error"}); + } + // One line, as every record of the stream is: a reader splits stdout on + // newlines and parses each line, the last of which is this envelope. + std::println("{}", mcpp::wire::to_json(env).dump()); + return rc; +} + +int cmd_build_body(const mcpplibs::cmdline::ParsedArgs& parsed) { bool verbose = parsed.is_flag_set("verbose") || mcpp::log::is_verbose(); bool print_fp = parsed.is_flag_set("print-fingerprint"); bool no_cache = parsed.is_flag_set("no-cache"); @@ -201,6 +281,10 @@ export int cmd_build(const mcpplibs::cmdline::ParsedArgs& parsed) { // `configured`, when given, receives the plan's output directory and the // plan leaves the root compile database to the caller, which publishes // it once for every configuration it planned. + // Where the project's compile database is published, as the last plan + // resolved it (`[build] compile_commands`); the caller that publishes the + // root copy of several configurations reads it. + std::string rootCompileDb; auto configure_member = [&](mcpp::build::BuildOverrides memberOv, std::string_view label, std::vector* configured = nullptr) -> int { @@ -233,6 +317,7 @@ export int cmd_build(const mcpplibs::cmdline::ParsedArgs& parsed) { } if (configured) { ctx->plan.publishRootCompileDb = false; + rootCompileDb = ctx->plan.rootCompileDb; configured->push_back(ctx->outputDir); } return mcpp::build::run_configure_plan(*ctx, verbose); @@ -290,11 +375,13 @@ export int cmd_build(const mcpplibs::cmdline::ParsedArgs& parsed) { if (!ctx && g.size() > 1) { configure_one_by_one(); continue; } if (!ctx) { mcpp::ui::error(std::format("{}: {}", g.front(), ctx.error())); rc = 2; continue; } ctx->plan.publishRootCompileDb = false; + rootCompileDb = ctx->plan.rootCompileDb; if (int r = mcpp::build::run_configure_plan(*ctx, verbose); r != 0) rc = r; configured.push_back(ctx->outputDir); } if (!configured.empty()) - mcpp::build::publish_workspace_compile_commands((*selection)->root, configured); + mcpp::build::publish_workspace_compile_commands((*selection)->root, configured, + rootCompileDb); return rc; } const bool plain = !print_fp && ov.target_triple.empty() && !ov.force_static @@ -364,7 +451,9 @@ export int cmd_build(const mcpplibs::cmdline::ParsedArgs& parsed) { if (r != 0 && rc == 0) rc = r; std::vector dirs; for (auto const& c : contexts) dirs.push_back(c.outputDir); - mcpp::build::publish_workspace_compile_commands((*selection)->root, dirs); + mcpp::build::publish_workspace_compile_commands( + (*selection)->root, dirs, + contexts.empty() ? std::string{} : contexts.front().plan.rootCompileDb); if (rc == 0) mcpp::build::progress::finish_deferred(); return rc; } @@ -885,6 +974,7 @@ export int cmd_run(const mcpplibs::cmdline::ParsedArgs& parsed, if (parsed.is_flag_set("list-runners")) return mcpp::build::list_runners(package_filter, cache_mode, no_cache, target_triple, features, profile, accel); + mcpp::build::set_print_launch(parsed.is_flag_set("print-launch")); // Closed before the program starts: the program owns the terminal. mcpp::build::progress::open(mcpp::log::is_verbose()); return mcpp::build::build_run_target(targetName, passthrough, package_filter, @@ -1339,6 +1429,49 @@ export int cmd_dyndep(const mcpplibs::cmdline::ParsedArgs& parsed) { // every source is byte-identical and otherwise fails, naming every source // and the destination. One source — every invocation before this feature — // takes the exact path it always has. +// `mcpp link-output --output -- ` (internal: +// invoked by build.ninja's link edges on Windows; design 2026-10-26 D22). +// +// A RUNNING PROGRAM KEEPS ITS FILE; THE NEW ONE TAKES THE NAME. +// +// Windows refuses to overwrite an executable or DLL that is running or loaded +// (error 5), and a relink while `mcpp run` or a test held the old one failed +// with LNK1104. It does allow the file to be RENAMED. So the artifact is moved +// aside to `.old-` and the linker writes the name afresh; the +// running program keeps the renamed file, and the previous `.old-*` files that +// nothing holds any more are deleted here. A process started while the linker +// writes fails to start (file not found, sharing violation) -- Windows never +// starts an image that is being written -- rather than running half of one. +// (POSIX links write `.tmp` and rename it over the name in the +// edge's own shell command: a rename replaces a running program there.) +export int cmd_link_output(const mcpplibs::cmdline::ParsedArgs& parsed, + std::span command) { + namespace fs = std::filesystem; + const fs::path out = parsed.option_or_empty("output").value(); + if (out.empty() || command.empty()) { + std::println(stderr, "error: link-output needs --output -- "); + return 2; + } + std::error_code ec; + const auto dir = out.parent_path().empty() ? fs::path(".") : out.parent_path(); + const auto stem = out.filename().string() + ".old-"; + for (auto it = fs::directory_iterator(dir, ec); !ec && it != fs::directory_iterator(); it.increment(ec)) + if (it->path().filename().string().starts_with(stem)) { + std::error_code rm; + fs::remove(it->path(), rm); // still held by a running program: kept + } + if (fs::exists(out, ec)) { + for (int n = 1; n < 10000; ++n) { + auto aside = out; + aside += std::format(".old-{}", n); + if (fs::exists(aside, ec)) continue; + fs::rename(out, aside, ec); + break; + } + } + return mcpp::platform::process::run_exec(std::vector(command.begin(), command.end())); +} + export int cmd_stage(const mcpplibs::cmdline::ParsedArgs& parsed) { // `--list FILE` (#734 E4): many placements, one process. Each destination // keeps the single-file semantics below -- content comparison, an diff --git a/src/cli/cmd_publish.cppm b/src/cli/cmd_publish.cppm index e5f879571..78d2b2020 100644 --- a/src/cli/cmd_publish.cppm +++ b/src/cli/cmd_publish.cppm @@ -25,6 +25,9 @@ import mcpp.manifest; import mcpp.publish.pipeline; import mcpp.ui; import mcpp.wire; +import mcpp.home; // --list-formats plans outside the project +import mcpp.pack.stage_tree; // the built-in formats +import mcpp.toolchain.fingerprint; // hash_string namespace mcpp::cli { @@ -192,7 +195,62 @@ int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, // the archive or tree of a built-in format, the terminal outputs of a // dispatched one, or the library package. A leg is a `targets` entry of the // artifact it went into, not an artifact of its own. +// `mcpp pack --list-formats` (design 2026-10-26 §5.6): the formats `--format` +// accepts here -- the engine's own (`tar`, `dir`) and each one a package of +// the resolved graph declares with `mcpp::provides_pack_format`, with the +// packages that provide it -- as the `mcpp.pack.formats` envelope. A plan +// that describes (plan_only) in a directory of the mcpp home, as `emit +// build-database` plans: the build programs run, because they are what +// declares a format, and nothing is written into the project. +int cmd_pack_list_formats(const mcpplibs::cmdline::ParsedArgs& parsed) { + auto root = mcpp::project::find_manifest_root(std::filesystem::current_path()); + if (!root) { + mcpp::ui::error("no mcpp.toml found in current directory or any parent"); + return 2; + } + mcpp::build::BuildOverrides ov; + ov.profile = mcpp::build::profile_override_from_flags( + parsed.value("profile").value_or(""), + parsed.is_flag_set("release"), parsed.is_flag_set("dev")); + if (auto v = parsed.value("features")) ov.features = *v; + ov.plan_only = true; + ov.work_dir = mcpp::home::root() / "cache" / "pack-formats" + / mcpp::toolchain::hash_string(root->lexically_normal().generic_string()); + std::expected ctx = std::unexpected(std::string{}); + { + mcpp::platform::terminal::StdoutToStderr narration; + ctx = mcpp::build::prepare_build(/*print_fingerprint=*/false, /*includeDevDeps=*/false, + {}, ov); + } + using mcpp::wire::Effect; + mcpp::wire::Envelope env{.kind = "mcpp.pack.formats", + .effects = {Effect::InitMcppHome, Effect::ReadProject, + Effect::WriteGlobalCache, Effect::ExecBuildScript}}; + if (!ctx) { + env.data = nullptr; + env.diagnostics.push_back({"MCPP_PACK_FORMATS_PLAN_FAILED", mcpp::wire::Severity::Error, + ctx.error()}); + mcpp::wire::emit(env); + return 1; + } + nlohmann::json formats = nlohmann::json::array(); + for (auto b : mcpp::pack::kBuiltinPackFormats) + formats.push_back({{"name", std::string(b)}, {"builtin", true}, + {"providers", nlohmann::json::array()}}); + for (auto const& f : ctx->plan.providedPackFormats) { + if (mcpp::pack::is_builtin_pack_format(f)) continue; + nlohmann::json providers = nlohmann::json::array(); + if (auto it = ctx->packFormatProviders.find(f); it != ctx->packFormatProviders.end()) + for (auto const& p : it->second) providers.push_back(p); + formats.push_back({{"name", f}, {"builtin", false}, {"providers", std::move(providers)}}); + } + env.data = {{"formats", std::move(formats)}}; + mcpp::wire::emit(env); + return 0; +} + export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { + if (parsed.is_flag_set("list-formats")) return cmd_pack_list_formats(parsed); std::string messageFormat = "human"; if (auto mf = parsed.value("message-format")) messageFormat = *mf; if (messageFormat != "human" && messageFormat != "json") { @@ -210,11 +268,16 @@ export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { mcpp::pack::PackOutcome outcome; mcpp::pack::LibraryPackReport library; bool libraryRoute = false; + // mcpp's own errors are the envelope's diagnostics, each located where + // its message says (design 2026-10-26 §5.6). + std::vector errors; + mcpp::ui::set_error_observer([&errors](std::string_view m) { errors.emplace_back(m); }); int rc = 0; { mcpp::platform::terminal::StdoutToStderr narration; rc = cmd_pack_body(parsed, &outcome, &library, &libraryRoute); } + mcpp::ui::set_error_observer({}); using mcpp::wire::Effect; std::vector effects{Effect::ReadProject, Effect::WriteProject, @@ -225,6 +288,8 @@ export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { env.data = nullptr; env.diagnostics.push_back({"MCPP_PACK_FAILED", mcpp::wire::Severity::Error, "mcpp pack did not produce a package; the reason is on standard error"}); + for (auto& e : errors) + env.diagnostics.push_back({"MCPP_PACK_FAILED", mcpp::wire::Severity::Error, std::move(e)}); // A pack of several members names each member that failed, which the // code above cannot: the others may have been packed. if (outcome.members.size() > 1) diff --git a/src/cli/cmd_registry.cppm b/src/cli/cmd_registry.cppm index fa0e1c4ad..b9fc9a48f 100644 --- a/src/cli/cmd_registry.cppm +++ b/src/cli/cmd_registry.cppm @@ -14,17 +14,35 @@ import mcpp.ui; namespace mcpp::cli { +// `--format json` or nothing; anything else is a usage error on stderr with +// no document on stdout (docs/50 §3). +std::optional bad_format(const mcpplibs::cmdline::ParsedArgs& parsed, bool& json) { + json = false; + if (auto f = parsed.value("format")) { + if (*f != "json") { + std::println(stderr, "error: unsupported --format '{}'; expected: json", *f); + return 2; + } + json = true; + } + return std::nullopt; +} + export int cmd_search(const mcpplibs::cmdline::ParsedArgs& parsed) { - std::string keyword = parsed.positional(0); - if (keyword.empty()) { - std::println(stderr, "error: `mcpp search` requires a keyword"); + bool json = false; + if (auto rc = bad_format(parsed, json)) return *rc; + std::string keyword = parsed.positional_count() > 0 ? parsed.positional(0) : std::string{}; + if (keyword.empty() && !json) { + std::println(stderr, "error: `mcpp search` requires a keyword (with --format json, none lists every package)"); return 2; } - return mcpp::pm::search_packages(keyword, parsed.is_flag_set("all-versions")); + return mcpp::pm::search_packages(keyword, parsed.is_flag_set("all-versions"), json); } -export int cmd_index_list(const mcpplibs::cmdline::ParsedArgs& /*parsed*/) { - return mcpp::pm::index_list(); +export int cmd_index_list(const mcpplibs::cmdline::ParsedArgs& parsed) { + bool json = false; + if (auto rc = bad_format(parsed, json)) return *rc; + return mcpp::pm::index_list(json); } export int cmd_index_add(const mcpplibs::cmdline::ParsedArgs& parsed) { diff --git a/src/config.cppm b/src/config.cppm index 5bd25bbc3..e2dec5b8b 100644 --- a/src/config.cppm +++ b/src/config.cppm @@ -124,6 +124,9 @@ struct GlobalConfig { // From config.toml [build] std::int64_t defaultJobs = 0; + // `[build] compile_commands`: the default for a project that does not say + // where its compile database goes (design 2026-10-26 D14). Empty: none. + std::string compileCommands; // From config.toml [toolchain] (M5.5) // default = "@" e.g. "gcc@15.1.0" @@ -451,6 +454,10 @@ search_ttl_seconds = 3600 [build] default_jobs = 0 +# Publish each project's compile database at this path under its root, for a +# project whose mcpp.toml does not say ([build] compile_commands). Unset: the +# database stays in the build directory (target///). +# compile_commands = "compile_commands.json" )"; } @@ -863,6 +870,16 @@ std::expected load_or_init( // 2. Create directory tree std::error_code ec; + // `--deny init-mcpp-home` (design 2026-10-26 §5.5): a home that holds no + // configuration yet is not set up, and the caller said not to. + if (initialization != Initialization::ReadOnly + && !std::filesystem::exists(cfg.configFile, ec) + && mcpp::platform::env::effect_denied("init-mcpp-home")) { + mcpp::platform::env::note_effect_refused("init-mcpp-home", cfg.mcppHome.generic_string()); + return std::unexpected(ConfigError{std::format( + "effect denied: init-mcpp-home -- {} is not set up yet, and the caller denied " + "creating it (--deny init-mcpp-home)", cfg.mcppHome.generic_string())}); + } if (initialization != Initialization::ReadOnly) { for (auto& d : { cfg.binDir, cfg.registryDir, cfg.bmiCacheDir, cfg.metaCacheDir, cfg.logDir }) { @@ -915,6 +932,7 @@ std::expected load_or_init( mcpp::provider::set_index_refresh_timeout(std::chrono::seconds{cfg.indexRefreshTimeoutSeconds}); cfg.searchTtlSeconds = doc->get_int("cache.search_ttl_seconds").value_or(3600); cfg.defaultJobs = doc->get_int("build.default_jobs").value_or(0); + cfg.compileCommands = doc->get_string("build.compile_commands").value_or(""); cfg.defaultToolchain = doc->get_string("toolchain.default").value_or(""); cfg.defaultTarget = doc->get_string("toolchain.default_target").value_or(""); // The system-level configuration (design 2026-10-16 §8.1, §9): what an diff --git a/src/diag.cppm b/src/diag.cppm index cd9bb8e46..3f7db8aba 100644 --- a/src/diag.cppm +++ b/src/diag.cppm @@ -133,7 +133,7 @@ std::string Record::format() const { // before any elaboration. std::string out(what); if (!impact.empty()) out += std::format("\n impact: {}", impact); - if (!hint.empty()) out += std::format("\n hint: {}", hint); + if (!hint.empty()) out += std::format("\n help: {}", hint); return out; } diff --git a/src/doctor.cppm b/src/doctor.cppm index a4d466fca..907e3d5f3 100644 --- a/src/doctor.cppm +++ b/src/doctor.cppm @@ -356,7 +356,7 @@ export int doctor_report() { : std::format("consistent ({} notice(s))", notices)); else warn(std::format("{} record(s) in mcpp's package home point at nothing (first: {})\n" - " hint: `{} self doctor --fix` repairs them", + " help: `{} self doctor --fix` repairs them", broken, example, provider->command_prefix())); } diff --git a/src/modgraph/validate.cppm b/src/modgraph/validate.cppm index a2a22bc10..ff852e8aa 100644 --- a/src/modgraph/validate.cppm +++ b/src/modgraph/validate.cppm @@ -178,7 +178,7 @@ ValidateReport validate(const Graph& g, "lib target without conventional lib root '{}'\n" " impact: `mcpp pack` publishes this library without a module " "interface\n" - " hint: add '{}' re-exporting the public modules with " + " help: add '{}' re-exporting the public modules with " "`export import`, or set [lib].path; a library that " "publishes headers only can ignore this", lib_root_rel.string(), lib_root_rel.string())}); diff --git a/src/pm/commands.cppm b/src/pm/commands.cppm index e71f5f0fa..712d8af4f 100644 --- a/src/pm/commands.cppm +++ b/src/pm/commands.cppm @@ -288,7 +288,7 @@ inline int cmd_add(const mcpplibs::cmdline::ParsedArgs& parsed) { "namespace:" + hint; } if (registryInvolved) { - hint += "\n hint: `mcpp index update` if it was published " + hint += "\n help: `mcpp index update` if it was published " "recently"; } if (!selector.candidates.empty()) { diff --git a/src/pm/index_management.cppm b/src/pm/index_management.cppm index 788823aa0..cadbff123 100644 --- a/src/pm/index_management.cppm +++ b/src/pm/index_management.cppm @@ -20,6 +20,8 @@ import mcpp.pm.index_route; import mcpp.ui; import mcpp.provider; import mcpp.provider.select; +import mcpp.libs.json; +import mcpp.wire; // `--format json` (design 2026-10-26 §5.6) namespace mcpp::pm { @@ -29,7 +31,13 @@ namespace mcpp::pm { // its descriptor's per-OS tables (#487): the latest three by default, all of // them under --all-versions. A hit with no readable descriptor prints exactly // as before — enrichment is best-effort display, never a failure. -export int search_packages(const std::string& keyword, bool allVersions) { +// +// `--format json` (design 2026-10-26 §5.6): the `mcpp.search` envelope, one +// record per package with every version and the descriptor's licenses and +// repository. The keyword may be empty there, which lists the whole catalogue +// in one call -- what an editor's library browser reads instead of parsing +// the index's Lua itself. +export int search_packages(const std::string& keyword, bool allVersions, bool json = false) { auto cfg = mcpp::config::load_or_init(/*quiet=*/false, mcpp::fetcher::make_bootstrap_progress_callback()); if (!cfg) { mcpp::ui::error(cfg.error().message); return 4; } @@ -51,6 +59,37 @@ export int search_packages(const std::string& keyword, bool allVersions) { mcpp::fetcher::Fetcher f(*cfg); auto hits = f.search(keyword); + if (json) { + mcpp::wire::Envelope env{.kind = "mcpp.search", + .effects = {mcpp::wire::Effect::InitMcppHome}}; + if (!hits) { + env.data = nullptr; + env.diagnostics.push_back({"MCPP_SEARCH_FAILED", mcpp::wire::Severity::Error, + hits.error().message}); + mcpp::wire::emit(env); + return 1; + } + nlohmann::json packages = nlohmann::json::array(); + const auto described = f.describe_hits(*hits); + for (std::size_t i = 0; i < hits->size(); ++i) { + const auto& h = (*hits)[i]; + const auto colon = h.name.find(':'); + const auto& catalog = described[i].catalog; + packages.push_back({ + {"id", h.name}, + {"namespace", colon == std::string::npos ? std::string{} : h.name.substr(0, colon)}, + {"name", colon == std::string::npos ? h.name : h.name.substr(colon + 1)}, + {"description", h.description}, + {"versions", described[i].versions}, + {"licenses", catalog.licenses}, + {"repo", catalog.repo}, + {"index", h.source}, + }); + } + env.data = {{"keyword", keyword}, {"packages", std::move(packages)}}; + mcpp::wire::emit(env); + return 0; + } if (!hits) { mcpp::ui::error(hits.error().message); return 1; } if (hits->empty()) { @@ -75,13 +114,46 @@ export int search_packages(const std::string& keyword, bool allVersions) { } // `mcpp index list` — global registries + project [indices]. -export int index_list() { +// `--format json`: the `mcpp.index` envelope -- `registries` (name, url, +// default) and the project's effective `indices` (name, and a path or a url +// with its pin, tag or revision). +export int index_list(bool json = false) { auto cfg = mcpp::config::load_or_init(/*quiet=*/false, mcpp::fetcher::make_bootstrap_progress_callback()); if (!cfg) { mcpp::ui::error(cfg.error().message); return 4; } mcpp::fetcher::Fetcher f(*cfg); auto repos = f.list_repos(); if (!repos) { mcpp::ui::error(repos.error().message); return 1; } + if (json) { + nlohmann::json registries = nlohmann::json::array(); + if (repos->empty()) + for (auto& r : cfg->indexRepos) + registries.push_back({{"name", r.name}, {"url", r.url}, + {"default", r.name == cfg->defaultIndex}}); + else + for (auto& r : *repos) + registries.push_back({{"name", r.name}, {"url", r.url}, + {"default", r.name == cfg->defaultIndex}}); + nlohmann::json project = nlohmann::json::array(); + if (auto root = mcpp::project::find_manifest_root(std::filesystem::current_path())) + for (auto& [name, spec] : mcpp::pm::effective_indices(*root)) { + nlohmann::json e{{"name", name}}; + if (spec.is_local()) e["path"] = spec.path.generic_string(); + else { + e["url"] = spec.url; + if (spec.is_builtin()) e["pin"] = true; + if (!spec.tag.empty()) e["tag"] = spec.tag; + if (!spec.rev.empty()) e["rev"] = spec.rev; + } + project.push_back(std::move(e)); + } + mcpp::wire::Envelope env{.kind = "mcpp.index", + .effects = {mcpp::wire::Effect::InitMcppHome, + mcpp::wire::Effect::ReadProject}}; + env.data = {{"registries", std::move(registries)}, {"indices", std::move(project)}}; + mcpp::wire::emit(env); + return 0; + } if (repos->empty()) { for (auto& r : cfg->indexRepos) { bool isDefault = (r.name == cfg->defaultIndex); diff --git a/src/pm/lock_io.cppm b/src/pm/lock_io.cppm index fafb3e5b3..be0b53239 100644 --- a/src/pm/lock_io.cppm +++ b/src/pm/lock_io.cppm @@ -63,8 +63,13 @@ struct LockError { }; std::expected load(const std::filesystem::path& path); +// Replaces the file atomically (a temporary name beside it, then a rename); +// an unchanged lock is not rewritten. std::expected write(const Lockfile& lock, const std::filesystem::path& path); +// The file's text with CRLF read as LF; nullopt when there is no file. +std::expected, LockError> read_text(const std::filesystem::path& path); + std::string serialize(const Lockfile& lock); std::string compute_hash(const Lockfile& lock); @@ -187,37 +192,70 @@ std::string serialize(const Lockfile& lock) { return out; } -std::expected write(const Lockfile& lock, const std::filesystem::path& path) { - const auto text = serialize(lock); +std::expected, LockError> read_text(const std::filesystem::path& path) { std::error_code ec; const bool exists = std::filesystem::exists(path, ec); if (ec) return std::unexpected(LockError{std::format("cannot inspect '{}'", path.string())}); - if (exists) { - // Reading is enough for an unchanged lock, including a read-only one. - // Accept either line ending without rewriting a checkout on another host. - std::ifstream in(path, std::ios::binary); - if (!in) return std::unexpected(LockError{std::format("cannot read '{}'", path.string())}); - std::string prior; - for (char c; in.get(c); ) prior.push_back(c); - if (in.bad() || !in.eof()) - return std::unexpected(LockError{std::format("cannot read '{}'", path.string())}); - std::string normalized; - normalized.reserve(prior.size()); - for (std::size_t i = 0; i < prior.size(); ++i) { - if (prior[i] == '\r' && i + 1 < prior.size() && prior[i + 1] == '\n') continue; - normalized.push_back(prior[i]); - } - if (normalized == text) return {}; + if (!exists) return std::optional{}; + std::ifstream in(path, std::ios::binary); + if (!in) return std::unexpected(LockError{std::format("cannot read '{}'", path.string())}); + std::string prior; + for (char c; in.get(c); ) prior.push_back(c); + if (in.bad() || !in.eof()) + return std::unexpected(LockError{std::format("cannot read '{}'", path.string())}); + // Either line ending, so a checkout on another host is not rewritten. + std::string normalized; + normalized.reserve(prior.size()); + for (std::size_t i = 0; i < prior.size(); ++i) { + if (prior[i] == '\r' && i + 1 < prior.size() && prior[i + 1] == '\n') continue; + normalized.push_back(prior[i]); } + return std::optional{std::move(normalized)}; +} + +std::expected write(const Lockfile& lock, const std::filesystem::path& path) { + const auto text = serialize(lock); + // Reading is enough for an unchanged lock, including a read-only one. + auto prior = read_text(path); + if (!prior) return std::unexpected(prior.error()); + if (*prior && **prior == text) return {}; + std::error_code ec; if (!path.parent_path().empty()) { std::filesystem::create_directories(path.parent_path(), ec); if (ec) return std::unexpected(LockError{std::format("cannot create directory for '{}'", path.string())}); } - std::ofstream os(path); - if (!os) return std::unexpected(LockError{std::format("cannot write '{}'", path.string())}); - os << text; - os.close(); - if (!os) return std::unexpected(LockError{std::format("cannot write '{}'", path.string())}); + // A linked lock stays a link: the file it names is the one replaced. + auto file = path; + if (std::filesystem::is_symlink(path, ec)) { + auto resolved = std::filesystem::weakly_canonical(path, ec); + if (ec) return std::unexpected(LockError{std::format("cannot resolve '{}'", path.string())}); + file = resolved; + } + // A read-only lock is refused as writing it in place was: the rename below + // would otherwise replace a file its owner made read-only. + if (*prior) { + const auto perms = std::filesystem::status(file, ec).permissions(); + if (!ec && (perms & std::filesystem::perms::owner_write) == std::filesystem::perms::none) + return std::unexpected(LockError{std::format("cannot write '{}'", path.string())}); + } + // Atomic: a concurrent reader sees the old lock or the new one, never part. + auto tmp = file; + tmp += std::format(".{:x}.tmp", std::random_device{}()); + { + std::ofstream os(tmp, std::ios::trunc); // text mode, as before: CRLF on Windows + if (!os) return std::unexpected(LockError{std::format("cannot write '{}'", path.string())}); + os << text; + os.close(); + if (!os) { + std::filesystem::remove(tmp, ec); + return std::unexpected(LockError{std::format("cannot write '{}'", path.string())}); + } + } + std::filesystem::rename(tmp, file, ec); + if (ec) { + std::filesystem::remove(tmp, ec); + return std::unexpected(LockError{std::format("cannot write '{}'", path.string())}); + } return {}; } diff --git a/src/pm/package_fetcher.cppm b/src/pm/package_fetcher.cppm index 2a4007c4f..b71dd6efe 100644 --- a/src/pm/package_fetcher.cppm +++ b/src/pm/package_fetcher.cppm @@ -116,6 +116,17 @@ public: // carries no `ns:name` prefix or has no readable descriptor — display // degrades to the bare line, the search itself never fails on this. std::vector versions_for_hit(const SearchHit& hit) const; + // The descriptor's catalogue fields (licenses, repo); empty as above. + mcpp::manifest::XpkgCatalogFields catalog_for_hit(const SearchHit& hit) const; + // Every hit's versions and catalogue fields at once: the index roots are + // enumerated once and each descriptor is read once, which is what makes a + // whole-catalogue listing (`mcpp search --format json` with no keyword) one + // pass instead of two reads and one provider lookup per package. + struct HitDescription { + std::vector versions; + mcpp::manifest::XpkgCatalogFields catalog; + }; + std::vector describe_hits(std::span hits) const; struct PackageInstall { std::string name; @@ -568,6 +579,15 @@ Fetcher::versions_for_hit(const SearchHit& hit) const { return mcpp::manifest::all_xpkg_versions_desc(*lua); } +mcpp::manifest::XpkgCatalogFields Fetcher::catalog_for_hit(const SearchHit& hit) const { + auto colon = hit.name.find(':'); + if (colon == std::string::npos || colon == 0 || colon + 1 == hit.name.size()) return {}; + auto lua = read_xpkg_lua(std::string_view(hit.name).substr(0, colon), + std::string_view(hit.name).substr(colon + 1)); + if (!lua) return {}; + return mcpp::manifest::xpkg_catalog_fields(*lua); +} + std::expected, CallError> Fetcher::plan_install(const std::vector& targets) { auto args = make_targets_args(targets); @@ -861,6 +881,65 @@ Fetcher::read_xpkg_lua(std::string_view ns, std::string_view shortName) const return std::nullopt; } +std::vector +Fetcher::describe_hits(std::span hits) const { + // EACH INDEX IS READ ONCE, BY DECLARED IDENTITY. A catalogue listing holds + // every package of every xlings index, most of which have no mcpp + // descriptor; looked up one by one, read_xpkg_lua's identity-first fallback + // scanned the whole index for each of those (267 413 opens for 660 hits, + // 8.4 s). One pass over each root's `pkgs/` keyed by the descriptor's own + // `namespace:name` answers every hit, whatever the file is called or + // wherever it sits (the flat `mwrap/` layout included). Identity is read + // from the descriptor's text; a descriptor that declares no namespace is + // keyed by its name alone. + std::map byIdentity; + const auto field = [](std::string_view lua, std::string_view key) -> std::string { + std::size_t pos = 0; + while ((pos = lua.find(key, pos)) != std::string_view::npos) { + const bool lineStart = pos == 0 || lua[pos - 1] == ' ' || lua[pos - 1] == '\t' + || lua[pos - 1] == '\n'; + std::size_t e = pos + key.size(); + while (e < lua.size() && (lua[e] == ' ' || lua[e] == '\t')) ++e; + if (lineStart && e < lua.size() && lua[e] == '=') { + const auto q1 = lua.find('"', e); + const auto nl = lua.find('\n', e); + if (q1 == std::string_view::npos || (nl != std::string_view::npos && q1 > nl)) return {}; + const auto q2 = lua.find('"', q1 + 1); + if (q2 == std::string_view::npos) return {}; + return std::string(lua.substr(q1 + 1, q2 - q1 - 1)); + } + pos += key.size(); + } + return {}; + }; + for (auto const& root : mcpp::provider::open(location_of(cfg_))->index_roots()) { + std::error_code ec; + const auto pkgs = root / "pkgs"; + if (!std::filesystem::is_directory(pkgs, ec)) continue; + for (auto it = std::filesystem::recursive_directory_iterator(pkgs, ec); + !ec && it != std::filesystem::recursive_directory_iterator(); it.increment(ec)) { + if (it->path().extension() != ".lua") continue; + std::ifstream is(it->path(), std::ios::binary); + std::string lua{std::istreambuf_iterator(is), {}}; + const auto name = field(lua, "name"); + if (name.empty()) continue; + byIdentity.try_emplace(field(lua, "namespace") + ":" + name, std::move(lua)); + } + } + std::vector out(hits.size()); + for (std::size_t i = 0; i < hits.size(); ++i) { + auto it = byIdentity.find(hits[i].name); + if (it == byIdentity.end()) { + const auto colon = hits[i].name.find(':'); + if (colon != std::string::npos) it = byIdentity.find(":" + hits[i].name.substr(colon + 1)); + } + if (it == byIdentity.end()) continue; + out[i].versions = mcpp::manifest::all_xpkg_versions_desc(it->second); + out[i].catalog = mcpp::manifest::xpkg_catalog_fields(it->second); + } + return out; +} + // ─── read_xpkg_lua from local path index ──────────────────────────── // // For [indices] entries with `path = "/some/dir"`, read the xpkg .lua @@ -1185,7 +1264,7 @@ Fetcher::resolve_xpkg_path(std::string_view target, if (action == PayloadAction::Refuse) { return std::unexpected(CallError{std::format( "payload of '{}' is not installed\n" - " hint: `mcpp toolchain install` installs a toolchain; a build installs what it needs", + " help: `mcpp toolchain install` installs a toolchain; a build installs what it needs", mcpp::provider::to_string(ref))}); } @@ -1250,7 +1329,7 @@ Fetcher::resolve_xpkg_path(std::string_view target, // said, not stamped over. return std::unexpected(CallError{std::format( "'{}' reported a successful install, but its payload is not complete\n" - " hint: {}", targets[0], install_failure_hint(inst->error, cfg_.xlingsHome()))}); + " help: {}", targets[0], install_failure_hint(inst->error, cfg_.xlingsHome()))}); } // The install failed. What happens next depends on why (D5). @@ -1287,11 +1366,11 @@ Fetcher::resolve_xpkg_path(std::string_view target, auto after = here(); if (!after.dir.empty() && after.usable) return std::unexpected(CallError{std::format( - "{}\n packaging revision {} is still installed and in use\n hint: {}", + "{}\n packaging revision {} is still installed and in use\n help: {}", installError, after.installed, install_failure_hint(inst->error, cfg_.xlingsHome()))}); } return std::unexpected(CallError{std::format( - "{}\n hint: {}", installError, + "{}\n help: {}", installError, install_failure_hint(inst->error, cfg_.xlingsHome()))}); } diff --git a/src/provider/xlings/xlings.cppm b/src/provider/xlings/xlings.cppm index e888a5b74..059d37e20 100644 --- a/src/provider/xlings/xlings.cppm +++ b/src/provider/xlings/xlings.cppm @@ -153,7 +153,14 @@ namespace pinned { // space a script package's hook could not load (mcpp#832). The index // works around it for gcc (openxlings/xim-pkgindex#1035); this floor // closes it for every script package. - inline constexpr std::string_view kXlingsVersion = "2026.10.24.1"; + // + // Thirteenth, at 2026.10.26.1 (openxlings/xlings#707). An interface call + // ends when its capability does (`payload_status`, which every query + // reads, took 517 ms of fixed waits); an upgraded home's projection links + // are proven by its registrations; `--home` selects one run's home; and a + // hook in a project's store finds `xlings` in its bin directory, which + // `gcc.libstdcxx` (an `xim:gcc` edge) needs. + inline constexpr std::string_view kXlingsVersion = "2026.10.26.1"; inline constexpr std::string_view kNasmVersion = "3.02"; } @@ -994,6 +1001,41 @@ mcpp::provider::ProviderInfo describe(const std::filesystem::path& binary, if (ec) return info; const auto key = std::tuple{executable, size, modified}; if (auto it = cache.find(key); it != cache.end()) return it->second; + // AND ACROSS PROCESSES (design 2026-10-26 §8): the answer describes the + // binary, so it is kept under the home keyed as the in-process cache is, + // by the file's path, size and modification time. Every query (`emit + // build-database`, `toolchain list`) started a process for it. + const auto parse = [&](const nlohmann::json& j) { + const auto strings = [&](const char* field) { + std::vector values; + if (j.contains(field) && j[field].is_array()) + for (const auto& value : j[field]) if (value.is_string()) values.push_back(value.get()); + return values; + }; + info.version = j.value("version", std::string{}); + info.protocol = j.value("protocol_version", std::string{}); + info.capabilities = strings("capabilities"); + info.features = strings("features"); + info.homeLayouts.clear(); + if (j.contains("home_layouts") && j["home_layouts"].is_array()) + for (const auto& value : j["home_layouts"]) if (value.is_number_integer()) info.homeLayouts.push_back(value.get()); + info.answered = !info.version.empty(); + }; + const auto stamp = std::format("{}\t{}\t{}", executable.generic_string(), + static_cast(size), static_cast(modified.time_since_epoch().count())); + const auto persisted = mcpp::home::root() / "cache" / "provider-handshake" + / (std::format("{:016x}", std::hash{}(stamp)) + ".json"); + if (std::ifstream in(persisted, std::ios::binary); in) { + auto j = nlohmann::json::parse(in, nullptr, false); + if (j.is_object() && j.value("stamp", std::string{}) == stamp && j.contains("answer")) { + parse(j["answer"]); + if (info.answered) { + cache.emplace(key, info); + return info; + } + info = {}; + } + } const auto temporary = std::filesystem::temp_directory_path(ec); if (ec) return info; const auto home = temporary / std::format("mcpp-provider-handshake-{}-{}", @@ -1018,19 +1060,18 @@ mcpp::provider::ProviderInfo describe(const std::filesystem::path& binary, const auto j = !timedOut && output.exit_code == 0 ? nlohmann::json::parse(output.output, nullptr, false) : nlohmann::json{}; if (j.is_object()) { - const auto strings = [&](const char* field) { - std::vector values; - if (j.contains(field) && j[field].is_array()) - for (const auto& value : j[field]) if (value.is_string()) values.push_back(value.get()); - return values; - }; - info.version = j.value("version", std::string{}); - info.protocol = j.value("protocol_version", std::string{}); - info.capabilities = strings("capabilities"); - info.features = strings("features"); - if (j.contains("home_layouts") && j["home_layouts"].is_array()) - for (const auto& value : j["home_layouts"]) if (value.is_number_integer()) info.homeLayouts.push_back(value.get()); - info.answered = !info.version.empty(); + parse(j); + // Written only by a run that writes the global cache, for the + // reason the probe cache gives. + if (info.answered && mcpp::platform::env::writes_global_cache() + && std::filesystem::is_directory(mcpp::home::root(), ec)) { + std::filesystem::create_directories(persisted.parent_path(), ec); + auto tmp = persisted; + tmp += std::format(".{:x}", std::random_device{}()); + { std::ofstream(tmp, std::ios::binary) << nlohmann::json{{"stamp", stamp}, {"answer", j}}.dump(); } + std::filesystem::rename(tmp, persisted, ec); + if (ec) std::filesystem::remove(tmp, ec); + } } } catch (const std::exception&) { info = {}; } cache.emplace(key, info); diff --git a/src/runtime/elf.cppm b/src/runtime/elf.cppm index f56426493..93988b911 100644 --- a/src/runtime/elf.cppm +++ b/src/runtime/elf.cppm @@ -169,6 +169,26 @@ constexpr bool is_vague_linkage_binding(unsigned char binding) { std::expected, std::string> defined_object_symbols(const std::filesystem::path& object); +// The names an ELF relocatable object REFERENCES and does not define: GLOBAL +// and WEAK entries of `.symtab` with section index SHN_UNDEF. What the island +// check of a C++ runtime reads (design 2026-10-26 §6.3, condition 4): which +// runtime symbols an object compiled against another runtime will bind to. +std::expected, std::string> +undefined_object_symbols(const std::filesystem::path& object); + +// One undefined entry of an image's DYNAMIC symbol table, with the version +// it was linked against (`GLIBCXX_3.4`, `CXXABI_1.3`; empty for none). The +// version names the library that answered the reference at link time, which +// is what the island check reads (condition 5): an image whose own C++ +// runtime is embedded must not import the ABI layer at the other runtime's +// versions. +struct VersionedImport { + std::string name; + std::string version; +}; +std::expected, std::string> +versioned_dynamic_imports(const std::filesystem::path& image); + // What an ELF object's DYNAMIC symbol table says it provides. // // Deliberately not folded into ElfRuntimeFacts: every caller of @@ -805,8 +825,98 @@ inspect_dynamic_symbols(const std::filesystem::path& object) { return out; } +namespace { +std::expected, std::string> +object_symbols(const std::filesystem::path& object, bool defined); +} + std::expected, std::string> defined_object_symbols(const std::filesystem::path& object) { + return object_symbols(object, true); +} + +std::expected, std::string> +undefined_object_symbols(const std::filesystem::path& object) { + return object_symbols(object, false); +} + +std::expected, std::string> +versioned_dynamic_imports(const std::filesystem::path& image) { + detail::Reader reader; + if (!reader.load(image)) return std::unexpected(std::format( + "cannot open ELF image '{}'", image.string())); + if (reader.bytes.size() < 0x40 || reader.bytes[0] != 0x7f || reader.bytes[1] != 'E' + || reader.bytes[2] != 'L' || reader.bytes[3] != 'F') + return std::unexpected(std::format("image '{}' is not ELF", image.string())); + if (reader.bytes[4] != 2 || reader.bytes[5] != 1) + return std::unexpected(std::format("image '{}' is not ELF64 little-endian", image.string())); + auto shoff = reader.u64(0x28); + auto shentsize = reader.u16(0x3A); + auto shnum = reader.u16(0x3C); + if (!shoff || !shentsize || !shnum || *shentsize < 0x40 || *shnum > 65000 + || !reader.range(*shoff, static_cast(*shentsize) * *shnum)) + return std::unexpected(std::format("image '{}' has a truncated section table", image.string())); + struct Sec { std::uint32_t type = 0; std::uint64_t offset = 0, size = 0; std::uint32_t link = 0; }; + std::vector secs(*shnum); + for (std::uint16_t i = 0; i < *shnum; ++i) { + const auto at = *shoff + static_cast(i) * *shentsize; + secs[i] = {reader.u32(at + 0x04).value_or(0), reader.u64(at + 0x18).value_or(0), + reader.u64(at + 0x20).value_or(0), reader.u32(at + 0x28).value_or(0)}; + } + constexpr std::uint32_t kShtDynsym = 11, kVersym = 0x6fffffff, kVerneed = 0x6ffffffe; + const Sec* dynsym = nullptr; const Sec* versym = nullptr; const Sec* verneed = nullptr; + for (auto const& sec : secs) { + if (sec.type == kShtDynsym) dynsym = &sec; + else if (sec.type == kVersym) versym = &sec; + else if (sec.type == kVerneed) verneed = &sec; + } + std::vector out; + if (!dynsym || dynsym->link >= secs.size()) return out; // static: no imports + const auto& dynstr = secs[dynsym->link]; + // Version index -> name, from the needed-version records. + std::map versionName; + if (verneed && verneed->link < secs.size() && reader.range(verneed->offset, verneed->size)) { + const auto& vstr = secs[verneed->link]; + std::uint64_t at = verneed->offset; + for (int guard = 0; guard < 4096; ++guard) { + auto cnt = reader.u16(at + 2); auto aux = reader.u32(at + 8); auto next = reader.u32(at + 12); + if (!cnt || !aux || !next) break; + std::uint64_t a = at + *aux; + for (std::uint16_t k = 0; k < *cnt; ++k) { + auto other = reader.u16(a + 6); auto nameOff = reader.u32(a + 8); auto anext = reader.u32(a + 12); + if (!other || !nameOff || !anext) break; + if (*nameOff < vstr.size) + if (auto n = reader.cstr(vstr.offset + *nameOff, vstr.size - *nameOff)) + versionName[static_cast(*other & 0x7fff)] = *n; + if (*anext == 0) break; + a += *anext; + } + if (*next == 0) break; + at += *next; + } + } + const auto count = dynsym->size / detail::kSymEntrySize; + for (std::uint64_t i = 1; i < count; ++i) { + const auto sym = dynsym->offset + i * detail::kSymEntrySize; + auto nameOff = reader.u32(sym); + auto shndx = reader.u16(sym + 6); + if (!nameOff || !shndx || *shndx != 0) continue; // defined + if (*nameOff >= dynstr.size) continue; + auto name = reader.cstr(dynstr.offset + *nameOff, dynstr.size - *nameOff); + if (!name || name->empty()) continue; + std::string version; + if (versym && reader.range(versym->offset + i * 2, 2)) + if (auto v = reader.u16(versym->offset + i * 2)) + if (auto it = versionName.find(static_cast(*v & 0x7fff)); it != versionName.end()) + version = it->second; + out.push_back({std::move(*name), std::move(version)}); + } + return out; +} + +namespace { +std::expected, std::string> +object_symbols(const std::filesystem::path& object, bool defined) { detail::Reader reader; if (!reader.load(object)) return std::unexpected(std::format( "cannot open ELF object '{}'", object.string())); @@ -856,7 +966,7 @@ defined_object_symbols(const std::filesystem::path& object) { auto nameOffset = reader.u32(sym); auto shndx = reader.u16(sym + 6); if (!nameOffset || !shndx) break; - if (*shndx == 0) continue; // SHN_UNDEF + if ((*shndx == 0) == defined) continue; // SHN_UNDEF: wanted only when !defined const unsigned char bind = static_cast(reader.bytes[sym + 4] >> 4); if (bind == detail::kStbLocal) continue; @@ -869,6 +979,7 @@ defined_object_symbols(const std::filesystem::path& object) { names.erase(std::ranges::unique(names).begin(), names.end()); return names; } +} // namespace std::expected inspect_elf_runtime(const std::filesystem::path& artifact) { diff --git a/src/toolchain/clang.cppm b/src/toolchain/clang.cppm index 75dd42a0e..9c39ef79f 100644 --- a/src/toolchain/clang.cppm +++ b/src/toolchain/clang.cppm @@ -249,10 +249,18 @@ std::filesystem::path staged_std_bmi_path(const std::filesystem::path& outputDir // the module purview, which clang reports. One answer for `std` and // `std.compat`, decided by the file and not by the machine running the build; // a libc++ `.cppm` receives nothing, so its commands are unchanged. +// The language of a `std` module source clang does not know by its name. +// `.cppm` (libc++'s) and the other module-interface extensions are +// recognised; the MSVC STL's `.ixx` and libstdc++'s `bits/std.cc` are not -- +// the second precompiled as an ordinary translation unit, and the object step +// then answered `unable to load precompiled file` (design 2026-10-26 §6.4: a +// graph-supplied libstdc++ under clang). The `.ixx` spelling is unchanged, so +// the std module caches built from it keep their identity. static std::string module_source_language_flags(const std::filesystem::path& source) { - return source.extension() == ".ixx" - ? " -x c++-module -Wno-include-angled-in-module-purview" - : ""; + const auto ext = source.extension().string(); + if (ext == ".ixx") return " -x c++-module -Wno-include-angled-in-module-purview"; + if (ext == ".cppm" || ext == ".ccm" || ext == ".cxxm" || ext == ".c++m") return ""; + return " -x c++-module"; } std::vector std_module_build_commands(const Toolchain& tc, diff --git a/src/toolchain/lifecycle.cppm b/src/toolchain/lifecycle.cppm index 403bfc2b0..689a3e657 100644 --- a/src/toolchain/lifecycle.cppm +++ b/src/toolchain/lifecycle.cppm @@ -987,7 +987,7 @@ export int toolchain_install(const mcpp::config::GlobalConfig& cfg, auto bsProblem = mcpp::config::check_base_init(cfg); if (!bsProblem.empty()) { mcpp::ui::error(std::format( - "{}\n hint: run `mcpp self init --force` to reset and re-initialize", + "{}\n help: run `mcpp self init --force` to reset and re-initialize", bsProblem)); return 1; } diff --git a/src/toolchain/probe.cppm b/src/toolchain/probe.cppm index 642ec4468..db14ee1a3 100644 --- a/src/toolchain/probe.cppm +++ b/src/toolchain/probe.cppm @@ -19,6 +19,8 @@ import mcpp.provider; import mcpp.provider.select; import mcpp.platform; import mcpp.log; +import mcpp.home; // the probe cache lives under the home's cache +import mcpp.toolchain.fingerprint; // hash_string import mcpp.fallback.sysroot_complete; import mcpp.fallback.probe_sysroot; @@ -99,8 +101,65 @@ std::string env_prefix_for_dirs(const std::vector& dirs) } // namespace +namespace { +// THE ANSWERS OF A COMPILER'S QUERIES, CACHED (design 2026-10-26 §8). Every +// plan asked the same compiler `--version`, `-dumpmachine` and `-print-sysroot` +// through a shell and an `env` -- three processes each, about 20 ms of a 100 ms +// query -- and a compiler answers them identically until its files change. +// Keyed by the command text and by the size and modification time of every +// file the command names (the compiler, its configuration), so a payload +// replaced in place, a different environment prefix or another compiler is a +// different key. Only a successful answer is kept; a failed query runs again. +std::string probe_cache_key(const std::string& cmd) { + std::string key = cmd; + std::size_t pos = 0; + while ((pos = cmd.find('\'', pos)) != std::string::npos) { + const auto end = cmd.find('\'', pos + 1); + if (end == std::string::npos) break; + const std::filesystem::path p(cmd.substr(pos + 1, end - pos - 1)); + std::error_code ec; + if (p.is_absolute() && std::filesystem::is_regular_file(p, ec)) { + const auto size = std::filesystem::file_size(p, ec); + const auto time = std::filesystem::last_write_time(p, ec); + key += std::format("\n{}\t{}\t{}", p.generic_string(), + static_cast(size), + static_cast(time.time_since_epoch().count())); + } + pos = end + 1; + } + return key; +} +std::filesystem::path probe_cache_file(const std::string& key) { + return mcpp::home::root() / "cache" / "probe" / (mcpp::toolchain::hash_string(key) + ".out"); +} +} // namespace + std::expected run_capture(const std::string& cmd) { + const auto key = probe_cache_key(cmd); + const auto cached = probe_cache_file(key); + if (std::ifstream in(cached, std::ios::binary); in) { + std::string recordedKey; + std::getline(in, recordedKey, '\0'); + if (recordedKey == key) return std::string(std::istreambuf_iterator(in), {}); + } auto r = mcpp::platform::process::capture_host_tool(cmd); + // Written only by a run that writes the global cache: a query (`self + // doctor`, `toolchain list`) reads the answer and leaves the home as it + // found it, and a cold home is never created for it. + std::error_code homeEc; + if (r.exit_code == 0 && mcpp::platform::env::writes_global_cache() + && std::filesystem::is_directory(mcpp::home::root(), homeEc)) { + std::error_code ec; + std::filesystem::create_directories(cached.parent_path(), ec); + auto tmp = cached; + tmp += std::format(".{:x}", std::random_device{}()); + { + std::ofstream out(tmp, std::ios::binary | std::ios::trunc); + out << key << '\0' << r.output; + } + std::filesystem::rename(tmp, cached, ec); + if (ec) std::filesystem::remove(tmp, ec); + } if (r.exit_code != 0 && r.output.empty()) { return std::unexpected(DetectError{std::format("failed to execute: {}", cmd)}); } diff --git a/src/ui.cppm b/src/ui.cppm index 374446de9..fafc589c9 100644 --- a/src/ui.cppm +++ b/src/ui.cppm @@ -84,6 +84,9 @@ void finished(std::string_view profile, std::chrono::milliseconds elapsed, // "warning:" / "error:" prefix lines (yellow / red). void warning(std::string_view message); void error(std::string_view message); +// Receives each message `error` prints, while set: an enveloped command +// carries its errors as diagnostics (design 2026-10-26 §5.6). Empty clears. +void set_error_observer(std::function observer); // "note:" prefix line (cyan), on stderr like the two above: a statement that // changes nothing the build does and that a reader may want to act on. void note(std::string_view message); @@ -437,7 +440,17 @@ bool detect_color(term::Stream s) { if (auto* e = std::getenv("NO_COLOR"); e && *e) return false; // On Windows this also turns the console's escape processing on, without // which the colour sequences would be printed as text. - return term::can_move_cursor(s); + const bool terminal = term::can_move_cursor(s); + // COLOUR ASKED FOR WHERE THERE IS NO TERMINAL (design 2026-10-26 D15): + // an IDE's console, a CI log or a pager reads mcpp through a pipe and + // renders the escapes itself. `CLICOLOR_FORCE` (any value but "0") and + // `FORCE_COLOR` (any value but "0" or "false") are the two conventions + // such hosts set; NO_COLOR and MCPP_NO_COLOR, above, still win. Colour + // only: the status row needs a terminal and stays off in a pipe. + if (auto* e = std::getenv("CLICOLOR_FORCE"); e && *e && std::string_view(e) != "0") return true; + if (auto* e = std::getenv("FORCE_COLOR"); e && *e && std::string_view(e) != "0" + && std::string_view(e) != "false") return true; + return terminal; } std::string with_color(std::string_view code, std::string_view text) { @@ -796,7 +809,21 @@ void warning(std::string_view message) { : std::format("warning: {}\n", message)); } +namespace { +std::mutex g_errorObserverMutex; +std::function g_errorObserver; +} // namespace + +void set_error_observer(std::function observer) { + std::lock_guard lock(g_errorObserverMutex); + g_errorObserver = std::move(observer); +} + void error(std::string_view message) { + { + std::lock_guard lock(g_errorObserverMutex); + if (g_errorObserver) g_errorObserver(message); + } init(); emit(term::Stream::Err, colored(term::Stream::Err) ? std::format("{}{}error:{} {}\n", kBold, kBrightRed, kReset, message) diff --git a/src/wire.cppm b/src/wire.cppm index f548923c5..b14b59315 100644 --- a/src/wire.cppm +++ b/src/wire.cppm @@ -68,7 +68,7 @@ inline constexpr int kEnvelopeVersion = 1; // from what comes back. struct KindVersion { std::string_view kind; int version; }; -inline constexpr std::array kKinds{{ +inline constexpr std::array kKinds{{ {"mcpp.env", 1}, {"mcpp.xpkg", 1}, {"mcpp.cache", 1}, @@ -102,6 +102,17 @@ inline constexpr std::array kKinds{{ // made from (`data.stage`). `--format` names the package format on that // command, hence the `test`-style spelling (docs/50 §3). {"mcpp.pack", 1}, + // `mcpp build --message-format json` (design 2026-10-26 §5.1): the result + // record of the build (`data`), after the NDJSON `mcpp.event` lines. + {"mcpp.build", 1}, + // `mcpp run --print-launch --format json` (§5.3): how `mcpp run` would + // start the program -- argv, the environment it injects, cwd, the runner. + {"mcpp.launch", 1}, + // `mcpp search --format json` and `mcpp index list --format json` (§5.6). + {"mcpp.search", 1}, + {"mcpp.index", 1}, + // `mcpp pack --list-formats --format json` (§5.6). + {"mcpp.pack.formats", 1}, }}; // What running a command does, beyond writing to stdout. @@ -179,7 +190,58 @@ inline int kind_version(std::string_view kind) { return 0; } -inline nlohmann::json to_json(const Diagnostic& d) { +// The source location a message begins with: `::: `, +// `:: ` or `: error: ` -- the form every located message of +// mcpp's is written in (a manifest's rejections, a compiler's lines). A +// Windows drive letter's colon belongs to the path. +struct Located { std::string path; int line = 0; int column = 0; }; +inline std::optional message_location(std::string_view msg) { + auto number = [](std::string_view s, int& v) { + return !s.empty() && std::from_chars(s.data(), s.data() + s.size(), v).ec == std::errc{} + && v > 0; + }; + const auto end = msg.find(": "); + if (end == std::string_view::npos || end == 0) return std::nullopt; + auto loc = msg.substr(0, end); + if (loc.find(' ') != std::string_view::npos && loc.find('/') == std::string_view::npos + && loc.find('\\') == std::string_view::npos) + return std::nullopt; + Located l; + int a = 0, b = 0; + auto c2 = loc.rfind(':'); + if (c2 != std::string_view::npos && c2 > 1 && number(loc.substr(c2 + 1), a)) { + auto head = loc.substr(0, c2); + auto c1 = head.rfind(':'); + if (c1 != std::string_view::npos && c1 > 1 && number(head.substr(c1 + 1), b)) { + l.path = std::string(head.substr(0, c1)); l.line = b; l.column = a; + } else { + l.path = std::string(head); l.line = a; + } + return l; + } + if (msg.substr(end + 2).starts_with("error: ") || msg.substr(end + 2).starts_with("warning: ")) { + l.path = std::string(loc); + return l; + } + return std::nullopt; +} + +// A diagnostic without a range whose message begins with a location carries +// that location as `path` and `range` (docs/50 §2): a caller that already +// named the file keeps its spelling of it, and gains the line. +inline nlohmann::json to_json(const Diagnostic& in) { + Diagnostic d = in; + if (!d.range) { + if (auto l = message_location(d.message); + l && (d.path.empty() || std::filesystem::path(l->path).filename() + == std::filesystem::path(d.path).filename())) { + if (d.path.empty()) d.path = l->path; + if (l->line > 0) { + const int col = std::max(l->column, 1); + d.range = Range{{l->line, col}, {l->line, col}}; + } + } + } nlohmann::json j{ {"code", d.code}, {"severity", std::string(severity_name(d.severity))}, diff --git a/tests/e2e/100_cppfly_reflection.sh b/tests/e2e/100_cppfly_reflection.sh index 2e04dc6f5..38cb52f41 100755 --- a/tests/e2e/100_cppfly_reflection.sh +++ b/tests/e2e/100_cppfly_reflection.sh @@ -7,6 +7,9 @@ # Tail section: standard = "c++latest" regression — it used to reach the GNU # driver as the invalid spelling -std=c++latest (design §11-Q5). set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap "rm -rf $TMP" EXIT diff --git a/tests/e2e/105_asm_sources_nasm.sh b/tests/e2e/105_asm_sources_nasm.sh index ad50a5fd6..ec8b0e433 100755 --- a/tests/e2e/105_asm_sources_nasm.sh +++ b/tests/e2e/105_asm_sources_nasm.sh @@ -5,6 +5,9 @@ # include dirs + %include tracked through the depfile (-MD), excluded from # compile_commands.json, and linked into the binary. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json [[ "$(uname -m)" == "x86_64" ]] || { echo "SKIP: x86-only NASM"; exit 0; } diff --git a/tests/e2e/126_default_features_opt_out.sh b/tests/e2e/126_default_features_opt_out.sh index c50d81f76..22a8e7278 100755 --- a/tests/e2e/126_default_features_opt_out.sh +++ b/tests/e2e/126_default_features_opt_out.sh @@ -14,6 +14,9 @@ # # No `requires:` capability → runs on all three CI platforms. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap "rm -rf $TMP" EXIT diff --git a/tests/e2e/128_feature_forwarding.sh b/tests/e2e/128_feature_forwarding.sh index 1bdeb7734..216964628 100755 --- a/tests/e2e/128_feature_forwarding.sh +++ b/tests/e2e/128_feature_forwarding.sh @@ -22,6 +22,9 @@ # # No `requires:` capability → runs on all three CI platforms. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap "rm -rf $TMP" EXIT diff --git a/tests/e2e/171_bmi_staging_locked_dest.sh b/tests/e2e/171_bmi_staging_locked_dest.sh index b90bbb0a0..f35a0effe 100755 --- a/tests/e2e/171_bmi_staging_locked_dest.sh +++ b/tests/e2e/171_bmi_staging_locked_dest.sh @@ -132,7 +132,7 @@ if [[ $rc -eq 0 ]]; then cat stage_err.log echo "FAIL: staging onto an unusable destination reported success"; exit 1 fi -grep -q "hint:" stage_err.log || { +grep -q "help:" stage_err.log || { cat stage_err.log; echo "FAIL: staging failure carried no hint"; exit 1; } grep -q "clangd" stage_err.log || { cat stage_err.log; echo "FAIL: hint does not name the usual holder (clangd)"; exit 1; } diff --git a/tests/e2e/175_cpp20_import_std.sh b/tests/e2e/175_cpp20_import_std.sh index ee388c6dc..d0bf19797 100755 --- a/tests/e2e/175_cpp20_import_std.sh +++ b/tests/e2e/175_cpp20_import_std.sh @@ -13,6 +13,9 @@ # Asserts: build + run succeed, -std=c++20 reaches every surface (global # cxxflags, compile_commands.json, the std BMI prebuild), and no c++23 leaks in. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap "rm -rf $TMP" EXIT diff --git a/tests/e2e/177_cpp20_msvc.sh b/tests/e2e/177_cpp20_msvc.sh index 76bd9379d..7315e6933 100755 --- a/tests/e2e/177_cpp20_msvc.sh +++ b/tests/e2e/177_cpp20_msvc.sh @@ -13,6 +13,9 @@ # `import std;` compiles and runs, and the runner's cl version is printed so the # log records WHICH toolset the claim was verified on. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json CONF="${MCPP_HOME:-$HOME/.mcpp}/config.toml" ORIG_DEFAULT="" diff --git a/tests/e2e/211_configure_only_cdb.sh b/tests/e2e/211_configure_only_cdb.sh index 5f5be2080..6ba36cade 100755 --- a/tests/e2e/211_configure_only_cdb.sh +++ b/tests/e2e/211_configure_only_cdb.sh @@ -4,6 +4,9 @@ # compile database before compiling source files. The generated database is # intended for clangd, so a broken source is deliberately part of the test. set -euo pipefail +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap 'rm -rf "$TMP"' EXIT diff --git a/tests/e2e/304_the_target_side_reaches_every_unit.sh b/tests/e2e/304_the_target_side_reaches_every_unit.sh index 7b4d3536d..9c1b65b26 100755 --- a/tests/e2e/304_the_target_side_reaches_every_unit.sh +++ b/tests/e2e/304_the_target_side_reaches_every_unit.sh @@ -29,6 +29,9 @@ # says what each unit is actually compiled with; a grep over `build.ninja` would # also match the global rule and pass for the wrong reason. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json MCPP="${MCPP:-mcpp}" work="$(mktemp -d)" diff --git a/tests/e2e/305_built_from_is_not_published.sh b/tests/e2e/305_built_from_is_not_published.sh index d0373d500..f5c8db705 100755 --- a/tests/e2e/305_built_from_is_not_published.sh +++ b/tests/e2e/305_built_from_is_not_published.sh @@ -25,6 +25,9 @@ # the consumer's command line — and, separately, that a consumer using the name # as an ordinary identifier compiles, which is what the user actually reported. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json MCPP="${MCPP:-mcpp}" work="$(mktemp -d)" diff --git a/tests/e2e/321_workspace_inheritance.sh b/tests/e2e/321_workspace_inheritance.sh index 583f42fde..def60a8e6 100755 --- a/tests/e2e/321_workspace_inheritance.sh +++ b/tests/e2e/321_workspace_inheritance.sh @@ -19,6 +19,9 @@ # an assertion on the first proves # nothing about it set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap "rm -rf $TMP" EXIT diff --git a/tests/e2e/324_fast_path_sees_path_deps.sh b/tests/e2e/324_fast_path_sees_path_deps.sh index f65ef41ca..fd32100fa 100755 --- a/tests/e2e/324_fast_path_sees_path_deps.sh +++ b/tests/e2e/324_fast_path_sees_path_deps.sh @@ -18,6 +18,9 @@ # changing") in a directory that fix did not reach. It matters most where it is # hardest to notice: workspace members depend on each other by `path`. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap "rm -rf $TMP" EXIT diff --git a/tests/e2e/328_layer_predicates_apply_and_unknown_ones_say_so.sh b/tests/e2e/328_layer_predicates_apply_and_unknown_ones_say_so.sh index 7a271874c..00fe257af 100755 --- a/tests/e2e/328_layer_predicates_apply_and_unknown_ones_say_so.sh +++ b/tests/e2e/328_layer_predicates_apply_and_unknown_ones_say_so.sh @@ -23,6 +23,9 @@ # # requires: gcc unix-shell set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json MCPP="${MCPP:-mcpp}" TMP=$(mktemp -d) diff --git a/tests/e2e/47_cdb_prebuilt_module_path_abs.sh b/tests/e2e/47_cdb_prebuilt_module_path_abs.sh index 8a9d58438..d74491a0b 100755 --- a/tests/e2e/47_cdb_prebuilt_module_path_abs.sh +++ b/tests/e2e/47_cdb_prebuilt_module_path_abs.sh @@ -12,6 +12,9 @@ # break clangd's module resolution while `mcpp build` itself keeps working # (ninja unescapes its own escape sequences). set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap "rm -rf $TMP" EXIT diff --git a/tests/e2e/59_cpp_standard_config.sh b/tests/e2e/59_cpp_standard_config.sh index 1ac2834e4..4f429e679 100644 --- a/tests/e2e/59_cpp_standard_config.sh +++ b/tests/e2e/59_cpp_standard_config.sh @@ -2,6 +2,9 @@ # requires: gcc # C++ standard config: [package].standard drives build flags, CDB, and std BMI. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap "rm -rf $TMP" EXIT diff --git a/tests/e2e/647_abi_threads_is_one_switch_for_the_graph.sh b/tests/e2e/647_abi_threads_is_one_switch_for_the_graph.sh index 87f4de70a..b22becda1 100755 --- a/tests/e2e/647_abi_threads_is_one_switch_for_the_graph.sh +++ b/tests/e2e/647_abi_threads_is_one_switch_for_the_graph.sh @@ -16,6 +16,9 @@ # 3. a dependency that writes the table itself is reported and changes nothing; # 4. a table scoped to another target changes nothing in a host build. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap "rm -rf $TMP" EXIT diff --git a/tests/e2e/654_a_requirement_on_the_target_axis.sh b/tests/e2e/654_a_requirement_on_the_target_axis.sh index 7420a6f29..d7913b257 100755 --- a/tests/e2e/654_a_requirement_on_the_target_axis.sh +++ b/tests/e2e/654_a_requirement_on_the_target_axis.sh @@ -19,6 +19,9 @@ # nothing, in both directions -- present or absent, the Linux build is # unaffected by a requirement scoped to a target it is not building. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap "rm -rf $TMP" EXIT diff --git a/tests/e2e/688_emit_build_database.sh b/tests/e2e/688_emit_build_database.sh index 3a1e21b43..0f70d242f 100755 --- a/tests/e2e/688_emit_build_database.sh +++ b/tests/e2e/688_emit_build_database.sh @@ -36,6 +36,9 @@ # BMI. The control leg at the end runs `build --configure-only` on a copy # in that home and sees the std module compiled there. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) # the measured tree: the project and its dev-dependency OUT=$(mktemp -d) # everything this script writes, outside the measured tree diff --git a/tests/e2e/735_offline_missing_package_has_its_own_code.sh b/tests/e2e/735_offline_missing_package_has_its_own_code.sh index c36d8e8a5..6b6639709 100755 --- a/tests/e2e/735_offline_missing_package_has_its_own_code.sh +++ b/tests/e2e/735_offline_missing_package_has_its_own_code.sh @@ -6,7 +6,9 @@ # The package comes from a local path index, so the descriptor is present and # the download is the only thing missing. Criteria: # A. offline, the envelope's code is MCPP_OFFLINE_DOWNLOAD_REQUIRED, its -# message names the package and its version, and the exit status is 1; +# message names the package and its version, and the exit status is 5: +# the run stopped before a denied effect (`--offline` is `--deny network`, +# SPEC-003, 2026.10.26.1; it was 1 before); # B. a plan that fails for another reason keeps MCPP_BUILD_DATABASE_PLAN_FAILED. set -e source "$(dirname "$0")/_host_path.sh" @@ -53,7 +55,7 @@ set +e MCPP_OFFLINE=1 "$MCPP" emit build-database --format json > a.json 2> a.err rc=$? set -e -[ "$rc" -eq 1 ] || fail "A: exit status $rc, expected 1" a.err +[ "$rc" -eq 5 ] || fail "A: exit status $rc, expected 5" a.err code_of a.json > a.txt sed -n 1p a.txt | grep -qx MCPP_OFFLINE_DOWNLOAD_REQUIRED || fail "A: wrong code" a.txt grep -q "widget" a.txt && grep -q "1.0.0" a.txt || fail "A: the message does not name the package and version" a.txt diff --git a/tests/e2e/736_compile_flag_words_reach_the_compiler_and_the_databases.sh b/tests/e2e/736_compile_flag_words_reach_the_compiler_and_the_databases.sh index 8e3b0bb35..eb3c28faa 100644 --- a/tests/e2e/736_compile_flag_words_reach_the_compiler_and_the_databases.sh +++ b/tests/e2e/736_compile_flag_words_reach_the_compiler_and_the_databases.sh @@ -14,6 +14,9 @@ # and does not name the ones whose words did not; # E. a build that repeats the plan names nothing. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap 'rm -rf "$TMP"' EXIT diff --git a/tests/e2e/738_a_graph_supplied_target_closes_the_hosts_own_search.sh b/tests/e2e/738_a_graph_supplied_target_closes_the_hosts_own_search.sh index fa9111e5f..a536ca388 100755 --- a/tests/e2e/738_a_graph_supplied_target_closes_the_hosts_own_search.sh +++ b/tests/e2e/738_a_graph_supplied_target_closes_the_hosts_own_search.sh @@ -37,6 +37,9 @@ source "$(dirname "${BASH_SOURCE[0]}")/_toolchain_env.sh" # only then can their absence from graph-supplied commands prove isolation. # Ordinary suites can still use the distro's host MinGW prefix. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json MCPP="${MCPP:-mcpp}" TMP=$(mktemp -d) diff --git a/tests/e2e/746_macos_deployment_target_is_target_keyed_not_host_keyed.sh b/tests/e2e/746_macos_deployment_target_is_target_keyed_not_host_keyed.sh index 4e253b8b6..8cce94252 100755 --- a/tests/e2e/746_macos_deployment_target_is_target_keyed_not_host_keyed.sh +++ b/tests/e2e/746_macos_deployment_target_is_target_keyed_not_host_keyed.sh @@ -27,6 +27,9 @@ # `[target.aarch64-macos] toolchain = "llvm@"` override is the escape # hatch that reaches this without one (measured). set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json find_llvm() { local home="${MCPP_HOME:-$HOME/.mcpp}" diff --git a/tests/e2e/765_a_consumer_include_directory_stays_in_the_consumer.sh b/tests/e2e/765_a_consumer_include_directory_stays_in_the_consumer.sh index f79ad538a..95a6cffa6 100755 --- a/tests/e2e/765_a_consumer_include_directory_stays_in_the_consumer.sh +++ b/tests/e2e/765_a_consumer_include_directory_stays_in_the_consumer.sh @@ -23,6 +23,9 @@ # now fails, and the output names the root directory that has it, on the # full path and again on the fast path. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap "rm -rf $TMP" EXIT diff --git a/tests/e2e/76_compile_commands_generated.sh b/tests/e2e/76_compile_commands_generated.sh index b812ff773..5119882f7 100755 --- a/tests/e2e/76_compile_commands_generated.sh +++ b/tests/e2e/76_compile_commands_generated.sh @@ -9,6 +9,9 @@ # # No `requires:` capability → runs on all three CI platforms. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap "rm -rf $TMP" EXIT diff --git a/tests/e2e/770_workspace_member_as_dependency.sh b/tests/e2e/770_workspace_member_as_dependency.sh index 2eccd7bd0..9f5584a03 100644 --- a/tests/e2e/770_workspace_member_as_dependency.sh +++ b/tests/e2e/770_workspace_member_as_dependency.sh @@ -18,6 +18,9 @@ # D. A member of a git-hosted workspace, consumed through `git`, receives its # repository's `[workspace.build]`, as its own checkout does. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap 'rm -rf "$TMP"' EXIT diff --git a/tests/e2e/774_an_index_member_inherits_its_archive_workspace.sh b/tests/e2e/774_an_index_member_inherits_its_archive_workspace.sh index 36a70f0c9..94da8f750 100755 --- a/tests/e2e/774_an_index_member_inherits_its_archive_workspace.sh +++ b/tests/e2e/774_an_index_member_inherits_its_archive_workspace.sh @@ -13,6 +13,9 @@ # The archive is seeded into the install path, so the test needs no network: # the resolver accepts an installed tree whose layout matches the descriptor. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json T=$(mktemp -d) trap 'rm -rf "$T"' EXIT diff --git a/tests/e2e/776_a_path_with_no_utf8_spelling_is_named.sh b/tests/e2e/776_a_path_with_no_utf8_spelling_is_named.sh index 26737c3b8..ce095fdd4 100755 --- a/tests/e2e/776_a_path_with_no_utf8_spelling_is_named.sh +++ b/tests/e2e/776_a_path_with_no_utf8_spelling_is_named.sh @@ -17,6 +17,9 @@ # into it. # `elf` stands for Linux here: macOS refuses such a name at creation. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap "rm -rf $TMP" EXIT diff --git a/tests/e2e/777_c_standard_applies_to_the_package_that_declares_it.sh b/tests/e2e/777_c_standard_applies_to_the_package_that_declares_it.sh index 7285d1398..4f2512c67 100755 --- a/tests/e2e/777_c_standard_applies_to_the_package_that_declares_it.sh +++ b/tests/e2e/777_c_standard_applies_to_the_package_that_declares_it.sh @@ -13,6 +13,9 @@ # cdep declares gnu11 201112L, and no __STRICT_ANSI__ (a GNU dialect) # plain declares none 201112L with __STRICT_ANSI__: the default, c11 set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap "rm -rf $TMP" EXIT diff --git a/tests/e2e/77_cdb_preserves_test_entries.sh b/tests/e2e/77_cdb_preserves_test_entries.sh index 2d762cdf2..debfb0795 100755 --- a/tests/e2e/77_cdb_preserves_test_entries.sh +++ b/tests/e2e/77_cdb_preserves_test_entries.sh @@ -13,6 +13,9 @@ # No `requires:` capability → runs on all three CI platforms, mirroring the # other `mcpp test` e2e tests (15/16/17). set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap "rm -rf $TMP" EXIT diff --git a/tests/e2e/781_deleted_root_cdb_is_restored.sh b/tests/e2e/781_deleted_root_cdb_is_restored.sh index a653d2b25..ce74bda35 100755 --- a/tests/e2e/781_deleted_root_cdb_is_restored.sh +++ b/tests/e2e/781_deleted_root_cdb_is_restored.sh @@ -8,6 +8,9 @@ # reached a compile-database writer, so a deleted database stayed deleted # forever (#397 C-1, #677 B1, report §3.6). set -euo pipefail +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap 'rm -rf "$TMP"' EXIT diff --git a/tests/e2e/782_foreign_root_cdb_entries_are_replaced.sh b/tests/e2e/782_foreign_root_cdb_entries_are_replaced.sh index afd562e12..72e616500 100755 --- a/tests/e2e/782_foreign_root_cdb_entries_are_replaced.sh +++ b/tests/e2e/782_foreign_root_cdb_entries_are_replaced.sh @@ -7,6 +7,9 @@ # When the replaced file held entries mcpp did not write, one warning states # how many. set -euo pipefail +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap 'rm -rf "$TMP"' EXIT diff --git a/tests/e2e/783_cdb_switches_whole_with_the_configuration.sh b/tests/e2e/783_cdb_switches_whole_with_the_configuration.sh index 5d03b8d8d..6b040d30b 100755 --- a/tests/e2e/783_cdb_switches_whole_with_the_configuration.sh +++ b/tests/e2e/783_cdb_switches_whole_with_the_configuration.sh @@ -10,6 +10,9 @@ source "$(dirname "${BASH_SOURCE[0]}")/_toolchain_env.sh" # (item 2: replaced, never merged across configurations); switching back # restores that configuration's entries, its test units included. set -euo pipefail +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap 'rm -rf "$TMP"' EXIT diff --git a/tests/e2e/784_cdb_replays_from_its_directory.sh b/tests/e2e/784_cdb_replays_from_its_directory.sh index 9684f843e..c5014d1ae 100755 --- a/tests/e2e/784_cdb_replays_from_its_directory.sh +++ b/tests/e2e/784_cdb_replays_from_its_directory.sh @@ -10,6 +10,9 @@ source "$(dirname "${BASH_SOURCE[0]}")/_toolchain_env.sh" # ROOT when replayed from there, and two of three GCC entries failed outright # (report §3.2). Must FAIL for gcc on 2026.9.26.1. set -euo pipefail +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap 'rm -rf "$TMP"' EXIT diff --git a/tests/e2e/785_cdb_interface_flag_module_extensions.sh b/tests/e2e/785_cdb_interface_flag_module_extensions.sh index 4e2c9a693..53503f421 100755 --- a/tests/e2e/785_cdb_interface_flag_module_extensions.sh +++ b/tests/e2e/785_cdb_interface_flag_module_extensions.sh @@ -10,6 +10,9 @@ source "$(dirname "${BASH_SOURCE[0]}")/_toolchain_env.sh" # `.ixx` on its own and hands it to the linker with no `-x`, so this is a real # failure there, not a cosmetic one. Must FAIL on 2026.9.26.1. set -euo pipefail +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap 'rm -rf "$TMP"' EXIT diff --git a/tests/e2e/786_std_unit_in_the_database_and_build_id.sh b/tests/e2e/786_std_unit_in_the_database_and_build_id.sh index cb5dae59f..198c33645 100755 --- a/tests/e2e/786_std_unit_in_the_database_and_build_id.sh +++ b/tests/e2e/786_std_unit_in_the_database_and_build_id.sh @@ -12,6 +12,9 @@ source "$(dirname "${BASH_SOURCE[0]}")/_toolchain_env.sh" # string), and `ide.toolchains..build-id` is present and stable across two # runs (S1 §6, S1-11.2-3). set -euo pipefail +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap 'rm -rf "$TMP"' EXIT diff --git a/tests/e2e/805_rooted_workspace_reaches_its_own_path_dependency.sh b/tests/e2e/805_rooted_workspace_reaches_its_own_path_dependency.sh index d994349e6..3fd767edf 100755 --- a/tests/e2e/805_rooted_workspace_reaches_its_own_path_dependency.sh +++ b/tests/e2e/805_rooted_workspace_reaches_its_own_path_dependency.sh @@ -24,6 +24,9 @@ # resolution fails loudly (a network fetch of an https://example.invalid URL) # rather than silently passing, which is what let 2026.9.26.1 look correct. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap 'rm -rf "$TMP"' EXIT diff --git a/tests/e2e/806_dash_p_resolves_the_package_first.sh b/tests/e2e/806_dash_p_resolves_the_package_first.sh index 5bb07d117..9b5bff833 100755 --- a/tests/e2e/806_dash_p_resolves_the_package_first.sh +++ b/tests/e2e/806_dash_p_resolves_the_package_first.sh @@ -7,6 +7,9 @@ # basename or its `[workspace] members` path -- the option's own name, # `--package`, promised a package that neither ever read. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap 'rm -rf "$TMP"' EXIT diff --git a/tests/e2e/808_device_source_is_not_a_compile_unit.sh b/tests/e2e/808_device_source_is_not_a_compile_unit.sh index 5f4435fe9..9f2511778 100755 --- a/tests/e2e/808_device_source_is_not_a_compile_unit.sh +++ b/tests/e2e/808_device_source_is_not_a_compile_unit.sh @@ -19,6 +19,9 @@ # E. The build still succeeds and the kernel still reaches the program: the # fix must not stop the device source from being compiled BY THE RULE. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json SRC="$(cd "$(dirname "$0")/../.." && pwd)/examples/12-a-new-device-language" [[ -d "$SRC" ]] || { echo "FAIL: $SRC is missing"; exit 1; } diff --git a/tests/e2e/80_feature_defines.sh b/tests/e2e/80_feature_defines.sh index 231999d0e..b78964e42 100755 --- a/tests/e2e/80_feature_defines.sh +++ b/tests/e2e/80_feature_defines.sh @@ -7,6 +7,9 @@ # # No `requires:` capability → runs on all three CI platforms. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap "rm -rf $TMP" EXIT diff --git a/tests/e2e/813_conditional_dialect_cxxflags_root_only.sh b/tests/e2e/813_conditional_dialect_cxxflags_root_only.sh index 660e58838..ea9c2dc22 100755 --- a/tests/e2e/813_conditional_dialect_cxxflags_root_only.sh +++ b/tests/e2e/813_conditional_dialect_cxxflags_root_only.sh @@ -21,6 +21,9 @@ # excluded from a package's own per-package fingerprint contribution, # design §6.2 finding 7). set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap "rm -rf $TMP" EXIT diff --git a/tests/e2e/815_the_database_describes_what_a_rule_generates.sh b/tests/e2e/815_the_database_describes_what_a_rule_generates.sh index 1881f0a6b..aa92826db 100755 --- a/tests/e2e/815_the_database_describes_what_a_rule_generates.sh +++ b/tests/e2e/815_the_database_describes_what_a_rule_generates.sh @@ -16,6 +16,9 @@ # following `mcpp build` writes the header at exactly that path; # C. compile_commands.json carries no such field. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap 'rm -rf "$TMP"' EXIT diff --git a/tests/e2e/826_plugin_diagnostics_features_and_names.sh b/tests/e2e/826_plugin_diagnostics_features_and_names.sh index 482b22238..87e663925 100755 --- a/tests/e2e/826_plugin_diagnostics_features_and_names.sh +++ b/tests/e2e/826_plugin_diagnostics_features_and_names.sh @@ -63,7 +63,7 @@ printf 'import mcpp.core;\nimport mcpp.acme.gen;\nint main() { acme::tell(); ret "$MCPP" build > d1.log 2>&1 || fail "D1: the build failed" d1.log grep -q "the generator found no schema" d1.log || fail "D1: the message is missing" d1.log grep -q "impact: no bindings are generated" d1.log || fail "D1: the impact line is missing" d1.log -grep -q "hint: add schema/\*.proto" d1.log || fail "D1: the hint line is missing" d1.log +grep -q "help: add schema/\*.proto" d1.log || fail "D1: the help line is missing" d1.log grep -q "mcpp.acme.gen'.*mcpp\.\|claims an origin" d1.log && fail "R1: an own-namespace module was warned about" d1.log # D2. `touch` first, as e2e 139 does: an unmodified build takes the project diff --git a/tests/e2e/83_feature_defines_propagate.sh b/tests/e2e/83_feature_defines_propagate.sh index 660b1ddf0..dc5d0da54 100755 --- a/tests/e2e/83_feature_defines_propagate.sh +++ b/tests/e2e/83_feature_defines_propagate.sh @@ -10,6 +10,9 @@ # # No `requires:` capability → runs on all three CI platforms. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap "rm -rf $TMP" EXIT diff --git a/tests/e2e/840_a_workspace_database_describes_each_package_once.sh b/tests/e2e/840_a_workspace_database_describes_each_package_once.sh index 549328ba0..8d7225471 100644 --- a/tests/e2e/840_a_workspace_database_describes_each_package_once.sh +++ b/tests/e2e/840_a_workspace_database_describes_each_package_once.sh @@ -21,6 +21,9 @@ # with its own `mcpp.toml` as the diagnostic's path, and the others are # described (R5.2). set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap "rm -rf $TMP" EXIT diff --git a/tests/e2e/857_a_host_module_several_members_import_is_compiled_once.sh b/tests/e2e/857_a_host_module_several_members_import_is_compiled_once.sh index 6ebeb1c37..07afe0171 100755 --- a/tests/e2e/857_a_host_module_several_members_import_is_compiled_once.sh +++ b/tests/e2e/857_a_host_module_several_members_import_is_compiled_once.sh @@ -25,6 +25,9 @@ # D. With `--cache global`, nothing of the path dependency is written below # the global cache root; the engine interface is. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json source "$(dirname "$0")/_host_path.sh" TMP=$(mktemp -d) diff --git a/tests/e2e/87_build_default_profile.sh b/tests/e2e/87_build_default_profile.sh index d12441c4b..ac1436da3 100755 --- a/tests/e2e/87_build_default_profile.sh +++ b/tests/e2e/87_build_default_profile.sh @@ -6,6 +6,9 @@ # Precedence: --profile/--release/--dev flag > [build].default-profile > "dev". # See .agents/docs/2026-06-29-manifest-environment-and-platform-design.md. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap "rm -rf $TMP" EXIT diff --git a/tests/e2e/881_pe_auto_exports_accept_llvm_bitcode.sh b/tests/e2e/881_pe_auto_exports_accept_llvm_bitcode.sh index 67d25f7e6..cd52fe8e8 100644 --- a/tests/e2e/881_pe_auto_exports_accept_llvm_bitcode.sh +++ b/tests/e2e/881_pe_auto_exports_accept_llvm_bitcode.sh @@ -4,6 +4,9 @@ source "$(dirname "${BASH_SOURCE[0]}")/_toolchain_env.sh" # 881 -- mcpp#762: PE auto-export accepts LLVM LTO objects while preserving an # explicitly annotated surface. Opting out removes the scanner from the graph. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json source "$(dirname "$0")/_host_path.sh" TMP=$(mktemp -d) trap 'rm -rf "$TMP"' EXIT diff --git a/tests/e2e/913_the_build_database_describes_the_build_programs.sh b/tests/e2e/913_the_build_database_describes_the_build_programs.sh index df6e2ea1c..26fab9b33 100755 --- a/tests/e2e/913_the_build_database_describes_the_build_programs.sh +++ b/tests/e2e/913_the_build_database_describes_the_build_programs.sh @@ -14,6 +14,9 @@ # D `mcpp self env` states `engineModules`, the directory of the units of # `mcpp:core`, and it holds the interface. set -e +# The project-root copy of the compile database is opt-in since +# 2026.10.26.1 ([build] compile_commands); this test reads it. +export MCPP_COMPILE_COMMANDS=compile_commands.json TMP=$(mktemp -d) trap 'rm -rf "$TMP"' EXIT fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } diff --git a/tests/e2e/933_one_cxx_runtime_per_process.sh b/tests/e2e/933_one_cxx_runtime_per_process.sh new file mode 100644 index 000000000..1bd634209 --- /dev/null +++ b/tests/e2e/933_one_cxx_runtime_per_process.sh @@ -0,0 +1,134 @@ +#!/usr/bin/env bash +# 933 -- one C++ runtime per process, and the islands that may differ +# (design 2026-10-26 §6.3, SPEC-002 §6; mcpp#838). +# +# An llvm build (libc++) with libstdc++ named on its link line: +# A a static library and a test that catches `std::exception` thrown by it: +# the line puts libstdc++ AFTER the build's own runtime, so the typed catch +# works (the previous release split the ABI layer and only `catch (...)` +# caught it), and the undeclared island is reported; +# B the same with a C++ shared library target: refused when planned, +# [cxx-runtime-island]; +# C a library built by g++ (on libstdc++) whose exception type the llvm +# program catches by type: refused after the link, [cxx-runtime-collision]; +# D the island declared, `requires = [{ layer = "mcpp:c++-abi=libstdc++", +# visibility = "private" }]` under `[target.'cfg(os = "linux")']`: no +# report; +# E `visibility = "private"` on another layer: a manifest error naming +# requires-private-layer. +set -e +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } +source "$(dirname "$0")/_toolchain_env.sh" +[[ "$(uname -s)" == Linux ]] || { echo "SKIP: 933 is about ELF"; exit 0; } +[[ -x "$LLVM_ROOT/bin/clang++" ]] || { echo "SKIP: no llvm payload"; exit 0; } +GLIB="$MCPP_E2E_GCC_ROOT/lib64" +[[ -e "$GLIB/libstdc++.so.6" && -x "$MCPP_E2E_GCC_ROOT/bin/g++" ]] || { echo "SKIP: no gcc payload"; exit 0; } + +project() { # $1 dir, $2 target kind, $3 extra manifest text + mkdir -p "$1/src" "$1/tests" + cat > "$1/mcpp.toml" < "$1/src/xrt.cppm" + cat > "$1/tests/catch.cpp" <<'EOF' +import std; +import xrt; +int main() { + auto ep = std::make_exception_ptr(std::logic_error("x")); // a libc++abi-only entry point + try { xrt::checked(-1); } + catch (const std::exception& e) { std::println("caught as std::exception: {}", e.what()); return 0; } + catch (...) { std::println("std::exception missed -> (...)"); return 1; } +} +EOF +} + +# A +project "$TMP/a" lib "" +( cd "$TMP/a" && "$MCPP" test > out.log 2>&1 ) || fail "A: the typed catch failed" "$TMP/a/out.log" +grep -q "caught as std::exception" "$TMP/a/out.log" || fail "A: no typed catch" "$TMP/a/out.log" +grep -q "no package declares it" "$TMP/a/out.log" || fail "A: the undeclared island is not reported" "$TMP/a/out.log" + +# B +project "$TMP/b" shared "" +set +e +( cd "$TMP/b" && "$MCPP" build > out.log 2>&1 ); rc=$? +set -e +[[ $rc -ne 0 ]] || fail "B: a C++ shared library beside the island was accepted" "$TMP/b/out.log" +grep -q "cxx-runtime-island" "$TMP/b/out.log" || fail "B: not refused as an island" "$TMP/b/out.log" + +# C -- libfoo is built by mcpp with the gcc payload, so it is a C++ library +# on libstdc++ whose exception type crosses its interface. +mkdir -p "$TMP/c/foo/src" "$TMP/c/app/src" +cat > "$TMP/c/foo/mcpp.toml" < "$TMP/c/foo/src/foo.cpp" <<'EOF' +#include +struct FooError : std::runtime_error { FooError() : std::runtime_error("foo") {} }; +void foo_throws() { throw FooError(); } +EOF +( cd "$TMP/c/foo" && "$MCPP" build > out.log 2>&1 ) || fail "C: libfoo did not build with gcc" "$TMP/c/foo/out.log" +FOO_DIR=$(dirname "$(find "$TMP/c/foo/target" -name 'libfoo.so' | head -1)") +[[ -e "$FOO_DIR/libfoo.so" ]] || fail "C: no libfoo.so" "$TMP/c/foo/out.log" +cat > "$TMP/c/app/mcpp.toml" < "$TMP/c/app/src/main.cpp" <<'EOF' +#include +struct FooError : std::runtime_error { FooError(); }; +void foo_throws(); +int main() { try { foo_throws(); } catch (const FooError&) { return 0; } return 1; } +EOF +set +e +( cd "$TMP/c/app" && "$MCPP" build > out.log 2>&1 ); rc=$? +set -e +[[ $rc -ne 0 ]] || fail "C: a typed catch across runtimes was accepted" "$TMP/c/app/out.log" +grep -q "cxx-runtime-collision" "$TMP/c/app/out.log" || fail "C: not refused as a collision" "$TMP/c/app/out.log" + +# D +project "$TMP/d" lib "[target.'cfg(os = \"linux\")'] +requires = [{ layer = \"mcpp:c++-abi=libstdc++\", visibility = \"private\" }]" +( cd "$TMP/d" && "$MCPP" test > out.log 2>&1 ) || fail "D: the declared island failed" "$TMP/d/out.log" +! grep -q "no package declares it" "$TMP/d/out.log" || fail "D: a declared island is reported as undeclared" "$TMP/d/out.log" + +# E +mkdir -p "$TMP/e/src" +printf '[package]\nname = "e"\nversion = "0.1.0"\nrequires = [{ layer = "mcpp:c-abi=glibc", visibility = "private" }]\n' > "$TMP/e/mcpp.toml" +printf 'int main() { return 0; }\n' > "$TMP/e/src/main.cpp" +set +e +( cd "$TMP/e" && "$MCPP" build > out.log 2>&1 ); rc=$? +set -e +[[ $rc -ne 0 ]] && grep -q "requires-private-layer" "$TMP/e/out.log" \ + || fail "E: private on another layer was accepted" "$TMP/e/out.log" + +echo "PASS: 933 one C++ runtime per process, and verified islands" diff --git a/tests/e2e/934_llvm_with_a_graph_supplied_libstdcxx.sh b/tests/e2e/934_llvm_with_a_graph_supplied_libstdcxx.sh new file mode 100644 index 000000000..e6d17e277 --- /dev/null +++ b/tests/e2e/934_llvm_with_a_graph_supplied_libstdcxx.sh @@ -0,0 +1,108 @@ +#!/usr/bin/env bash +# 934 -- an llvm build whose C++ layer is libstdc++, supplied by a package of +# the graph (design 2026-10-26 §6.2, D10; the shape of `gcc.libstdcxx`). +# +# The package links the gcc payload's headers and runtime directory into its +# own tree and declares `provides = ["mcpp:c++-abi=libstdc++"]`, its std module +# (`bits/std.cc`) and its library directory. Criteria: +# A `import std` builds and runs; the typed catch works; +# B a C++ shared library of the build throws, the program catches +# `const std::exception&` across the .so (one runtime: U, nothing to +# verify as an island); the image needs no libc++; +# C a package that requires `mcpp:c++-abi=libstdc++` is satisfied by it; +# D without the provider the same requirement is refused when planned. +set -e +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } +source "$(dirname "$0")/_toolchain_env.sh" +[[ "$(uname -s)" == Linux ]] || { echo "SKIP: 934 uses the Linux gcc payload"; exit 0; } +[[ -x "$LLVM_ROOT/bin/clang++" ]] || { echo "SKIP: no llvm payload"; exit 0; } +G="$MCPP_E2E_GCC_ROOT" +[[ -e "$G/include/c++/$GCC_VERSION/bits/std.cc" && -e "$G/lib64/libstdc++.so.6" ]] \ + || { echo "SKIP: no gcc payload with bits/std.cc"; exit 0; } + +# The provider, as `gcc.libstdcxx`'s install() lays it out. +P="$TMP/libstdcxx" +mkdir -p "$P/lib" +ln -s "$G/include/c++/$GCC_VERSION" "$P/cxx" +for f in libstdc++.so libstdc++.so.6 libstdc++.a libgcc_s.so libgcc_s.so.1; do + [[ -e "$G/lib64/$f" ]] && ln -s "$G/lib64/$f" "$P/lib/$f" +done +cat > "$P/mcpp.toml" <<'EOF' +[package] +namespace = "gcc" +name = "libstdcxx" +version = "16.1.0.1" +provides = ["mcpp:c++-abi=libstdc++"] +requires = ["mcpp:compiler=llvm", "mcpp:c-abi=glibc"] +[build] +sources = [] +include_dirs = ["cxx", "cxx/x86_64-linux-gnu", "cxx/backward"] +std-module = "cxx/bits/std.cc" +[runtime] +library_dirs = ["lib"] +EOF + +# A requirer, as `compat.sycl-runtime` is. +R="$TMP/needs" +mkdir -p "$R/src" +cat > "$R/mcpp.toml" <<'EOF' +[package] +name = "needs" +version = "0.1.0" +requires = ["mcpp:c++-abi=libstdc++"] +[build] +sources = [] +EOF + +mkdir -p "$TMP/app/src" "$TMP/app/tests" +cat > "$TMP/app/mcpp.toml" < "$TMP/app/src/xrt.cppm" +cat > "$TMP/app/src/main.cpp" <<'EOF' +import std; +import xrt; +int main() { + std::vector v{3, 1, 2}; + std::ranges::sort(v); + try { xrt::checked(-1); } + catch (const std::exception& e) { std::println("{} caught {}", v, e.what()); return 0; } + catch (...) { std::println("missed"); return 1; } +} +EOF +( cd "$TMP/app" && "$MCPP" run app > out.log 2>&1 ) || fail "A/B/C: the build or the run failed" "$TMP/app/out.log" +grep -q "caught negative" "$TMP/app/out.log" || fail "B: no typed catch across the shared library" "$TMP/app/out.log" +grep -q "c++-abi.*libstdc++" "$TMP/app/out.log" || true +APP=$(find "$TMP/app/target" -path '*/bin/app' -type f | head -1) +readelf -d "$APP" | grep -q 'libc++' && fail "B: the image needs libc++" "$TMP/app/out.log" +SO=$(find "$TMP/app/target" -name 'libxrt.so*' -type f | head -1) +[[ -n "$SO" ]] || fail "B: no libxrt.so" "$TMP/app/out.log" +readelf -d "$SO" | grep -q 'libc++' && fail "B: the shared library needs libc++" "$TMP/app/out.log" + +# D: no provider, the requirement is refused before anything compiles. +sed -i '/^libstdcxx = /d' "$TMP/app/mcpp.toml" +set +e +( cd "$TMP/app" && "$MCPP" build > d.log 2>&1 ); rc=$? +set -e +[[ $rc -ne 0 ]] || fail "D: the requirement was not refused" "$TMP/app/d.log" +grep -q "requires the c++-abi to be .libstdc++." "$TMP/app/d.log" \ + || fail "D: the refusal does not name the requirement" "$TMP/app/d.log" + +echo "PASS: 934 llvm with a graph-supplied libstdc++" diff --git a/tests/e2e/935_two_builds_of_one_configuration.sh b/tests/e2e/935_two_builds_of_one_configuration.sh new file mode 100644 index 000000000..66bc7dab0 --- /dev/null +++ b/tests/e2e/935_two_builds_of_one_configuration.sh @@ -0,0 +1,61 @@ +#!/usr/bin/env bash +# 935 -- two builds of one configuration at once (mcpp#839, design 2026-10-26 +# §4): the second waits for the build directory and both succeed. +# +# Before the build directory had a lock, two cold builds started 0.1-0.5 s apart +# wrote one directory at once, and in three rounds of five both failed +# (`failed to write/read compiled module`). Criteria, over five rounds: +# A both builds exit 0 in every round; +# B in at least one round the later build reports that it waited, naming +# the holder; +# C the result record is complete after the last round (status ok) and +# `target/.last-result.json` names it; +# D two different configurations (dev and --release) do not wait on each +# other's lock (they share nothing to lock). +set -e +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +mkdir -p "$TMP/p/src" +cd "$TMP/p" +printf '[package]\nname = "lk"\nversion = "0.1.0"\nstandard = "c++23"\n' > mcpp.toml +for i in $(seq 1 24); do + printf 'export module m%d;\nimport std;\nexport int f%d(){ std::vector v(%d); return (int)v.size(); }\n' $i $i $i > "src/m$i.cppm" +done +{ echo 'import std;'; for i in $(seq 1 24); do echo "import m$i;"; done + echo 'int main(){ std::println("{}", f1()+f24()); }'; } > src/main.cpp + +"$MCPP" build > warm.log 2>&1 || fail "the first build failed" warm.log # std module and toolchain in place +waited=0 +for r in 1 2 3 4 5; do + rm -rf target + ( "$MCPP" build > a.log 2>&1; echo $? > a.rc ) & + ( sleep "0.$r"; "$MCPP" build > b.log 2>&1; echo $? > b.rc ) & + wait + [[ "$(cat a.rc)" == 0 && "$(cat b.rc)" == 0 ]] || fail "A: round $r: rc=$(cat a.rc),$(cat b.rc)" a.log b.log + grep -q "Waiting.*held by pid" a.log b.log && waited=1 +done +[[ $waited == 1 ]] || echo "note: no round overlapped long enough to wait (B not observed on this machine)" + +last=target/.last-result.json +[[ -f $last ]] || fail "C: no $last" +record=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["data"]["result"])' "$last") +python3 - "$record" <<'EOF' || fail "C: the result record is incomplete" "$record" +import json, sys +d = json.load(open(sys.argv[1])) +assert d["kind"] == "mcpp.result", d["kind"] +data = d["data"] +assert data["status"] == "ok", data["status"] +assert data["finishedAt"], data +assert any(a["kind"] == "bin" and a.get("launch") for a in data["artifacts"]), data["artifacts"] +EOF + +# D: dev and release are two directories, two locks. +( "$MCPP" build > dev.log 2>&1; echo $? > dev.rc ) & +( "$MCPP" build --release > rel.log 2>&1; echo $? > rel.rc ) & +wait +[[ "$(cat dev.rc)" == 0 && "$(cat rel.rc)" == 0 ]] || fail "D: rc=$(cat dev.rc),$(cat rel.rc)" dev.log rel.log +! grep -q "Waiting.*held by pid" dev.log rel.log || fail "D: two configurations waited on one lock" dev.log rel.log + +echo "PASS: 935 two builds of one configuration" diff --git a/tests/e2e/936_the_control_contract.sh b/tests/e2e/936_the_control_contract.sh new file mode 100644 index 000000000..22dfd0a57 --- /dev/null +++ b/tests/e2e/936_the_control_contract.sh @@ -0,0 +1,162 @@ +#!/usr/bin/env bash +# requires: python3 +# 936 -- the control contract (design 2026-10-26 §5): what a program, an +# agent or an editor reads of a build without a terminal. +# +# A `mcpp build --message-format json`: NDJSON `mcpp.event` lines on stdout +# (`plan`, `artifact`), then the `mcpp.build` envelope whose data is the +# result record; a failing build carries the compiler's diagnostics with +# path and line, in the events and in the record; +# B `mcpp run --print-launch`: the `mcpp.launch` envelope (argv, env, cwd), +# and the program is not started; +# C `--deny write-project` stops before the build writes, exit 5, naming the +# effect; an unknown effect is a usage error (exit 2); +# D CLICOLOR_FORCE colours a pipe, NO_COLOR wins over it; +# E `--home ` is MCPP_HOME for one run (`self env` reports it); +# F `search --format json` and `index list --format json` answer with their +# envelopes; `pack --list-formats` names tar and dir; +# G `--protocol-version` lists the new kinds; +# H a build ended by SIGTERM while it writes its directory exits 143 and +# leaves its record `cancelled` (and `.last-result.json` says so); the +# next build completes. +set -e +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } +PY=python3 + +cd "$TMP" +"$MCPP" new app > /dev/null +cd app + +# A +"$MCPP" build --message-format json > a.out 2> a.err || fail "A: the build failed" a.err +$PY - a.out <<'EOF' || fail "A: the stream is not as specified" a.out +import json, sys, os +same = lambda a, b: os.path.normcase(os.path.normpath(a)) == os.path.normcase(os.path.normpath(b)) +lines = [json.loads(l) for l in open(sys.argv[1]) if l.strip()] +events = [l for l in lines if l.get("kind") == "mcpp.event"] +assert any(e["event"] == "plan" for e in events), "no plan event" +assert any(e["event"] == "artifact" and e["artifact"]["kind"] == "bin" for e in events), "no artifact event" +env = lines[-1] +assert env["kind"] == "mcpp.build" and env["schemaVersion"] == 1, env +data = env["data"] +assert data["status"] == "ok", data["status"] +art = [a for a in data["artifacts"] if a["kind"] == "bin"][0] +assert same(art["launch"]["argv"][0], art["path"]), art +EOF +printf 'int main() { return undeclared_name; }\n' > src/main.cpp +set +e +"$MCPP" build --message-format json > a2.out 2> a2.err; rc=$? +set -e +[[ $rc -ne 0 ]] || fail "A: a failing build exited 0" a2.err +$PY - a2.out <<'EOF' || fail "A: the failure carries no located diagnostic" a2.out +import json, sys +lines = [json.loads(l) for l in open(sys.argv[1]) if l.strip()] +diags = [l["diagnostic"] for l in lines if l.get("event") == "diagnostic"] +assert diags, "no diagnostic event" +assert any(d["path"].endswith("main.cpp") and d["range"]["start"]["line"] == 1 for d in diags), diags +env = lines[-1] +assert env["data"]["status"] == "failed", env["data"]["status"] +assert env["data"]["diagnostics"], "the record has no diagnostics" +EOF +printf 'import std;\nint main() { std::println("started"); }\n' > src/main.cpp + +# B +"$MCPP" run --print-launch > b.out 2> b.err || fail "B: --print-launch failed" b.err +grep -q started b.out && fail "B: the program was started" b.out +$PY - b.out <<'EOF' || fail "B: not an mcpp.launch envelope" b.out +import json, sys, os +same = lambda a, b: os.path.normcase(os.path.normpath(a)) == os.path.normcase(os.path.normpath(b)) +d = json.load(open(sys.argv[1])) +assert d["kind"] == "mcpp.launch", d["kind"] +assert d["data"]["argv"] and same(d["data"]["argv"][0], d["data"]["program"]), d["data"] +assert "env" in d["data"] and "cwd" in d["data"] +EOF + +# C +rm -rf target +set +e +"$MCPP" --deny write-project build > c.out 2> c.err; rc=$? +set -e +[[ $rc -eq 5 ]] || fail "C: --deny write-project exited $rc, expected 5" c.err +grep -q "effect denied: write-project" c.err || fail "C: the effect is not named" c.err +[[ ! -d target ]] || [[ -z "$(find target -name build.ninja)" ]] || fail "C: the project was written" c.err +set +e +"$MCPP" --deny no-such-effect build > c2.out 2> c2.err; rc=$? +set -e +[[ $rc -eq 2 ]] || fail "C: an unknown effect exited $rc, expected 2" c2.err + +# D +CLICOLOR_FORCE=1 "$MCPP" build 2> d.err > /dev/null || fail "D: the build failed" d.err +grep -q $'\033\[' d.err || fail "D: CLICOLOR_FORCE did not colour a pipe" d.err +CLICOLOR_FORCE=1 NO_COLOR=1 "$MCPP" build 2> d2.err > /dev/null || fail "D: the build failed" d2.err +grep -q $'\033\[' d2.err && fail "D: NO_COLOR did not win" d2.err + +# E +OTHER="$TMP/other-home" +mkdir -p "$OTHER" +"$MCPP" --home "$OTHER" self env --format json > e.out 2> e.err || true +$PY - e.out <<'EOF' || fail "E: --home did not select the home for the run" e.out e.err +import json, sys, os +d = json.load(open(sys.argv[1]))["data"] +# By the directory's name: the shell's spelling of $OTHER is not the one a +# Windows program reports. +home = os.path.normpath(d["mcppHome"].replace("\\", "/")) +assert os.path.basename(home) == "other-home", d["mcppHome"] +EOF + +# F +"$MCPP" search --format json fmt > f1.out 2> f1.err || true +$PY -c 'import json,sys; d=json.load(open(sys.argv[1])); assert d["kind"]=="mcpp.search"; assert "packages" in d.get("data", {}) or d["diagnostics"]' f1.out \ + || fail "F: search --format json" f1.out f1.err +"$MCPP" index list --format json > f2.out 2> f2.err || fail "F: index list --format json failed" f2.err +$PY -c 'import json,sys; d=json.load(open(sys.argv[1])); assert d["kind"]=="mcpp.index"; assert isinstance(d["data"]["registries"], list)' f2.out \ + || fail "F: index list --format json" f2.out +"$MCPP" pack --list-formats > f3.out 2> f3.err || fail "F: pack --list-formats failed" f3.err +$PY -c 'import json,sys; d=json.load(open(sys.argv[1])); n=[f["name"] for f in d["data"]["formats"]]; assert d["kind"]=="mcpp.pack.formats" and "tar" in n and "dir" in n, n' f3.out \ + || fail "F: pack --list-formats" f3.out + +# G +"$MCPP" --protocol-version > g.out +for k in mcpp.build mcpp.launch mcpp.search mcpp.index mcpp.pack.formats; do + grep -q "\"$k\"" g.out || fail "G: --protocol-version does not list $k" g.out +done + +# H -- POSIX only: a shell's `kill` ends a native Windows process outright, +# which no handler sees (Windows reaches mcpp's handler by a console control +# event, SPEC-003 §3.7). +case "$(uname -s)" in + MINGW*|MSYS*|CYGWIN*) echo "PASS: 936 the control contract (H is POSIX)"; exit 0 ;; +esac +mkdir -p "$TMP/h/src" +cd "$TMP/h" +printf '[package]\nname = "h"\nversion = "0.1.0"\nstandard = "c++23"\n' > mcpp.toml +for i in $(seq 1 40); do + printf 'export module m%d;\nimport std;\nexport int f%d(){ std::vector v(%d); return (int)v.size(); }\n' $i $i $i > "src/m$i.cppm" +done +{ echo 'import std;'; for i in $(seq 1 40); do echo "import m$i;"; done + echo 'int main(){ std::println("{}", f1()+f40()); }'; } > src/main.cpp +"$MCPP" build > h0.log 2>&1 || fail "H: the first build failed" h0.log # the std module in place +rm -rf target +"$MCPP" build > h.log 2>&1 & +pid=$! +for _ in $(seq 1 500); do + grep -qs '"status": "running"' target/.last-result.json && break + sleep 0.02 +done +kill -TERM $pid +set +e +wait $pid; rc=$? +set -e +$PY - target/.last-result.json <<'EOF2' || fail "H: the interrupted build left no cancelled record (rc=$rc)" h.log target/.last-result.json +import json, sys +last = json.load(open(sys.argv[1]))["data"] +assert last["status"] == "cancelled", last +rec = json.load(open(last["result"]))["data"] +assert rec["status"] == "cancelled", rec["status"] +EOF2 +[[ $rc -eq 143 ]] || fail "H: SIGTERM exited $rc, expected 143" h.log +"$MCPP" build > h2.log 2>&1 || fail "H: the build after the interrupted one failed" h2.log + +echo "PASS: 936 the control contract" diff --git a/tests/e2e/937_the_root_compile_database_is_opt_in.sh b/tests/e2e/937_the_root_compile_database_is_opt_in.sh new file mode 100644 index 000000000..3741b91ee --- /dev/null +++ b/tests/e2e/937_the_root_compile_database_is_opt_in.sh @@ -0,0 +1,52 @@ +#!/usr/bin/env bash +# 937 -- the project-root compile database is published where the project asks +# for it, and nowhere by default (design 2026-10-26 D14). +# +# A a plain build writes the configuration's database in its build directory +# and none at the project root; +# B `[build] compile_commands = "compile_commands.json"` publishes it at the +# root, on the full path and on the fast path (a deleted copy comes back); +# C a root copy an earlier build left, in a project that no longer asks for +# one, is named once (a note), and not updated; +# D MCPP_COMPILE_COMMANDS overrides the manifest: "off" publishes nothing. +set -e +unset MCPP_COMPILE_COMMANDS +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +cd "$TMP" +"$MCPP" new app > /dev/null +cd app + +# A +"$MCPP" build > a.log 2>&1 || fail "A: the build failed" a.log +[[ ! -e compile_commands.json ]] || fail "A: a root database was written by default" +[[ -n "$(find target -name compile_commands.json)" ]] || fail "A: no configuration database" + +# B +printf '\n[build]\ncompile_commands = "compile_commands.json"\n' >> mcpp.toml +"$MCPP" build > b.log 2>&1 || fail "B: the build failed" b.log +[[ -f compile_commands.json ]] || fail "B: the key published nothing" b.log +rm compile_commands.json +"$MCPP" build > b2.log 2>&1 || fail "B: the fast build failed" b2.log +[[ -f compile_commands.json ]] || fail "B: the fast path did not publish the asked copy" b2.log + +# C +cp compile_commands.json keep.json +grep -v '^compile_commands = ' mcpp.toml > mcpp.toml.new && mv mcpp.toml.new mcpp.toml # BSD and GNU sed spell -i differently +"$MCPP" build > c.log 2>&1 || fail "C: the build failed" c.log +grep -q "is no longer updated" c.log || fail "C: the left copy was not named" c.log +touch src/main.cpp +"$MCPP" build > c2.log 2>&1 || fail "C: the second build failed" c2.log +grep -q "is no longer updated" c2.log && fail "C: named twice" c2.log +cmp -s compile_commands.json keep.json || fail "C: the left copy was updated" + +# D +rm -f compile_commands.json +printf '\n[build]\ncompile_commands = "compile_commands.json"\n' >> mcpp.toml +touch src/main.cpp +MCPP_COMPILE_COMMANDS=off "$MCPP" build > d.log 2>&1 || fail "D: the build failed" d.log +[[ ! -e compile_commands.json ]] || fail "D: MCPP_COMPILE_COMMANDS=off published a copy" d.log + +echo "PASS: 937 the root compile database is opt-in" diff --git a/tests/e2e/938_a_relink_never_exposes_a_partial_file.sh b/tests/e2e/938_a_relink_never_exposes_a_partial_file.sh new file mode 100644 index 000000000..5492ac68d --- /dev/null +++ b/tests/e2e/938_a_relink_never_exposes_a_partial_file.sh @@ -0,0 +1,42 @@ +#!/usr/bin/env bash +# 938 -- a linked artifact is published whole (design 2026-10-26 D22): a +# process that starts the program while it is relinked starts the old file or +# the new one, never half of either. +# +# GNU ld unlinks and writes in place, gold writes in place, and macOS lets a +# running program's file be overwritten: four launchers over 25 relinks saw +# thousands of EACCES / ETXTBSY / ENOEXEC (mcpp#840, E23). The link now writes +# `.tmp` and renames it over the name. +# A the link rule writes a temporary name and renames it (POSIX); +# B starting the program in a loop while it is relinked never fails to exec. +set -e +case "$(uname -s)" in Linux|Darwin) ;; *) echo "SKIP: POSIX publication"; exit 0 ;; esac +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +cd "$TMP" +"$MCPP" new app > /dev/null +cd app +printf 'int main() { return 0; }\n' > src/main.cpp +"$MCPP" build > a.log 2>&1 || fail "the build failed" a.log +ninja_file=$(find target -name build.ninja | head -1) +grep -A1 '^rule cxx_link' "$ninja_file" | grep -q '\$out.tmp.*&& mv -f \$out.tmp \$out' \ + || fail "A: the link rule does not publish through a temporary name" "$ninja_file" + +exe=$(find target -path '*/bin/app' -type f | head -1) +errors="$TMP/errors" +: > "$errors" +( end=$((SECONDS + 20)) + while [[ $SECONDS -lt $end && ! -e "$TMP/stop" ]]; do + "$exe" 2>>"$errors" >/dev/null || echo "exec failed: $?" >> "$errors" + done ) & +for i in $(seq 1 12); do + printf 'int main() { volatile int x = %d; return x - %d; }\n' "$i" "$i" > src/main.cpp + "$MCPP" build > b.log 2>&1 || { touch "$TMP/stop"; wait; fail "B: relink $i failed" b.log; } +done +touch "$TMP/stop" +wait +[[ ! -s "$errors" ]] || fail "B: a start met a partially written program" "$errors" + +echo "PASS: 938 a relink never exposes a partial file" diff --git a/tests/e2e/939_a_lock_written_meanwhile_is_not_overwritten.sh b/tests/e2e/939_a_lock_written_meanwhile_is_not_overwritten.sh new file mode 100644 index 000000000..c83cf9b18 --- /dev/null +++ b/tests/e2e/939_a_lock_written_meanwhile_is_not_overwritten.sh @@ -0,0 +1,100 @@ +#!/usr/bin/env bash +# requires: gcc python3 fresh-sandbox +# 939 -- mcpp.lock is written optimistically (design 2026-10-26 §4.1): no lock +# is held while a build resolves its dependencies, so the write at the end +# compares what is there with what the run read when it started. +# +# A the first build records the index dependency in mcpp.lock; +# B a lock that changes while a build resolves (here: the project's own +# build program rewrites it, between the read and the write) is not +# overwritten; the build stops, naming [lockfile-concurrent] and the +# remedy; +# C run again, the build resolves against the lock as it is now and +# succeeds. +# +# The dependency is served by a project-local index and pre-extracted into the +# private xlings data directory (as 844), so the build touches no network. +set -e + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +export MCPP_HOME="$TMP/mcpp-home" +source "$(dirname "$0")/_inherit_toolchain.sh" + +mkdir -p "$TMP/p" && cd "$TMP/p" + +mkdir -p local-index/pkgs/c +cat > local-index/pkgs/c/cmdline.lua <<'EOF' +package = { + spec = "1", + namespace = "mcpplibs", + name = "cmdline", + description = "fixture for the optimistic lock write", + licenses = {"MIT"}, + type = "package", + xpm = { linux = { ["1.0.0"] = { + url = "https://example.invalid/cmdline-1.0.0.tar.gz", + sha256 = "0000000000000000000000000000000000000000000000000000000000000000", + } } }, + mcpp = { + language = "c++23", + import_std = false, + sources = { "src/cmdline.cppm" }, + targets = { ["cmdline"] = { kind = "lib" } }, + deps = {}, + }, +} +EOF +XP=".mcpp/.xlings/data/xpkgs/mcpplibs-x-cmdline/1.0.0" +mkdir -p "$XP/src" +cat > "$XP/src/cmdline.cppm" <<'EOF' +export module fixture.cmdline; +export int cmdline_value() { return 7; } +EOF +printf 'ok\n' > "$XP/.mcpp_ok" + +mkdir -p src +cat > mcpp.toml <<'EOF' +[package] +name = "p" +version = "0.1.0" + +[indices] +default = { path = "local-index" } + +[dependencies.mcpplibs] +cmdline = "1.0.0" +EOF +cat > src/main.cpp <<'EOF' +import fixture.cmdline; +int main() { return cmdline_value() == 7 ? 0 : 1; } +EOF + +# A +"$MCPP" build > a.log 2>&1 || fail "A: the first build failed" a.log +grep -q 'cmdline' mcpp.lock || fail "A: mcpp.lock does not record the dependency" mcpp.lock + +# B -- the build program runs after the lock is read and before it is written. +cat > build.mcpp <<'EOF' +#include +int main() { + std::ofstream f("mcpp.lock", std::ios::app); + f << "# written by another run\n"; + return f ? 0 : 1; +} +EOF +set +e +"$MCPP" build > b.log 2>&1; rc=$? +set -e +[[ $rc -ne 0 ]] || fail "B: a lock changed meanwhile was overwritten (the build succeeded)" b.log mcpp.lock +grep -q 'lockfile-concurrent' b.log || fail "B: the refusal does not name lockfile-concurrent" b.log +grep -q 'run the command again' b.log || fail "B: the refusal names no remedy" b.log +grep -q '# written by another run' mcpp.lock || fail "B: the other run's lock was overwritten" mcpp.lock + +# C +rm build.mcpp +"$MCPP" build > c.log 2>&1 || fail "C: the build after the concurrent write failed" c.log + +echo "PASS: 939 a lock written meanwhile is not overwritten" diff --git a/tests/unit/test_build_stage.cpp b/tests/unit/test_build_stage.cpp index c253bbcf1..1eb9229f4 100644 --- a/tests/unit/test_build_stage.cpp +++ b/tests/unit/test_build_stage.cpp @@ -185,7 +185,7 @@ TEST(BuildStage, ReadOnlyDestinationOutcomeIsPlatformDefined) { auto r = stage_file(src, dst, no_retry()); #if defined(_WIN32) ASSERT_FALSE(r.has_value()); - EXPECT_NE(r.error().message.find("hint:"), std::string::npos); + EXPECT_NE(r.error().message.find("help:"), std::string::npos); EXPECT_EQ(read_file(dst), "old"); #else ASSERT_TRUE(r.has_value()) << r.error().message; @@ -207,7 +207,7 @@ TEST(BuildStage, UnwritableDestinationFailsWithActionableHint) { auto r = stage_file(src, dst, no_retry()); ASSERT_FALSE(r.has_value()); - EXPECT_NE(r.error().message.find("hint:"), std::string::npos); + EXPECT_NE(r.error().message.find("help:"), std::string::npos); EXPECT_NE(r.error().message.find("clangd"), std::string::npos); EXPECT_NE(r.error().message.find(dst.string()), std::string::npos); } diff --git a/tests/unit/test_diag.cpp b/tests/unit/test_diag.cpp index ba1c1122f..19664b71f 100644 --- a/tests/unit/test_diag.cpp +++ b/tests/unit/test_diag.cpp @@ -55,7 +55,7 @@ TEST_F(DiagTest, FormatKeepsWhatOnTheFirstLine) { ASSERT_EQ(rs.size(), 1u); EXPECT_TRUE(rs[0].format().starts_with("what happened\n")); EXPECT_NE(rs[0].format().find("impact: the impact"), std::string::npos); - EXPECT_NE(rs[0].format().find("hint: the hint"), std::string::npos); + EXPECT_NE(rs[0].format().find("help: the hint"), std::string::npos); } TEST_F(DiagTest, StrictPromotesDegradationsOnly) { diff --git a/tests/unit/test_ninja_backend.cpp b/tests/unit/test_ninja_backend.cpp index 08381514e..a836c69ee 100644 --- a/tests/unit/test_ninja_backend.cpp +++ b/tests/unit/test_ninja_backend.cpp @@ -1254,7 +1254,7 @@ TEST(NinjaBackend, FilterKeepsStagingDiagnosticsAndFailedTarget) { "FAILED: [code=1] pcm.cache/std.pcm \n" "/opt/mcpp/bin/mcpp stage --output pcm.cache/std.pcm /cache/std.pcm\n" "error: cannot stage file into the build directory\n" - "hint: another process has this file memory-mapped, loaded or open.\n" + "help: another process has this file memory-mapped, loaded or open.\n" "ninja: build stopped: subcommand failed.\n"; auto filtered = filter_ninja_output(raw, prefixes); @@ -1266,7 +1266,7 @@ TEST(NinjaBackend, FilterKeepsStagingDiagnosticsAndFailedTarget) { EXPECT_EQ(filtered.find("[code=1]"), std::string::npos) << filtered; // The diagnostic survives; the echoed command line does not. EXPECT_NE(filtered.find("error: cannot stage file"), std::string::npos) << filtered; - EXPECT_NE(filtered.find("hint:"), std::string::npos) << filtered; + EXPECT_NE(filtered.find("help:"), std::string::npos) << filtered; EXPECT_EQ(filtered.find("stage --output"), std::string::npos) << filtered; EXPECT_EQ(filtered.find("ninja: Entering directory"), std::string::npos) << filtered; }