xtask: repair 7 stale wasm-fixture tests and gate cargo test -p xtask#836
Open
brandonpayton wants to merge 2 commits into
Open
Conversation
Seven xtask unit tests have failed since PR #605 (`3430c5bbc`) added program output/cache artifact validation (and, for one, since ABI_VERSION moved past 4). They are stale *test* defects, not product regressions: the fixtures emit artifacts the (correct) validation rejects. - Class A (5 cmd_resolve_* tests): build scripts `touch`ed an empty `.wasm`, rejected as "is not a wasm binary". They now emit a valid minimal module via new `emit_wasm_build_script` + `minimal_executable_wasm` helpers; the kernel output emits the full `HOST_ADAPTER_REQUIRED_KERNEL_EXPORTS` set from the shared const so it self-updates. - Class B (binaries_dir_program_fetch_does_not_require_built_deps): the fetched archive fixture was a wasm header only (no exports), so cache validation forced a source-build fallback (baddep `exit 42` -> panic). Now uses valid wasm, so the remote-first path it means to prove is actually exercised. - Class C (cli_produces_archive_with_canonical_filename): assertion hardcoded `abi4`; the canonical filename correctly encodes the real ABI. Now version-relative via `shared::ABI_VERSION`, doubling as a guard that the filename encodes ABI_VERSION. Production validation is unchanged. A new negative test (`wasm_artifact_policy_rejects_empty_and_exportless_when_exports_required`) locks in that empty/non-wasm -> "is not a wasm binary" and export-less wasm -> "missing required exports" stay rejected, so the class cannot be "fixed" by weakening validation. xtask's ~298 unit tests were entirely un-gated in CI, which is how these rotted. Adds a `cargo-xtask` suite to `scripts/ci-run-test-suite.sh` and wires it into the prepare-merge, staging-build, and force-rebuild suite matrices (not kernel-gated; ~16s, no toolchain/workspace). Documents the suite in `docs/agent-guidance/validation.md` and `docs/repository-organization.md`. cargo test -p xtask: 290 passed / 7 failed -> 298 passed / 0 failed. Implements kd-xc19 triage design (docs/plans/2026-07-02-xtask-stale-wasm-fixture-test-failures-triage.md). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Durable evidence for the stale-fixture repair: before 290 passed / 7 failed, after 298 passed / 0 failed on the convoy base (ABI 15), and 315 / 0 rebased on origin/main (ABI 16). Includes per-run passed/failed/skipped outcome lists (the skipped lists are empty by construction — the xtask suite declares no #[ignore] tests) and the serial repro logs. Mirrors the kd-xc19 triage artifact layout. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
cargo test -p xtaskhas been red with 7 failing unit tests since PR #605(
3430c5bbc, "Fix browser demos and artifact validation") added programoutput/cache artifact validation — and, for one test, since
ABI_VERSIONmovedpast 4. These are stale test defects, not product regressions: the fixtures
emit artifacts the (correct) validation rejects. They went unnoticed because
xtask's ~298 unit tests were entirely un-gated in CI —
ci-run-test-suite.shand
prepare-merge.ymlbuild and run xtask but never run its own tests.This blocks the validation-gates work (internal kd-u7f): a pre-merge gate that
runs xtask tests can't be enabled while these 7 are red.
Implements the triage design in
docs/plans/2026-07-02-xtask-stale-wasm-fixture-test-failures-triage.md(internal kd-xc19 → kd-872c).
What
Repair the 7 stale tests (test-only; production validation unchanged):
cmd_resolve_*tests): build scriptstouched an empty.wasm, rejected as "is not a wasm binary". They now emit a valid minimalmodule via new
emit_wasm_build_script+minimal_executable_wasmtesthelpers. The kernel output emits the full
HOST_ADAPTER_REQUIRED_KERNEL_EXPORTSset built from the shared const, so it self-updates if that set changes.
binaries_dir_program_fetch_does_not_require_built_deps): thefetched-archive fixture was a wasm header only (no exports), so cache
validation forced a source-build fallback (a dep that
exit 42s → panic). Nowuses valid wasm, so the remote-first path the test means to prove is actually
exercised.
cli_produces_archive_with_canonical_filename): assertionhardcoded
abi4while the canonical filename correctly encodes the real ABI.Now version-relative via
shared::ABI_VERSION, doubling as a guard that thefilename encodes
ABI_VERSION. (Its sibling..._uses_build_toml_revisionstays abi-pinned on purpose — it passes
--abi "4"to test the revisionfield.)
Lock the class shut: a new negative test
wasm_artifact_policy_rejects_empty_and_exportless_when_exports_requiredassertsempty/non-wasm → "is not a wasm binary" and export-less wasm → "missing
required exports" stay rejected, so the tests can't be "fixed" later by
weakening validation.
Close the gate gap: add a
cargo-xtasksuite toscripts/ci-run-test-suite.shand wire it into theprepare-merge,staging-build, andforce-rebuildsuite matrices (not kernel-gated; ~16s, notoolchain/workspace). Documented in
docs/agent-guidance/validation.mdanddocs/repository-organization.md.Validation
cargo test -p xtask --target aarch64-apple-darwin: 290 passed / 7 failed →298 passed / 0 failed (the 7 repaired + 1 new negative test). Confirmed green
at both the convoy base (ABI 15) and rebased on
main(ABI 16). Verified the CIpath end-to-end via
ci-run-test-suite.sh cargo-xtask, andcheck-abi-version.shreports the ABI snapshot unchanged. Not run (change touches none of these
surfaces): vitest, libc, POSIX, browser, kernel suites.
Follow-up (design open question): consider broadening the gate to
cargo test --workspaceafter auditing other un-gated workspace crates(e.g.
tools/mkrootfs).🤖 Generated with Claude Code