[2/5] test: harden the build-output assertions - #1110
Merged
Conversation
The deferral check counted preload=none globally, so a stray string elsewhere could mask an eager video; the .jpg scan covered only the homepage of 106 built pages; an empty poster folder passed; and the frozen byte baseline would falsely fail when a poster is legitimately added. Assertions are now per-video-tag, all-pages, presence-checked, and per-poster (110KB cap), with fileURLToPath for path safety.
ralph-cmyk
changed the title
AvdLee
marked this pull request as ready for review
July 27, 2026 12:34
AvdLee
added a commit
to ralph-cmyk/RocketSimApp
that referenced
this pull request
Jul 27, 2026
Rebased onto master (now includes the AvdLee#1109 retitle) and onto the hardened test harness from AvdLee#1110. The FAQ block and its test are unchanged in substance from the original branch.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merge order 2 of 5 — hardens the test harness introduced in #1105; merge right after it. Shows #1105's commits until that merges — the diff on top is tests/build-output.test.js only. I will rebase after each merge.
Problem. The build-output tests added in #1105 have four blind spots: the
video-deferral check counts
preload="none"anywhere on the page rather thanon each video tag, the
.jpg-reference scan covers only the homepage, anempty poster folder passes the WebP check, and the payload guard is a frozen
byte total that falsely fails the moment a poster is legitimately added.
Why it matters. These tests are the CI guard that keeps the poster
optimization from silently regressing. A guard that can be masked by an
unrelated string, or that only watches 1 of 106 built pages, protects less
than it appears to.
What changed.
tests/build-output.test.jsonly. Assertions are nowper-video-tag, scan every built HTML page, require at least 13 WebP posters
to exist, and cap each poster at 110 KB instead of freezing a directory
total.
readDistusesfileURLToPathso checkouts with spaces in the pathwork.
KPI — starting point. Measured by mutation testing on the built site
(25 Jul 2026): removing
preload="none"from one video while the literalstring appears in an inline script passes the old assertion; the
.jpgscancovers 1 of 106 built pages; an empty
posters/directory passes the WebPcheck.
KPI — expectation. The same mutations now fail the suite (verified before
opening this PR). No behavior change to the site itself —
dist/output isbyte-identical.
How to measure it.
npm testafternpm run build. To reproduce themutation check: strip a
preload="none"from one<video>tag indist/index.htmland re-run — the suite must go red.Risk / rollback. Test-only change; the site does not change. If an
assertion proves too strict, revert this single commit — #1105's original
tests remain the fallback.