Skip to content

ci(workflows): add PR validation merge gate and fix docusaurus workflow#2213

Merged
WilliamBerryiii merged 13 commits into
mainfrom
fix/docusaurus-workflow
Jun 29, 2026
Merged

ci(workflows): add PR validation merge gate and fix docusaurus workflow#2213
WilliamBerryiii merged 13 commits into
mainfrom
fix/docusaurus-workflow

Conversation

@jkim323

@jkim323 jkim323 commented Jun 28, 2026

Copy link
Copy Markdown
Collaborator

Pull Request

Description

Adds a required merge gate for PR validation and fixes the Docusaurus workflow along with several broken documentation links.

  • PR validation gate: Adds a pr-validation-success aggregator job to pr-validation.yml (if: always(), depends on every non-gate job, fails when any required job fails, is cancelled, or is skipped) to provide a single, stable status context for branch-protection rules.
  • Gate-completeness validator: Adds scripts/security/Test-PrValidationGate.ps1 (+ Pester tests and fixtures) that fails CI when a new job is added to pr-validation.yml without being wired into the aggregator''s needs:, preventing silent gaps in the gate. Wired into package.json as lint:pr-gate (included in lint:all) and documented in workflows.instructions.md and the workflows README.md.
  • Docusaurus workflow: Fixes docusaurus-tests.yml configuration.
  • Broken links + docs accuracy: Repairs broken links surfaced by markdown link checking and corrects stale references in docs/agents/project-planning/ pages.
  • Gate Bypass: pr-validation-success previously passed even when a needed job failed. The aggregation jq now selects result != "success" and != "skipped" and runs under set -euo pipefail, so any real failure blocks the gate while intentional skips (e.g. python jobs gated on has-projects) still pass.
  • "Phantom" stale guard: Filter null/empty needs: elements before computing stale entries. A stray YAML null (~) or empty string ("") is not a real job ID, so it would surface as a false "stale" violation and fail the check. Now elements are filtered, not just the container, so the validator only flags genuine drift.
  • Tests: Added empty-needs.yml fixture (needs: [lint, test, ~, "", build]) and a new context asserting null/empty entries are filtered with no stale/missing.

Related Issue(s)

fix #2214

Type of Change

Select all that apply:

Code & Documentation:

  • Bug fix (non-breaking change fixing an issue)
  • New feature (non-breaking change adding functionality)
  • Breaking change (fix or feature causing existing functionality to change)
  • Documentation update

Infrastructure & Configuration:

  • GitHub Actions workflow
  • Linting configuration (markdown, PowerShell, etc.)
  • Security configuration
  • DevContainer configuration
  • Dependency update

Other:

  • Script/automation (.ps1, .sh, .py)
  • Other (please describe):

Testing

  • npm run lint:pr-gate exits 0.
  • pr-validation.yml and docusaurus-tests.yml parse as valid YAML.
  • Test-PrValidationGate.Tests.ps1 passes (16 tests) against the complete / missing-job / stale-needs fixtures.

Checklist

Required Checks

  • Documentation is updated (if applicable)
  • Files follow existing naming conventions
  • Changes are backwards compatible (if applicable)
  • Tests added for new functionality (if applicable)

Required Automated Checks

The following validation commands must pass before merging:

  • Markdown linting: npm run lint:md
  • Spell checking: npm run spell-check
  • Frontmatter validation: npm run lint:frontmatter
  • Skill structure validation: npm run validate:skills
  • Link validation: npm run lint:md-links
  • PowerShell analysis: npm run lint:ps
  • Plugin freshness: npm run plugin:generate
  • Docusaurus tests: npm run docs:test

Security Considerations

  • This PR does not contain any sensitive or NDA information
  • Any new dependencies have been reviewed for security issues
  • Security-related scripts follow the principle of least privilege

Additional Notes

  • A maintainer must add the status context PR Validation / PR Validation Success to the branch-protection ruleset for the gate to become enforcing.
  • markdown-link-check is intentionally left as soft-fail (grace period) and can be flipped to blocking after ignorePatterns are tuned.

@jkim323 jkim323 requested a review from a team as a code owner June 28, 2026 07:15
@jkim323 jkim323 changed the title ci(workflows): add PR validation merge gate and fix docusaurus workflow [DRAFT] ci(workflows): add PR validation merge gate and fix docusaurus workflow Jun 28, 2026
@github-actions

github-actions Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

OpenSSF Scorecard

PackageVersionScoreDetails
pip/regex 2026.6.28 UnknownUnknown
pip/regex 2026.5.9 UnknownUnknown

Scanned Files

  • scripts/evals/moderation/requirements.txt
  • scripts/evals/moderation/uv.lock

@codecov-commenter

codecov-commenter commented Jun 28, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.88889% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.31%. Comparing base (88dc7f2) to head (461192f).

Files with missing lines Patch % Lines
scripts/security/Test-PrValidationGate.ps1 88.88% 8 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2213      +/-   ##
==========================================
+ Coverage   81.29%   81.31%   +0.02%     
==========================================
  Files         129      130       +1     
  Lines       19068    19140      +72     
  Branches       12       12              
==========================================
+ Hits        15501    15564      +63     
- Misses       3564     3573       +9     
  Partials        3        3              
Flag Coverage Δ
docusaurus 61.84% <ø> (ø)
pester 86.02% <88.88%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
scripts/security/Test-PrValidationGate.ps1 88.88% <88.88%> (ø)

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

Eval Execution

Status: Passed

  • Artifacts evaluated: 0
  • Specs run: 0
  • Assertions passed: 0
  • Assertions failed (blocking): 0
  • Assertions failed (advisory): 0
  • Failed specs (merge-blocking): 0

No changed AI artifacts required evaluation.

@jkim323 jkim323 changed the title [DRAFT] ci(workflows): add PR validation merge gate and fix docusaurus workflow [DRAFT!] ci(workflows): add PR validation merge gate and fix docusaurus workflow Jun 28, 2026
@github-actions github-actions Bot mentioned this pull request Jun 28, 2026
@jkim323 jkim323 changed the title [DRAFT!] ci(workflows): add PR validation merge gate and fix docusaurus workflow ci(workflows): add PR validation merge gate and fix docusaurus workflow Jun 28, 2026
@jkim323

jkim323 commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator Author

🧭 Reviewer's guide

This PR bundles three logically separate changes under one branch.

Changes Look at these files What to verify
The merge gate .github/workflows/pr-validation.yml (+70) The pr-validation-success job lists every other job in needs: and the jq filter rejects anything that isn't success or skipped
The gate validator scripts/security/Test-PrValidationGate.ps1 (+273) + 4 fixtures + Pester tests (16 tests) YAML parsing is structural (not regex), and null/empty needs: entries are filtered before the stale check
Docusaurus + docs docusaurus-tests.yml, docs/agents/project-planning/*.md Path filter widens from docs/docusaurusdocs (every doc-only PR now pays the a11y/build cost — intentional). Broken README links become README.md
Eval coverage skip scripts/evals/Modules/ArtifactDetection.psm1, Invoke-VallyEvals.ps1, Test-StimulusPresence.ps1 New Test-RepoRootArtifact helper exempts .github/<kind>/... from eval coverage enforcement. Note: this is orthogonal to the rest of the PR — flag if it should be split out.

Two things worth a second look

  1. Skip-tolerant policy — the jq selector is result != "success" and != "skipped", so any skipped need passes the gate. This is intentional (matrix children gated on has-projects), but it means: if a future contributor adds an if: to a job without thinking it through, the gate will silently let that job not-run.
  2. Gate enforcement is a follow-up — after merge, a maintainer must add the status context PR Validation / PR Validation Success to the branch-protection ruleset. Until then, the gate runs but is advisory only.

Suggested review order

  1. Read the PR description— the "Gate Bypass" and "Phantom stale guard" bullets explain the two non-obvious design choices
  2. Skim pr-validation.yml lines 369–438 — the entire merge gate fits in one screen
  3. Skim Test-PrValidationGate.ps1Get-PrValidationGateResult is the only function that does real work
  4. Spot-check fixtures + tests — the empty-needs.yml fixture is the regression case for the phantom-stale bug

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: add required PR validation merge gate and fix docusaurus workflow

4 participants