Skip to content

feat(cli): stackwright test:a11y — portable WCAG 2.1 AA accessibility audit (#270)#456

Merged
perasperaactual merged 6 commits into
devfrom
feat/a45-test-a11y
May 20, 2026
Merged

feat(cli): stackwright test:a11y — portable WCAG 2.1 AA accessibility audit (#270)#456
perasperaactual merged 6 commits into
devfrom
feat/a45-test-a11y

Conversation

@perasperaactual
Copy link
Copy Markdown
Contributor

Summary

Closes #270 / stackwright-a45.

Implements the stackwright test:a11y CLI command in 5 phases as specified in the issue:

Phase 1 — Page Discovery Utility

  • packages/cli/src/utils/a11y-page-discovery.tsdiscoverPageSlugs() wraps existing listPages(); zero duplication

Phase 2 — Portable Axe Runner

  • packages/cli/src/utils/a11y-runner.ts — axe-core + Playwright, cookie-based dark-mode switching, failOn threshold, graceful optional-peer-dep errors
  • @axe-core/playwright added as optional peer dependency to @stackwright/cli

Phase 3 — CLI Command

  • packages/cli/src/commands/a11y.tsstackwright test:a11y [slug] with --pages, --no-dark-mode, --tags, --fail-on, --json, --base-url
  • Wired into cli.ts and exported from index.ts

Phase 4 — launch-stackwright Integration

  • packages/launch-stackwright/README.md updated with new CLI section and test:a11y entry

Phase 5 — MCP/Otter Integration

  • packages/mcp/src/tools/a11y.tsstackwright_test_a11y tool with structured violation output and actionable error messages
  • Registered in packages/mcp/src/server.ts

Testing

  • 31 new unit tests across a11y-runner.test.ts and a11y.test.ts
  • All 211 CLI tests pass
  • Follows existing preview.ts pattern for optional Playwright peer dep
  • Mirrors dark-mode logic from packages/e2e/tests/a11y/wcag-compliance.spec.ts

Usage

# With dev server running (pnpm dev):
stackwright test:a11y                    # audit all pages, light + dark
stackwright test:a11y /about             # audit one page
stackwright test:a11y --no-dark-mode     # light mode only
stackwright test:a11y --fail-on critical # only fail on critical violations
stackwright test:a11y --json             # machine-readable output

Requirements

  • playwright (optional peer dep, already listed)
  • @axe-core/playwright (new optional peer dep added)
  • Running dev server at http://localhost:3000 (or --base-url)

@github-actions
Copy link
Copy Markdown
Contributor

❌ Visual Regression Test Results

Status: ❌ Visual regression tests failed

Visual differences were detected! Please review:

  1. Download the artifacts from this workflow run
  2. Check the visual-regression-diffs artifact for comparison images
  3. If changes are intentional, update snapshots locally:
    pnpm test:e2e --update-snapshots
    git add packages/e2e/tests/__screenshots__
    git commit -m "Update visual regression baselines"

📸 Baseline screenshots are committed to the repo for reproducible testing.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 20, 2026

♿ Accessibility Test Results

Overall Status: ✅ 0/0 tests passed

🦮 WCAG 2.1 AA Compliance

No WCAG test results available

⌨️ Keyboard Navigation

No keyboard navigation test results available


⚠️ No accessibility tests were executed. Check the workflow logs for setup issues.

📊 Detailed Report

Download the full HTML accessibility report from the workflow artifacts for:

  • Detailed WCAG violation descriptions
  • Specific element selectors and fixes
  • Color contrast issues
  • Keyboard navigation flow analysis

🔍 Testing Checklist

Our accessibility tests verify:

  • ✅ WCAG 2.1 Level AA compliance
  • ✅ Color contrast in light and dark modes
  • ✅ Tab key navigation through all interactive elements
  • ✅ Focus indicators are visible
  • ✅ No keyboard traps
  • ✅ Skip links and ARIA landmarks
  • ✅ Screen reader compatibility

Powered by @axe-core/playwright and Playwright

@github-actions
Copy link
Copy Markdown
Contributor

✅ Visual Regression Test Results

Status: ✅ All visual tests passed!

All screenshots match the baseline. No visual regressions detected! 🎉

@github-actions
Copy link
Copy Markdown
Contributor

⚡ Performance Benchmark Results

✅ Build Time Benchmarks: PASSED

✅ Bundle Size Benchmarks: PASSED

❌ Runtime Vitals Benchmarks: FAILED

📝 Note: Detailed results are available in the job logs.

🎯 Performance Budgets:

  • Build time: <70s total
  • First-load JS: <100KB gzipped
  • FCP: <1.5s, LCP: <2.5s, TTI: <3s

Updated: 2026-05-20T18:09:20.620Z

@perasperaactual perasperaactual merged commit 8dbbe0b into dev May 20, 2026
14 of 15 checks passed
@perasperaactual perasperaactual deleted the feat/a45-test-a11y branch May 20, 2026 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: stackwright test:a11y — portable accessibility audit for any Stackwright site

1 participant