Skip to content

feat: products/ output layout + PDF/UA-1 validation gate#15

Merged
agiacalone merged 11 commits into
mainfrom
ada-products-and-pdfua
Jun 23, 2026
Merged

feat: products/ output layout + PDF/UA-1 validation gate#15
agiacalone merged 11 commits into
mainfrom
ada-products-and-pdfua

Conversation

@agiacalone

Copy link
Copy Markdown
Owner

Summary

Two changes, both test-first. +23 tests; full suite 191 green (npm test).

1. Output hygiene — products/ layout + intermediate cleanup ✅ complete

Generated artifacts and pdflatex's scratch files were landing in the topic root beside the kept _lecture_main.md source, drifting the classes-layout-doctrine (hit intro_to_compsec + 999/ac_measurement_principles).

  • Default --out is now a products/ subdir beside the source (lib/out-dir.js); explicit --out still honored verbatim.
  • compileLatex sweeps .aux/.log/.out/.nav/.snm/.toc after each successful compile, keeping only .tex + .pdf (lib/latex-clean.js; never touches the .tex/.pdf or other decks' files).
  • products/ added to .gitignore.
  • Verified end-to-end on the example lecture: artifacts in products/, topic root clean, no stray intermediates.

2. ADA Title II / WCAG audit chain — advances #7

PDF/UA-1 validation gate (Phase 3). The pre-generation stages check the source; this checks the compiled PDFs.

  • lib/a11y/pdfua.js: veraPDF PDF/UA-1 deep check when present, else a pdfinfo Tagged: smoke-check fallback. A tagged PDF passes the smoke-check but its report row says "smoke-check only — install veraPDF" — a fallback pass is never mistaken for a full PDF/UA pass (no silent pass).
  • generate.js appends a pdf-ua stage to a11y-report.json and gates the build (exit 1) on any untagged artifact.
  • Pure interpreters + orchestration unit-tested (lib/a11y/pdfua.test.js, 8 tests). 5/5 example PDFs pass on TL2026.

Table header /TH tagging (Phase 2 polish — table semantics). Found and pikepdf-verified the working mechanism on TL2026 (\thead{} was a red herring):

  • \DocumentMetadata{…testphase={phase-III,table}} + group-scoped \tagpdfsetup{table/header-rows={1}} → header row tags as /TH, data rows /TD, and the group scope keeps non-header layout tables (e.g. the Cornell cue/notes table) at /TD with no leak to later tables.
  • Applied to texComparisonTable + cornellComparisonTable, unit-tested.
  • Caveat (intentionally surfaced): those comparison-table emitters aren't yet called by the live markdown-monolith generators, so the recipe is correct + ready but no shipping artifact emits /TH yet — wiring them in is a follow-up.

Scope / what this does NOT do

Advances #7 — does not close it. Remaining on the issue:

  • Phase 4 — CI enforcement over examples/ artifacts.
  • Phase 5 — changelog/AGENTS.md doctrine propagation.
  • veraPDF install (userspace Java) so the deep check actually runs — the gate is wired and auto-detects it on PATH.
  • Comparison-table wiring so the /TH mechanism fires on real artifacts; then heading-hierarchy + figure-actualtext, driven against veraPDF.

Docs

  • CHANGELOG.md — entries under [Unreleased] (Fixed: output hygiene; Added: PDF/UA gate, table /TH).
  • docs/plans/wcag-audit-pass-plan.md — Phase 3 marked done, and the verified /TH recipe recorded (incl. the two auto-mechanisms that don't work, so they aren't retried).

Test plan

  • npm test → 191 passing.
  • npm run check → clean.
  • node generate.js --main examples/file_systems_abstraction_lecture_main.md → artifacts in examples/products/, ✓ pdf-ua: 5 PDF(s) tagged (pdfinfo smoke-check), build gated.

🤖 Generated with Claude Code

agiacalone and others added 2 commits June 23, 2026 11:06
Two changes, both test-first (+23 tests; suite 191 green):

1. Output hygiene — generated artifacts and pdflatex intermediates no
   longer litter the topic root. Default --out is now a products/ subdir
   beside the source (lib/out-dir.js); compileLatex sweeps
   .aux/.log/.out/.nav/.snm/.toc after each successful compile
   (lib/latex-clean.js). products/ gitignored.

2. ADA Title II / WCAG audit chain (advances #7) — a post-generation
   PDF/UA-1 validation gate (lib/a11y/pdfua.js): veraPDF deep check when
   present, an honest pdfinfo Tagged: smoke-check fallback otherwise
   (labeled as such — no silent pass). generate.js appends a pdf-ua stage
   to a11y-report.json and gates the build (exit 1) on any untagged PDF.
   Plus the verified table-header /TH tagging mechanism
   (testphase={phase-III,table} + group-scoped
   \tagpdfsetup{table/header-rows={1}}) applied to both comparison-table
   emitters — correct and unit-tested, latent until the emitters are wired
   into the live markdown-monolith generators.

Advances #7 (Phase 3 validation stage + table-semantics polish). Does not
close it: Phase 4 (CI), Phase 5 (docs), veraPDF install, and the
comparison-table wiring remain.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S98DYe1qKVSuEeh5S2U6nZ
…visory

Installed veraPDF 1.30.2 and ran the deep check for real — the example
artifacts are all tagged but FAIL full PDF/UA-1 (4–6 rules each: missing
XMP PDF/UA identifier + dc:title, DisplayDocTitle not set, PDF header EOL,
untagged header/footer content). Full PDF/UA-1 compliance is genuine
remediation work, so the gate must not break every lecture build the moment
veraPDF is on PATH.

Corrected the gate's blocking model:
- BLOCKING invariant = tagged-presence (pdfinfo Tagged:): an untagged PDF
  has no accessibility tree → exit 1 (regression guard, unchanged intent).
- veraPDF PDF/UA-1 verdict is now ADVISORY (row.ua1): logged loudly and
  written to a11y-report.json, but never flips stage.ok / the exit code.
  A future --strict-a11y flag can opt into blocking.

generate.js now exits 0 with the advisory surfaced; the failing rules are
documented in docs/plans/wcag-audit-pass-plan.md as the remediation backlog.
Suite 193 green (+2 advisory tests).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S98DYe1qKVSuEeh5S2U6nZ
@agiacalone

Copy link
Copy Markdown
Owner Author

veraPDF installed — deep check now runs, and it surfaced the real state

Installed veraPDF 1.30.2 (userspace, ~/verapdf/~/bin/verapdf) and pointed the gate at it. Honest result: every example artifact is tagged but fails full PDF/UA-1 (4–6 rules each).

Failing rule classes (the remediation backlog, now documented in the plan doc):

  1. PDF/UA identifier missing in XMP (clause 5)
  2. dc:title missing + DisplayDocTitle not true (clause 7.1)
  3. PDF header EOL nit (clause 6.1)
  4. Untagged real contentfancyhdr header/footer rules need /Artifact marking (clause 7.1 t9)

Gate behavior corrected (commit 3522ddc)

That finding exposed a bug in my first commit: the gate blocked on any non-pass, so installing veraPDF broke node generate.js (exit 1) for every lecture. Fixed:

  • Blocking invariant = tagged-presence — an untagged PDF fails the build (exit 1). Unchanged intent.
  • veraPDF PDF/UA-1 verdict = advisory (row.ua1) — logged loudly + written to a11y-report.json, never flips the exit code. Full PDF/UA-1 compliance is remediation work that shouldn't block lecture generation. A future --strict-a11y flag can opt into blocking.

node generate.js now exits 0 with the advisory surfaced; suite 193 green. The PDF/UA-1 remediation is tracked as a follow-up (still advances #7, doesn't close it).

…t 1 rule away

Driven red->green against veraPDF 1.30.2. Both preambles now:
- \DocumentMetadata{...pdfstandard=ua-1,pdfversion=1.7,...} — writes the
  PDF/UA-1 identifier into XMP and forces the PDF-1.7 header PDF/UA-1
  requires (default \DocumentMetadata output is PDF 2.0).
- \hypersetup{pdftitle={<headerLeft>},pdfdisplaydoctitle=true} — satisfies
  dc:title + the DisplayDocTitle ViewerPreference.

Result on the example artifacts: 4-6 failed PDF/UA-1 rules each -> quiz and
quiz key FULLY compliant (0 failures); lecture-notes + Cornell handout + key
down to a single rule (clause 7.1 t3 — colortbl \cellcolor/\columncolor and
an mdframed fill emit untagged decorative content needing /Artifact marking;
known colortbl+tagging limitation, tracked as the final step).

Suite 196 green (+3 preamble tests). Advances #7.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S98DYe1qKVSuEeh5S2U6nZ
@agiacalone

Copy link
Copy Markdown
Owner Author

PDF/UA-1 metadata remediation — driven against veraPDF (commit 9eb86a1)

Found the recipe and applied it to both preambles, red→green against veraPDF 1.30.2:

  • \DocumentMetadata{…pdfstandard=ua-1,pdfversion=1.7,…} → PDF/UA-1 XMP identifier + forces the PDF-1.7 header (default was 2.0)
  • \hypersetup{pdftitle={<title>},pdfdisplaydoctitle=true}dc:title + DisplayDocTitle

Result: 4–6 failed rules each → quiz + quiz-key fully PDF/UA-1 compliant (0 failures); lecture-notes + Cornell handout + key down to 1 rule (clause 7.1 t3 — colortbl \cellcolor/\columncolor + an mdframed fill draw untagged decorative backgrounds; known colortbl+tagging limitation, the final remediation step). Suite 196 green.

agiacalone and others added 6 commits June 23, 2026 12:00
Investigated clause 7.1 t3 (untagged decorative content) against veraPDF:
isolated to table border rules (hline/|/booktabs all fail; only borderless
passes) and mdframed background fills (lecture-notes navy strips). phase-III
does not auto-artifact either, and a naive \tagmcbegin{artifact} made it worse
(1 -> 2 rules) — so it is toolchain-gated, not a quick generator fix. Two paths
(design-side vs wait-for-toolchain) documented.

Adds docs/plans/2026-06-23-a11y-roadmap.md (current prioritized next steps:
P1 wire comparison-table emitters, P2 last rule, P3 --strict-a11y, P4 CI,
P5 doctrine). Advances #7.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S98DYe1qKVSuEeh5S2U6nZ
…king

New pure evaluatePdfUaGate(stage,{strict}) decides the gate: untagged PDFs
always block; veraPDF PDF/UA-1 non-compliance blocks ONLY under --strict-a11y
(advisory by default). Wired into generate.js + usage; intended for CI.
201 tests green. Advances #7 (roadmap P3).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S98DYe1qKVSuEeh5S2U6nZ
The /TH header-cell tagging shipped earlier was latent — the comparison-table
emitters had no callers in the markdown-monolith path. Wired end to end:

- parser captures GFM pipe tables (parsed.tables + tablesForSection()),
  associating each with its enclosing Roman section.
- lecture-notes + cornell-handout generators render them via texComparisonTable
  / cornellComparisonTable (header row tagged /TH).
- example lecture gains a real Hard-link vs Symlink comparison table under III.

Verified: pikepdf StructElem scan shows TH=3 header cells in both
lecture_notes.pdf and cornell_handout.pdf (was 0). Suite 206 green.
Advances #7.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S98DYe1qKVSuEeh5S2U6nZ
…oadmap P4)

.github/workflows/a11y.yml installs a tagging-capable TeX Live (2025) +
veraPDF (headless), regenerates the example, runs the DEFAULT gate as blocking
(tagged-presence regression guard) and --strict-a11y as INFORMATIONAL
(continue-on-error, so the toolchain-gated clause-7.1-t3 rule does not permared
CI), and uploads a11y-report.json. Conservative triggers (PR-path-filtered +
workflow_dispatch) to respect the org Actions budget.

Needs one live validation run (TeX Live package list + veraPDF install).
Roadmap statuses updated (P1/P3 done, P4 written). Advances #7.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S98DYe1qKVSuEeh5S2U6nZ
Adds docs/ACCESSIBILITY.md — the authoritative, version-controlled
compliance document for auditors: regulatory scope (ADA Title II, DOJ 2024
rule, WCAG 2.1 AA -> PDF/UA-1), the two-tier audit-chain architecture, the
blocking model, the tagging mechanism, evidence (a11y-report.json + veraPDF),
an honest per-artifact status table, and the maintenance process. README +
CHANGELOG link it. Advances #7.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S98DYe1qKVSuEeh5S2U6nZ
Public-safe summary of the Keep-a-Changelog house standard + the
regression-citation rule. Propagated identically to lectern + oracle.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S98DYe1qKVSuEeh5S2U6nZ
@agiacalone

Copy link
Copy Markdown
Owner Author

Roadmap P1–P5 cleared + audit documentation

Pushed the full accessibility roadmap on top of the metadata remediation:

  • P1 — GFM comparison tables → tagged /TH (86a1ab1): parser captures pipe tables (tables/tablesForSection), lecture-notes + Cornell generators render them through the /TH emitters, example gains a Hard-link vs Symlink table under §III. pikepdf confirms TH=3 — the /TH mechanism is no longer latent.
  • P3 — --strict-a11y (3fa3ab8): untagged always blocks; PDF/UA-1 non-compliance blocks only under strict (advisory by default). Pure evaluatePdfUaGate, unit-tested.
  • P4 — CI (bb45bad): .github/workflows/a11y.yml installs TeX Live 2025 + veraPDF, default gate blocking + --strict-a11y informational (so the toolchain-gated rule doesn't permared CI). Needs one live validation run.
  • P5 — changelog philosophy (4b30d00 here; lectern docs: changelog philosophy + technical-audience sweep lectern#18, oracle agiacalone/oracle#31).
  • Audit doc — docs/ACCESSIBILITY.md: the authoritative ADA Title II / PDF/UA-1 compliance record (scope, architecture, blocking model, evidence, honest status, maintenance).

206 tests green; build exits 0. P2 (last PDF/UA-1 rule) remains toolchain-gated and documented; the advisory gate + docs/ACCESSIBILITY.md §5 carry the honest status. This branch is ready for review/merge.

agiacalone and others added 2 commits June 23, 2026 12:55
- ACCESSIBILITY.md §2.1: the pipeline IS a compiler — source->AST(parser)->
  codegen(generators)->backend(pdflatex); tagging = type annotations; veraPDF =
  the type-checker; --strict-a11y = -Werror. Three ways the analogy frays
  (prose spec, preconditions-not-adequacy, post-hoc verifier). §5 reframed: the
  residual clause-7.1-t3 rule is a codegen-gap in the LaTeX tagging backend,
  experimentally confirmed (full de-styling left it unchanged at ~100 untagged
  items) — not fixable from the source.
- CLAUDE.md + AGENTS.md: documented the products/ default output + the two-tier
  accessibility gate for the agent/technical audience; pointer to ACCESSIBILITY.md.

Advances #7.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S98DYe1qKVSuEeh5S2U6nZ
PR #15 run failed at job setup with "Repository access blocked" — the org
Actions policy does not allowlist the third-party setup-texlive-action. The
workflow also still needs one live validation run. Switch to workflow_dispatch
only so it stops red-flagging PRs; re-enable the pull_request trigger once the
action is allowlisted and the TeX Live + veraPDF install is proven on a live run.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S98DYe1qKVSuEeh5S2U6nZ
@agiacalone agiacalone merged commit b3a8347 into main Jun 23, 2026
1 check passed
@agiacalone agiacalone deleted the ada-products-and-pdfua branch June 23, 2026 20:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant