Skip to content

wcag: the automated half of a WCAG-EM evaluation, for the W3C report tool - #68

Merged
ralyodio merged 1 commit into
masterfrom
feat/wcag
Sep 13, 2026
Merged

ralyodio merged 1 commit into
masterfrom
feat/wcag

Conversation

@ralyodio

Copy link
Copy Markdown
Contributor

The bot@ forward asked for https://www.w3.org/WAI/eval/report-tool/ as a command in cli-tools. That tool is a Svelte web app with no CLI and no headless mode, so this adds wcag, which does the automated half of a WCAG-EM evaluation and hands the file to the tool for the human half.

wcag audit https://example.org --pages 8 --level AA   # sample, headless Chrome + axe-core, one row per criterion, exit 1 on failure
wcag report wcag-report.json                          # the tool's own evaluation.json, for "Open evaluation"
wcag open                                             # the hand-off, step by step
  • Sample (step 3): home page, then one page per sitemap section before a second of any; falls back to the start page's links; --url adds pages by hand.
  • Audit (step 4): every page in headless Chrome over CDP (Node's WebSocket, no Puppeteer/Playwright), axe-core 4.13 filtered to the WCAG tags for the target. Nothing is ever marked passed: a violation is earl:failed, anything else earl:cantTell with the checks listed.
  • Chrome: CHROME_PATH, then PATH, then the Puppeteer and Playwright caches; the staged ~/.local/share/chrome-deps libraries are wired in so a bare Puppeteer build runs on this box.
  • Evaluation file: mirrors the tool's export from its source and was verified by replaying its open() with the jsonld version it pins — 33/33 assertions restore for 2.2, 30/30 for 2.1. The replay caught that defining a WCAG22: prefix in the context (which looks correct) makes the tool drop every 2.2 assertion, so the prefix is deliberately left undefined, as the tool does.

Live run on profullstack.com, 3 pages: 1.4.3 Contrast (Minimum) fails on 2 of 3 pages, color-contrast on 36 elements; everything else axe checked needs a person.

Tests: 43 new, suite 923/924 (the one failure is torrent.test.ts against a real client, flaky on master). pnpm typecheck has no errors in the new files; argontv.ts and free-names.ts already fail it on master.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Khk1C6Ese6xjdHAWLVstca

…tool

The forward asked for the W3C's WCAG-EM Report Tool as a command. It is a
Svelte page with no CLI and no headless mode, and cannot be one: it is a
form for the five steps of the evaluation methodology, and most success
criteria need a person to decide them. What a machine can do is the part
that is machine-shaped, and `wcag` does that: choose the structured sample
(step 3), load each page in headless Chrome, run axe-core over it, print
one row per success criterion, and write the tool's own evaluation file so
"Open evaluation" starts the evaluator at the judgement calls (step 4)
rather than at an empty form.

The sample is the home page and then one page per section of the sitemap
before a second of any, because WCAG-EM wants pages that differ and the
first N a crawler lists are N posts from one archive. A site with no
sitemap gives its own links; `--url` names a page whatever the sample says.

Nothing is ever marked passed. axe proves a failure but "no rule fired"
proves nothing about a criterion, since its rules cover a part of each one,
so a criterion with only passing checks lands as "cannot tell" with the
checks listed. The exit status is 1 on any failure within the target.

The browser is driven over the DevTools protocol with Node's own WebSocket
rather than through Puppeteer or Playwright, which would each bring a Chrome
download and a driver to open a page and evaluate two scripts. The box's
Chrome is found instead — CHROME_PATH, PATH, the two caches — and a bare
Puppeteer build runs when its libraries are staged under
~/.local/share/chrome-deps, which is what this box has. axe-core is the one
dependency; it is pure JavaScript and is read from the package, not fetched.

The evaluation file mirrors the tool's export, read from its source rather
than guessed, and was verified by replaying its open() with the jsonld
version it pins: every assertion restores, for 2.2 and for 2.1. That replay
caught the one thing reading the code did not. The tool's context stops at
WCAG21, and a 2.2 id like `WCAG22:reflow` survives its processor only as an
IRI with an unknown scheme; defining the prefix, as seemed correct, expanded
every id into a form the tool cannot fold back and dropped all 33.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Khk1C6Ese6xjdHAWLVstca
@socket-security

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedaxe-core@​4.13.01001001009970

View full report

Comment thread src/wcag.ts

/** `linux-152.0.7977.42` after `linux-131.0.6778.204`: newest first by version, not by string. */
const byVersionDesc = (a: string, b: string): number => {
const parse = (name: string): number[] => (/(\d+(?:\.\d+)*)/.exec(name)?.[1] ?? '0').split('.').map(Number);
@github-actions

Copy link
Copy Markdown

ThreatCrush Security Scan

13 finding(s)

HIGH/CRITICAL: 4 | MEDIUM: 5 | LOW: 4

Severity Rule Location
HIGH sh-remote-script-execution root-ubuntu.sh:3227
HIGH sh-remote-script-execution root-ubuntu.sh:3228
HIGH sh-remote-script-execution root-ubuntu.sh:4840
HIGH sh-remote-script-execution root-ubuntu.sh:4844
MEDIUM sh-remote-script-execution root-ubuntu.sh:4993
MEDIUM sh-remote-script-execution root-ubuntu.sh:6055
MEDIUM redos-nested-quantifier src/domain-free.ts:56
MEDIUM redos-nested-quantifier src/mail.ts:1042
MEDIUM redos-nested-quantifier src/wcag.ts:556
LOW secret-generic-credential src/credentials.ts:36
LOW secret-generic-api-key test/credentials.test.ts:208
LOW secret-generic-credential test/mail.test.ts:135
LOW secret-generic-credential test/shorten.test.ts:36

Snippets are redacted; ThreatCrush never prints matched credential material.

@ralyodio
ralyodio marked this pull request as ready for review September 13, 2026 01:50
@ralyodio
ralyodio merged commit b4e8587 into master Sep 13, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants