feat(wpt-wintertc): run fetch tests against a real wptserve, refine expectations, stabilize the ratchet#4
Open
Alan Wisper (wisper-dev) wants to merge 4 commits into
Open
Conversation
The fetch/api tests resolve relative URLs against the document location and
fetch WPT resources and stateful handlers; with no server they die in setup
("Failed to parse URL"). Deno and workerd both run upstream Python wptserve as
a sidecar — this mirrors that.
- setup.ts: add tools/ to the WPT sparse checkout and patch it to run
`wpt serve` without a virtualenv (system Python + WPT's vendored deps).
- wpt-server.ts: start wptserve on a free 127.0.0.1 port, wait until it serves,
stop it via SIGINT so its multiprocessing child servers are reaped, not
orphaned.
- adapter: start the server only when fetch tasks are in scope, hand its origin
to the runner, stop it when the run ends; a start failure does not sink the
serverless encoding/url tests.
- wintertc-runner.js: root the document location and get-host-info at the
server so relative fetches reach real resources.
Scope: HTTP, same-origin only. Cross-origin subdomains and .https/.h2 tests
still fail — they need the web-platform.test subdomain aliases and a client
that trusts WPT's CA — but now produce a real fetch-conformance signal instead
of a setup crash.
.window.js tests need a document / window / navigation the Elide runtime has no equivalent for, and fetch-later needs window / iframe / bfcache — they are structurally unreachable, not regressions. Skipping them (74 of 313 files) keeps the pass rate over reachable tests. encodeInto.any.js is deliberately not skipped: its ArrayBuffer branch passes and skip is file-level, so skipping would drop the passing subtests; only its SharedArrayBuffer branch (needs WebAssembly.Memory) stays a known failure.
wpt-server: - read the bound HTTP port from wptserve stdout (removes the pre-allocation race), bind loopback explicitly, reap child listeners on stop, and clean up the temp config dir wpt-wintertc: - normalize the ephemeral server port out of test ids so they stay ratchetable - gate the server on fetch/ paths and attribute a server-start failure explicitly instead of collapsing fetch metrics silently setup: - guard commands.json parsing, verify the offline serve patch, and make the paths edit idempotent expectations: - skip the structurally-unreachable .https / .h2 tests, list the browser-only window.js files precisely (keeping those that pass headless), and correct the encodeInto note - regenerate the ratchet (union across runs) to prune now-skipped entries tests: - cover the server lifecycle, port parsing, and the server-failure gate
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes the WinterTC WPT
fetch/suite actually execute by running upstream Pythonwptserveas aloopback HTTP sidecar (the way Deno and workerd run fetch conformance), refines the expectations so
the pass rate reflects what is structurally achievable in this cut, and makes the committed ratchet
stable against the fetch suite's run-to-run flakiness.
What changed
Server sidecar --
harness/src/adapters/wpt-server.tswpt serveon loopback HTTP for thefetch/group only; encoding/url run serverless.bind-then-hope-it's-free (TOCTOU) race.
bind_address+browser_host), confirmed 127.0.0.1-only.config-dir cleanup.
Adapter --
harness/src/adapters/wpt-wintertc.tsfetch/paths; a server-start failure is attributed explicitly (each fetchtask errors loudly rather than silently collapsing the fetch metric).
(e.g.
mode-same-origin) stay ratchetable across runs.Setup --
bin/setup.tswpt serveto run offline (system Python, no venv), guards commands.jsonparsing, makes the paths edit idempotent, and verifies the patch under
setup --check.Expectations --
expectations/wpt-wintertc.toml+ ratchet.https/.h2(HTTP-only cut: no WPT-CA trust, noHTTP/2) and the browser-only
.window.jsfiles with zero headless passes..window.jsfiles that DO pass headless, so their passing subtests count (previouslymasked by a blanket skip).
new-pass, never a spurious regression.
Tests -- cover the server lifecycle, port parsing, the
fetch/gate, and the server-failure path.Conformance
wpt-wintertc: 88.2% pass (threads 4) / 88.0% (threads 1), 0 regressions against the committedratchet, both single- and multi-threaded.
Notes
TextDecoder, URL setters,data:-URLXMLHttpRequest, relative-fetch()base resolution).cpythonsuite is not checked outlocally, and the
javac-jtregregistry test assertsjtregCaseTimeoutSeconds = 300whileregistry.tomlis now900.