fix(e2e): make the runtime-error tests able to fail, drop a save race

[nk-o]

Toolkit 0.6.0 plus the fixes that came out of profiling this suite. Two of these are tests that currently cannot fail — those matter more than the seconds.

The runtime-error tests never observe the errors they exist to catch

lottie-block.spec.js and gist-block.spec.js both do:

const frontendPage = await publishAndGetFrontendPage(page, editor);
const runtimeErrors = trackRuntimeErrors(frontendPage);   // ← too late

publishAndGetFrontendPage does newPage() → goto(href). Playwright does not buffer console or pageerror for listeners attached afterwards, and goto resolves on load — which has already waited for the async plugin scripts to both load and execute (classes/class-assets.php:413-414). So an error thrown during script evaluation was emitted and dropped before the listener existed.

relevantErrors was therefore always [], and expect(relevantErrors).toEqual([]) always passed.

The lottie one is the regression test for the duplicate customElements.define('lottie-player', …) fix. As written it would not catch a reintroduction.

The helper now takes an onPageReady callback and hands the page over before navigating.

Heads up for review: these two assertions have never actually run. If either now goes red, that is a real finding, not a broken test — the whole point of the change is that they can fail again.

The typography save is a race it loses

setFontSetting slept 1000 ms for a save the settings page debounces by exactly 1000 ms (settings/pages/typography.js:28-31) and then fires without awaiting (:254-260). The sleep expired as the POST was going out, and the page fixture closes the page at teardown, discarding anything in flight.

When it lost, the symptom appeared in a different test — Check fonts available on backend and frontend — as a mismatch against EXPECTED_GOOGLE_FONTS_FAMILY. Classic intermittent-and-baffling.

It now waits on the POST itself. No time saved; the 1000 ms is in the product.

The gist counters were asserted without retry

await expect(mockStats.jsonRequests).toBeGreaterThan(0) — mockStats is a plain object the route handlers mutate, so await on a non-thenable buys nothing and the check ran once. That is precisely why a flat 1 s sleep had to sit in front of it. expect.poll retries and ends as soon as the mock fires.

Both sleeps (editor and frontend) are gone. ~1.6 s.

Smaller

• Two waitForLoadState('domcontentloaded') calls that ran after goto had resolved on load — strictly later, so no-ops.
• The build step is gone from the lint job. lint:js is biome check, and the shared Biome config excludes **/build; lint:css globs the SCSS sources. Neither reads anything the build produces. 8–15 s per PR.
• @nk-crew/plugin-toolkit → 0.6.0 (toolkit#2): no tracing on tests that pass, and reportSlowTests restored so the next run prints its own slowest files.

Lockfile touched surgically — four fields for that one package. npm install --package-lock-only re-resolves the whole tree and, on a sibling repo, dropped two optional peers while flipping ~60 entries to dev.

Sleeps deliberately kept

console-errors.spec.js:100 and the two in lottie-block.spec.js are settle windows for the absence of a future event. There is no signal for "no error is coming", so a sleep is the correct tool.

Flagged, not fixed

ghostkit_typography is never reset. wp-env's database survives env:stop, so from the second run onward every editor boot enqueues a real fonts.googleapis.com stylesheet inside the load gate — which is likely why console-errors.spec.js has to ignore /fonts\.gstatic\.com/i and /net::ERR_/i. Test 4 also silently depends on tests 1–3 having run, so --grep on it alone fails.

I did not fix it: the obvious reset via update_custom_typography merges rather than replaces (classes/class-rest.php:1834-1856), so {} lands as [], and getting it right needs a local run.

Also flagged: publishAndGetFrontendPage and trackRuntimeErrors are copy-pasted across three spec files, which is why this fix had to be applied twice. Worth a tests/e2e/utils/ directory like the sibling plugins have.

Not included

assets-loading.spec.js boots the editor and publishes six times to produce content REST could create (est. 15–30 s). Detection is content-driven, so it should be equivalent — but it would stop the markup round-tripping through the editor's parse/serialize, and that wants a local run first.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8dea5e0891

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Wait for the mocked stylesheet to finish loading

In the editor test, cssRequests is incremented before route.fulfill() completes, so this poll only proves that the stylesheet request started. The bundled gist-simple code inserts the mocked gist and can emit the errors under test from the stylesheet's later load callback; because the fixed delay was removed, runtimeErrors can be filtered before that callback runs, allowing asynchronous regressions to pass. Wait for the rendered mocked gist content (such as .gist-file) or another post-load signal before inspecting the error array.

Useful? React with 👍 / 👎.