Add VS Code UI tests and fix viewer edge cases by rafaelha · Pull Request #17 · QuEraComputing/vscode-stim

rafaelha · 2026-10-01T02:51:36Z

Summary

Add end-to-end UI coverage using the same Extension Tester/Selenium + Mocha approach as vscode-flamegraph, then fix three edge cases reproduced with that framework.

Developed directly in the local checkout on a branch from main. This PR does not include the separate SVG zoom-mode PR.

Add npm run test:ui and a pinned VS Code 1.137.0 test runtime, with isolated profile/extensions and copied fixtures under .ui-tests/.
Exercise the packaged extension in an actual VS Code window, with real WASM rendering and host/webview messaging rather than mocked DOM or VS Code APIs.
Add a Linux CI job using Xvfb, with screenshots, DOM snapshots, and VS Code/WebDriver logs uploaded on failure.
Use Node 22 in CI/release workflows for the test tooling; no new runtime dependencies.
Document execution and exclude test assets from Git and VSIX packaging. Also exclude local .claude/worktrees, which the packaging check found inside the VSIX.

Confirmed Bug Fixes

Invalid Tsim shorthand such as CCX 0 1 or R_XX(0.25) 0 0 threw before the visualization error handler, leaving a stale diagram. Show the error, clear stale statistics, and recover after corrected source is saved.
A zero maximum layer was treated as an unknown bound. Disable next-layer navigation and clamp numeric/keyboard input for single-layer circuits.
Trailing TICK inside nested repeats was overlooked, allowing navigation into an empty final slice. Account for closing repeat braces when determining the last meaningful instruction.

The framework and follow-up fixes are separate commits. Only these confirmed bugs receive additional UI regression tests.

UI Coverage

.stim activation, rendered circuit, SVG tools, and statistics.
.tsim activation and non-Clifford labels.
.dem activation and restricted matching-graph controls.
Removing and restoring noise.
Slice-layer navigation, all-ticks mode, and detector-operation overlays.
Ctrl+wheel zoom and mouse-drag panning.
Parse-error display and recovery after editing/saving the source.
Tsim lowering errors and recovery after correction.
Single-layer navigation bounds, including numeric input and End.
Nested-repeat trailing-tick bounds and backward navigation.

No version bump or publication. CDN-dependent 3D viewers and OS clipboard integration remain outside this initial suite.

Validation

Reproduced each of the three bugs against the original implementation in isolated VS Code before fixing it.
npm run test:ui: all 10 tests passed locally.
npx tsc --noEmit -p tsconfig.json
npm run test:engine: 28 passing tests.
npm run test:grammar
npm run test:wasm
Extension Tester production build, VSIX packaging, and installation.
vsce ls --no-dependencies: no test runtime, test files, or nested worktrees included.
npm audit: 0 vulnerabilities when adding the framework dependencies.
git diff --check

rafaelha

Reviewed the UI test framework and the three viewer fixes. The } handling in endsWithTick, the zero-layer bounds, and the Tsim lowering catch all address the reported repros, and the test suite covers a lot of real interaction.

Main points (details inline):

media/main.js: with the tickMax > 0 guard gone, the tickMax: 0 that non-slice renders post becomes a hard bound right after switching to a slice base, so input in that window resets the layer to 0.
src/panel.ts: the synchronous lowering-error path can be overwritten by an earlier in-flight refresh. A refresh sequence guard would fix this.
endsWithTick is case-sensitive; stim isn't.
Tests: openVisualizer can't actually reset full, and a throwing captureFailure would skip the rest of the suite.

The rest are nits.

🤖 Generated with Claude Code

rafaelha · 2026-10-06T15:59:32Z

    let next = Math.floor(t);
    if (!Number.isFinite(next) || next < 0) next = 0;
-    if (state.tickMax > 0) next = Math.min(next, state.tickMax);
+    next = Math.min(next, state.tickMax);


Stale tickMax = 0 now clamps the layer to 0 after a base switch. Timeline/matchgraph renders still post tickMax: 0, and with the > 0 guard removed, 0 is now a hard bound. Example: on timeslice at layer 7, switch to timeline, then back to timeslice. Until the new render arrives, any wheel, ←/→, End or typed value clamps to min(n, 0) = 0 and posts setTick(0), so the layer is lost (▶ is also disabled during that window). Suggest having the host send tickMax: null (or omit it) for non-slice renders and skipping the clamp in the webview while it's unknown.

rafaelha · 2026-10-06T15:59:32Z

+    try {
+      text = shorthandToStim(this.doc.getText());
+    } catch (e: any) {
+      this.postStats(null);


This early return can still be overwritten by an earlier refresh. refresh() has no generation guard. If a previous refresh is still awaiting the WASM module (first open, or the first call after an engine exception resets modulePromise), it posts its svg and stats after this synchronous error, so the stale diagram comes back. A refreshSeq counter that drops posts from superseded refreshes would make the 'clear stale diagram/statistics' guarantee hold.

rafaelha · 2026-10-06T15:59:32Z

    const body = (hash >= 0 ? lines[i].slice(0, hash) : lines[i]).trim();
-    if (!body) continue; // skip blank / comment-only lines
+    if (!body || body === "}") continue;
    return /^TICK(\[[^\]]*\])?$/.test(body);


Stim gate names are case-insensitive, so a trailing tick (including the newly handled tick + } case) isn't detected and the empty final slice stays reachable.

Suggested change

return /^TICK(\[[^\]]*\])?$/.test(body);

return /^TICK(\[[^\]]*\])?$/i.test(body);

rafaelha · 2026-10-06T15:59:33Z

 const DIM_CAPABLE_BASES: BaseType[] = ["timeline", "matchgraph"];

-// True if the last meaningful line of the circuit is a bare TICK (optionally
+// True if the last instruction, including inside a repeat, is a TICK (optionally


Optional, longer term: detecting an empty final slice by scanning text keeps needing special cases (}, case, multiple trailing TICKs: H 0\nTICK\nTICK still leaves an empty slice in range). The binding already has the parsed circuit, so it could report whether the flattened circuit ends in TICK, or count the trailing empty slices, by walking the last op into REPEAT blocks.

rafaelha · 2026-10-06T15:59:33Z

+  if (!name.endsWith(".dem")) {
+    await selectBase("timeline");
+    await setToggle("#toggle-noise", false);
+    await setToggle("#toggle-full", false);


This normalization is a no-op. It runs after selectBase("timeline"), where #toggle-full is hidden and its aria-pressed is state.full && dependent, which is always "false". A persisted full=true (for example, the all-ticks test failing between its two toggles) is never cleared, so later slice tests open in full mode with the stepper hidden and fail too. The extester file order comes from an unsorted globSync. Consider normalizing full while on a slice base.

rafaelha · 2026-10-06T15:59:33Z

+      const error = await driver().wait(until.elementLocated(By.css("#view .error")), 15000);
+      assert.match(await error.getText(), message);
+      assert.equal((await driver().findElements(By.css("#view svg"))).length, 0);
+      assert.equal(await (await find("#save-btn")).isDisplayed(), false);


The PR says lowering errors also clear stale statistics (postStats(null)), but nothing asserts it. Something like checking that #info-tip innerHTML contains Cannot parse would cover that line.

rafaelha · 2026-10-06T15:59:33Z

+      this.postStats(null);
+      this.panel.webview.postMessage({
+        command: "error",
+        message: String(e?.message ?? e),


Nit: stim parse errors are routed through explainStimError and quote the offending line, but lowering errors post the bare message (CCX expects bare qubit integer targets in groups of three.). Including the source line would match the other error path.

rafaelha · 2026-10-06T15:59:33Z

+
+  it("reports Tsim lowering errors and recovers after correction", async () => {
+    const view = await openVisualizer("invalid-gates.tsim");
+    async function edit(text) {


Nit: this switchBack → openResources → setText → save → switchToFrame sequence is also written out twice in visualizer.test.cjs's parse-error test. It could live in helpers.cjs.

rafaelha · 2026-10-06T15:59:33Z

+} = require("./helpers.cjs");
+
+describe("Stim visualizer UI", () => {
+  let view;


Nit: view is assigned in every test but only read in the parse-error test. The beforeEach/afterEach hooks are also identical to the ones in edgeCases.test.cjs and could be shared.

rafaelha · 2026-10-06T15:59:33Z

+      - run: npm ci
+
+      - name: Real VS Code UI tests
+        run: xvfb-run -a npm run test:ui


Nit: consider an actions/cache step for the VS Code/ChromeDriver downloads under .ui-tests/, keyed on extester.config.json. That avoids re-downloading the pinned runtime on every push.

Resolve conflicts with the OIDC release publishing change (#18): keep both the UI test devDependencies (mocha, vscode-extension-tester) and ovsx, merge package-lock.json from both sides (restoring the optional platform binaries npm dropped during auto-resolution), and drop the .vscodeignore entries main already added. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Viewer: - Treat an unknown slice range as unbounded in the webview. Non-slice renders now report tickMax: null and a base switch clears it, so input before the new render no longer clamps the layer to 0. - Drop results from superseded refreshes, so a slow render or stats call can't overwrite a later error (or diagram) with stale output. - Replace the endsWithTick line scan with countTrailingTicks/lastSliceTick in stimEngine: case-insensitive, unrolls REPEAT blocks, and excludes every empty trailing slice. Lowercase `tick` previously left a slice that traps stim's timeslice renderer. - Quote the offending line in Tsim lowering errors, matching stim parse errors. UI tests: - Reset the full toggle on a slice base, where its state is visible. - Make failure capture best effort so it can't fail afterEach. - Share editor hooks and an editAndSave helper that reuses the existing editor instead of opening a second one that races for focus. - Emulate page focus so local runs pass while another app is in front. - Assert that statistics are cleared on Tsim lowering errors. - Cache the VS Code and ChromeDriver downloads in CI. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

rafaelha added 2 commits September 30, 2026 22:51

Add real VS Code UI tests with Extension Tester

14a92f2

Fix Tsim error handling and slice navigation edge cases

bcef16d

rafaelha changed the title ~~Add real VS Code UI tests and CI coverage~~ Add VS Code UI tests and fix viewer edge cases Oct 1, 2026

rafaelha commented Oct 6, 2026

View reviewed changes

rafaelha and others added 2 commits October 6, 2026 12:01

rafaelha merged commit a2d8381 into main Oct 7, 2026
2 checks passed

Navigation Menu

Repository navigation

Add VS Code UI tests and fix viewer edge cases - #17

Add VS Code UI tests and fix viewer edge cases#17
rafaelha merged 4 commits into
mainfrom
codex/ui-tests

rafaelha commented Oct 1, 2026 •

edited

Loading

Uh oh!

rafaelha left a comment

Uh oh!

rafaelha Oct 6, 2026

Uh oh!

rafaelha Oct 6, 2026

Uh oh!

rafaelha Oct 6, 2026

Uh oh!

rafaelha Oct 6, 2026

Uh oh!

rafaelha Oct 6, 2026

Uh oh!

rafaelha Oct 6, 2026

Uh oh!

rafaelha Oct 6, 2026

Uh oh!

rafaelha Oct 6, 2026

Uh oh!

rafaelha Oct 6, 2026

Uh oh!

rafaelha Oct 6, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

	return /^TICK(\[[^\]]*\])?$/.test(body);
	return /^TICK(\[[^\]]*\])?$/i.test(body);

Conversation

rafaelha commented Oct 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Confirmed Bug Fixes

UI Coverage

Validation

Uh oh!

rafaelha left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

rafaelha commented Oct 1, 2026 •

edited

Loading