Bypass paranext-core first-run wizard overlay in e2e tests - #169
Conversation
paranext-core added a first-run wizard overlay that gates the app on a fresh profile; when the registration service is unreachable (dev machines, CI) it renders as a full-screen modal Dialog that intercepts pointer events, so e2e tests time out clicking the toolbar/menus behind it. Seed platform.firstRunComplete before launching Electron so the overlay never shows, mirroring paranext-core's own e2e app.fixture: the smoke fixture seeds+restores around each worker's app, and the CDP setup/teardown seed+restore (with a .cdp-settings-backup marker) around the launched instance. The developer's dev-appdata settings are restored after every run. Verified locally: smoke passes; the CDP tier's 5 feature tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Warning Review limit reached
Next review available in: 22 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThe E2E infrastructure now seeds and restores settings around Electron runs, centralizes process and directory cleanup helpers, improves CDP failure handling, and supports alternate Home-tab startup layouts for interlinearizer readiness. ChangesE2E lifecycle management
Estimated code review effort: 4 (Complex) | ~45 minutes Possibly related issues
Possibly related PRs
Suggested reviewers: Sequence Diagram(s)sequenceDiagram
participant PlaywrightFixture
participant Electron
participant ProcessUtils
participant CDPTeardown
PlaywrightFixture->>Electron: seed settings and launch
Electron-->>PlaywrightFixture: expose test UI
PlaywrightFixture->>ProcessUtils: wait for app exit and remove user data
CDPTeardown->>Electron: kill process by PID
CDPTeardown->>ProcessUtils: wait for PID exit and remove user data
CDPTeardown->>PlaywrightFixture: restore backed-up settings
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…failure Shares the CDP tier's on-disk backup/self-heal logic with the smoke tier (previously in-memory only, so a hard-killed process lost the developer's original settings). Also has CDP global setup restore settings and kill the launched app immediately on a readiness failure, instead of leaving that to the next run's self-heal.
…helpers Restoring settings before killing the launched app risked the still-running process flushing the seeded value back over the just-restored original, with no backup left to recover it. Kill (and wait for actual exit) now happens first, mirroring globalTeardownCdp's ordering. Also extracts the wait-for-exit-then-remove-directory retry logic, previously duplicated with slightly different behavior across the smoke teardown, CDP setup failure path, and CDP teardown, into shared process-utils helpers (waitForProcessExit, waitForPidExit, removeDirWithRetry) so all three sites behave identically.
paranext-core's newest build hides the Home/Editor columns' tab bars entirely in the default Simple interface mode and ships Resources-column tabs that never resolve on a bare e2e profile, breaking both the strict dock-tab-title readiness gate and WEB-tab clicks. Seed platform.interfaceMode: 'power' (E2E_SETTINGS_OVERRIDES) alongside the existing firstRunComplete seed to restore the visible/clickable dock-tab layout the suite is written against — a stand-in until Simple-mode e2e coverage is added. Separately, a fresh profile now opens directly to an already-open Home tab rather than an empty "Scripture Editor" tab. Widen openInterlinearizerFromScriptureEditor's and ensureInterlinearizerOpenOnWeb's initial readiness gates to accept either starting layout, and skip the redundant toolbar Home-button click when Home is already open. Full local e2e suite (smoke + CDP, 10/10) passes against the current paranext-core.
- Stop the renderer dev server in global-setup-cdp's failure path too, since Playwright skips globalTeardown when globalSetup throws; shares a new DEV_SERVER_PID_FILE constant with global-setup.ts/global-teardown.ts instead of each recomputing the same path. - Clear the pending setTimeout in waitForProcessExit so a fast process exit doesn't leave a timer keeping the event loop alive. - Rename seedFirstRunComplete to seedE2ESettingsOverrides: it seeds all of E2E_SETTINGS_OVERRIDES (including interfaceMode), not just firstRunComplete. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The readiness gate only confirmed paranext-core's own subsystems (service hosts, dock tabs) before declaring the launched instance ready, leaving each CDP feature test's short per-test budget to also cover the extension's first-ever activation - a budget sized for confirming an already-settled instance, not for that. Add waitForInterlinearizerReady to the same gate so a slow-to-activate extension fails setup once, clearly, instead of cascading into every feature test failing against a shared instance nothing confirmed was actually ready. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Tighten the KillFromPidFileResult.pid doc to match what globalTeardownCdp actually reads, and force focus on an already-open Home tab instead of assuming it's the focused one before driving its iframe.
8ad8d1c to
0c6d34f
Compare
The bundled sample project installs into the project root asynchronously, so a test that raced straight from app-ready to clicking WEB's row in Home could lose that race — most visibly on the CDP tier's single shared instance, where losing it once failed every downstream feature test for the rest of the run. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Unlike SERVICE_HOST_OBJECT_METHODS's bare existence handlers, this constant depends on both the object id and method name staying stable upstream — flagged in PR review. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
alex-rawlings-yyc
left a comment
There was a problem hiding this comment.
@alex-rawlings-yyc reviewed 8 files and all commit messages, and made 1 comment.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on imnasnainaec).
e2e-tests/global-teardown-cdp.ts line 49 at r2 (raw file):
} restoreSeededSettings();
⛏️ Not ownership-gated, unlike everything else in this teardown
This runs even after a warm-instance-reuse run, which seeds nothing. Since the backup file is deliberately shared across both tiers (helpers.ts:176-180), a reuse-path teardown can consume and delete a backup belonging to a smoke run — leaving that run's seeded test values as the developer's "restored" settings.
Every other resource here is gated on the marker files; this is the one that isn't. Suggested fix:
const launchedApp = fs.existsSync(CDP_PID_FILE) || fs.existsSync(CDP_USER_DATA_FILE);then if (launchedApp) restoreSeededSettings();
Two details that aren't obvious: the snapshot has to be taken before killProcessFromPidFile, which unlinks CDP_PID_FILE — read it after and ownership always reports false. And || rather than &&, so a run that crashed between the two marker writes still restores the seed it owns.
Low severity — the tiers run sequentially under npm run test:e2e, so this needs an interleaving to bite. Raising it because it's a one-line guard, and because the app kill and dir removal directly above both gate on the marker files. Setup's clearStaleOwnershipMarkers() comment says clearing them "keeps teardown a true no-op on this reuse path" — which this line quietly breaks.
restoreSeededSettings() ran unconditionally, unlike the app-kill and user-data-dir cleanup above it. Since the settings backup file is shared across both e2e tiers, a warm-instance-reuse teardown (which seeds nothing) could consume and delete a still-in-flight smoke run's backup. Flagged in PR #169 review (alex-rawlings-yyc). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
e2e-tests/global-teardown-cdp.ts (1)
35-58: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winEnsure shared teardown runs when CDP cleanup fails.
The
readFileSyncat Line 36 is not guarded, so a concurrent marker deletion or filesystem error can abort this function beforeglobalTeardown(config)runs. Likewise, an exception fromrestoreSeededSettings()at Line 55 skips shared cleanup. Wrap marker cleanup and restoration in error handling withglobalTeardown(config)in afinallypath.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@e2e-tests/global-teardown-cdp.ts` around lines 35 - 58, Update the teardown flow around CDP_USER_DATA_FILE cleanup and restoreSeededSettings so readFileSync, marker removal, and restoration errors cannot prevent shared cleanup. Wrap the cleanup and restoration logic in try/catch as appropriate, and place globalTeardown(config) in a finally path so it always executes, preserving the existing launchedApp gating and cleanup behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@e2e-tests/global-teardown-cdp.ts`:
- Around line 19-21: The launchedApp check in globalTeardownCdp must not infer
ownership from shared CDP_PID_FILE or CDP_USER_DATA_FILE existence. Introduce or
reuse a per-run ownership token or run-specific marker/backup established by
globalSetupCdp, and have teardown verify that token before killing the PID or
calling restoreSeededSettings(); apply the same ownership guard to the related
cleanup at the additional referenced block.
---
Outside diff comments:
In `@e2e-tests/global-teardown-cdp.ts`:
- Around line 35-58: Update the teardown flow around CDP_USER_DATA_FILE cleanup
and restoreSeededSettings so readFileSync, marker removal, and restoration
errors cannot prevent shared cleanup. Wrap the cleanup and restoration logic in
try/catch as appropriate, and place globalTeardown(config) in a finally path so
it always executes, preserving the existing launchedApp gating and cleanup
behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: e7a6bd8c-e368-4407-9bda-a19dd0cf1a36
📒 Files selected for processing (3)
e2e-tests/fixtures/helpers.tse2e-tests/global-setup-cdp.tse2e-tests/global-teardown-cdp.ts
🚧 Files skipped from review as they are similar to previous changes (2)
- e2e-tests/global-setup-cdp.ts
- e2e-tests/fixtures/helpers.ts
alex-rawlings-yyc
left a comment
There was a problem hiding this comment.
@alex-rawlings-yyc reviewed 1 file and all commit messages, and resolved 1 discussion.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on imnasnainaec).
readFileSync(CDP_USER_DATA_FILE) was unguarded, so a marker deleted between the existsSync check and this read (or any other fs error in this block) would abort globalTeardownCdp before it reached globalTeardown(config), leaking the renderer dev server and any lingering Electron process. Wrapped the block so any such error is logged instead of skipping shared cleanup. Flagged in PR #169 review. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
imnasnainaec
left a comment
There was a problem hiding this comment.
Fixed CodeRabbit's "Outside diff range comments" catch.
@imnasnainaec made 3 comments and resolved 1 discussion.
Reviewable status: 7 of 8 files reviewed, all discussions resolved (waiting on alex-rawlings-yyc).
e2e-tests/global-teardown-cdp.ts line 49 at r2 (raw file):
Previously, alex-rawlings-yyc (Alex Rawlings) wrote…
⛏️ Not ownership-gated, unlike everything else in this teardown
This runs even after a warm-instance-reuse run, which seeds nothing. Since the backup file is deliberately shared across both tiers (
helpers.ts:176-180), a reuse-path teardown can consume and delete a backup belonging to a smoke run — leaving that run's seeded test values as the developer's "restored" settings.Every other resource here is gated on the marker files; this is the one that isn't. Suggested fix:
const launchedApp = fs.existsSync(CDP_PID_FILE) || fs.existsSync(CDP_USER_DATA_FILE);then
if (launchedApp) restoreSeededSettings();Two details that aren't obvious: the snapshot has to be taken before
killProcessFromPidFile, which unlinksCDP_PID_FILE— read it after and ownership always reports false. And||rather than&&, so a run that crashed between the two marker writes still restores the seed it owns.Low severity — the tiers run sequentially under
npm run test:e2e, so this needs an interleaving to bite. Raising it because it's a one-line guard, and because the app kill and dir removal directly above both gate on the marker files. Setup'sclearStaleOwnershipMarkers()comment says clearing them "keeps teardown a true no-op on this reuse path" — which this line quietly breaks.
Done
alex-rawlings-yyc
left a comment
There was a problem hiding this comment.
@alex-rawlings-yyc reviewed 1 file and all commit messages.
Reviewable status:complete! all files reviewed, all discussions resolved (waiting on imnasnainaec).
restoreSeededSettings() sat inside the same try as the kill and user-data cleanup, so an earlier throw there (e.g. readFileSync on a marker deleted mid-race) would skip it along with everything else, leaving the developer's real settings.json overwritten by seeded test values until the next e2e run self-heals it. Moved the ownership snapshot and restore out to their own unconditional step so a failure in the unrelated kill/dir cleanup can't suppress it. Flagged in PR #169 review. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
alex-rawlings-yyc
left a comment
There was a problem hiding this comment.
@alex-rawlings-yyc reviewed 1 file and all commit messages.
Reviewable status:complete! all files reviewed, all discussions resolved (waiting on imnasnainaec).
This is an e2e-harness-only catch-up fix for recent paranext-core behavior changes, plus reliability hardening for the harness itself. No production/extension code changes.
Problem
Against current paranext-core, the e2e suite can't reliably reach the dock/Home UI it drives:
platform.interfaceModedefaults to'simple', which hides the Home/Editor tab bars entirely and populates the Resources column with default tabs that never resolve real titles on a bare e2e profile — breaking both the cold-start readiness gate and the WEB-tab click flow.Fix
Settings seeding.
E2E_SETTINGS_OVERRIDES(platform.firstRunComplete: true,platform.interfaceMode: 'power') is written into paranext-core'sdev-appdata/data/settings.jsonbefore Electron launches, in both tiers — the smoke tier's per-worker fixture (preConfigureSettings()) and the CDP tier's global setup (backupAndSeedSettings()). Forcing Power mode is a stopgap, not a permanent choice — see Follow-up. The original file is backed up to disk (not just in memory) and self-heals a stale backup left by a prior hard-killed run, so a CI timeout or Ctrl+C never loses a developer's real settings; it's restored once the app closes (smoke) or the launched instance is killed (CDP).Flexible dock-layout handling.
openInterlinearizerFromScriptureEditor()andensureInterlinearizerOpenOnWeb()(fixtures/helpers.ts) now accept either starting layout — an empty Scripture Editor tab with no Home tab, or an already-open Home tab — and skip the redundant toolbar Home-button click when Home is already open.WEB-project readiness.
waitForAtLeastOneProjectMetadata()(mirrors paranext-core's own e2e helper) pollsProjectLookupService.getMetadataForAllProjectsuntil a project is registered. It's called inline, right before clicking WEB's row in Home, and also once in the CDP tier's global setup, alongside the interlinearizer extension's own activation gate (waitForInterlinearizerReady()). Since the CDP tier launches one shared instance for the whole run, gating setup on both means a broken or slow-starting instance fails setup once with a clear message, instead of every downstream feature test burning its own timeout.CDP setup failure-path cleanup. If any readiness wait throws (ports never open, renderer never settles, extension never activates, project never registers), global setup now kills the launched process tree, removes the isolated user-data dir, restores the seeded settings, and removes its PID/user-data marker files before rethrowing. Playwright skips
globalTeardownwhenglobalSetupthrows, so previously all of that leaked to the next run.Teardown/process-utility hardening. Shared helpers in
process-utils.ts—waitForProcessExit(race a live exit event against a timeout),waitForPidExit(poll a bare PID for exit, for callers with no live process handle),removeDirWithRetry(retry directory removal once after a delay) — replace duplicated one-off sleep/retry logic in both teardown files.killProcessFromPidFilenow returns{ killed, pid }instead of a bare boolean, so callers can wait on that specific PID's exit before touching files it may still hold open.Full local e2e suite (smoke + CDP) passes against current paranext-core.
Follow-up
Filed #170 to add real Simple-mode e2e coverage instead of permanently forcing Power mode.
🤖 Generated with Claude Code
Devin: https://app.devin.ai/review/sillsdev/interlinearizer-extension/pull/169
This change is
Summary by CodeRabbit