fix: unify divergence screen capture with snapshot's full-window scope#1265
Conversation
Size Report
Startup median (7 runs, lower is better):
Top changed chunks:
|
|
Verified this diff against One framing note (correction posted on #1264): the Android divergence-scope "bug" that partly motivated this was a measurement (timing) confound on my end — the code has no app-scoping in the divergence path. This PR's genuine value is unification + macOS/Linux + the ADR invariant (now provably true) + the test-id fix. Merge on those merits; the live acceptance (held overlay in |
|
I found two issues and would request changes before merging:
All completed CI checks are green; the iOS smoke test was still pending when reviewed. |
|
Review: keep the refactor, but it does not fix #1264 — the archived live evidence points at divergence-side ref selection, not capture scope. Verified against
Actionable changes for this fix:
Mitigating nuance worth recording: |
|
Verified your review — cap burial is correct. The archived JSON has Implementing your actionable list on this branch now:
Recording your nuance: |
|
Thanks — the cap-burial diagnosis is correct, and I've verified it against the code and reproduced it in a unit test. Pushed 1. Keep the 2. Rank within the cap instead of document-order slicing. Done in 3. Occlusion interaction. Done. When a system overlay mass-covers the app, 4. Fix the unit test to actually catch cap burial. Done. I replaced the small-fixture 5. Nuance preserved. ADR-0012 decision 4's amendment is reworded from "capture scope" to two guarantees — capture scope and within-cap ref selection (ranking + occlusion) — so it no longer overclaims that scope alone upholds the invariant. Live acceptance (for your device setup — not run here; device env is down): re-run the archived experiment — held volume dialog and persistent QS shade must appear in Verify (in the worktree): |
|
Re-review of exact head 7afa758: the cap ranking and occlusion regression is improved, but the new ADR invariant is still not proven by the production route.
All checks are green, but this head is not merge-ready. |
|
Addressed the re-review of 1. Complete the capture unification (freshness/retry parity). You're right — Regression added ( 2. Clean divergence-capture flags policy. Fixed. The capture no longer spreads Regression added ( (The helper extraction keeps 3. Live overlay acceptance — stays a maintainer step. Kept documented in the PR body, not faked. The unit fixtures prove ref selection after nodes are supplied and now also prove the divergence uses the same fresh-capture wrapper as Everything from the prior rounds is kept (cap-ranking + occlusion + test-id/comment fixes + unification). Verify (in the worktree): Head: |
|
Re-review of exact head 9cb3188: the two functional findings are fixed. Divergence now uses the full captureSnapshot freshness wrapper, the narrowing action flags are stripped, and the regressions fail on the prior behavior. One small code issue remains: the new sharp-drop freshness regression waits on the real 250 ms Android retry delay. Repo guidance forbids real-time waits in unit tests; mock the delay or use fake timers while retaining proof that the retry branch executes. Separately, merge readiness still requires the documented live Android overlay run. The current mocked fixtures do not prove that the helper returns separate-window overlay nodes at divergence time. Not merge-ready yet. |
|
Fixed the real-time-wait in the freshness regression. Pushed How: the retry delay is Retained proof of the retry branch: the capture-parity test still asserts (a) Measured effect: the divergence suite's Live Android overlay acceptance stays the maintainer's on-device step (env down) — documented in the PR body, not faked: held volume dialog + persistent QS shade must appear in Verify (worktree): Head: |
340b667 to
d7046b0
Compare
|
Rebased onto the new Conflicts: none — the rebase applied all four of my commits cleanly with no conflict markers. I still hand-verified the two overlap areas #1267 touched:
Re-verified against the merged base: Head: |
Route captureDivergenceObservation through captureSnapshotData — the same function the snapshot command itself builds its capture with (Android's snapshot-helper full-window route with its graceful app-scoped fallback, iOS's bounded system-modal probe path, macOS/Linux surface-scoped branches) — instead of a parallel hand-rolled dispatchCommand call. The chrome filter and meaningful-target filter stay layered on top as filters over that full capture, never as a scoping. Amends ADR-0012 decision 4 to state the invariant: an agent must never see a healthier `screen` in a divergence report than a plain `snapshot` would show it, so a separate-window system overlay (volume dialog, quick-settings shade, permission dialog) must survive into `screen.refs` exactly as `snapshot` would present it. Also fixes the synthetic `volume_dialog_slider` id in snapshot-chrome-android-statusbar.test.ts to the real, live-verified `volume_new_ringer_active_icon_container` id and rewords the test comment to read as a filter-logic unit test rather than a live-capture-path claim, and adds unit coverage for the invariant itself. Fixes #1264
…uried The #1264 root cause is cap burial, not capture scope: buildReplayDivergenceScreenRefs sliced candidates in document order, so a fully-captured separate-window overlay (volume dialog, QS shade, permission dialog) that enumerates after the app window's ~77 nodes lands past position 20 and is truncated away — the report shows a healthy-looking app under a covering overlay it cannot see (archived evidence: screen.truncated: true, zero volume refs). - Rank within the cap instead of document-order slicing: foreign-window (non-app-bundleId) hittable nodes — the dismiss targets for whatever covers the app — are promoted ahead of app content, otherwise stable (document order preserved within each tier; equal-priority app nodes never reshuffled). The 20-cap is a byte bound, not a first-20-in-tree-order policy. - Occlusion fallback: when a system overlay mass-covers the app (every app node annotated interactionBlocked: 'covered'), surface those covered nodes rather than emitting an empty screen.refs — a report whose capture holds meaningful nodes but whose refs is empty is broken by construction. - repairHint/suggestions consume the full captured node list, not the capped refs slice, so hint routing is unaffected; only screen.refs selection changes. Detection keys off node.bundleId (Android-only, from the a11y package); iOS/macOS leave per-node bundleId undefined, so ranking degrades to document order there (safe — those platforms surface modals via the probe path, not by cap-competing). Guarded on a known appBundleId so a sessionless capture never reorders. Tests: replaces the small-fixture #1264 test (which the overlay fit inside the cap regardless of order, so it did not prove the invariant) with a realistic full fixture (24 app controls + overlay dismiss-target captured LAST) that fails on document-order slicing and passes with ranking; plus occlusion tests (mass-covered app -> overlay surfaces, refs non-empty; bare-scrim fallback -> covered app nodes surfaced, refs non-empty). Both were verified to fail before the fix. ADR-0012 decision 4 amendment reworded to cover ref-selection ranking and the occlusion guarantee, not only capture scope. Refs #1264
… flags policy Completes the #1264 capture unification. The prior round routed captureDivergenceObservation through captureSnapshotData (the inner single-shot capture), but plain `snapshot`'s backend calls the HIGHER captureSnapshot wrapper, which owns Android freshness + post-action retry. A divergence could therefore consume the first stale/app-scoped dump while a plain `snapshot` retries to the fresh full-window tree — a divergence staler/narrower than `snapshot`, violating the invariant. - Route the divergence capture through the same `captureSnapshot` wrapper as plain snapshot, so it inherits freshness/post-action retry parity. No fork: the wrapper's params (device, session, flags, logPath) are all suppliable from the divergence path. - Build the divergence capture's flags from a clean, fixed policy (`divergenceCaptureFlags`: full-window, non-raw, default depth) instead of spreading the failed action's flags — so a failed `snapshot --raw`/scoped/`-d` action can no longer narrow the diagnostic tree. Only the interactive-only policy is carried (extracted as a helper so captureDivergenceObservation stays within complexity budget). Tests: a freshness-retry regression (session carries an active Android freshness marker; capture-1 is a stale near-empty dump that trips sharp-drop, capture-2 holds the overlay — asserts the divergence uses the retried fresh tree and dispatched twice), and a clean-flags regression (a failed raw/scoped/depth action — asserts the snapshot dispatch context drops snapshotRaw/scope/depth while still applying interactive-only). Both verified to fail on the pre-fix code. ADR-0012 decision 4 amendment updated to state the same-wrapper (freshness parity) and clean-flags guarantees. Live overlay acceptance remains a maintainer device step (env down): unit fixtures prove ref SELECTION after nodes are supplied, not that the Android helper returns the separate-window overlay at divergence time. Refs #1264
…'t real-wait The #1264 capture-parity regression exercised the real Android sharp-drop retry, which awaited the real ~250 ms `sleep` delay — repo guidance forbids real-time waits in unit tests. Mock `sleep` (the delay the retry path in snapshot-capture.ts awaits) to a no-op at the module level, so the retry BRANCH still executes (loop runs, retries, re-captures) without a wall-clock wait. The test still proves the branch: two on-device dispatches and use of the retried fresh tree (overlay present). Verified it still fails on the pre-fix single-shot path (1 dispatch) with the delay stubbed, so the stub does not make it vacuous. No production change; the delay stub needs no DI seam since `sleep` is a plain module export. Refs #1264
…ef frame Rebase reconciliation. #1257 (ADR-0014 session ref-frame lifetime) landed on main and changed captureDivergenceObservation to activate a PARTIAL ref frame (markSessionPartialRefsIssued) authorizing exactly the divergence screen's emitted refs — computing that "digestBodies" set with its own document-order, non-covered-only filter. My #1264 change made buildReplayDivergenceScreenRefs emit a DIFFERENT set (ranked, occlusion-fallback, meaningful-filtered), so the authorized frame would no longer match the shown screen: in the mass-covered fallback the screen surfaces covered refs that #1257's non-covered-only frame filter excluded, leaving the agent a ref the screen advertised but the frame rejects. Extract selectDivergenceScreenRefNodes as the single source of truth for which nodes screen.refs publishes and in what order. Both the rendered digest (buildReplayDivergenceScreenRefs) and the partial-frame authorization (captureDivergenceObservation -> markSessionPartialRefsIssued) derive from it, so the frame authorizes exactly the emitted set — preserving BOTH #1257's ADR-0014 intent and #1264's ranking/occlusion intent. Also refresh the captureDivergenceObservation doc to the partial-frame sequence. Test: assert the partial ref frame scope (session.refFrameScope) equals the emitted screen.refs set in the mass-covered fallback (covered refs included) — verified to fail on #1257's original non-covered-only digestBodies. Refs #1264 #1257
d7046b0 to
0f7a41b
Compare
|
Rebased onto the new Unlike the #1267 rebase, this one had real conflicts on both overlapping files. Resolved preserving both intents: src/daemon/handlers/session-replay-divergence.ts — conflict + a semantic reconciliation.
docs/adr/0012-interactive-replay.md — auto-merged, verified coherent. Decision 4 now carries three composed amendments, non-overlapping: my Re-verified against the merged base: Head: |
|
Summary
Implements #1264's decided direction: unify the replay-divergence
screencapture withsnapshot'sfull-window scope, so an agent never sees a healthier screen in a divergence report than a plain
snapshotwould show — separate-window system overlays (volume dialog, quick-settings shade, permissiondialogs) must appear in divergence
screen.refsexactly assnapshotshows them, instead of beinginvisible to both the dismiss target and
repairHint.captureDivergenceObservation(src/daemon/handlers/session-replay-divergence.ts)now calls
captureSnapshotData— the same functionsnapshotitself builds its capture with(
src/daemon/handlers/snapshot-capture.ts) — instead of a parallel hand-rolleddispatchCommandcall.This picks up every scope branch
snapshotsupports (Android's snapshot-helper full-window route withits graceful app-scoped fallback, iOS's bounded system-modal probe path, macOS/Linux surface-scoped
captures) instead of duplicating only the generic branch.
collectSettleChromeRefs/ANDROID_SYSTEM_CHROME_MARKER_LEAF_IDS,src/core/snapshot-chrome.ts) and the meaningful-target filter(
isMeaningfulDivergenceTarget) are unchanged — they already operate as filters over whatevernodesthey're handed, never as a scoping. The fix was ensuring the capture itself is full-window, not narrowing
or widening these filters.
docs/adr/0012-interactive-replay.mddecision 4'sscreensemantics gain anexplicit amendment paragraph (plus a Status-section entry) stating the invariant:
screen.refsis afiltered digest of the exact same tree
snapshotwould return, and an agent must never see a healthierscreenin a divergence report than a plainsnapshotwould show it.src/core/__tests__/snapshot-chrome-android-statusbar.test.ts's synthetic volume-dialog test used anon-existent
volume_dialog_sliderid — replaced with the real, live-verifiedvolume_new_ringer_active_icon_containerid, and the test's comment now reads explicitly as afilter-logic unit test (isolated
collectSettleChromeRefsexercise), not a live-capture-path claim.src/daemon/handlers/__tests__/session-replay-divergence.test.tsgains atest that feeds
buildReplayFailureDivergencea full Android capture (real non-raw walk) containing asystemui overlay window (volume dialog, real ids) alongside status-bar/IME chrome and app content —
asserting the overlay's actionable ref survives into
screen.refswhile status/nav/IME chrome is stillfiltered and app content is untouched.
Why
captureSnapshotDataand not a deeper native changecaptureDivergenceObservation's previous hand-rolleddispatchCommand('snapshot', ...)call andsnapshot's own capture path already converged on the identical low-level per-platform capture(Android/iOS both ultimately call the same
dispatchCommand('snapshot', ...)under the hood), so this isnot a "genuinely different code path requiring a larger refactor" per the task's stop condition. But the
hand-rolled call bypassed
captureSnapshotData's platform branching entirely for macOS surface-scopedsessions (
isMacOs(device) && session.surface !== 'app'→runMacOsSnapshotAction) and Linux AT-SPIsessions (
snapshotLinux), always falling through to the generic XCTest/Android branch instead — aconcrete, verifiable full-scope gap for those platforms. Routing through
captureSnapshotDatacloses thatgap and removes any possibility of future incidental drift between the two call sites, which is the
literal ask in the issue ("find where
snapshotbuilds its full capture and reuse that scope").I could not reproduce the Android live-investigation numbers from #1264 on a physical/emulator device in
this environment (no Android/iOS runtime available here), so I can't independently confirm the exact
native-layer mechanism behind the "0 systemui/volume nodes" finding. The unification above is the
maintainer-directed fix and is verified at the unit level (new test above); the live acceptance recipe
below re-runs the maintainer's own experiment for confirmation on a real device.
Live acceptance recipe (for the maintainer's device setup — not run in this PR)
openthe test-app, and start areplay --save-script(or any) run that fails while a system overlay is up.adb shell input keyevent KEYCODE_VOLUME_UP) repeatedly across the failing step soVolumeDialogstays up through the capture.The resulting
REPLAY_DIVERGENCE'sdetails.divergence.screen.refsshould include the volume dialog'shittable control(s) (e.g.
volume_new_ringer_active_icon_container), not just app-content refs.adb shell cmd statusbar expand-settings) before and keep it up after the failing step.screen.refsshould include QS'sactionable tiles, not be empty.
clock/battery/status-bar and nav-barentries are absent from
screen.refs(the fix: exclude keyboard/IME chrome from replay divergence screen.refs #1233/fix: Android status/nav-bar systemui chrome leaks into non-raw captures (#1251) #1256 filter still applies on top of the now-fullcapture).
springboard-system-alert-reprorecipe's Android analogue, or any runtime permission dialog coveringthe app) and confirm the dialog's own refs appear in
screen.refsandrepairHintdoes not read theapp underneath as healthy.
This recipe is the same one documented live in #1264's investigation; it requires the maintainer's device
setup (emulator/simulator + snapshot-helper) and is not part of this PR's automated verification.
Test plan
npx tsc --noEmitnpx oxlint --deny-warningsnpx oxfmt --check(no new formatting issues introduced; the 38 files flagged pre-exist onorigin/main)npx vitest runoversnapshot-chrome*,session-replay-divergence*,snapshot-capture*,settle-related,session-replay-target-verification-runtime,replay/__tests__/divergence, andthe full
src/daemon/handlers/__tests__+src/replay+src/core/__tests__suites (105 files /1021 tests, all passing)
npx fallow audit --base origin/main— no issues in the 4 changed filesStatus update (head
9cb3188ab) — the real fix evolved during reviewThe summary above describes the round-1 framing (capture scope). Maintainer review established that #1264's
root cause is cap burial in
screen.refsselection, not capture scope, and flagged two remainingcapture-parity gaps. The branch now carries three layered changes; see the PR comment thread for the full
point-by-point mapping. Net state:
captureSnapshotwrapper asplain
snapshot(full-window scope + Android freshness / post-action retry parity), under a clean,fixed capture-flags policy (full-window, non-raw, default depth) so a failed
--raw/scoped/-dactioncan't narrow the diagnostic tree. (Supersedes the round-1
captureSnapshotDatarouting described above.)foreign-window (non-app-
bundleId) hittable dismiss targets are promoted ahead of app content (stablewithin tiers), so a fully-captured overlay is never buried past the cap. Plus an occlusion fallback: a
mass-covered app surfaces its covered nodes rather than emitting an empty
refs.repairHint/suggestionsconsume the full captured node list, not the capped slice, so hint routing isunaffected. ADR-0012 decision 4's amendment reflects all of the above.
Live overlay acceptance remains a maintainer device step (env down): the unit fixtures prove ref
selection and capture-wrapper parity, not that the Android helper returns the separate-window overlay at
divergence time — see the recipe above.