Skip to content

fix: Android status/nav-bar systemui chrome leaks into non-raw captures (#1251)#1256

Open
thymikee wants to merge 2 commits into
mainfrom
fix/android-statusbar-divergence-leak
Open

fix: Android status/nav-bar systemui chrome leaks into non-raw captures (#1251)#1256
thymikee wants to merge 2 commits into
mainfrom
fix/android-statusbar-divergence-leak

Conversation

@thymikee

Copy link
Copy Markdown
Member

Root cause

walkUiHierarchyNode (src/platforms/android/ui-hierarchy.ts) includes every node when options.raw, but in the non-raw path runs shouldIncludeAndroidNodeshouldIncludeStructuralAndroidNode, which drops unlabeled/unidentified structural nodes and re-parents their children upward.

That silently swallows the com.android.systemui:id/status_bar* / navigation_bar* wrapper nodes — the only nodes hasAndroidSystemChromeMarker (in src/core/snapshot-chrome.ts) matches. A non-raw capture is left with only their labeled/identified leaves (clock, battery, wifi_signal/wifi_combo, mobile_signal/mobile_combo, etc.), whose own resource-ids carry no status_bar/navigation_bar prefix. The systemui run loses its marker, collectAndroidSystemChromeRunIndexes stops dropping it, and status-bar chrome leaks into --settle and the replay-divergence screen.refs cap. --raw keeps the wrapper markers, so it was unaffected.

Fix

ANDROID_SYSTEM_CHROME_MARKER_LEAF_IDS in src/core/snapshot-chrome.ts: a set of stable AOSP status-bar/nav-bar leaf resource-ids (clock, battery, statusIcons, notificationIcons, notification_icon_area, system_icons, cutout_space_view, mobile_signal/mobile_combo/mobile_group, wifi_signal/wifi_combo/wifi_group, start_side_notif_and_chip_container; nav: back, home, recent_apps, home_handle), matched by exact id (not prefix) so actionable systemui overlays (volume dialog, media/output pickers) keep surviving. The existing prefix check stays for --raw, which still carries the wrapper markers.

Deferred alternative noted in a code comment: thread the AOSP window-type constants (TYPE_STATUS_BAR/TYPE_NAVIGATION_BAR, already parsed in readNodeAttributes but not copied to the output node) through to the snapshot node and key off that instead of resource-ids — left for later unless the id-based approach proves insufficient.

Test

src/core/__tests__/snapshot-chrome-android-statusbar.test.ts derives its fixture from a real --raw Android capture (checkout-form fixture app, Gboard open, status bar visible). A simulateNonRawWalk helper reproduces the walk's drop+reparent for the specific status_bar*/navigation_bar* marker wrappers (verified node-for-node against the source capture), producing a realistic "leaves only" tree. Assertions:

  • every surviving status-bar leaf is classified as chrome (fails without the fix — confirmed red/green locally)
  • the whole systemui run drops as a unit
  • app fields (field-name/field-email/field-phone) and the IME keyboard, from the same real capture, are handled unchanged
  • a synthetic volume-dialog run (no real capture of that surface was available) still survives — guards against broadening the id set to "any systemui id"
  • a synthetic nav-bar case (the real capture's test device has no nav bar — gesture nav) covers back/home/recent_apps/home_handle

Verification

  • npx tsc --noEmit — clean
  • npx oxlint --deny-warnings / npx oxfmt --check on touched files — clean
  • Full npx vitest run — 4130 tests, all pass on a clean re-run after pnpm install in the fresh worktree (an initial run without deps installed hit an unrelated build-tooling failure; a couple of runs also hit known-flaky iOS runner-client/runtime-hints tests under parallel contention, all confirmed passing in isolation — see unit-suite-flaky-under-contention.md)
  • Targeted re-run of Android + core + settle + replay + daemon-handlers suites together: 155 files / 1482 tests, all green
  • pnpm check:fallow --base origin/main — no issues in the 2 changed files

Fixes #1251.

)

The non-raw Android walk (walkUiHierarchyNode/shouldIncludeStructuralAndroidNode
in ui-hierarchy.ts) drops unlabeled/unidentified structural nodes, re-parenting
their children upward. That silently swallows the status_bar*/navigation_bar*
WRAPPER nodes carrying the marker ids collectAndroidSystemChromeRunIndexes keys
on, leaving only their labeled/identified LEAVES (clock, battery, wifi/mobile
icons, nav buttons) in a non-raw capture. Those leaves' own ids have no
status_bar/navigation_bar prefix, so the systemui run loses its marker and is
no longer dropped -- leaking status-bar chrome into --settle and replay
divergence screen.refs. --raw keeps the wrapper markers, so it was unaffected.

Recognize the surviving leaves directly by EXACT resource-id (not prefix, to
stay tight -- actionable systemui overlays like the volume dialog or a media
picker must keep surviving).

Test derives a faithful non-raw tree from a real --raw Android capture
(checkout-form fixture app, Gboard + status bar) by simulating the walk's
drop+reparent for the specific marker-bearing wrappers, then asserts: every
surviving status-bar leaf is classified as chrome, the whole systemui run
drops, app fields and the IME keyboard are handled unchanged, and a synthetic
volume-dialog run still survives. A second synthetic case covers the nav-bar
leaves (no real nav-bar capture was available on the gesture-nav test device).
@github-actions

github-actions Bot commented Jul 13, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 1.7 MB 1.7 MB +401 B
JS gzip 542.7 kB 542.9 kB +169 B
npm tarball 653.4 kB 653.5 kB +176 B
npm unpacked 2.3 MB 2.3 MB +401 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 29.3 ms 29.2 ms -0.1 ms
CLI --help 60.0 ms 59.2 ms -0.8 ms

Top changed chunks:

Chunk Raw diff Gzip diff
dist/src/tv-remote.js +329 B +143 B
dist/src/session.js +72 B +26 B

@thymikee

thymikee commented Jul 13, 2026

Copy link
Copy Markdown
Member Author

Code review on bfc00a11 found the exact-ID classifier change narrowly scoped, and the new core regression fails under the old behavior. Readiness is still blocked on practical Android evidence for the actual path fixed by #1251: run a post-fix replay/target-binding divergence on an emulator and show that non-raw screen.refs excludes status/nav chrome before the 20-ref cap while retaining actionable app controls and an actionable SystemUI overlay/dialog. The PR currently provides fixture/classifier evidence only, so it does not yet prove the live capture-to-bounded-refs workflow.

…led structural nodes

The get/is/wait divergence uses a full (non-interactive) capture so static-text
targets survive, but that also pulls in unlabeled structural containers
(ViewGroups/ComposeViews) that carry a ref yet no identity and aren't tappable.
On deeply-nested RN trees they consume the SCREEN_REF_CAPTURE_LIMIT budget ahead
of the actionable controls (and the app content the excluded status/nav chrome
just freed room for). Filter divergence screen.refs to nodes an agent could
actually re-target: identifiable (display label/value/non-generic id) or
interactive (hittable).
@thymikee

Copy link
Copy Markdown
Member Author

Additional blocking test-validity finding on 8fc549988:

The Android “non-raw” fixture does not reproduce the production walker closely enough for the nav-bar assertion. simulateNonRawWalk removes only status_bar*/navigation_bar* wrappers, while production also drops structural/visual nodes whose generic resource ID is their only identity unless they are hittable or have a hittable descendant. For example, the synthetic unlabeled, non-hittable android.view.View with com.android.systemui:id/home_handle is retained by the test and asserted as chrome, but production shouldIncludeStructuralAndroidNode would drop it before the chrome classifier runs.

Please run the fixture through the real Android inclusion/walk behavior (or reproduce all of its relevant decisions exactly) and test the Android target-binding divergence route rather than an iOS session with mocked dispatch. The previously requested practical emulator evidence for the actual non-raw capture-to-bounded-screen.refs path is also still absent.

Readiness remains withheld. No fixer was dispatched.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Android status-bar systemui chrome leaks into replay divergence screen.refs (non-raw capture drops status_bar resource-id markers)

1 participant