From 44050b47c68aff8318223d4bf9937554edff31a2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Wed, 15 Jul 2026 19:29:28 +0200 Subject: [PATCH 1/3] fix(replay): demote non-unique ids from writer identity/selector chain (#1269) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Android list-row GET replays bind the wrong row because the recorder uses the non-unique framework resource id `android:id/title` (matchCount 11 on Settings root) as primary identity; positional drift then makes the identity verifier correctly refuse with `identity-mismatch`. Demote an id from identity whenever it matches more than one node in the record-time tree (capture-time uniqueness, not an `android:id/*` namespace check — a reused RN FlatList testID hits the same class on iOS). Applied in both places a recorded id feeds identity: - `computeTargetEvidence` (session-target-evidence.ts): the `target-v1` identity tuple falls back to role+label when the id's own capture-time match count exceeds one, reusing the existing `filterIdentitySet` domain machinery (an empty ancestry degrades it to a plain id scan). - `buildSelectorChainForNode` (selectors/build.ts): the recorded selector chain omits a non-unique id rather than leading with it. Every writer call site (get/press/fill recording, plus the divergence-suggestion path) now passes the record-time tree so the check has something to count against; omitting it preserves prior behavior for isolated-node callers (tests). Resolver-side `resolveSelectorChain` and live press/fill resolution are untouched per ADR 0012 (disclosed-not-changed disambiguation) — this is writer/replay-scoped only. Amends ADR 0012 decision 3: an id may serve as identity (and lead the selector chain) only when it uniquely denotes the target in the record-time tree. Adds fixtures: an Android duplicated-`android:id/title` list (the measured repro) and an iOS/RN duplicated-testID FlatList shape, both demoted and still verifying via the now-selective label; a regression case confirming an already-unique id is unaffected. Out of scope: the Android list-*press* class (matchCount 12, label-less `role="linearlayout"` container with no id at all to demote) needs a separate design decision — deriving identity from the labeled descendant. Tracked as a follow-up, not attempted here. --- docs/adr/0012-interactive-replay.md | 13 ++ .../interaction/runtime/resolution.ts | 1 + .../interaction/runtime/selector-read.ts | 2 + .../__tests__/session-target-evidence.test.ts | 123 +++++++++++++++ .../handlers/session-replay-divergence.ts | 1 + src/daemon/session-target-evidence.ts | 30 +++- src/selectors/build.test.ts | 147 ++++++++++++++++++ src/selectors/build.ts | 30 +++- 8 files changed, 344 insertions(+), 3 deletions(-) create mode 100644 src/selectors/build.test.ts diff --git a/docs/adr/0012-interactive-replay.md b/docs/adr/0012-interactive-replay.md index d43bb3841..fb8e26ed3 100644 --- a/docs/adr/0012-interactive-replay.md +++ b/docs/adr/0012-interactive-replay.md @@ -19,6 +19,11 @@ Accepted (2026-07-10); partially implemented (last updated 2026-07-13). See [Mig filters, not scopings; and `screen.refs` is ranked within the byte cap (foreign-window dismiss targets ahead of app content; mass-covered app nodes surfaced rather than emptied) instead of sliced in document order, so a captured overlay is never buried past the cap (#1264). +- Decision 3 amendment, id demotion under non-unique capture-time match — a recorded id no longer serves + as identity, or leads the selector chain, unless it is unique in the record-time tree; a non-unique id + (a shared Android framework resource id such as `android:id/title`, or a reused RN `FlatList` `testID`) + falls back to role+label in both `computeTargetEvidence`'s `target-v1` tuple and + `buildSelectorChainForNode`'s chain (#1269). **Accepted but NOT yet implemented** (this amendment; tracked by #1235 — repair-transaction lifecycle): the R7 repair-transaction keep-alive and its distinct `resume.repairSessionHeld` signal, the ARMED → @@ -316,6 +321,14 @@ or, when the recording carries no `id`, when their normalized roles are equal an are equal (label absent on both sides counts as equal; label present on exactly one side is a mismatch). A recorded `id` never matches a node without that id. +> **Amendment (#1269).** An id may serve as identity (and lead the selector chain) only when it uniquely +> denotes the target in the record-time tree: the writer computes the id's own capture-time match count +> (independent of ancestry) and, whenever more than one node in the record-time tree carries the recorded +> id, demotes it — falling back to role+label exactly as an unrecorded id already does. A shared +> Android framework resource id (`android:id/title`, present on every titled list row) is the measured +> case, but the rule is capture-time uniqueness, not an `android:id/*` namespace heuristic: a reused RN +> `FlatList` `testID` hits the same demotion on iOS. + **Ancestry.** The chain is the nearest **K = 8** ancestors of the target, ordered **leaf→root** (nearest ancestor first), each entry `{ role, label? }` under the same normalization (`role` may be the empty string when the node has no type; `label` is omitted when empty). Truncation drops entries from the diff --git a/src/commands/interaction/runtime/resolution.ts b/src/commands/interaction/runtime/resolution.ts index 0c4fa85e4..54b900bdf 100644 --- a/src/commands/interaction/runtime/resolution.ts +++ b/src/commands/interaction/runtime/resolution.ts @@ -403,6 +403,7 @@ function describeResolvedInteractionNode( node, selectorChain: buildSelectorChainForNode(node, runtime.backend.platform, { action: action === 'fill' ? 'fill' : 'click', + nodes, }), refLabel: resolveRefLabel(node, nodes), ...describeNonHittableTarget(node, action), diff --git a/src/commands/interaction/runtime/selector-read.ts b/src/commands/interaction/runtime/selector-read.ts index a7d6696ee..1d90a3dfe 100644 --- a/src/commands/interaction/runtime/selector-read.ts +++ b/src/commands/interaction/runtime/selector-read.ts @@ -210,6 +210,7 @@ export const getCommand: RuntimeCommand = a ); const selectorChain = buildSelectorChainForNode(resolved.node, runtime.backend.platform, { action: 'get', + nodes: capture.snapshot.nodes, }); const target = { kind: 'ref' as const, ref: `@${resolved.ref}` }; const preActionNodes = capture.snapshot.nodes; @@ -233,6 +234,7 @@ export const getCommand: RuntimeCommand = a const selectorChain = buildSelectorChainForNode(resolved.node, runtime.backend.platform, { action: 'get', + nodes: resolved.capture.snapshot.nodes, }); if (options.property === 'attrs') { diff --git a/src/daemon/__tests__/session-target-evidence.test.ts b/src/daemon/__tests__/session-target-evidence.test.ts index 81d278db7..9717a284c 100644 --- a/src/daemon/__tests__/session-target-evidence.test.ts +++ b/src/daemon/__tests__/session-target-evidence.test.ts @@ -466,3 +466,126 @@ test('computeTargetEvidence: a parent cycle records unverifiable, never verified assert.ok(evidence); assert.equal(evidence.verification, 'unverifiable'); }); + +// --------------------------------------------------------------------------- +// #1269: ADR 0012 decision 3 amendment — an id is identity only when it +// uniquely denotes the target in the record-time tree. A shared framework +// resource id (Android's `android:id/title`, present on every titled list +// row) is not selective: the recorded ancestry is label-less generic +// containers (linearlayout/recyclerview/framelayout) contributing no +// disambiguation, so an id-led identity set spans every row and replay binds +// the wrong one under positional drift. The fix demotes the id back to +// role+label whenever it matches more than one node at record time — the +// same fallback an unrecorded id already uses. +// --------------------------------------------------------------------------- + +function androidSettingsListFixture(): SnapshotNode[] { + return toSnapshotNodes([ + { index: 0, type: 'FrameLayout', depth: 0 }, + { index: 1, type: 'RecyclerView', depth: 1, parentIndex: 0 }, + // Each row is a label-less LinearLayout wrapper (real Android list + // shape) whose title TextView shares one framework resource id. + { index: 2, type: 'LinearLayout', depth: 2, parentIndex: 1 }, + { + index: 3, + type: 'TextView', + identifier: 'android:id/title', + label: 'Network & internet', + rect: { x: 0, y: 100, width: 300, height: 48 }, + depth: 3, + parentIndex: 2, + }, + { index: 4, type: 'LinearLayout', depth: 2, parentIndex: 1 }, + { + index: 5, + type: 'TextView', + identifier: 'android:id/title', + label: 'Connected devices', + rect: { x: 0, y: 148, width: 300, height: 48 }, + depth: 3, + parentIndex: 4, + }, + { index: 6, type: 'LinearLayout', depth: 2, parentIndex: 1 }, + { + index: 7, + type: 'TextView', + identifier: 'android:id/title', + label: 'Apps', + rect: { x: 0, y: 196, width: 300, height: 48 }, + depth: 3, + parentIndex: 6, + }, + ]); +} + +test('computeTargetEvidence: a shared Android framework id (matchCount 3 > 1) is demoted to role+label, and still verifies via the now-selective label', () => { + const nodes = androidSettingsListFixture(); + const winner = findByLabel(nodes, 'Network & internet'); + const evidence = computeTargetEvidence({ node: winner, preActionNodes: nodes }); + assert.ok(evidence); + assert.equal( + evidence.id, + undefined, + 'the non-unique android:id/title must not be recorded as identity', + ); + assert.equal(evidence.role, 'textview'); + assert.equal(evidence.label, 'Network & internet'); + assert.deepEqual(evidence.ancestry[0], { role: 'linearlayout' }); + // With the id demoted, role+label uniquely isolates this row among the + // three sharing the id, so the record-time self-check still verifies — + // this is the measured "get-only stripped-id: 20/20 clean" fix. + assert.equal(evidence.verification, 'verified'); +}); + +test('computeTargetEvidence: an RN FlatList reusing one testID across rows (iOS/RN, matchCount > 1) is demoted the same way', () => { + // The class is not Android-specific: an RN `FlatList` `renderItem` that + // assigns the SAME testID to every row reproduces one shared accessibility + // identifier across siblings on iOS too. Capture-time uniqueness — not an + // `android:id/*` namespace check — is the rule. + const nodes = toSnapshotNodes([ + { index: 0, type: 'Application', depth: 0 }, + { index: 1, type: 'ScrollView', identifier: 'contacts-list', depth: 1, parentIndex: 0 }, + { + index: 2, + type: 'Cell', + identifier: 'contact-row', + label: 'Ada Lovelace', + rect: { x: 0, y: 0, width: 320, height: 60 }, + depth: 2, + parentIndex: 1, + }, + { + index: 3, + type: 'Cell', + identifier: 'contact-row', + label: 'Grace Hopper', + rect: { x: 0, y: 60, width: 320, height: 60 }, + depth: 2, + parentIndex: 1, + }, + { + index: 4, + type: 'Cell', + identifier: 'contact-row', + label: 'Katherine Johnson', + rect: { x: 0, y: 120, width: 320, height: 60 }, + depth: 2, + parentIndex: 1, + }, + ]); + const winner = findByLabel(nodes, 'Grace Hopper'); + const evidence = computeTargetEvidence({ node: winner, preActionNodes: nodes }); + assert.ok(evidence); + assert.equal(evidence.id, undefined, 'the shared testID must not be recorded as identity'); + assert.equal(evidence.role, 'cell'); + assert.equal(evidence.label, 'Grace Hopper'); + assert.equal(evidence.verification, 'verified'); +}); + +test('computeTargetEvidence: a unique id is still recorded as identity (already-clean case unchanged)', () => { + const nodes = toolbarFixture(); + const winner = findByLabel(nodes, 'Save'); + const evidence = computeTargetEvidence({ node: winner, preActionNodes: nodes }); + assert.ok(evidence); + assert.equal(evidence.id, 'save'); +}); diff --git a/src/daemon/handlers/session-replay-divergence.ts b/src/daemon/handlers/session-replay-divergence.ts index 59e995e33..6adc3b23b 100644 --- a/src/daemon/handlers/session-replay-divergence.ts +++ b/src/daemon/handlers/session-replay-divergence.ts @@ -557,6 +557,7 @@ export function buildReplayDivergenceSuggestionForNode(params: { const selectorChain = buildSelectorChainForNode(node, session.device.platform, { action: action.command === 'fill' ? 'fill' : isTouchTargetCommand(action.command) ? 'click' : 'get', + nodes, }); const role = formatRole(node.type ?? 'Element'); const label = displayLabel(node, role); diff --git a/src/daemon/session-target-evidence.ts b/src/daemon/session-target-evidence.ts index ebcc771e0..f6f8cf470 100644 --- a/src/daemon/session-target-evidence.ts +++ b/src/daemon/session-target-evidence.ts @@ -45,7 +45,7 @@ export function computeTargetEvidence( const { node, preActionNodes: nodes } = capture; if (typeof node.index !== 'number') return undefined; const byIndex = buildIndexMap(nodes); - const identity = boundedLocalIdentity(node); + const identity = demoteNonUniqueId(boundedLocalIdentity(node), nodes, byIndex); const ancestryWalk = buildAncestryChain(node, byIndex, TARGET_ANNOTATION_MAX_ANCESTRY); const fullAncestry = ancestryWalk.chain; const sibling = computeSiblingOrdinal(nodes, node); @@ -237,6 +237,34 @@ export function filterIdentitySet( }); } +/** + * ADR 0012 decision 3 amendment (#1269): an id is identity only when it + * uniquely denotes the target in the record-time tree. `boundedLocalIdentity` + * reads a node's id unconditionally, but a shared framework resource id + * (Android's `android:id/title` matching every list row is the measured + * case — #1269) is not selective: on replay the id-led identity set spans + * every row, position drifts, and verification correctly refuses a + * confident bind. Passing an empty `ancestry` to `filterIdentitySet` + * degrades its ancestry-prefix check to vacuously-true, so it returns + * exactly the nodes sharing this id anywhere in the tree — the id's own + * capture-time match count, independent of structural context. When that + * count is greater than one, fall back to role+label, exactly the identity + * an unrecorded id already computes. The rule is capture-time uniqueness, + * not an id-namespace heuristic: a reused RN `FlatList` `testID` hits the + * same demotion on iOS. + */ +function demoteNonUniqueId( + identity: LocalIdentity, + nodes: readonly SnapshotNode[], + byIndex: Map, +): LocalIdentity { + if (identity.id === undefined) return identity; + const idMatchCount = filterIdentitySet(nodes, byIndex, identity, []).length; + if (idMatchCount <= 1) return identity; + const { role, label } = identity; + return { role, ...(label !== undefined ? { label } : {}) }; +} + function computeDisambiguationDomain(params: { nodes: readonly SnapshotNode[]; byIndex: Map; diff --git a/src/selectors/build.test.ts b/src/selectors/build.test.ts new file mode 100644 index 000000000..d383280c3 --- /dev/null +++ b/src/selectors/build.test.ts @@ -0,0 +1,147 @@ +import { test } from 'vitest'; +import assert from 'node:assert/strict'; +import type { SnapshotNode } from '../kernel/snapshot.ts'; +import { buildNodes } from '../__tests__/test-utils/snapshot-builders.ts'; +import { buildSelectorChainForNode } from './build.ts'; + +function findByLabel(nodes: SnapshotNode[], label: string): SnapshotNode { + const found = nodes.find((node) => node.label === label); + if (!found) throw new Error(`fixture missing node with label ${label}`); + return found; +} + +// --------------------------------------------------------------------------- +// #1269: a recorded id only leads the chain when it is unique in the +// record-time tree. `android:id/title` — a shared Android framework +// resource id present on every titled list row — is the measured repro +// shape; matchCount reflects how many nodes in `nodes` carry the id. +// --------------------------------------------------------------------------- + +function androidSettingsListFixture(): SnapshotNode[] { + return buildNodes([ + { index: 0, type: 'FrameLayout', depth: 0 }, + { index: 1, type: 'RecyclerView', depth: 1, parentIndex: 0 }, + // Each row is a label-less LinearLayout wrapper (real Android list shape) + // whose title TextView shares the same framework resource id. + { index: 2, type: 'LinearLayout', depth: 2, parentIndex: 1 }, + { + index: 3, + type: 'TextView', + identifier: 'android:id/title', + label: 'Network & internet', + rect: { x: 0, y: 100, width: 300, height: 48 }, + depth: 3, + parentIndex: 2, + }, + { index: 4, type: 'LinearLayout', depth: 2, parentIndex: 1 }, + { + index: 5, + type: 'TextView', + identifier: 'android:id/title', + label: 'Connected devices', + rect: { x: 0, y: 148, width: 300, height: 48 }, + depth: 3, + parentIndex: 4, + }, + { index: 6, type: 'LinearLayout', depth: 2, parentIndex: 1 }, + { + index: 7, + type: 'TextView', + identifier: 'android:id/title', + label: 'Apps', + rect: { x: 0, y: 196, width: 300, height: 48 }, + depth: 3, + parentIndex: 6, + }, + ]); +} + +test('buildSelectorChainForNode: a shared Android framework id (matchCount > 1) is demoted — chain leads with role+label, not id', () => { + const nodes = androidSettingsListFixture(); + const row = findByLabel(nodes, 'Network & internet'); + const chain = buildSelectorChainForNode(row, 'android', { action: 'get', nodes }); + assert.deepEqual(chain, [ + 'role="textview" label="Network & internet"', + 'label="Network & internet"', + ]); + assert.ok( + !chain.some((entry) => entry.startsWith('id=')), + 'a non-unique id must never lead (or appear in) the recorded selector chain', + ); +}); + +test('buildSelectorChainForNode: an RN FlatList reusing one testID across rows (iOS/RN, matchCount > 1) is demoted the same way', () => { + // RN's common FlatList antipattern: `renderItem` assigns the SAME testID to + // every row. On iOS this surfaces as one accessibility identifier shared by + // every cell — the same non-selective-id class as the Android framework + // resource id above, reproduced on a different platform to prove the rule + // is capture-time uniqueness, not an `android:id/*` namespace check. + const nodes = buildNodes([ + { index: 0, type: 'Application', depth: 0 }, + { index: 1, type: 'ScrollView', identifier: 'contacts-list', depth: 1, parentIndex: 0 }, + { + index: 2, + type: 'Cell', + identifier: 'contact-row', + label: 'Ada Lovelace', + rect: { x: 0, y: 0, width: 320, height: 60 }, + depth: 2, + parentIndex: 1, + }, + { + index: 3, + type: 'Cell', + identifier: 'contact-row', + label: 'Grace Hopper', + rect: { x: 0, y: 60, width: 320, height: 60 }, + depth: 2, + parentIndex: 1, + }, + { + index: 4, + type: 'Cell', + identifier: 'contact-row', + label: 'Katherine Johnson', + rect: { x: 0, y: 120, width: 320, height: 60 }, + depth: 2, + parentIndex: 1, + }, + ]); + const row = findByLabel(nodes, 'Grace Hopper'); + const chain = buildSelectorChainForNode(row, 'ios', { action: 'get', nodes }); + assert.deepEqual(chain, ['role="cell" label="Grace Hopper"', 'label="Grace Hopper"']); + assert.ok(!chain.some((entry) => entry.startsWith('id='))); +}); + +test('buildSelectorChainForNode: a unique id still leads the chain (existing already-clean behavior is preserved)', () => { + const nodes = buildNodes([ + { index: 0, type: 'Window', depth: 0 }, + { + index: 1, + type: 'Button', + identifier: 'save', + label: 'Save', + rect: { x: 10, y: 10, width: 40, height: 20 }, + depth: 1, + parentIndex: 0, + }, + ]); + const node = findByLabel(nodes, 'Save'); + const chain = buildSelectorChainForNode(node, 'ios', { action: 'get', nodes }); + assert.deepEqual(chain, ['id="save"', 'role="button" label="Save"', 'label="Save"']); +}); + +test('buildSelectorChainForNode: without a record-time tree, an id is trusted as-is (back-compat default)', () => { + const nodes = buildNodes([ + { + index: 0, + type: 'Button', + identifier: 'save', + label: 'Save', + rect: { x: 10, y: 10, width: 40, height: 20 }, + depth: 0, + }, + ]); + const chain = buildSelectorChainForNode(nodes[0]!, 'ios', { action: 'get' }); + assert.deepEqual(chain, ['id="save"', 'role="button" label="Save"', 'label="Save"']); +}); diff --git a/src/selectors/build.ts b/src/selectors/build.ts index 2511b2ad0..200274df7 100644 --- a/src/selectors/build.ts +++ b/src/selectors/build.ts @@ -6,11 +6,11 @@ import { extractNodeText, normalizeType } from '../snapshot/snapshot-processing. export function buildSelectorChainForNode( node: SnapshotNode, _platform: Platform | PublicPlatform, - options: { action?: 'click' | 'fill' | 'get' } = {}, + options: { action?: 'click' | 'fill' | 'get'; nodes?: readonly SnapshotNode[] } = {}, ): string[] { const chain: string[] = []; const role = normalizeType(node.type ?? ''); - const id = normalizeSelectorText(node.identifier); + const id = selectableId(node, options.nodes); const label = normalizeSelectorText(node.label); const value = normalizeSelectorText(node.value); const text = normalizeSelectorText(extractNodeText(node)); @@ -66,6 +66,32 @@ export function buildSelectorChainForNode( return deduped; } +/** + * ADR 0012 decision 3 amendment (#1269): an id may lead the selector chain + * only when it uniquely denotes the node in the record-time tree it was + * captured from — a shared framework resource id (Android's + * `android:id/title` matching every list row is the measured case) resolves + * the wrong element under positional drift on replay. The rule is + * capture-time uniqueness, not an id-namespace heuristic: a reused RN + * `FlatList` `testID` hits the same demotion. `nodes` is the record-time + * tree the node was captured from; every writer call site passes it. When + * absent (e.g. a node built in isolation with no tree to check against) the + * id is trusted as-is. + */ +function selectableId( + node: SnapshotNode, + nodes: readonly SnapshotNode[] | undefined, +): string | null { + const id = normalizeSelectorText(node.identifier); + if (!id || !nodes) return id; + let matches = 0; + for (const candidate of nodes) { + if (normalizeSelectorText(candidate.identifier) === id) matches += 1; + if (matches > 1) return null; + } + return id; +} + function uniqueStrings(values: readonly string[]): string[] { return Array.from(new Set(values)); } From 80c1b3e9f51b123fb845d72651f80e258b8418ad Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Wed, 15 Jul 2026 21:41:27 +0200 Subject: [PATCH 2/3] fix(replay): unify the id-uniqueness predicate across both writer sites (#1269 review) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Address the maintainer review on #1272: 1. ONE shared uniqueness predicate. The two demotion sites were counting id matches with DIFFERENT semantics — `demoteNonUniqueId` via `filterIdentitySet` (NFC + 256-byte cap, and a broken-parent-walk exclusion), `selectableId` via a raw `normalizeSelectorText` scan (trim, no NFC/cap, no exclusion) — so the identity tuple and the selector chain could disagree and half-demote (id gone from one, kept in the other). Extract `idMatchCountInTree(nodes, id)` in target-identity-node.ts, counting over the canonical `readNodeLocalIdentity` id the replay verifier keys on, with no ancestry/parent-walk exclusion. Both `demoteNonUniqueId` and `selectableId` now call it. Corrects the inaccurate "vacuously-true / plain id scan" comment. Cross-invariant test (build.test.ts): for the same node+tree, evidence.id === undefined iff the built chain has no id= clause — across demoted, unique, and a non-NFC (decomposed vs precomposed) edge case. Verified it fails under the old raw-scan and passes under the unified predicate. 2. End-to-end reorder proof (session-replay-target-classification.test.ts): record against a tree whose rows share android:id/title, then classify against a DIFFERENT replay tree where the shared-id rows reorder — the demoted role+label identity rebinds the correct row (verified, matchCount 1) while `id="android:id/title"` resolves ambiguously (null). This pins the FDR 1.0 -> 0 mechanism, not just record-time demotion. 3. Removed the conflated "20/20 clean" live-number comment from the unit test; it now states the mechanism (role+label selectivity) instead. Behavior for the already-clean unique-id path is unchanged: for ordinary ascii ids the canonical count equals the old raw count. The only outcomes that change are the edge cases the old split mishandled (non-NFC, broken parent walk) — where demotion is the correct result. The kept clause still emits the chain's own normalizeSelectorText id string, so unique ids lead the chain exactly as before. --- .../__tests__/session-target-evidence.test.ts | 4 +- ...ssion-replay-target-classification.test.ts | 99 +++++++++++++- src/daemon/session-target-evidence.ts | 33 +++-- src/replay/target-identity-node.ts | 27 ++++ src/selectors/build.test.ts | 127 ++++++++++++++++++ src/selectors/build.ts | 29 ++-- 6 files changed, 289 insertions(+), 30 deletions(-) diff --git a/src/daemon/__tests__/session-target-evidence.test.ts b/src/daemon/__tests__/session-target-evidence.test.ts index 9717a284c..01d7986e7 100644 --- a/src/daemon/__tests__/session-target-evidence.test.ts +++ b/src/daemon/__tests__/session-target-evidence.test.ts @@ -532,8 +532,8 @@ test('computeTargetEvidence: a shared Android framework id (matchCount 3 > 1) is assert.equal(evidence.label, 'Network & internet'); assert.deepEqual(evidence.ancestry[0], { role: 'linearlayout' }); // With the id demoted, role+label uniquely isolates this row among the - // three sharing the id, so the record-time self-check still verifies — - // this is the measured "get-only stripped-id: 20/20 clean" fix. + // three sharing the id, so the record-time self-check still verifies via + // the now-selective label rather than the non-selective id. assert.equal(evidence.verification, 'verified'); }); diff --git a/src/daemon/handlers/__tests__/session-replay-target-classification.test.ts b/src/daemon/handlers/__tests__/session-replay-target-classification.test.ts index 69e96e8f1..c65e97121 100644 --- a/src/daemon/handlers/__tests__/session-replay-target-classification.test.ts +++ b/src/daemon/handlers/__tests__/session-replay-target-classification.test.ts @@ -1,7 +1,10 @@ import { test } from 'vitest'; import assert from 'node:assert/strict'; -import type { SnapshotNode } from '../../../kernel/snapshot.ts'; +import type { RawSnapshotNode, SnapshotNode } from '../../../kernel/snapshot.ts'; import type { TargetAnnotationV1 } from '../../../replay/target-identity.ts'; +import { computeTargetEvidence } from '../../session-target-evidence.ts'; +import { buildSelectorChainForNode } from '../../../selectors/build.ts'; +import { parseSelectorChain, resolveSelectorChain } from '../../../selectors/index.ts'; import { classifyReplayTarget } from '../session-replay-target-classification.ts'; import { bottomTabsRealCaptureFixture, @@ -464,3 +467,97 @@ test('classifyReplayTarget: document-order determinism for equal rect centers', }); // --------------------------------------------------------------------------- +// #1269 end-to-end mechanism (Ask 2): the load-bearing proof that id-demotion +// makes replay clean — not just that the id is dropped at record time. Record +// against a tree whose rows share `android:id/title`, then replay against a +// DIFFERENT tree where those shared-id rows REORDER (positions drift). The +// demoted role+label identity binds the correct row; the id path would not. +// --------------------------------------------------------------------------- + +function androidSharedIdListFixture( + rows: { id: string; label: string; y: number }[], +): SnapshotNode[] { + const raw: RawSnapshotNode[] = [ + { index: 0, type: 'FrameLayout', depth: 0 }, + { index: 1, type: 'RecyclerView', depth: 1, parentIndex: 0 }, + ]; + rows.forEach((row, position) => { + const wrapperIndex = 2 + position * 2; + const titleIndex = wrapperIndex + 1; + raw.push({ index: wrapperIndex, type: 'LinearLayout', depth: 2, parentIndex: 1 }); + raw.push({ + index: titleIndex, + type: 'TextView', + identifier: row.id, + label: row.label, + rect: { x: 0, y: row.y, width: 300, height: 48 }, + depth: 3, + parentIndex: wrapperIndex, + }); + }); + return toSnapshotNodes(raw); +} + +test('#1269 e2e: a demoted shared-id row rebinds by role+label after the shared-id rows reorder on replay', () => { + const ANDROID = 'android' as const; + // Record-time Settings root: three rows all carrying the framework id. + const recordNodes = androidSharedIdListFixture([ + { id: 'android:id/title', label: 'Network & internet', y: 100 }, + { id: 'android:id/title', label: 'Connected devices', y: 148 }, + { id: 'android:id/title', label: 'Apps', y: 196 }, + ]); + const target = recordNodes.find((node) => node.label === 'Connected devices')!; + + // The writer demotes the non-unique id in BOTH the tuple and the chain. + const recorded = computeTargetEvidence({ node: target, preActionNodes: recordNodes })!; + assert.equal(recorded.id, undefined); + assert.equal(recorded.verification, 'verified'); + const chain = buildSelectorChainForNode(target, ANDROID, { action: 'get', nodes: recordNodes }); + assert.ok(!chain.some((entry) => entry.startsWith('id='))); + const token = chain.join(' || '); // the recorded selector positional the replay loop re-resolves + + // Replay-time tree: a new conditional row appears at the top and the + // shared-id rows are in a DIFFERENT document/viewport order. The recorded + // row is now third, at a drifted position. + const replayNodes = androidSharedIdListFixture([ + { id: 'android:id/title', label: 'Wi-Fi', y: 100 }, + { id: 'android:id/title', label: 'Apps', y: 148 }, + { id: 'android:id/title', label: 'Connected devices', y: 196 }, + { id: 'android:id/title', label: 'Network & internet', y: 244 }, + ]); + const expected = replayNodes.find((node) => node.label === 'Connected devices')!; + + const result = classifyReplayTarget({ + recorded, + token, + nodes: replayNodes, + platform: ANDROID, + refLabel: undefined, + requireRect: true, + allowDisambiguation: true, + }); + assertVerified(result, { winnerRef: expected.ref, matchCount: 1 }); + + // Contrast — the reorder is genuinely hostile to the id path: the shared id + // matches all four rows (no unique bind → resolution refuses), while the + // demoted role+label resolves the correct row uniquely. This is the + // FDR 1.0 → 0 difference the demotion buys. + const idResolved = resolveSelectorChain( + replayNodes, + parseSelectorChain('id="android:id/title"'), + { + platform: ANDROID, + requireRect: true, + requireUnique: true, + }, + ); + assert.equal(idResolved, null); // non-unique: refuses to bind + const labelResolved = resolveSelectorChain( + replayNodes, + parseSelectorChain('role="textview" label="Connected devices"'), + { platform: ANDROID, requireRect: true, requireUnique: true }, + ); + assert.equal(labelResolved?.node.ref, expected.ref); +}); + +// --------------------------------------------------------------------------- diff --git a/src/daemon/session-target-evidence.ts b/src/daemon/session-target-evidence.ts index f6f8cf470..1fd01f199 100644 --- a/src/daemon/session-target-evidence.ts +++ b/src/daemon/session-target-evidence.ts @@ -17,7 +17,11 @@ import type { SnapshotNode } from '../kernel/snapshot.ts'; import { resolveRectCenter } from '../utils/rect-center.ts'; import { findNearestScrollableContainer } from './snapshot-presentation/tree.ts'; -import { readNodeLocalIdentity, siblingOrdinal } from '../replay/target-identity-node.ts'; +import { + idMatchCountInTree, + readNodeLocalIdentity, + siblingOrdinal, +} from '../replay/target-identity-node.ts'; import { classifyTargetBindingMatch, matchesAncestryPrefix, @@ -45,7 +49,7 @@ export function computeTargetEvidence( const { node, preActionNodes: nodes } = capture; if (typeof node.index !== 'number') return undefined; const byIndex = buildIndexMap(nodes); - const identity = demoteNonUniqueId(boundedLocalIdentity(node), nodes, byIndex); + const identity = demoteNonUniqueId(boundedLocalIdentity(node), nodes); const ancestryWalk = buildAncestryChain(node, byIndex, TARGET_ANNOTATION_MAX_ANCESTRY); const fullAncestry = ancestryWalk.chain; const sibling = computeSiblingOrdinal(nodes, node); @@ -244,23 +248,18 @@ export function filterIdentitySet( * (Android's `android:id/title` matching every list row is the measured * case — #1269) is not selective: on replay the id-led identity set spans * every row, position drifts, and verification correctly refuses a - * confident bind. Passing an empty `ancestry` to `filterIdentitySet` - * degrades its ancestry-prefix check to vacuously-true, so it returns - * exactly the nodes sharing this id anywhere in the tree — the id's own - * capture-time match count, independent of structural context. When that - * count is greater than one, fall back to role+label, exactly the identity - * an unrecorded id already computes. The rule is capture-time uniqueness, - * not an id-namespace heuristic: a reused RN `FlatList` `testID` hits the - * same demotion on iOS. + * confident bind. `idMatchCountInTree` — the SAME predicate + * `buildSelectorChainForNode` uses for the selector chain — counts nodes + * sharing this canonical id across the whole tree; when more than one, fall + * back to role+label, exactly the identity an unrecorded id already computes. + * Both sites sharing one predicate is what keeps the tuple and the chain from + * disagreeing (demoting one but not the other). The rule is capture-time + * uniqueness, not an id-namespace heuristic: a reused RN `FlatList` `testID` + * hits the same demotion on iOS. */ -function demoteNonUniqueId( - identity: LocalIdentity, - nodes: readonly SnapshotNode[], - byIndex: Map, -): LocalIdentity { +function demoteNonUniqueId(identity: LocalIdentity, nodes: readonly SnapshotNode[]): LocalIdentity { if (identity.id === undefined) return identity; - const idMatchCount = filterIdentitySet(nodes, byIndex, identity, []).length; - if (idMatchCount <= 1) return identity; + if (idMatchCountInTree(nodes, identity.id) <= 1) return identity; const { role, label } = identity; return { role, ...(label !== undefined ? { label } : {}) }; } diff --git a/src/replay/target-identity-node.ts b/src/replay/target-identity-node.ts index 148b16655..9d634919a 100644 --- a/src/replay/target-identity-node.ts +++ b/src/replay/target-identity-node.ts @@ -41,6 +41,33 @@ export function localIdentitiesEqual(a: LocalIdentity, b: LocalIdentity): boolea return a.id === b.id && a.role === b.role && a.label === b.label; } +/** + * ADR 0012 decision 3 amendment (#1269): how many nodes in `nodes` carry the + * canonical identity `id`. THE single uniqueness predicate behind both + * id-demotion sites — `computeTargetEvidence`'s `target-v1` identity tuple + * and `buildSelectorChainForNode`'s selector chain. Sharing it is the + * correctness guarantee: a non-unique id is dropped from BOTH or NEITHER, + * never half-demoted (dropped from identity while still leading the chain, or + * the reverse — the exact split #1269's fix exists to close). + * + * `id` is a canonical identity id (`readNodeLocalIdentity(node).id`: NFC + + * 256-byte field cap, no trimming), and every candidate is measured through + * the SAME reader, so the count is over exactly the identities the replay + * verifier keys on. There is deliberately NO ancestry / parent-walk + * exclusion: an id is non-selective the moment two nodes anywhere in the tree + * carry it, independent of structural context or a broken parent linkage. + */ +export function idMatchCountInTree( + nodes: readonly Pick[], + id: string, +): number { + let count = 0; + for (const node of nodes) { + if (readNodeLocalIdentity(node).id === id) count += 1; + } + return count; +} + /** * ADR 0012 decision 3: the STRUCTURAL denotation of a node within its capture * — the discriminators path 6 uses to isolate ONE member among several nodes diff --git a/src/selectors/build.test.ts b/src/selectors/build.test.ts index d383280c3..ca2983a62 100644 --- a/src/selectors/build.test.ts +++ b/src/selectors/build.test.ts @@ -2,6 +2,7 @@ import { test } from 'vitest'; import assert from 'node:assert/strict'; import type { SnapshotNode } from '../kernel/snapshot.ts'; import { buildNodes } from '../__tests__/test-utils/snapshot-builders.ts'; +import { computeTargetEvidence } from '../daemon/session-target-evidence.ts'; import { buildSelectorChainForNode } from './build.ts'; function findByLabel(nodes: SnapshotNode[], label: string): SnapshotNode { @@ -10,6 +11,24 @@ function findByLabel(nodes: SnapshotNode[], label: string): SnapshotNode { return found; } +/** + * #1269 cross-invariant: the two writers demote through ONE shared + * uniqueness predicate, so for the same node+tree the `target-v1` identity + * tuple carries an `id` iff the built selector chain leads with an `id=` + * clause — never a half-demoted split (id in one, gone from the other). + */ +function assertIdParityAcrossWriters(nodes: SnapshotNode[], node: SnapshotNode): void { + const evidence = computeTargetEvidence({ node, preActionNodes: nodes }); + const chain = buildSelectorChainForNode(node, 'android', { action: 'get', nodes }); + const evidenceHasId = evidence?.id !== undefined; + const chainHasId = chain.some((entry) => entry.startsWith('id=')); + assert.equal( + evidenceHasId, + chainHasId, + `id parity broken: evidence.id=${JSON.stringify(evidence?.id)} chain=${JSON.stringify(chain)}`, + ); +} + // --------------------------------------------------------------------------- // #1269: a recorded id only leads the chain when it is unique in the // record-time tree. `android:id/title` — a shared Android framework @@ -145,3 +164,111 @@ test('buildSelectorChainForNode: without a record-time tree, an id is trusted as const chain = buildSelectorChainForNode(nodes[0]!, 'ios', { action: 'get' }); assert.deepEqual(chain, ['id="save"', 'role="button" label="Save"', 'label="Save"']); }); + +// --------------------------------------------------------------------------- +// #1269 cross-invariant (Ask 1): the identity tuple and the selector chain go +// through ONE shared uniqueness predicate (`idMatchCountInTree`), so they +// demote in lockstep. The pre-fix bug was two predicates with different +// normalization/exclusions that could disagree — the non-NFC case below is +// exactly where a raw `normalizeSelectorText` chain scan kept an id the +// NFC-normalized identity tuple demoted. +// --------------------------------------------------------------------------- + +test('#1269 cross-invariant: evidence.id present iff chain leads with id= — demoted, unique, and non-NFC edge cases', () => { + // Demoted: a shared framework id across 3 rows → both writers drop it. + const demoted = buildNodes([ + { index: 0, type: 'FrameLayout', depth: 0 }, + { index: 1, type: 'RecyclerView', depth: 1, parentIndex: 0 }, + { index: 2, type: 'LinearLayout', depth: 2, parentIndex: 1 }, + { + index: 3, + type: 'TextView', + identifier: 'android:id/title', + label: 'Network & internet', + rect: { x: 0, y: 100, width: 300, height: 48 }, + depth: 3, + parentIndex: 2, + }, + { index: 4, type: 'LinearLayout', depth: 2, parentIndex: 1 }, + { + index: 5, + type: 'TextView', + identifier: 'android:id/title', + label: 'Apps', + rect: { x: 0, y: 148, width: 300, height: 48 }, + depth: 3, + parentIndex: 4, + }, + { index: 6, type: 'LinearLayout', depth: 2, parentIndex: 1 }, + { + index: 7, + type: 'TextView', + identifier: 'android:id/title', + label: 'Battery', + rect: { x: 0, y: 196, width: 300, height: 48 }, + depth: 3, + parentIndex: 6, + }, + ]); + assertIdParityAcrossWriters(demoted, findByLabel(demoted, 'Network & internet')); + + // Unique: a distinct id → both writers keep it. + const unique = buildNodes([ + { index: 0, type: 'Window', depth: 0 }, + { + index: 1, + type: 'Button', + identifier: 'save', + label: 'Save', + rect: { x: 10, y: 10, width: 40, height: 20 }, + depth: 1, + parentIndex: 0, + }, + ]); + assertIdParityAcrossWriters(unique, findByLabel(unique, 'Save')); + + // Non-NFC edge: two ids equal under NFC but different raw bytes (decomposed + // vs precomposed "é"). The identity tuple always NFC-normalizes, so it sees + // one shared id and demotes; the pre-fix raw chain scan compared the raw + // bytes, saw two DIFFERENT ids, and kept the id in the chain — the exact + // half-demoted split this cross-invariant now forbids. + const decomposed = 'cafe\u0301'; // c a f e + U+0301 combining acute accent + const precomposed = 'caf\u00e9'; // c a f é (single U+00E9) + assert.notEqual(decomposed, precomposed); // distinct raw code-point sequences + assert.equal(decomposed.normalize('NFC'), precomposed.normalize('NFC')); // equal under NFC + const nonNfc = buildNodes([ + { index: 0, type: 'RecyclerView', depth: 0 }, + { index: 1, type: 'LinearLayout', depth: 1, parentIndex: 0 }, + { + index: 2, + type: 'TextView', + identifier: decomposed, + label: 'Coffee', + rect: { x: 0, y: 100, width: 300, height: 48 }, + depth: 2, + parentIndex: 1, + }, + { index: 3, type: 'LinearLayout', depth: 1, parentIndex: 0 }, + { + index: 4, + type: 'TextView', + identifier: precomposed, + label: 'Espresso', + rect: { x: 0, y: 148, width: 300, height: 48 }, + depth: 2, + parentIndex: 3, + }, + ]); + assertIdParityAcrossWriters(nonNfc, findByLabel(nonNfc, 'Coffee')); + // And concretely: the id is dropped from BOTH sides (not kept in the chain). + const evidence = computeTargetEvidence({ + node: findByLabel(nonNfc, 'Coffee'), + preActionNodes: nonNfc, + }); + assert.equal(evidence?.id, undefined); + const chain = buildSelectorChainForNode(findByLabel(nonNfc, 'Coffee'), 'android', { + action: 'get', + nodes: nonNfc, + }); + assert.ok(!chain.some((entry) => entry.startsWith('id='))); +}); diff --git a/src/selectors/build.ts b/src/selectors/build.ts index 200274df7..94f49aa9d 100644 --- a/src/selectors/build.ts +++ b/src/selectors/build.ts @@ -2,6 +2,7 @@ import type { Platform, PublicPlatform } from '../kernel/device.ts'; import type { SnapshotNode } from '../kernel/snapshot.ts'; import { isNodeVisible } from './node.ts'; import { extractNodeText, normalizeType } from '../snapshot/snapshot-processing.ts'; +import { idMatchCountInTree, readNodeLocalIdentity } from '../replay/target-identity-node.ts'; export function buildSelectorChainForNode( node: SnapshotNode, @@ -73,10 +74,18 @@ export function buildSelectorChainForNode( * `android:id/title` matching every list row is the measured case) resolves * the wrong element under positional drift on replay. The rule is * capture-time uniqueness, not an id-namespace heuristic: a reused RN - * `FlatList` `testID` hits the same demotion. `nodes` is the record-time - * tree the node was captured from; every writer call site passes it. When - * absent (e.g. a node built in isolation with no tree to check against) the - * id is trusted as-is. + * `FlatList` `testID` hits the same demotion. + * + * The uniqueness DECISION goes through `idMatchCountInTree` — the SAME + * predicate `computeTargetEvidence` uses for the `target-v1` identity tuple, + * over the canonical identity id (`readNodeLocalIdentity`: NFC + 256-byte + * cap) — so the chain and the tuple demote in lockstep and never disagree. + * Only the decision is shared; the kept clause still emits the chain's own + * `normalizeSelectorText` id (an id that survives the check keeps its + * existing string form — no behavior change for the already-unique path). + * `nodes` is the record-time tree; every writer call site passes it. When + * absent (a node built in isolation, e.g. a test with no tree) the id is + * trusted as-is. */ function selectableId( node: SnapshotNode, @@ -84,12 +93,12 @@ function selectableId( ): string | null { const id = normalizeSelectorText(node.identifier); if (!id || !nodes) return id; - let matches = 0; - for (const candidate of nodes) { - if (normalizeSelectorText(candidate.identifier) === id) matches += 1; - if (matches > 1) return null; - } - return id; + // A non-null `normalizeSelectorText` id guarantees a defined canonical id + // (NFC/cap never empty a non-whitespace string), so this is always the + // identity id the tuple would carry. + const canonicalId = readNodeLocalIdentity(node).id; + if (canonicalId === undefined) return id; + return idMatchCountInTree(nodes, canonicalId) > 1 ? null : id; } function uniqueStrings(values: readonly string[]): string[] { From fcc9b605db5657022640509e10681033748b438e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Pierzcha=C5=82a?= Date: Wed, 15 Jul 2026 21:53:30 +0200 Subject: [PATCH 3/3] fix(replay): thread record-time tree through the extracted suggestion helper MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rebase-conflict resolution against origin/main. #1217 (typed direct Maestro engine) extracted `buildReplayDivergenceSuggestionForNode` out of `resolveSuggestionCandidate` and added a second caller in `session-replay-maestro-failure.ts`. My #1269 change had added `nodes` to the `buildSelectorChainForNode` call that #1217 moved into the extracted helper, so after rebase the helper referenced an out-of-scope `nodes`. Thread the record-time tree as a required `nodes` param on the helper and pass it from BOTH callers (each already has it in scope). This keeps the non-unique-id demotion applied wherever a divergence/repair suggestion chain is built — now including the typed-Maestro suggestion path — with no behavior change for the already-unique-id case. --- src/daemon/handlers/session-replay-divergence.ts | 5 ++++- src/daemon/handlers/session-replay-maestro-failure.ts | 1 + 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/src/daemon/handlers/session-replay-divergence.ts b/src/daemon/handlers/session-replay-divergence.ts index 6adc3b23b..cad7f1472 100644 --- a/src/daemon/handlers/session-replay-divergence.ts +++ b/src/daemon/handlers/session-replay-divergence.ts @@ -536,6 +536,7 @@ function resolveSuggestionCandidate(params: { return { suggestion: buildReplayDivergenceSuggestionForNode({ node: resolved.node, + nodes, session, action, basis, @@ -548,12 +549,14 @@ function resolveSuggestionCandidate(params: { export function buildReplayDivergenceSuggestionForNode(params: { node: SnapshotNode; + /** The record-time tree the node came from, for #1269 non-unique-id demotion in the chain. */ + nodes: readonly SnapshotNode[]; session: SessionState; action: ReplayReportAction; basis: ReplayDivergenceSuggestionBasis; sanitize: DivergenceFieldSanitizer; }): ReplayDivergenceSuggestion { - const { node, session, action, basis, sanitize } = params; + const { node, nodes, session, action, basis, sanitize } = params; const selectorChain = buildSelectorChainForNode(node, session.device.platform, { action: action.command === 'fill' ? 'fill' : isTouchTargetCommand(action.command) ? 'click' : 'get', diff --git a/src/daemon/handlers/session-replay-maestro-failure.ts b/src/daemon/handlers/session-replay-maestro-failure.ts index 0ee796dbc..ce48679cf 100644 --- a/src/daemon/handlers/session-replay-maestro-failure.ts +++ b/src/daemon/handlers/session-replay-maestro-failure.ts @@ -210,6 +210,7 @@ function collectTypedMaestroSuggestions(params: { ).map(({ node, basis }) => buildReplayDivergenceSuggestionForNode({ node, + nodes: params.nodes, session: params.session, action: params.action, basis,