Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
6fc1c18
test: add pinned Maestro conformance harness
thymikee Jul 11, 2026
a8dd730
feat: add typed Maestro program IR parser
thymikee Jul 11, 2026
5dc9285
docs: define direct Maestro engine architecture
thymikee Jul 11, 2026
67b6539
test: compare Maestro oracle with typed IR
thymikee Jul 11, 2026
375832b
feat: add direct Maestro program engine
thymikee Jul 11, 2026
380ea2c
refactor: narrow Maestro execution context
thymikee Jul 11, 2026
a5602a7
refactor: tighten Maestro program parsing
thymikee Jul 11, 2026
693c725
fix: verify iOS Maestro visibility waits
thymikee Jul 11, 2026
acf17f2
refactor: isolate retained Maestro runtimes
thymikee Jul 12, 2026
c73a9e3
refactor: type Maestro target resolution
thymikee Jul 12, 2026
db4bbf1
refactor: harden typed Maestro execution
thymikee Jul 12, 2026
d5ed1e9
refactor: share in-page swipe planning
thymikee Jul 12, 2026
f9fcdd7
feat: add typed Maestro runtime port
thymikee Jul 12, 2026
96bc961
refactor: parse Maestro suite metadata from typed IR
thymikee Jul 12, 2026
2bb8088
refactor: centralize Maestro include loading
thymikee Jul 12, 2026
3b4bc42
feat: execute Maestro files through typed engine
thymikee Jul 12, 2026
ad80ebd
refactor: share replay built-in variables
thymikee Jul 12, 2026
ba9e9bb
fix: make Maestro target intent explicit
thymikee Jul 12, 2026
4c891fa
fix: refresh Maestro targets before input
thymikee Jul 12, 2026
e46ec3f
refactor: format Maestro progress from typed IR
thymikee Jul 12, 2026
69f3ee3
feat: compile typed Maestro replay plans
thymikee Jul 12, 2026
2d4d59c
feat: bind typed Maestro runtime to public commands
thymikee Jul 12, 2026
b26559f
feat: route Maestro YAML through typed runtime
thymikee Jul 12, 2026
13386a5
refactor: remove legacy Maestro runtime
thymikee Jul 12, 2026
4db3e6a
refactor: remove obsolete replay control model
thymikee Jul 12, 2026
dd8b9a7
refactor: split typed Maestro plan modules
thymikee Jul 12, 2026
bfc3e55
fix: harden typed Maestro runtime semantics
thymikee Jul 12, 2026
8aa78a6
docs: update direct Maestro architecture
thymikee Jul 12, 2026
e6c6ace
fix: reconcile Maestro runtime with merged contracts
thymikee Jul 13, 2026
b7c8d26
fix: harden typed Maestro execution boundaries
thymikee Jul 13, 2026
e906174
fix: harden typed Maestro runtime evidence
thymikee Jul 13, 2026
43a3d4d
perf: avoid eager Maestro device resolution
thymikee Jul 13, 2026
883419c
refactor: finalize typed Maestro execution
thymikee Jul 14, 2026
17a4d6f
fix: reject Android system-only helper snapshots
thymikee Jul 14, 2026
7352d32
fix: preserve Android system dialog snapshots
thymikee Jul 14, 2026
3682be6
fix: make helper-backed CI deterministic
thymikee Jul 14, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 10 additions & 5 deletions CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,8 @@
seam. Helper long-press executes its absolute stationary path without a viewport probe; provider
long-press receives its paired provider-owned viewport. See ADR 0013.
- Multi-touch geometry: the internal initial span and angle plus centroid translation, scale, and rotation used to build both contact trajectories. Geometry is viewport-aware and fails early when the requested motion cannot fit; it is not a public tuning surface.
- Maestro program: source-preserving typed representation of supported Maestro YAML. It is interpreted directly through the compatibility runtime port and never lowered through generic replay action strings. See ADR 0015.
- Maestro observation generation: explicit compatibility-engine state identifying evidence captured since the most recent mutation. Queries may share semantic evidence within one generation; every mutation attempt invalidates it before dispatch. Interaction geometry is action-local: unique exact iOS selectors resolve and tap atomically in XCTest, while coordinate dispatch uses a fresh target snapshot. Rectangles are never shared across command boundaries.
- Guarantee cell: one (dispatch path, guarantee) entry in `src/contracts/interaction-guarantees.ts`, classified as runtime/runner/delegated/inapplicable/waived. Completeness is a compile error; honesty is gate-tested.
- Owned waiver: a `gap:`-prefixed waived cell carrying a `trackingIssue` URL. Waivers are diffable debt with an owner, never folklore.
- Parity table: golden JSON fixture under `contracts/fixtures/` consumed by both vitest and the runner's gated Swift tests, so a cross-language rule (e.g. tap-point policy) cannot drift silently. Change the rule only via the table.
Expand Down Expand Up @@ -154,10 +156,13 @@ the observable freshness and failure semantics below before any runtime refactor
disables direct iOS selector shortcuts while pending.
- `setSessionSnapshot` is the centralized session snapshot mutation path. Sparse captures do not
write back, and empty `@ref`-scoped snapshot output must not replace the stored session snapshot.
- Maestro target matching remains snapshot-based, fresh, and policy-rich. Native selector
simplification must not erase Maestro regex/string selector behavior, visibility filtering,
ranking, fuzzy fallback, visible-context preference, Android duplicate handling, tab-strip
inference, or assertion/wait semantics.
- Maestro target matching remains snapshot-based and policy-rich. Coordinate dispatch always uses a
fresh target snapshot. A unique exact iOS match may instead reuse bound same-generation semantic
evidence and dispatch through XCTest's atomic selector tap; structured live-selector failures return
to fresh Maestro resolution. This optimization must not erase Maestro regex/string selector behavior,
visibility filtering, ranking, visible-context preference, Android duplicate handling, tab-strip
inference, or assertion/wait semantics. Plain text is exact and regex-aware; do not add
substring/fuzzy recovery that changes authored selector meaning.

Evidence: [ADR 0002](docs/adr/0002-persistent-platform-helper-sessions.md),
[ADR 0004](docs/adr/0004-ios-snapshot-backend-strategy.md),
Expand All @@ -166,7 +171,7 @@ Evidence: [ADR 0002](docs/adr/0002-persistent-platform-helper-sessions.md),
[`find.test.ts`](src/daemon/handlers/__tests__/find.test.ts),
[`snapshot-handler.test.ts`](src/daemon/handlers/__tests__/snapshot-handler.test.ts),
[`snapshot-scoped-refs.test.ts`](src/daemon/handlers/__tests__/snapshot-scoped-refs.test.ts),
[`runtime-targets.test.ts`](src/compat/maestro/__tests__/runtime-targets.test.ts), and
[`runtime-targets-typed.test.ts`](src/compat/maestro/__tests__/runtime-targets-typed.test.ts), and
[`android-test-suite.test.ts`](test/integration/provider-scenarios/android-test-suite.test.ts).

## Testing Principles
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,7 @@ extension RunnerTests {
}
let command = try runnerCommandFixture(
"""
{"command":"gesture","commandId":"gesture-fling-fallback","gesturePlan":{"topology":"single","intent":"fling","durationMs":100,"viewport":{"x":0,"y":0,"width":200,"height":300},"pointers":[{"pointerId":0,"samples":[{"offsetMs":0,"point":{"x":160,"y":150}},{"offsetMs":100,"point":{"x":40,"y":150}}]}]}}
{"command":"gesture","commandId":"gesture-fling-fallback","gesturePlan":{"topology":"single","intent":"fling","executionProfile":"endpoint-hold","durationMs":100,"viewport":{"x":0,"y":0,"width":200,"height":300},"pointers":[{"pointerId":0,"samples":[{"offsetMs":0,"point":{"x":160,"y":150}},{"offsetMs":100,"point":{"x":40,"y":150}}]}]}}
"""
)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1186,6 +1186,13 @@ extension RunnerTests {
: plan.intent == "pan" || plan.intent == "pinch" || plan.intent == "rotate"
|| plan.intent == "transform"
guard supportedIntent else { return "planned gesture has unsupported intent for its topology" }
if plan.topology == "single" {
guard plan.executionProfile == "endpoint-hold" || plan.executionProfile == "timed-pan" else {
return "single-pointer gesture requires a supported execution profile"
}
} else if plan.executionProfile != nil {
return "multi-touch gesture cannot define a single-pointer execution profile"
}
guard plan.durationMs.isFinite, plan.durationMs >= 16, plan.durationMs <= 10_000 else {
return "planned gesture durationMs must be between 16 and 10000"
}
Expand Down Expand Up @@ -1245,7 +1252,9 @@ extension RunnerTests {
}

func plannedGestureExecution(for plan: RunnerGesturePlan) -> PlannedGestureExecution {
plan.topology == "single" && plan.intent == "fling" ? .fastSwipe : .sampled
plan.topology == "single" && plan.executionProfile == "endpoint-hold"
? .fastSwipe
: .sampled
}

func sampledPlannedGesture(
Expand Down Expand Up @@ -1459,6 +1468,7 @@ extension RunnerTests {
)

XCTAssertNil(plannedGestureValidationError(plan))
XCTAssertEqual(plannedGestureExecution(for: plan), .sampled)
}

func testPlannedMultiTouchGestureRejectsMismatchedOffsets() throws {
Expand All @@ -1479,7 +1489,7 @@ extension RunnerTests {
let plan = try JSONDecoder().decode(
RunnerGesturePlan.self,
from: Data(
#"{"topology":"single","intent":"fling","durationMs":100,"viewport":{"x":0,"y":0,"width":200,"height":300},"pointers":[{"pointerId":0,"samples":[{"offsetMs":0,"point":{"x":160,"y":150}},{"offsetMs":100,"point":{"x":40,"y":150}}]}]}"#.utf8
#"{"topology":"single","intent":"fling","executionProfile":"endpoint-hold","durationMs":100,"viewport":{"x":0,"y":0,"width":200,"height":300},"pointers":[{"pointerId":0,"samples":[{"offsetMs":0,"point":{"x":160,"y":150}},{"offsetMs":100,"point":{"x":40,"y":150}}]}]}"#.utf8
)
)

Expand All @@ -1490,13 +1500,39 @@ extension RunnerTests {
let plan = try JSONDecoder().decode(
RunnerGesturePlan.self,
from: Data(
#"{"topology":"single","intent":"pan","durationMs":500,"viewport":{"x":0,"y":0,"width":200,"height":300},"pointers":[{"pointerId":0,"samples":[{"offsetMs":0,"point":{"x":160,"y":150}},{"offsetMs":250,"point":{"x":100,"y":150}},{"offsetMs":500,"point":{"x":40,"y":150}}]}]}"#.utf8
#"{"topology":"single","intent":"pan","executionProfile":"timed-pan","durationMs":500,"viewport":{"x":0,"y":0,"width":200,"height":300},"pointers":[{"pointerId":0,"samples":[{"offsetMs":0,"point":{"x":160,"y":150}},{"offsetMs":250,"point":{"x":100,"y":150}},{"offsetMs":500,"point":{"x":40,"y":150}}]}]}"#.utf8
)
)

XCTAssertEqual(plannedGestureExecution(for: plan), .sampled)
}

func testSinglePointerEndpointHoldUsesFastSwipeExecution() throws {
let plan = try JSONDecoder().decode(
RunnerGesturePlan.self,
from: Data(
#"{"topology":"single","intent":"pan","executionProfile":"endpoint-hold","durationMs":500,"viewport":{"x":0,"y":0,"width":200,"height":300},"pointers":[{"pointerId":0,"samples":[{"offsetMs":0,"point":{"x":160,"y":150}},{"offsetMs":500,"point":{"x":40,"y":150}}]}]}"#.utf8
)
)

XCTAssertNil(plannedGestureValidationError(plan))
XCTAssertEqual(plannedGestureExecution(for: plan), .fastSwipe)
}

func testSinglePointerGestureRejectsMissingExecutionProfile() throws {
let plan = try JSONDecoder().decode(
RunnerGesturePlan.self,
from: Data(
#"{"topology":"single","intent":"pan","durationMs":500,"viewport":{"x":0,"y":0,"width":200,"height":300},"pointers":[{"pointerId":0,"samples":[{"offsetMs":0,"point":{"x":160,"y":150}},{"offsetMs":500,"point":{"x":40,"y":150}}]}]}"#.utf8
)
)

XCTAssertEqual(
plannedGestureValidationError(plan),
"single-pointer gesture requires a supported execution profile"
)
}

func testDesktopScrollWheelDeltasMapDirections() throws {
XCTAssertEqual(try XCTUnwrap(desktopScrollWheelDeltas(direction: "up", pixels: 120)).vertical, 120)
XCTAssertEqual(try XCTUnwrap(desktopScrollWheelDeltas(direction: "down", pixels: 120)).vertical, -120)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,7 @@ struct Command: Codable {
struct RunnerGesturePlan: Codable {
let topology: String
let intent: String
let executionProfile: String?
let durationMs: Double
let viewport: RunnerGestureViewport
let pointers: [RunnerGesturePointer]
Expand Down
4 changes: 3 additions & 1 deletion docs/adr/0002-persistent-platform-helper-sessions.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,9 @@ For Android snapshots, productize a persistent helper mode that keeps `UiAutomat
serves fresh snapshot requests over an `adb forward` socket. Do not add snapshot result caching as
part of that first step. The first reliable win is infrastructure reuse, not data reuse. The current
implementation keeps the existing one-shot instrumentation helper as the fallback for startup,
socket, protocol, and request failures.
socket, protocol, and request failures. Both transports execute the same packaged helper contract;
agent-device must fail closed when that helper is unavailable or invalid instead of substituting
the legacy `adb uiautomator dump` snapshot engine.

Android permits only one reliable instrumentation-owned `UiAutomation` context per device. Before
starting a different instrumentation helper, such as touch synthesis, the daemon must stop the
Expand Down
26 changes: 17 additions & 9 deletions docs/adr/0013-unified-gesture-plans.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,13 +29,17 @@ model; compatibility is owed at CLI, Node.js, and MCP.
The runtime plans canonical intent in `src/contracts/gesture-plan.ts`. Contact topology is separate
from motion:

- one contact: pan or fling with a complete pointer trajectory;
- one contact: pan or fling with a complete pointer trajectory and an explicit execution profile;
- two contacts: pan, pinch, rotate, or transform with two complete, synchronized trajectories.

`swipe` is public sugar for a fixed-duration fling. Its historical optional duration remains a
thin compatibility alias to pan and reports a deprecation. The same rule applies to the historical
fling duration. Pinch fixes translation and rotation at zero; rotate fixes translation at zero and
scale at one; two-finger pan fixes scale at one and rotation at zero; transform can apply all three
`swipe` without a duration is public sugar for a fixed-duration fling. Its historical optional
duration normalizes to pan intent with an endpoint-hold execution profile and reports a deprecation
toward explicit pan. Maestro-authored swipes follow the same normalization and materialize
Maestro's 400 ms default when duration is omitted. A genuine pan uses the timed-pan profile, so
compatibility aliases retain their release behavior without becoming a new semantic intent. The
same deprecation-to-pan rule applies to the historical fling duration. Pinch fixes translation and rotation at zero; rotate fixes
translation at zero and scale at one; two-finger pan fixes scale at one and rotation at zero;
transform can apply all three
components atomically. Intent remains on the plan even when aliases share an executor.

The planner owns deterministic multi-touch geometry. Contacts start at -90 degrees, except Android
Expand Down Expand Up @@ -65,10 +69,14 @@ Platform adapters consume the canonical plan:
injected coordinates against zero-origin extents that include the viewport offset. The snapshot helper is stopped
before local gesture instrumentation because Android permits only one instrumentation owner of
`UiAutomation`.
- iOS converts every planned point to native orientation and feeds the exact arrays to the existing
private XCTest event bridge. macOS lowers a one-contact plan to its drag executor and tvOS lowers
it to remote direction. Core admission and the Apple adapter both consume the same shared
multi-touch support policy; multi-touch remains capability-gated to iOS simulators.
- iOS lowers one-contact endpoint-hold plans to the established fast-swipe synthesis profile. That
profile reaches the endpoint in 100 ms, then holds there for the planned duration before lifting,
matching Maestro's XCTest driver. Timed-pan and two-contact plans convert every point to native
orientation and feed the exact planned arrays to the private XCTest event bridge. Android and
WebDriver continue to execute the plan samples across the authored duration, matching their
native Maestro drivers. macOS lowers a one-contact plan to its drag executor and tvOS lowers it to remote
direction. Core admission and the Apple adapter both consume the same shared multi-touch support
policy; multi-touch remains capability-gated to iOS simulators.
- WebDriver lowers a supported plan to synchronized W3C pointer action sources. Multi-touch remains
capability-gated until a provider proves it.

Expand Down
Loading
Loading