Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
152 commits
Select commit Hold shift + click to select a range
5ad987b
Show hierarchy and check-syntax commands in help (#2904)
Fl0p Feb 5, 2026
bdd8e03
Feature: Adding scrolling for Flutter Web Apps! (#2828)
ff-vivek Feb 5, 2026
579d425
Add tags & properties to test reports (#2979)
proksh Feb 6, 2026
5b37d53
Fix connection failures due to failing driver tests while auto granti…
amanjeetsingh150 Feb 6, 2026
fa86718
fix: screenshot exceptions (#2981)
proksh Feb 9, 2026
e9f47a7
Fix --no-ansi breaks and stops printing nested sub flows (#2970)
eldare Feb 9, 2026
11474c6
Fix: only dismiss notifications dialog automatically (#2988)
amanjeetsingh150 Feb 10, 2026
c392d10
fix: iOS call timeout being 60 seconds (#2990)
amanjeetsingh150 Feb 10, 2026
ce1ec83
Tidy up after screenshot test (#2992)
Fishbowler Feb 10, 2026
bb610b4
timeout to match read on XCTest http client (#2993)
amanjeetsingh150 Feb 11, 2026
e1a63db
fix: timeouts creating errors in runtime (#2994)
amanjeetsingh150 Feb 11, 2026
6e17381
Bump some dependencies (#2991)
Fishbowler Feb 12, 2026
e47ba45
Prepare for release v2.2.0 (#2989)
Fishbowler Feb 20, 2026
b2db891
Fix e2e tests following changes in demo_app, and make inputRandomPers…
Fishbowler Feb 24, 2026
975559f
Fix specifying --device when also specifying --host (#3014)
Fishbowler Feb 24, 2026
509f32f
Add web support for clearState (#2996)
jkronborg Feb 24, 2026
2066d58
Moving all the demo app E2E tests in demo app repo (#3023)
amanjeetsingh150 Mar 4, 2026
1712019
fix: iPad landscape orientation support (#2980)
leggomuhgreggo Mar 5, 2026
4717f5a
remove: integration test and cover in E2E (#3033)
amanjeetsingh150 Mar 5, 2026
b674a3a
Fix: make assertScreenshot work without specifying the extension (#3034)
SosenWiosen Mar 6, 2026
753ac75
Add assertScreenshot paths to DependencyResolver (#3028)
Fishbowler Mar 6, 2026
919a5e3
fix: tests didn't fail when the screenshot size was mismatched. (#3020)
SosenWiosen Mar 6, 2026
d4c2f35
Allow relative paths from flows to assertScreenshot reference images …
Fishbowler Mar 6, 2026
cd88b61
revert: unintentional apk changes (#3040)
amanjeetsingh150 Mar 8, 2026
a6f1298
Use device info when retrying cloud uploads (#3016)
Fishbowler Mar 9, 2026
3d01bc8
chore: update drivers for iOS (#3042)
amanjeetsingh150 Mar 10, 2026
f3dd692
Prepare for release v2.3.0 (#3043)
Fishbowler Mar 10, 2026
46b646d
Strongly Typed Device Configuration (#3001)
proksh Mar 11, 2026
a90c8db
feat: make all options for resolve to be passed explisitly + added os…
proksh Mar 12, 2026
6b2ad09
Replace flat resolve() params with sealed DeviceRequest (#3055)
proksh Mar 16, 2026
0c1ec99
Never have callTimeout lower than readTimeout (#3047)
Fishbowler Mar 16, 2026
f758446
Fix step ordering in HTML detailed report (#3048)
Fishbowler Mar 16, 2026
692adb7
Testing a single context, IIFE-based approach to isolating vars set i…
steviec Mar 17, 2026
7bb6720
feat: added list device command (#3046)
proksh Mar 17, 2026
e15f68d
Fix JsEngine resource leak, close in finally block (#3062)
amanjeetsingh150 Mar 17, 2026
8bf4df6
Read hierarchy from iFrames (#3067)
Fishbowler Mar 18, 2026
b6f0cbf
fix platform from string (#3068)
proksh Mar 19, 2026
3ab8798
fix start device command for os-version flag (#3073)
proksh Mar 20, 2026
c970fb3
fix: removed deviceCatalog and updated deviceSpec (#3074)
proksh Mar 20, 2026
b329463
Add deprecation warning to legacy Maestro Studio (#3077)
Fishbowler Mar 23, 2026
84aae73
SetOrientation with env variables + Improved validation (#3088)
sailsdima Mar 27, 2026
a5566c2
fix: reduce websocket response timeout to 5 seconds (#3096)
sailsdima Mar 31, 2026
dd38131
Fix input text crashing on native passcode screen when providing nume…
amanjeetsingh150 Mar 31, 2026
430f033
fix: implement clearAppState for Selenium-based WebDriver (#3103)
amanjeetsingh150 Apr 1, 2026
411e3bd
Fail startRecording gracefully with error message (#3105)
amanjeetsingh150 Apr 1, 2026
296dc7b
Feat/extract validators from cloud interactor (#3106)
amanjeetsingh150 Apr 1, 2026
5ec9862
Update CONTRIBUTING.md (#3109)
Fishbowler Apr 1, 2026
44cb012
Enrich AppMetadata as single source of truth for app analysis (#3112)
proksh Apr 1, 2026
43981c0
fix: add cases for validating workspace (#3113)
amanjeetsingh150 Apr 1, 2026
2661992
Add cross-domain iframe support (#3079)
Fishbowler Apr 2, 2026
e95b2e0
Fix return statement for deriving XPath for non elements not in an if…
Fishbowler Apr 2, 2026
8362ae6
updated cloud flags (#3072)
proksh Apr 2, 2026
8800807
Prepare for release of v2.4.0 (#3119)
Fishbowler Apr 2, 2026
f24a6f8
Remove recipes (#3110)
Fishbowler Apr 3, 2026
03f067b
Fix list-cloud-devices command suggestion typo (#3126)
simon-gilmurray Apr 3, 2026
41d6dec
fix: handle empty text list in ToastAccessibilityListener (#3111)
HarlonWang Apr 5, 2026
a01aa59
Rebuild drivers (#3144)
Fishbowler Apr 5, 2026
645ce28
Maestro-cli build script handles generating new drivers (#3082)
amanjeetsingh150 Apr 5, 2026
f21871e
feat: build demo_app from source in e2e workflow (#3129)
amanjeetsingh150 Apr 6, 2026
08fa7da
Remove CLI-side app binary validation (#3125)
proksh Apr 6, 2026
32c08aa
fix: added normalised path (#3134)
proksh Apr 7, 2026
75fc037
Fix --android-api-level flag being silently ignored in cloud uploads …
proksh Apr 7, 2026
8e39fe5
Fix log folder name generation (#3120)
Fishbowler Apr 7, 2026
b979e6c
fix: resolve config path before filtering flow files (#3159)
proksh Apr 9, 2026
ba849f3
Skip additional config files in workspace execution planner (#3150)
proksh Apr 9, 2026
3532f77
fix: Orchestra properly respects coroutine cancellation (#3157)
steviec Apr 9, 2026
6337008
feat: make Maestro natively suspend-aware (#3162)
steviec Apr 9, 2026
fdbf66e
Detect Azure as CI provider (#3163)
jkronborg Apr 10, 2026
92c9aaa
Improvements on android E2E job (#3147)
amanjeetsingh150 Apr 12, 2026
fb078c4
build: replace processResources->maestro-android dependsOn with sha25…
amanjeetsingh150 Apr 13, 2026
b3b0867
Fix download app script, to fix update samples (#3173)
amanjeetsingh150 Apr 13, 2026
77f082e
Fix workspace ZIP path traversal for single-flow cloud uploads (#3154)
proksh Apr 13, 2026
cefe17c
rebuild driver on 26.2 (#3174)
amanjeetsingh150 Apr 13, 2026
121b6ea
fix: stabilize :maestro-orchestra:test against cyclic-flow SOE (#3175)
steviec Apr 14, 2026
d727ed0
feat: use direct ADB socket for gRPC instead of TCP port forwarding (…
Leland-Takamine Apr 14, 2026
b4c88bf
refactor: slim DeviceSpec to required fields + sparse serialization (…
steviec Apr 15, 2026
f3fad23
fix: guard WorkingDirectory.resolve(String) against absolute paths (#…
pedro18x Apr 15, 2026
2316457
fix: drop use_fuzzy_matching from tap_on MCP schema (#3181)
pedro18x Apr 16, 2026
fe579f2
chore: upgrade MCP Kotlin SDK to official 0.11.1 (#3182)
Leland-Takamine Apr 16, 2026
946ed9f
feat: native Maestro Cloud tools in the MCP server (#3184)
pedro18x Apr 17, 2026
96013b0
Added the missing node text for the textarea in the view hierarchy (#…
dineshvenugopal Apr 20, 2026
5beeb16
feat(mcp): consolidate run_flow + run_flow_files into single `run` to…
pedro18x Apr 20, 2026
b54bdfa
feat(mcp): drop 8 redundant MCP tools (#3195)
pedro18x Apr 21, 2026
c60939f
refactor: extract maestro debug artifact generation from CLI to core …
amanjeetsingh150 Apr 21, 2026
c9a2f2d
feat(mcp): server instructions + clean stdout handshake (#3201)
pedro18x Apr 21, 2026
5926ac1
Add missing string command config for random inputs, and add e2e test…
Fishbowler Apr 21, 2026
72a1a80
feat(mcp): add list_cloud_devices + device_model param on run_on_clou…
pedro18x Apr 21, 2026
86c0ec0
feat(mcp): tighten instructions and inspect_view_hierarchy descriptio…
pedro18x Apr 21, 2026
6a1cf87
Fix `--config` on `maestro cloud` and unify workspace-config handling…
proksh Apr 21, 2026
65f4175
included workspace in progress bar (#3199)
proksh Apr 21, 2026
b5b28eb
Allow evaluation of JS expressions in waitForAnimationToEnd timeout (…
Fishbowler Apr 22, 2026
aba3e86
feat(mcp): ground text selectors in the hierarchy, not screenshots (#…
pedro18x Apr 22, 2026
b5cf9a0
feat(mcp): nudge on assertScreenshot + broaden cheat_sheet guidance (…
pedro18x Apr 22, 2026
c21d85b
Adds a soft CLI warning when the workspace zip produced by maestro cl…
proksh Apr 22, 2026
b108ab9
fix(orchestra): retryCommand only retries on MaestroException, propag…
amanjeetsingh150 Apr 23, 2026
f05c930
fix(ios-runner): suppress XCTest UI-interruption preflight hang (#3206)
amanjeetsingh150 Apr 23, 2026
8c99cae
feat(mcp): rename inspect_view_hierarchy to inspect_screen, ship comp…
pedro18x Apr 23, 2026
3c93e5b
Add support for MANAGE_EXTERNAL_STORAGE permission on Android (#3217)
Fishbowler Apr 24, 2026
1135651
prepare for release 2.5.0 (#3210)
proksh Apr 27, 2026
34890c4
fix(maestro-client): remove redundant proto srcDirs to unblock source…
proksh Apr 27, 2026
e5da624
fix(ios-driver): fail-fast on transport failure instead of rethrowing…
amanjeetsingh150 Apr 27, 2026
c139f0f
Improve CLI help for start-device command (#3228)
Fishbowler Apr 28, 2026
d046e54
Fix detection of successful emulator installation in start-device (#3…
Fishbowler Apr 28, 2026
772cedd
Improve device info help in maestro cloud --help (#3230)
Fishbowler Apr 28, 2026
164d115
Improve Windows developer experience (#3236)
Fishbowler Apr 29, 2026
1d82579
Fix emulator discovery on Windows, fixes start-device (#3238)
Fishbowler Apr 29, 2026
36414a8
Fix NoSuchMethodError on Ktor server startup (login/studio/cloud) (#3…
Leland-Takamine Apr 30, 2026
a4c7c95
Prepare for release v2.5.1, and add prepare-release skill (#3244)
Fishbowler Apr 30, 2026
eecd552
Fix release instructions (#3246)
Fishbowler Apr 30, 2026
d0236f1
Bump dependencies in GitHub Actions (#3245)
Fishbowler Apr 30, 2026
03d0c79
chore(claude): add bump-android-version skill + diagnose-maestro-fail…
amanjeetsingh150 Apr 30, 2026
a7fdc11
feat(mcp): cheat_sheet works without Maestro Cloud auth (#3257)
pedro18x May 4, 2026
96c4a88
Attach YAML source provenance to MaestroCommand (#3251)
Leland-Takamine May 5, 2026
76a075d
chore!: remove deprecated Rhino JS engine (#3255)
amanjeetsingh150 May 5, 2026
ff3e2ed
feat(orchestra): structured parser errors with summary + detail (#3249)
proksh May 6, 2026
1942c60
make DeviceService.getDeviceScreen suspend (#3178)
proksh May 6, 2026
292a172
fix(assertScreenshot): report match percent that aligns with the thre…
proksh May 6, 2026
816498c
Add feature to demo_app to expose assertScreenshot threshold issue (#…
simon-gilmurray May 6, 2026
b1da80f
Fix the mutation setting of hierarchy-inert commands (#3232)
Fishbowler May 7, 2026
48b1269
Don't assume lowercase .yaml extension when parsing flownames (#3219)
Fishbowler May 8, 2026
5ee03fe
Update .idea for new Flutter additions (#3161)
Fishbowler May 8, 2026
c855236
Use emoji on Windows (#3268)
Fishbowler May 8, 2026
11889d9
fix(js): convert PolyglotException to JsEvaluationException at the en…
amanjeetsingh150 May 11, 2026
0518408
Route xctest runner logs into per-run debug output (#3267)
amanjeetsingh150 May 11, 2026
e8e257e
fix(env): fail fast with helpful message when env value is null (#3264)
proksh May 11, 2026
c49ed47
Warn when deprecated --android-api-level / --ios-version flags are us…
proksh May 11, 2026
db366b0
feat: enable parallel iOS simulator execution with per-device session…
qwertey6 May 12, 2026
c757709
feat: make JUnit XML id and classname attributes configurable (#3235)
dezsibalint May 12, 2026
af4138b
revert: restore iOS simulator driver zips to pre-#3138 state (#3283)
proksh May 12, 2026
7afd1b2
Add cloud URL and app binary ID to HTML report (#3260)
mikamikaWolt May 12, 2026
f51ac7d
Add file attribute to JUnit report (#3266)
hSATAC May 12, 2026
1feac72
Make hierarchy command JSON only (#3286)
Fishbowler May 12, 2026
9fec55b
MCP visualizer (#3288)
Leland-Takamine May 14, 2026
3e17f7c
feat(android-driver): enable gRPC channel-level retries for transient…
proksh May 15, 2026
a71e5b5
feat(http-client): opt-in retry on transient 5xx and IOException, ena…
proksh May 15, 2026
39eeb99
fix(web): swallow CDP ServiceConfigurationError in WebDriver.open() (…
proksh May 15, 2026
b1e9da6
fix(orchestra): use literal runScript file as sourceDescription (#3278)
proksh May 15, 2026
a5e06a1
chore(deps): bump selenium to 4.43.0 and CDP bindings to v145 (#3285)
proksh May 15, 2026
7cd8817
Add CI job to test building on all 3 OSes, and test version output (#…
Fishbowler May 15, 2026
6cba126
fix(mcp-visualizer): support physical Android devices (#3297)
Leland-Takamine May 15, 2026
b74ef53
fix(web): web inputText for native date inputs (#3270)
LVSant May 18, 2026
134c35b
fix: preserve hostname in DummyDns so devtools augmentation works (#3…
proksh May 18, 2026
f0da81b
chore(cli): remove bundled Maestro Studio in favor of desktop app (#3…
proksh May 18, 2026
5abacaa
feat(mcp-visualizer): pending-commands UX and panel redesign (#3298)
Leland-Takamine May 18, 2026
b7471ff
Rename MCP Visualizer to Maestro Viewer + add open_maestro_viewer too…
Leland-Takamine May 19, 2026
2313ad5
fix: throw correct test error in case of invalid timeout (#3179)
proksh May 19, 2026
eda43b0
feat(analytics): attach sanitized commandStringUsed to PostHog events…
proksh May 20, 2026
0f3182c
MCP take_screenshot: downscale screenshots to 2000px (#3305)
pedro18x May 20, 2026
741325d
Revert "feat(android-driver): enable gRPC channel-level retries for t…
proksh May 21, 2026
7899659
Revert "feat(http-client): opt-in retry on transient 5xx and IOExcept…
proksh May 21, 2026
6213e61
chore: prepare release 2.6.0 (#3302)
proksh May 21, 2026
5ae5181
Merge upstream v2.6.0 into tb-mods
jochen-testingbot May 22, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
117 changes: 117 additions & 0 deletions .claude/agents/diagnose-maestro-failure.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
---
name: diagnose-maestro-failure
description: Use when a Maestro test-android run has failed and a structured per-flow diagnosis is needed without bloating the caller's context. Accepts either a local artifact directory or a GitHub Actions run/job URL. Reads commands JSON, screenshots, and maestro.log; classifies each newly-failing passing/ flow; identifies cascade groups; and returns a structured report with proposed fixes. Does NOT edit code or commit — the caller is responsible for consent and application.
tools: Bash, Read, Agent
---

# Diagnose Maestro Failure (subagent)

You diagnose Maestro `test-android` failures from CI artifacts and return a structured report. You do not modify code, commit, or push — the caller handles consent and application.

## Input

The caller passes one of:

- **Local artifact directory** — e.g. `/Users/.../maestro-root-dir-android` or `/tmp/maestro-android-<run_id>`. Must contain a `tests/` subtree.
- **GitHub Actions URL** — `https://github.com/mobile-dev-inc/Maestro/actions/runs/<run_id>` or `.../runs/<run_id>/job/<job_id>`.

If a URL, extract the `run_id` and download:

```bash
RUN_ID=<extracted>
DEST=/tmp/maestro-android-$RUN_ID
gh run download "$RUN_ID" --repo mobile-dev-inc/Maestro \
--name maestro-root-dir-android -D "$DEST"
```

If the directory has no `tests/demo_app/passing/` subtree (e.g. iOS artifact, or the run failed before tests), report that and stop.

## Where to Look

Only `<artifact_root>/tests/demo_app/passing/` is signal. `failing/` is the negative-path suite — its commands are *supposed* to fail. Ignore it entirely.

```bash
cd <artifact_root>/tests/demo_app/passing
grep -l '"FAILED"' commands-*.json
```

The status field lives at `.[].metadata.status`. Possible values: `COMPLETED` (success) and `FAILED` (terminal failure). `commands-*.json` is the **only** authoritative source — `screenshot-❌-*.png` is not.

**`commands-(retry).json` is the canonical example.** This flow exercises the `retryCommand` directive: a tap fails, Maestro retries it, the retry succeeds, and the flow passes. In the artifact:

- All entries (including the recovered `tapOnElement`) have `metadata.status = "COMPLETED"` — Maestro writes only the final outcome of a recovered command, not the failed attempt.
- A `screenshot-❌-<ts>-(retry).png` still exists, captured at the moment of the failed attempt before the retry succeeded.
- The grep above does **not** match this file (no `"FAILED"` string anywhere), so the agent never investigates it. This is correct behaviour.

The risk is treating the `❌` screenshot as evidence of failure on its own. Don't. Trust the candidates from grep — they are the flows whose JSON contains a terminal `FAILED`. Any other `❌` screenshot in `passing/` is from a recovered retry and is irrelevant.

## Per-Flow Investigation

For each candidate flow returned by the grep above:

1. **Open `commands-(<flow>).json`.** Locate the entry whose `metadata.status == "FAILED"` — that's the terminal failure. Record: command type (the single key under `command`), error message, `metadata.timestamp` (ms epoch), `metadata.sequenceNumber`, `metadata.duration` (≈17000 ms = default-wait timeout, indicating UI-blocked, not crash).
2. **Open the screenshot `screenshot-❌-<timestamp>-(<flow>).png` whose timestamp matches the FAILED entry.** This is the highest-signal artifact — JSON and logs alone misclassify when a system dialog is invisible to them. Always read it. Ignore `❌` screenshots from other timestamps in the same file — those are recovered retries.
3. **Grep `maestro.log`** for the flow name and the timestamp window for surrounding driver activity, gRPC errors, and stack traces.

## Classification

| Screen at failure | Class | Where the fix lives |
|---|---|---|
| Expected app screen, but selector wouldn't match | Test flow issue | `e2e/demo_app/.maestro/**/*.yaml` |
| Android / GMS system dialog blocking the app | Maestro driver gap (auto-grant / auto-dismiss missing) | `maestro-client/src/main/java/maestro/drivers/AndroidDriver.kt` or `.github/workflows/test-e2e.yaml` (emulator pre-config) |
| Pixel launcher visible behind a system dialog | Same — dialog stole focus across `launchApp` / `clearState` / `stopApp` | Same as above |
| App crash dialog / blank screen | App-side issue | Out of scope — report only |
| Stack trace from `maestro.*` in `maestro.log` | Maestro framework bug | `maestro-client/` / `maestro-orchestra/` / `maestro-android/` |

For anything classified as **driver gap** or **framework bug**, dispatch an `Explore` subagent into the relevant module(s) to find the offending code path before proposing line numbers. Never guess.

## Cascade Detection (critical)

A single system dialog can fail many subsequent flows because `clearState` / `launchApp` / `stopApp` don't dismiss system overlays. List FAILED entries by ascending timestamp; if downstream screenshots show the *same* overlay, group them under the **root trigger** (the earliest flow that raised the dialog). Propose **one** fix per root, not N fixes for the cascade.

## Required Output Format

Return this structure verbatim — the caller parses it:

````markdown
## Summary
<one-line: how many flows failed, how many root causes, how many cascade groups>

## Root causes (caller acts on each)

### 1. <flow-or-group-label> — <classification>
- **Trigger:** <what raised this>
- **Affected flows:** <flow1>, <flow2>, ... (N flows)
- **Cascade?** yes/no
- **Proposed fix:**
- File: `<exact path>:<line range>`
- What it does: <one sentence>
- Diff (unified or pseudo):
```
- old line
+ new line
```
- Side effects: <if any, else "none">
- Verified via Explore? <yes — line numbers confirmed | no — fix is in CI yaml or test flow yaml, line numbers from direct read>

### 2. ...

## Out of scope
- `<flow>` — <reason, e.g. app crash, environmental>

## Notes
<anything the caller should know but isn't a fix proposal>
````

## Constraints

- Do NOT edit, write, or commit any file. Diagnosis only.
- Do NOT propose a separate fix for each member of a cascade group.
- Do NOT guess at Kotlin line numbers — verify via Explore subagent first.
- Do NOT report `failing/` flows as failures.
- Do NOT trust `❌` screenshots on their own. The grep filter (`grep -l '"FAILED"' commands-*.json`) is the source of truth for which flows had a terminal failure. A `❌` screenshot inside a flow whose JSON has zero `FAILED` entries is from a recovered retry and is not evidence of regression. `commands-(retry).json` is the canonical example — see "Where to Look" for details.
- Always read the screenshot, even when JSON+log seem to point clearly at a cause — the screenshot is what reveals system overlays.

## When to Stop and Hand Back

Hand back as soon as every passing-suite FAILED command has been either (a) classified with a proposed fix, or (b) classified as out-of-scope with a reason. Do not loop, retry, or attempt to apply fixes.
151 changes: 151 additions & 0 deletions .claude/skills/bump-android-version/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
---
name: bump-android-version
description: Use when bumping Maestro's Android compileSdk/targetSdk to a new API level and validating end-to-end against the test-e2e GHA workflow until the test-android job is green.
---

# Bump Android Version

## Overview

Drive an Android-version bump in Maestro commit-by-commit on a single feature branch: edit gradle, rebuild the on-device driver APKs, push a draft PR, dispatch `test-e2e.yaml` against the new system image, watch the `test-android` job, and diagnose+fix any newly-failing `passing/` flows. Loop until green.

## When to Use

- User asks to bump Maestro's Android `compileSdk` / `targetSdk` (e.g. "bump to API 36").
- User wants to validate Maestro against a new Android system image.

One commit per logical step. Don't bundle the gradle bump and the APK rebuild — they need to be separately revertible. After a failing run, fixes are committed and the workflow re-dispatched. Loop until green; do not stop on the first re-dispatch.

**The loop only acts on `passing/` failures.** `tests/demo_app/passing/` is the regression-detection suite — its flows are expected to pass, so a failure there is a real signal. `tests/demo_app/failing/` is the negative-path suite (its flows are *expected* to fail) and is ignored end-to-end. The diagnose agent rejects `failing/` artifacts and rejects retry-recovered flows in `passing/` (see [`.claude/agents/diagnose-maestro-failure.md`](../../agents/diagnose-maestro-failure.md)). If the run is otherwise green except for `failing/`, that's a green run.

## Pre-flight

- Working tree clean (`git status`).
- `main` up to date: `git fetch origin && git checkout main && git pull --ff-only origin main`.
- The workflow's `workflow_dispatch` inputs `android_version` (and optional `app` / `flow`) must exist on the branch.
- `android_version` is a `choice` input — its options live in `.github/workflows/test-e2e.yaml`. If `<new>` isn't already listed (e.g. bumping to API 37), add it to the `options:` list in a separate commit before dispatching, otherwise the workflow rejects the input.
- The `validate-inputs` job rejects `android_version <= android-29`. Bumps below API 29 aren't supported.
- Branch name: `bump-android-api-<old>-<new>` (e.g. `bump-android-api-34-36`).

```bash
git checkout -b bump-android-api-<old>-<new> origin/main
```

## Commit 1: bump compile/target SDK

Read current values from `maestro-android/build.gradle.kts` (`compileSdk` and `targetSdk`). Tell the user the current values verbatim, then ask for the new target:

> Current `compileSdk` / `targetSdk` = `<N>`. What's the new API level?

Edit both lines. Also `rg -n "compileSdk\s*=|targetSdk\s*=" --type kotlin --type-add 'kotlin:*.kts'` to catch consistent occurrences in other modules — only update those that intentionally track the same SDK. Commit:

```bash
git add maestro-android/build.gradle.kts <other touched files>
git commit -m "chore(android): bump compile/target SDK from <old> to <new>"
```

## Commit 2: rebuild driver APKs

```bash
./gradlew :maestro-android:assemble :maestro-android:assembleAndroidTest
```

The build's `copyMaestroAndroid` / `copyMaestroServer` finalizers update three checked-in files. Stage and commit *only* those:

```bash
git add maestro-client/src/main/resources/maestro-app.apk \
maestro-client/src/main/resources/maestro-server.apk \
maestro-client/src/main/resources/maestro-android-source.sha256
git commit -m "chore(android): rebuild driver APKs against API <new>"
```

If `assemble` fails, **stop and surface the failure to the user** with the gradle error and a suggested fix. Do not proceed without rebuilt driver APKs. Once we have assemble fixed and new drivers are commited, we can proceed to the next step.

## Step 3: push draft PR + dispatch

```bash
git push -u origin bump-android-api-<old>-<new>
gh pr create --draft \
--title "chore(android): bump compile/target SDK to <new>" \
--body "Validates Maestro against API <new>. Driver APKs rebuilt. Test-e2e dispatched with system-images;android-<new>;google_apis;x86_64."

gh workflow run test-e2e.yaml --ref bump-android-api-<old>-<new> \
-f android_version="android-<new>"

# capture the run id
sleep 3
gh run list --workflow test-e2e.yaml --branch bump-android-api-<old>-<new> --limit 1 --json databaseId,status
```

Always pass `-f android_version=android-<new>` — the default is `android-32` and would silently re-test the old API. The workflow constructs the full `system-images;...` string internally. `app` and `flow` are optional narrowing knobs (defaults: `demo_app`, all flows); leave them off for the full bump-validation run.

## Step 4: watch the test-android job

```bash
gh run watch <run_id>
```

Block in the foreground; the run is the bottleneck. **Green here means every `passing/` flow passed (terminally — retry-recovered counts as passed). `failing/` outcomes do not affect the verdict.** On green: update PR from draft → ready, done.

On red, download artifacts:

```bash
gh run download <run_id> --name maestro-root-dir-android -D /tmp/maestro-android-<run_id>
```

## Step 5: diagnose (delegated to subagent)

Dispatch the **`diagnose-maestro-failure`** subagent with the artifact directory `/tmp/maestro-android-<run_id>` (or the GHA run URL — the subagent handles both). It diagnoses each passing-suite failure in isolation and returns a structured report grouped by root cause (cascades collapsed). It does **not** apply fixes — that's your job here.

When the subagent returns:

**Fixes go in Maestro source — not in `.github/workflows/test-e2e.yaml`.** A failing flow on a new API level reflects a behaviour Maestro users will also hit on their own machines/CI, not just ours. Patching the GHA workflow (e.g. extra `adb shell settings put …`, command-line tweaks, AVD setup steps) hides the regression from real users and ships a Maestro that's only green inside our CI.

Acceptable fix targets and their roles are documented in [`AGENTS.md`](../../../AGENTS.md) (modules `maestro-android/`, `maestro-client/`, `maestro-orchestra/`, `e2e/demo_app/`). Driver-behaviour fixes belong in `maestro-android/` or `maestro-client/`; fixture-only issues belong in `e2e/demo_app/`. Editing the driver source means rebuilding the driver APKs (`./gradlew :maestro-android:assemble :maestro-android:assembleAndroidTest`) before re-dispatching.

Valid edits to `test-e2e.yaml` during this loop are workflow-shape changes (matrix, retention, dispatch inputs) — and the narrow exception below. Anything else (e.g. extra `adb shell settings put …` to mask a Maestro driver gap) is off-limits. If the subagent proposes a workflow patch, push back: ask it (or yourself) where the same fix would live in `maestro-android/` or `maestro-client/` so users on the new API level inherit it automatically.

**Exception — third-party app first-run UI not triggered by a Maestro API.** A pre-installed app's own onboarding (Chrome Welcome / "Make Chrome your own", browser default-app picker, Play Protect prompts that fire at app launch) is environment-harness state, not driver behaviour. Maestro doesn't expose a public API that triggers them — they're side effects of the OS image we picked. These may be pre-disabled in the AVD setup step (typical knobs: `setprop debug.chrome.command_line`, `settings put global …` flags). Test: *if Maestro called nothing related to this dialog, would it still appear?* If yes → CI workaround is fair game. If no (e.g. `setLocation` triggers GMS Location Accuracy via `FusedLocationProviderClient` inside the driver) → fix the driver instead.

1. For each root cause in its report, present the proposed diff to the user (file path + one-line summary + diff + side effects). Reject any proposal that targets `test-e2e.yaml` for driver-behaviour reasons before showing it — re-scope to Maestro source first.
2. **Ask consent explicitly per fix:**
> Proposed fix for `<root cause>`: `<one-liner>`. Apply?
3. On approval: edit, commit with a focused message — don't bundle unrelated fixes.
```bash
git commit -m "fix(android-<new>): <summary>"
```
4. On rejection: skip that fix; record what was rejected so you don't re-propose it next iteration.

After all approved fixes for this iteration are committed:

- **If any fix was applied:** push and re-dispatch, then loop back to **Step 4** with the new `run_id`.

```bash
git push
gh workflow run test-e2e.yaml --ref bump-android-api-<old>-<new> \
-f android_version="android-<new>"
```

- **If no fixes were applied** (everything was out-of-scope, or the user rejected every proposal): pause and ask the user how to proceed. Do not keep dispatching.

## Termination

Stop when `test-android` is green on the latest run. Then:

```bash
gh pr ready <pr_number>
```

If the loop has gone three iterations without progress (same flow keeps failing for different reasons after fixes), pause and ask the user how to proceed — don't spin indefinitely.

## Anti-patterns

- **Bundling gradle + APK rebuild into one commit** — kills bisectability. Always two commits.
- **Dispatching with default inputs** — `android_version` defaults to `android-32` and hits the old API. Always pass `-f android_version=android-<new>`.
- **Forgetting to extend the `android_version` choice enum** — the workflow rejects API levels not listed in `options:`. Add `<new>` to the enum in a separate commit before the first dispatch.
- **Auto-applying Maestro source fixes without consent** — every Kotlin patch needs explicit user approval first.
- **Patching `.github/workflows/test-e2e.yaml` to mask a driver behaviour gap** — workflow band-aids hide the regression from users running Maestro outside our CI. Fix `maestro-android/` or `e2e/demo_app/` instead so the fix ships with the driver APKs.
- **Treating `failing/` artifacts as regressions** — that suite is expected to fail.
- **Treating `❌` screenshots in `passing/` as regressions on their own** — when `retryCommand` recovers a failed attempt, Maestro writes only the final `COMPLETED` entry to `commands-(<flow>).json` but leaves the `❌` screenshot from the failed attempt on disk (e.g. `screenshot-❌-<ts>-(retry).png` for the `retry` flow itself). The diagnose agent uses `commands-*.json` containing a terminal `FAILED` entry as the source of truth — orphan `❌` screenshots without a matching FAILED entry are noise.
- **Committing on `main`** — always work on the bump branch. Verify with `git status` before every commit.
- **Skipping the screenshot read** — the screenshot tells you what `maestro.log` and the JSON cannot. It's the highest-signal artifact. Always read it.
Loading
Loading