Repository navigation
Navigator: a step that changed nothing and sent no request does not resolve - #302
Conversation
Without an expected URL, resolveState accepted the first code block that ran without an error. A click on the wrong element (same page, nothing changed) came back to the Tester as a successful interact(). Compare the page after the block with the page before it and treat an unchanged page as a failed attempt, fed back to the AI like the existing "page state did not change" case. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
One question: capture() runs right after the attempt — a delayed effect (animation, background request) could be misread as "nothing changed". Is there a settle before capture, or is the retry feedback enough? |
Before calling a step "nothing changed", count it as done when it sent an XHR/fetch to the app's own domain (subdomains included, third parties such as analytics excluded), or when the page changes or the app sends a request within 1s after the first capture. Action now counts app requests as they start, so a slow request that has not answered yet is still seen. The "XHR/fetch to the app" check moves into isAppRequest() and is shared by Action, XhrCapture and the new waitForAppRequest(). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…method attemptChangedPage() runs the request, capture and settle checks as early returns in order, so the loop body only records progress or the failure. Same behaviour. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…uccess # Conflicts: # CHANGELOG.md # src/ai/navigator.ts
Explorbot Self-RegressionCommit
Change compares it with the median passing time of the last 5 runs; ±20% or more is flagged. Run details
Session analysis — basic (native): Session AnalysisIssue listing works: title search, status filtering, opening a detail page, and creating an issue from the top navigation all behave as expected. The toolbar creation path could not be confirmed because its test never asserts the new row in the action log. Coverage
What works
Execution Issues
Session analysis — slow (native): Session AnalysisIssue creation works end to end: a new issue submitted from the drawer form appears in the issues list while existing issues remain visible. No defects or friction were observed on this path. Coverage
What works
Session analysis — gated (native): Session AnalysisThe toolbar New Issue flow creates an issue that appears in the issues table with the entered title, priority and bug label. The create form has no Status field, so the status expectation was not tested as written. Coverage
UX issues
Execution Issues
|
The merge left both PRs' pushes in formatSessionLog: #303's numbered push into the current page group (with #300's messages appended after it), and #300's leftover groups.get(), which crashed now that groups is an array. Every Pilot session-log test failed on main. Append the messages before the one numbered push, and start the landed page's group after a navigating action, so the action stays under the page it ran on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem
Navigator.resolveStatewithout anexpectedUrl(the pathinteract()takes) setresolved = attempt.ok. The first code block that ran without throwing ended the loop as a success, even if it did nothing.Seen in Langfuse session
ThoroughIndustrialSalmon765: the Tester askedinteractto open project settings. Four blocks failed, thenI.click({role:'link', text:'Project'})ran fine but most likely hit another visible "Project" link on the same page. URL, ARIA and HTML were all unchanged, yetinteractreturnedsuccess: true("Successfully executed: Navigate to project settings…"). The Tester then improvised a directamOnPage.Fix
When no URL is expected, a block that ran without an error is checked before it counts as done. It counts as done when any of these holds:
Actionnow counts XHR/fetch requests as they start during the attempt (sentRequests), so a slow request that has not answered yet still counts. "The app" means the base URL's host and its subdomains (isSameHostFamily), so analytics and other third-party calls are ignored.Diff.hasChanges(): URL, ARIA, HTML).waitForAppRequest, new), or the DOM changes (waitForDomChange, existing) and a second capture shows a difference. This covers effects that land after the readiness wait, which can end after ~350ms of DOM quiet.If none holds, the attempt counts as failed: it is logged, added to the batch failures and fed back to the AI with
Ran without error but nothing on the page changed, mirroring the existingReached … but the page state did not changehandling. No prompt text changed.The "XHR/fetch to the app" check was written out in
ActionandXhrCapture; it is now oneisAppRequest()inurl-matcher.ts, used by both and bywaitForAppRequest().Trade-off
The 1s settle only runs when a block looks like it did nothing, so the normal path costs nothing. Steps that are done yet change nothing and send no request (hover, scroll, copy to clipboard) still count as failed and the Navigator moves to the next solution. An app that polls its own backend can make a no-op click look like it did something; that only makes the Navigator accept a click, as it always did before this PR.
Not in this PR
The same session showed the Navigator using the ARIA snapshot's
{img}placeholder as a link name ({role:'link', text:'{img}'}), which can never match. Fixing that needs prompt wording, so it is left for a separate change once the wording is agreed.Tests
analytics.example.net) does not.bun test tests/integration/passes (175). The 8 Provider/Langfuse-env failures intests/unit/fail the same way on cleanmain. Lint andtscclean on the changed files.src/ai/, so regression is queued behind the approval gate. Not started.🤖 Generated with Claude Code