Release 107 feedback: clean desktop chat previews - #84
Open
alichherawalla wants to merge 318 commits into
Open
Conversation
startModelServer is async (it awaits pickFreePort and listen); two tests called it unawaited and fetched immediately, passing only when the event loop happened to run listen first. On this machine's scheduling the fetch consistently won (ECONNREFUSED in 4ms) and the whole db-suite coverage report was suppressed with it. One word at two call sites; 7/7 green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The gate's own measure drove these: branches 49.7->60.3 (floor 57) and functions 41.8->60.0 (floor 52) on the code this branch adds, with statements/lines at 99.5. - use-runtime.integration.dbtest: the real composition on a real DB with only electron and the native helper mocked - lazy singleton, a reminder end to end through the real worker, waitForOutcome timeout, and the approval-hook probe across both hook names. - use-driver unit: the reader/non-reader routing matrix on a structural fake (the real-SQLite behaviour stays proven in the dbtest suite). - native-helper unit: candidate-miss, first-existing-candidate, the non-zero-exit-with-stdout salvage, and spawn failure - all degrade to reported failures, with electron and child_process mocked at the boundary. - gate-host: park-signal corners (immediate and later whenActionParked, onGateParked subscribe/unsubscribe, pending count + abandon). - extension: the spec-table completeness sweep (every title/buildArgs/ formatResult), schemas + systemHint, and the edited/poisoned/no-detail outcome arms. - emit: escaped quotes inside balanced-object strings; helper-logic: long invalid line truncation; semantic-rail: lookup with no kind; verification: unparseable calendar start never lists. - devices-sync e2e: the pro implementation import loads lazily behind PRO_PRESENT - a static import failed spec collection in a core-only checkout before the guard could skip. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Core carried pro/main/licensing/__tests__ (tests only, no implementations) even though .gitignore already ignores /pro/ - tracked files override the ignore. With desktop-pro access restored and the files verified byte- identical there, core stops tracking them: pro/ is the desktop-pro clone (the submodule-style layout CLAUDE.md describes), fully ignored by core. This is also what made the fast suite collect orphan tests in core-only checkouts; the hasPro glob gate covers anyone without access. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…id/use The first Windows CI run of the engine branch (31778949364) failed its typecheck on 'Cannot find module @offgrid/use' and '@offgrid/sync/portable': windows-build.yml predates the shared adoption and never checked out ../shared at all, and release.yml (both build-mac and build-win) builds only packages/sync - file-dep prepare alone does not emit @offgrid/use's dist types before the typecheck. windows-build now mirrors release.yml's shared checkout (matching-branch ref, main fallback, root ci) and all three shared steps build packages/use alongside packages/sync. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…x 17) Calendar, reminders (tasks), and mail run through LOCAL Outlook COM via PowerShell - the write lands in Outlook's local store and syncs when the network returns, matching the mac EventKit/Mail behaviour instead of failing offline like a cloud API. The scripts print the same one-JSON-line contract the mac helper speaks, so parseHelperResponse is shared. open goes through Electron's shell; message is refused honestly (iMessage is macOS- only). Microsoft Graph is the online-only fallback for setups without local Outlook: the port and fallback logic ship boundary-tested (only consulted when COM says not-registered AND the port reports signed-in); the OAuth wiring is a recorded fast-follow, so production passes no port and the failure names both paths. The runtime picks the rail by platform in exactly one place; a DSP test drives both rails through one dispatch with zero caller changes. 21 tests: script building and quoting (embedded quotes doubled), the COM-unavailable matcher, every mapping and refusal, ordinary-error vs absent-Outlook vs signed-out-Graph vs Graph-failure paths, and the never-throws guarantee. Follow-ups (win tool exposure, Outlook read-back verbs, Graph sign-in) recorded in the checklist. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Run 31779453356: shared checkout resolved at the matching branch, @offgrid/use built, typecheck + bundle passed, NSIS packaged, 414MB installer artifact uploaded. Cert + real-Windows model-load smoke remain recorded release items. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CI's dependency-boundaries gate caught the circle the category seam introduced: tools.ts imports selectToolExtensions, and extension-select imported ToolExtension back from tools.ts (a type import still counts). The selector only needs the category field, so it now asks for exactly that - a structural CategorizedExtension, generic over the caller's richer type - and imports nothing from tools.ts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… and routines follow The lead's steer plus R1 field feedback, in the build doc: R1 is done (PRs open); R2 (~5-6 days) ships all four rails chat-driven on both platforms - Windows exposure, the browser rail, the supervised vision rail (UI-TARS- 1.5-7B) - plus Approval UX v2: inline approval cards in chat, outcome feedback everywhere via the pro approval-executor migration to the engine gate, and risk-tiered gating (reversible mutations auto-run with verified confirmation + Undo; sends keep the gate). Notices-you moves to R3, routines to R4. The R1 checklist records the three pro-path UX verdicts that drove the approval rebuild. One versioned release ships after R2. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
R1: chat actions on the durable engine
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ook subset (R2-A1) specsForPlatform defined once in the logic file: darwin keeps the full eight; win32 exposes exactly the engine-routed set the local Outlook rail supports (calendar_create_event, reminders_create, mail_send, open_url); any other platform exposes nothing and stays unregistered. The model-facing hint follows the platform and never promises a tool it does not have (the Windows hint speaks Outlook, no iMessage or contact lookup). The extension carries its platform (injectable for tests); canHandle and execute refuse mac-only tools even when a model hallucinates them. The old darwin-only registration test updated to the new contract; 9 new platform tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…refuses (R2-A2) The non-engine path on Windows: makeWinInlineRunner handles system.openURL through the injected opener (Electron's shell at wiring) and refuses every other verb honestly, so nothing silently impersonates the Swift helper. The production boundary picks the inline runner by platform in exactly one place. 3 tests through the injected opener: open, opener failure degrades to a reported error, unknown verbs refuse. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…R2-A3) List scripts for the tasks folder (olFolderTasks 13, incomplete only) and a calendar range (olFolderCalendar 9, IncludeRecurrences + Restrict on [Start], locale 'g' formatting as Outlook filters expect), speaking EXACTLY the mac helper's result shapes - so makeReadBackVerifiers and buildRegistry work unchanged over either OS. makeOutlookNativeReader exposes them behind the mac command names, reads only, refusing the rest. The runtime picks the reader by platform in the same single place the rail is picked. 4 tests: script content per folder, the reader mapping + refusal, and the shared verifiers composed over a scripted PS boundary. R2 section A complete. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…hrough) inlineRunnerForPlatform and pickByPlatform extracted and exported so the one place an OS chooses an implementation is proven, not assumed: darwin gets the Swift helper runner, win32 gets the shell runner (refusals + opener- failure degradation exercised, including electron's inert shell under vitest), and both runtime picks (rail executor, read-back reader) go through the same tested helper. Plus the win runner's missing-url default. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… profile env Two report-ownership fixes for the merged new-code gate, which sums denominators across reports so a file mapped by both suites but exercised in one reads half-covered: - The unit report excludes the two new Electron/subprocess shells (use-runtime.ts - covered by its dbtest on a real DB; win-powershell.ts - the powershell.exe spawn twin of native-helper, its parsing is the shared covered parseHelperResponse), with the same by-name-with-reason precedent as the other excluded shells. - The db report excludes three files it only LOADS through use-runtime's import graph but never set out to measure (semantic-rail-win, the tool extension + its logic) - they are owned by the default run per the db config's own complementary-not-second-opinion doctrine. And use-runtime.integration.dbtest now sets OFFGRID_USER_DATA in beforeAll and restores it in afterAll: process.env is shared across files in a worker, and leaving it pointed at a deleted temp profile broke whichever dbtests ran after. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…h undo (R2-B1) The app half of Approval UX v2's tier policy, over the engine's new capability model (shared feat/use-approval-tiers): reversibility IS the undo capability, and that is what earns auto-run. - The Swift helper gains the undo verbs: reminders.delete and calendar.deleteEvent fetch by the exact id the create returned and remove it - undo of the thing itself, never a search-and-guess. Rebuilt. - The mac rail surfaces the created id as effectId; the Outlook adapter maps the same verb names onto GetItemFromID + Delete, so buildRegistry's undo capabilities are one code path across platforms. - Calendar and reminder handlers declare undo -> they auto-run with a verified confirmation instead of a pre-approval gate; messages and mail declare none -> they still gate. The runtime exposes undo(record). - Proven on a real DB: propose -> auto-run (gate never consulted) -> read-back verified -> effectId stamped -> undo deletes exactly that item. The chat Undo chip and the inline approval card land with B3. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…run-now (R2-B3, main + preload) The gate host gains an injectable inline surface: when the app registers it (actions-ipc at setup), a gated action with no pro queue listening PARKS and broadcasts the card request to the chat instead of auto-running - the free build's sends get a consent surface for the first time. Unregistered (tests, headless), behaviour is unchanged; a listening pro queue still wins until the B4 migration. The worker gains an outcome feed; the runtime enriches it with undoability (effectId + handler capability); actions-ipc broadcasts gate-pending and outcome events and handles resolve-gate/undo through fail- closed parsers (parseGateDecision accepts exactly the three decision shapes; undo revalidates the record). The preload bridges the four methods, swept. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…(R2-B2/B3) ActionGateDock renders pending gates as cards in-flow above the composer: the resolved values, the risk tier (amber mutate / red irreversible), and Approve / Edit / Reject resolved through the engine gate - an edit sends the changed args for re-binding and the re-gated card returns as its own event. Outcomes land in the same surface: 'Done - verified' with Undo when the handler can reverse the effect (the B2 chip), the honest needs- attention text otherwise, dismissable. Self-contained: it subscribes to the preload feed and never touches the chat's message model - all 39 existing chat behaviour suites pass untouched (the R1 lesson applied). Brand-conformant: mono, dense, emerald primary only. 8 component tests over the real logic with the feed as the only fake. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tcome fanout The inline-approval commit was below the new-code floor in four files; each gets its owner: - actions-ipc.test.ts pins the renderer's IPC contract (channel names, fail-closed decision/record parsing, gate-pending broadcast, outcome fanout with undoability) with electron and the runtime as the mocked boundaries. - The runtime dbtest now asserts onOutcome fans out the outcome enriched with undoable - the feed ActionGateDock subscribes to. - ActionGateDock tests grow the branch cases: mutate risk tone, edited outcomes never landing as rows, a failed undo reporting its detail, the poisoned error text, the three-row cap, and unmount unsubscribing. - vitest.db.config.ts: the db report no longer owns src/main/index.ts or renderer .tsx it merely loads through a jsdom journey - index.ts is entry wiring owned by the e2e tour, and .tsx is rendered-behavior surface owned by e2e + render tests everywhere else already. New-code gate: statements/lines 98.3, branches 71.0, functions 62.1 - all floors met. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ro PR #42) Approval rows carry action_id and the queue's verdict resolves resolveActionGate - the engine executes, verifies and journals; the row records only the outcome the queue observes. Retires the Windows-PRO watch-list entry and adds the model-transfer FileHandle flake to the watch list. (pro/ is gitignored in core - no submodule pointer to bump; the code ships in desktop-pro.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…nts (R2-C1) Ported design per the porting map: nanobrowser's injected dom module + browser-use's clickable-element detection and numeric indexing, as one pure in-page function graph. pageScriptSource() serializes the exact unit-tested functions for Runtime.evaluate - the injected code IS the tested code, and a test evaluates the serialized source to prove the graph is self-contained. Identity boundary built into the eyes: password / one-time-code fields are flagged (takeover markers) and their values never enter the snapshot - the agent cannot leak what it cannot see. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…(R2-C1)
snapshot / navigate / click / type / pressKey over raw CDP, with the Electron
webContents.debugger attach kept OUT of this module (CdpTransport is the
seam) so every dispatch decision is tested against a fake transport.
The takeover boundary is enforced here, not in the prompt: typing into an
identity field returns {reason: 'takeover'} with zero events dispatched -
prompt injection cannot talk the agent past a rule the driver refuses to
execute. Clicking one stays allowed (focusing the login form is how the
human takes over). Type selects-all first so prefilled values are replaced,
never appended to.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…parking (R2-C3 core) Stagehand-shaped step decisions, grammar-constrained (STEP_RESPONSE_FORMAT -> GBNF) and fail-closed parsed: free text, unknown actions, and non-http URLs (file:, javascript:) never become an action. The loop parks on the identity boundary - both when the driver refuses a credential field and when the model hands over voluntarily - and resumes after the user acts in the watched pane. A step budget bounds how far a hijacked page could steer even a fully fooled model; a missing element index is reported back, never clicked blind. Every boundary injected (driver, model, takeover wait); 11 tests pin the control flow, the parser matrix, and the injection stance in the prompt source. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
TakeoverCoordinator owns the identity-boundary handoff the same way the gate host owns approvals: park keyed by task id, an injectable pane surface, a fail-closed resolve, and - the safe default - resume immediately when no pane is registered so a task never wedges on a UI that is not there. WatchedBrowserPane reuses the ArtifactCanvas slide-over layout: the live step feed, and at the boundary a takeover prompt (Resume / Cancel) that states the privacy promise on the surface - 'Off Grid never sees your password or codes'. The live page is a main-process WebContentsView laid over the reserved region; this component owns the chrome, narration and handoff. Preload gains a browser namespace (resolveTakeover + onStep/onTakeover/onTaskState) mirroring actions. 11 tests: 5 on the coordinator's park/resume/cancel/no-pane paths, 6 on the pane render + IPC resolution. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rivable (R2-C3) The rail is now reachable from chat end to end. A web_task tool (cross- platform - Electron CDP is the same everywhere) proposes a durable web_task Action; the engine routes it to the browser rail, gates it for approval like any mutation, and on approve runs the watched loop in a WebContentsView the user sees. Takeover parks it at every identity boundary; the outcome reports back inline. - browser-rail.ts: registerBrowserRail (web_task on the browser rail, none_fuzzy on purpose - a web task is never auto-retried; re-running an order double-orders) + makeBrowserRailExecutor (Action -> run -> result, final URL as the effect handle). Both unit-tested. - use-runtime.ts: the device gains a browser branch; the live host is created lazily on first web_task. buildRegistry composes the browser rail. - browser-host.ts: the Electron shell (WebContentsView + CDP debugger as the driver transport + the local model as the step decider + step broadcasts) - excluded from in-process coverage like the other rail hosts, over the unit-tested collector/driver/loop/executor. - browser-ipc.ts: the watched-pane takeover handoff (resolve + broadcast), fail-closed, tested with electron mocked. - The web_task tool is engine-only: it never falls to the legacy pro queue (no connector runs a web task; with B4 the queue resolves the engine gate anyway), and refuses cleanly when no engine is wired. - WatchedBrowserPane mounts in MemoryChat; scrollTo added to the shared jsdom shim so the step-feed effect doesn't take down the render in tests. Tests: browser rail suites (collector, driver, loop, takeover, rail adapter, ipc, pane) + the engine/logic/platform tool suites, all green. The dbtest asserts web_task registers and routes to the browser rail. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Ported from @ui-tars/sdk (Apache-2.0), reduced to the supervised-tier verbs and retyped closed: click/double/right, drag, type, hotkey, scroll, wait, finished, call_user. Parses every coordinate spelling UI-TARS-1.5 uses (<point>x y</point>, (x,y), start_box=...), denormalizes 0-1000 to real pixels within the target bounds, and clamps an out-of-range prediction onto the screen. Fail-closed: an unknown verb or a point-less action is null, so the loop re-observes rather than clicking a guessed spot. Pure and screen-free; 17 tests over the verb set, the coordinate math, a Thought prefix, content escaping, and the junk matrix. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…adapter (R2-D) The vision rail's brain, all screen-free and tested; the host (screenshot + actuation + overlay) is the native shell, added next behind a capability gate. - vision-guard.ts: the supervised safety state machine. Kill switch (Esc) is terminal and outranks everything; a user touch pauses until they resume; a step budget halts a flailing model. canActuate() is the one gate the loop checks before every action. - vision-agent.ts: screenshot -> ground -> actuate under the guard, until the model reports finished, calls the user (handoff + resume), or the guard stops it. Re-checks the guard right before dispatch, so an Esc mid-decision actuates nothing more. Every boundary injected (screen, model, guard, takeover). - vision-rail.ts: the engine adapter. computer_task registers on the vision rail as a no-retry mutation (a live-desktop GUI action is never safely auto-retried); the executor maps a run to an ExecuteResult. Also: exclude the rail hosts (browser-host, vision-host) from the DB coverage report - the dbtest loads browser-host through use-runtime's import graph but never drives a display, so measuring it there dragged the merged branch/ function ratio for code the e2e/real-machine pass owns. 42 vision tests (parser 17, guard 7, loop 8, rail 3, + the earlier parser suite). computer_task added to the shared ACTION_TYPES enum. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ability gate (R2-D)
buildRegistry composes registerVisionRail and the device gains a vision branch;
the host is created lazily on first computer_task.
vision-host.ts is the Electron shell: screen capture (desktopCapturer), the
grounding model (llm with the UI-TARS action-space system prompt + the
screenshot), the Esc kill switch (globalShortcut), and actuation through an
ActuationPort. Actuation is CAPABILITY-GATED: synthetic input needs a native
addon (@nut-tree-fork/robotjs) plus Accessibility + Screen-Recording
entitlements and a notarization pass - a real packaging decision, not a silent
dependency. Until it lands, loadActuation() is null and the rail refuses
cleanly ('vision actuation is not available in this build') instead of
half-working; computer_task is NOT offered to the model, so the supervised
tier ships labeled or not at all. Excluded from coverage like the browser host.
The dbtest asserts computer_task registers and routes to the vision rail.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ted (D2b pending) The vision-rail brain (parser D1a, guard+loop+adapter D2a) is done, tested and wired into the engine. What remains is native (D2b: the actuation addon + entitlements + real-machine pass) and the model catalog entry (D1b). Records the shared-branch ref-matching note and the local flake-retry guidance. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Too many files changed for review (486 files, 100 file limit). Bypass the limit by tagging |
|
Too many files changed for review (499 files, 100 file limit). Bypass the limit by tagging |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Verification
Summary by CodeRabbit
New Features
Bug Fixes
<think>and__LABEL:markers are hidden.