Skip to content

feat(shared-browser): add natural input and improve display controls - #274

Open
RyanEwen wants to merge 5 commits into
omercnet:mainfrom
RyanEwen:shared-browser-interaction-quality
Open

RyanEwen wants to merge 5 commits into
omercnet:mainfrom
RyanEwen:shared-browser-interaction-quality

Conversation

@RyanEwen

@RyanEwen RyanEwen commented Oct 4, 2026

Copy link
Copy Markdown

Summary

Make Shared Browser behave more like a browser you are using directly, with natural input and simpler display controls.

Changes

  • Forward mouse hover with cursor feedback, wheel scrolling, right-click, double-click, continuous dragging and touch gestures. Native scrollbar dragging updates the page before release.
  • Support direct keyboard input on desktop/web, including shortcuts and committed composition text. Add a native-phone software-keyboard relay for basic typing and Backspace, with explicit Done submission for composed text.
  • Replace click/drag mode selectors and rows below the browser with compact address-bar menus and a Send keys submenu.
  • Group resolutions by aspect ratio, add filled-star favorites and presets through 2560 × 2560, and support sharper Pixel captures.
  • Add Fit/Actual size with correct local overflow behavior. Changing mobile emulation preserves the viewport; device defaults apply on first control rather than while observing.
  • Improve mobile address readability, frame retention and viewer-expiry recovery. Add owned Linux Xvfb support for native desktop hover capability.
  • Align the manifest with the Paseo 0.11 SDK APIs already used upstream, retaining the tested 0.11.0-beta.3 allowance.

Safety and limits

Human control retains priority over agents. Ordered gesture channels release held state when ownership or geometry changes; uncertain actions are not replayed. Resolution and emulation changes affect the shared browser, while Fit/Actual size remains local to each viewer.

Images use JPEG transport. Local plain-text paste is supported, but remote clipboard synchronization is not. Native hardware shortcuts and full live IME composition are outside the phone relay. The Linux hover correction requires existing Xvfb and does not install it automatically.

Validation

Unit coverage includes pointer mapping, input ordering, cancellation, keyboard handling, menus, display preferences, frame settlement and viewer recovery. Development checks also exercised isolated Chromium input and mounted DOM interactions. Physical-phone keyboard behavior is not established by those checks.

The interaction branch passes 264 unit regressions, the real Chromium browser smoke, TypeScript, pinned Biome checks and actual Paseo client/server compilation. Tests use owned temporary profiles and runtime resources; the default Paseo daemon is untouched.

AI disclosure: this comment and the related code were written with the assistance of AI.

Add direct keyboard, pointer and touch controls, display presets and favorites, toolbar menus, sharp captures, native desktop hover, and stable frame buffering. Preserve page state during mode changes and reduce redundant frame transfers. Include regression tests and fork reconciliation notes.
… review

Apply pinned formatting, type the viewport fixture, align the smoke with default capture quality, and document isolated validation and the supported SDK.
Resolve CSS auto cursors over selectable glyphs and editable surfaces while preserving explicit cursor styles and existing control authority. Cover native text, padding, vertical text and shadow content with isolated Chromium checks.
Use recognizable height labels, add 1440-wide desktop choices, and project favorites in picker order without rewriting saved IDs. Include shared contract tests in the normal regression command.
Use each icon control accessibility label as its native web hover title, including contextual mode labels. Keep native accessibility and shared toolbar behavior intact.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant