Skip to content

feat(drivers): add the Hybrid Computer Use driver - #730

Open
hqhq1025 wants to merge 3 commits into
scaleapi:mainfrom
hqhq1025:feat/hcu-driver
Open

hqhq1025 wants to merge 3 commits into
scaleapi:mainfrom
hqhq1025:feat/hcu-driver

Conversation

@hqhq1025

@hqhq1025 hqhq1025 commented Nov 17, 2026 •

Copy link
Copy Markdown

Closes #729.

Adds Hybrid Computer Use as a Linux X11 computer driver on amd64, using the
published v1.0.1 archive. The archive, extracted executable and model-visible
MCP contract are pinned; mismatches fail readiness.

  • Forward HCU's AX and GUI tools while retaining X11Driver's standard computer
    controls and the existing tools policy.
  • Keep one isolated HCU stdio process per driver so references survive between
    calls. Bound initialization, reads and each worker call to 300 seconds;
    a failed session fails its queue without replaying input, and a later call
    starts a fresh session.
  • Set the system accessibility default before apps start and provide HCU with
    a private runtime directory and the desktop's session bus.
  • Include the driver alias, manifest regeneration helper, documentation and
    unit and integration coverage.

Validation

  • Full framework and protocol unit suites: 17,297 passed, 8 skipped.
  • make lint and the plugin API compatibility check pass.
  • The real GTK integration test passes after merging the current base branch.
    It installs the pinned HCU release, starts the app before the first AX call,
    checks the startup accessibility setting, finds and clicks a named control,
    and verifies both AX state and a file written by that same app process.
  • Real stdio subprocess tests cover a stalled call with an active or cancelled
    caller, queue failure, old-process cleanup, and a fresh subsequent session.

The documentation site should cover selecting the HCU driver, its tool policy,
and updating the release and MCP contract pins. Windows support is a separate
follow-up so its release-asset dependency does not block this Linux driver.

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge; no new actionable issues were found.

Summary

Adds the hcu driver for Linux X11 on amd64, with HCU accessibility and pixel tools beside the standard computer tools.

  • Linux X11 desktops can use HCU’s accessibility and graphical tools.
  • Apps started with an HCU desktop expose accessibility trees from launch.

Both previous findings are addressed. No new actionable issues found. Integration tests were not run during this review.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    Agent[Agent] --> Driver[HCU driver]
    Driver --> Queue[Ordered calls with deadline]
    Queue --> HCU[HCU stdio server]
    HCU --> Desktop[Linux X11 desktop]
    Driver --> X11[X11Driver standard tools]
    X11 --> Desktop
Loading

Reviews (2) · Last reviewed commit: "Merge remote-tracking branch 'origin/mai..." · Reviewed by Greptile

Pin HCU 1.0.1 release and executable hashes, and forward its AX and GUI tools.

Preserve Linux accessibility setup, add manifest regeneration, and test installation, server isolation and readiness offline.
@hqhq1025
hqhq1025 requested a review from a team as a code owner November 17, 2026 02:32
Comment thread tst/unit/drivers/hcu_driver_test.py Resolved

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a driver for Hybrid Computer Use (HCU)

1 participant