ProjectDiscovery fork of go-rod

This repository is a fork of go-rod/rod v0.116.2, maintained by ProjectDiscovery for the headless browser needs of nuclei, katana, httpx and related tools. The module path is unchanged, so existing code moves to the fork with a replace directive and no import edits.

Fork changes

Upstream go-rod is no longer released. These changes are the reason for the fork.

• Local browsers talk over a pipe. Browser.Connect launches Chrome with --remote-debugging-pipe on Linux and macOS, so no debugging port is open for other local processes to attach to. Windows falls back to a port, since Go does not pass extra file descriptors to child processes there. Launcher.Launch still returns a URL for callers that need one.
• Connect cleans up after itself. A cancelled context fails before a browser starts, and a browser that cannot finish connecting is closed rather than left running.
• A page pool that checks its pages. NewHealthPagePool hands out at most N pages. Before reuse, each page is checked: the browser answers, the target still exists, and the renderer runs script. A failed page is closed and its slot is freed, so a closed target is never handed out again. Returned pages go back to about:blank and popups they opened are closed. Cookies and storage are kept, so use a separate browser context when that state must not carry over.
• Access to the paused network request. HijackRequest.Event() returns the paused Fetch.requestPaused event, including the request id, the frame and the resource type.
• Service worker requests. A router attached to one page does not see requests that page's service worker makes itself, because those belong to the worker target. Browser.HijackRequests does see them. This is documented on Page.HijackRequests and covered by a test.
• Chrome for Testing by default. Without a Bin, the launcher downloads the latest Stable full Chrome, and the archive hash is pinned on first download. HeadlessShell(true) selects the smaller headless-shell build. Musl hosts use the installed browser instead of a download that will not run.
• Browser exits with its parent. Linux uses Pdeathsig, Windows a job object, and macOS and the BSDs a watchdog process. A crashed test run no longer leaves browsers behind.
• Opt-in compliant mode. Compliant(true) removes the automation switch, so navigator.webdriver is false, and makes the user agent and client hints match the real browser, without the HeadlessChrome token.
• Examples and page tests run offline. They use local fixture pages and a local HTTP server instead of live sites. Some launcher tests still download a browser and need network access.

Upstream overview

Documentation | API reference | FAQ

Rod is a high-level driver directly based on DevTools Protocol. It's designed for web automation and scraping for both high-level and low-level use, senior developers can use the low-level packages and functions to easily customize or build up their own version of Rod, the high-level functions are just examples to build a default version of Rod.

中文 API 文档

Features

• Chained context design, intuitive to timeout or cancel the long-running task
• Auto-wait elements to be ready
• Debugging friendly, auto input tracing, remote monitoring headless browser
• Thread-safe for all operations
• Automatically find or download browser
• High-level helpers like WaitStable, WaitRequestIdle, HijackRequests, WaitDownload, etc
• Two-step WaitEvent design, never miss an event (how it works)
• Correctly handles nested iframes or shadow DOMs
• No zombie browser process after the crash (how it works)
• CI enforced 100% test coverage

Examples

Please check the examples_test.go file first, then check the examples folder.

For more detailed examples, please search the unit tests. Such as the usage of method HandleAuth, you can search all the *_test.go files that contain HandleAuth, for example, use GitHub online search in repository. You can also search the GitHub issues or discussions, a lot of usage examples are recorded there.

Here is a comparison of the examples between rod and Chromedp.

If you have questions, please raise an issues/discussions or join the chat room.

Sponsors

Rod is sponsored by many organizations and individuals, thank you for your support!

Please contact yad@ysmood.org if you want to be listed here.

Browser testing via

Join us

Your help is more than welcome! Even just open an issue to ask a question may greatly help others.

Please read How To Ask Questions The Smart Way before you ask questions.

If you want to contribute please read the Contributor Guide.