Skip to content

feat!: bound connection time and harden the transport - #9

Merged
Gikoskos merged 4 commits into
mainfrom
fix/transport-hardening
Nov 13, 2026
Merged

Gikoskos merged 4 commits into
mainfrom
fix/transport-hardening

Conversation

@artiom

@artiom artiom commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Bound the whole page connection setup with connect_timeout, shared by async and sync APIs. Clean up failed or cancelled setup without cancelling another operation's shared connection attempt.
  • Apply 5000 ms of client headroom once for schema mutations with server-side timeouts. Keep raw GraphQL sends and response waits unchanged.
  • Make deadline expiry win over late completion, end failed subscriptions on both sides, handle malformed error entries per operation, accept empty error lists, and make unsubscribe safe after socket failure.
  • Keep connection errors redacted without attached exception context or tokenized URL locals, harden log filtering, and propagate synchronous operation timeouts instead of continuing to poll.
  • Document timeout and logging behavior, generate the new connect option, and explicitly check schema exclusion from wheels and inclusion in source distributions.

Related issues

None.

Type of change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • CI checks

Checklist

  • ruff format --check .
  • ruff check .
  • mypy
  • pytest -m "not e2e" — 241 passed, 1 skipped, 5 deselected on Python 3.13
  • pytest -m e2e — 5 passed against a local BrowserQL runtime at c9b91f5, before the final abandoned-task callback fix
  • python scripts/codegen.py leaves no diff
  • python scripts/sync_schema.py --check
  • Tests and documentation updated without hand-editing generated output
  • Wheel/sdist build, twine check, fresh-wheel imports, and archive contents checked
  • actionlint 1.7.12 with ShellCheck 0.11.0; zizmor 1.32.0 offline
  • Title and commits follow Conventional Commits and public repository rules

Notes for reviewers

Live BrowserQL acceptance passed: all five tests ran against a local runtime, including async/sync happy paths, rejected authentication, malformed endpoints, and the missing-selector regression receiving the server's BrowserQLError mentioning 1500ms, rather than the client's timeout. No production endpoint was used.

The final follow-up retrieves shared connection-task failures when all waiters cancel and narrows the documented redaction scope to the WebSocket logger. Its cancellation regression failed before the fix and passes afterward; the full offline suite passes. Live tests were not rerun for this callback-only change.

Follow-up regressions also verify that suppressed cancellation cannot accept an expired connection deadline, stalled failed-setup cleanup is limited to five seconds while preserving the original error, and malformed log records are dropped before handler diagnostics can expose credentials.

Offline regressions cover stalled real-socket handshakes and socket closure, synchronous connection and operation deadlines, cancellation isolation, failed setup cleanup, timeout/completion races, stream termination, malformed and empty error payloads, error context and traceback-local redaction, and the operation-timeout truth table. The remaining offline skip is the existing optional source-parity test. A separate focused security scanner is unavailable; the consolidated code review covered security and privacy.

No runtime dependency, schema, package version, or changelog changes. Custom connection factories still receive the same URL string. Operation deadline expiry does not close the page; only connection timeout is terminal for that connection attempt.

Summary by CodeRabbit

  • New Features
    • Added an optional connection timeout for async and synchronous APIs. It defaults to the existing timeout; values at or below zero disable the connection deadline.
  • Bug Fixes
    • Improved timeout handling for page operations and connection failures, with clearer errors and safer cleanup.
    • Logs and error messages redact tokens and authorization values; unexpected GraphQL error formats now produce readable messages.
  • Documentation
    • Added guidance on timeout behavior, logging, and sensitive data that may appear in WebSocket frame logs.
    • Clarified package contents checked for wheel and source distribution releases.

BREAKING CHANGE: Opening a page now times out after connect_timeout (default: the timeout value, 30000 ms). Pass connect_timeout=0 to restore the previous unbounded wait.

Bound page connection setup with an independent deadline, apply server timeout headroom once, and harden operation completion and connection cleanup. Keep credentials out of connection error context and traceback locals, propagate synchronous timeouts, and check both distribution formats.

BREAKING CHANGE: Opening a page now times out after connect_timeout, which defaults to timeout. Set connect_timeout=0 to restore an unbounded connection wait.
@coderabbitai

coderabbitai Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The change adds connection deadlines to async and sync APIs, adjusts operation timeout handling, and updates transport error and logging behavior. Tests and documentation cover these changes. Workflow checks validate wheel and source distribution contents.

Changes

Transport and timeout behavior

Layer / File(s) Summary
Connection deadlines and API wiring
src/browserless_bap/types/options.py, scripts/codegen/config/client_only.py, src/browserless_bap/browserless.py, src/browserless_bap/sync_api/browserless.py, src/browserless_bap/browser.py, src/browserless_bap/transport.py, src/browserless_bap/sync_api/_bridge.py, tests/test_browser.py, tests/test_transport.py, tests/test_transport_protocol.py, tests/sync/*, tests/e2e/*, README.md, llms.txt, AGENTS.md
The APIs accept and forward connect_timeout. The transport applies it to connection setup and closes unsuccessful connections. Tests cover timeout, failure, cancellation, and cleanup behavior. The documentation describes connection deadlines and retry behavior.
Operation timeout handling
src/browserless_bap/page.py, src/browserless_bap/transport.py, tests/test_operation_timeouts.py, tests/test_live_complete.py, tests/test_page_overrides.py, tests/sync/test_sync_page.py, README.md, llms.txt, AGENTS.md
Page operations use the shared 5,000 ms cushion for mutations with a timeout argument. The transport expires timed-out operations and sends completion frames. Tests cover timeout values and results.
Protocol errors and secret redaction
src/browserless_bap/transport.py, src/browserless_bap/errors.py, tests/test_transport_protocol.py, tests/test_errors.py, tests/e2e/test_live.py, README.md, llms.txt
Transport processing handles subscription errors, completion frames, and connection errors. BrowserQLError normalizes error entries. Error messages and log records redact secrets; logging guidance notes that frame logs can contain page or application data.

Distribution-content checks

Layer / File(s) Summary
Distribution content validation
.github/workflows/publish.yml, .github/workflows/test.yml
Both workflows fail if the wheel contains mutations.graphql or the source distribution lacks src/browserless_bap/mutations.graphql.

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant Browserless
  participant Browser
  participant Transport
  participant WebSocket
  Browserless->>Browser: pass connect_timeout
  Browser->>Transport: create transport and connect
  Transport->>WebSocket: dial and initialize within deadline
  WebSocket-->>Transport: connection result
  Transport-->>Browser: result or connect timeout
Loading

Suggested reviewers: gikoskos

Merge Risk: Minimal · up to 6473f

This change adds a bounded connection timeout and tightens operation-timeout, error-handling and log-redaction behavior. No unresolved concrete defect was identified in the supplied evidence. Live tests were not rerun after the final callback-only change, so ordinary pre-merge checks still apply.

Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title "feat!: bound connection time and harden the transport" matches the main change. The PR adds a connect_timeout option that bounds page connection setup, and it also hardens the transport wit…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title "feat!: bound connection time and harden the transport" matches the main change. The PR adds a connect_timeout option that bounds page connection setup, and it also hardens the transport with error handling, redaction, and cleanup. The title is specific and uses the conventional commit form. The "!" marks a breaking change, which the objectives do not confirm, but this does not make the title unrelated to the changeset.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

A rabbit checks the dial-time clock,
Then bounds each hop around the block.
A timeout lands; the page stands still,
New pages answer when they will.
The wheel stays clear; the source stays whole.

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Team
  • Run ID: 80f5edd0-8cc8-4825-81dc-426dacd50868
Commits

Reviewing files that changed from the base of the PR and between d1a3a7d and 6473f3e.

Files selected for processing: 17

  • AGENTS.md
  • scripts/codegen/config/client_only.py
  • src/browserless_bap/browser.py
  • src/browserless_bap/browserless.py
  • src/browserless_bap/page.py
  • src/browserless_bap/sync_api/_bridge.py
  • src/browserless_bap/sync_api/browserless.py
  • src/browserless_bap/transport.py
  • tests/e2e/test_config.py
  • tests/e2e/test_live.py
  • tests/sync/_server.py
  • tests/test_browser.py
  • tests/test_errors.py
  • tests/test_live_complete.py
  • tests/test_page_overrides.py
  • tests/test_transport.py
  • tests/test_transport_protocol.py

Included review availability: This review used your included allowance. 0 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

coderabbitai[bot]

This comment was marked as resolved.

Bound failed-setup cleanup to five seconds, reject connection attempts whose deadline cancellation was suppressed, and drop unformattable log records before handlers can expose credentials.
@artiom
artiom marked this pull request as ready for review November 13, 2026 13:20
@artiom
artiom requested a review from a team as a code owner November 13, 2026 13:20

artiom commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator Author

All three review findings are fixed with regression coverage: malformed log records are dropped, expired connection attempts cannot succeed after suppressed cancellation, and failed-setup cleanup has a five-second budget. Current head passes 240 offline tests (one existing optional parity skip), all five live BrowserQL tests against a local runtime, formatting, lint, typing, generation checks, and hosted CI. The missing-selector case returns the server error mentioning 1500ms. No unresolved review threads or merge conflicts remain. Ready for human review; no merge or release performed.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

artiom commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator Author

All three initial review findings are fixed: malformed log records are dropped before handler diagnostics, expired connection deadlines remain errors after suppressed cancellation, and failed-setup cleanup is bounded. Regression coverage passes: 240 offline tests and five live BrowserQL tests. All 14 current-head checks pass; no unresolved review threads or merge conflicts remain. The PR is ready for human review. No remaining review findings were rejected.

@Gikoskos

Copy link
Copy Markdown
Collaborator

Review: ready to merge

Every planned item is present. 38 of the 79 new non-sync tests fail on main's code and the new sync test hangs there, so the tests catch the old bugs. All three CodeRabbit findings are fixed, each with a regression test. Lint, mypy and the offline suite pass on 3.11, 3.12 and 3.14 (240 passed, 1 skipped). Codegen and the schema check leave no diff, and pytest -m e2e passes 5/5 on dev.

Live probes on dev

  • connect_timeout against an unroutable wss://10.255.255.1: raises at 2.00 s; main hangs until killed.
  • wait_for_selector(timeout=1500): the server's own timeout error arrives at 1.69 s, which confirms the 5 s cushion.
  • evaluate(timeout=1000) on a 3 s script: the server's error arrives at 1.19 s, and the page stays usable.
  • The same sync wait_for_selector: hangs on main until killed at 240 s; on this PR it raises at 1.69 s.
  • Fake-token probes through both APIs (unroutable, stalled, refused, DNS failure, bad scheme, rejected token): the token never appears in the error text, repr, attached errors, traceback locals or DEBUG logs.

Minor (none blocking)

  1. src/browserless_bap/transport.py:280 and :320: if every caller waiting on a connection attempt is cancelled, the attempt keeps running with no owner. If it then fails, asyncio logs "Task exception was never retrieved". Reproduced on 3.11 and 3.14. Add a done-callback on _ready that retrieves the exception.
  2. src/browserless_bap/transport.py:41-44: the log filter drops the whole record when formatting fails. Replacing the message with fixed text instead would keep the diagnostic and still avoid the leak.
  3. src/browserless_bap/transport.py:440-456: a next frame whose errors is not a list, or whose payload is not a dict, counts as success {}. Consider raising a protocol error instead, in a later PR.
  4. src/browserless_bap/transport.py:457-461: an error frame whose payload is a single dict becomes "Unknown BrowserQL error", which drops the server's message. This predates the PR.
  5. README: "SDK logs redact tokens" is broader than the code. The filter is attached to browserless_bap.websockets only, not to the parent browserless_bap logger. Narrow the wording or move the filter.

Merge order: whichever of #8 and #9 lands second needs a rebase, because #8 renames the exceptions to BAP* and adds opt_in. #7 also touches _bridge.py, so the future.done() guard must survive its rebase.

Observe shared connection-task failures even when every waiter cancels, without changing what remaining awaiters receive. Scope logging redaction documentation to the filtered WebSocket logger.

Co-authored-by: Artiom Lunev <artiom@browserless.io>

artiom commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator Author

Items 1 and 5 are fixed in d1a3a7d: shared connection-task failures are retrieved even after all waiters cancel, and README/llms redaction claims now name the filtered WebSocket logger. The new cancellation regression reproduced the asyncio warning before the fix; the updated offline suite passes 241 tests (one existing optional skip), along with Ruff, mypy, generation, schema and package checks. Item 2 retains the deliberate fail-closed record drop to avoid handler diagnostics leaking malformed input. The later protocol-handling suggestions in items 3 and 4 remain unchanged. Live acceptance passed five tests before this callback-only follow-up and was not rerun afterward. Rebase integration still needs to preserve BAP-prefixed exceptions, opt_in forwarding and the completed-future sync guard.

artiom commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator Author

Follow-up verification is complete at d1a3a7d: all 13 current-head checks pass, including Python 3.11–3.14, and the latest CodeRabbit review reports no actionable comments. No unresolved review threads or merge conflicts remain. The PR remains ready for human review; the prior live-test attribution and non-blocking protocol notes are unchanged.

Keep both new live tests and count six skipped live tests.

@Gikoskos Gikoskos left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved: review findings handled, and live tests pass on the merged head.

@Gikoskos
Gikoskos merged commit 1689f3b into main Nov 13, 2026
15 checks passed
@Gikoskos
Gikoskos deleted the fix/transport-hardening branch November 13, 2026 16:30
Gikoskos pushed a commit that referenced this pull request Nov 13, 2026
🤖 I have created a release *beep* *boop*
---


##
[0.3.0](v0.2.0...v0.3.0)
(2026-11-13)


### ⚠ BREAKING CHANGES

* `Page.query_selector()` and `Page.query_selector_all()` no longer
fetch `innerHTML`/`outerHTML`; pass `html=True`. Reading
`ElementHandle.inner_html` or `outer_html` on a handle queried without
it raises `BAPValueError`. `Page.response()` no longer returns `body`;
pass `body=True` (or `body_encoding=True`, which implies it). the
exceptions `TimeoutError`, `ConnectionError` and `ValueError` are
removed from `browserless_bap` and `browserless_bap.errors`; use
`BAPTimeoutError`, `BAPConnectionError` and `BAPValueError` (they still
subclass the matching built-in exceptions).
`browserless_bap.graphql.RESPONSE_FIELDS` no longer has the
`HTTPResponseWithRemoteMetadata` and `ResponseResponseWithBodyEncoding`
keys; use `OPTIONAL_FIELDS` with `selection_for()`.
`QuerySelectorResponse["childElementCount"]` is typed `int` (was
`float`), and
`browserless_bap.sync_api.Browserless.connect(transport=...)` is typed
`TransportFactory | None` (was `Any`).
* Opening a page now times out after `connect_timeout` (default: the
`timeout` value, 30000 ms). Pass `connect_timeout=0` to restore the
previous unbounded wait.

### Features

* bound connection time and harden the transport
([#9](#9))
([1689f3b](1689f3b))
* make element HTML and response bodies opt-in and rename
builtin-shadowing errors
([#8](#8))
([bc80671](bc80671))


### Bug Fixes

* encode element handle selectors so nested and special-character
queries work ([#6](#6))
([f22f12b](f22f12b))
* sync API listener delivery, wait_for_function predicates and parity
checks ([#7](#7))
([e5a106d](e5a106d))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: browserless-actions-bot[bot] <186328842+browserless-actions-bot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants