Repository navigation
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🟡 Changes recommended
Proxy passwords can leak through client representations, and invalid proxy-host values can silently cause direct connections that bypass the requested proxy.
5 open issues
Reject invalid proxy hosts before warning and discarding them · New Redact proxy credentials from client representations · New Use the owning client’s proxy settings for automatic decryption · New Defer proxy cross-option checks until client options are merged · New Use TLS options only when the test client uses TLS · New
What changed in this PR
This PR adds standard-library SOCKS5 proxy support to PyMongo’s synchronous and asynchronous clients, including database connections, monitoring pools, and encryption KMS requests.
Changes:
- Adds proxy configuration options, validation, and client documentation.
- Implements SOCKS5 negotiation, authentication, timeout handling, and connection routing.
- Adds URI validation, wire-protocol, and integration tests for both APIs.
| File | Description |
|---|---|
test/uri_options/proxy-options.json |
Adds proxy URI specification cases. |
test/test_uri_parser.py |
Tests proxy option validation and normalization. |
test/test_socks5.py |
Adds synchronous SOCKS5 protocol and routing tests. |
test/test_socks5_prose.py |
Adds synchronous proxy integration scenarios. |
test/asynchronous/test_socks5.py |
Adds asynchronous protocol, routing, and cancellation tests. |
test/asynchronous/test_socks5_prose.py |
Adds asynchronous proxy integration scenarios. |
pymongo/uri_parser_shared.py |
Checks duplicate options, credentials, and configuration conflicts. |
pymongo/topology_shared.py |
Propagates proxy settings to monitoring pools. |
pymongo/synchronous/mongo_client.py |
Documents proxy options and handles empty credentials. |
pymongo/synchronous/encryption.py |
Applies proxy settings to synchronous KMS requests. |
pymongo/pool_shared.py |
Implements SOCKS5 connections and proxy-specific error handling. |
pymongo/pool_options.py |
Stores immutable proxy settings in pool options. |
pymongo/common.py |
Registers proxy options and validates hosts and ports. |
pymongo/client_options.py |
Builds proxy settings for connection pools. |
pymongo/asynchronous/mongo_client.py |
Documents proxy options and handles empty credentials. |
pymongo/asynchronous/encryption.py |
Applies proxy settings to asynchronous KMS requests. |
doc/changelog.rst |
Announces SOCKS5 support. |
.evergreen/remove-unimplemented-tests.sh |
Stops removing proxy URI specification tests. |
Review effort: Balanced
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
NoahStapp
left a comment
There was a problem hiding this comment.
The prose tests need to be run in an Evergreen task, as specified by the specification.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
NoahStapp
left a comment
There was a problem hiding this comment.
Stopping my review here. This prompt against 5.5 Opus on medium had 10 findings on the latest commit, please resolve or provide reasoning as to why ignoring them is the correct choice:
Review the changes in https://github.com/mongodb/mongo-python-driver/pull/3174 using @CODING-STANDARDS.md and @.github/copilot-instructions.md as guidelines. Source your findings. Make sure you review the latest version of the PR.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Stopping my review here, additional LLM-found findings below. I don't think this approach is going to produce a worthwhile result.
1. The PR description doesn't match the pushed code
The body says "implemented and validated locally, but has not yet been committed or pushed." It then describes things that aren't in 9a0c88d:
- test/asynchronous/test_socks5.py
- test/socks5_helpers.py
- .evergreen/scripts/test_socks5_proxy.py
- test_kms_connect_rejects_callback_with_socks5_proxy
- A kms_connect_callback/proxy check "at KMS connection establishment". _connect_kms only had its docstring changed; the check exists only at construction.
The test plan was therefore not run against this commit.
2. Large tests that cover several unrelated behaviours
- Source: Testing 3, "Prefer many small tests over one large one." You raised this on Nov 4 and it is still present.
- Examples:
- test/test_socks5.py:258 test_bad_versions_eof_timeout_and_tls_sni mixes version errors, EOF, timeout and TLS SNI.
- :107 test_greeting_and_auth, :160, :206, :311, and the async counterparts at :756 and :835.
- test/test_kms_connect.py:1064 test_socks5_proxy_options_flow_to_kms covers the kms_request path, auto-encryption with and without bypass, explicit encryption, and two conflict errors in one test.
- test/test_uri_parser.py:662 and :704.
- Misplaced tests: test_socks5_deadline_helpers and test_socks5_protocol_validation (test/test_uri_parser.py:704+) test pymongo.socks5_shared protocol and deadline helpers. They aren't URI parsing tests and belong in the SOCKS5 test file.
3. Bare assert, pytest.raises and a tautological assertion in unittest classes
- Source: Testing 4, "Use unittest-style assertions when using unittest."
- Where: test/test_socks5.py:491-536, used by the IntegrationTest prose classes.
- Tautology: line 534, assert address == (("localhost", 12345) if mapped else address), compares address with itself in the unmapped case. Split it into an explicit if mapped: / else: with assertEqual / assertIn.
4. The mock proxy's errors are never checked
_Socks5Server stores exceptions from its server thread in self.error (test/test_socks5.py:613, set at :714), but no test reads it.
- Effect: server-side failures, such as a malformed request that the server failed to parse, are swallowed. A test can pass without the exchange it claims to check.
- Fix: assert server.error is None in close() for the success-path tests.
- Source: Testing 5, safe setup and cleanup for both success and failure.
5. Exception handling in _async_create_proxy_connection is more complex than needed
At pymongo/asynchronous/socks5_support.py:92-136:
- err.__cause__ = exc is assigned by hand, and if isinstance(err, socket.timeout): raise err re-raises outside the chain. Error Handling 2 says to use raise ... from ....
- _Socks5Timeout (pymongo/socks5_shared.py:73) doesn't inherit from _Socks5Error. That is why pool_shared.py:347 has to list both types, and why the loop needs the nested try: _proxy_remaining() except _Socks5Timeout as timeout: raise timeout from exc.
- class _Socks5Timeout(_Socks5Error, socket.timeout) would shorten the isinstance tuple and the handler chain. Structure 2 ("bias towards less code") and Structure 6 (maintainability) apply.
6. I/O helpers that network_layer already provides
- pymongo/network_layer.py already has _async_socket_receive (reads exactly n bytes from a non-blocking socket and raises on EOF) and async_socket_sendall (line 76).
- _async_socks5_read_exact, _async_socks5_recv and _async_socks5_sendall (asynchronous/socks5_support.py:51, socks5_shared.py) re-implement them, with a deadline added.
- Consider building on the existing helpers. Source: Structure 1, prefer shared helpers to duplication.
7. Magic numbers in the protocol code
- Source: Code Style 2. The PR adds named constants for most protocol bytes, but these are still bare literals:
- _socks5_bound_address_size returns 6, 18 and domain_length + 2 (socks5_shared.py:102-108).
- The greeting method counts 2 and 1 (:110-123).
- The read sizes 2, 4 and 1 in _socks5_handshake.
- port.to_bytes(2, "big").
- Fix: named sizes, for example _SOCKS5_PORT_LEN and IPv4/IPv6 address lengths, would make the RFC 1928 mapping readable.
8. A per-connection local import works around a circular import
pool_shared.py:630 and :869 import socks5_support inside _create_connection because socks5_support imports _new_tcp_socket and _configure_tcp_socket from pool_shared.
- Moving those two helpers to a lower-level module, such as socks5_shared or network_layer, removes the cycle and the comment.
- Source: Structure 6.
- Severity: low.
9. The changelog entry doesn't end with the JIRA link
doc/changelog.rst:12 puts (PYTHON-2943_) before the sentence "SOCKS5 proxy options cannot be combined with kms_connect_callback."
- Source: Changelog 7, "Changelog entries must end with a link to the JIRA ticket."
- Fix: move that sentence ahead of the link.
10. Minor
- Misplaced comment: asynchronous/encryption.py:398, "KMS requests use the encrypted client's SOCKS5 proxy options, not the key vault client's", sits above the key-vault-client selection. The proxy is actually chosen at :421; move the comment there. Source: Comments 1.
- Stale comment: pool_shared.py:601, "SOCK_CLOEXEC was added in CPython 3.2… PEP 446… 3.4". The PR rewrote this block, and those versions don't matter with a 3.11 minimum. The point that matters is that the flag may be missing on some platforms or kernels. Source: Comments 2 and Backwards compatibility 1.
- Unrelated whitespace change: a blank line was removed at .evergreen/scripts/setup_tests.py:55. The PR body says unrelated formatting changes were removed.
|
|
||
| """Tests for SOCKS5 proxy connection establishment.""" | ||
|
|
||
| from __future__ import annotations |
There was a problem hiding this comment.
This file uses a totally different structure than the existing tests that cover both async and sync APIs. We expect async tests to live in test/asynchronous and sync tests to live in test/ instead of combining both into one metaprogrammed file.
| then `just run-tests`. Set `FUZZ_MAX_TIME` (seconds, default 300) to change | ||
| the per-target budget. | ||
|
|
||
| ## SOCKS5 proxy tests |
There was a problem hiding this comment.
This whole entry isn't true, there's no change to setup-tests with socks5.


PYTHON-2943
The review follow-up below is implemented and validated locally, but has not yet been committed or pushed.
Changes in this PR
MongoClientandAsyncMongoClientthrough theproxyHost,proxyPort(default1080),proxyUsername, andproxyPasswordURI and keyword options.pymongo/common.pyand cross-option checks inpymongo/uri_parser_shared.py. Reject duplicate URI options, ignore empty proxy credentials, enforce the 255-byte UTF-8 credential limit during normal validation, and reject proxy settings with Unix domain sockets. Do not partially revalidate individual options whenvalidate=False._ProxyOptionsto server pools, SDAM/RTT monitor pools, and CSFLE/QE KMS requests. Automatic encryption uses the encrypted client's proxy, including withbypass_auto_encryption; explicit encryption uses the key-vault client's proxy. Leavemongocryptdunproxied.kms_connect_callbackcombined withproxyHostwithConfigurationError, both at public encryption construction and at KMS connection establishment, rather than silently bypassing SOCKS5. Document this restriction and default KMS routing.SOCKS5 proxy timed out, and exclude proxy failures from server-overload labels.test/asynchronous/test_socks5.pyand its generated synchronous mirror,test/test_socks5.py; the prose class isTestSocks5Prose. Cover real public encryption wiring, callback conflicts, overload-label positive controls, TLS SNI, connection fallback/deadlines, and command-monitoring addresses independently of the connection-string table.socks5srv.pyprocesses directly, remove the wrapper and redundant port prechecks, and retain readiness, failure cleanup, and PID-ownership verification. Run setup/teardown tests through the existinggenerate-configpre-commit hook used by CI.Test Plan
.venv/bin/python -m pytest -q -m '' test/test_socks5.py test/asynchronous/test_socks5.py test/test_uri_parser.py test/test_uri_spec.py test/test_kms_connect.py::test_init_kms_connect_callback test/test_kms_connect.py::test_kms_connect_rejects_callback_with_socks5_proxy test/test_kms_connect.py::test_kms_request_does_not_retry_a_contract_violation .evergreen/scripts/test_socks5_proxy.py .evergreen/scripts/test_generate_config.py: passed (375 passed, 9 skipped, 96 subtests passed). Skips are API-specific cases, prose cases without the default local server, and an unrelated optional snappy case./home/ubuntu/dev/mongo-python-driver/.venv/bin/python -m pytest -q test/test_socks5.py test/asynchronous/test_socks5.py -k TestSocks5Prose --junitxml /home/ubuntu/.copilot/session-state/b558bef0-e8f8-4df3-ad07-538ecc00cb95/files/socks5-prose-results.xml: passed (4 passed, 84 deselected, 48 subtests passed, no skips). Ran against an isolated Dockermongo:8.0.29replica set with the authenticated and unauthenticated SOCKS5 proxies; supplied the replica-set address throughDB_IP/DB_PORTand enabledSOCKS5_PROXIES_STARTED. Verified the JUnit output contained no skipped/failed/error cases, then stopped all test processes and the container..venv/bin/python tools/synchro.py: passed.git --no-pager diff --name-only --diff-filter=AM -z -- '*.py' | xargs -0 -r .venv/bin/ruff check: passed.git --no-pager diff --name-only --diff-filter=AM -z -- '*.py' | xargs -0 -r .venv/bin/ruff format --check: passed (22 files already formatted).git --no-pager diff --name-only --diff-filter=AM -z -- pymongo/ | xargs -0 -r .venv/bin/python -m mypy --disable-error-code import-untyped --disable-error-code unused-ignore: passed (14 source files). Default strict mypy still reports two pre-existingimport-untypederrors for the unchangedpymongocrypt.mongocryptimports: that dependency lacks stubs/py.typed. The flags isolate this dependency-metadata issue and its consequent unused-ignore diagnostics; no source-level ignores were added..venv/bin/python -m mypy --config-file mypy_test.ini test/test_socks5.py test/asynchronous/test_socks5.py test/socks5_helpers.py test/test_uri_parser.py test/test_kms_connect.py: passed (5 source files).PATH="/home/ubuntu/dev/mongo-python-driver/.venv/bin:$PATH" bash .evergreen/scripts/generate-config.sh: passed (25 tests passed; configuration generation succeeded).PYTHONPATH="/home/ubuntu/.copilot/session-state/b558bef0-e8f8-4df3-ad07-538ecc00cb95/files/socks5-doc-deps${PYTHONPATH:+:$PYTHONPATH}" .venv/bin/python -m sphinx -W -b html doc /home/ubuntu/.copilot/session-state/b558bef0-e8f8-4df3-ad07-538ecc00cb95/files/socks5-doc-build: passed, without warnings. Missing documentation dependencies were installed into an isolated temporary directory rather than changing the development environment.git --no-pager diff --check: passed.Checklist
Checklist for Author
Checklist for Reviewer