Add bolt+routing wrapper#86
Open
mattkjames7 wants to merge 19 commits into
Open
Conversation
Turn the flat `mgclient` C extension into a `mgclient` Python package that re-exports a renamed `mgclient._mgclient` extension. This gives a home for Python-level code (the client-side routing wrapper lands here next) alongside the compiled DB-API implementation, with no change to the public import surface: `import mgclient` and every name on it behave exactly as before. - setup.py: build the extension as `mgclient._mgclient` - pyproject.toml: map the `mgclient` package to `python/mgclient` - src/mgclientmodule.c: rename the module init to `PyInit__mgclient` - python/mgclient/__init__.py: `from mgclient._mgclient import *` Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Introduce a `mgclient.TransientError` exception (a subclass of `OperationalError`, so existing `except DatabaseError`/`except OperationalError` handlers still catch it) and raise it instead of the generic error whenever a failure is transient: a server-signalled Bolt TransientError, or a low-level transport failure that is worth retrying during a high-availability failover. The classification is delegated to `mg_error_is_transient` from libmgclient rather than duplicated here, so the client and the driver agree on what counts as transient. Both the connect path and the per-query error path are covered. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add an `owns_session` flag to `ConnectionObject` and a `connection_wrap_session` helper that builds a Connection around an already-established `mg_session`. When the session is borrowed, close and dealloc leave it intact for its owner. This is the foundation for routed managed transactions: the work callback needs a normal Connection/Cursor to run its queries, but over a session that the router owns and reuses across retries, so the wrapper must not destroy it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Expose libmgclient's routing engine (`mg_router`) to Python. The engine -- routing-table fetch and TTL caching, read load-balancing, coordinator failover, and the managed-transaction retry/backoff loop -- lives in libmgclient and is shared with every driver built on it; pymgclient only adapts it. - src/router.c: a `_mgclient._Router` type wrapping `mg_router`, with connect_read/write, execute_read/write, refresh and routing_table. Two trampolines bridge Python callables to the C callbacks (an address resolver and the unit of work); a raised exception is stashed and re-raised after the top-level call, and its type decides whether the retry loop sees a transient failure. The work runs against a borrowed, autocommit connection so the C layer owns the write transaction boundary. - python/mgclient/routing.py: a thin, Pythonic facade -- `Router`, `connect(routing=...)`, the access-mode constants and the is_transient_error / is_committed_on_main_error helpers. - python/mgclient/__init__.py: re-export the routing API; the routing-aware `connect` supersedes the C one (routing disabled by default). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add cluster-gated tests for the routing facade: one-shot connect(routing=True), the reusable Router (connect, refresh, routing_table), and managed execute_read/execute_write, plus no-cluster tests for argument handling and the is_transient_error / is_committed_on_main_error helpers. Move the shared HA-cluster and address-resolver fixtures into conftest.py so both test_connection.py and test_routing.py can use them. The engine's internals (routing-table parsing, read round-robin, coordinator failover, retry/backoff) are unit-tested in libmgclient's own suite, so the tests here are black-box: they confirm the Python facade wires the engine up correctly rather than re-testing the algorithms. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add the routing section to the module reference and usage guide: the routing-aware connect(routing=True, ...), the Router class and its managed execute_read/execute_write, the access-mode constants and resolver, and the TransientError exception with the is_transient_error / is_committed_on_main_error helpers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR introduces client-side routing support for Memgraph HA clusters by adding a low-level C extension wrapper over libmgclient’s routing engine and a Python-facing mgclient.Router + connect(routing=True, ...) facade, along with CI-backed HA routing tests.
Changes:
- Adds a new
_RouterC extension type and Pythonmgclient.routing.Routerfacade withconnect,execute_read,execute_write,refresh, androuting_table. - Introduces
mgclient.TransientErrorand propagates transient-vs-non-transient classification into connection and routing operations. - Adds HA routing integration tests and updates CI to provision an HA cluster and run the routing test suite.
Reviewed changes
Copilot reviewed 17 out of 17 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| test/test_routing.py | New HA routing integration tests for connect(routing=True, ...) and Router. |
| test/test_connection.py | Adds transient error tests and a regression test for routing=False. |
| src/router.h | Declares the new _Router C extension type. |
| src/router.c | Implements the _Router wrapper: resolver trampoline, routed connect, managed transactions, routing table access. |
| src/mgclientmodule.c | Registers _Router type and new TransientError; renames the extension module init/name for packaging as mgclient._mgclient. |
| src/exceptions.h | Exposes the new TransientError exception symbol. |
| src/connection.h | Adds owns_session and declares connection_wrap_session for router-managed sessions. |
| src/connection.c | Implements connection_wrap_session; classifies connect errors as transient vs operational; honors owns_session on close/dealloc. |
| src/connection-int.c | Classifies runtime/session errors as TransientError when appropriate. |
| setup.py | Renames built extension to mgclient._mgclient to support a Python mgclient package wrapper. |
| python/mgclient/routing.py | Adds the public routing API (Router, routed connect, access-mode constants, transient classifier). |
| python/mgclient/init.py | Introduces the mgclient package that re-exports the extension and overrides connect with the routing-aware wrapper. |
| pyproject.toml | Updates packaging to include the new mgclient Python package (and tools). |
| docs/source/usage.rst | Adds user-facing documentation for HA routing and managed transactions. |
| docs/source/module.rst | Documents routing API and adds TransientError to the exception reference. |
| .github/workflows/reusable_buildtest.yml | Updates CI to start/stop an HA cluster via submodule script and run routing tests. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
mattkjames7
marked this pull request as ready for review
July 10, 2026 16:49
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
18 tasks
as51340
approved these changes
Jul 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TODO BEFORE MERGE: release new version and update
mgclientsubmodule to latest tag!!!!Adds a Python wrapper around the new bolt_routing with automatic failover and retry logic in
mgclient(see memgraph/mgclient#84).Changes include:
Routerclass to manage a routed session to an HA cluster ->execute_readandexecute_writemembers which can be provided with acallablefunction to do work using acursor.mgclient.connect()function to allow a routed connection to be created usingrouting=Trueandaccess_mode="READ"|"WRITE".