Support Client-Level Exception Handlers for httpx.Client and httpx.AsyncClient

[dhruvjain1512]

🚀 Feature Request: Client-Level Exception Handlers for httpx.Client and httpx.AsyncClient

Problem Statement

Currently, httpx.Client and httpx.AsyncClient do not support registering custom exception handlers at the client level.
Developers are forced to wrap every request call with repetitive try/except blocks whenever consistent error handling is required.

This results in:

• Repetitive boilerplate across multiple request sites.
• Increased code maintenance complexity.
• Higher chances of missing or inconsistent error handling in large projects.

In many real-world scenarios, applications need uniform exception management across all outgoing HTTP calls from a shared client instance.

---

Proposed Solution

Introduce the ability to register custom exception handlers at the client level, allowing centralized and automatic handling of all exceptions raised during requests.

Key aspects of the proposed behavior:

• Allow developers to pass a mapping of exception types to handler callables during client creation, e.g.:

  def handle_timeout(exc):
      logger.warning(f"Request timed out: {exc}")
      return None
  
  client = httpx.Client(exception_handlers={httpx.TimeoutException: handle_timeout})

• When any request made through the client raises a matching exception:
  • The corresponding handler should automatically be invoked.
  • Handlers for more specific exception types should take precedence over base classes.
  • Handlers should support inheritance-based type matching.
• Both synchronous and asynchronous clients should support this feature in a consistent manner.

---

Example Use Case

def log_connection_error(exc):
    logger.error(f"Connection issue: {exc}")
    raise CustomConnectionError from exc

def retry_on_timeout(exc):
    logger.info("Retrying due to timeout...")
    return retry_request()

client = httpx.Client(
    exception_handlers={
        httpx.ConnectError: log_connection_error,
        httpx.TimeoutException: retry_on_timeout
    }
)

# No need for repetitive try/except blocks
response = client.get("https://example.com/data")

This pattern ensures that all requests from a client share the same consistent error handling behavior without requiring per-call wrappers.

---

Expected Behavior

• Registered exception handlers automatically intercept exceptions raised within client.request(), client.get(), client.post(), etc.
• More specialized exception handlers (e.g., httpx.ReadTimeout) take precedence over general ones (e.g., httpx.RequestError).
• Compatible with both synchronous and asynchronous interfaces.
• Maintains backward compatibility for clients without registered handlers.

---

Benefits

• Reduces repetitive boilerplate and improves code clarity.
• Centralizes error handling for better maintainability and testability.
• Simplifies integration with logging, monitoring, and retry frameworks.
• Encourages a clean, declarative configuration style consistent with modern Python design patterns.

---

Discussion Points

• Should handler callables return fallback responses, re-raise exceptions, or both (configurable)?
• How to propagate handler return values in async contexts?
• Should exception_handlers be modifiable post client initialization?

---

References & Inspiration

• Similar patterns exist in frameworks such as FastAPI (exception_handlers), aiohttp middlewares, and requests-hooks.
• Would align with modern best practices for composable HTTP clients and centralized observability.

[amasen02]

In httpx, the design philosophy typically prefers composable layers (such as custom Transports and Client method hooks) over adding dictionary dispatchers inside the core Client.

Here is how you can achieve centralized, inheritance-aware exception handling today in both sync and async, along with the core architectural considerations around streaming and return values.

---

Solution 1: Custom Transport Middleware (Recommended for network/transport errors)

httpx routes all network operations through its BaseTransport (or AsyncBaseTransport). Wrapping the default transport gives you a clean, centralized interception point without modifying call sites:

import logging
from typing import Callable, Type
import httpx

logger = logging.getLogger(__name__)

class ExceptionHandlerTransport(httpx.HTTPTransport):
    def __init__(self, handlers: dict[Type[Exception], Callable[[Exception, httpx.Request], httpx.Response | None]] = None, **kwargs):
        super().__init__(**kwargs)
        self.handlers = handlers or {}

    def _resolve_handler(self, exc: Exception):
        # MRO-based matching: find the most specific registered exception type
        for cls in type(exc).mro():
            if cls in self.handlers:
                return self.handlers[cls]
        return None

    def handle_request(self, request: httpx.Request) -> httpx.Response:
        try:
            return super().handle_request(request)
        except Exception as exc:
            handler = self._resolve_handler(exc)
            if handler is not None:
                res = handler(exc, request)
                if isinstance(res, httpx.Response):
                    return res
            raise

# Usage:
def on_timeout(exc: Exception, request: httpx.Request):
    logger.warning(f"Timeout on {request.url}: {exc}")
    # Can return a synthetic fallback response or re-raise
    return httpx.Response(status_code=504, content=b"Gateway Timeout (Handled)")

client = httpx.Client(
    transport=ExceptionHandlerTransport(
        handlers={
            httpx.TimeoutException: on_timeout,
        }
    )
)

resp = client.get("https://example.com")

For async, implement the exact same wrapper on httpx.AsyncHTTPTransport by overriding async def handle_async_request(self, request: httpx.Request).

---

Solution 2: Overriding Client.send() (Handles both network errors and status validation)

In httpx, all HTTP verbs (get(), post(), request()) eventually route through Client.send(). Subclassing Client allows you to catch transport errors as well as application-level response inspection:

import httpx

class ResilientClient(httpx.Client):
    def __init__(self, exception_handlers: dict = None, **kwargs):
        super().__init__(**kwargs)
        self.exception_handlers = exception_handlers or {}

    def _dispatch_exception(self, exc: Exception, request: httpx.Request):
        for cls in type(exc).mro():
            if cls in self.exception_handlers:
                return self.exception_handlers[cls](exc, request)
        raise exc

    def send(self, request: httpx.Request, **kwargs) -> httpx.Response:
        try:
            return super().send(request, **kwargs)
        except Exception as exc:
            fallback = self._dispatch_exception(exc, request)
            if isinstance(fallback, httpx.Response):
                return fallback
            raise

---

Architectural Challenges for First-Class Inclusion in Core httpx

If exception_handlers were considered for inclusion in core httpx, there are a few edge cases to consider:

1. Streaming Responses (client.stream(...)):
   When streaming with with client.stream(...) as response:, exceptions during response.iter_bytes() or response.iter_text() occur after client.request() has already returned. A client-level handler intercepting only the request phase would miss body stream read timeouts or connection drops mid-transfer.
2. Return Type Invariants:
   client.get(...) is typed to return httpx.Response. If an exception handler returns None or suppresses the error without returning a response, type checkers and downstream code expecting .status_code or .json() will fail with AttributeError. A handler must either return a valid fallback httpx.Response or re-raise a domain exception.
3. Retry State & Cloned Requests:
   Handlers that attempt retries (e.g. retry_on_timeout) need to rewind or re-create streaming request bodies (request.stream), which is why retry logic is typically delegated to specialized transport layers or packages like hishel / tenacity.