Race condition after async cancellations leads to degraded pool state

[TheTechromancer]

This bug has been haunting us recently. It happens after certain async cancellations, which cause the HTTP pool to get stuck in a degraded state, unable to issue requests. It looks like a resurfacing of this issue.

httpcore version: 1.0.5
python version: 3.12.4

Code to reproduce:

import random
import asyncio
import httpcore
from collections import defaultdict

# Global dictionary to store status code counts
status_counts = defaultdict(int)


async def make_request(pool):
    try:
        response = await pool.request(
            method="GET",
            url=("https://localhost:8000/"),
        )
        status_counts[response.status] += 1
    except BaseException:
        status_counts[0] += 1


async def spawn_and_cancel_tasks(pool):
    while True:
        tasks = [asyncio.create_task(make_request(pool)) for _ in range(100)]
        await asyncio.sleep(random.random())  # Allow some time for the tasks to start
        for task in tasks:
            task.cancel()
        await asyncio.gather(*tasks, return_exceptions=True)


async def monitor_stats(pool):
    while True:
        status_str = ", ".join(f"{code}: {count}" for code, count in sorted(status_counts.items()))
        print(f"Status codes: {status_str} | Pool {pool}")
        status_counts.clear()
        await asyncio.sleep(1)


async def main():

    async with httpcore.AsyncConnectionPool() as pool:
        stats_task = asyncio.create_task(monitor_stats(pool))
        spawn_task = asyncio.create_task(spawn_and_cancel_tasks(pool))

        try:
            await asyncio.gather(stats_task, spawn_task)
        except KeyboardInterrupt:
            print("Stopping...")

if __name__ == "__main__":
    asyncio.run(main())

Output:

Status codes:  | Pool <AsyncConnectionPool [Requests: 0 active, 0 queued | Connections: 0 active, 0 idle]>
Status codes: 200: 200 | Pool <AsyncConnectionPool [Requests: 0 active, 0 queued | Connections: 0 active, 10 idle]>
Status codes: 0: 202, 200: 139 | Pool <AsyncConnectionPool [Requests: 62 active, 0 queued | Connections: 8 active, 2 idle]>
Status codes: 0: 142, 200: 117 | Pool <AsyncConnectionPool [Requests: 5 active, 0 queued | Connections: 5 active, 0 idle]>
Status codes: 0: 35, 200: 165 | Pool <AsyncConnectionPool [Requests: 5 active, 0 queued | Connections: 5 active, 5 idle]>
Status codes: 200: 100 | Pool <AsyncConnectionPool [Requests: 5 active, 0 queued | Connections: 5 active, 5 idle]>
Status codes: 0: 155, 200: 45 | Pool <AsyncConnectionPool [Requests: 5 active, 100 queued | Connections: 10 active, 0 idle]>
Status codes: 0: 300 | Pool <AsyncConnectionPool [Requests: 5 active, 0 queued | Connections: 10 active, 0 idle]>
Status codes:  | Pool <AsyncConnectionPool [Requests: 5 active, 100 queued | Connections: 10 active, 0 idle]>
Status codes: 0: 200 | Pool <AsyncConnectionPool [Requests: 5 active, 0 queued | Connections: 10 active, 0 idle]> <- stops responding here
Status codes: 0: 100 | Pool <AsyncConnectionPool [Requests: 5 active, 100 queued | Connections: 10 active, 0 idle]>
Status codes: 0: 200 | Pool <AsyncConnectionPool [Requests: 5 active, 0 queued | Connections: 10 active, 0 idle]>
Status codes: 0: 100 | Pool <AsyncConnectionPool [Requests: 5 active, 100 queued | Connections: 10 active, 0 idle]>
Status codes: 0: 100 | Pool <AsyncConnectionPool [Requests: 5 active, 100 queued | Connections: 10 active, 0 idle]>
Status codes: 0: 200 | Pool <AsyncConnectionPool [Requests: 5 active, 100 queued | Connections: 10 active, 0 idle]>
Status codes: 0: 200 | Pool <AsyncConnectionPool [Requests: 5 active, 100 queued | Connections: 10 active, 0 idle]>
Status codes: 0: 300 | Pool <AsyncConnectionPool [Requests: 5 active, 100 queued | Connections: 10 active, 0 idle]>

This is reproducible using the default python http server:

echo 'a' > index.html
python -m http.server

The bug applies to both asyncio and trio. However in trio it's much narrower. With asyncio it triggers almost immediately, while trio takes about 30 seconds.

Code to reproduce in trio:

import random
import trio
import httpcore
from collections import defaultdict

# Global dictionary to store status code counts
status_counts = defaultdict(int)


async def make_request(pool):
    try:
        response = await pool.request(
            method="GET",
            url="http://localhost:8000/",
        )
        status_counts[response.status] += 1
    except BaseException:
        status_counts[0] += 1


async def spawn_and_cancel_tasks(pool):
    while True:
        with trio.CancelScope() as cancel_scope:
            async with trio.open_nursery() as nursery:
                for _ in range(100):
                    nursery.start_soon(make_request, pool)
                await trio.sleep(random.random())  # Allow some time for the tasks to start
                cancel_scope.cancel()


async def monitor_stats(pool):
    while True:
        status_str = ", ".join(f"{code}: {count}" for code, count in sorted(status_counts.items(), reverse=True))
        print(f"Status codes: {status_str} | Pool {pool}")
        # Clear the counts for the next iteration
        status_counts.clear()
        await trio.sleep(1)


async def main():

    async with httpcore.AsyncConnectionPool() as pool:
        async with trio.open_nursery() as nursery:
            nursery.start_soon(monitor_stats, pool)
            await spawn_and_cancel_tasks(pool)


if __name__ == "__main__":
    try:
        trio.run(main)
    except KeyboardInterrupt:
        print("Stopping...")

Anyway sorry to bring this up again. I know these suck to troubleshoot. Let me know if there's anything else I can do to help in getting it fixed.

[TayfurYldz]

There has been a meaningful cleanup change since the 1.0.5 reproducer, and current main is worth retesting before treating this as the same bug.

The pool now shields response cleanup from cancellation. In particular, PoolByteStream.aclose() wraps the underlying stream close in AsyncShieldCancellation(), and both async HTTP/1.1 and HTTP/2 response streams similarly shield their aclose() path when iteration exits with an exception/cancellation.

That matters for the symptom in the reproducer: the permanently “active” requests/connections can occur when cancellation interrupts cleanup after a request has acquired pool/connection state but before the response-close path releases it. Shielding that critical cleanup is the right invariant: cancellation may abort the request, but it must not abort releasing pool bookkeeping or protocol capacity.

I would therefore rerun this exact cancellation stress test against current httpcore before changing pool logic. If it still degrades, the next useful diagnostic is to identify which resource remains owned after cancellation: pool._requests, an HTTP/1 connection state, or an HTTP/2 stream semaphore/event entry. Those now have separate shielded cleanup paths, so “5 active forever” would point to a more specific missed cancellation window.

One place I'd scrutinize if it still reproduces is the pool's exception path itself: it removes/reassigns the pool request and then awaits closing connections. The stream-close paths are explicitly shielded; cancellation safety should be checked consistently for every await between removing a failed/cancelled request and completing the corresponding connection cleanup.