Skip to content

bitreq: Support multiple connections per origin #658

Closed

Description

@tnull

Client currently caches one AsyncConnection per ConnectionKey, so capacity limits cached origins rather than connections to one origin.

Consequently, concurrent requests to the same HTTP/1.1 server share one socket. Pipelining does not provide server-side concurrency with common servers such as Go's net/http, resulting in head-of-line blocking. This is causing serialized remote-storage reads in: lightningdevkit/orange-sdk#90

We should add support for a configurable max_connections_per_origin, with connections selected by availability/load. The existing cache capacity could remain a separate limit on cached origins.

This limitation and possible support for two or three connections was previously noted here: lightningdevkit/vss-client#56 (comment)

Activity

  1. changed the title [-]Support multiple connections per origin[/-] [+]`bitreq`: Support multiple connections per origin[/+] on Jul 10, 2026
  2. TheBlueMatt commented on Jul 10, 2026

    @TheBlueMatt
    Member

    Pipelining does not provide server-side concurrency with common servers such as Go's net/http, resulting in head-of-line blocking. This is causing serialized remote-storage reads in: lightningdevkit/orange-sdk#90

    That shouldn't matter here, though. For VSS, the server-side processing of a single request should be a few milliseconds, so the pipelining should still help a lot.

  3. tnull commented on Jul 11, 2026

    @tnull
    CollaboratorAuthor

    That shouldn't matter here, though. For VSS, the server-side processing of a single request should be a few milliseconds, so the pipelining should still help a lot.

    But out-of-scope for this issue here, but in VSS we can pipeline reads can't pipeline writes/removes due to idempotency concerns. So IMO using multiple connections would help a lot here to avoid head of line blocking, especially in cases where writes and reads are interleaved, as any write would be a serialization point/bottleneck.

  4. TheBlueMatt commented on Jul 11, 2026

    @TheBlueMatt
    Member

    Yea I feel like we should treat non-pipelined requests when there's no available no-requests-pending connections as needing a fresh connection always.

  5. jamillambert commented on Sep 1, 2026

    @jamillambert
    Collaborator
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions