Embedded client TCP tunnel support - #51
Open
jagerman wants to merge 11 commits into
Open
Conversation
map_udp_remote_port auto-vivified the _udp_handles entry and incremented holders before constructing the socket, so a throwing constructor left an entry behind with a holder that nothing could release: the exception propagates out, no claim is returned, and the count never comes back down. A later call for the same remote:port then saw a null socket, counted to two, and constructed again. Construct the socket first and only insert and count once it exists.
TCPHandle listened on IPv4 INADDR_ANY, which exposed every mapped tunnel port to the local network and contradicted the documented contract of handing back a port on [::1]; it also passed sizeof(sockaddr) as the socklen for a sockaddr_in. Bind [::1] via quic::Address instead. Close handling could not distinguish an orderly shutdown from a failure: both EOF and error closed the whole stream, which truncates a transfer silently in the EOF case. A clean EOF now sends FIN and keeps receiving, so a half-close stays half-closed; an error closes the stream with an error code, distinguishing "could not connect at all" from "connection broke" so the far end can tell them apart; and a received FIN shuts down the write side only once queued output has drained. Backpressure now uses the stream watermark and pause/resume APIs, which did not exist when the previous chunking loop was written. A bufferevent output buffer grows without limit and so signals nothing by itself, so the stream is paused once the application falls behind and resumed from the write callback. Also removes the dead surface (make_client, the empty _init_client, the undefined on_stream_data declaration, a forward declaration of a class that does not exist) and reworks TCPHandle::connect, which took a raw event_base and a source address it used as the destination, was IPv4 only, and never installed bufferevent callbacks at all.
The tunnel was 230 lines of struct sitting in session.cpp with its accepting half disabled behind #if 0, referencing APIs that no longer exist (set_stream_data_cb, and a Router::loop member access missing its call parens), so it could not have compiled if simply re-enabled. The accepting side now reads the two-byte destination port from the head of each stream and connects to the tun address. Handing the stream to a TCPConnection installs a new data callback, which cannot be done from inside the callback being replaced -- that assigns over the std::function currently executing -- so the connect is deferred to the next loop iteration and anything arriving meanwhile is buffered and written ahead of the socket's own traffic. Only a client with a tun device can terminate a stream locally, so only such a client listens. The tunnel is now built on first use rather than in both Session constructors, which gave every session a QUIC endpoint it would most likely never use. Connections are tracked by an id of our own rather than by stream id: a stream opened past the peer's stream limit is pending and has no id assigned yet, and those placeholder ids are not distinct, so keying on them silently dropped connections once more than 32 were open at once. The connection is capped at the QUIC minimum packet size, raises the stream limit well above the default of 32 (one stream per TCP connection), and holds itself open with keep-alives so that a merely idle TCP connection is not killed; the connection is instead torn down a minute after its last stream goes away, so a dormant tunnel costs no path traffic.
Only a client with a tun device can terminate a tunnelled stream into a local TCP connection, so an initiator has no way to tell whether a mapping it makes could ever carry anything. Advertise the capability so it can refuse up front instead of waiting to fail. This takes a new bit rather than reusing 1.0.x's QUIC_TUNNEL (1 << 1), which advertised the opposite role -- "I am embedded, so reach me via a tunnel" -- and was therefore set by exactly the clients that cannot accept one. Bit 0 (1.0.x EXIT) is likewise still set in the wild. Session::remote_accepts_tcp reports it as an optional<bool> because "no client contact yet" and "remote says no" call for different behaviour from a caller: the former is worth attempting, the latter never will be.
The TCP counterpart of map_udp_remote_port, sharing its holder-count semantics: asking for a remote and port that is already mapped hands back the same listener, and the mapping is torn down by the last holder to release it. The listener lives here rather than on the session because it has to outlive session churn -- the tunnel beneath it can be reset by a dead path, an idle teardown, or a session timing out, none of which should change the port an application was handed. Each accepted connection re-obtains the session, as the UDP receive path does per datagram, so a mapping re-establishes itself on next use. Releasing the last holder closes the connections established through the mapping as well as the listener: holding a claim is what keeps a tunnel alive, so nothing is left running once the last one goes. A remote already known not to accept tunnelled TCP is refused without binding a port, and re-checked per connection, since a mapping may have been made before its client contact arrived.
Mirrors establish_udp, including its three-outcome contract and the RAII claim that keeps the mapping alive: an application holds a tcp_tunnel for as long as it wants the tunnel and lets it go when it does not, and dropping the last claim closes the listening port and the connections made through it. Two differences from UDP. A remote whose client contact says it does not accept tunnelled TCP is refused up front, and gets its own tunnel_failure value when that only becomes apparent after the mapping is made, since "will never work" and "not reachable right now" call for different responses. And suggested_mtu is nullopt, because segment sizes are not an application's to choose.
Replaces the commented-out sketch with a real establish_tcp call, and makes SIGUSR2 re-open the tunnel so that releasing and re-establishing can be tested by hand.
A full client defaulted to a global info level, at which quic logs a
line per stream open and close -- noise for a client tunnelling TCP,
where that is one pair of lines per connection. Default to
"*=info, quic=warn" instead.
Setting a level in config replaced that default outright, so naming a
single category ("tcp=debug") silently discarded the rest of it.
Category settings are cumulative, so appending is enough: naming only
categories now adjusts the defaults, while naming a global level resets
everything before it and so still replaces them.
Rewrites the 2021 design proposal into a description of what exists: the supported direction and what is deliberately not supported, the wire details, packet sizing, idle handling, backpressure, close semantics, and the remaining gaps.
jagerman
force-pushed
the
embedded-tcp-tunnel
branch
from
August 14, 2026 03:33
6f1ee06 to
964aa50
Compare
The inner connection carried nothing the session layer had not already encrypted end to end and the path layer onion-encrypted, and its peer is authenticated by the session itself, so QUIC's own crypto bought a second AEAD pass over every byte and a TLS handshake before any of it could flow. It was standing on a hardcoded well-known keypair to get that far. Use libquic's DangerouslyUnencryptedCreds instead, which needs libquic 1.9.0. Both ends must be doing this -- such a connection cannot talk to an ordinary QUIC endpoint -- which is what the TCP_TUNNEL capability flag already ensures.
Two cases the harness could not previously reach. establish_tcp() hands back the bound port synchronously and documents that a connection made before the session is up is held rather than refused, but every test connected only after on_established had fired. With SR_TCP_EARLY set the tunnel is opened before anything has established a session and connected to immediately, which is the case that documents. A target may now carry several comma-separated ports, mapping each to the same remote, so that multiple mappings sharing one session (and one tunnel connection) get covered.
jagerman
force-pushed
the
embedded-tcp-tunnel
branch
from
August 14, 2026 03:59
964aa50 to
c0fbaab
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds TCP tunnel support for embedded clients. It deliberately only implements embedded -> full client tunnels (no embedded-to-embedded, client-to-embedded, or embedded-to-relay). An embedded device as a TCP recipient could be useful, but isn't worth creating an API for until there is an actual use case, and realistically the only thing you would access on a relay is the storage server, which is available over quic (UDP) already.
Also fixes a minor potential issue in the UDP tunnel mapping if the actual tunnel construction raised an exception that would leave a stale entry.
This relies on
This also includes a minor vaguely related commits:
*=info, quic=warningbecause quic info is too chatty; this also fixes an unexpected behaviour wherelevel=tcp=debugin the config file was ignoring the default and just setting everything to info. (Now whatever you specify in the config applies on top of the default).Requires session-foundation/libquic#15 (for null crypto).