TCPIP: add multiwaiter support for TCP connect - #119
Open
nding0405 wants to merge 7 commits into
Open
Conversation
Allow a single thread to wait for incoming connections across multiple listening sockets at once using multiwaiter_wait(), instead of having to block in network_socket_accept_tcp() on one socket at a time. Each socket now exposes an array of futex called eventFutexState, that a caller can register them with a multiwaiter. For this PR, only the added the accept functionality, others remains commented out for now. A thread can watch the accept futexes of several listening sockets simultaneously, block in multiwaiter_wait() until one of them signals a ready connection, and then accept that connection. The blocking wait therefore moves out of accept() and into the multiwaiter: accept() is called only once a connection is known to be ready, so it no longer has to block, which is what makes it possible to wait on several listening sockets at the same time from a single thread. Key changes: Added Event futex array (tcpip-internal.h, NetAPI.h) for socket wrapper: Only SocketAcceptEvent is implemented today, room reserved for future receive/send events. The value of the futex counts the ready events currently pending on the socket. For SocketAcceptEvent, it represents the number of connections that are ready to be accepted. The reserved value SocketNotAvailable (0xFFFFFFFF) means the socket will be freed soon. Read-only getter: network_socket_get_event_source() returns a capability to a socket's event futex stripped to read-only. Producing events: on_tcp_connect increments the SocketAcceptEvent futex and calls notify_all() when a new connection becomes ready. To let the callback can find the wrapper from the raw socket, network_socket_create_and_bind() now applys a bidirectional link between the wrapper and raw socket. Consuming events: network_socket_accept_tcp() decrements the futex after it successfully accepts a connection. This only keeps the futex value align with the semantic desbribed above. It does not call notify_all(), since dequeuing an existing connection is not a new, so no need to wake the threads up. Avoid waiting forever when socket is not available: the futex lives in the socket wrapper's heap memory, so a thread must never be left blocked on a socket that is being freed. Both network_socket_close() and the reset handler (reset_network_stack_state) set every event futex to SocketNotAvailable and call notify_all() before that memory goes away. The value change wakes any thread blocked in multiwaiter_wait(), which then observes the sentinel and returns instead of sleeping on a dead socket. A subsequent accept() will see the sentinel and reports the socket as unavailable.
- Claim `signal_event_futex` and `consume_event_futex` are owned by
SealedSocket
- Use `TimeoutWaitForever` instead of timeout{UnlimitedTimeout}
A TCP child socket can become visible and accept-ready to `FreeRTOS_accept()` before `on_tcp_connect()` increments the accept futex. If a waiter accepts the child while the counter is zero, the old futex consume helper does not decrement it to -1, because -1 represents an invalid futex. The delayed `on_tcp_connect` callback then increments the counter to one, though no connection is pending. This leaves pending multi-waiters with a misleading futex. They will repeatedly treat the socket as ready, call `accept()`, and never block, defeating the purpose of multi-waiter waiting. Use a signed counter and always decrement after a successful accept. An early accept records a temporary negative debt that the callback later repays. Reserve INT32_MIN for an invalid futex so that normal negative debt cannot conflict with the sentinel value.
Add a TCP send event for multiwaiter, so one thread can wait for free send space on several connected TCP sockets at once. Workflow: The caller first calls `network_socket_send()` with a zero timeout. The first non-empty send creates `txStream` and sets the send event to its exact free space. A send queues as many bytes as it can and subtracts that count from the send event futex. When the peer ACKs bytes, `on_tcp_sent()` adds that count to the futex and wakes all waiters. If more bytes remain, the caller waits with multiwaiter and then tries the send again. If the connection closes, the waiter wakes and its next call to any network API will returns `-ENOTCONN`. Key changes: - Add `SocketTCPSendEvent`, whose corresponding futex represents the free byte count in `txStream`. - Keep `txStream` setup lazy just like FreeRTOS_send(), and use `FreeRTOS_tx_space()` for its usable size. - Register the sent callback for TCP clients and listeners. Because accepted sockets inherit it from their listener, although the listening socket itself cannot send application bytes. - Add `SocketConnectionClosed` for a disconnected TCP state. Keep `SocketNotAvailable` for a socket wrapper that is being removed. - On disconnect, set the send event futex to `SocketConnectionClosed` and wake its waiters.
Added the futex initialization helper to avoid code duplication. The send futex is initialized to the maximum txStream size before txStream is initialized. This is to prevent the user from calling multiwaiter_wait before calling the first send, which would result in waiting forever if we initialized it to 0. Later, on the first send, FreeRTOS initializes the txStream buffer, and the futex value is updated to the exact value instead of an advertised placeholder value.
Add a TCP connect event so one thread can start several outgoing TCP connections and wait for any of them to finish. Workflow: The caller first calls `network_socket_connect_tcp_internal()` with a zero timeout. FreeRTOS starts the handshake and returns `-EWOULDBLOCK` immediately because it cannot wait. The caller waits on the multiwaiter until with expected futex value equals to 0. When the connection succeeds, FreeRTOS will call `on_tcp_connect()` which changes the futex to 1 and wakes all waiters. The caller then retries the connect with a zero timeout and FreeRTOS returns `-EISCONN`. The event stays one while the socket is connected. If the connection fails or later closes, the callback changes the event to `SocketConnectionClosed` and wakes all waiters. Key changes: - Add `SocketTCPConnectEvent`: 0 means not connected and 1 means connected. - In `network_socket_connect_tcp_internal()`. Handle `-EWOULDBLOCK` and `-EINPROGRESS` cases in switch. `-EWOULDBLOCK` means we call `FreeRTOS_connect` without non-zero timeout, while `-EINPROGRESS` means the connection is in progress, but not finished. - When a TCP connection succeeds, signal the connect futex in `on_tcp_connect`. - Set the connect event to `SocketConnectionClosed` and wake its waiters when the connection terminates.
nding0405
force-pushed
the
tcp-connect-multiwaiter
branch
from
August 16, 2026 00:59
64f9880 to
d641304
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a TCP connect event so one thread can start several outgoing TCP
connections and wait for any of them to finish.
Workflow:
The caller first calls
network_socket_connect_tcp_internal()with a zerotimeout. FreeRTOS starts the handshake and returns
-EWOULDBLOCKimmediatelybecause it cannot wait. The caller waits on the multiwaiter until with expected
futex value equals to 0. When the connection succeeds, FreeRTOS will call
on_tcp_connect()which changes the futex to 1 and wakes all waiters. The callerthen retries the connect with a zero timeout and FreeRTOS returns
-EISCONN.The event stays one while the socket is connected. If the connection fails or
later closes, the callback changes the event to
SocketConnectionClosedand wakesall waiters.
Key changes:
SocketTCPConnectEvent: 0 means not connected and 1 means connected.network_socket_connect_tcp_internal(). Handle-EWOULDBLOCKand-EINPROGRESScases in switch.-EWOULDBLOCKmeans we callFreeRTOS_connectwithout non-zero timeout, while
-EINPROGRESSmeans the connection is inprogress, but not finished.
on_tcp_connect.SocketConnectionClosedand wake its waiters whenthe connection terminates.