fix(streamable-http-server): release the session map lock before waiting on a session - #1323
Open
TheSeydiCharyyev wants to merge 1 commit into
Open
TheSeydiCharyyev wants to merge 1 commit into
TheSeydiCharyyev wants to merge 1 commit into
Conversation
…ing on a session LocalSessionManager kept the read guard on its session map while it waited on a session worker. initialize_session keeps it for the whole ServerHandler::initialize, so one slow initialize blocks create_session for every new client. tokio's RwLock is fair, so has_session calls for other sessions then wait behind that writer, and the whole server waits until the slow initialize returns. Clone the session handle out of the map and drop the guard before waiting, like close_session already does.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
While one client's
initializeis running, no other client can connect. Once a new client tries, requests on all other sessions wait too. With a server whoseinitializetakes 3 s, a second client'sinitializetook 2.8 s and apingon an existing session took 2.6 s. With this change, both take about 3 ms.Motivation and Context
LocalSessionManagerkeeps all sessions in oneRwLock<HashMap<..>>. Five methods take the read guard and keep it while they wait on the session worker:initialize_sessionwaits for the initialize response, so it keeps the guard during the wholeServerHandler::initialize.create_stream,create_standalone_streamandresumewait for the worker to reply.accept_messagewaits when the session's event channel is full.create_sessionneeds the write lock, so a new client waits for that guard. tokio'sRwLockis fair: when a writer is waiting, new readers wait behind it. The tower service callshas_sessionfor every POST and GET that has a session id, so requests on other sessions wait as well. The wait lasts as long as the slowinitializeruns.Fix
A private helper,
session_handle, clones theLocalSessionHandleout of the map and drops the guard before the caller waits on the worker. The handle is only an id and a channel sender, so the clone is cheap. The five methods use the helper.close_sessionalready drops the lock before it awaits; now the other methods work the same way.How Has This Been Tested?
tests/test_streamable_http_session_isolation.rs. One session waits on its worker: nothing serves its transport, the same setup astest_streamable_http_init_timeout.rs. Thencreate_sessionandhas_sessionfor another session must finish within 1 s. There is one case for each of the five methods. Onmainall five cases fail withhas_session blocked: Err(Elapsed(())). With this change they pass (30 of 30 runs).initialize_sessionfails all five cases, because every case starts with a pending initialize.StreamableHttpServicewith aServerHandlerwhoseinitializesleeps 3 s, then a second client'sinitializeand apingon an existing session. The numbers are at the top.cargo test -p rmcpwith the CI feature set withoutlocal, for the 19 streamable HTTP server test files: all 152 tests pass. Also the nightly rustfmt check and both clippy commands from CI.Tested on Windows 11.
Breaking Changes
None. The public API does not change: the helper is private.
Types of changes
Checklist
Additional context
One small change in behavior:
close_sessionno longer waits for calls that hold the read guard. The worker handles session events in order, so events queued before the close are still handled. Events queued after the close are not handled, and a call that waits for a reply getsSessionError::SessionServiceTerminated.