Repository navigation
perf(audio): aggregate receive frames before waking Rust consumers - #1645
Draft
theomonnom wants to merge 1 commit into
Draft
theomonnom wants to merge 1 commit into
theomonnom wants to merge 1 commit into
Conversation
Contributor
Changeset incompleteThis PR's changeset is missing version bumps for packages that are affected by the change. The following packages still require a bump:
Already covered:
A package must be bumped when its own files change, and whenever a package it depends on is bumped (so downstream consumers get a matching release). Click here to create a changeset for the missing packages The link pre-populates a changeset file with If this change doesn't require a version bump, add the |
This was referenced Nov 8, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR: Aggregate requested receive frames at the native sink to reduce Rust wakeups and avoid an extra PCM copy.
FFI/protobuf scope: PCM is passed by pointer with rate/channel/sample-count metadata; protobuf does not serialize the audio samples. For an existing 50 ms stream, this PR reduces native Rust queue deliveries from roughly 100/s to 20/s, while Python already receives 20 frame events/s. It therefore does not provide a 5× reduction in Python protobuf parsing or callbacks. Follow-up live instrumentation of Python and Rust codec calls attributed about 1.14% of total native-patched session CPU to protobuf encode/decode (about 0.71% Python and 0.43% Rust; 1.20% combined before timer-overhead adjustment), across two 16 s profiling runs with four sessions each. This excludes surrounding GIL acquisition, queues, PCM copies, processing, and protobuf object/field work outside the timed calls. It is profiling attribution, not an end-to-end protobuf-removal A/B. Warm message replay estimated only 0.16% including common request construction and understated the live cost; use the live measurement when assessing serialization.
Capacity projections use
new_sessions = current_sessions / (1 - measured_CPU_reduction), assuming CPU is the bottleneck, unchanged workload, and unchanged per-session cost at higher load. They apply proportionally per core or machine, are not additive across PRs, and require deployment saturation/tail-latency validation.When an unbounded audio stream requests 50 ms frames, the native sink currently wakes the Rust consumer for every 10 ms chunk and the FFI layer copies those chunks into the requested frame. Accumulate at the sink observer instead, so the Rust consumer receives one complete frame per requested interval. Move an already correctly sized frame through FFI without a second PCM copy.
The WebRTC 10 ms callback and requested Python frame duration stay the same. Bounded queues retain their existing drop-oldest accounting; streams with native audio filters and requests at or below 10 ms use the existing path. Closing the stream clears the partial accumulation buffer. The additive
NativeAudioStream::with_frame_sizeconstructor lets livekit-ffi select this behavior without changing existing option struct literals.Validation:
Performance context: receive aggregation, the separate Opus AVX2 patch, and optional native log filtering together reduced whole-session CPU by 6.21% in two paired trials (4.97% and 7.47%). This does not isolate this PR's contribution. The workload used four AgentSession processes per core, real RTC/native VAD/AGC, standard asyncio, threaded VAD, and fixture/stub cloud providers, with 16 s warmup and 64 s measurement. Both native-only trials passed the benchmark's receive, loop, playout, and local interrupt gates. These checks are not a production capacity or universal RTC-equivalence proof.
The separate combined experiment including uvloop and inline VAD reduced CPU by 15.14%, but had timing failures in both cohorts. That number must not be attributed to this PR or used as a measured capacity increase.
Related draft PRs: Opus AVX2, Python log filtering, inline VAD, RTC regression tests.