Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .changeset/harden_incoming_data_streams.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
livekit-data-stream: patch
livekit-rpc: patch
livekit: patch
livekit-ffi: patch
livekit-uniffi: patch
---

Reject incoming stream packets from other participants, enforce decompression limits before buffering output, and prevent received files from overwriting existing files or following destination symlinks.
6 changes: 6 additions & 0 deletions .changeset/network_quality_start_bitrate_hints.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
livekit: patch
libwebrtc: patch
---

Add publish-time network quality signals for start bitrate hints.
5 changes: 5 additions & 0 deletions .changeset/platform_audio_invalid_device_id_test.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
livekit: patch
---

test: cover `DeviceNotFound` for invalid PlatformAudio device IDs.
6 changes: 6 additions & 0 deletions .changeset/trim_uniffi_bindgen_from_uniffi_runtime.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
livekit-uniffi: patch
livekit-datatrack: patch
---

Stop enabling UniFFI bindgen CLI dependencies in runtime UniFFI builds.
10 changes: 10 additions & 0 deletions .changeset/use_typed_platform_audio_device_ids.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
livekit: minor
livekit-ffi: patch
---

refactor: route platform audio device selection through typed device IDs in Rust and FFI.

fix: narrow hidden platform audio index/count helpers to internal APIs and use `usize` for public device indices.

fix: return `DeviceNotFound` when selecting an unavailable platform audio device ID.
Comment on lines +6 to +10

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Platform audio changelog loses bullet format

The multi-paragraph use_typed_platform_audio_device_ids summary becomes a heading with loose body text instead of one changelog bullet.

Suggested change
refactor: route platform audio device selection through typed device IDs in Rust and FFI.
fix: narrow hidden platform audio index/count helpers to internal APIs and use `usize` for public device indices.
fix: return `DeviceNotFound` when selecting an unavailable platform audio device ID.
Route platform audio device selection through typed IDs, narrow the internal index and count helpers, use `usize` for public device indices, and return `DeviceNotFound` for unavailable IDs.

Devin Review


Was this helpful? React with 👍 or 👎 to provide feedback.

6 changes: 6 additions & 0 deletions .changeset/video_frame_drop_visibility_and_buffer_reuse.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
libwebrtc: patch
livekit-ffi: patch
---

Report dropped video frames to the application via `VideoFrameReceived.dropped_frames`, keep libwebrtc log messages at their own severity instead of flattening every one to debug, and reuse video frame conversion buffers rather than allocating one per frame.
51 changes: 49 additions & 2 deletions libwebrtc/src/native/peer_connection_factory.rs
Original file line number Diff line number Diff line change
Expand Up @@ -35,12 +35,35 @@ lazy_static! {
static ref LOG_SINK: Mutex<Option<UniquePtr<sys_rtc::ffi::LogSink>>> = Default::default();
}

/// Maps a libwebrtc severity onto the matching `log` level.
///
/// Warning and error move up so consumers can see them. Verbose and info both
/// move down a step -- libwebrtc logs both very heavily, and the sink is
/// installed at `LS_VERBOSE`, so mapping info to `Info` would flood every
/// binding that shows it. Keeping info at `Debug` preserves today's behaviour
/// for the noisy levels while making the two that matter visible.
fn log_level(severity: sys_rtc::ffi::LoggingSeverity) -> Option<log::Level> {
match severity {
sys_rtc::ffi::LoggingSeverity::Verbose => Some(log::Level::Trace),
sys_rtc::ffi::LoggingSeverity::Info => Some(log::Level::Debug),
sys_rtc::ffi::LoggingSeverity::Warning => Some(log::Level::Warn),
sys_rtc::ffi::LoggingSeverity::Error => Some(log::Level::Error),
_ => None,
}
}

fn ensure_log_sink() {
let mut log_sink = LOG_SINK.lock();
if log_sink.is_none() {
*log_sink = Some(sys_rtc::ffi::new_log_sink(|msg, _| {
*log_sink = Some(sys_rtc::ffi::new_log_sink(|msg, severity| {
// Every message used to arrive as Debug regardless of severity, so
// libwebrtc warnings and errors were discarded by any consumer
// filtering at Info -- which is every binding's default.
let Some(level) = log_level(severity) else {
return;
};
let msg = msg.strip_suffix("\r\n").or(msg.strip_suffix('\n')).unwrap_or(&msg);
log::debug!(target: "libwebrtc", "{}", msg);
log::log!(target: "libwebrtc", level, "{}", msg);
}));
}
}
Expand Down Expand Up @@ -351,6 +374,7 @@ impl PeerConnectionFactory {
mod tests {
use super::*;
use std::sync::Mutex;
use webrtc_sys::webrtc::ffi::LoggingSeverity;

static TEST_MUTEX: Mutex<()> = Mutex::new(());

Expand All @@ -373,4 +397,27 @@ mod tests {
assert_eq!(factory.adm_recording_enabled(), !initial_recording);
factory.set_adm_recording_enabled(initial_recording);
}

/// Warnings and errors used to arrive as `Debug` and were filtered out by
/// default in every binding; these pin the mapping that fixed that.
#[test]
fn libwebrtc_warnings_and_errors_map_to_visible_levels() {
assert_eq!(log_level(LoggingSeverity::Warning), Some(log::Level::Warn));
assert_eq!(log_level(LoggingSeverity::Error), Some(log::Level::Error));
}

#[test]
fn the_noisy_levels_stay_below_info() {
// libwebrtc logs both heavily and the sink is installed at LS_VERBOSE,
// so neither may reach a level a binding shows by default.
for severity in [LoggingSeverity::Verbose, LoggingSeverity::Info] {
let level = log_level(severity).expect("still forwarded");
assert!(level > log::Level::Info, "{severity:?} mapped to {level:?}");
}
}

#[test]
fn severity_none_is_dropped() {
assert_eq!(log_level(LoggingSeverity::None), None);
}
}
42 changes: 42 additions & 0 deletions libwebrtc/src/native/video_stream.rs
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,12 @@ impl NativeVideoStream {
self.video_track.clone()
}

/// Frames this stream has dropped because the consumer did not keep up
/// with the WebRTC sink, counted since the stream was created.
pub fn dropped_frames(&self) -> u64 {
self.frame_queue.dropped_frames()
}

pub fn close(&mut self) {
let video = unsafe { sys_vt::ffi::media_to_video(self.video_track.sys_handle()) };
video.remove_sink(&self.native_sink);
Expand Down Expand Up @@ -289,6 +295,10 @@ impl VideoFrameQueue {
}
}

fn dropped_frames(&self) -> u64 {
self.dropped_frames.load(Ordering::Relaxed)
}

fn record_drop(&self) {
let dropped_frames = self.dropped_frames.fetch_add(1, Ordering::Relaxed) + 1;
if dropped_frames == 1 || dropped_frames % 100 == 0 {
Expand Down Expand Up @@ -361,6 +371,38 @@ mod tests {
assert_eq!(pop_timestamp(&queue), None);
}

#[test]
fn dropped_frames_is_readable_and_cumulative() {
let queue = VideoFrameQueue::new(Some(1));

assert_eq!(queue.dropped_frames(), 0);

queue.push(test_frame(1));
assert_eq!(queue.dropped_frames(), 0, "the first frame fits");

queue.push(test_frame(2));
assert_eq!(queue.dropped_frames(), 1);

queue.push(test_frame(3));
assert_eq!(queue.dropped_frames(), 2, "the count accumulates");

// Draining does not reset it: the counter reports losses over the
// stream's whole life, so a consumer can diff it between frames.
assert_eq!(pop_timestamp(&queue), Some(3));
assert_eq!(queue.dropped_frames(), 2);
}

#[test]
fn an_unbounded_queue_never_reports_drops() {
let queue = VideoFrameQueue::new(None);

for timestamp_us in 1..=64 {
queue.push(test_frame(timestamp_us));
}

assert_eq!(queue.dropped_frames(), 0);
}

#[test]
fn unbounded_queue_retains_all_frames() {
let queue = VideoFrameQueue::new(None);
Expand Down
11 changes: 11 additions & 0 deletions libwebrtc/src/video_stream.rs
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,17 @@ pub mod native {
self.handle.track()
}

/// Frames this stream has dropped because the consumer did not keep up
/// with the WebRTC sink, counted since the stream was created.
///
/// A stream built with `queue_size_frames: Some(0)` never drops, so
/// this stays at zero. For any bounded queue a rising count means the
/// consumer is too slow for the incoming frame rate; the frames are
/// gone and no WebRTC stat reports them.
pub fn dropped_frames(&self) -> u64 {
self.handle.dropped_frames()
}

pub fn close(&mut self) {
self.handle.close();
}
Expand Down
17 changes: 17 additions & 0 deletions livekit-ffi-node-bindings/proto/video_frame_pb.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions livekit-ffi-node-bindings/proto/video_frame_pb.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 13 additions & 0 deletions livekit-ffi/protocol/video_frame.proto
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,19 @@ message VideoFrameReceived {
required int64 timestamp_us = 2; // In microseconds
required VideoRotation rotation = 3;
optional FrameMetadata metadata = 4;

// Frames this stream has dropped since it was created, because the FFI
// forwarding task did not keep up with the WebRTC sink.
//
// Dropped frames are decoded but never delivered. No WebRTC stat reports
// them -- `framesDecoded` counts them and `framesDropped` does not -- so
// this counter is the only way an application can observe the loss. It is
// cumulative and monotonic: compare it against the previous frame's value
// to find how many were lost in between.
//
// Always zero for a stream created with `queue_size_frames = 0`, which is
// unbounded and never drops.
optional uint64 dropped_frames = 5;
}

message VideoStreamEOS {}
Expand Down
Loading
Loading