Feat/vulkan video recorder#19
Open
dennisdevulder wants to merge 5 commits into
Open
Conversation
Queue families are fixed at vkCreateDevice, so a recorder plugin cannot add an encode queue later. When the device advertises an encode-capable queue family plus VK_KHR_video_queue, VK_KHR_video_encode_queue and at least one codec extension, enable them and create a dedicated encode queue up front. A shared graphics family yields a second queue at lower priority; a single-queue family yields none, since sharing the render queue with a recorder thread is unsafe. synchronization2 is enabled alongside (required by the video extensions) and gated on device API 1.3. -Dvkgpu.disableVideoEncode=true switches the whole path off. VulkanEncodeContext now returns live queue handles, and unavailableReason() states the exact refusal. recordAfterComposite hands extensions the final composited frame on the graphics command buffer after the last render pass, at the same point the screenshot readback runs, with a documented layout-restore contract — the capture point for recorder-style consumers. Assisted-by: Claude Fable 5
Proves the encode plumbing added in the previous commit actually works: VideoRecorderExtension grabs the composited frame in recordAfterComposite, converts it to NV12 (BT.709 limited) with a compute pass on the graphics queue, and hands it to H264EncodeSession on the video encode queue from a worker thread. Output is an Annex-B .h264 elementary stream (IDR-only, constant QP, rate control disabled where supported) in ~/.runelite/vulkan-recordings/. Cross-queue ordering uses a new frame timeline semaphore: every graphics submit signals a monotonically increasing value, the hook context carries the value for its frame, and the recorder host-waits it before submitting encode work. The encode-enabled device now also enables the timelineSemaphore feature (mandatory in 1.3 alongside synchronization2). LWJGL moves 3.3.2 -> 3.3.6, core and vulkan together: 3.3.2 only ships the provisional video encode bindings, which no current driver implements. The 45f09ed Windows breakage came from mixing core and vulkan versions, not from 3.3.6 itself; the pin comment in build.gradle now says so. WinBase.GetModuleHandle gained a lastError parameter in 3.3.6. Recording survives canvas resizes by rolling over to a new file and session. Not available on macOS (MoltenVK has no Vulkan Video); the config toggle no-ops with a logged reason. Assisted-by: Claude Fable 5
Encoded frames now land in an in-memory ring covering the last clipSeconds (default 10) instead of streaming to disk; a configurable hotkey snapshots the ring and writes it as clip-<time>.h264 on a background thread. IDR-only encoding pays off here: the ring can be trimmed and cut at any frame without reference bookkeeping. A capture FPS cap (default 30) bounds memory — standalone IDR frames are large, and unlocked render FPS would balloon the buffer. A 256 MiB hard cap backstops the time window and logs once when hit. Canvas resizes restart the buffer, since frames with different SPS dimensions cannot share a stream. The hotkey is registered by the plugin through KeyManager and forwards to the extension, which stays plugin-agnostic. Assisted-by: Claude Fable 5
Clips now land as clip-<time>.mp4 instead of a raw Annex-B elementary stream. Mp4Writer is a minimal ISO BMFF muxer for exactly what the replay buffer produces: one H.264 video track, every sample an IDR picture (no stss box - absence marks all samples sync), one chunk, per-sample durations taken from real capture timestamps so variable capture pacing plays back correctly. Output is written in one pass: ftyp + mdat + moov, no temp files, no re-encode - the muxing cost is container bookkeeping around already-compressed bytes on the clip writer thread. Annex-B start codes are rewritten to 4-byte AVCC length prefixes for mdat; avcC carries the session's SPS/PPS verbatim. Validated against a real libx264 IDR-only stream: ffprobe reports the expected codec/dimensions/frame count and a full decode runs clean. Unit test covers box tiling, mdat sizing, and parameter-set embedding. Assisted-by: Claude Fable 5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.