[jiwenc-nv]

Description

Stack 2/3 — depends on #1077. Follow-up: #998 (the SIPL plugin).

Zero-copy handoff of camera frames from a capture process to a Python consumer on the same Jetson. The publisher exports CUDA VMM allocations and passes the OS handles over a unix socket with SCM_RIGHTS; camera_viz gains a matching type: cuda_ipc source that maps them directly, so frames never round-trip through host memory.

The VMM-plus-socket shape is not a preference: the legacy cudaIpcGetMemHandle path does not work on Tegra.

No SIPL and no camera dependency — the transport builds and is tested on its own. sensing_ipc_testsrc publishes a synthetic pattern with the frame number stamped into the pixels, so a stale or misordered buffer slot fails the test rather than looking plausible.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation update

Testing

AGX Orin, JetPack 7.2.1 / L4T R39.2.1:

• sensing_ipc_testsrc → camera_viz cuda_ipc source, rendered in window mode.
• test_cuda_ipc_source.py drives the publisher and asserts the stamped frame numbers.
• Configured and built with the SIPL SDK absent, confirming the transport pulls in no camera headers.
• SKIP=check-copyright-year pre-commit run --all-files passes.

Checklist

• I have read and understood the contribution guidelines
• I have run the linter and formatter with SKIP=check-copyright-year pre-commit run --all-files
• I have made corresponding changes to the documentation
• I have added tests that prove my fix/feature works (or explained why not)
• I have signed off all my commits (git commit -s) per the DCO

Summary by CodeRabbit

• New Features

  • Added optional CUDA IPC camera streaming between local producer and consumer processes.
  • Added support for zero-copy GPU RGBA frames, producer reconnection, and frame geometry validation.
  • Added an animated CUDA test-pattern source for development without a camera.
  • Added a ready-to-use camera visualization configuration for multiple streams.
  • Added an opt-in SENSING camera plugin build option.

• Documentation

  • Documented CUDA IPC setup, socket behavior, frame lifetime, synchronization, and usage requirements.

• Tests

  • Added protocol, configuration, recovery, frame delivery, and GPU image validation coverage.

[coderabbitai]

Important

Review skipped

The saved review history does not include the base for the last reviewed commit. This saved history cannot establish the base for an incremental review. Comment @coderabbitai full review to establish a new review baseline. No full review was started, and the last reviewed checkpoint was preserved.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

📝 Walkthrough

Adds an optional Jetson SENSING plugin with a CUDA IPC publisher and synthetic test producer. Defines a fixed wire protocol for CUDA allocation handshakes, frame notifications, and slot releases. Adds a Python camera source that maps shared CUDA memory and supports reconnects. Adds camera configuration, frame-lifetime documentation, and end-to-end tests for frame delivery, recovery, and geometry validation.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant TestProducer
  participant CudaIpcPublisher
  participant CudaIpcSource
  participant CUDADeviceMemory
  TestProducer->>CudaIpcPublisher: publish device frame
  CudaIpcPublisher->>CudaIpcSource: send handshake and frame metadata
  CudaIpcSource->>CUDADeviceMemory: import and map shared allocation
  CudaIpcSource->>CudaIpcSource: expose zero-copy RGBA8 frame
  CudaIpcSource->>CudaIpcPublisher: release consumed slot

Loading

Merge Risk: 🟡 Moderate · up to 9644f

The new transport can fail validation or deployment, leak resources on startup errors, corrupt its protocol under socket pressure, and expose or disrupt camera sharing for local users. These issues should be resolved before merge.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 25.40% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 63 functions across 10 files. (6 skipped:…	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely summarizes the main changes: adding the SENSING CUDA IPC transport and the corresponding camera_viz source.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 25.40% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 63 functions across 10 files. (6 skipped: 6 unsupported.)

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch jiwenc/sensing-cuda-ipc

_{Comment @coderabbitai help to get the list of available commands.}

[gareth-morgan-nv]

🤖 Agent-generated (Claude Code · claude-sonnet-5-5)

Summary

Adds the CUDA IPC transport: a publisher that exports one CUDA VMM allocation over SCM_RIGHTS, a type: cuda_ipc camera_viz source that maps it, a synthetic producer, and tests. The design is sound. The VMM-over-socket choice is explained where a maintainer would be tempted to undo it, the handshake bounds-checks slot_count/slot_stride/total_bytes before sizing any mapping, the mailbox drop policy is explicit, and the wire format is pinned by static_asserts mirrored in Python tests. CI is green except publish-wheel, which was cancelled during the Artifactory upload and looks infrastructural; a rerun should clear it. The main gaps are access control on the socket, a timing-based teardown on the consumer, and end-to-end tests that cannot run in CI.

Legend: 🚫 Blocker · 💡 Suggestion · 🔍 Nit

	Finding
💡	src/plugins/sensing/cuda_ipc/cuda_ipc_publisher.cpp:188 — the socket's mode follows the process umask, and "last connect wins" (:213) means any local process that can connect receives the camera allocation's fd and evicts the real viewer. Restrict the socket (fchmod 0600 before listen, and/or a SO_PEERCRED uid check in accept_client). Hardening already raised by CodeRabbit on this range. Separately, :188 calls unlink on the configured path unconditionally, so ipc= pointing at a regular file deletes it; lstat and unlink only when S_ISSOCK.
💡	examples/camera_viz/sources/cuda_ipc.py:385 — before unmapping, _teardown sleeps _RETIRE_GRACE_S (0.25 s) and assumes the render thread has stopped touching the slot views by then. A render hitch longer than that unmaps memory under an in-flight submit(). Guard the views with the existing lock (or a refcount) so the unmap waits for the renderer instead of a timer.
💡	examples/camera_viz/sources/cuda_ipc.py:183,288 — _reader_loop only catches ValueError, CudaDriverError and OSError. A producer-reported device_id that does not exist makes cp.cuda.Device(device_id).use() raise a CuPy runtime error, which ends the reader thread; the source then never reconnects and no frame arrives. Validate device_id against the device count in _handshake and catch Exception in the loop with the same back-off.
💡	tests/python/examples/camera_viz/test_cuda_ipc_source.py — every end-to-end test is skipped without a built sensing_ipc_testsrc plus a CUDA device, so CI only runs the struct-size, magic and config checks. _find_testsrc (:45) only looks in build/ and cmake-build-debug/, startup is a fixed sleep(1.5) (:136) instead of waiting for the socket, and the assert at :137 runs inside __enter__, so a failed start leaks the producer process. Nothing exercises backpressure, slot release or consumer replacement on the C++ side.
🔍	cuda_ipc_publisher.cpp:60,265 — send_record reports success as sent == size and the caller then reads errno; on a short write errno can be stale (for example from the preceding accept4), and drain_releases treats any read shorter than 8 bytes as a desync although a stream socket may split it. Short-write side already raised by CodeRabbit.
🔍	examples/camera_viz/sources/cuda_ipc.py:6,17,35 (and the test docstring) — the paths src/plugins/sensing/core/cuda_ipc_publisher.cpp and core/cuda_ipc_protocol.hpp do not exist; the files are under cuda_ipc/ and cuda_ipc/inc/sensing_cuda_ipc/.
🔍	src/plugins/sensing/CMakeLists.txt:22 — the comment says Thor is 101 but "80;86;87;89;90" omits it; add 101 or reword. Related architecture-default point already raised by CodeRabbit.
🔍	examples/camera_viz/sources/cuda_ipc.py:124 — the reader is a non-daemon thread whose loop only exits on stop(), so any path that skips stop() hangs interpreter exit. tools/ipc_testsrc.cu:144,165 uses std::runtime_error without <stdexcept> (raised by CodeRabbit) and ignores the return of publish().

Actionables (for bots — copy-paste-ready for AI)

Agent-generated suggestions, not human-vetted obligations. Skip anything wrong, already
addressed, or not worth the churn.

• src/plugins/sensing/cuda_ipc/cuda_ipc_publisher.cpp:188 — lstat the path and unlink only if it is a socket; set mode 0600 on the socket after bind and verify the peer uid with SO_PEERCRED in accept_client.
• examples/camera_viz/sources/cuda_ipc.py:385 — replace the timed grace sleep with a lock or refcount so unmapping waits for any frame the renderer still holds.
• examples/camera_viz/sources/cuda_ipc.py:183 — validate device_id in _handshake and catch Exception (log and back off) in _reader_loop so the reader cannot die silently.
• tests/python/examples/camera_viz/test_cuda_ipc_source.py:136 — wait for the socket to appear instead of sleeping, move the startup assert out of __enter__ (or stop the process on failure), and resolve the producer binary from the CMake build directory.
• examples/camera_viz/sources/cuda_ipc.py:6 — point the docstring paths at cuda_ipc/ and cuda_ipc/inc/sensing_cuda_ipc/.