feat(teleop): show live camera feeds beside the 3D viewer#42
Open
cn0303 wants to merge 1 commit into
Open
Conversation
Add an optional camera panel to the teleoperation page, off by default so landing on the page never calls getUserMedia (same consent pattern as the calibration camera toggle). Teleoperation opens no cv2 cameras, so the browser can stream them directly while the arm runs. The panel mirrors the selected robot's configured cameras: it shows a live feed for each camera on the robot record (e.g. wrist_cam, webcam), stacked vertically, and shows nothing when the robot has none -- teleop never surfaces a device that was not deliberately configured.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds an optional camera panel to the teleoperation page so you can watch your cameras live while
you drive the arm; handy for verifying framing and catching a camera that came loose.
getUserMedia; the user opts in with atoggle; the same consent pattern as the calibration camera toggle (fix(calibration): don't open cameras until the user turns them on #21).
selected robot record (e.g.
wrist_cam,webcam), stacked vertically with their names.add them during calibration — it never surfaces a device that wasn't deliberately configured.
it's obvious it's expected but not detected.
Why this works during teleop
Teleoperation opens no cv2 cameras (
SO101FollowerConfighas nocameras), so the browser isfree to stream them directly via
getUserMediawhile the arm runs. (Recording is different; therecv2 owns the devices, so this browser-source approach is teleop-only by design.)
Implementation notes
CameraFeedis a small presentational component bound to a browserdeviceId.TeleopCameraPanelreads the selected robot's cameras viauseRobots()and renders aCameraFeedper configured camera. A retry button remounts the feeds for a fresh
getUserMediaattempt that ishandy if a camera was unplugged and reconnected. (No backend camera enumeration: the configured
cameras already carry the browser
device_idwe stream directly.)VisualizerPanelvia a newrightSlot.Testing
unplugged-but-configured shows the placeholder.
npx tsc --noEmitclean; ESLint clean on the changed files (the repo's fullnpm run linthaspre-existing errors/warnings unrelated to this change);
npm run buildsucceeds.frontend/distis left to CI'sbuild_frontend.ymlto rebuild on merge.