diff --git a/.gitignore b/.gitignore index 45c2754576..3b4043a02c 100644 --- a/.gitignore +++ b/.gitignore @@ -59,5 +59,9 @@ project.spdx # MuJoCo writes this to the working directory, not next to the model. MUJOCO_LOG.TXT +# examples/mcap_record_replay writes here by default (Path.cwd() / "recordings"), +# and its README has you run from the repo root. +/recordings/ + # Runtime device files injected by MCP server infrastructure .mcp.json diff --git a/docs/source/device/trackers.rst b/docs/source/device/trackers.rst index ba6bbfcfed..a8ccc1c818 100644 --- a/docs/source/device/trackers.rst +++ b/docs/source/device/trackers.rst @@ -260,9 +260,9 @@ reads the PICO ``XR_BD_body_tracking`` extension directly. - :code-file:`examples/schemaio/full_body_printer.cpp` - :code-file:`examples/mcap_record_replay/cpp/record_full_body.cpp` - - :code-file:`examples/mcap_record_replay/python/live_full_body.py` - - :code-file:`examples/mcap_record_replay/python/record_full_body.py` - - :code-file:`examples/mcap_record_replay/python/replay_full_body.py` + - :code-file:`examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/live_full_body.py` + - :code-file:`examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/record_full_body.py` + - :code-file:`examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/replay_full_body.py` .. note:: diff --git a/docs/source/references/mcap_record_replay.rst b/docs/source/references/mcap_record_replay.rst index eb53d36953..5fd51939fc 100644 --- a/docs/source/references/mcap_record_replay.rst +++ b/docs/source/references/mcap_record_replay.rst @@ -130,7 +130,7 @@ Runnable Example ---------------- A complete record / replay example lives at -``examples/mcap_record_replay/python/``: +``examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/``: - ``common.py`` — pipeline builders (``build_hand_pipeline()``, ``build_controller_pipeline()``, ``build_full_body_pipeline()``) plus the @@ -168,19 +168,18 @@ A C++ recorder lives at ``examples/mcap_record_replay/cpp/``: - ``record_full_body.cpp`` — records the ``full_body`` channel by passing a ``core::McapRecordingConfig`` to ``DeviceIOSession::run()``. It uses the same channel base name as ``FullBodySource``, so the resulting file replays - with ``replay_full_body.py``. + with ``replay_full_body``. Live preview ^^^^^^^^^^^^ -From the example directory: +From the repo root: .. code-block:: bash - cd examples/mcap_record_replay/python - uv sync - uv run python live_full_body.py --accept-eula - uv run python live_full_body.py --port 8090 --accept-eula # change viser port + uv pip install -e ./examples/mcap_record_replay + python -m isaacteleop_examples.mcap_record_replay.live_full_body --accept-eula + python -m isaacteleop_examples.mcap_record_replay.live_full_body --port 8090 --accept-eula Open the printed URL (default ``http://localhost:8080``) in a browser. The viewers bind every interface, since they run where the headset is and get @@ -189,23 +188,25 @@ opened from another machine; pass ``--host 127.0.0.1`` to keep one local. Recording ^^^^^^^^^ -From the example directory: +From the repo root: .. code-block:: bash - cd examples/mcap_record_replay/python - uv sync - uv run python record_hand.py # 5 s → ../recordings/hands_.mcap - uv run python record_hand.py 10 # record for 10 seconds - uv run python record_hand.py 10 out.mcap # custom output path + uv pip install -e ./examples/mcap_record_replay + R="python -m isaacteleop_examples.mcap_record_replay.record_hand" + $R # 5 s → ./recordings/hands_.mcap + $R 10 # record for 10 seconds + $R 10 out.mcap # custom output path + +Recordings are written to ``./recordings/`` relative to where you run the +command, and the replay scripts look there when given no path. The example never downloads a published wheel — it runs against the -``isaacteleop`` next to it. Above, ``uv sync`` builds it from this checkout; the -first sync compiles the extension modules and takes a few minutes, later ones -reuse the cached build, and the install is editable, so edits under -``src/python/`` need no rebuild. From -``install/examples/mcap_record_replay/python`` (after ``cmake --install``) it -picks up the wheel you just built instead — see +``isaacteleop`` next to it, built from this checkout. The first install +compiles the extension modules and takes a few minutes; later ones reuse the +cached build, and the install is editable, so edits under ``src/python/`` need +no rebuild. From ``install/examples/mcap_record_replay`` (after ``cmake +--install``) it picks up the wheel you just built instead — see :doc:`/getting_started/build_from_source/index`. An active OpenXR runtime / headset must be connected, just like any other @@ -221,19 +222,23 @@ Replay runs headless — no headset required: .. code-block:: bash - uv run python replay_hand.py # newest file in ../recordings/ - uv run python replay_hand.py path/to/file.mcap # explicit file - uv run python replay_hand.py --loop # repeat until Ctrl+C - uv run python replay_hand.py --port 8090 # change viser port + R="python -m isaacteleop_examples.mcap_record_replay.replay_hand" + $R # newest hands_*.mcap under ./recordings/ + $R path/to/file.mcap # explicit file + $R --loop # repeat until Ctrl+C + $R --port 8090 # change viser port Open the printed URL (default ``http://localhost:8080``) in a browser to see the left (green) and right (blue) hand skeletons update each frame. -The ``record_controller.py`` / ``replay_controller.py`` and -``record_full_body.py`` / ``replay_full_body.py`` pairs use the same CLI -(positional MCAP path, ``--host``, ``--port``, ``--loop``). -``live_full_body.py`` accepts ``--host``, ``--port``, and the CloudXR launcher -flags (including ``--accept-eula``). +The ``record_controller`` / ``replay_controller`` and ``record_full_body`` / +``replay_full_body`` module pairs use the same CLI (positional MCAP path, +``--host``, ``--port``, ``--loop``). ``live_full_body`` accepts ``--host``, +``--port``, and the CloudXR launcher flags (including ``--accept-eula``). + +``uv run`` (instead of an activated venv) needs an explicit ``--python 3.11`` +— without it, ``uv`` may resolve against the system Python and fail to match +the lockfile. API Reference ------------- diff --git a/examples/deviceio_live_view/python/isaacteleop_examples/deviceio_live_view/deviceio_viser.py b/examples/deviceio_live_view/python/isaacteleop_examples/deviceio_live_view/deviceio_viser.py index 5feac7f94f..059d32951a 100644 --- a/examples/deviceio_live_view/python/isaacteleop_examples/deviceio_live_view/deviceio_viser.py +++ b/examples/deviceio_live_view/python/isaacteleop_examples/deviceio_live_view/deviceio_viser.py @@ -42,6 +42,9 @@ TRACKED_COLOR: tuple[float, float, float] = (0.25, 0.85, 0.35) +# Duplicated verbatim in mcap_record_replay/common.py: each example package +# is self-contained (see examples/README.md), so this stays a copy rather +# than a cross-example dependency. Keep the two in sync by hand. class GroundGrid: """The ground plane and the default camera, anchored to what is tracked. diff --git a/examples/mcap_record_replay/CMakeLists.txt b/examples/mcap_record_replay/CMakeLists.txt index 7a977c5e27..d59cf24f12 100644 --- a/examples/mcap_record_replay/CMakeLists.txt +++ b/examples/mcap_record_replay/CMakeLists.txt @@ -7,4 +7,4 @@ cmake_minimum_required(VERSION 3.20) add_subdirectory(cpp) include(${CMAKE_SOURCE_DIR}/cmake/InstallPythonExample.cmake) -install_python_example(DESTINATION examples/mcap_record_replay/python) +install_python_example(DESTINATION examples/mcap_record_replay) diff --git a/examples/mcap_record_replay/README.md b/examples/mcap_record_replay/README.md new file mode 100644 index 0000000000..4ebcf7e88f --- /dev/null +++ b/examples/mcap_record_replay/README.md @@ -0,0 +1,41 @@ + + +# MCAP record / replay + +Record DeviceIO tracking to an MCAP file and replay it into a viser 3D view. +Live viewers are included for watching without recording. + +```bash +uv pip install -e ./examples/mcap_record_replay +python -m isaacteleop_examples.mcap_record_replay.record_hand # 5 s +python -m isaacteleop_examples.mcap_record_replay.replay_hand # newest take +``` + +Recordings are written to `./recordings/` relative to where you run the +command; a replay given no path picks the newest matching recording there +(e.g. `replay_hand` picks the newest `hands_*.mcap`) and exits if none +match -- no silent fallback to a wrong-type recording. + +`uv run` (instead of an activated venv) needs an explicit `--python 3.11` -- +without it, `uv` may resolve against the system Python and fail to match the +lockfile. + +The live and replay viewers bind every interface, so a browser on another +machine can reach them at `http://:8080`. Pass `--host 127.0.0.1` to +keep a viewer local. + +| Channel | Live | Record | Replay | +| --- | --- | --- | --- | +| Hands | `live_hand` | `record_hand` | `replay_hand` | +| Controllers | `live_controller` | `record_controller` | `replay_controller` | +| Full body | `live_full_body` | `record_full_body` | `replay_full_body` | +| VIVE SE3 trackers | — | `record_se3_vive` | `replay_se3_vive` | + +`record_*` takes an optional duration in seconds and an optional output path. +Recording needs a live OpenXR runtime; replay needs only the file. + +A C++ recorder lives in `cpp/`. Docs: +`docs/source/references/mcap_record_replay.rst`. diff --git a/examples/mcap_record_replay/python/pyproject.toml b/examples/mcap_record_replay/pyproject.toml similarity index 53% rename from examples/mcap_record_replay/python/pyproject.toml rename to examples/mcap_record_replay/pyproject.toml index 440fe3d8c2..9bed222988 100644 --- a/examples/mcap_record_replay/python/pyproject.toml +++ b/examples/mcap_record_replay/pyproject.toml @@ -1,8 +1,14 @@ # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + [project] -name = "mcap-record-replay-example" +# The dist name mirrors the import path, so an installed example claims no bare +# top-level name in site-packages. +name = "isaacteleop-examples-mcap-record-replay" version = "0.0.0" # Internal example - not versioned description = "Isaac Teleop MCAP record / replay example with viser visualization" requires-python = ">=3.11,<3.14" @@ -15,8 +21,16 @@ dependencies = [ "viser>=0.2.0", ] +# `isaacteleop_examples` is a PEP 420 namespace shared by every example dist and +# must stay without an __init__.py. `only-include` + `sources`, not `packages`: +# the latter keeps only the last path component and would root the wheel at a +# bare top-level `mcap_record_replay/`. +[tool.hatch.build.targets.wheel] +only-include = ["python/isaacteleop_examples/mcap_record_replay"] +sources = ["python"] + [tool.uv.sources] # Editable, so edits under src/python/ need no rebuild. Dropped from the # installed copy, which has no repo above it. # TODO(#880): nothing runs this example in CI; neither path is covered. -isaacteleop = { path = "../../..", editable = true } +isaacteleop = { path = "../..", editable = true } diff --git a/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/__init__.py b/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/__init__.py new file mode 100644 index 0000000000..52a7a9daf0 --- /dev/null +++ b/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/__init__.py @@ -0,0 +1,2 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 diff --git a/examples/mcap_record_replay/python/common.py b/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/common.py similarity index 85% rename from examples/mcap_record_replay/python/common.py rename to examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/common.py index 835db44bd8..f4aea8d3c9 100644 --- a/examples/mcap_record_replay/python/common.py +++ b/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/common.py @@ -48,6 +48,74 @@ ) +# Duplicated verbatim in deviceio_live_view/deviceio_viser.py: each example +# package is self-contained (see examples/README.md), so this stays a copy +# rather than a cross-example dependency. Keep the two in sync by hand. +class GroundGrid: + """The ground plane and the default camera, anchored to what is tracked. + + The session asks OpenXR for a stage (floor-relative) space, but a runtime + that cannot supply one falls back to a head-relative origin: y=0 then sits + at eye height and the skeleton hangs below a grid drawn at zero. Following + the lowest tracked joint puts the grid on the floor in either space, and + the camera is framed against that floor rather than against y=0 -- aiming + at a fixed height leaves the subject at the bottom of the viewport in a + head-relative space. + """ + + def __init__(self, server, handle, smoothing: float = 0.05): + self._server = server + self._handle = handle + self._smoothing = smoothing + self._y: float | None = None + + @server.on_client_connect + def _(client) -> None: + self._frame(client) + + def _frame(self, client) -> None: + """Stand back from the floor at eye height, looking at torso height.""" + floor = 0.0 if self._y is None else self._y + client.camera.position = (0.0, floor + 1.5, 2.5) + client.camera.look_at = (0.0, floor + 0.9, 0.0) + + def follow(self, positions: np.ndarray, valid: np.ndarray) -> None: + points = np.asarray(positions, dtype=np.float32)[np.asarray(valid, dtype=bool)] + if points.size == 0: + return + lowest = float(np.min(points[:, 1])) + # Ease toward it: a single mistracked frame should not drop the floor. + first = self._y is None + self._y = lowest if first else self._y + self._smoothing * (lowest - self._y) + self._handle.position = (0.0, self._y, 0.0) + + # Re-aim once, when the floor is first known. Doing it every frame would + # fight the mouse. + if first: + for client in self._server.get_clients().values(): + self._frame(client) + + +def setup_scene(server) -> GroundGrid: + """Up axis, ground grid and a starting camera, shared by every viewer here. + + viser's ``add_grid`` defaults to the XY plane, which stands up as a wall + once the up direction is +y -- it has to be ``xz`` to lie on the ground. + Returns the grid so a caller with tracked joints can keep it on the floor. + """ + server.scene.set_up_direction("+y") + grid = server.scene.add_grid( + name="/grid", + width=6.0, + height=6.0, + plane="xz", + cell_size=0.25, + section_size=1.0, + ) + + return GroundGrid(server, grid) + + _ZERO_POSITIONS = np.zeros((NUM_HAND_JOINTS, 3), dtype=np.float32) @@ -456,7 +524,8 @@ def update(self, state: dict) -> None: class FullBodyViz: """Viser handles for full-body skeleton (joint cloud + skeleton segments).""" - def __init__(self, server: viser.ViserServer): + def __init__(self, server: viser.ViserServer, ground: "GroundGrid | None" = None): + self._ground = ground self.color = np.array(TRACKED_COLOR, dtype=np.float32) zero_pts = np.zeros((len(BODY_JOINT_NAMES), 3), dtype=np.float32) zero_segs = np.zeros((0, 2, 3), dtype=np.float32) @@ -494,3 +563,6 @@ def update(self, positions: np.ndarray | None, valid: np.ndarray | None) -> None segs = _valid_bone_segments(positions, valid_bool) self.bones.points = segs self.bones.colors = np.tile(self.color, (segs.shape[0], 2, 1)) + + if self._ground is not None: + self._ground.follow(positions, valid_bool) diff --git a/examples/mcap_record_replay/python/live_controller.py b/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/live_controller.py similarity index 91% rename from examples/mcap_record_replay/python/live_controller.py rename to examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/live_controller.py index 850fe12802..a13e91535d 100644 --- a/examples/mcap_record_replay/python/live_controller.py +++ b/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/live_controller.py @@ -5,12 +5,13 @@ Visualize live OpenXR controller poses in real time with viser. ``CloudXRLauncher`` starts the CloudXR runtime and WSS proxy automatically. -Open the URL viser prints (default http://localhost:8080) in a browser to see +Open the URL viser prints in a browser (binds all interfaces, so another +machine can reach it at http://:8080) to see aim / grip points for both controllers, a ray between them, and a live HUD showing thumbstick, trigger, squeeze, and button state. Usage: - python live_controller.py [--port 8080] [--host 127.0.0.1] [--accept-eula] + python -m isaacteleop_examples.mcap_record_replay.live_controller [--port 8080] [--host 127.0.0.1] [--accept-eula] Press Ctrl+C to stop. @@ -26,7 +27,8 @@ from isaacteleop.cloudxr import CloudXRLauncher from isaacteleop.teleop_session_manager import TeleopSession, TeleopSessionConfig -from common import ( +from .common import ( + setup_scene, ControllerViz, LEFT_COLOR, RIGHT_COLOR, @@ -47,8 +49,7 @@ def main(argv: list[str]) -> int: args = parser.parse_args(argv[1:]) server = viser.ViserServer(host=args.host, port=args.port) - server.scene.set_up_direction("+y") - server.scene.add_grid(name="/grid", width=2.0, height=2.0, cell_size=0.1) + setup_scene(server) config = TeleopSessionConfig( app_name="LiveControllerExample", diff --git a/examples/mcap_record_replay/python/live_full_body.py b/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/live_full_body.py similarity index 88% rename from examples/mcap_record_replay/python/live_full_body.py rename to examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/live_full_body.py index 883a1f39d9..4de6efa71f 100644 --- a/examples/mcap_record_replay/python/live_full_body.py +++ b/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/live_full_body.py @@ -5,12 +5,13 @@ Visualize live OpenXR full-body pose tracking in real time with viser. ``CloudXRLauncher`` starts the CloudXR runtime and WSS proxy automatically. -Open the URL viser prints (default http://localhost:8080) in a browser to see +Open the URL viser prints in a browser (binds all interfaces, so another +machine can reach it at http://:8080) to see the full PICO body skeleton — joints colored green when valid, red when lost — updating live as you move. Usage: - python live_full_body.py [--port 8080] [--host 127.0.0.1] [--accept-eula] + python -m isaacteleop_examples.mcap_record_replay.live_full_body [--port 8080] [--host 127.0.0.1] [--accept-eula] Press Ctrl+C to stop. @@ -28,7 +29,7 @@ from isaacteleop.retargeting_engine.tensor_types.indices import FullBodyInputIndex from isaacteleop.teleop_session_manager import TeleopSession, TeleopSessionConfig -from common import BODY_JOINT_NAMES, FullBodyViz, build_full_body_pipeline +from .common import BODY_JOINT_NAMES, FullBodyViz, build_full_body_pipeline, setup_scene def main(argv: list[str]) -> int: @@ -43,8 +44,7 @@ def main(argv: list[str]) -> int: args = parser.parse_args(argv[1:]) server = viser.ViserServer(host=args.host, port=args.port) - server.scene.set_up_direction("+y") - server.scene.add_grid(name="/grid", width=2.0, height=2.0, cell_size=0.1) + ground = setup_scene(server) config = TeleopSessionConfig( app_name="LiveFullBodyExample", @@ -57,7 +57,7 @@ def main(argv: list[str]) -> int: print("[live] waiting for headset connection… (Ctrl+C to stop)") with TeleopSession(config) as session: - viz = FullBodyViz(server) + viz = FullBodyViz(server, ground) print( f"[live] viser listening on {args.host}:{args.port} " f"(http://localhost:{args.port})" diff --git a/examples/mcap_record_replay/python/live_hand.py b/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/live_hand.py similarity index 90% rename from examples/mcap_record_replay/python/live_hand.py rename to examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/live_hand.py index 7349dea622..8ad82ad5de 100644 --- a/examples/mcap_record_replay/python/live_hand.py +++ b/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/live_hand.py @@ -5,11 +5,12 @@ Visualize live OpenXR hand-tracking in real time with viser. ``CloudXRLauncher`` starts the CloudXR runtime and WSS proxy automatically. -Open the URL viser prints (default http://localhost:8080) in a browser to see +Open the URL viser prints in a browser (binds all interfaces, so another +machine can reach it at http://:8080) to see both hands rendered as joint clouds + bone segments, updating live as you move. Usage: - python live_hand.py [--port 8080] [--host 127.0.0.1] [--accept-eula] + python -m isaacteleop_examples.mcap_record_replay.live_hand [--port 8080] [--host 127.0.0.1] [--accept-eula] Press Ctrl+C to stop. @@ -26,7 +27,7 @@ from isaacteleop.cloudxr import CloudXRLauncher from isaacteleop.teleop_session_manager import TeleopSession, TeleopSessionConfig -from common import HandViz, LEFT_COLOR, RIGHT_COLOR, build_hand_pipeline +from .common import HandViz, LEFT_COLOR, RIGHT_COLOR, build_hand_pipeline, setup_scene def main(argv: list[str]) -> int: @@ -41,8 +42,7 @@ def main(argv: list[str]) -> int: args = parser.parse_args(argv[1:]) server = viser.ViserServer(host=args.host, port=args.port) - server.scene.set_up_direction("+y") - server.scene.add_grid(name="/grid", width=2.0, height=2.0, cell_size=0.1) + setup_scene(server) config = TeleopSessionConfig( app_name="LiveHandExample", diff --git a/examples/mcap_record_replay/python/record_controller.py b/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/record_controller.py similarity index 91% rename from examples/mcap_record_replay/python/record_controller.py rename to examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/record_controller.py index 87fb58bce3..2d074941c3 100644 --- a/examples/mcap_record_replay/python/record_controller.py +++ b/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/record_controller.py @@ -10,9 +10,9 @@ exactly the ``controllers`` channel — no head, no hands. Usage: - python record_controller.py [duration_seconds] [output.mcap] [--accept-eula] + python -m isaacteleop_examples.mcap_record_replay.record_controller [duration_seconds] [output.mcap] [--accept-eula] -Defaults: 5 seconds → ../recordings/controllers_.mcap +Defaults: 5 seconds → ./recordings/controllers_.mcap See: https://nvidia.github.io/IsaacTeleop/main/references/mcap_record_replay.html """ @@ -28,7 +28,7 @@ from isaacteleop.retargeting_engine.tensor_types.indices import ControllerInputIndex from isaacteleop.teleop_session_manager import TeleopSession, TeleopSessionConfig -from common import build_controller_pipeline +from .common import build_controller_pipeline def main(argv: list[str]) -> int: @@ -46,7 +46,7 @@ def main(argv: list[str]) -> int: mcap_path = Path(args.output) mcap_path.parent.mkdir(parents=True, exist_ok=True) else: - out_dir = Path(__file__).resolve().parent.parent / "recordings" + out_dir = Path.cwd() / "recordings" out_dir.mkdir(exist_ok=True) mcap_path = out_dir / f"controllers_{datetime.now():%Y%m%d_%H%M%S}.mcap" diff --git a/examples/mcap_record_replay/python/record_full_body.py b/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/record_full_body.py similarity index 91% rename from examples/mcap_record_replay/python/record_full_body.py rename to examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/record_full_body.py index d2c76c3577..c616c3ace1 100644 --- a/examples/mcap_record_replay/python/record_full_body.py +++ b/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/record_full_body.py @@ -10,9 +10,9 @@ ``TeleopSession`` records the ``full_body`` and ``controllers`` channels. Usage: - python record_full_body.py [duration_seconds] [output.mcap] [--accept-eula] + python -m isaacteleop_examples.mcap_record_replay.record_full_body [duration_seconds] [output.mcap] [--accept-eula] -Defaults: 5 seconds → ../recordings/full_body_.mcap +Defaults: 5 seconds → ./recordings/full_body_.mcap See: https://nvidia.github.io/IsaacTeleop/main/references/mcap_record_replay.html """ @@ -30,7 +30,7 @@ from isaacteleop.retargeting_engine.tensor_types.indices import FullBodyInputIndex from isaacteleop.teleop_session_manager import TeleopSession, TeleopSessionConfig -from common import BODY_JOINT_NAMES, build_full_body_pipeline +from .common import BODY_JOINT_NAMES, build_full_body_pipeline def main(argv: list[str]) -> int: @@ -48,7 +48,7 @@ def main(argv: list[str]) -> int: mcap_path = Path(args.output) mcap_path.parent.mkdir(parents=True, exist_ok=True) else: - out_dir = Path(__file__).resolve().parent.parent / "recordings" + out_dir = Path.cwd() / "recordings" out_dir.mkdir(exist_ok=True) mcap_path = out_dir / f"full_body_{datetime.now():%Y%m%d_%H%M%S}.mcap" diff --git a/examples/mcap_record_replay/python/record_hand.py b/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/record_hand.py similarity index 91% rename from examples/mcap_record_replay/python/record_hand.py rename to examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/record_hand.py index cefa06ee8a..b8c9b644fa 100644 --- a/examples/mcap_record_replay/python/record_hand.py +++ b/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/record_hand.py @@ -10,9 +10,9 @@ the ``hands`` channel — no head, no controllers. Usage: - python record_hand.py [duration_seconds] [output.mcap] [--accept-eula] + python -m isaacteleop_examples.mcap_record_replay.record_hand [duration_seconds] [output.mcap] [--accept-eula] -Defaults: 5 seconds → ../recordings/hands_.mcap +Defaults: 5 seconds → ./recordings/hands_.mcap See: https://nvidia.github.io/IsaacTeleop/main/references/mcap_record_replay.html """ @@ -27,7 +27,7 @@ from isaacteleop.deviceio import McapRecordingConfig from isaacteleop.teleop_session_manager import TeleopSession, TeleopSessionConfig -from common import build_hand_pipeline +from .common import build_hand_pipeline def main(argv: list[str]) -> int: @@ -45,7 +45,7 @@ def main(argv: list[str]) -> int: mcap_path = Path(args.output) mcap_path.parent.mkdir(parents=True, exist_ok=True) else: - out_dir = Path(__file__).resolve().parent.parent / "recordings" + out_dir = Path.cwd() / "recordings" out_dir.mkdir(exist_ok=True) mcap_path = out_dir / f"hands_{datetime.now():%Y%m%d_%H%M%S}.mcap" diff --git a/examples/mcap_record_replay/python/record_se3_vive.py b/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/record_se3_vive.py similarity index 94% rename from examples/mcap_record_replay/python/record_se3_vive.py rename to examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/record_se3_vive.py index 3e19b47c7d..c46ab11326 100644 --- a/examples/mcap_record_replay/python/record_se3_vive.py +++ b/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/record_se3_vive.py @@ -7,7 +7,7 @@ Reads the per-tracker "vive_tracker_" tensor collections pushed by the vive_se3_tracker plugin (one core.Se3Tracker per collection) and records each one to its own MCAP channel pair ( / _tracked) via the standard -Se3TrackerRecordingTraits. Replay with replay_se3_vive.py. +Se3TrackerRecordingTraits. Replay with replay_se3_vive. Prerequisites (separate terminals): 1. CloudXR runtime: python -m isaacteleop.cloudxr @@ -15,11 +15,11 @@ (VIVEHub tracker_server running, or VIVE_SE3_SYNTHETIC=1 for a smoke test) With no arguments it records 10 s of every collection the pusher currently -advertises (see --collections), to a timestamped file under ../recordings/. +advertises (see --collections), to a timestamped file under ./recordings/. Usage: source ~/.cloudxr/run/cloudxr.env - uv run record_se3_vive.py [duration_s] [output.mcap] [--collections a,b,c] + python -m isaacteleop_examples.mcap_record_replay.record_se3_vive [duration_s] [output.mcap] [--collections a,b,c] """ import argparse @@ -85,7 +85,7 @@ def main(argv: list[str]) -> int: mcap_path = Path(args.output) mcap_path.parent.mkdir(parents=True, exist_ok=True) else: - out_dir = Path(__file__).resolve().parent.parent / "recordings" + out_dir = Path.cwd() / "recordings" out_dir.mkdir(exist_ok=True) mcap_path = out_dir / f"se3_vive_{datetime.now():%Y%m%d_%H%M%S}.mcap" diff --git a/examples/mcap_record_replay/python/replay_controller.py b/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/replay_controller.py similarity index 90% rename from examples/mcap_record_replay/python/replay_controller.py rename to examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/replay_controller.py index 426d9b7c3b..c49019aa3e 100644 --- a/examples/mcap_record_replay/python/replay_controller.py +++ b/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/replay_controller.py @@ -10,9 +10,10 @@ HUD (thumbstick, trigger, squeeze, buttons). Usage: - python replay_controller.py [path/to/file.mcap] [--port 8080] [--loop] + python -m isaacteleop_examples.mcap_record_replay.replay_controller [path/to/file.mcap] [--port 8080] [--loop] -If no path is given, the newest file under ``../recordings/`` is used. +If no path is given, the newest ``controllers_*.mcap`` under ``./recordings/`` is +used. ``--loop`` keeps replaying the file end-to-end until the process is killed. See: https://nvidia.github.io/IsaacTeleop/main/references/mcap_record_replay.html @@ -33,7 +34,8 @@ TeleopSessionConfig, ) -from common import ( +from .common import ( + setup_scene, ControllerViz, LEFT_COLOR, RIGHT_COLOR, @@ -67,12 +69,12 @@ def resolve_mcap(path_arg: str | None) -> Path: sys.exit(f"[replay] error: {path} does not exist") return path - recordings = Path(__file__).resolve().parent.parent / "recordings" - candidates = list(recordings.glob("*.mcap")) + recordings = Path.cwd() / "recordings" + candidates = list(recordings.glob("controllers_*.mcap")) if not candidates: sys.exit( - f"[replay] error: no .mcap files in {recordings}. " - "Run record_controller.py first or pass a path." + f"[replay] error: no controllers_*.mcap in {recordings}. " + "Run record_controller first or pass a path." ) return max(candidates, key=lambda p: p.stat().st_mtime) @@ -136,8 +138,7 @@ def main(argv: list[str]) -> int: duration_s = mcap_duration_s(mcap_path) server = viser.ViserServer(host=args.host, port=args.port) - server.scene.set_up_direction("+y") - server.scene.add_grid(name="/grid", width=2.0, height=2.0, cell_size=0.1) + setup_scene(server) viz_left = ControllerViz(server, "controller_left", LEFT_COLOR) viz_right = ControllerViz(server, "controller_right", RIGHT_COLOR) diff --git a/examples/mcap_record_replay/python/replay_full_body.py b/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/replay_full_body.py similarity index 88% rename from examples/mcap_record_replay/python/replay_full_body.py rename to examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/replay_full_body.py index 336a2f4b03..70a2fb3cb6 100644 --- a/examples/mcap_record_replay/python/replay_full_body.py +++ b/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/replay_full_body.py @@ -9,9 +9,10 @@ http://localhost:8080) in a browser to see the body skeleton. Usage: - python replay_full_body.py [path/to/file.mcap] [--port 8080] [--loop] + python -m isaacteleop_examples.mcap_record_replay.replay_full_body [path/to/file.mcap] [--port 8080] [--loop] -If no path is given, the newest file under ``../recordings/`` is used. +If no path is given, the newest ``full_body_*.mcap`` under ``./recordings/`` is +used. ``--loop`` keeps replaying the file end-to-end until the process is killed. See: https://nvidia.github.io/IsaacTeleop/main/references/mcap_record_replay.html @@ -34,7 +35,7 @@ TeleopSessionConfig, ) -from common import BODY_JOINT_NAMES, FullBodyViz, build_full_body_pipeline +from .common import BODY_JOINT_NAMES, FullBodyViz, build_full_body_pipeline, setup_scene def mcap_duration_s(path: Path) -> float: @@ -62,12 +63,12 @@ def resolve_mcap(path_arg: str | None) -> Path: sys.exit(f"[replay] error: {path} does not exist") return path - recordings = Path(__file__).resolve().parent.parent / "recordings" - candidates = list(recordings.glob("*.mcap")) + recordings = Path.cwd() / "recordings" + candidates = list(recordings.glob("full_body_*.mcap")) if not candidates: sys.exit( - f"[replay] error: no .mcap files in {recordings}. " - "Run record_full_body.py first or pass a path." + f"[replay] error: no full_body_*.mcap in {recordings}. " + "Run record_full_body first or pass a path." ) return max(candidates, key=lambda p: p.stat().st_mtime) @@ -137,9 +138,8 @@ def main(argv: list[str]) -> int: duration_s = mcap_duration_s(mcap_path) server = viser.ViserServer(host=args.host, port=args.port) - server.scene.set_up_direction("+y") - server.scene.add_grid(name="/grid", width=2.0, height=2.0, cell_size=0.1) - viz = FullBodyViz(server) + ground = setup_scene(server) + viz = FullBodyViz(server, ground) print( f"[replay] viser listening on {args.host}:{args.port} " diff --git a/examples/mcap_record_replay/python/replay_hand.py b/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/replay_hand.py similarity index 89% rename from examples/mcap_record_replay/python/replay_hand.py rename to examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/replay_hand.py index 0402badc68..801b566aa3 100644 --- a/examples/mcap_record_replay/python/replay_hand.py +++ b/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/replay_hand.py @@ -9,9 +9,10 @@ http://localhost:8080) in a browser to see the hands move. Usage: - python replay_hand.py [path/to/file.mcap] [--port 8080] [--loop] + python -m isaacteleop_examples.mcap_record_replay.replay_hand [path/to/file.mcap] [--port 8080] [--loop] -If no path is given, the newest file under ``../recordings/`` is used. +If no path is given, the newest ``hands_*.mcap`` under ``./recordings/`` is +used. ``--loop`` keeps replaying the file end-to-end until the process is killed. See: https://nvidia.github.io/IsaacTeleop/main/references/mcap_record_replay.html @@ -33,7 +34,7 @@ TeleopSessionConfig, ) -from common import HandViz, LEFT_COLOR, RIGHT_COLOR, build_hand_pipeline +from .common import HandViz, LEFT_COLOR, RIGHT_COLOR, build_hand_pipeline, setup_scene def mcap_duration_s(path: Path) -> float: @@ -61,12 +62,12 @@ def resolve_mcap(path_arg: str | None) -> Path: sys.exit(f"[replay] error: {path} does not exist") return path - recordings = Path(__file__).resolve().parent.parent / "recordings" - candidates = list(recordings.glob("*.mcap")) + recordings = Path.cwd() / "recordings" + candidates = list(recordings.glob("hands_*.mcap")) if not candidates: sys.exit( - f"[replay] error: no .mcap files in {recordings}. " - "Run record_hand.py first or pass a path." + f"[replay] error: no hands_*.mcap in {recordings}. " + "Run record_hand first or pass a path." ) return max(candidates, key=lambda p: p.stat().st_mtime) @@ -132,8 +133,7 @@ def main(argv: list[str]) -> int: duration_s = mcap_duration_s(mcap_path) server = viser.ViserServer(host=args.host, port=args.port) - server.scene.set_up_direction("+y") - server.scene.add_grid(name="/grid", width=2.0, height=2.0, cell_size=0.1) + setup_scene(server) viz_left = HandViz(server, "hand_left", LEFT_COLOR) viz_right = HandViz(server, "hand_right", RIGHT_COLOR) diff --git a/examples/mcap_record_replay/python/replay_se3_vive.py b/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/replay_se3_vive.py similarity index 93% rename from examples/mcap_record_replay/python/replay_se3_vive.py rename to examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/replay_se3_vive.py index 2b460addca..5768af23a3 100644 --- a/examples/mcap_record_replay/python/replay_se3_vive.py +++ b/examples/mcap_record_replay/python/isaacteleop_examples/mcap_record_replay/replay_se3_vive.py @@ -4,17 +4,17 @@ """ Replay VIVE Ultimate Tracker SE3 pose streams from an MCAP file — no runtime needed. -Opens a recording made by record_se3_vive.py, replays each per-tracker channel +Opens a recording made by record_se3_vive, replays each per-tracker channel through ReplaySession + core.Se3Tracker (ReplaySe3TrackerImpl), and shows every tracker as a coordinate frame in a viser 3D view (browser). Replay needs no OpenXR runtime and no hardware. -With no arguments it replays the most recent recording under ../recordings/, +With no arguments it replays the most recent recording under ./recordings/, auto-discovers its collections, and plays back at the recording's own capture rate. Open the printed viser URL to see the trackers in 3D. Usage: - uv run replay_se3_vive.py [recording.mcap] [--loop] [--rate N] [--no-viz] \ + python -m isaacteleop_examples.mcap_record_replay.replay_se3_vive [recording.mcap] [--loop] [--rate N] [--no-viz] \ [--collections a,b,c] """ @@ -39,16 +39,16 @@ def _summary(mcap_path: Path): def resolve_mcap(path_arg: str | None) -> Path: - """Use the given path, or the newest .mcap under ../recordings/.""" + """Use the given path, or the newest .mcap under ./recordings/.""" if path_arg: return Path(path_arg) - recordings = Path(__file__).resolve().parent.parent / "recordings" + recordings = Path.cwd() / "recordings" candidates = list(recordings.glob("se3_vive_*.mcap")) or list( recordings.glob("*.mcap") ) if not candidates: sys.exit( - f"[replay-se3] no .mcap files in {recordings}. Run record_se3_vive.py first." + f"[replay-se3] no .mcap files in {recordings}. Run record_se3_vive first." ) return max(candidates, key=lambda p: p.stat().st_mtime) @@ -112,8 +112,11 @@ class Se3Viz: def __init__(self, server, collections: list[str]): import viser # noqa: F401 (import here so --no-viz runs without viser) - server.scene.set_up_direction("+y") - server.scene.add_grid(name="/grid", width=2.0, height=2.0, cell_size=0.1) + # Also deferred: common imports viser at module level, so a top-level + # import here would defeat --no-viz. + from .common import setup_scene + + setup_scene(server) self._frames = {} self._labels = {} for cid in collections: @@ -178,7 +181,7 @@ def main(argv: list[str]) -> int: parser.add_argument( "mcap", nargs="?", - help="Recording to replay (default: newest under ../recordings/)", + help="Recording to replay (default: newest under ./recordings/)", ) parser.add_argument( "--collections", diff --git a/src/plugins/noitom_mocap/README.md b/src/plugins/noitom_mocap/README.md index 636ec5fdf2..f6d8cc750a 100644 --- a/src/plugins/noitom_mocap/README.md +++ b/src/plugins/noitom_mocap/README.md @@ -63,7 +63,7 @@ script: uv run python examples/noitom/record_noitom_full_body.py \ 10 examples/noitom/recordings/noitom_full_body.mcap -cd examples/mcap_record_replay/python -uv sync -uv run python replay_full_body.py ../../noitom/recordings/noitom_full_body.mcap +uv pip install -e ./examples/mcap_record_replay +python -m isaacteleop_examples.mcap_record_replay.replay_full_body \ + examples/noitom/recordings/noitom_full_body.mcap ``` diff --git a/src/plugins/vive_se3_tracker/README.md b/src/plugins/vive_se3_tracker/README.md index 4ff1b95f57..23ddfba7d3 100644 --- a/src/plugins/vive_se3_tracker/README.md +++ b/src/plugins/vive_se3_tracker/README.md @@ -102,16 +102,15 @@ with no arguments. Stale files from a crashed run are cleared on the next startu ## Recording / replay (Python examples) -Both run with no arguments once the plugin is up (from the example directory, so -`uv` picks up its `pyproject.toml`): +Both run with no arguments once the plugin is up: ```bash -cd examples/mcap_record_replay/python -# records 10 s of every advertised collection to ../recordings/.mcap -uv run record_se3_vive.py +uv pip install -e ./examples/mcap_record_replay +# records 10 s of every advertised collection to ./recordings/.mcap +python -m isaacteleop_examples.mcap_record_replay.record_se3_vive # replays the newest recording, auto-discovering collections + capture rate, # and serves a viser 3D view at http://localhost:8080 -uv run replay_se3_vive.py +python -m isaacteleop_examples.mcap_record_replay.replay_se3_vive ``` Under the hood this is just the standard MCAP tooling — a `core::Se3Tracker(cid)`