feat: add viser as new manipulation vis backend#2475
Open
TomCC7 wants to merge 25 commits into
Open
Conversation
# Conflicts: # dimos/manipulation/planning/monitor/world_monitor.py
1 task
TomCC7
force-pushed
the
cc/feat/viser-vis-rework
branch
from
283a6da to
9087b4d
Compare
TomCC7
changed the title
Contributor
Greptile SummaryThis PR adds Viser as a new manipulation visualization backend and replaces the old
Confidence Score: 5/5Safe to merge; the only findings are cosmetic improvements to the fallback path of the adapter and a dead config field. The new Viser backend is well-structured: server lifecycle, scene management, GUI panel, and threading model are cleanly separated. The initialization ordering is correct. Previously identified state-race issues in the GUI have been addressed. The only noteworthy items are a dead preview_request_timeout config field, a duplicate FK call in the fallback evaluation path, and a semantically misleading success=True for collision returns in the same fallback path, none of which affect the happy path. dimos/manipulation/visualization/viser/adapter.py - the evaluate_joint_target fallback path has two minor issues worth cleaning up. Important Files Changed
Sequence DiagramsequenceDiagram
participant MM as ManipulationModule
participant WM as WorldMonitor
participant VF as VisualizationFactory
participant VV as ViserManipulationVisualizer
participant GUI as ViserPanelGui
participant ADP as InProcessViserAdapter
MM->>WM: create_planning_specs()
MM->>VF: create_manipulation_visualization(config, world, world_monitor, self)
VF->>VV: ViserManipulationVisualizer(...)
VV->>ADP: InProcessViserAdapter(world_monitor, manipulation_module)
VV->>GUI: ViserPanelGui(server, adapter, config, scene)
GUI->>GUI: start() - refresh() - WAITING_FOR_ROBOT
VF-->>MM: visualization
MM->>WM: add_robot(config) x N
MM->>WM: finalize()
MM->>WM: set_visualization(visualization)
MM->>WM: sync_visualization_scene()
WM->>VV: initialize_scene(PlanningSceneInfo)
VV->>GUI: refresh() - READY
Note over GUI,ADP: User interaction
GUI->>ADP: evaluate_joint_target(joints, robot_name)
ADP->>WM: is_state_valid(robot_id, target)
ADP-->>GUI: result
GUI->>ADP: plan_to_joints(target, robot_name)
ADP->>MM: plan_to_joints(joints)
MM-->>GUI: bool
GUI->>ADP: preview_path(robot_name)
ADP->>MM: preview_path()
MM->>VV: animate_path(robot_id, path, duration)
VV->>VV: PreviewAnimator.animate() blocking
Reviews (5): Last reviewed commit: "fix: make viser theme test tolerate fall..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
github-actions
Bot
added
the
ready-to-merge
github-actions
Bot
removed
the
ready-to-merge
github-actions
Bot
added
the
ready-to-merge
github-actions
Bot
removed
the
ready-to-merge
github-actions
Bot
added
the
ready-to-merge
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #2412 and supersedes #2413
Adds Viser as a manipulation planning visualizer and rewires manipulation visualization config around a single validated
visualizationfield. The new visualizer is more feature-rich than the original drake-bundled meshcat vis and provides a moveit style UI for plan/preview/execute.Main changes:
visualization_backend/visualization_optionsconfig with a Pydantic discriminatedvisualizationconfig fornone,meshcat, andviser.WorldSpecPlanningSpecsVisualizationSpecWorldMonitor.viser[urdf]in themanipulationextra and move Viser usage docs into the manipulation docs.Notes
allow_plan_execute.set_planning_target/clear_planning_targetVisualizationSpecAPI was reverted; target controls remain internal to the Viser GUI/scene.How to Test
Contributor License Agreement