Skip to content

refactor(libsy): use conversation IR request and response types#58

Open
nachiketb-nvidia wants to merge 10 commits into
mainfrom
nachiketb/libsy-conversation-ir
Open

refactor(libsy): use conversation IR request and response types#58
nachiketb-nvidia wants to merge 10 commits into
mainfrom
nachiketb/libsy-conversation-ir

Conversation

@nachiketb-nvidia

Copy link
Copy Markdown
Contributor

Why

PR #17 introduced libsy with bespoke LlmRequest / LlmResponse structs that only carried a text prompt and completion. That makes libsy look like it owns a new LLM wire abstraction, when Switchyard already has a shared provider-neutral conversation IR in switchyard-translation.

This change makes libsy align with that direction now: LlmRequest and LlmResponse are semantic aliases over ConversationRequest and ConversationResponse, while provider-specific details stay in clients/hosts rather than in router algorithms.

What

  • Replaces libsy-owned LlmRequest and LlmResponse structs with aliases to ConversationRequest and ConversationResponse.
  • Adds small text helpers for text-only router logic and examples: text_request, text_response, request_text, and response_text.
  • Updates llm_class, ensemble, and rand to inspect conversation IR via the helpers instead of old prompt / completion fields.
  • Boxes Step::ReturnToAgent so the larger conversation response does not inflate the stream enum.
  • Updates libsy examples and README to teach the conversation IR contract.
  • Updates the libsy proxy demo to the current Algorithm / Request / Response API and compiles it against the shared router builder.

How

The algorithms still operate on plain text where their current logic requires text, but that text is now extracted from the shared IR. Generated subcalls are built as text-only ConversationRequests with the inbound model preserved in model; routed target selection remains on Decision::selected_model().

The proxy demo still lets libsy make routing decisions and Switchyard own HTTP/profile serving. The demo now creates a text-only libsy request from inbound prompt text, runs the classifier algorithm directly, and returns a chat-completion response for Switchyard to translate back to the inbound format.

What to Review

  • Whether aliasing LlmRequest / LlmResponse to the translation IR is the right libsy contract for this PR.
  • Whether the helper functions are the right minimum bridge for text-based router algorithms without reintroducing a second IR.
  • Whether boxing Step::ReturnToAgent is acceptable for the stream API after the response type becomes larger.
  • Whether the demo proxy update is appropriately scoped to compiling against the current libsy/server APIs.

Validation

  • cargo test -p libsy
  • cargo check -p libsy --examples
  • cargo check -p libsy-proxy
  • cargo test -p libsy-proxy
  • pre-commit hook: cargo fmt, cargo clippy

messiaen and others added 9 commits July 10, 2026 13:05
Signed-off-by: Greg Clark <grclark@nvidia.com>

feat: poc rand router

Signed-off-by: Greg Clark <grclark@nvidia.com>

feat: poc llm class router

Signed-off-by: Greg Clark <grclark@nvidia.com>

docs: add @route decorator design spec for libsy optimizer port

docs: clarify abstract optimize and classifier preamble in @route spec

feat(agentapi): add chat data types for optimizer port

feat(agentapi): add optimizer role interfaces and decision types

feat(agentapi): add weighted random router

feat(agentapi): add multi-round LLM-classifier router

feat(agentapi): add @route decorator driving the optimizer loop

feat(agentapi): export public @route API from subpackage

test(agentapi): cover default-adapter prompt and system-message paths

feat: python decorator example

Signed-off-by: Greg Clark <grclark@nvidia.com>

docs: libsy design doc

Signed-off-by: Greg Clark <grclark@nvidia.com>

docs: fix design doc

Signed-off-by: Greg Clark <grclark@nvidia.com>

docs: fix design doc

Signed-off-by: Greg Clark <grclark@nvidia.com>

feat(libsy): add RoutedClient LLM-client wrapper over the optimizer traits

docs(libsy): reflect RoutedClient wrapper in design doc

style(libsy): apply rustfmt to client.rs

docs(libsy): add README and LLM-classifier research-agent examples

feat: multi llm orch and algo

Signed-off-by: Greg Clark <grclark@nvidia.com>

refactor(libsy): migrate routers and examples to the OrchAlgo/MultiLlmOrchestrator model

feat: orchestrate_direct

Signed-off-by: Greg Clark <grclark@nvidia.com>

fix: concurrency bug

Signed-off-by: Greg Clark <grclark@nvidia.com>

fix: promise.set_reponse need to take Result

Signed-off-by: Greg Clark <grclark@nvidia.com>

chore: move interfaces to lib

Signed-off-by: Greg Clark <grclark@nvidia.com>

feat: proxy demo

Signed-off-by: Greg Clark <grclark@nvidia.com>

feat(libsy): add ensemble router with per-session state

docs: libsy documentation update

Signed-off-by: Greg Clark <grclark@nvidia.com>

chore: remove old files

Signed-off-by: Greg Clark <grclark@nvidia.com>

docs: add srd

Signed-off-by: Greg Clark <grclark@nvidia.com>

docs: srd rm image

Signed-off-by: Greg Clark <grclark@nvidia.com>

docs: remove srd doc from branch

chore: review feedback

Signed-off-by: Greg Clark <grclark@nvidia.com>

docs: fix llm_class eg

Signed-off-by: Greg Clark <grclark@nvidia.com>

chore: review cleanup

Signed-off-by: Greg Clark <grclark@nvidia.com>

fix: unbounded_channel in orch for promise_tx

Signed-off-by: Greg Clark <grclark@nvidia.com>

chore: add get_target_set to algo

Signed-off-by: Greg Clark <grclark@nvidia.com>

chore: remove OrchAlgo builder

Signed-off-by: Greg Clark <grclark@nvidia.com>

docs: improve README

Signed-off-by: Greg Clark <grclark@nvidia.com>

docs: cleanup readme

Signed-off-by: Greg Clark <grclark@nvidia.com>
Signed-off-by: Greg Clark <grclark@nvidia.com>
Signed-off-by: Greg Clark <grclark@nvidia.com>
Signed-off-by: Greg Clark <grclark@nvidia.com>
Signed-off-by: Greg Clark <grclark@nvidia.com>
Signed-off-by: Greg Clark <grclark@nvidia.com>
Signed-off-by: Greg Clark <grclark@nvidia.com>
Signed-off-by: Greg Clark <grclark@nvidia.com>
@nachiketb-nvidia nachiketb-nvidia requested a review from a team as a code owner July 13, 2026 21:15
@messiaen messiaen force-pushed the grclark/feat/libsy-routers-poc branch from fcdc0ad to eab29f6 Compare July 14, 2026 14:20
Base automatically changed from grclark/feat/libsy-routers-poc to main July 14, 2026 14:28
@messiaen messiaen self-requested a review July 14, 2026 14:31
@messiaen

Copy link
Copy Markdown
Contributor

#17 is merged can you rebase? @nachiketb-nvidia

Comment thread crates/libsy/src/types.rs

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's move these types into a protocol or similar crate.

@messiaen messiaen requested a review from grahamking July 14, 2026 14:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants