refactor(translation): use Dynamo Responses request schema#57
refactor(translation): use Dynamo Responses request schema#57nachiketb-nvidia wants to merge 1 commit into
Conversation
nachiketb-nvidia
force-pushed
the
nachiketb/use-dynamo-protocols-translation
branch
from
e242703 to
90c1e74
Compare
WalkthroughChangesTyped Provider Decoding
Estimated code review effort: 4 (Complex) | ~45 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
crates/switchyard-translation/src/codecs/openai_chat/buffered.rs (1)
460-466: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDocument the deprecated token-limit precedence.
Add a concise comment explaining why
max_completion_tokenswins and why deprecatedmax_tokensremains supported.As per coding guidelines, “Add concise comments/doc comments for public items and non-obvious helpers.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/switchyard-translation/src/codecs/openai_chat/buffered.rs` around lines 460 - 466, Add a concise comment above openai_chat_max_output_tokens explaining that max_completion_tokens takes precedence while deprecated max_tokens remains supported for backward compatibility. Keep the existing fallback behavior unchanged.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/switchyard-translation/src/codecs/anthropic/buffered.rs`:
- Around line 75-82: Read temperature and top_p directly from the raw body
instead of preferring decoded schema values, preserving precise JSON numbers as
f64. Apply this change in
crates/switchyard-translation/src/codecs/anthropic/buffered.rs lines 75-82,
crates/switchyard-translation/src/codecs/openai_chat/buffered.rs lines 69-76,
and crates/switchyard-translation/src/codecs/responses/buffered.rs lines 95-102;
update the corresponding request translation fields at each site and remove
schema-based fallbacks.
In `@crates/switchyard-translation/src/provider_types.rs`:
- Around line 10-25: Add concise /// doc comments to each public provider-type
re-export in provider_types.rs, including the OpenAI chat aliases, Anthropic
types, and OpenAI Responses aliases. Describe the exposed wire type and its
Switchyard-facing name without changing the re-exported symbols or behavior.
- Around line 32-45: Update the provider decoding flow around
maybe_decode_object() and field() so nested provider JSON retains the original
raw Value alongside the typed view, or configure the provider structs to
preserve unknown fields. Ensure field lookup does not replace raw messages,
content, tools, or input data with normalized typed serialization, and add a
regression test covering extra keys at those nested levels.
---
Nitpick comments:
In `@crates/switchyard-translation/src/codecs/openai_chat/buffered.rs`:
- Around line 460-466: Add a concise comment above openai_chat_max_output_tokens
explaining that max_completion_tokens takes precedence while deprecated
max_tokens remains supported for backward compatibility. Keep the existing
fallback behavior unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 2cd1c27e-e83a-4589-9ae2-67e7c8ba7e63
⛔ Files ignored due to path filters (1)
Cargo.lockis excluded by!**/*.lock,!Cargo.lock
📒 Files selected for processing (7)
crates/switchyard-translation/Cargo.tomlcrates/switchyard-translation/src/codecs/anthropic/buffered.rscrates/switchyard-translation/src/codecs/openai_chat/buffered.rscrates/switchyard-translation/src/codecs/responses/buffered.rscrates/switchyard-translation/src/lib.rscrates/switchyard-translation/src/provider_types.rscrates/switchyard-translation/tests/request_translation.rs
| temperature: schema | ||
| .as_ref() | ||
| .and_then(|request| request.temperature.map(f64::from)) | ||
| .or_else(|| body.get("temperature").and_then(Value::as_f64)), | ||
| top_p: schema | ||
| .as_ref() | ||
| .and_then(|request| request.top_p.map(f64::from)) | ||
| .or_else(|| body.get("top_p").and_then(Value::as_f64)), |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
rg -n -C3 'temperature: Option<f32>|top_p: Option<f32>|struct SamplingParams' \
crates/switchyard-translation Cargo.lockRepository: NVIDIA-NeMo/Switchyard
Length of output: 722
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n## outlines\n'
ast-grep outline crates/switchyard-translation/src/codecs/anthropic/buffered.rs --view expanded || true
ast-grep outline crates/switchyard-translation/src/codecs/openai_chat/buffered.rs --view expanded || true
ast-grep outline crates/switchyard-translation/src/codecs/responses/buffered.rs --view expanded || true
ast-grep outline crates/switchyard-translation/src/ir.rs --view expanded || true
printf '\n## relevant slices\n'
sed -n '1,220p' crates/switchyard-translation/src/codecs/anthropic/buffered.rs
printf '\n--- openai_chat ---\n'
sed -n '1,220p' crates/switchyard-translation/src/codecs/openai_chat/buffered.rs
printf '\n--- responses ---\n'
sed -n '1,240p' crates/switchyard-translation/src/codecs/responses/buffered.rs
printf '\n--- ir ---\n'
sed -n '180,215p' crates/switchyard-translation/src/ir.rsRepository: NVIDIA-NeMo/Switchyard
Length of output: 38140
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n## provider_types declarations\n'
rg -n -C3 'fn f64_field|fn field|struct OpenAiChatRequest|struct OpenAiResponsesRequest|struct AnthropicCreateMessageRequest|temperature|top_p' crates/switchyard-translation/src/provider_types.rs crates/switchyard-translation/src/provider_types -g '!**/target/**' || true
printf '\n## provider_types file slice\n'
sed -n '1,260p' crates/switchyard-translation/src/provider_types.rsRepository: NVIDIA-NeMo/Switchyard
Length of output: 3605
🌐 Web query:
dynamo-protocols CreateChatCompletionRequest temperature top_p f32 OpenAiResponsesRequest OpenAI chat completions schema
💡 Result:
The dynamo-protocols crate (formerly dynamo-async-openai) provides request and response structures for OpenAI-compatible inference servers [1][2]. Within this schema, the CreateChatCompletionRequest structure includes the temperature and top_p fields, which are defined as Option types [3]. In the context of this API schema: - temperature: Defines the sampling temperature (a float between 0 and 2) used to control output randomness [3]. - top_p: Defines the nucleus sampling parameter (a float used to determine the cumulative probability mass for token consideration) [3]. The use of f32 for these parameters aligns with standard practices for representing single-precision floating-point numbers in Rust-based LLM protocol definitions to maintain compatibility with the underlying OpenAI API requirements while optimizing for memory and performance [3][4]. The library is designed as a protocol-schema crate that re-exports types from upstream sources (specifically async-openai) while introducing select extensions for specific inference engine needs [1][5].
Citations:
- 1: https://docs.rs/crate/dynamo-protocols/latest
- 2: refactor(2/3): rename dynamo-async-openai to dynamo-protocols ai-dynamo/dynamo#7565
- 3: https://docs.rs/dynamo-async-openai/latest/dynamo_async_openai/types/struct.CreateChatCompletionRequest.html
- 4: https://github.com/ai-dynamo/dynamo/blob/main/lib/llm/src/protocols/common.rs
- 5: RFC: Publish dynamo-protocols and dynamo-parsers as standalone crates ai-dynamo/dynamo#7563
🌐 Web query:
AnthropicCreateMessageRequest temperature top_p Option<f32> dynamo-protocols
💡 Result:
AnthropicCreateMessageRequest is a Rust struct defined within the dynamo-protocols crate [1][2], which provides protocol type definitions for OpenAI-compatible and Anthropic-compatible inference APIs [2][3]. In the context of the dynamo-protocols library, AnthropicCreateMessageRequest represents the structure of requests sent to the Anthropic Messages API [1]. While the official Anthropic API supports temperature and top_p as floating-point parameters [4], the dynamo-protocols implementation specifically uses Option for these fields to accommodate their optional nature in the request body [1]. The dynamo-protocols crate (formerly dynamo-async-openai [5]) is designed to manage these types independently of upstream crates, allowing for custom inference-serving extensions [3][6]. As of the current development state in 2026, the crate has undergone significant refactoring to serve as a unified source for multi-protocol type definitions [1][5]. Developers working with this crate should ensure they are referencing the current types in dynamo_protocols::types::anthropic [1].
Citations:
- 1: RFC: Publish dynamo-protocols and dynamo-parsers as standalone crates ai-dynamo/dynamo#7563
- 2: https://docs.rs/dynamo-protocols/latest/dynamo_protocols/
- 3: https://docs.rs/crate/dynamo-protocols/latest
- 4: https://github.com/anthropics/anthropic-sdk-python/blob/main/src/anthropic/types/beta/message_create_params.py
- 5: refactor(2/3): rename dynamo-async-openai to dynamo-protocols ai-dynamo/dynamo#7565
- 6: https://crates.io/crates/dynamo-protocols
🌐 Web query:
OpenAiResponsesRequest temperature top_p Option<f32> dynamo-protocols
💡 Result:
In technical contexts involving the OpenAiResponsesRequest structure, such as those found in the dynamo-protocols Rust crate [1][2] or similar OpenAI-compatible API implementations [3][4][5], the temperature and top_p fields are defined as optional parameters [3][4][6]. In Rust, these are typically represented using Option<f32> or Option<f64> to denote their optionality [3][4][6], allowing them to be omitted from the request body if not specified by the user. Key technical details include: 1. Dynamo-protocols: This Rust crate provides protocol type definitions for OpenAI-compatible inference APIs, including support for the Responses API [1][2]. It is built on async-openai with selective overrides for specific inference behaviors [2][7]. 2. Optionality: In implementations like new-api (Go) or various TypeScript provider definitions, these fields have been transitioned to optional (pointer types or ? optional modifiers) to ensure compatibility with requests that do not explicitly set sampling parameters [3][4][6]. 3. Usage: When constructing an OpenAiResponsesRequest, these fields allow the developer to pass None (or leave the field undefined in JSON), leaving the inference engine to use its default sampling settings [3][4][6]. If you are encountering this in a Rust codebase using dynamo-protocols or an agent-sdk-provider, the use of Option<f32> is the standard idiomatic way to handle these nullable/optional JSON fields [1][8].
Citations:
- 1: https://docs.rs/dynamo-protocols/latest/dynamo_protocols/
- 2: https://docs.rs/crate/dynamo-protocols/latest
- 3: https://github.com/ProviderProtocol/ai/blob/bdb03ec946d88e219254141a0cd336a156d8a72a/src/providers/openai/types.ts
- 4: https://github.com/QuantumNous/new-api/blob/f7dae5cb/dto/openai_request.go
- 5: Add support for OpenAI Responses API dotnet/dev-proxy#1492
- 6: feat: /v1/chat/completion -> /v1/response QuantumNous/new-api#2629
- 7: https://crates.io/crates/dynamo-protocols
- 8: https://docs.rs/agent-sdk-provider/latest/agent_sdk_provider/
🌐 Web query:
docs.rs OpenAiResponsesRequest temperature top_p f32 dynamo-protocols create response struct
💡 Result:
The crate dynamo-protocols provides protocol type definitions for OpenAI-compatible inference APIs [1][2]. It is built on top of the async-openai crate (version 0.34) and re-exports many standard OpenAI types [1][3]. Regarding the request struct you mentioned, there is no single struct named OpenAiResponsesRequest in the dynamo-protocols crate [4]. The crate re-exports the OpenAI Responses API types from async-openai [1][3], where the relevant struct is named CreateResponse [5]. In async-openai's CreateResponse struct, the temperature and top_p fields are defined as Option [5]: pub struct CreateResponse { //... pub temperature: Option, pub top_p: Option, //... } If you are looking to create a request struct, you should use the types provided in dynamo-protocols::types [2][6]. For chat completions, for example, the crate uses CreateChatCompletionRequest [6][4], which mirrors the structure of the upstream async-openai type while incorporating local extensions like reasoning_content [1][3]. To use these types, you would typically import them from the dynamo_protocols::types module [6].
Citations:
- 1: https://docs.rs/crate/dynamo-protocols/latest
- 2: https://docs.rs/dynamo-protocols/latest/dynamo_protocols/
- 3: https://crates.io/crates/dynamo-protocols
- 4: https://docs.rs/dynamo-protocols/latest/dynamo_protocols/all.html
- 5: https://docs.rs/async-openai/latest/async_openai/types/responses/struct.CreateResponse.html
- 6: RFC: Publish dynamo-protocols and dynamo-parsers as standalone crates ai-dynamo/dynamo#7563
Avoid routing temperature and top_p through the schema object
provider_types::f64_field prefers the decoded schema value, and these request types model both fields as f32, so precise JSON numbers can be rounded before they reach the IR's f64. Read these keys from the raw body here instead, in:
crates/switchyard-translation/src/codecs/anthropic/buffered.rscrates/switchyard-translation/src/codecs/openai_chat/buffered.rscrates/switchyard-translation/src/codecs/responses/buffered.rs
📍 Affects 3 files
crates/switchyard-translation/src/codecs/anthropic/buffered.rs#L75-L82(this comment)crates/switchyard-translation/src/codecs/openai_chat/buffered.rs#L69-L76crates/switchyard-translation/src/codecs/responses/buffered.rs#L95-L102
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/switchyard-translation/src/codecs/anthropic/buffered.rs` around lines
75 - 82, Read temperature and top_p directly from the raw body instead of
preferring decoded schema values, preserving precise JSON numbers as f64. Apply
this change in crates/switchyard-translation/src/codecs/anthropic/buffered.rs
lines 75-82, crates/switchyard-translation/src/codecs/openai_chat/buffered.rs
lines 69-76, and crates/switchyard-translation/src/codecs/responses/buffered.rs
lines 95-102; update the corresponding request translation fields at each site
and remove schema-based fallbacks.
| pub use dynamo_protocols::types::{ | ||
| ChatCompletionRequestMessage as OpenAiChatRequestMessage, | ||
| ChatCompletionResponseMessage as OpenAiChatResponseMessage, | ||
| ChatCompletionTool as OpenAiChatTool, ChatCompletionToolChoiceOption as OpenAiChatToolChoice, | ||
| CreateChatCompletionRequest as OpenAiChatRequest, | ||
| CreateChatCompletionResponse as OpenAiChatResponse, | ||
| CreateChatCompletionStreamResponse as OpenAiChatStreamResponse, | ||
| }; | ||
|
|
||
| pub use dynamo_protocols::types::anthropic::{ | ||
| AnthropicCreateMessageRequest, AnthropicMessageResponse, AnthropicStreamEvent, | ||
| }; | ||
|
|
||
| pub use dynamo_protocols::types::responses::{ | ||
| CreateResponse as OpenAiResponsesRequest, Response as OpenAiResponsesResponse, | ||
| }; |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Document the public provider-type re-exports.
Add concise /// documentation for the public aliases so their exposed wire types and Switchyard naming are clear in generated API docs.
As per coding guidelines, public items should have concise comments or doc comments.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/switchyard-translation/src/provider_types.rs` around lines 10 - 25,
Add concise /// doc comments to each public provider-type re-export in
provider_types.rs, including the OpenAI chat aliases, Anthropic types, and
OpenAI Responses aliases. Describe the exposed wire type and its
Switchyard-facing name without changing the re-exported symbols or behavior.
Source: Coding guidelines
| pub(crate) fn maybe_decode<T>(body: &Value) -> Option<T> | ||
| where | ||
| T: DeserializeOwned, | ||
| { | ||
| serde_json::from_value(body.clone()).ok() | ||
| } | ||
|
|
||
| /// Best-effort serialization used to feed existing IR mappers from provider | ||
| /// structs without making schema serialization itself a new failure mode. | ||
| pub(crate) fn maybe_encode<T>(value: &T) -> Option<Value> | ||
| where | ||
| T: Serialize, | ||
| { | ||
| serde_json::to_value(value).ok() |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
set -euo pipefail
# Map the target file and nearby symbols
ast-grep outline crates/switchyard-translation/src/provider_types.rs --view expanded || true
echo '--- maybe_decode/maybe_encode usages ---'
rg -n "maybe_decode|maybe_encode|raw fallback|fallback" crates/switchyard-translation/src -S || true
echo '--- provider_types.rs with line numbers ---'
cat -n crates/switchyard-translation/src/provider_types.rs | sed -n '1,220p'Repository: NVIDIA-NeMo/Switchyard
Length of output: 4810
🏁 Script executed:
set -euo pipefail
ast-grep outline crates/switchyard-translation/src/provider_types.rs --view expanded || true
echo '--- maybe_decode/maybe_encode usages ---'
rg -n "maybe_decode|maybe_encode|raw fallback|fallback" crates/switchyard-translation/src -S || true
echo '--- provider_types.rs with line numbers ---'
cat -n crates/switchyard-translation/src/provider_types.rs | sed -n '1,220p'Repository: NVIDIA-NeMo/Switchyard
Length of output: 4810
🏁 Script executed:
set -euo pipefail
for file in \
crates/switchyard-translation/src/codecs/openai_chat/buffered.rs \
crates/switchyard-translation/src/codecs/anthropic/buffered.rs \
crates/switchyard-translation/src/codecs/responses/buffered.rs
do
echo "===== $file ====="
# Show nearby regions around each maybe_decode_object call
rg -n "maybe_decode_object|schema|raw" "$file" -n -S | head -n 80
echo "--- snippets ---"
# Print limited windows around the call sites
while IFS=: read -r line _; do
start=$((line-25))
end=$((line+120))
if [ "$start" -lt 1 ]; then start=1; fi
sed -n "${start},${end}p" "$file" | cat -n | sed -n "1,180p"
echo "----"
done < <(rg -n "maybe_decode_object" "$file" -n -S)
doneRepository: NVIDIA-NeMo/Switchyard
Length of output: 2919
🏁 Script executed:
set -euo pipefail
for file in \
crates/switchyard-translation/src/codecs/openai_chat/buffered.rs \
crates/switchyard-translation/src/codecs/anthropic/buffered.rs \
crates/switchyard-translation/src/codecs/responses/buffered.rs
do
echo "===== $file ====="
rg -n "maybe_decode_object" "$file" -S
echo "--- around matches ---"
while IFS=: read -r line _; do
start=$((line-35))
end=$((line+140))
if [ "$start" -lt 1 ]; then start=1; fi
echo ">>> lines ${start}-${end}"
sed -n "${start},${end}p" "$file" | cat -n
echo "----"
done < <(rg -n "maybe_decode_object" "$file" -S)
doneRepository: NVIDIA-NeMo/Switchyard
Length of output: 490
🏁 Script executed:
set -euo pipefail
for file in \
crates/switchyard-translation/src/codecs/openai_chat/buffered.rs \
crates/switchyard-translation/src/codecs/anthropic/buffered.rs \
crates/switchyard-translation/src/codecs/responses/buffered.rs
do
echo "===== $file ====="
rg -n "maybe_decode_object" "$file" -S
while IFS=: read -r line _; do
start=$((line-30))
end=$((line+110))
[ "$start" -lt 1 ] && start=1
echo "--- lines ${start}-${end} ---"
sed -n "${start},${end}p" "$file" | cat -n
done < <(rg -n "maybe_decode_object" "$file" -S)
doneRepository: NVIDIA-NeMo/Switchyard
Length of output: 467
🏁 Script executed:
set -euo pipefail
# Inspect the exact mapping code around the schema/raw usage
sed -n '1,220p' crates/switchyard-translation/src/codecs/openai_chat/buffered.rs | cat -n | sed -n '1,220p'
echo '---'
sed -n '220,340p' crates/switchyard-translation/src/codecs/openai_chat/buffered.rs | cat -n | sed -n '1,180p'
echo '---'
sed -n '1,320p' crates/switchyard-translation/src/codecs/anthropic/buffered.rs | cat -n | sed -n '1,320p'
echo '---'
sed -n '1,260p' crates/switchyard-translation/src/codecs/responses/buffered.rs | cat -n | sed -n '1,260p'
echo '--- search for serde unknown-field retention in local source ---'
rg -n "flatten|deny_unknown_fields|extra|unknown fields|preserve" crates -S || trueRepository: NVIDIA-NeMo/Switchyard
Length of output: 50378
🏁 Script executed:
set -euo pipefail
echo '--- openai_chat request path ---'
sed -n '30,180p' crates/switchyard-translation/src/codecs/openai_chat/buffered.rs | cat -n
echo '--- openai_chat response path ---'
sed -n '240,340p' crates/switchyard-translation/src/codecs/openai_chat/buffered.rs | cat -n
echo '--- anthropic path ---'
sed -n '30,320p' crates/switchyard-translation/src/codecs/anthropic/buffered.rs | cat -n
echo '--- responses path ---'
sed -n '30,260p' crates/switchyard-translation/src/codecs/responses/buffered.rs | cat -n
echo '--- serde attrs in workspace source ---'
rg -n "deny_unknown_fields|flatten" crates -S || trueRepository: NVIDIA-NeMo/Switchyard
Length of output: 39688
Don't normalize nested provider JSON here. maybe_decode_object() round-trips through serde and field() prefers that map over raw, so unknown keys inside messages/content/tools/input disappear once the typed subtree is used. Keep the original nested Value alongside the typed view, or make the provider structs retain unknown fields, and add a regression test with extra nested keys.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/switchyard-translation/src/provider_types.rs` around lines 32 - 45,
Update the provider decoding flow around maybe_decode_object() and field() so
nested provider JSON retains the original raw Value alongside the typed view, or
configure the provider structs to preserve unknown fields. Ensure field lookup
does not replace raw messages, content, tools, or input data with normalized
typed serialization, and add a regression test covering extra keys at those
nested levels.
90c1e74 to
34f019c
Compare
nachiketb-nvidia
changed the title
1 participant
What
Why
The earlier approach made Dynamo look like a generic provider-schema preprocessor. That added ceremony without removing provider-specific logic.
The useful boundary is narrower: Dynamo already carries the Responses API request schema and its real-world compatibility fixes, including object-form tool-choice modes and relaxed Responses input parsing. Switchyard should reuse that schema at the request boundary, then keep its current IR conversion logic rather than duplicate Dynamo or async-openai internals.
How
What to review
Tested