fix(cloud-functions): accept per-model routing expressions - #1422
Draft
along-2017 wants to merge 1 commit into
Draft
fix(cloud-functions): accept per-model routing expressions#1422along-2017 wants to merge 1 commit into
along-2017 wants to merge 1 commit into
Conversation
Replace the routingMethod allow-list with format-only validation of the routing expression grammar (a profile of RFC 8941 Item with Parameters): an algorithm token plus optional key=value tuning parameters. Well-formed expressions persist even when semantically wrong; the router remains the semantic authority and rejects unknown methods, unknown or incompatible parameters, and invalid values at request time. Normalize the algorithm token at write time (lowercase, underscore to hyphen, alias map) so newly persisted values converge on the router's canonical spellings; deprecated multiregion aliases normalize to their replacements. Do not release ahead of router-side expression support: persisted expressions reach the router verbatim, and routers without the parser reject them per request. Relates to #536 Signed-off-by: along <along@nvidia.com>
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueComment |
Contributor
🛡️ CodeQL Analysis🚨 Found 11 issue(s) Severity Breakdown:
📋 Top Issues🔗 View full details in Security tab 🕐 Last updated: 2026-08-31 20:53:51 UTC | Commit: d22207d |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
Function owners can persist per-model routing expressions such as
pulsar;seed=stable-a;n=2inllmConfig.routingMethod. The API now validatesformat only and normalizes the algorithm token to the router's canonical
spelling; the router stays the semantic authority and rejects invalid
expressions at request time.
Additional Details
The routing method allow-list is replaced by grammar validation (a profile of
RFC 8941 Item with Parameters): algorithm token, optional
;key=valueparameters, values as integer, decimal, token, or quoted string, no commas, no
duplicate keys, 1024-byte cap. Well-formed but semantically wrong expressions
persist by design, so the validator and the router cannot drift on semantics.
Write-time normalization (lowercase, underscore to hyphen, alias map) converges
newly persisted values on canonical algorithm names; the deprecated
multiregion aliases normalize to their replacements.
Do not merge until the router-side expression support has rolled out:
persisted expressions reach the router verbatim, and routers without the
parser reject those models' requests. Sequencing is tracked on the parent
issue.
For the Reviewer
The grammar and its profile rules are the review core:
LlmConfigValidator.validateAndNormalizeRoutingMethodand its parameterizedtest table. The two call sites now persist the returned normalized value.
For QA
bazel test //src/control-plane-services/cloud-functions/nvcf-core:tests:LlmConfigValidatorTest passes fully (all parameterized cases). The target
overall fails locally without Docker because unrelated Testcontainers
integration tests need it; CI provides Docker.
Issues
Relates to #536
Checklist