Skip to content

fix(cloud-functions): accept per-model routing expressions - #1422

Draft
along-2017 wants to merge 1 commit into
mainfrom
fix/llm-routing/expression-format-validation
Draft

fix(cloud-functions): accept per-model routing expressions#1422
along-2017 wants to merge 1 commit into
mainfrom
fix/llm-routing/expression-format-validation

Conversation

@along-2017

Copy link
Copy Markdown
Contributor

TL;DR

Function owners can persist per-model routing expressions such as
pulsar;seed=stable-a;n=2 in llmConfig.routingMethod. The API now validates
format only and normalizes the algorithm token to the router's canonical
spelling; the router stays the semantic authority and rejects invalid
expressions at request time.

Additional Details

The routing method allow-list is replaced by grammar validation (a profile of
RFC 8941 Item with Parameters): algorithm token, optional ;key=value
parameters, values as integer, decimal, token, or quoted string, no commas, no
duplicate keys, 1024-byte cap. Well-formed but semantically wrong expressions
persist by design, so the validator and the router cannot drift on semantics.
Write-time normalization (lowercase, underscore to hyphen, alias map) converges
newly persisted values on canonical algorithm names; the deprecated
multiregion aliases normalize to their replacements.

Do not merge until the router-side expression support has rolled out:
persisted expressions reach the router verbatim, and routers without the
parser reject those models' requests. Sequencing is tracked on the parent
issue.

For the Reviewer

The grammar and its profile rules are the review core:
LlmConfigValidator.validateAndNormalizeRoutingMethod and its parameterized
test table. The two call sites now persist the returned normalized value.

For QA

bazel test //src/control-plane-services/cloud-functions/nvcf-core:tests:
LlmConfigValidatorTest passes fully (all parameterized cases). The target
overall fails locally without Docker because unrelated Testcontainers
integration tests need it; CI provides Docker.

Issues

Relates to #536

Checklist

  • I am familiar with the Contributing Guidelines.
  • I have signed off my commits for Developer Certificate of Origin (DCO) compliance.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

Replace the routingMethod allow-list with format-only validation of the
routing expression grammar (a profile of RFC 8941 Item with Parameters):
an algorithm token plus optional key=value tuning parameters. Well-formed
expressions persist even when semantically wrong; the router remains the
semantic authority and rejects unknown methods, unknown or incompatible
parameters, and invalid values at request time.

Normalize the algorithm token at write time (lowercase, underscore to
hyphen, alias map) so newly persisted values converge on the router's
canonical spellings; deprecated multiregion aliases normalize to their
replacements.

Do not release ahead of router-side expression support: persisted
expressions reach the router verbatim, and routers without the parser
reject them per request.

Relates to #536

Signed-off-by: along <along@nvidia.com>
@coderabbitai

coderabbitai Bot commented Aug 31, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Comment @coderabbitai help to get the list of available commands.

@along-2017 along-2017 self-assigned this Aug 31, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🛡️ CodeQL Analysis

🚨 Found 11 issue(s)

Severity Breakdown:

  • 🔴 Errors: 0
  • 🟡 Warnings: 0
  • 🔵 Notes: 0
📋 Top Issues

🔗 View full details in Security tab

🕐 Last updated: 2026-08-31 20:53:51 UTC | Commit: d22207d

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant