fix(llm-routing): keep plaintext gRPC independent of QUIC trust - #1372
Conversation
Apply the merged transport bundle to Stargate gRPC only for HTTPS registration while preserving reverse-QUIC trust and explicit gRPC CA overrides. Closes #1367 Signed-off-by: Stephanie Baum <sbaum@nvidia.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (6)
Included review availability: Your plan provides up to 12 included reviews per hour; 8 remain after this review. 📝 WalkthroughWalkthroughNVCA now injects the gRPC CA path only for HTTPS Stargate registration. Pylon applies the same condition when reusing reverse QUIC trust. Tests cover HTTP, HTTPS, address precedence, and explicit CA overrides. ChangesLLM routing trust handling
Estimated code review effort: 3 (Moderate) | ~20 minutes Merge Risk: ⚪ Minimal · up to Plaintext gRPC remains independent of the reverse-QUIC trust bundle while HTTPS registration keeps CA validation, and split-address handling remains unaffected. No actionable merge-blocking risk remains beyond normal checks. Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Full details: Linked Issues checkExplanation The changes address issue
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Warning Some tools did not complete. Review the errors below. 🔧 golangci-lint (2.12.2)level=error msg="Running error: context loading failed: failed to load packages: failed to load packages: failed to load with go/packages: err: exit status 1: stderr: go: inconsistent vendoring in /src/compute-plane-services/nvca:\n\tgithub.com/NVIDIA/KAI-scheduler@v0.12.6: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt\n\tgithub.com/NVIDIA/k8s-dra-driver-gpu@v0.0.0-20251017125642-cfe35ffd3d2c: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt\n\tgithub.com/NVIDIA/nvcf/src/libraries/go/lib@v0.0.0-20260722095202-f5e2792f5630: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt\n\tgithub.com/aws/aws-sdk-go@v1.55.5: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt\n\tgithub.com/bombsimon/logrusr/v4@v4.1.0: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt\n\tgithub.com/evanphx/json-patch/v5@v5.9.11: is explicitly required in ... [truncated 21721 characters] ... i: is replaced in go.mod, but not marked as replaced in vendor/modules.txt\n\tk8s.io/apiextensions-apiserver: is replaced in go.mod, but not marked as replaced in vendor/modules.txt\n\tk8s.io/apimachinery: is replaced in go.mod, but not marked as replaced in vendor/modules.txt\n\tk8s.io/client-go: is replaced in go.mod, but not marked as replaced in vendor/modules.txt\n\tk8s.io/component-base: is replaced in go.mod, but not marked as replaced in vendor/modules.txt\n\tsigs.k8s.io/controller-runtime: is replaced in go.mod, but not marked as replaced in vendor/modules.txt\n\tgolang.org/x/crypto: is replaced in go.mod, but not marked as replaced in vendor/modules.txt\n\n\tTo ignore the vendor directory, use -mod=readonly or -mod=mod.\n\tTo sync the vendor directory, run:\n\t\tgo mod vendor\n" 🔧 Clippy (1.97.1)Clippy execution failed Comment |
🛡️ CodeQL Analysis🚨 Found 5 issue(s) Severity Breakdown:
📋 Top Issues🔗 View full details in Security tab 🕐 Last updated: 2026-08-30 18:59:33 UTC | Commit: 44950cc |
|
🎉 This PR is included in version stargate-v0.14.3 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
2 participants
TL;DR
Keep the reverse-QUIC trust bundle out of plaintext Stargate gRPC registration while preserving verified HTTPS registration.
Additional Details
Pylon reverse mode uses
STARGATE_TLS_CERT_PATHas its QUIC trust anchor. NVCA also injected the same bundle throughSTARGATE_GRPC_TLS_CA_CERT_PATH. Current Pylon releases then attempted to load that gRPC CA even when--stargate-addressused explicit HTTP, preventing worker registration.This PR fixes both ends of that contract:
The paired changes belong together because either change alone still leaves the current released counterpart applying the CA to plaintext registration.
Customer Release Notes
Self-managed LLM workers can register over explicit plaintext in-cluster gRPC while continuing to authenticate reverse QUIC independently.
Plan Summary
No Kubernetes resources or public API shapes change. Worker environment injection becomes scheme-aware.
Usage
No operator action is required. HTTPS endpoints continue to use the configured trust bundle. Explicit HTTP and scheme-less development endpoints remain plaintext.
For the Reviewer
Please review the address-precedence behavior in
llmWorkerUsesHTTPSRegistrationand the Pylon fallback ordering inload_grpc_tls_ca_cert.For QA
QA needed: publish and consume both component artifacts through #1362, then run the final self-managed BDD integration.
Tests run:
go test ./internal/transporttlswith the required NVCA linker flaggo test ./pkg/nvca -run TestCreatePodArtifactInstancesTransportTLSBundleInjectsOnlyLLMWorkerwith the required linker flaggo test ./internal/miniservicewith Kubernetes 1.34 envtest assets and the required linker flaggolangci-lint --new-from-rev=origin/mainfor the affected NVCA packagescargo test -p pyloncargo clippy -p pylon --all-targets -- -D warningscargo fmt --all -- --checkgit diff --checkNotes
The NVCA
make testwrapper requires an ignored.envfile that is absent from the public checkout, so the affected packages were run directly with the documented linker and envtest settings.Package-wide NVCA lint still reports three existing
goconstfindings in unrelated MiniService files. Linting only changes fromorigin/mainreports zero issues.Issues
Closes #1367
Related Pull Requests
Dependencies
No new or updated third-party dependencies. NOTICE is unchanged.
Checklist
Summary by CodeRabbit