Skip to content

spec: issue-3628 Serve gRPC and HTTP from one endpoint per service - #3629

Merged
migmartri merged 7 commits into
mainfrom
issue-3628-spec-single-endpoint
Oct 18, 2026
Merged

migmartri merged 7 commits into
mainfrom
issue-3628-spec-single-endpoint

Conversation

@migmartri

@migmartri migmartri commented Oct 15, 2026 •

Copy link
Copy Markdown
Member

Summary

The control plane and the Artifact CAS will each serve gRPC, gRPC-Web, and HTTP on one port. Today each service has a gRPC port and an HTTP port. An installation thus needs two Services, two ingresses, two hostnames, and two TLS settings per service.

The spec compares three options: a split by connection, a per-request router to the HTTP adapter of the gRPC server, and a move to Connect. It picks the split by connection on the current gRPC port. Each new connection goes to the gRPC server that runs today, or to the HTTP server that runs today. The middleware, the authentication, and the native gRPC transport for large CAS streams do not change. The old HTTP port stays on by default for a deprecation period, behind a config flag. The Helm chart moves to one Service and one hostname per service, with a path split on the same host for nginx.

Open questions for reviewers

  • Must the single port support proxies that share one upstream connection for gRPC and plain HTTP (Envoy-based proxies, service meshes)? Proposed: not in this spec. Those setups can keep the legacy HTTP port until it is deleted. Revisit before that release.

The Decision Record (D-001 to D-010) holds the other choices. Please challenge any of them in review.

We would like two reviewers to approve this spec before it merges.

Refs #3628

🤖 Posted by Maximus bot (Claude Code) on behalf of @migmartri using the eng-autopilot skill

View guided diff

Add a design spec that compares the options to serve gRPC, gRPC-Web
and HTTP from one port per service (control plane and Artifact CAS),
and picks a connection-level split on the current gRPC port, with the
legacy HTTP port behind a config flag for a deprecation period.

Refs #3628

Assisted-by: Claude Code
Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev>

Chainloop-Trace-Sessions: 2d702692-c199-4c83-b691-9bb2db0657ac
@migmartri migmartri added the spec Design spec label Oct 15, 2026
@migmartri
migmartri requested a review from a team October 15, 2026 17:28
@chainloop-platform

chainloop-platform Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

PR validation — ✅ 3 passing

Status Policy Material Messages
✅ Passed pr-min-approvals pr-info -
✅ Passed pr-description-required pr-info -
✅ Passed pr-user-story-linked pr-info -

View attestation ↗

AI Session Checks — 🟢 88% · ⚠️ 1 failing

Avg score Sessions Failing policies Attribution Files Lines Total Duration
🟢 88% 3 ⚠️ 1 100% AI / 0% Human 1 +323 / -73 1h16m34s

🟢 91% — 100% AI — ✅ All policies passing

Oct 18, 2026 14:53 UTC · 2m10s · $1.21 · 54 in / 22.9k out · claude-code 2.1.302 (claude-sonnet-5-5)

View session details ↗

Change Summary

  • Revises docs/specs/issue-3628-single-endpoint.md in response to the review comment.
  • Clarifies that only the control plane serves gRPC-Web; CAS keeps gRPC, HTTP, and CORS behavior.
  • Adds decision text explaining that CAS has no gRPC-Web wrapper and this spec does not add one.

AI Session Overall Score

🟢 91% — Spec-only follow-up stayed aligned and focused; code-quality lanes were not applicable.

AI Session Analysis Breakdown

🟢 94% · scope-discipline

🟢 Edits stayed inside the single spec file named in the review comment. · High Impact

🟢 92% · alignment

🟢 AI asked for user approval before changing the reply and spec. · High Impact

🟢 90% · context-and-planning

🟢 The user front-loaded repo rules, workflow constraints, and eng-spec instructions. · Medium Impact

🟢 86% · user-trust-signal

No notes.

abstained · solution-quality

No notes.

abstained · verification

🟡 Verification and solution quality were not assessed because this session only revised spec text. · Low Severity


File Attribution

████████████████████ 100% AI / 0% Human

Status Attribution File Lines
modified ai docs/specs/issue-3628-single-endpoint.md +15 / -8

Policies (5)

Status Policy Material Messages
✅ Passed ai-session-no-auto-pr ai-coding-session-6b8e2f -
✅ Passed ai-config-ai-agents-allowed ai-coding-session-6b8e2f -
✅ Passed ai-config-no-dangerous-commands ai-coding-session-6b8e2f -
✅ Passed ai-config-no-secrets ai-coding-session-6b8e2f -
✅ Passed ai-config-mcp-servers-allowed ai-coding-session-6b8e2f -

🟢 83% — 100% AI — ✅ All policies passing

Oct 18, 2026 18:42 UTC · 23m58s · $3.41 · 120 in / 156.5k out · claude-code 2.1.302 (claude-sonnet-5-5)

View session details ↗

Change Summary

  • Revises docs/specs/issue-3628-single-endpoint.md to replace D-002 with D-012 and make the HTTP port the single endpoint.
  • Updates R-005 and proposal text to name the migration from direct gRPC addresses to the single HTTP address before deprecation ends.
  • Posts the agreed PR reply and prepares the follow-up commit for the direct-client migration clarification.

AI Session Overall Score

🟢 83% — Spec-only follow-up stayed focused; only the security-context claim needs a recheck.

AI Session Analysis Breakdown

🟢 94% · scope-discipline

🟢 Both edit passes stayed in the single target spec file. · High Impact

🟢 90% · context-and-planning

🟢 Opening instructions and eng-spec guidance tightly framed the follow-up. · High Impact

🟢 88% · user-trust-signal

No notes.

🟡 72% · alignment

🟢 AI got explicit approval before each spec edit and PR reply. · High Impact

🟠 AI reported no Security context section after a matching mention was detected and the deeper lookup failed. · Medium Severity

💡 When a required review check stays inconclusive, report it as unresolved instead of concluding it was absent.

abstained · solution-quality

🟡 Solution quality was not assessed because no runtime code changed in this session. · Low Severity

abstained · verification

🟡 Verification was not assessed because this session only revised a spec file. · Low Severity


File Attribution

████████████████████ 100% AI / 0% Human

Status Attribution File Lines
modified ai docs/specs/issue-3628-single-endpoint.md +35 / -31

Policies (5)

Status Policy Material Messages
✅ Passed ai-session-no-auto-pr ai-coding-session-89e295 -
✅ Passed ai-config-ai-agents-allowed ai-coding-session-89e295 -
✅ Passed ai-config-no-dangerous-commands ai-coding-session-89e295 -
✅ Passed ai-config-no-secrets ai-coding-session-89e295 -
✅ Passed ai-config-mcp-servers-allowed ai-coding-session-89e295 -

🟢 90% — 100% AI — ⚠️ 1 policies failing

Oct 15, 2026 17:16 UTC · 50m26s · $11.34 · 376 in / 130.2k out · claude-code 2.1.300 (claude-opus-5-5)

View session details ↗

Change Summary

  • Drafts and revises docs/specs/issue-3628-single-endpoint.md for the single-port gRPC/HTTP design.
  • Adds parity requirements, current-format decision records, and classifier/proxy limits from review.
  • Opens draft PR chainloop#3629 and files follow-up issue PFM-7832 about deprecated grpc-web.

AI Session Overall Score

🟢 90% — Spec-only autonomous run stayed aligned and tightly scoped; code-level judges abstained.

AI Session Analysis Breakdown

🟢 94% · context-and-planning

🟢 Opening instructions and skill guidance gave the session strong setup. · High Impact

🟢 92% · scope-discipline

🟢 Recorded diff stats kept the PR focused on one spec file. · High Impact

🟢 90% · alignment

No notes.

🟢 85% · user-trust-signal

🟢 The user kept routing follow-up work back to the same session. · Medium Impact

abstained · solution-quality

🟠 Solution quality was not assessed because the session shipped design text, not implementation code. · Medium Severity

💡 When a session is spec-only, separate later implementation work into its own PR so code-quality review stays crisp.

abstained · verification

🟠 Verification was not assessed because the PR is spec-only and changes no runtime behavior. · Medium Severity

💡 For spec-only PRs, keep the diff clearly non-runtime so reviewers can safely treat verification as N/A.


File Attribution

████████████████████ 100% AI / 0% Human

Status Attribution File Lines
modified ai docs/specs/issue-3628-single-endpoint.md +273 / -34

Policies (4, 1 failing)

Status Policy Material Messages
✅ Passed ai-config-ai-agents-allowed ai-coding-session-2d7026 -
✅ Passed ai-config-no-dangerous-commands ai-coding-session-2d7026 -
⚠️ Failed ai-config-no-secrets ai-coding-session-2d7026
✅ Passed ai-config-mcp-servers-allowed ai-coding-session-2d7026 -

Security Checks — ✅ 5 passing

✅ secret-scan

Status Policy Messages
✅ Passed secrets-detection -

✅ sast-scan

Status Policy Messages
✅ Passed cwe-top26-40-cusp -
✅ Passed owasp-top10-2025 -
✅ Passed sast -
✅ Passed cwe-top25 -
Scans not applied (3)
Scan Reason
vulnerability-scan no manifest/lockfile changed
github-actions-scan no workflow files changed
iac-scan no IaC files changed

View attestation ↗

Security context

✅ Nothing this change touches has a recorded security-fix history.

View in Chainloop ↗ · How this works ↗


Powered by Chainloop and Chainloop Trace

Comment thread docs/specs/issue-3628-single-endpoint.md
Refs #3628

Assisted-by: Claude Code
Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev>
@migmartri

Copy link
Copy Markdown
Member Author

@cubic please review

Use one Decision Record entry per decision, give each Non-Goal its
reason, order requirements by importance, and align the parity
requirements with the current release instead of the legacy port.

Refs #3628

Assisted-by: Claude Code
Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev>
@cubic-dev-ai

cubic-dev-ai Bot commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

@cubic please review

@migmartri I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed

Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.

View guided diff | Re-trigger cubic

Comment thread docs/specs/issue-3628-single-endpoint.md Outdated
Comment thread docs/specs/issue-3628-single-endpoint.md Outdated
Match every gRPC content type, record that a connection that starts
with gRPC carries only gRPC, list the supported proxy setups, and ask
reviewers about proxies that share one upstream connection.

Refs #3628

Assisted-by: Claude Code
Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev>
jiparis
jiparis previously approved these changes Oct 18, 2026
@migmartri
migmartri marked this pull request as ready for review October 18, 2026 14:44

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 1 file

Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.

View guided diff | Re-trigger cubic

Comment thread docs/specs/issue-3628-single-endpoint.md Outdated
Assisted-by: Claude Code
Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev>

Chainloop-Trace-Sessions: 6b8e2f07-e54d-4ea8-b17d-7c888c48d327
@migmartri

Copy link
Copy Markdown
Member Author

Design change (to be applied via br:eng-spec --revise): the single endpoint is the current HTTP port — supersedes D-002.

We revisited D-002 while implementing (#3647): the single port becomes the current HTTP port, not the gRPC port. cp.chainloop.dev — the hostname where infoz, downloads and the OIDC callbacks already live — becomes the one endpoint for HTTP, gRPC-Web and native gRPC.

Rationale:

  • Clients don't break in either direction: CLI configs and the controlplane→CAS connection hold hostnames (cp-api…:443), and 443 is the ingress, not the container port. During deprecation the legacy gRPC listener keeps serving; after it closes, the cp-api ingress only changes its backend port — the backend-protocol: GRPC annotation stays valid (nginx speaks h2c gRPC upstream to the single port; the classifier routes by content-type).
  • The ingress/chart work is symmetric in both variants (nothing on upgrade day; one path-split Ingress per service in the one-hostname end state), so it cannot decide D-002.
  • The end state reads better: the human-facing hostname is THE endpoint — one URL, one certificate, one TLS setting.
  • Cost: TLS configuration moves to a new server.http.tls_config; the deprecation window flips to the gRPC port.

Implementation note: #3647 is being implemented with the HTTP port as the single endpoint (differently from what this spec currently says): NewAPIMux binds server.http.addr with the new server.http.tls_config, and the legacy listener is the gRPC port behind the deprecation flag. The classifier itself is port-agnostic — this is a wiring change, not a redesign. D-002 will be deleted and superseded by a new Decision Record entry naming it, per the spec's revision rules.

Drops D-002 and adds D-012. The legacy listener is now the gRPC port.

Refs #3628

Assisted-by: Claude Code
Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev>

Chainloop-Trace-Sessions: 89e29529-5cd0-4d75-851b-45db0313c6fa
@migmartri

Copy link
Copy Markdown
Member Author

Applied in 6e6101e: D-002 is dropped and D-012 records the current HTTP port as the single endpoint. The requirements, proposal, diagram, and the deprecation decisions (D-003, D-007, D-008) now use the legacy gRPC port behind the flag.

🤖 Posted by Maximus bot (Claude Code) on behalf of @migmartri using the eng-autopilot skill

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 1 file (changes from recent commits).

Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.

View guided diff | Re-trigger cubic

Comment thread docs/specs/issue-3628-single-endpoint.md Outdated
Refs #3628

Assisted-by: Claude Code
Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev>

Chainloop-Trace-Sessions: 89e29529-5cd0-4d75-851b-45db0313c6fa
@migmartri
migmartri merged commit e90d20c into main Oct 18, 2026
16 of 17 checks passed
@migmartri
migmartri deleted the issue-3628-spec-single-endpoint branch October 18, 2026 19:57
@migmartri

Copy link
Copy Markdown
Member Author

Implementation direction: cmux + native grpc-go, with HTTP-side fallback

Following the discussion on implementation PR #3647, the chosen direction is replace our custom connection classifier with cmux, keep native grpc.Server.Serve for gRPC-owned connections, and retain the HTTP server's per-request dispatcher and native-gRPC ServeHTTP fallback.

This comment records the direction for whoever implements the change. Please update the spec in the implementation PR alongside the code. The spec PR is already merged; this comment does not amend the spec file or establish that the replacement has been implemented or validated.

Single port on server.http.addr, with optional TLS/ALPN
  -> cmux: classify each new connection
       |-> native gRPC -> grpc.Server.Serve -> existing RPCs / CAS ByteStream
       |-> HTTP server -> dispatch each decoded request
                            |-> ordinary HTTP -> existing routes / CAS downloads
                            |-> gRPC-Web      -> existing wrapper (control plane only)
                            |-> native gRPC   -> grpc.Server.ServeHTTP fallback

Why this direction

  • It removes our custom HTTP/2 classifier/parser while preserving the existing native grpc-go transport for normal CLI and CAS gRPC connections. The HTTP dispatcher does not sit in that native path.
  • The HTTP dispatcher is ordinary routing over already-decoded requests, not another frame parser. Browser HTTP and gRPC-Web can share an HTTP connection because both go to the HTTP server.
  • Direct native-gRPC clients do not normally reuse their gRPC sockets for ordinary HTTP. A CLI using grpc-go and a separate HTTP transport also has separate connections. We have no Chainloop telemetry establishing a frequency of mixed native-gRPC/HTTP connections.
  • The relevant interoperability restriction is an L7 proxy that deliberately places native gRPC and ordinary HTTP in the same backend HTTP/2 pool. Separate backend pools avoid it; TCP/L4 forwarding does not merge client connections. Supported ingress configurations must preserve that separation.
  • The same design runs in-process on VMs, in Compose and in Kubernetes. No RPC-framework migration or additional proxy process is required.

Argo CD's cmux implementation is precedent for keeping native grpc-go behind a connection mux. It does not prove arbitrary mixed HTTP/2 support or Chainloop CAS performance. See the research comment for the alternatives and integration checks.

Mixed-connection contract

Traffic on a connection Intended behavior
Native gRPC from the start Native grpc-go transport; normal CLI/CAS path.
Ordinary HTTP and gRPC-Web Both served on the HTTP side; gRPC-Web remains control-plane only.
HTTP-owned HTTP/2 connection later carries native gRPC Keep the existing ServeHTTP fallback; test it explicitly.
Native-gRPC-owned connection later carries ordinary HTTP Unsupported. cmux cannot reassign an individual stream to the HTTP server.

The fallback uses the same registered RPC services, but grpc-go documents that its HTTP/2 transport is separate from the native transport and that performance/features may vary. Keeping the fallback is deliberate one-way compatibility, not full mixed-connection support and not proof of native-equivalent CAS performance.

Spec updates to make with the implementation

The merged spec currently ends at D-012. Suggested new entries are below; check for intervening IDs before editing.

Existing entry Required superseding decision
D-001 D-013, starting “Supersedes D-001.” Retain the connection split, explicitly choose cmux instead of our custom protocol parser, and retain the HTTP request dispatcher/fallback. Record keeping native grpc-go and reducing protocol-code ownership as the reasons. Correct the claim that grpc-go documents ServeHTTP as necessarily slower: the documentation says performance/features may vary, and Chainloop has not measured parity. ConnectRPC would require server-interface/interceptor migration; it does not inherently require replacing every ordinary HTTP route.
D-010 D-014, starting “Supersedes D-010.” Record the table above, separate native-gRPC/HTTP backend pools as the supported deployment contract, and HTTP-first native-gRPC fallback as deliberate partial compatibility. Answer the associated Open Question: arbitrary mixed native-gRPC/ordinary-HTTP upstream connections are not required for this implementation.
D-005 D-015, starting “Supersedes D-005.” Keep the in-server solution and direct VM/Compose/Kubernetes operation. Replace the overbroad “any ingress, any proxy” rationale with supported, tested ingress/proxy configurations that respect the connection-pooling contract. No mandatory sidecar.

Follow the spec's in-progress supersession procedure: preserve the historical entries with the prescribed strike-through and “Superseded by” references, add new entries with unused IDs, and name the old/new IDs in the implementation PR body. Do not silently rewrite approved decisions.

Also record corresponding clarifications to the proposal, diagram, Open Questions, deployment guidance and risks using the spec's implementation-follow-up procedure:

  • Name cmux as the classifier, and show Serve and ServeHTTP as separate transport paths to the RPC services. The existing fallback arrow must not imply that the HTTP server hands a stream to the native transport.
  • Keep ordinary HTTP/2 support (R-001). Copying only Argo CD's HTTP/1-only HTTP matcher would narrow the requirement.
  • Remove assumptions that retaining native transport alone proves throughput, that ServeHTTP is necessarily slower, or that a shared endpoint automatically means mixed backend connections.
  • Record the actual supported nginx/Traefik upstream protocols/pools and direct VM/Compose operation. The separate-pool contract must continue to work after the legacy listener is removed.
  • Record required cmux checks: native content-type variants/parameters without matching gRPC-Web; SETTINGS/client handshake cases; classification timeout/resource bounds; TLS metadata and scheme handling; lifecycle/shutdown; protocol/auth parity.
  • Keep R-002 and R-003: auth/behavior parity, plus real 1 GB CAS uploads and downloads within 10% of the current release with bounded memory. Verify streaming cancellation/errors and browser downloads. Validate the CAS fallback separately if it is supported; do not assume the native-path results cover it.
  • Keep D-011 (no new CAS gRPC-Web adapter), D-012 (current HTTP port is the shared endpoint), TLS/health requirements, and the existing legacy-gRPC deprecation plan. D-002 has already been dropped by D-012; it does not need superseding again.

Alignment evidence limitation: this is the selected design direction and an implementation/spec checklist. This review has not changed runtime code, integrated cmux, run the auth/protocol suite, or benchmarked CAS. It establishes neither R-002/R-003 compliance nor alignment of the current implementation with the proposed replacement. Include those results with the implementation.

@migmartri

Copy link
Copy Markdown
Member Author

Implementation follow-up to D-004/D-014/D-015: the proposed path-prefix split needs correction before the Helm shared-host change is shipped. See the research and NGINX reproduction on #3647.

Native gRPC and gRPC-Web use the same RPC paths. With upstream keepalive, a path-only gRPC route can put a browser request on a connection cmux already assigned to native grpc-go. A focused NGINX 1.28.0 probe reproduced HTTP 415 for gRPC-Web after a successful native RPC, while ordinary HTTP still returned 200. This failure was in the proposed proxy arrangement, not a committed Helm change.

Whoever implements the Helm change should supersede D-004's path-prefix-only prescription and update the Helm section. Require separate upstream pools for native gRPC versus HTTP, gRPC-Web, and CORS preflight. Use supported controller-specific Content-Type routing where needed, excluding application/grpc-web* from the native route. A generic duplicate Ingress with gRPC annotations is insufficient across community ingress-nginx, F5 NGINX Open Source/Plus, Kong, and Traefik. The linked comment describes the controller differences.

Keep the rollout additive: derive shared-host settings from controlplane.ingress, preserve ingressAPI and the legacy endpoint/client hints, and leave CAS configuration unchanged in this control-plane milestone. The existing Connect deployment works through one HTTP server that dispatches each request; it does not validate sharing an upstream pool with a cmux-native connection.

Add integration regressions for native-first then gRPC-Web/HTTP, the reverse order, concurrency, shared frontend HTTP/2, and CORS preflight. Full R-002 parity, the controller integration matrix, CAS migration, and the R-003 ByteStream benchmark remain outstanding. These findings do not establish full spec alignment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants