Add @varlock/cloudflare-gateway: credential proxy adapters for Cloudflare Workers - #966
Add @varlock/cloudflare-gateway: credential proxy adapters for Cloudflare Workers#966theoephraim wants to merge 3 commits into
Conversation
…lare Workers New (private, not yet published) package that runs the varlock credential proxy pipeline (varlock/proxy-core) inside a Cloudflare Worker, in the two ways traffic can reach one: - asSandboxOutbound(): transparent outbound handler for the Cloudflare Sandbox SDK / Containers. CF's per-sandbox TLS interception hands the handler the original request, so nothing inside the sandbox needs a gateway URL or varlock binary. - asFetchHandler(): explicit HTTPS gateway for workloads outside Cloudflare (E2B, Fly, CI). Destination rides the x-varlock-target header; data-plane auth via Proxy-Authorization basic or x-varlock-token (constant-time compare, fail closed when unconfigured). Both share one pipeline: policy -> substitution guards -> placeholder substitution -> upstream fetch (workerd verifies upstream TLS before sending) -> response scrubbing (buffered redaction with the Invariant #6 leak fail-safe, plus chunk-by-chunk TransformStream scrubbing for SSE). Secrets never live in the worker bundle: values resolve lazily per request via an async getSecretValue seam (defaults to worker secret env bindings), and a request carrying a placeholder whose value is missing fails closed. Egress defaults to strict (a hosted gateway should not be an open forwarder). Approval rules throw at startup since there is no approval provider yet. Audit defaults to structured console logs.
The schema stays the source of truth for egress mode. In sandbox-outbound mode ALL sandbox egress (package installs, git) flows through the handler, so a strict default would break everything unruled out of the box; and without substitution a forwarded request carries no secrets. Infra-level lockdown in sandbox mode is the Sandbox SDK's allowedHosts; strict remains an explicit opt-in.
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
There was a problem hiding this comment.
Important
The core policy integration is sound, but the Workers transport currently loses valid HTTP behavior and can fail on supported request and destination shapes. The inline findings should be addressed before merging.
Reviewed changes in the initial 322a838 implementation of the Cloudflare credential gateway:
- Worker adapters: Added transparent Sandbox outbound and authenticated explicit HTTPS gateway handlers over the shared proxy pipeline.
- Credential handling: Added per-request secret resolution, strict egress defaults, substitution guards, audit reporting, and reflected-secret scrubbing.
- Package surface: Added the private integration package, exports, build configuration, and workspace metadata.
- Tests: Added 18 unit tests covering policy, substitution, authentication, destination routing, and buffered and streamed redaction.
azure/gpt-5.6-sol ｜ 𝕏
| function headersToRecord(headers: Headers): HeadersRecord { | ||
| const out: HeadersRecord = {}; | ||
| headers.forEach((value, key) => { | ||
| out[key.toLowerCase()] = value; |
There was a problem hiding this comment.
Repeated Set-Cookie fields are overwritten here, so any redacted response preserves only its final cookie. This breaks origins that issue separate session and CSRF cookies, and can be avoided by retaining Workers' getAll('Set-Cookie') result as an array.
Technical details
# Preserve repeated response cookies
## Affected sites
- `packages/integrations/cloudflare-gateway/src/gateway.ts:37-41` stores each iterated header under one scalar key.
- `packages/integrations/cloudflare-gateway/src/gateway.ts:137-142` routes responses requiring redaction through the lossy record.
## Required outcome
- Preserve every upstream `Set-Cookie` field independently through response redaction and reconstruction.
- Add a test with at least two upstream cookies on a request that enables redaction.
## Suggested approach
- Read Workers' special `Headers.getAll('Set-Cookie')` API and store that value as the record's array entry before reconstructing headers.
Cloudflare documents that `Set-Cookie` cannot be folded and requires `getAll`: https://developers.cloudflare.com/workers/runtime-apis/headers/#getall-method| return new Response(scrubbed, { status: statusCode, headers: outgoingHeaders }); | ||
| } | ||
| report(headerKeys, false); | ||
| return new Response(upstreamRes.body, { status: statusCode, headers: outgoingHeaders }); |
There was a problem hiding this comment.
Reconstructing an upgraded response without upstreamRes.webSocket discards the Workers WebSocket handle, so a successful outbound upgrade cannot be returned to the sandbox. Please either preserve the handle in the Workers response init or reject upgrade requests before contacting the upstream.
Technical details
# Preserve or explicitly reject WebSocket upgrades
## Affected sites
- `packages/integrations/cloudflare-gateway/src/gateway.ts:156-165` constructs a new response from only body, status, and headers.
- `packages/integrations/cloudflare-gateway/src/gateway.ts:182-183` does the same after buffered redaction.
## Required outcome
- A successful upstream 101 response must retain its Workers `webSocket` handle, or the gateway must explicitly declare and enforce that upgrades are unsupported before forwarding credentials.
## Suggested approach
- Special-case `upstreamRes.webSocket` and include it in the Cloudflare-specific response init without attempting body redaction.
Cloudflare documents the response handle returned by a successful client handshake: https://developers.cloudflare.com/workers/examples/websockets/#write-a-websocket-client| return textResponse(pre.status, pre.message); | ||
| } | ||
|
|
||
| const bodyBytes = new Uint8Array(await request.arrayBuffer()); |
There was a problem hiding this comment.
Every request is buffered in full and then duplicated as a decoded string before upstream fetch() begins. A valid large or streaming upload can therefore exceed Workers' shared 128 MB isolate limit or never be forwarded, even on routes with no credential substitution.
Technical details
# Bound request buffering and preserve streaming where possible
## Affected sites
- `packages/integrations/cloudflare-gateway/src/gateway.ts:220-228` materializes both the complete byte body and a UTF-16 string before policy phase two.
- `packages/integrations/cloudflare-gateway/src/gateway.ts:281-285` forwards only the buffered copy.
## Required outcome
- Requests that do not need body inspection or rewriting should retain streaming behavior.
- Requests that must be inspected need a deliberate size bound and a controlled client response rather than isolate exhaustion.
- Add coverage for the no-substitution streaming path and the configured or documented inspection limit.
Cloudflare recommends streaming to avoid buffering large bodies and documents the per-isolate memory limit: https://developers.cloudflare.com/workers/runtime-apis/streams/ and https://developers.cloudflare.com/workers/platform/limits/#memory| const scheme = target.isHttps ? 'https' : 'http'; | ||
| const defaultPort = target.isHttps ? 443 : 80; | ||
| const portSuffix = target.port && target.port !== defaultPort ? `:${target.port}` : ''; | ||
| const upstreamUrl = `${scheme}://${target.host}${portSuffix}${outcome.rewrittenTarget}`; |
There was a problem hiding this comment.
parseTargetHost() explicitly accepts bracketed IPv6 but strips the brackets, and this interpolation then creates invalid URLs such as https://2001:db8::1/ping. Re-bracket IP literals when constructing the authority so both default and custom-port IPv6 targets can be forwarded instead of returning 502.
commit: |
…onfig New subcommand emitting the schema's proxy policy as a JSON artifact (version, schema fingerprint, egress mode, rules, placeholder map) for hosted gateway adapters to bake into their bundle. Never values, and no resolver runs: placeholders now derive from the schema alone via the new EnvGraph.getProxyPlaceholderMap(), so the command works in low-trust CI (including Cloudflare Workers Builds) where op/keychain resolvers are unavailable, and never triggers a biometric prompt. getProxyManagedItems() now builds on the same map so a key's placeholder is identical however consumed (local runtime vs compiled config), even in the uniqueness-suffix edge case. The gateway config type accepts the artifact directly (optional version/schemaFingerprint fields), so 'import config from ./varlock-gateway.config.json' drops straight into createVarlockGateway().
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
varlock-website | dd793c8 | Commit Preview URL Branch Preview URL |
Jul 31 2026, 06:04 AM |
|
The changes in this PR will be included in the next version bump.
|
📦 Bundle size
dist/ only; native binaries are versioned separately and not counted here. |
There was a problem hiding this comment.
Important
varlock proxy config can execute supported dynamic policy resolvers and silently emit incomplete rules when one fails, contradicting its resolver-free contract.
Reviewed changes since the prior Pullfrog review at 322a838:
- Changed egress defaults: Switched the gateway default from strict to permissive while preserving explicit strict mode.
- Added static config compilation: Added
varlock proxy configto emit rules, placeholders, egress mode, a version, and a schema fingerprint without item values. - Unified placeholder generation: Added
EnvGraph.getProxyPlaceholderMap()and reused it for local proxy managed items so both paths derive identical placeholders. - Expanded package contracts and docs: Made compiler artifacts structurally consumable by the Cloudflare gateway and documented the new CLI surface.
azure/gpt-5.6-sol ｜ 𝕏
| checkForSchemaErrors(envGraph); | ||
| checkForNoEnvFiles(envGraph); | ||
|
|
||
| const { staticConfig, genericPlaceholderKeys } = await buildProxyStaticConfig(envGraph); |
There was a problem hiding this comment.
buildProxyStaticConfig() is reached only after checkForSchemaErrors(), but getProxyPlaceholderMap() calls getProxyRules(), which resolves supported dynamic @proxy(...) arguments. For attached decorators, a resolver failure is recorded after that check and returns an empty domain, so this command can both run external resolvers in low-trust CI and silently emit an artifact with the affected rule missing.
Technical details
# Keep static config compilation resolver-free and fail closed
## Affected sites
- `packages/varlock/src/cli/commands/proxy.command.ts:2370-2373` checks existing graph errors before policy resolution starts.
- `packages/varlock/src/env-graph/lib/env-graph.ts:1323-1342` resolves root and attached proxy decorators, then skips entries whose resolved domain is empty.
- `packages/varlock/src/env-graph/lib/decorators.ts:350-353` explicitly permits dynamic proxy expressions.
- `packages/varlock/src/env-graph/lib/decorators.ts:203-225` records resolver failures and returns without throwing.
## Required outcome
- Preserve the documented guarantee that this command does not invoke external or secret-bearing resolvers, or narrow the command contract and reject unsupported dynamic policy expressions explicitly.
- Fail config generation before writing JSON when policy compilation discovers a resolution error; never turn that error into an omitted rule.
- Cover an attached `@proxy` option backed by a failing or side-effecting resolver, not only an unresolved item value.1 participant
Note
Stacked on #964 (proxy-core extraction). Re-target to
mainonce that merges.What
New package
@varlock/cloudflare-gateway(markedprivatefor now; flips public when the docs guide ships) that runs the varlock credential proxy pipeline inside a Cloudflare Worker, via the two ways traffic can reach one:asSandboxOutbound()- transparent outbound handler for the Cloudflare Sandbox SDK / Containers. CF terminates the sandbox's TLS per instance and invokes the handler with the original request, so nothing inside the sandbox needs a gateway URL, a CA, or a varlock binary. Attach to the Sandbox class (MySandbox.outbound = gw.asSandboxOutbound()) alongsideallowedHostsfor infra-enforced strict egress.asFetchHandler()- explicit HTTPS gateway for workloads outside Cloudflare (E2B, Fly, CI, or the futurevarlock proxy run --gatewayguest shim). True destination rides thex-varlock-targetheader (hostorhost:port, https only); data-plane auth viaProxy-Authorization: Basic varlock:<token>orx-varlock-token, constant-time compared, fail closed when no token is configured.Both modes share one request path built on
varlock/proxy-core: policy -> substitution guards (placement + occurrence caps) -> placeholder substitution -> upstreamfetch()(workerd validates upstream TLS before sending, which is what the local runtime'sverifyUpstreamIdentityexists to guarantee) -> response scrubbing (buffered redaction with the Invariant #6 leak fail-safe, plus chunk-by-chunkTransformStreamscrubbing for SSE, sharingStreamingScrubber's hold-back logic with the node runtime).Design decisions:
getSecretValue(itemKey, env)seam (default: worker secret bindings named after the item key). Async from day one so Secrets Store / DO-backed stores slot in without an interface change. A request carrying a placeholder whose value is unavailable fails closed with a pointed error.allowedHosts;strictremains an explicit opt-in.approvalthrow at gateway creation - no approval provider exists in this tier, and silently denying every matching request would be a confusing failure mode.console.logby default (lands in Workers Logs), overridable viaonAudit; optionalonResponsesurfaces scrubbed-key info.Schema -> config compiler (
varlock proxy config)Second commit adds the varlock-side compiler (in varlock core, not the wrangler wrapper): a new
varlock proxy configsubcommand that compiles the schema's@proxypolicy into the static JSON artifact the gateway bakes into its bundle:{ "version": 1, "schemaFingerprint": "c81a72…", "egressMode": "strict", "rules": [{ "domain": ["api.stripe.com"], "itemKeys": ["STRIPE_KEY"] }], "placeholders": { "STRIPE_KEY": "sk_test_fake123" } }--output <file>); warnings go to stderr so stdout stays pipeable.EnvGraph.getProxyPlaceholderMap(), so the command works in low-trust CI (including Cloudflare Workers Builds) where op/keychain resolvers are unavailable, and never triggers a biometric prompt.getProxyManagedItems()now builds on the same map, so a key's placeholder is identical however it is consumed (local runtime vs compiled config).version/schemaFingerprintfields), soimport config from './varlock-gateway.config.json'drops straight intocreateVarlockGateway(config).ProxyStaticConfigtype lives invarlock/proxy-coreso both sides share the shape.Tests
Gateway: 18 unit tests run the handlers as plain functions with a stubbed
fetch: substitution, strict/permissive egress, location + occurrence guards, cleartext refusal, missing-secret fail-closed, buffered + cross-chunk streamed scrubbing, token auth paths, target parsing, internal-header stripping.Compiler: env-graph tests prove placeholders + egress derive from an unresolved graph and match the resolved path exactly; a command-level test covers the emitted artifact shape; verified end-to-end against a scratch schema whose only value is an unresolvable
exec().Not yet covered (next slices): live workerd verification (miniflare / vitest-pool-workers), the deploy-time secrets sync in
varlock-wrangler(values to worker secrets + fingerprint drift check), the guest shim, and the docs guide.