Skip to content

Log what answered when a Gateway reply is not encrypted - #17

Open
Marketen wants to merge 1 commit into
mainfrom
fix/log-plaintext-gateway-replies
Open

Marketen wants to merge 1 commit into
mainfrom
fix/log-plaintext-gateway-replies

Conversation

@Marketen

Copy link
Copy Markdown
Contributor

When a reply to an encrypted request has no Ehbp-Response-Nonce, the proxy only logged:

failed to decrypt response: missing Ehbp-Response-Nonce header

The Gateway sets that header on every reply once its EHBP handler has decrypted the request, so a reply without it came from something in front of that step: an edge, a load balancer or an early rejection. The real HTTP status was dropped before anyone could read it.

This happened on a Dappnode on 2026-09-14: six small requests (likely Hermes title generation) failed in two bursts at 11:49 and 11:51 UTC, while large requests sent at the same moment succeeded, with no Gateway redeploy or key change. Small requests reach the encrypted handler normally (verified against the live TEE Gateway), so the cause is outside it, but the logs can't say what.

Change

GuardEHBPResponses now fails an unencrypted reply to an encrypted request itself, describing it:

Gateway replied without EHBP encryption: HTTP 429 (Content-Type=text/plain, Retry-After=5): rate limited at the edge
  • Includes Content-Type, Server, Retry-After, Cf-Ray when present and a 512-byte single-line body excerpt.
  • The reply was plaintext on the wire, so no prompt or completion can appear in it. The integration test asserts the prompt canary never reaches the log.
  • The 422 problem+json key-mismatch reply still passes through, so the EHBP client re-verifies as before (existing key-rotation test passes).
  • Requests that were never encrypted are untouched.

The caller-facing behaviour is unchanged: still a 502.

Tests

  • Guard unit tests: descriptive error with capped, single-line excerpt and closed body; key mismatch and unencrypted requests pass through. The main test fails with the change disabled.
  • Proxy integration test: a plaintext 429 edge gives the caller 502 and a log line with the status, Retry-After and body.
  • go vet ./... and go test ./... pass.

🤖 Generated with Claude Code

A reply to an encrypted request without Ehbp-Response-Nonce came from
something in front of the Gateway's EHBP handler: an edge, a load balancer
or an early rejection. The EHBP client only reported the missing header, so
the real status was lost and a brief burst of failures could not be
diagnosed.

The response guard now fails such replies itself with the HTTP status,
Content-Type, Server, Retry-After, Cf-Ray and a 512-byte single-line body
excerpt. The reply was plaintext on the wire, so nothing confidential is
logged. The 422 key-mismatch reply still reaches the EHBP client so it can
re-verify.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant