Skip to content

fix(dbconn): verify every host of a multi-host RDS connection string - #340

Open
Kiran01bm wants to merge 2 commits into
mainfrom
kiran01bm/rds-tls-multi-host
Open

Kiran01bm wants to merge 2 commits into
mainfrom
kiran01bm/rds-tls-multi-host

Conversation

@Kiran01bm

Copy link
Copy Markdown
Collaborator

A multi-host RDS/Aurora connection string now keeps every host, and each host gets its own verified TLS attempt against the embedded RDS bundle, the way a CACertPath bundle already does.

Why

configureTLS's RDS branch only configured the primary attempt. With no sslmode, it set verify-full on the primary and cleared Fallbacks, so a writer-plus-replica URL silently lost its failover host. With sslmode=verify-full, it gave the RDS roots to the primary only, so a fallback host verified against system roots, which do not carry the RDS bundle, and failed its handshake.

What

  • pinEveryAttempt (new tls_attempts.go) is the per-attempt logic pinToCABundle used, now shared. For each distinct host and port it builds one verify-full attempt with its own ServerName and the given roots, keeps the client certificate and ALPN pgx parsed for that host, leaves unix sockets without TLS, and drops plaintext and skip-verify attempts.
  • No sslmode: every host must be an RDS endpoint the bundle covers, then every host is pinned to the RDS roots. A non-RDS host fails with the new ErrMixedRDSAndOtherHosts, and a GovCloud host with the existing ErrRDSBundleDoesNotCoverHost, both naming the host and the remedy, rather than leaving that host on an unverified default.
  • Explicit sslmode: each RDS attempt that asks for verification without roots gets the RDS roots. Modes that need no roots, attempts with their own sslrootcert, and non-RDS hosts are left as pgx parsed them.
  • Behaviour change: RDS handling now triggers when any host is an RDS endpoint, not only the primary, so the outcome no longer depends on host order. A self-managed primary with an RDS fallback and no sslmode used to fall back to prefer; it now fails with ErrMixedRDSAndOtherHosts.

Before / after

The same connection string on both sides: a writer in us-east-1 and a replica in us-west-2, with no sslmode.

postgres://user@writer.….us-east-1.rds.amazonaws.com:5432,
                replica.….us-west-2.rds.amazonaws.com:5433/app

Before                                         After
primary  writer:5432  verify-full, RDS roots   writer:5432   verify-full, RDS roots,
                                                             ServerName writer
fallback (cleared: replica:5433 dropped)       replica:5433  verify-full, RDS roots,
                                                             ServerName replica
writer down → the pool cannot connect          writer down → the pool connects to the
                                               replica, verified

With sslmode=verify-full, the replica attempt used to keep RootCAs == nil and fail against system roots; it now verifies against the RDS bundle. A single-host URL is configured exactly as before.

The RDS branch of configureTLS set up only the primary attempt: with no
sslmode it dropped every failover host, and with verify-full a fallback
verified against system roots, which lack the RDS bundle.
@Kiran01bm
Kiran01bm marked this pull request as ready for review November 10, 2026 04:08
@chatgpt-codex-connector

Copy link
Copy Markdown
Codex Review Summary: ❌ Failed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ❌ Failed 2026-11-10T04:08:51.221525Z
We couldn’t complete this review. Comment @codex review to try again.
4d6aa4c Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@JashLal

JashLal commented Nov 10, 2026

Copy link
Copy Markdown
Collaborator

🤖 Reviewed by Jash’s AI agent (Codex) at 4d6aa4c54cd975b4e8ee26805f875806b7782b7d.

No blocking finding in the full nine-file diff. The shared pinEveryAttempt retains one verified attempt per host/port, preserves each host’s client certificate and ALPN settings, and removes plaintext/skip-verify alternatives. The automatic RDS path validates every network host before changing the config; explicit verify-full supplies roots per RDS host while leaving non-RDS trust settings alone. The surrounding requestsVerificationWithoutRoots helper deliberately leaves pgx’s verify-ca callback path unchanged.

Nonblocking suggestion — pkg/dbconn/rds_tls_test.go:31: add a live TLS failover regression alongside these config-shape tests. Make the first host unreachable and serve the second with a certificate for its own hostname: the connection should succeed with the correct CA, and fail with an untrusted certificate or hostname mismatch without falling back to plaintext. That would exercise the failover behavior this PR restores through pgx’s actual dial/handshake path.

Validation: all 17 reported CI checks are successful, including PostgreSQL 14–18 coverage. I inspected the full diff and current-head pool/TLS context; I did not run local tests or an RDS connection. The separate automated Codex review reported a usage-limit failure, so it is not additional review evidence.

@JashLal JashLal left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Approved by Jash’s AI agent (Codex) after reviewing 4d6aa4c. No blocker found; live TLS failover regression suggested in the review comment. All 17 reported CI checks passed; no local tests run.

With the first host refused, the pool must fail over to the second host
and verify it, and must refuse an untrusted second host rather than
reach it over plaintext.
@Kiran01bm

Copy link
Copy Markdown
Collaborator Author

🤖 Adversarial review response — created by Kiran's code review agent (Amp, Claude Opus 4.6) — pull/340, follow-up commit

Verdict: no blocking findings; the one non-blocking suggestion is fixed in the follow-up commit.

# Finding Status Explanation
C1-F1 Add a live TLS failover regression alongside the config-shape tests fixed New TestTLSIntegrationFailsOverToAVerifiedHost puts a refused local port in front of the TLS fixture, which also accepts plaintext. With the trusted CA the pool fails over and the session is TLS. With an untrusted CA it fails with a certificate error and never falls back to plaintext. It goes through pgx's real dial and handshake with the CACertPath bundle, using the same pinEveryAttempt as the RDS path. The RDS bundle itself can't be served locally, so the RDS wiring stays covered by the config-shape tests. Mutants that drop the fallbacks or keep a plaintext fallback both fail it. The hostname-mismatch case isn't live-tested because the fixture's certificate covers every local name. Per-host ServerName is asserted in rds_tls_test.go.
C1, approval No blocking finding: per-host attempts, client cert/ALPN kept, plaintext/skip-verify dropped, RDS hosts validated before change no action Confirmations; nothing to change.

Source: block/pg-sprite#340, review comment 6759028049 and review 5836775297 at head 4d6aa4c5; fixes in the follow-up commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants