Skip to content

fix(trino): refuse plain-HTTP identity on every engine (port 8080 hardening, PR A) - #842

Merged
viethqb merged 16 commits into
mainfrom
feat/port-8080-hardening
Nov 8, 2026
Merged

viethqb merged 16 commits into
mainfrom
feat/port-8080-hardening

Conversation

@viethqb

@viethqb viethqb commented Nov 7, 2026 •

Copy link
Copy Markdown
Member

Summary

This is port 8080 hardening, PR A: Trino identity over plain HTTP. It is the first of two prerequisites for SP2b Task 26 (OPA on by default).

The hole (measured). egret rendered http-server.authentication.allow-insecure-over-http=true on every Trino engine except jwt-mode with TLS. At true, Trino does not authenticate a plain-HTTP request at all, so anything that could reach a coordinator's port 8080 could run as any user.

  • This includes egret's pods, the engine namespace and an engine's ingress controller.
  • trino --user ann --session-user bob ran as bob with no password, on Ranger and OPA alike.
  • An engine with no trinoConfig.auth had no authenticator at all.
  • The old comment ("safe because impersonation is delegator-only") was wrong.

The spec is docs/en/specs/2026-11-07-port-8080-hardening-design.md, Part A, and the plan is docs/en/plans/2026-11-07-port-8080-hardening.md, Tasks A1–A6. Part B, a dedicated OPA listener, is a separate PR.

Changes

  • Every authenticated engine refuses authentication over plain HTTP. The render is now allow-insecure-over-http=false.
    • A plain-HTTP request without X-Forwarded-Proto: https gets 403.
    • A request with that header but no credential gets 401.
    • egret's monitor (Basic plus the header), delegated queries (HTTPS 8443) and /v1/info probes keep working.
  • Every provisioned Trino engine authenticates.
    • A spec without auth is stored and applied as {"password":{}}.
    • egret-monitor is injected for every engine except a true jwt-only coordinator, which is jwt mode with an auth.jwt block. The delegator is injected in impersonate mode.
    • Migration 0215 backfills the stored auth on existing provisioned Trino rows.
  • "Re-apply needed".
    • Migration 0215 adds engine_mgmt.engines.applied_security_rev.
    • Every successful Trino apply records TRINO_SECURITY_REV = 1.
    • The engine API gains securityReapplyNeeded, and the engine list shows a "Re-apply needed" badge.
    • jwt-only engines without password users are excluded, because they were already closed.
  • Every Trino apply now re-materializes the engine's workspace connections. A newly minted delegator secret is then bound on every apply path (rotate, mechanism switch, catalog attach or detach, restart), not only on PUT.
    • A failure leaves the apply successful but names the engine in the existing "not recorded" note.
    • So a mechanism switch reports that engine and a resend retries.
  • monitor_headers branches on is_jwt_only_coordinator(): Bearer only for a real jwt-only engine, Basic plus the header otherwise.
  • The opa-e2e harness reaches Trino as egret does.
    • It uses the delegator's Basic credential with X-Forwarded-Proto: https, or the CLI over 8443, impersonating each user.
    • The credential is decrypted from the e2e stack's own DB with the public dev master key, and masked in CI logs. It is e2e-only.
    • The new identity-check.sh asserts 403, 401 and 200 on both engines and fails the job on a regression.
  • Docs corrections. "As corrected" notes in the 2026-07-04 and 2026-07-15 specs, and in deploy/trino/etc/README.md and config.properties.

Test plan

  • A1 live measurement, on a standalone Trino 483 that mirrors egret's render (scripts/trino-insecure-http/check.sh), with and without an HTTPS listener. Results are in docs/en/plans/2026-11-07-port-8080-hardening-live-result.md, with the raw output alongside.
    • At false: plain HTTP gives 403 "Authentication over HTTP is not enabled"; forwarded HTTPS with no credential gives 401; monitor-pattern Basic gives 200; the delegator impersonating ann gives ann; /v1/info gives 200; the worker registers.
  • Unit and DB tests, each with a mutant killed by a failing assertion. Results: egret-provisioner 212, egret-engine-mgmt lib 233 and routes 215/215, egret-metastore lib, and frontend vitest control-plane 568. Covered:
    • the flag render;
    • auth normalisation across 6 spec shapes;
    • the security revision and badge population;
    • the legacy no-auth row re-applied through rotate-secrets, which ends with auth stored, the connection bound to the delegator, and the monitor sending Basic plus the header;
    • the monitor Bearer/Basic choice.
  • fmt and clippy -D warnings are clean.
  • e2e: the CI opa-e2e job on this PR is the run of record. Locally, Layer 2 passed (501 cases) and the identity check passed, but only on a pre-A2 image with the flag set to false by hand. Building a fresh backend image was not safe with other stacks running.
  • Reviews: a task review after every task, a final whole-branch review (opus), and one fix wave with a re-review. The final review found one Important issue, now fixed: a legacy no-auth row re-applied through a non-PUT path got a broken monitor and an unbound connection.

Known follow-ups (not in this PR)

  • Provisioned Trino workers have no access control, so the graceful-shutdown preStop (PUT /v1/info/state) gets 403 "Management only resource" at both flag values. This was measured, and graceful shutdown never happens. It is pre-existing.
  • StarRocks FE 9030 accepts root with no password when root_password is unset.
  • Ingress and kube-proxy behaviour was simulated with the header, not measured against a real ingress controller.
  • PR B moves the OPA surface to its own engines-only listener.

Documentation

egret-data/docs#69 covers several pages:

  • every Trino engine authenticates;
  • the plain-HTTP refusal, as a breaking change for direct http://…:8080 clients without a password;
  • the "Re-apply needed" badge;
  • an upgrade note;
  • KI-355.

Checklist

  • Linked to an issue (no tracking issue)
  • Tests added or updated
  • Docs updated in egret-data/docs
  • Commit messages follow Conventional Commits
  • No secrets, tokens or real data added

🤖 Generated with Claude Code

viethqb and others added 10 commits November 7, 2026 23:50
…a dedicated OPA listener

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The spec gains the security revision (migration 0215) behind the
re-apply marker; the OPA URL column moves to 0216.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…laims

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…o engine

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A Trino engine with no trinoConfig.auth got no authenticator, so any caller
on 8080 could be anyone. Normalise the spec to PASSWORD auth (no human users)
in create and update, and on a clone in the apply build, so old rows gain
auth when next applied. egret-monitor and egret_delegator then render the
authentication type. A jwt-delegation engine also gets a jwt block, since it
has no monitor user.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…malising auth

normalise_trino_auth only ever adds the password block. The egret-monitor
user is now skipped only for a real jwt-only coordinator (jwt mode AND an
auth.jwt block), so a jwt-mode spec with no jwt block renders PASSWORD with
the monitor user and stays non-delegating (no TLS). Add update-path tests.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… re-apply

Engines applied before the port-8080 hardening still run the old render.
Record TRINO_SECURITY_REV (migration 0215) beside the applied mechanism on
every successful Trino apply, expose EngineDto.security_reapply_needed for
provisioned Trino engines below it, and show a "Re-apply needed" badge in
the engines list.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…r re-apply

The pre-hardening render already set allow-insecure-over-http=false for a
jwt-only coordinator with no human password users, so the "Re-apply needed"
badge was a false alarm there. Also rename mechanism_not_recorded to
not_recorded now that it carries both notes.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The 2026-07-15 delegated-JWT spec and the 2026-07-04 ingress spec (EN and
VI) said allow-insecure-over-http=true was safe. It is not: at true, Trino
does not authenticate a plain-HTTP request and runs it as whatever
X-Trino-User names. Add an append-only correction to each, pointing at
the port-8080 hardening spec and its live result. The dev Trino config's
README and header comment no longer describe the flag as set to true and
say why it must not be.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
viethqb and others added 6 commits November 8, 2026 04:06
Every authenticated engine now renders allow-insecure-over-http=false, so
the opa-e2e harness can no longer reach Trino with a bare X-Trino-User on
plain HTTP 8080.

- identity-check.sh (new), run by up.sh on trino-opa while it is on the
  Ranger plugin and by layer2.sh after the switch to OPA, on trino-opa
  and trino-ranger: plain HTTP with X-Trino-User and no credential must
  answer 403, the same with X-Forwarded-Proto: https 401, and the
  delegator (Basic, forwarded https) must still become bob. A regression
  of the flag to true fails the job.
- trino-auth.sh (new, sourced): reads the engine's delegator password
  from the stack's Postgres and opens it with the public dev master key
  in deploy/opa-e2e/egret.env (AES-GCM via openssl AES-CTR); egret has
  no route that reveals it. Also the CLI helper over HTTPS 8443.
- trino-batch.sh, latency-run.sh: authenticate as the delegator with
  X-Forwarded-Proto: https; a direct job runs as egret's query path does
  (X-Trino-Original-User: egret_delegator); nextUri followed on 8080.
- up.sh step 7b + copy_auth.py: trino-ranger gets trino-opa's rendered
  authentication (config lines, password file, keystore), so both sides
  are reached the same way.
- layer2.sh, live-check.sh, latency.sh: the Trino CLI goes through
  trino_cli; the password reaches containers through the environment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ns on every apply

A provisioned Trino row stored before the hardening may lack
`trinoConfig.auth`. Every apply rendered it as PASSWORD auth from a
normalised clone, but the stored spec kept no auth block, so the monitor
still sent a bare `X-Trino-User` and was refused. Migration 0215 now
backfills `{"password":{}}` into such rows (idempotent; never a jwt block).

The apply of such an impersonate engine mints the delegator secret, but
only the PUT routes re-materialized workspace connections, so a re-apply by
rotation, a catalog change, restart-recreate or a mechanism switch left the
connection bound to no secret. `finish_apply` now re-materializes after
every successful Trino apply; a failure keeps the apply successful and is
logged and noted like the other post-apply records.

Tests: a legacy no-auth row re-applied by rotate-secrets binds the
delegator and the monitor signs in as egret-monitor; the security-rev test
now creates an open engine so its first assertion proves the write; a jwt
mode test split out; provision branch test asserts is_jwt_only_coordinator.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A jwt-only coordinator (jwt mode and an auth.jwt block) gets the viewer's
engine token; a jwt-mode engine without a jwt block renders PASSWORD and
gets Basic egret-monitor plus X-Forwarded-Proto: https.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The port-8080 spec gains an "As measured" note (standalone Trino for the
flag, opa-e2e as the run of record, worker preStop 403 pre-existing and out
of scope). The live result's rows 10 and 10s rejoin their table and it
records the Layer 2 and latency caveats for Task 26. The ingress
X-Forwarded-Proto statement is marked as not measured against a real
ingress controller.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@viethqb
viethqb force-pushed the feat/port-8080-hardening branch from 708c78d to b79c4e0 Compare November 8, 2026 00:03
@viethqb
viethqb merged commit 6661357 into main Nov 8, 2026
9 checks passed
@viethqb
viethqb deleted the feat/port-8080-hardening branch November 8, 2026 03:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant