Skip to content

fix(engines): StarRocks impersonation users, CN password and delete leftovers - #850

Merged
viethqb merged 21 commits into
mainfrom
fix/starrocks-delegation-and-lifecycle
Nov 12, 2026
Merged

viethqb merged 21 commits into
mainfrom
fix/starrocks-delegation-and-lifecycle

Conversation

@viethqb

@viethqb viethqb commented Nov 12, 2026 •

Copy link
Copy Markdown
Member

Summary

Three StarRocks defects found live on kind (package D of the StarRocks follow-ups):

  • D1 (KI-356, S1): an impersonation-mode StarRocks engine under Ranger could not run any query through Egret. The delegator had no Ranger right to impersonate, and the members had no StarRocks user.
  • D2: the CN had no MYSQL_PWD and crash-looped on a passworded engine.
  • D3 (KI-357): deleting an engine left the {name}-initpwd Job pods and the fe-meta PVCs behind. A same-name re-create then kept the old root password, users and catalogs.

No issue; tracked in the StarRocks follow-up programme. Spec: docs/en/specs/2026-11-11-starrocks-delegation-and-lifecycle-design.md. Plan: docs/en/plans/2026-11-11-starrocks-delegation-and-lifecycle.md (both with vi twins).

Changes

  • Ranger row: Egret renders an egret-delegator-impersonate row (resource user, isExcludes over root, egret_delegator, egret_query_log) for StarRocks impersonate engines. The OPA decide arm (InfraObject::AnyUserExcept) matches it. The G22 golden gains the row.
  • User sync:
    • Egret creates each member's StarRocks user, with an unusable random password, and drops it when access ends. Platform admins are included.
    • The bookkeeping lives in migration 0219_starrocks_users.sql (engine_mgmt.starrocks_users, root reserved in the username ledger, statement-level pg_notify triggers).
    • Sync runs on a LISTEN worker (1 s debounce, Busy retry), on every apply including a PUT that rolls the FE, at boot, and on a 10-minute sweep.
    • Reserved names are refused. A same-name user Egret did not create is left alone and logged.
  • Native access control (no profile): per-user GRANT IMPERSONATE ON USER for the delegator, then REVOKE … ON ALL USERS, and a revoke before every drop. The bootstrap no longer grants ON ALL USERS.
  • Provisioner: MYSQL_PWD comes from <name>-initpwd-secret on FE and CN (an admin-set value wins). Delete removes the initpwd Job (background propagation) and every fe-meta-<name>-fe-<i> claim. The FE StatefulSet has PVC retention whenDeleted: Delete.
  • Create: POST /engines answers 409 engine_metadata_left_over while fe-meta-<name>-fe-0 exists, with the kubectl delete pvc command (502 if the probe cannot read).

Test plan

  • Unit and integration: unit and DB tests for every task. Each final-review fix is pinned by a test that failed before it (a_starrocks_update_syncs_the_users_again).
  • CI parity, all green locally:
    • cargo fmt --check and workspace clippy -D warnings;
    • the core, perm-ir, policy-api, provisioner, engine-mgmt, auth, perm-compile (harness), perm-eval, policy and ranger-eval suites;
    • goldens (Ranger and EGRET_GOLDEN_MECHANISM=opa), ranger_engine_gate, opa_differential, ranger-eval differential.
  • Real FE (ghcr.io/egret-data/egret-starrocks-allin1:4.1.7-egret.4): new tests/starrocks_users_live.rs, gated on EGRET_TEST_STARROCKS_URL, passes in 1.4 s.
    • It covers sync, foreign user, drop and native access control.
    • On native access control: ON ALL USERS is gone, EXECUTE AS a member works, EXECUTE AS root and EXECUTE AS <foreign> are refused.
  • Fork plugin: the fork's e2e with the row Egret renders gives 51/51 and 10/10 audit checks. root, egret_query_log and egret_delegator are refused (5204); an ordinary user is allowed.
  • Live on kind (Ranger + OIDC, image built from this branch and grep-verified):
    1. Impersonation: member-sso ran SELECT current_user() through Egret and got 'member-sso'@'%', with no user created by hand. Removing them from the workspace dropped the user within about 1 s; re-adding brought it back within 5 s. The delegator is refused EXECUTE AS root and egret_query_log.
    2. CN: MYSQL_PWD comes from the initpwd secret on FE and CN. The CN ran with 0 restarts, was Alive: true, and re-registered after its pod was deleted.
    3. Delete: no pod, Job, PVC or PV was left after about 20 s.
    4. Same-name re-create: the new root password works and the user list is fresh.
    5. Leftover claim: with a leftover fe-meta claim, create gives 409 with the command. After running the command, create succeeds.
  • Not covered on kind: native access control on a profile-less engine. Package B binds a profile on every create under Ranger, so it is covered by the real-FE test above.

Documentation

egret-data/docs#76:

  • KI-356 and KI-357 are fixed.
  • KI-358 (orphan per-user grants, with manual cleanup) and KI-359 (lost CREATE USER reply, with recovery) are new.
  • Updated pages: engines and Ranger, workspace databases, engine detail (including the kubectl delete sts cascade note), provision (the 409), upgrade notes, REST reference.

Checklist

  • Linked to an issue (none; programme item)
  • Tests added or updated (or not applicable)
  • Docs updated in egret-data/docs for any user-visible feature, UI, API or config change (or not applicable: internal only)
  • Commit messages follow Conventional Commits
  • No secrets, tokens or real data added

🤖 Generated with Claude Code

viethqb and others added 20 commits November 11, 2026 22:29
Design for the three StarRocks defects found on kind: the Ranger
impersonate row and eager StarRocks user sync for impersonate mode,
MYSQL_PWD for the FE and CN, and engine delete removing the initpwd
Job's pods and the FE metadata claims. English and Vietnamese.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Twelve tasks: measurements on the fork's plugin and FE first, the D1b
infra row with its goldens, MYSQL_PWD, delete cleanup, migration 0219
and the StarRocks user sync with its hook points, a CI parity gate,
real-StarRocks and kind live checks, and the docs. English and
Vietnamese.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Platform admins are synced; native access control gets per-user
IMPERSONATE grants and loses ON ALL USERS in this package; users egret
did not create stay untouched; a StarRocks create answers 409 when an
old FE metadata claim is left. Adds M6 and M7. English and Vietnamese.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
New Task 6 (create refuses a leftover FE metadata claim) and Task 10
(native per-user IMPERSONATE, ON ALL USERS revoked, bootstrap stops
granting it); platform admins in the sync and its triggers; M6 and M7
in Task 1; live checks for both on kind. English and Vietnamese.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e but root and egret's users

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…oldens with it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…MYSQL_PWD

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… metadata claims

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…end metadata volume

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fy on membership writes

Migration 0219 adds engine_mgmt.starrocks_users, reserves `root` in the
username ledger, and wakes the StarRocks user sync (channel
egret_starrocks_user_sync) from statement triggers on access_grants,
user_groups, users, engine_workspaces and roles. The store gains the
candidate, record, forget and per-engine advisory-lock methods the sync
uses.

The two perm-compile tests that seed a person named `root` now write that
row with the ledger's triggers off, as a holder from before 0219.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eftover claim

A StarRocks create that reuses a live engine's name now answers the
ordinary name-exists 409 before the leftover-claim check, so the message
never advises deleting a running engine's metadata volume. The claim
name comes from the provisioner's fe_meta_claim_name, the helper moves
above mod tests, and a failed claim read is tested as a 502.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…gine's members

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…applies, the bootstrap and a sweep

The bootstrap syncs the users at a new checkpoint 7, after the delegation
statements and before the query log (now checkpoint 8). Every StarRocks
update apply spawns a sync once the FE answers. A LISTEN worker on
migration 0219's channel and a 10-minute sweep (first tick = boot
reconcile) feed one worker that debounces and drains wake-ups, so a burst
of statement-level NOTIFYs costs one pass. Every sync plans from the
engine row re-read at call time, never an earlier snapshot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d or a stale row

A username that cannot be minted fails the sync instead of skipping the
person; the platform admins' stored names are read in one query. The plan
uses the engine row re-read under the sync lock. A create refused as
"already exists" always forgets its record, so a lost user someone else
re-created is never adopted. Tests assert on their own users, not counts a
concurrent platform-admin test can change, and clean up panic-safely.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…Busy pass retries

sqlx's PgListener::recv() reconnects silently, so the "wake on reconnect"
never ran for an ordinary dropped connection. The listener now loops on
try_recv() and wakes the worker on Ok(None) (connection lost and
re-established), stopping once the pool closes. A pass that finds an
engine Busy asks for another pass shortly after instead of leaving the
write for the sweep. The coalescing test runs on paused time, and the
checkpoint-8 test no longer assumes the member's create is the last
statement.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… revoke ON ALL USERS

The bootstrap no longer grants IMPERSONATE ON ALL USERS. On an engine bound
to no profile, the user sync reads the delegator's grants by header, revokes
ON ALL USERS, and grants IMPERSONATE ON USER for each user egret created and
records. Reserved and foreign users are never granted; bound engines get no
native statement. StarRocks keeps a grant on a dropped user and cannot revoke
it afterwards, so the sync revokes it before DROP USER and skips the drop when
the revoke or the read fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…SERS, revoke before every drop

Per-user grants are sent first and ON ALL USERS is revoked only when all of
them landed, so a failure never leaves members un-impersonatable. The
delegator's grants are read once per sync; a listed per-user grant is revoked
before its user is dropped on bound engines too, since DROP USER keeps it.
The grants parser unquotes exactly as quote_string quotes. A none-mode engine
is not converged. The spec and plan record the measured DROP behaviour, the
new order and the 2026-11-12 ruling on orphan grants.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n guard

The workspace guard in egret-workflow's execution_timestamps reads
`ApplyOp::Update {` as an UPDATE whose table is interpolated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@viethqb
viethqb merged commit 88a0e9e into main Nov 12, 2026
9 checks passed
@viethqb
viethqb deleted the fix/starrocks-delegation-and-lifecycle branch November 12, 2026 06:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant