Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions platform/control-plane/query-engines/engine-detail.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,48 @@ The header shows the engine name, its namespace (`egret-engines`), the image, th

A provisioned StarRocks engine that a policy profile names shows, above the tabs, the warning that unless the engine runs `egret-starrocks-fe` `4.1.7-egret.4` or later its StarRocks lets users past the profile: official images serve the Iceberg metadata tables without asking Ranger and apply no mask or row filter to prepared statements, and official and earlier patched images apply none through the HTTP SQL API or Arrow Flight SQL, let anyone read object storage through `FILES()` and let a masked or filtered user store a table's rows in the clear in a materialized view, and images before `4.1.7-egret.4` show a filtered user the statistics of hidden rows through `EXPLAIN COSTS` and let anyone reach object storage with the cluster's own identity through `BROKER LOAD`, `EXPORT` and `SELECT INTO OUTFILE` without keys (since platform #822, widened in #826, #827 and #832; see [External engine](#overview-1), KI-340, KI-345 to KI-348, KI-350, KI-351). It is not shown while the profile list loads, or to a viewer who cannot read it. It is not shown for a provisioned engine on Egret's patched image `ghcr.io/egret-data/egret-starrocks-fe` `4.1.7-egret.4` or later (the default for new engines since platform #832), which fixes all of them; it is shown for an engine still on `4.1.7-egret.1` or `4.1.7-egret.3`; an external engine always shows it, because Egret does not know its image.

### Security warnings on a StarRocks engine

Two warnings can appear above the tabs of a StarRocks engine (since platform #847).

#### Not governed

With Ranger on, a StarRocks engine that is bound to no policy profile shows:

*Not governed: this engine is bound to no policy profile, so egret's grants, masks and row filters do not reach it. Bind it to a StarRocks profile in its settings.*

The engine list shows the same engine with a **Not governed** badge. To fix it, bind a StarRocks profile in the engine's settings (`authzServiceId` on `PUT /api/control-plane/engines/{id}`). With Ranger off no engine is governed, so the warning is not shown. Add engine now offers a profile for StarRocks (see [Provision an engine](./provision.md#create-an-engine)), so a new engine is bound from the start.

#### Security upgrade needed

A provisioned StarRocks engine that lacks egret's security fixes shows the warning *This engine needs a security upgrade* with one sentence per reason, and an **Upgrade security** button. The older warning about metadata tables is not shown while this one is.

![The engine page of a StarRocks engine with the security upgrade warning and the Upgrade security button](/img/platform/admin/cp-engine-security-upgrade-warning.png)

| Reason | API value | What it means |
|---|---|---|
| Image | `unpatchedImage` | The FE image is not `egret-starrocks-fe` `4.1.7-egret.4` or later (or a mirror of one). StarRocks then lets users past the engine's profile: Iceberg metadata tables, prepared statements, the HTTP SQL API, `FILES()` and more (KI-340, KI-345). |
| Root password | `noRootPassword` | The FE `root` account has no password: anyone who reaches port 9030 can log in as `root`. Engines created before the root-password change have none. |
| Not applied | `notApplied` | The last security upgrade stored its changes but did not finish applying them, for example because the apply failed. |

The warning needs the `engine:update` permission to act on; without it the page says *Upgrading it needs the engine:update permission.* An external engine never shows it: egret does not change an external engine's image.

**Upgrade security** opens a confirmation dialog, *Upgrade this engine's security?*, that lists what will change:

- `FE image: <current> → <new>`, and *The CN image moves to the same release.* When egret has no FE image recorded for the engine, `<current>` reads *the current image (not recorded)*;
- *root password will be set*, when the engine has none;
- *The stored changes are applied again.*, for `notApplied`.

![The Upgrade this engine's security dialog listing the FE image change and the root password](/img/platform/admin/cp-engine-security-upgrade-dialog.png)

It also warns that the FE and CN pods restart, and that a client outside egret that logs in as `root` with no password is refused afterwards. When a root password will be set, it adds *egret's own admin statements on this engine fail until the FE takes the new password, usually within a minute.* Press **Upgrade security** in the dialog to run it. The engine is applied on every run, so running it again is safe.

If the FE image is of a newer release than egret's default fork image (an official 4.1.8, for example), egret does not change the image, because StarRocks FE metadata cannot be downgraded. The root password is still set, and the dialog and the result say:

*No patched image for release* `<x>` *yet: the image was not changed.*

The `unpatchedImage` reason then stays. If the apply fails, the page shows the error, the engine keeps the **Security upgrade needed** badge with the reason *Not applied*, and you run **Upgrade security** again. Any later successful apply of the engine, a settings save for example, also clears that reason.

### Overview

A strip shows the **phase**, **pods** (ready/total; **containers** on the Docker provisioner), **replicas** (for example *Coord 1 · Worker 0*, the counts the engine runs with), **catalogs**, **workspaces** and **auth** (*JWT auth*, impersonation, password auth or no delegation). For StarRocks, **FE replicas** and **CN replicas** replace *replicas*.
Expand Down
7 changes: 7 additions & 0 deletions platform/control-plane/query-engines/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,13 @@ A summary strip counts **engines**, how many are **running** and **not running**

A provisioned Trino engine can also carry a **Re-apply needed** badge (the API field is `securityReapplyNeeded` on the engine). Its tooltip says *This engine still accepts unauthenticated requests on port 8080. Re-apply it to turn on authentication.* The engine was applied before platform #842 and still runs the old configuration. The badge clears after the next successful apply. An engine with JWT only and no password users never shows it. See [Provision an engine](./provision.md#security--exposure).

Since platform #847 a StarRocks engine can carry two more badges beside the status:

- **Security upgrade needed** for a provisioned engine on an image or login that lacks egret's security fixes. Open the engine and press **Upgrade security** (see [Engine detail](./engine-detail.md#security-upgrade-needed)).
- **Not governed** for a StarRocks engine with no policy profile while Ranger is on (see [Engine detail](./engine-detail.md#not-governed)).

![The Engines list with the Security upgrade needed and Not governed badges on StarRocks engines](/img/platform/admin/cp-engines-security-upgrade-badge.png)

## Add an engine

Two buttons sit at the top right:
Expand Down
4 changes: 4 additions & 0 deletions platform/control-plane/query-engines/provision.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@ You need **engine:create** and **engine:provision**. Egret runs the engine where
4. Under **Configuration**, adjust what you need. Leave a field blank to use the default.
5. Press **Create engine**.

For a StarRocks engine the form has a **Policy profile** select (since platform #847; fixes KI-336). It lists the StarRocks profiles that no engine uses yet. Its default, **Create a profile named after the engine**, makes a new StarRocks profile with the engine's name when Ranger is on; with Ranger off nothing is bound. Without a profile, egret's grants, masks and row filters do not reach the engine, and with Ranger on the engine shows **Not governed**. If a profile with the engine's name already exists and is of another type, or another engine uses it, the create is refused (409).

![The Add engine form for StarRocks with the Policy profile select open](/img/platform/admin/cp-engine-create-starrocks-profile.png)

The engine page opens with the status **Provisioning**. It turns **Running** when the coordinator answers. Measured on the demo stack: 72 seconds for the first engine on a Docker host that already had the image, and 38 seconds for a restart.

:::tip Single-node engine
Expand Down
8 changes: 8 additions & 0 deletions platform/deploy-operate/upgrade.md
Original file line number Diff line number Diff line change
Expand Up @@ -678,6 +678,14 @@ default, and mounts the OPA surface with no configuration change, so a profile c
- **Tag-service warning:** a Trino profile with no usable tag service now carries `no_tag_service_warning` on the `PATCH …/mechanism` answer and on each explain profile, and the console shows it. See [Trino on OPA](../data-governance/engines-and-policy/trino-on-opa.md#through-the-api).
- **Known issues** on OPA engines: KI-352, KI-353 and KI-354 in [Known issues](../known-issues.md).

### StarRocks engine posture (platform #847)

Nothing changes on its own when you upgrade: no migration rewrites an engine's image or sets a password, because the FE restarts. Migration 0217 adds one flag, which records a security upgrade that was stored but not applied.

- **After the upgrade, open Control Plane → Infrastructure → Engines and look for Security upgrade needed.** An engine created before platform #826 and #832 can still run an official StarRocks FE or an egret image before `4.1.7-egret.4`, or have a passwordless FE `root`. Open each one and press **Upgrade security** (or `POST /api/control-plane/engines/{id}/security-upgrade`) in a quiet window: the FE and CN pods restart. A client outside egret that logs in as `root` with no password is refused afterwards. See [Engine detail](../control-plane/query-engines/engine-detail.md#security-upgrade-needed).
- Look for **Not governed** too: with Ranger on, a StarRocks engine with no policy profile enforces none of egret's policies. Bind a profile in its settings.
- **API change:** the engine adds `notGoverned`, `securityUpgradeNeeded`, `securityUpgradeReasons`, `securityUpgradeFeImage` and `securityUpgradeBlockedRelease`, and a new route `POST /api/control-plane/engines/{id}/security-upgrade`. See [Engines API](../reference/rest-api/control-plane-engines.md).

## Further reading

- [Backup and restore](./backup-restore)
Expand Down
Loading
Loading