From 7d4bc3815300b0e131f8fd91fe0dc16e22d692a0 Mon Sep 17 00:00:00 2001 From: 1Pro-L4R8 Date: Sun, 8 Nov 2026 16:31:14 +0700 Subject: [PATCH 1/6] docs(opa): OPA on by default, tag-service warning Document [governance.opa] enabled and Helm opa.enabled defaulting to true, how to turn it off, the no_tag_service_warning on the mechanism switch and explain, and an upgrade note. KI-352..354 already exist and are unchanged. Co-Authored-By: Claude Opus 5.5 --- .../engines-and-policy/trino-on-opa.md | 33 +++++++++++++------ platform/deploy-operate/kubernetes.md | 4 +-- platform/deploy-operate/upgrade.md | 12 +++++++ platform/reference/configuration-keys.md | 2 +- 4 files changed, 38 insertions(+), 13 deletions(-) diff --git a/platform/data-governance/engines-and-policy/trino-on-opa.md b/platform/data-governance/engines-and-policy/trino-on-opa.md index eed44fc..a920247 100644 --- a/platform/data-governance/engines-and-policy/trino-on-opa.md +++ b/platform/data-governance/engines-and-policy/trino-on-opa.md @@ -12,18 +12,28 @@ You switch a **policy profile**, not an engine. Every engine bound to the profil ## Enable the feature -The feature is gated by one key, **off by default**: +The feature is gated by one key, **on by default** from this release: | Key | Environment variable | Default | |---|---|---| -| `[governance.opa] enabled` | `EGRET_GOVERNANCE__OPA__ENABLED` | `false` | +| `[governance.opa] enabled` | `EGRET_GOVERNANCE__OPA__ENABLED` | `true` | + +Nothing needs to be set to use OPA, as long as `ranger.enabled` is on. Turning the key on does not move any profile: every profile stays on Ranger until an operator switches it (see [Switch a profile](#who-can-switch)), so the default alone changes no enforcement. It does bind the OPA listener (`0.0.0.0:8082` by default) on every installation. + +To turn the feature off, use any one of these: + +```toml +[governance.opa] +enabled = false +``` ```bash -EGRET_GOVERNANCE__OPA__ENABLED=true -EGRET_RANGER__ENABLED=true +EGRET_GOVERNANCE__OPA__ENABLED=false ``` -Egret serves the OPA routes on their own port, **8082** by default, not on the API port 8080 (see [Network exposure](#network-exposure)). On Kubernetes, the Helm value `opa.enabled=true` sets both keys of the OPA listener for you (see [Kubernetes](../../deploy-operate/kubernetes.md)). +On Kubernetes, set the Helm value `opa.enabled=false`. The chart renders the environment variable explicitly either way. + +Egret serves the OPA routes on their own port, **8082** by default, not on the API port 8080 (see [Network exposure](#network-exposure)). On Kubernetes, the Helm value `opa.enabled` (default `true`) sets both keys of the OPA listener for you (see [Kubernetes](../../deploy-operate/kubernetes.md)). It also needs `ranger.enabled = true`, because the OPA routes check the engine credentials of the Ranger surface (see [Engines and Ranger setup](../../deploy-operate/engines-and-ranger.md)). With the Ranger surface off, OPA does not run and a switch to OPA is refused with the same message as a disabled gate. @@ -37,7 +47,7 @@ A profile can switch to OPA when all of these hold: - it is a **Trino** profile; - Egret generates it: it holds grants or rules from the permission model, not hand-written policies only; -- the installation has `[governance.opa] enabled = true`; +- the installation has `[governance.opa] enabled = true` (the default from this release; see [Enable the feature](#enable-the-feature)); - every bound engine that **Egret provisions** runs `trinodb/trino:483`, the one image Egret has measured to ship `plugin/opa` (a pinned digest must be the known one); - it holds no `all - …` default policies; - neither the profile nor its tag service holds hand-written policies; @@ -57,7 +67,7 @@ A refused switch lists every condition that holds, one message each, and writes | Message | What to do | |---|---| -| *OPA is not enabled on this installation ([governance.opa] enabled)* | Set `EGRET_GOVERNANCE__OPA__ENABLED=true` (and keep `ranger.enabled`), then restart Egret. | +| *OPA is not enabled on this installation ([governance.opa] enabled)* | The key is on by default, so someone turned it off: remove `enabled = false` (or set `EGRET_GOVERNANCE__OPA__ENABLED=true`, or Helm `opa.enabled=true`), keep `ranger.enabled`, then restart Egret. | | *only a Trino profile can use OPA; this profile is of type \* | Nothing: StarRocks profiles stay on Ranger. | | *only a generated profile can use OPA: hand-written policies are Ranger JSON that OPA cannot evaluate* | Express the profile's access as grants and rules (see [Migrate policies](./migrate.md)), so that Egret generates it. | | *engine \ runs \, which does not ship the Trino OPA plugin; supported: trinodb/trino:483* | Move the engine to `trinodb/trino:483` in the Control plane, or bind it to another profile. | @@ -80,6 +90,8 @@ You need the **policy:update** permission. Without it the section is read-only a On success a toast says *Switched to OPA.*, and the section lists the engines it reconfigured (*Reconfigured: …*), the external engines it did not restart (*Not restarted (external): …*) and what their operator must do. On a refusal the section lists every message (above) and keeps the previous selection. Switching back works the same way, from **Ranger** and **Switch to Ranger?**. +A Trino profile that has no usable tag service is switched all the same, but the section also shows a warning after the switch (Governance → Engines → Advanced): *This profile has no tag service, so classification rules and clearances do not apply to its engines. Bind a tag service in the profile's settings.* Classification rules and clearances need a tag service on either mechanism, so the warning follows the profile, not the switch. StarRocks profiles never show it. + ### Through the API ```bash @@ -92,10 +104,10 @@ The body is `{"mechanism": "ranger"}` or `{"mechanism": "opa"}`. It needs `polic | Status | Meaning | |---|---| -| 200 | The switch is committed. The body is `{name, mechanism, engines_reconfigured, external_engines_unchanged, external_engines_warning?}`. | +| 200 | The switch is committed. The body is `{name, mechanism, engines_reconfigured, external_engines_unchanged, external_engines_warning?, no_tag_service_warning?}`. `no_tag_service_warning` is set for a Trino profile with no usable tag service. | | 404 | No such policy profile. | | 409 | The switch to OPA is refused, nothing is written. The body is `{error, code: "mechanism_refused", refusals}`: `refusals` holds every message of the table above that applies, and `error` is the first. | -| 502 | The switch is committed, but some bound engines were not reconfigured (below). | +| 502 | The switch is committed, but some bound engines were not reconfigured (below). The body carries `no_tag_service_warning` too when it applies. | ## What a switch does @@ -217,11 +229,12 @@ On an OPA profile, [Explain access](../data/explain-access.md) answers from the - A read shows the column mask as applied and the row filter, for example *Column mask: MASK_HASH, applied as …* and *Row filter: …*. - *Could not read the profile’s current state; try again.* means the snapshot could not be read at that moment. - *This answer is from the previous snapshot: the current model did not build, so egret answers from the previous data until it does.* appears when the profile's model did not lower. +- A Trino profile with no usable tag service shows once, in its card, *This profile has no tag service, so classification rules and clearances do not apply to its engines. Bind a tag service in the profile's settings.* (on Ranger and on OPA). Without a tag service, tag masks, tag row filters and clearances do not reach the engines, so the explanation of a read can differ from what the classification suggests. - After a switch, an engine not re-applied yet is named: *Not re-applied yet: \ still runs the previous plugin, so it may answer differently from this OPA explanation until re-applied (send the switch again).* ![Who can read? on lake.s1.t2 for bob: the profile opa_e2e with its Trino and OPA badges, and bob allowed to select s1.t2 by a grant](/img/platform/admin/gov-opa-explain.png) -Through the API, `POST /api/control-plane/perm/explain` carries `mechanism` (`ranger` or `opa`), `ir_reason` on each access, `engines_not_reapplied` when an engine still runs the other plugin, and, for a read, the rendered column mask and the AND of the applying row filters under `opa` (`column_mask`, `column_mask_type`, `row_filter`). A column mask is rendered with `varchar` where the engine substitutes the column's own type. +Through the API, `POST /api/control-plane/perm/explain` carries `mechanism` (`ranger` or `opa`), `ir_reason` on each access, `engines_not_reapplied` when an engine still runs the other plugin, `no_tag_service_warning` on each profile of the answer that has no usable tag service (the same text as above; StarRocks profiles carry none), and, for a read, the rendered column mask and the AND of the applying row filters under `opa` (`column_mask`, `column_mask_type`, `row_filter`). A column mask is rendered with `varchar` where the engine substitutes the column's own type. Explain follows the **profile's** mechanism, not each engine's. It also cannot see a snapshot the refresher has dropped, so for about one refresh period (1 s) it can show a read that the engine refuses, never the reverse. diff --git a/platform/deploy-operate/kubernetes.md b/platform/deploy-operate/kubernetes.md index 82f53e3..5608675 100644 --- a/platform/deploy-operate/kubernetes.md +++ b/platform/deploy-operate/kubernetes.md @@ -25,7 +25,7 @@ With default values `helm template` renders these objects: | Object | Purpose | |---|---| | `Deployment/egret` | The Egret server. Probes: startup `GET /healthz`, readiness `GET /readyz`, liveness `GET /healthz`. Requests 500m CPU and 1 GiB, limits 2 CPU and 4 GiB. `terminationGracePeriodSeconds` is 65, which must stay above the server's 55 second drain. | -| `Service/egret` | Port 8080 (8081 for Data Apps when `apps.origin` is set, and the port named `opa` when `opa.enabled` is set). | +| `Service/egret` | Port 8080 (8081 for Data Apps when `apps.origin` is set, and the port named `opa` while `opa.enabled` is on, which is the default). | | `Deployment/egret-frontend`, `Service/egret-frontend` | The built web app behind nginx, as a NodePort Service (30080). | | `Deployment/egret-renderer`, `Service/egret-renderer` | The screenshot service used by reports. | | `Job/egret-init` | A post-install and post-upgrade hook that runs `egret init`, creating the administrator and the `default` workspace. It is idempotent. | @@ -182,7 +182,7 @@ Running a script task also needs the workspace feature `workflow_containers` (of | `engineNetworkPolicy.operatorNamespaces` | `[starrocks]` | Namespaces of the StarRocks operator, let in on the front end's 8030 and 9030. | | `engineNetworkPolicy.ingressFrom` | `[]` | NetworkPolicy peers an engine Ingress forwards from (your ingress controller), let in on 8030 (StarRocks front end) and 8080 and 8443 (Trino). **Required when an engine has an Ingress**, or the Ingress gets no traffic. | | `engineNetworkPolicy.extraFrom` | `[]` | Other NetworkPolicy peers, let in on every port, for example a metrics scraper. | -| `opa.enabled` | `false` | Sets `EGRET_GOVERNANCE__OPA__ENABLED=true` and `EGRET_GOVERNANCE__OPA__LISTEN=0.0.0.0:`, and adds the `opa` port to the Deployment and the `Service/egret`. It still needs `ranger.enabled`. See [Trino on OPA](../data-governance/engines-and-policy/trino-on-opa.md). | +| `opa.enabled` | `true` | Sets `EGRET_GOVERNANCE__OPA__ENABLED=true` (`false` when you turn it off; the chart renders the variable either way) and `EGRET_GOVERNANCE__OPA__LISTEN=0.0.0.0:`, and adds the `opa` port to the Deployment and the `Service/egret`. Profiles stay on Ranger until an operator switches them. It still needs `ranger.enabled`. Set `opa.enabled=false` to turn OPA off. See [Trino on OPA](../data-governance/engines-and-policy/trino-on-opa.md). | | `opa.port` | `8082` | The port of Egret's OPA listener. Egret derives the URL engines use from `catalog.public_url` with this port, so no other value is needed while that URL is Egret's in-cluster address. | | `opa.extraFrom` | `[]` | NetworkPolicy peers beyond `engineNamespace` that may reach the OPA port, for example the namespace of an external Trino on OPA. | | `apps.origin`, `apps.egretOrigin` | empty | The Data Apps origin (a sibling subdomain such as `apps.`) and Egret's own origin. Empty turns the apps iframe and its listener off. | diff --git a/platform/deploy-operate/upgrade.md b/platform/deploy-operate/upgrade.md index ef5284a..e4ee7fd 100644 --- a/platform/deploy-operate/upgrade.md +++ b/platform/deploy-operate/upgrade.md @@ -665,6 +665,18 @@ while `[governance.opa] enabled` is `false`. See [Trino on OPA](../data-governan and `opa.extraFrom`. With `engineNetworkPolicy.enabled`, `egret-ingress` admits only 8080, 8081 (apps) and the OPA port to the backend pod: anything that reached another port on it is now refused. Add the namespace of an external Trino on OPA to `opa.extraFrom`. See [Kubernetes](./kubernetes.md#engine-network-policy). +### OPA is on by default + +`[governance.opa] enabled` now defaults to `true`, and so does the Helm value `opa.enabled`. **Profiles stay on Ranger until an operator +switches them**, so the change alone moves no enforcement. What changes is that Egret now binds the OPA listener, port **8082** by +default, on every installation, and that a profile can be switched to OPA without touching configuration first. See +[Trino on OPA](../data-governance/engines-and-policy/trino-on-opa.md). + +- **Turning it off:** `[governance.opa] enabled = false`, `EGRET_GOVERNANCE__OPA__ENABLED=false`, or Helm `opa.enabled=false`. The chart renders the variable explicitly either way. Switch any profile on OPA back to Ranger first: with the key off, its engines fail every query. +- **Network:** the chart's `NetworkPolicy/egret-ingress` (with `engineNetworkPolicy.enabled`) admits the OPA port only from `engineNamespace` and `opa.extraFrom`. Outside Kubernetes, keep port 8082 reachable from the engines only. +- **Tag-service warning:** a Trino profile with no usable tag service now carries `no_tag_service_warning` on the `PATCH …/mechanism` answer and on each explain profile, and the console shows it. See [Trino on OPA](../data-governance/engines-and-policy/trino-on-opa.md#through-the-api). +- **Known issues** on OPA engines: KI-352, KI-353 and KI-354 in [Known issues](../known-issues.md). + ## Further reading - [Backup and restore](./backup-restore) diff --git a/platform/reference/configuration-keys.md b/platform/reference/configuration-keys.md index fc488a2..8296594 100644 --- a/platform/reference/configuration-keys.md +++ b/platform/reference/configuration-keys.md @@ -156,7 +156,7 @@ The S3-compatible object store used for the catalog warehouse, query results, wo | Key | Environment variable | Default | Meaning | |---|---|---|---| -| `enabled` | `EGRET_GOVERNANCE__OPA__ENABLED` | `false` | Serve the OPA decision routes for Trino, on their own listener (`listen`), and allow a policy profile to switch to OPA. Needs `ranger.enabled`. See [Trino on OPA](../data-governance/engines-and-policy/trino-on-opa.md). | +| `enabled` | `EGRET_GOVERNANCE__OPA__ENABLED` | `true` | Serve the OPA decision routes for Trino, on their own listener (`listen`), and allow a policy profile to switch to OPA. On by default; profiles stay on Ranger until an operator switches them. Set `false` to turn it off. Needs `ranger.enabled`. See [Trino on OPA](../data-governance/engines-and-policy/trino-on-opa.md). | | `batch_body_limit_bytes` | `EGRET_GOVERNANCE__OPA__BATCH_BODY_LIMIT_BYTES` | `25165824` (24 MiB) | Largest request body the OPA `/batch` route accepts. Trino sends a listing of a whole schema as one batch; a larger body is answered 413 and the query fails. From 4194304 (4 MiB) to 67108864 (64 MiB); every other OPA route keeps 4 MiB. | | `body_read_timeout_ms` | `EGRET_GOVERNANCE__OPA__BODY_READ_TIMEOUT_MS` | `5000` | How long an admitted OPA request may take to send its whole body before it is answered 408 and its connection closed. From 1 to 60000. | | `listen` | `EGRET_GOVERNANCE__OPA__LISTEN` | `0.0.0.0:8082` | Where the OPA listener binds, `host:port`. It serves only the OPA routes; port 8080 answers 404 for `/opa/`. With `enabled`, Egret refuses to start when the port equals `server.port` or, with `apps.origin` set, the apps listener's port; with OPA off it is not checked. | From a305a7a7ed81baab04e8020aac7a92cd36127a85 Mon Sep 17 00:00:00 2001 From: 1Pro-L4R8 Date: Sun, 8 Nov 2026 16:34:27 +0700 Subject: [PATCH 2/6] docs(opa): OPA listener needs ranger.enabled, fix anchors and Helm wording Co-Authored-By: Claude Opus 5.5 --- .../data-governance/engines-and-policy/trino-on-opa.md | 4 ++-- platform/deploy-operate/kubernetes.md | 2 +- platform/deploy-operate/upgrade.md | 10 +++++----- 3 files changed, 8 insertions(+), 8 deletions(-) diff --git a/platform/data-governance/engines-and-policy/trino-on-opa.md b/platform/data-governance/engines-and-policy/trino-on-opa.md index a920247..619441c 100644 --- a/platform/data-governance/engines-and-policy/trino-on-opa.md +++ b/platform/data-governance/engines-and-policy/trino-on-opa.md @@ -18,7 +18,7 @@ The feature is gated by one key, **on by default** from this release: |---|---|---| | `[governance.opa] enabled` | `EGRET_GOVERNANCE__OPA__ENABLED` | `true` | -Nothing needs to be set to use OPA, as long as `ranger.enabled` is on. Turning the key on does not move any profile: every profile stays on Ranger until an operator switches it (see [Switch a profile](#who-can-switch)), so the default alone changes no enforcement. It does bind the OPA listener (`0.0.0.0:8082` by default) on every installation. +OPA also needs `ranger.enabled`, which is **off by default**. The OPA surface and its listener run only when both keys are on; with `ranger.enabled` off, nothing binds the OPA port and a switch to OPA is refused. When both are on, Egret binds the OPA listener (`0.0.0.0:8082` by default). An installation that already runs the Ranger surface therefore binds 8082 and mounts the OPA surface after the upgrade, with no configuration change. The key being on does not move any profile; each profile stays on Ranger until an operator switches it (see [Switch a profile](#switch-a-profile)), so the default alone changes no enforcement. To turn the feature off, use any one of these: @@ -31,7 +31,7 @@ enabled = false EGRET_GOVERNANCE__OPA__ENABLED=false ``` -On Kubernetes, set the Helm value `opa.enabled=false`. The chart renders the environment variable explicitly either way. +On Kubernetes, set the Helm value `opa.enabled=false`. The chart sets `EGRET_GOVERNANCE__OPA__ENABLED` from `opa.enabled`. Egret serves the OPA routes on their own port, **8082** by default, not on the API port 8080 (see [Network exposure](#network-exposure)). On Kubernetes, the Helm value `opa.enabled` (default `true`) sets both keys of the OPA listener for you (see [Kubernetes](../../deploy-operate/kubernetes.md)). diff --git a/platform/deploy-operate/kubernetes.md b/platform/deploy-operate/kubernetes.md index 5608675..f0044d9 100644 --- a/platform/deploy-operate/kubernetes.md +++ b/platform/deploy-operate/kubernetes.md @@ -182,7 +182,7 @@ Running a script task also needs the workspace feature `workflow_containers` (of | `engineNetworkPolicy.operatorNamespaces` | `[starrocks]` | Namespaces of the StarRocks operator, let in on the front end's 8030 and 9030. | | `engineNetworkPolicy.ingressFrom` | `[]` | NetworkPolicy peers an engine Ingress forwards from (your ingress controller), let in on 8030 (StarRocks front end) and 8080 and 8443 (Trino). **Required when an engine has an Ingress**, or the Ingress gets no traffic. | | `engineNetworkPolicy.extraFrom` | `[]` | Other NetworkPolicy peers, let in on every port, for example a metrics scraper. | -| `opa.enabled` | `true` | Sets `EGRET_GOVERNANCE__OPA__ENABLED=true` (`false` when you turn it off; the chart renders the variable either way) and `EGRET_GOVERNANCE__OPA__LISTEN=0.0.0.0:`, and adds the `opa` port to the Deployment and the `Service/egret`. Profiles stay on Ranger until an operator switches them. It still needs `ranger.enabled`. Set `opa.enabled=false` to turn OPA off. See [Trino on OPA](../data-governance/engines-and-policy/trino-on-opa.md). | +| `opa.enabled` | `true` | Sets `EGRET_GOVERNANCE__OPA__ENABLED=true` (the chart sets the variable from `opa.enabled`) and `EGRET_GOVERNANCE__OPA__LISTEN=0.0.0.0:`, and adds the `opa` port to the Deployment and the `Service/egret`. Profiles stay on Ranger until an operator switches them. It still needs `ranger.enabled`, which is off by default; without it nothing binds the OPA port. Set `opa.enabled=false` to turn OPA off. See [Trino on OPA](../data-governance/engines-and-policy/trino-on-opa.md). | | `opa.port` | `8082` | The port of Egret's OPA listener. Egret derives the URL engines use from `catalog.public_url` with this port, so no other value is needed while that URL is Egret's in-cluster address. | | `opa.extraFrom` | `[]` | NetworkPolicy peers beyond `engineNamespace` that may reach the OPA port, for example the namespace of an external Trino on OPA. | | `apps.origin`, `apps.egretOrigin` | empty | The Data Apps origin (a sibling subdomain such as `apps.`) and Egret's own origin. Empty turns the apps iframe and its listener off. | diff --git a/platform/deploy-operate/upgrade.md b/platform/deploy-operate/upgrade.md index e4ee7fd..245dee4 100644 --- a/platform/deploy-operate/upgrade.md +++ b/platform/deploy-operate/upgrade.md @@ -548,7 +548,7 @@ migrating it is refused until its missing levels are set (see [Migrate policies] ### Trino on OPA, preview (platform #838) Migration 0213 adds `mechanism` to every policy profile, with the default `ranger`, and `applied_mechanism` to every -engine. Nothing changes until `[governance.opa] enabled` is turned on, which it is not by default, and a profile is +engine. Nothing changes until `[governance.opa] enabled` is turned on, which it is not by default (changed in a later release; see below), and a profile is switched. See [Trino on OPA](../data-governance/engines-and-policy/trino-on-opa.md). - **API addition:** `ProfileDto.mechanism`, `PATCH /api/control-plane/policies/services/{service}` and @@ -556,7 +556,7 @@ switched. See [Trino on OPA](../data-governance/engines-and-policy/trino-on-opa. ### Trino on OPA, console and limits (platform #839) -No migration. `[governance.opa] enabled` stays `false` by default. See [Trino on OPA](../data-governance/engines-and-policy/trino-on-opa.md). +No migration. `[governance.opa] enabled` stays `false` by default (changed in a later release; see below). See [Trino on OPA](../data-governance/engines-and-policy/trino-on-opa.md). - **Console:** the **Access control** section on a Trino profile's **Advanced** tab switches between Ranger and OPA; **Download Trino config** and an external engine's **Plugin config** tab show the OPA files for a profile on OPA. @@ -668,11 +668,11 @@ while `[governance.opa] enabled` is `false`. See [Trino on OPA](../data-governan ### OPA is on by default `[governance.opa] enabled` now defaults to `true`, and so does the Helm value `opa.enabled`. **Profiles stay on Ranger until an operator -switches them**, so the change alone moves no enforcement. What changes is that Egret now binds the OPA listener, port **8082** by -default, on every installation, and that a profile can be switched to OPA without touching configuration first. See +switches them**, so the change alone moves no enforcement. What changes is that, where `ranger.enabled` is on (it is off by default), Egret now binds the OPA listener, port **8082** by +default, and mounts the OPA surface with no configuration change, so a profile can be switched to OPA without touching configuration first. With `ranger.enabled` off, nothing binds 8082 and a switch to OPA is refused. See [Trino on OPA](../data-governance/engines-and-policy/trino-on-opa.md). -- **Turning it off:** `[governance.opa] enabled = false`, `EGRET_GOVERNANCE__OPA__ENABLED=false`, or Helm `opa.enabled=false`. The chart renders the variable explicitly either way. Switch any profile on OPA back to Ranger first: with the key off, its engines fail every query. +- **Turning it off:** `[governance.opa] enabled = false`, `EGRET_GOVERNANCE__OPA__ENABLED=false`, or Helm `opa.enabled=false`. The chart sets the variable from `opa.enabled`. Switch any profile on OPA back to Ranger first: with the key off, its engines fail every query. - **Network:** the chart's `NetworkPolicy/egret-ingress` (with `engineNetworkPolicy.enabled`) admits the OPA port only from `engineNamespace` and `opa.extraFrom`. Outside Kubernetes, keep port 8082 reachable from the engines only. - **Tag-service warning:** a Trino profile with no usable tag service now carries `no_tag_service_warning` on the `PATCH …/mechanism` answer and on each explain profile, and the console shows it. See [Trino on OPA](../data-governance/engines-and-policy/trino-on-opa.md#through-the-api). - **Known issues** on OPA engines: KI-352, KI-353 and KI-354 in [Known issues](../known-issues.md). From c645d5ff10ba5e9faa41bb3240e58ed904bfd5a0 Mon Sep 17 00:00:00 2001 From: 1Pro-L4R8 Date: Sun, 8 Nov 2026 16:35:34 +0700 Subject: [PATCH 3/6] docs(opa): refusal row names both causes Co-Authored-By: Claude Opus 5.5 --- platform/data-governance/engines-and-policy/trino-on-opa.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/platform/data-governance/engines-and-policy/trino-on-opa.md b/platform/data-governance/engines-and-policy/trino-on-opa.md index 619441c..bcefbd9 100644 --- a/platform/data-governance/engines-and-policy/trino-on-opa.md +++ b/platform/data-governance/engines-and-policy/trino-on-opa.md @@ -67,7 +67,7 @@ A refused switch lists every condition that holds, one message each, and writes | Message | What to do | |---|---| -| *OPA is not enabled on this installation ([governance.opa] enabled)* | The key is on by default, so someone turned it off: remove `enabled = false` (or set `EGRET_GOVERNANCE__OPA__ENABLED=true`, or Helm `opa.enabled=true`), keep `ranger.enabled`, then restart Egret. | +| *OPA is not enabled on this installation ([governance.opa] enabled)* | Either the key was turned off or `ranger.enabled` (off by default) is off. Remove `enabled = false` (or set `EGRET_GOVERNANCE__OPA__ENABLED=true`, or Helm `opa.enabled=true`), keep `ranger.enabled`, then restart Egret. | | *only a Trino profile can use OPA; this profile is of type \* | Nothing: StarRocks profiles stay on Ranger. | | *only a generated profile can use OPA: hand-written policies are Ranger JSON that OPA cannot evaluate* | Express the profile's access as grants and rules (see [Migrate policies](./migrate.md)), so that Egret generates it. | | *engine \ runs \, which does not ship the Trino OPA plugin; supported: trinodb/trino:483* | Move the engine to `trinodb/trino:483` in the Control plane, or bind it to another profile. | From 721aa5d4f972d8875743e699b6f8c823b7fb6c57 Mon Sep 17 00:00:00 2001 From: 1Pro-L4R8 Date: Sun, 8 Nov 2026 18:54:43 +0700 Subject: [PATCH 4/6] docs(opa): note shared tag service rebuild latency Co-Authored-By: Claude Opus 5.5 --- platform/data-governance/engines-and-policy/trino-on-opa.md | 1 + 1 file changed, 1 insertion(+) diff --git a/platform/data-governance/engines-and-policy/trino-on-opa.md b/platform/data-governance/engines-and-policy/trino-on-opa.md index bcefbd9..959dfdd 100644 --- a/platform/data-governance/engines-and-policy/trino-on-opa.md +++ b/platform/data-governance/engines-and-policy/trino-on-opa.md @@ -282,6 +282,7 @@ A value outside its range stops Egret at startup, for example *governance.opa.ba - **Body deadline.** An admitted request must deliver its whole body within `body_read_timeout_ms`, or it gets 408. 5 s covers a 24 MiB body at about 40 Mbit/s; a 64 MiB body needs about 110 Mbit/s. Raise it on a slow link. - **In flight and parsing.** Egret counts bodies in 4 MiB units: at most 64 units (256 MiB) in flight before authentication, and 16 units (64 MiB) being parsed at once. A request reserves its declared `Content-Length`, or its route's whole limit when it declares none. A request that cannot get its units within 1 s gets 503 `{"error": "busy"}`, and its body is never read. A larger `batch_body_limit_bytes` lets one body take more of the same budget; it does not raise the memory bound. - **Rebuild delay.** Snapshots are rebuilt within about 1 s of a policy change. +- **Shared tag service.** After a policy or tag change, the OPA surface keeps answering from the previous snapshot until each affected profile is rebuilt. Profiles rebuild one after another on the 1 s refresher, so when many profiles share one tag service, a tag change reaches the last of them after roughly N rebuilds. On a loaded dev machine the last profile became current about 0.4 s after the change with 5 profiles sharing one tag service, about 1 s with 10 and about 3.5 s with 20, and the refresher's 1 s period comes on top. The time also grows with the total number of grants and rules on the installation. - **Other gaps.** An external Trino 483 engine on its own compiled profile cannot run any query: it gets no infrastructure rows (KI-354). The mask `{col}` placeholder is substituted unquoted, as with Ranger. StarRocks has no OPA mechanism. ## Further reading From a066574cf7c485e2d106163f6bebd3d57028526c Mon Sep 17 00:00:00 2001 From: 1Pro-L4R8 Date: Sun, 8 Nov 2026 19:26:49 +0700 Subject: [PATCH 5/6] docs(opa): Helm opens the OPA port only with ranger.enabled Co-Authored-By: Claude Opus 5.5 --- .../data-governance/engines-and-policy/trino-on-opa.md | 4 ++-- platform/deploy-operate/kubernetes.md | 8 ++++---- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/platform/data-governance/engines-and-policy/trino-on-opa.md b/platform/data-governance/engines-and-policy/trino-on-opa.md index 959dfdd..4e7d00b 100644 --- a/platform/data-governance/engines-and-policy/trino-on-opa.md +++ b/platform/data-governance/engines-and-policy/trino-on-opa.md @@ -31,9 +31,9 @@ enabled = false EGRET_GOVERNANCE__OPA__ENABLED=false ``` -On Kubernetes, set the Helm value `opa.enabled=false`. The chart sets `EGRET_GOVERNANCE__OPA__ENABLED` from `opa.enabled`. +On Kubernetes, set the Helm value `opa.enabled=false`. The chart always sets `EGRET_GOVERNANCE__OPA__ENABLED` from `opa.enabled`. With `ranger.enabled` off (the default) the chart opens no OPA port, and Egret logs an info line at start, *OPA surface idle: ranger.enabled is off*, not an error. -Egret serves the OPA routes on their own port, **8082** by default, not on the API port 8080 (see [Network exposure](#network-exposure)). On Kubernetes, the Helm value `opa.enabled` (default `true`) sets both keys of the OPA listener for you (see [Kubernetes](../../deploy-operate/kubernetes.md)). +Egret serves the OPA routes on their own port, **8082** by default, not on the API port 8080 (see [Network exposure](#network-exposure)). On Kubernetes, the Helm value `opa.enabled` (default `true`) sets the key and, when `ranger.enabled` is also true, the OPA listener, port and NetworkPolicy rule for you (see [Kubernetes](../../deploy-operate/kubernetes.md)). It also needs `ranger.enabled = true`, because the OPA routes check the engine credentials of the Ranger surface (see [Engines and Ranger setup](../../deploy-operate/engines-and-ranger.md)). With the Ranger surface off, OPA does not run and a switch to OPA is refused with the same message as a disabled gate. diff --git a/platform/deploy-operate/kubernetes.md b/platform/deploy-operate/kubernetes.md index f0044d9..25e7607 100644 --- a/platform/deploy-operate/kubernetes.md +++ b/platform/deploy-operate/kubernetes.md @@ -25,7 +25,7 @@ With default values `helm template` renders these objects: | Object | Purpose | |---|---| | `Deployment/egret` | The Egret server. Probes: startup `GET /healthz`, readiness `GET /readyz`, liveness `GET /healthz`. Requests 500m CPU and 1 GiB, limits 2 CPU and 4 GiB. `terminationGracePeriodSeconds` is 65, which must stay above the server's 55 second drain. | -| `Service/egret` | Port 8080 (8081 for Data Apps when `apps.origin` is set, and the port named `opa` while `opa.enabled` is on, which is the default). | +| `Service/egret` | Port 8080 (8081 for Data Apps when `apps.origin` is set, and the port named `opa` while `opa.enabled` and `ranger.enabled` are both on; `opa.enabled` is on by default, `ranger.enabled` is not). | | `Deployment/egret-frontend`, `Service/egret-frontend` | The built web app behind nginx, as a NodePort Service (30080). | | `Deployment/egret-renderer`, `Service/egret-renderer` | The screenshot service used by reports. | | `Job/egret-init` | A post-install and post-upgrade hook that runs `egret init`, creating the administrator and the `default` workspace. It is idempotent. | @@ -182,9 +182,9 @@ Running a script task also needs the workspace feature `workflow_containers` (of | `engineNetworkPolicy.operatorNamespaces` | `[starrocks]` | Namespaces of the StarRocks operator, let in on the front end's 8030 and 9030. | | `engineNetworkPolicy.ingressFrom` | `[]` | NetworkPolicy peers an engine Ingress forwards from (your ingress controller), let in on 8030 (StarRocks front end) and 8080 and 8443 (Trino). **Required when an engine has an Ingress**, or the Ingress gets no traffic. | | `engineNetworkPolicy.extraFrom` | `[]` | Other NetworkPolicy peers, let in on every port, for example a metrics scraper. | -| `opa.enabled` | `true` | Sets `EGRET_GOVERNANCE__OPA__ENABLED=true` (the chart sets the variable from `opa.enabled`) and `EGRET_GOVERNANCE__OPA__LISTEN=0.0.0.0:`, and adds the `opa` port to the Deployment and the `Service/egret`. Profiles stay on Ranger until an operator switches them. It still needs `ranger.enabled`, which is off by default; without it nothing binds the OPA port. Set `opa.enabled=false` to turn OPA off. See [Trino on OPA](../data-governance/engines-and-policy/trino-on-opa.md). | -| `opa.port` | `8082` | The port of Egret's OPA listener. Egret derives the URL engines use from `catalog.public_url` with this port, so no other value is needed while that URL is Egret's in-cluster address. | -| `opa.extraFrom` | `[]` | NetworkPolicy peers beyond `engineNamespace` that may reach the OPA port, for example the namespace of an external Trino on OPA. | +| `opa.enabled` | `true` | Always sets `EGRET_GOVERNANCE__OPA__ENABLED` from this value. Only when `ranger.enabled` is also true, the chart also sets `EGRET_GOVERNANCE__OPA__LISTEN=0.0.0.0:`, adds the `opa` port to the Deployment and the `Service/egret`, and renders the OPA rule of the NetworkPolicy. Profiles stay on Ranger until an operator switches them. It needs `ranger.enabled`, which is off by default: with the defaults the chart opens no OPA port, and Egret logs an info line at start (*OPA surface idle: ranger.enabled is off*), not an error. Without it nothing binds the OPA port. Set `opa.enabled=false` to turn OPA off. See [Trino on OPA](../data-governance/engines-and-policy/trino-on-opa.md). | +| `opa.port` | `8082` | The port of Egret's OPA listener, used only when `opa.enabled` and `ranger.enabled` are both true. Egret derives the URL engines use from `catalog.public_url` with this port, so no other value is needed while that URL is Egret's in-cluster address. | +| `opa.extraFrom` | `[]` | NetworkPolicy peers beyond `engineNamespace` that may reach the OPA port (rendered only when `opa.enabled` and `ranger.enabled` are both true), for example the namespace of an external Trino on OPA. | | `apps.origin`, `apps.egretOrigin` | empty | The Data Apps origin (a sibling subdomain such as `apps.`) and Egret's own origin. Empty turns the apps iframe and its listener off. | | `apps.pod.launcher` | `none` | `kubernetes` starts one hardened pod per author per workspace in `apps.pod.namespace` (`egret-apps`). Also `image`, `imagePullPolicy`, `resources`, `readyTimeoutSecs`, `idleSecs`, `networkPolicy.enabled` and `run.*`. | From 0c69677ebe2243d25871ef64d630cbb797cacf03 Mon Sep 17 00:00:00 2001 From: 1Pro-L4R8 Date: Sun, 8 Nov 2026 19:32:22 +0700 Subject: [PATCH 6/6] docs(opa): upgrade start-up failures, new refusal text, both-keys wording Co-Authored-By: Claude Opus 5.5 --- platform/data-governance/engines-and-policy/trino-on-opa.md | 6 ++---- platform/deploy-operate/kubernetes.md | 4 ++-- platform/deploy-operate/upgrade.md | 1 + platform/reference/configuration-keys.md | 2 +- 4 files changed, 6 insertions(+), 7 deletions(-) diff --git a/platform/data-governance/engines-and-policy/trino-on-opa.md b/platform/data-governance/engines-and-policy/trino-on-opa.md index 4e7d00b..37f22f9 100644 --- a/platform/data-governance/engines-and-policy/trino-on-opa.md +++ b/platform/data-governance/engines-and-policy/trino-on-opa.md @@ -35,9 +35,7 @@ On Kubernetes, set the Helm value `opa.enabled=false`. The chart always sets `EG Egret serves the OPA routes on their own port, **8082** by default, not on the API port 8080 (see [Network exposure](#network-exposure)). On Kubernetes, the Helm value `opa.enabled` (default `true`) sets the key and, when `ranger.enabled` is also true, the OPA listener, port and NetworkPolicy rule for you (see [Kubernetes](../../deploy-operate/kubernetes.md)). -It also needs `ranger.enabled = true`, because the OPA routes check the engine credentials of the Ranger surface (see [Engines and Ranger setup](../../deploy-operate/engines-and-ranger.md)). With the Ranger surface off, OPA does not run and a switch to OPA is refused with the same message as a disabled gate. - -With the gate off, the OPA routes are not mounted (they answer 404), no snapshot refresher runs, and a switch to OPA is refused: *OPA is not enabled on this installation ([governance.opa] enabled)*. The keys that tune the request bodies are under [Limits](#limits); the listener's keys are under [Network exposure](#network-exposure). +`ranger.enabled` is needed because the OPA routes check the engine credentials of the Ranger surface (see [Engines and Ranger setup](../../deploy-operate/engines-and-ranger.md)). When either key is off, the OPA routes are not mounted (they answer 404), no snapshot refresher runs, and a switch to OPA is refused: *OPA is not running on this installation: it needs both [governance.opa] enabled and ranger.enabled*. The keys that tune the request bodies are under [Limits](#limits); the listener's keys are under [Network exposure](#network-exposure). **Turning the gate off while profiles are on OPA makes their engines fail every query**, because the routes unmount. Switch those profiles back to Ranger first. @@ -67,7 +65,7 @@ A refused switch lists every condition that holds, one message each, and writes | Message | What to do | |---|---| -| *OPA is not enabled on this installation ([governance.opa] enabled)* | Either the key was turned off or `ranger.enabled` (off by default) is off. Remove `enabled = false` (or set `EGRET_GOVERNANCE__OPA__ENABLED=true`, or Helm `opa.enabled=true`), keep `ranger.enabled`, then restart Egret. | +| *OPA is not running on this installation: it needs both [governance.opa] enabled and ranger.enabled* | Turn on `ranger.enabled` (it is off by default), and make sure `[governance.opa] enabled` is not `false` (or set `EGRET_GOVERNANCE__OPA__ENABLED=true`, or Helm `opa.enabled=true`), then restart Egret. | | *only a Trino profile can use OPA; this profile is of type \* | Nothing: StarRocks profiles stay on Ranger. | | *only a generated profile can use OPA: hand-written policies are Ranger JSON that OPA cannot evaluate* | Express the profile's access as grants and rules (see [Migrate policies](./migrate.md)), so that Egret generates it. | | *engine \ runs \, which does not ship the Trino OPA plugin; supported: trinodb/trino:483* | Move the engine to `trinodb/trino:483` in the Control plane, or bind it to another profile. | diff --git a/platform/deploy-operate/kubernetes.md b/platform/deploy-operate/kubernetes.md index 25e7607..16e0cdb 100644 --- a/platform/deploy-operate/kubernetes.md +++ b/platform/deploy-operate/kubernetes.md @@ -33,7 +33,7 @@ With default values `helm template` renders these objects: | `ConfigMap/egret-env` | Every `EGRET_*` setting that is not a secret. | | `ServiceAccount`, `Role` and `RoleBinding` `egret-engine-provisioner` | Lets Egret manage workloads in the engine namespace. | | `NetworkPolicy/egret-engine-isolation` | In the engine namespace: only that namespace's own pods, Egret's pods and the StarRocks operator (front end ports 8030 and 9030) may connect to the engines (since platform #828). See [Engine network policy](#engine-network-policy). | -| `NetworkPolicy/egret-ingress` | In the release namespace, selecting Egret's backend pods (`app: egret`): 8080 open to every peer, 8081 open when `apps.origin` is set, and the OPA port only from `engineNamespace` and `opa.extraFrom`. Rendered when `engineNetworkPolicy.enabled`. See [Engine network policy](#engine-network-policy). | +| `NetworkPolicy/egret-ingress` | In the release namespace, selecting Egret's backend pods (`app: egret`): 8080 open to every peer, 8081 open when `apps.origin` is set, and the OPA port, when `opa.enabled` and `ranger.enabled` are both true, only from `engineNamespace` and `opa.extraFrom`. Rendered when `engineNetworkPolicy.enabled`. See [Engine network policy](#engine-network-policy). | | `Deployment/postgres`, `PersistentVolumeClaim/postgres-data` (1 GiB), `Service/postgres` | Bundled PostgreSQL 16 (`deps.enabled`). | | `Deployment/rustfs`, `PersistentVolumeClaim/rustfs-data`, `Service/rustfs`, `Job/egret-rustfs-bucket` | Bundled RustFS and a job that creates the bucket. | | `Pod/mailpit`, `Service/mailpit` | A mail catcher so you can read report emails. Never for real use. | @@ -216,7 +216,7 @@ Measured on kind 0.29 with a listener on 8030, 9030, 8080 and 9010: before the p after it, a task pod and a non-Egret pod of the release namespace were refused on all four, the operator's namespace got 8030 and 9030 only, and an `ingressFrom` peer got 8030 and 8080 only. -`NetworkPolicy/egret-ingress` protects Egret's own pods the other way round. It selects the backend pods (`app: egret`) and admits 8080 from every peer (the frontend, ingresses, app and task pods and the kubelet use it), 8081 when `apps.origin` is set, and the OPA port, 8082 by default, **only** from `engineNamespace` and `opa.extraFrom`. The OPA surface reads a request body before it can authenticate it, so a pod that could reach that port could hold stalled requests and exhaust the budget engines need. The policy renders whenever `engineNetworkPolicy.enabled` is set, with or without `opa.enabled`. This describes what the chart renders; it has not been run on a live cluster. +`NetworkPolicy/egret-ingress` protects Egret's own pods the other way round. It selects the backend pods (`app: egret`) and admits 8080 from every peer (the frontend, ingresses, app and task pods and the kubelet use it), 8081 when `apps.origin` is set, and, when `opa.enabled` and `ranger.enabled` are both true, the OPA port, 8082 by default, **only** from `engineNamespace` and `opa.extraFrom`. The OPA surface reads a request body before it can authenticate it, so a pod that could reach that port could hold stalled requests and exhaust the budget engines need. The policy renders whenever `engineNetworkPolicy.enabled` is set, with or without `opa.enabled`. This describes what the chart renders; it has not been run on a live cluster. An external engine on OPA lives outside `engineNamespace`: add its namespace to `opa.extraFrom`, for example `- namespaceSelector: { matchLabels: { kubernetes.io/metadata.name: trino-prod } }`. Any firewall between the engines and Egret must allow the OPA port too. When `catalog.public_url` is not Egret's in-cluster address, set `[governance.opa] engine_url` (see [Trino on OPA](../data-governance/engines-and-policy/trino-on-opa.md#network-exposure)). diff --git a/platform/deploy-operate/upgrade.md b/platform/deploy-operate/upgrade.md index 245dee4..4aa89a3 100644 --- a/platform/deploy-operate/upgrade.md +++ b/platform/deploy-operate/upgrade.md @@ -673,6 +673,7 @@ default, and mounts the OPA surface with no configuration change, so a profile c [Trino on OPA](../data-governance/engines-and-policy/trino-on-opa.md). - **Turning it off:** `[governance.opa] enabled = false`, `EGRET_GOVERNANCE__OPA__ENABLED=false`, or Helm `opa.enabled=false`. The chart sets the variable from `opa.enabled`. Switch any profile on OPA back to Ranger first: with the key off, its engines fail every query. +- **Installs outside Helm with `ranger.enabled` on can fail to start after the upgrade, with no config change.** Egret now validates and binds `[governance.opa] listen` (default `0.0.0.0:8082`) and runs the 1 s snapshot refresher. Start-up stops, naming the key, when: port 8082 is already taken on the host; `server.port` or `apps.listen` (with `apps.origin` set) is 8082; a leftover malformed `[governance.opa] listen` or `engine_url`, ignored while OPA was off, is now checked; or the OPA engine URL cannot be derived from `catalog.public_url`. Fix it by setting `[governance.opa] listen` and `engine_url`, or `enabled = false`. Helm installs are not affected, because the chart sets the listen value. - **Network:** the chart's `NetworkPolicy/egret-ingress` (with `engineNetworkPolicy.enabled`) admits the OPA port only from `engineNamespace` and `opa.extraFrom`. Outside Kubernetes, keep port 8082 reachable from the engines only. - **Tag-service warning:** a Trino profile with no usable tag service now carries `no_tag_service_warning` on the `PATCH …/mechanism` answer and on each explain profile, and the console shows it. See [Trino on OPA](../data-governance/engines-and-policy/trino-on-opa.md#through-the-api). - **Known issues** on OPA engines: KI-352, KI-353 and KI-354 in [Known issues](../known-issues.md). diff --git a/platform/reference/configuration-keys.md b/platform/reference/configuration-keys.md index 8296594..843c081 100644 --- a/platform/reference/configuration-keys.md +++ b/platform/reference/configuration-keys.md @@ -159,7 +159,7 @@ The S3-compatible object store used for the catalog warehouse, query results, wo | `enabled` | `EGRET_GOVERNANCE__OPA__ENABLED` | `true` | Serve the OPA decision routes for Trino, on their own listener (`listen`), and allow a policy profile to switch to OPA. On by default; profiles stay on Ranger until an operator switches them. Set `false` to turn it off. Needs `ranger.enabled`. See [Trino on OPA](../data-governance/engines-and-policy/trino-on-opa.md). | | `batch_body_limit_bytes` | `EGRET_GOVERNANCE__OPA__BATCH_BODY_LIMIT_BYTES` | `25165824` (24 MiB) | Largest request body the OPA `/batch` route accepts. Trino sends a listing of a whole schema as one batch; a larger body is answered 413 and the query fails. From 4194304 (4 MiB) to 67108864 (64 MiB); every other OPA route keeps 4 MiB. | | `body_read_timeout_ms` | `EGRET_GOVERNANCE__OPA__BODY_READ_TIMEOUT_MS` | `5000` | How long an admitted OPA request may take to send its whole body before it is answered 408 and its connection closed. From 1 to 60000. | -| `listen` | `EGRET_GOVERNANCE__OPA__LISTEN` | `0.0.0.0:8082` | Where the OPA listener binds, `host:port`. It serves only the OPA routes; port 8080 answers 404 for `/opa/`. With `enabled`, Egret refuses to start when the port equals `server.port` or, with `apps.origin` set, the apps listener's port; with OPA off it is not checked. | +| `listen` | `EGRET_GOVERNANCE__OPA__LISTEN` | `0.0.0.0:8082` | Where the OPA listener binds, `host:port`. It serves only the OPA routes; port 8080 answers 404 for `/opa/`. Used only when `enabled` and `ranger.enabled` are both on. Then Egret refuses to start when the port equals `server.port` or, with `apps.origin` set, the apps listener's port; otherwise it is not checked. | | `engine_url` | `EGRET_GOVERNANCE__OPA__ENGINE_URL` | unset | Base URL engines use for OPA. Unset: `catalog.public_url` (or the Docker callback URL; `http://egret:8080` if unset) with its port replaced by the port of `listen`. Set it when `public_url` is not Egret's in-cluster address, for example an ingress hostname. With `enabled`, it must be an absolute `http(s)` URL with a host. | ## engine