docs(self-hosting): add sandboxes and reference architectures, and correct the existing pages

An audit of 27 pages checked every concrete claim against the code it describes.
It found claims that made a documented path fail outright, and claims that
understated what ships.

Errors that broke a documented path:

- `openssl rand -hex 32` prints 64 hex characters, not the "32 hex chars" five
  entries claimed, and only two of those secrets are length-validated at all.
- `API_ENCRYPTION_KEY` was missing from the whole Kubernetes path. It is
  optional, and unset means API keys are stored in plain text behind one warning.
- Bucket fallback behaviour was wrong for all three object-storage providers, in
  both directions.
- The remote-sandbox table omitted the immutable template and snapshot refs, so
  the documented configuration could not work.
- The GKE snippet set a values key no template renders.
- Retention deletion is scheduled by neither shipped deployment, so a deployment
  that followed the docs deleted nothing.
- Every "Settings → Enterprise/Security → X" path named nav groups that do not
  exist, across eleven pages.

Claims that understated what ships:

- The outbound-proxy section said no platform traffic can be proxied. The server
  runs on Bun, whose fetch honors the proxy variables, so provider and Resend
  traffic already proxies; the guarded egress, SMTP, S3 and OTLP do not. Replaced
  with a per-path table and the NO_PROXY guidance internal services need.
- Telemetry is off by default on Helm, not on.
- The CronJob count was 18 in two places; there are 22.
- The permission-group table was stale throughout, named a deployment key that
  does not exist, and inverted the checkbox semantic.

Adds two pages. Sandboxes carries the base-image runbook that was buried in an
enterprise enablement page. Reference architectures answers what an operator
provisions versus what the chart provisions, per cloud — the question behind "do
you have Terraform or CloudFormation templates?".

Also documents around 25 environment variables nothing described, the migration
advisory lock and its operational knobs, and `sim-setup doctor`. Removes
redundancy where a page restated itself: FAQ blocks repeating their own body, and
four drifting copies of the same tables.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015BwsJTEQRzWJaY4BRCkPZt

Author: waleedlatif1

apps/docs/content/docs/platform/enterprise/access-control.mdx

@@ -14,7 +14,7 @@ Audit logs give your organization a tamper-evident record of every significant a
 
 ### In the UI
 
-Go to **Settings → Enterprise → Audit Logs** in your workspace. Logs are displayed in a table with the following columns:
+Go to **Settings → Organization → Audit logs** in your workspace. Logs are displayed in a table with the following columns:
 
 <Image src="/static/enterprise/audit-logs.png" alt="Audit Logs settings showing a table of events with columns for Timestamp, Event, Description, and Actor, along with search and filter controls" width={900} height={830} />
 
@@ -149,4 +149,13 @@ AUDIT_LOGS_ENABLED=true
 NEXT_PUBLIC_AUDIT_LOGS_ENABLED=true
 ```
 
-Once enabled, audit logs are viewable in **Settings → Enterprise → Audit Logs** and accessible via the API.
+Once enabled, audit logs are viewable in **Settings → Organization → Audit logs** and accessible via the API.
+
+`GET /api/v1/audit-logs` authenticates with an `x-api-key` whose owner is an admin or owner of an organization, so it is unreachable on a deployment where nobody belongs to one yet. On Sim Cloud it additionally requires an active Enterprise subscription; self-hosted, `AUDIT_LOGS_ENABLED` takes that role. The admin-key equivalent needs neither an organization nor a plan:
+
+```http
+GET /api/v1/admin/audit-logs
+x-admin-key: <admin-api-key>
+```
+
+It accepts the same filters plus `actorEmail`, drops `includeDeparted`, and paginates with `limit` (max 250) and `offset` instead of the organization endpoint's `limit` (max 100) and `cursor`. It returns entries across the whole deployment rather than one organization. `GET /api/v1/admin/audit-logs/<id>` returns a single entry. Set `ADMIN_API_KEY` to use it — see the [self-hosted enterprise guide](/platform/enterprise/self-hosted).

apps/docs/content/docs/platform/enterprise/audit-logs.mdx

@@ -14,7 +14,7 @@ A custom block always runs the **latest deployed version** of its source workflo
 
 ## Common uses
 
-Custom blocks turn a workflow one team owns into infrastructure the whole organization can safely reuse. Credentials and complexity stay with the block's author; everyone else gets a clean block that's always up to date. A few patterns:
+The block's author keeps the credentials and the workflow logic; by default consumers see only the inputs and outputs, unless the author enables **Trace runs in consumer logs**. Common patterns:
 
 - **Internal API gateway.** Wrap an authenticated internal or partner endpoint — "Create Ticket", "Charge Account", "Provision User" — behind a block that takes only the business inputs. Teammates call it without the base URL, API key, or auth headers, and when the endpoint changes you update one workflow instead of every consumer's.
 - **Blessed knowledge lookup.** Package a vetted retrieval pipeline — chunking, filters, reranking — as "Search Company Docs" with a single query input, so teams reuse the approved retrieval instead of each rebuilding it.
@@ -35,7 +35,7 @@ Custom blocks turn a workflow one team owns into infrastructure the whole organi
 
 ### 1. Open Custom blocks settings
 
-Go to **Settings → Enterprise → Custom blocks** and click **Create block**.
+Go to **Settings → Organization → Custom blocks** and click **Create block**.
 
 <Image src="/static/enterprise/custom-blocks-list.png" alt="Custom blocks settings page listing a published block with its icon, name, and description, with a Create block button in the header" width={900} height={570} />
 
@@ -90,19 +90,19 @@ Click **Save changes**. The block is published immediately and becomes available
 
 ## Using a custom block
 
-In the workflow editor, open the block toolbar. Published custom blocks appear under a **Custom blocks** section. Drag one onto the canvas like any other block, fill in its inputs (using the placeholders as a guide), and reference its outputs in downstream blocks.
+In the workflow editor, open the block toolbar. Published custom blocks appear under a **Custom blocks** section. Drag one into your workflow like any other block, fill in its inputs (using the placeholders as a guide), and reference its outputs in downstream blocks.
 
 <Image src="/static/enterprise/custom-blocks-toolbar.png" alt="Workflow editor block toolbar with a Custom Blocks section listing two published blocks below Core Blocks" width={400} height={476} />
 
 Consumers don't need any access to the source workflow. The block runs on its own, using only the inputs provided, and returns only the outputs you exposed. Its internal steps, models, and intermediate values stay hidden unless the block's publisher turned on **Trace runs in consumer logs**, in which case they appear under the block in the run's trace.
 
-<Image src="/static/enterprise/custom-blocks-canvas.png" alt="A custom block connected to a Start block on the workflow canvas, with its query input filled in and the run output showing the returned fields" width={900} height={570} />
+<Image src="/static/enterprise/custom-blocks-canvas.png" alt="A custom block connected to a Start block in the workflow editor, with its query input filled in and the run output showing the returned fields" width={900} height={570} />
 
 ---
 
 ## Managing blocks
 
-Open a block from **Settings → Enterprise → Custom blocks** to edit or delete it.
+Open a block from **Settings → Organization → Custom blocks** to edit or delete it.
 
 - **Editing** changes only the block's presentation, interface, and trace policy — name, description, icon, input placeholders, exposed outputs, and whether runs are traced in consumer logs. The source workflow can't be re-pointed.
 - **Changing what the block does** is done by editing and **redeploying the source workflow**. The block picks up the new deployment automatically; there's nothing to republish.
@@ -131,7 +131,7 @@ Open a block from **Settings → Enterprise → Custom blocks** to edit or delet
   },
   {
     question: "Can consumers see the workflow behind a block?",
-    answer: "No. A custom block only exposes the inputs it needs and the outputs you chose to share. The source workflow, its steps, and its intermediate values are never visible, and consumers don't need any access to it."
+    answer: "A custom block only exposes the inputs it needs and the outputs you chose to share, and consumers don't need any access to the source workflow. Its steps and intermediate values stay hidden unless you enable Trace runs in consumer logs, which surfaces them in the consumer's run trace."
   },
   {
     question: "Can I change which workflow a block points to?",

apps/docs/content/docs/platform/enterprise/custom-blocks.mdx

@@ -4,6 +4,7 @@ description: Continuously export workflow logs, audit logs, and Chat data to you
 ---
 
 import { FAQ } from '@/components/ui/faq'
+import { Image } from '@/components/ui/image'
 
 Data Drains let organization owners and admins on Enterprise plans continuously export Sim data to a destination they control — a customer-owned S3 bucket, Google Cloud Storage bucket, Azure Blob container, BigQuery table, Snowflake table, Datadog logs intake, or an HTTPS webhook. A drain runs on a schedule, picks up only new rows since its last successful run, and writes them to the destination. Viewing drain configuration and run history is restricted to owners and admins as well, since destinations expose internal bucket names, table identifiers, and webhook URLs.
 
@@ -13,11 +14,11 @@ Drains are independent of [Data Retention](/platform/enterprise/data-retention)
 
 ## Setup
 
-Go to **Settings → Enterprise → Data Drains** in your workspace, then click **New drain**.
+Go to **Settings → Organization → Data drains** in your workspace, then click **New drain**.
 
-![Data Drains settings page showing two configured drains — one exporting workflow logs to Amazon S3 daily, another exporting Copilot chats to an HTTPS webhook hourly](/static/enterprise/data-drains-list.png)
+<Image src="/static/enterprise/data-drains-list.png" alt="Data Drains settings page showing two configured drains — one exporting workflow logs to Amazon S3 daily, another exporting Chat conversations to an HTTPS webhook hourly" width={900} height={885} />
 
-![New data drain dialog with fields for name, source, cadence, destination, and S3 credentials](/static/enterprise/data-drains-new.png)
+<Image src="/static/enterprise/data-drains-new.png" alt="New data drain dialog with fields for name, source, cadence, destination, and S3 credentials" width={900} height={491} />
 
 Each drain has four pieces:
 
@@ -186,7 +187,7 @@ The **last 10 runs** for each drain are visible by expanding its row in the sett
 
 ## Pairing with Data Retention
 
-Drains and [Data Retention](/platform/enterprise/data-retention) are independent modules. Sim does **not** gate retention on drain progress — if a drain is failing, retention will still purge data on its own schedule. This matches the model used by Datadog Archives and AWS CloudWatch + S3 Export: keep the two configurations orthogonal and let the customer pair them deliberately.
+Drains and [Data Retention](/platform/enterprise/data-retention) are independent modules. Sim does **not** gate retention on drain progress — if a drain is failing, retention will still purge data on its own schedule. Keep the two configurations independent and pair them deliberately.
 
 To safely use both together, set the drain cadence shorter than the retention period for the same data category:
 
@@ -207,10 +208,6 @@ After data lands in your bucket or webhook system, archive lifecycle (transition
     question: "Who can configure data drains?",
     answer: "Only organization owners and admins can view, create, edit, run, or delete drains. On Sim Cloud, the organization must be on an Enterprise plan."
   },
-  {
-    question: "Will drained data be duplicated if a run fails?",
-    answer: "The drain cursor only advances on overall success, so a failure replays the same chunks on the next run. Every row has a stable `id` field and every webhook chunk has an `Idempotency-Key` header so receivers can dedupe."
-  },
   {
     question: "Can I export multiple sources to the same destination?",
     answer: "Yes — create one drain per source, all pointing at the same bucket or endpoint. S3 destinations namespace by source automatically; webhook receivers can branch on the `X-Sim-Source` header."
@@ -240,6 +237,29 @@ DATA_DRAINS_ENABLED=true
 NEXT_PUBLIC_DATA_DRAINS_ENABLED=true
 ```
 
-`NEXT_PUBLIC_DATA_DRAINS_ENABLED` shows the **Settings → Enterprise → Data Drains** page in the UI. `DATA_DRAINS_ENABLED` gates the server-side mutating endpoints and the cron dispatcher — when unset on a self-hosted deployment, drain create/update/delete/run requests return `404` and the dispatcher is a no-op. Both should be set to `true` together.
+`NEXT_PUBLIC_DATA_DRAINS_ENABLED` shows the **Settings → Organization → Data drains** page in the UI. `DATA_DRAINS_ENABLED` gates the server-side mutating endpoints and the cron dispatcher — when unset on a self-hosted deployment, drain create/update/delete/run requests return `404` and the dispatcher is a no-op. Both should be set to `true` together.
+
+### Scheduling the dispatcher
+
+The dispatcher is an HTTP endpoint, not a self-scheduling job — something has to call it:
+
+```
+GET /api/cron/run-data-drains
+```
+
+It authenticates with a bearer token equal to `CRON_SECRET` and returns `401` when that variable is unset, so a self-hosted deployment must set it:
+
+```bash
+openssl rand -hex 32
+```
+
+Set the same value as `CRON_SECRET` on both the app and whatever invokes the endpoint. Generating it in your shell does not configure either one.
+
+Both shipped deployments schedule this endpoint hourly for you — Helm through `cronjobs.jobs.runDataDrains`, Docker Compose through its `cron` service. A deployment that runs neither schedules it itself:
+
+```bash
+curl -H "Authorization: Bearer $CRON_SECRET" \
+  https://sim.example.com/api/cron/run-data-drains
+```
 
-Data Drains otherwise rely on the standard Trigger.dev background job infrastructure used elsewhere in Sim — no additional setup is required. The cron dispatcher runs hourly and fans out due drains as background jobs.
+Each due drain is then fanned out as a `run-data-drain` background job, so the deployment also needs `TRIGGER_DEV_ENABLED` and a configured Trigger.dev project. Without it the dispatcher still claims the work and enqueues it to the database, but nothing drains that queue for this job type, so the runs stay pending and never execute. See [background jobs](/platform/self-hosting/background-jobs).

apps/docs/content/docs/platform/enterprise/data-drains.mdx

@@ -18,7 +18,7 @@ Both are configured once at the **organization level** and apply to every worksp
 
 ## Setup
 
-Go to **Settings → Enterprise → Data Retention** in your workspace.
+Go to **Settings → Organization → Data retention** in your workspace.
 
 <Image src="/static/enterprise/data-retention.png" alt="Data Retention settings showing the Retention policies list with the Organization default row and its summary of retention periods and PII stages" width={900} height={537} />
 
@@ -97,12 +97,12 @@ The **Workflow input** and **Block outputs** stages alter what the workflow comp
 
 For each stage, choose the **entity types** to redact from the searchable grid. They are grouped as:
 
-- **Common** — person name, email, phone, credit card, IP address, URL, IBAN, crypto wallet, medical license, VIN
+- **Common** — person name, email, phone, credit card, IP address, location, date or time, URL, IBAN, crypto wallet, nationality/religious/political group, medical license, VIN
 - **United States** — SSN, passport, driver's license, bank account, ITIN
 - **United Kingdom** — NHS number, National Insurance number
-- **Other regions** — Singapore, Australian, and Indian identifiers
+- **Other regions** — Spanish (NIF, NIE), Italian (fiscal code, driver's licence, VAT code, passport, identity card), Polish (PESEL), Singaporean (NRIC/FIN, UEN), Australian (ABN, ACN, TFN, Medicare), Indian (PAN, Aadhaar, vehicle registration, voter ID, passport), and Finnish (personal identity code) identifiers
 
-The **Block outputs** stage is restricted to regex- and checksum-based recognizers, so it can run in-flight over large payloads without a performance penalty. Types that need name-model detection — person name, location, date or time — are not offered for that stage.
+The **Block outputs** stage is restricted to regex- and checksum-based recognizers, so it can run in-flight over large payloads without a performance penalty. Types that need name-model detection — person name, location, date or time, and nationality/religious/political group — are not offered for that stage.
 
 Detection is language-aware: pick the **language** whose recognizers should apply. English, Spanish, Italian, Polish, and Finnish are supported, and the grid filters to the identifiers available for the selected language.
 
@@ -167,7 +167,7 @@ By default, retention settings are unconfigured — no data is automatically del
   },
   {
     question: "What is the maximum retention period?",
-    answer: "5 years."
+    answer: "5 years, or Forever — the same option list applies to all three settings."
   }
 ]} />
 
@@ -182,15 +182,41 @@ NEXT_PUBLIC_DATA_RETENTION_ENABLED=true
 DATA_RETENTION_ENABLED=true
 ```
 
-Once enabled, retention settings are configurable through **Settings → Enterprise → Data Retention** the same way as Sim Cloud.
+Once enabled, retention settings are configurable through **Settings → Organization → Data retention** the same way as Sim Cloud.
+
+### Scheduling the deletion pass
+
+`DATA_RETENTION_ENABLED` permits deletion; it does not perform it. Deletion runs when a scheduled request reaches one of three endpoints, each authenticated with a bearer token equal to `CRON_SECRET`:
+
+| Category | Endpoint |
+|----------|----------|
+| Execution and job logs | `GET /api/logs/cleanup` |
+| Soft-deleted resources | `GET /api/cron/cleanup-soft-deletes` |
+| Chats and Chat runs | `GET /api/cron/cleanup-tasks` |
+
+<Callout type="warn">
+Neither shipped deployment schedules these three endpoints — not the Helm chart, not Docker Compose's `cron` service. An operator who sets `DATA_RETENTION_ENABLED=true` alone still deletes nothing. Add them to `cronjobs.jobs`, or call them daily from an external scheduler.
+</Callout>
+
+```bash
+# Use the value already configured as CRON_SECRET on the app — a token
+# generated here and not installed there returns 401.
+curl -H "Authorization: Bearer $CRON_SECRET" \
+  https://sim.example.com/api/logs/cleanup
+```
+
+Each call fans the work out as background jobs. Trigger.dev is not required: when it is not configured the dispatcher runs the cleanup chunks inline instead of enqueuing them, so deletion completes on a default self-host.
 
 ### PII redaction
 
 PII redaction runs against a standalone [Presidio](https://microsoft.github.io/presidio/) service. Deploy it (see `apps/pii`) and point Sim at it:
 
 ```bash
 # The Presidio service exposing /analyze and /anonymize
-PII_URL=http://localhost:5001
+# Helm — substitute your release name and namespace
+PII_URL=http://<release>-pii.<namespace>.svc.cluster.local:5001
+# Docker Compose — the PII service name on your network
+# PII_URL=http://pii:5001
 ```
 
-All PII stages are configurable under **Settings → Enterprise → Data Retention**.
+All PII stages are configurable under **Settings → Organization → Data retention**.