Skip to content

Commit 1063f1c

Browse files
waleedlatif1claude
waleedlatif1
and
claude
committed
docs(self-hosting): add per-cloud reference architectures
"Do you have CloudFormation or Terraform templates?" is a recurring enterprise question, and the honest answer needs a page: Sim ships a Helm chart, and the chart is what your own IaC should call. Nothing documented where the boundary sits, so the answer was being reconstructed per conversation. The page covers what the chart provisions by default versus what production needs you to bring, the prerequisites for EKS, AKS and GKE, how the three secret strategies map onto what your pipeline has to create, and how to wrap the chart in a helm_release. Deliberately not IaC of our own. The one strong vendor precedent, GitLab's Environment Toolkit, lives in a separate repository and is framed as-is; comparable products ship a chart and docs and leave the modules to the community. Maintaining Terraform across three clouds is a standing cost that buys less than documenting the boundary precisely. Two things the page is careful about. The Terraform snippet overrides every secret, because the example values files ship literal placeholders and the chart only rejects empty values and its own CHANGE-ME strings — inheriting them installs cleanly with a publicly known session-signing secret. And it states that existingSecret and External Secrets fail in opposite directions: the first silently, since app.env is inlined onto the pod spec and shadows the Secret, the second loudly, by refusing to render. Sizing, the secret inventory and the pre-launch checklist are cross-references rather than copies, so there is one place for each to drift. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015BwsJTEQRzWJaY4BRCkPZt
1 parent 4b1605b commit 1063f1c

3 files changed

Lines changed: 169 additions & 0 deletions

File tree

‎apps/docs/content/docs/platform/self-hosting/meta.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
"index",
55
"architecture",
66
"---Install---",
7+
"reference-architectures",
78
"docker",
89
"kubernetes",
910
"platforms",

‎apps/docs/content/docs/platform/self-hosting/reference-architectures.mdx‎

Lines changed: 167 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,167 @@
1+
---
2+
title: Reference Architectures
3+
description: What you provision and what the chart provisions, per cloud, before you run helm install
4+
---
5+
6+
import { Tab, Tabs } from 'fumadocs-ui/components/tabs'
7+
import { Callout } from 'fumadocs-ui/components/callout'
8+
9+
Sim ships a Helm chart, not infrastructure-as-code. You bring a Kubernetes cluster and, in production, managed Postgres and object storage; the chart installs Sim on top. This page draws that line precisely so you can map it onto whatever Terraform, Bicep, or CloudFormation you already run, and decide what to provision before you reach [Kubernetes](/platform/self-hosting/kubernetes).
10+
11+
<Callout type="info">
12+
There is no official Terraform module or CloudFormation template. The chart is the deployment interface, and it is what your own IaC should call — `helm_release` in Terraform, or a Helm task in your pipeline. Everything below describes the inputs that chart expects.
13+
</Callout>
14+
15+
## The boundary
16+
17+
Everything the chart can run itself, it runs by default. That is convenient for evaluation and wrong for production, because two of those defaults keep state.
18+
19+
| Component | Chart default | Production |
20+
|---|---|---|
21+
| App, realtime, migrations | Deployed | Deployed by the chart |
22+
| Scheduled jobs (CronJobs) | Deployed | Deployed by the chart |
23+
| PostgreSQL | Deployed in-cluster | **Replace** with managed Postgres — set `externalDatabase.*` and `postgresql.enabled: false` |
24+
| Redis | Deployed in-cluster | Replace with a managed cache, or keep the bundled one — it holds no committed data |
25+
| Object storage | None — local disk | **Required.** S3, Azure Blob, or GCS. Local disk is lost when a container is recreated and is not shared across replicas |
26+
| Ingress | Off | You install the controller; the chart renders the Ingress |
27+
| TLS certificates | Off | You provision them |
28+
| Remote sandbox | Off | Required for Python, Shell, and imported JavaScript — a provider **and** an immutable Function image. See [Security](/platform/self-hosting/security) |
29+
| PII redaction, Ollama, telemetry collector | Off | Optional, off unless you enable them |
30+
31+
The bundled Postgres uses a 10 Gi `ReadWriteOnce` volume and is intended for evaluation. Moving to managed Postgres is the single most important production change.
32+
33+
## Sizing
34+
35+
| Component | Requests | Limits |
36+
|---|---|---|
37+
| app | 4 Gi / 1000m | 8 Gi / 2000m |
38+
| realtime | 512 Mi / 250m | 1 Gi / 500m |
39+
| PostgreSQL (bundled) | 1 Gi / 500m | 2 Gi (no CPU limit) |
40+
41+
These are the chart defaults. Memory is the constraint rather than CPU, and the app's real ceiling depends on your execution load — see [Scaling](/platform/self-hosting/scaling) for the observed steady and peak figures, when to raise `app.resources.limits.memory`, and how to size replicas and connection pooling. Size nodes so a single app pod fits with headroom rather than packing them.
42+
43+
## Per-cloud requirements
44+
45+
<Tabs items={['AWS (EKS)', 'Azure (AKS)', 'GCP (GKE)']}>
46+
<Tab value="AWS (EKS)">
47+
48+
Use `helm/sim/examples/values-aws.yaml` as a starting point. It configures EBS GP3 storage, ALB ingress with AWS Certificate Manager, IRSA-style ServiceAccount annotations, and GPU tolerations for Ollama. Ollama itself ships disabled, and its node selectors are present but commented out.
49+
50+
**You provision**
51+
52+
| Resource | Notes |
53+
|---|---|
54+
| EKS cluster | Kubernetes 1.25 or newer |
55+
| EBS CSI driver add-on | `aws eks create-addon --addon-name aws-ebs-csi-driver`. Without it, PersistentVolumeClaims never bind |
56+
| AWS Load Balancer Controller | Required for ALB ingress |
57+
| ACM certificate | ALB discovers it from the ingress hosts, or pin it with `alb.ingress.kubernetes.io/certificate-arn` |
58+
| RDS PostgreSQL | With the `vector` extension available. Reachable from the cluster subnets |
59+
| S3 buckets | One per file purpose — see [Object storage](/platform/self-hosting/object-storage) |
60+
| ElastiCache | Optional; the bundled Redis is a supported alternative |
61+
62+
<Callout type="warn">
63+
cert-manager does not work with ALB. ALB cannot serve a Kubernetes TLS Secret, so certificates must come from ACM. This is the most common first-deploy failure on EKS.
64+
</Callout>
65+
66+
**Identity.** Bind an IAM role to the ServiceAccount with IRSA rather than issuing static keys. The role needs S3 access to the buckets you configured; grant nothing else unless you have enabled a feature that needs it.
67+
68+
**Timeouts.** ALB's default idle timeout will cut long-running executions and websockets. See [Networking](/platform/self-hosting/networking) before you go live.
69+
70+
</Tab>
71+
<Tab value="Azure (AKS)">
72+
73+
Use `helm/sim/examples/values-azure.yaml` as a starting point. It configures `managed-csi` / `managed-csi-premium` storage, NGINX ingress, role-based node targeting, and NVIDIA GPU node pool tolerations for Ollama.
74+
75+
**You provision**
76+
77+
| Resource | Notes |
78+
|---|---|
79+
| AKS cluster | Kubernetes 1.25 or newer |
80+
| NGINX ingress controller | The example values assume it |
81+
| cert-manager with a ClusterIssuer | Issues the ingress TLS secret |
82+
| Azure Database for PostgreSQL | With `pgvector` enabled |
83+
| Azure Blob Storage containers | One per file purpose |
84+
| Azure Cache for Redis | Optional |
85+
| GPU node pool | Only for Ollama. Taint it `sku=gpu:NoSchedule` to match the example tolerations |
86+
87+
**Identity.** Use a workload identity bound to the ServiceAccount rather than a storage account key in `app.env`.
88+
89+
</Tab>
90+
<Tab value="GCP (GKE)">
91+
92+
Use `helm/sim/examples/values-gcp.yaml` as a starting point. It configures Persistent Disk storage, Google Cloud Load Balancer with managed certificates, Workload Identity annotations, and GPU tolerations for Ollama. Ollama itself ships disabled, and the T4 node selectors are present but commented out.
93+
94+
**You provision**
95+
96+
| Resource | Notes |
97+
|---|---|
98+
| GKE cluster | Kubernetes 1.25 or newer |
99+
| Workload Identity | Must be enabled for the IAM-bound ServiceAccount |
100+
| ManagedCertificate | Created before the first deploy — nothing in the chart creates it. The example values reference `simstudio-ssl-cert`; match whatever name your ingress annotation uses |
101+
| Cloud SQL for PostgreSQL | With `pgvector` |
102+
| GCS buckets | One per file purpose. Every purpose-specific bucket falls back to the general one |
103+
| Memorystore | Optional |
104+
105+
<Callout type="warn">
106+
With a ManagedCertificate, keep `ingress.tls.enabled: false`. GCLB, like ALB, cannot serve a Kubernetes TLS Secret — leaving chart TLS on points the ingress at a Secret nothing creates, and the GKE controller reports sync errors for it. cert-manager only applies if you replace the `gce` ingress with an in-cluster controller such as NGINX.
107+
</Callout>
108+
109+
**Identity.** Workload Identity binds the ServiceAccount to a Google service account. Grant it object access to your buckets only.
110+
111+
**Timeouts.** GCLB's default backend timeout closes websockets every 30 seconds. See [Networking](/platform/self-hosting/networking) for the `BackendConfig` fix.
112+
113+
</Tab>
114+
</Tabs>
115+
116+
## Secrets
117+
118+
The chart offers three strategies. Pick one before you write any IaC, because it decides what your pipeline has to create.
119+
120+
| Strategy | What your IaC creates | When |
121+
|---|---|---|
122+
| Inline (`app.env`, or `--set`) | Nothing | Evaluation. Values land in your values file, your shell history, and `helm get values` |
123+
| `app.secrets.existingSecret.enabled` + `.name` | A Kubernetes Secret carrying **every** key the app needs, under the standard key names — it is consumed wholesale via `envFrom`, so key remapping is not supported | You already manage secrets out of band |
124+
| External Secrets Operator | Entries in AWS Secrets Manager, Azure Key Vault, or GCP Secret Manager, plus `externalSecrets.remoteRefs.app.<KEY>` mappings | Most production deployments |
125+
126+
The two non-inline modes fail in opposite directions, which is worth knowing before you pick one:
127+
128+
- **`existingSecret` fails quietly.** The chart renders no Secret of its own, so a key absent from yours and from `app.env` is simply empty at runtime. A key you also set in `app.env` is inlined onto the pod spec, where it shadows the Secret — so the Secret is not always the source of truth.
129+
- **External Secrets fails loudly.** The chart refuses to render when a key is set in `app.env` but not mapped, on the grounds that a silently empty secret is worse than a failed install.
130+
131+
See [Security](/platform/self-hosting/security) for the full secret inventory, which keys are unrecoverable if lost, and what to back up.
132+
133+
## Calling the chart from Terraform
134+
135+
If you already run Terraform, the chart is the resource to wrap — not something to reimplement. It is not published to a Helm repository or an OCI registry, so there is no `repository` to point at: vendor this repo as a submodule, a release tarball, or a `git clone` in your pipeline, and give `chart` the local path.
136+
137+
```hcl
138+
resource "helm_release" "sim" {
139+
name = "sim"
140+
namespace = "sim"
141+
create_namespace = true
142+
143+
# Local path, not a repository. Pin the git ref you vendor from.
144+
chart = "${path.module}/sim/helm/sim"
145+
146+
# Your own values file. The examples under helm/sim/examples/ carry
147+
# placeholder secrets and are starting points, not deployable as-is.
148+
values = [file("${path.module}/values-aws.yaml")]
149+
150+
# helm provider v3 syntax. On v2 each of these is a set_sensitive { } block.
151+
set_sensitive = [
152+
{ name = "app.env.BETTER_AUTH_SECRET", value = var.better_auth_secret },
153+
{ name = "app.env.ENCRYPTION_KEY", value = var.encryption_key },
154+
{ name = "app.env.API_ENCRYPTION_KEY", value = var.api_encryption_key },
155+
{ name = "app.env.INTERNAL_API_SECRET", value = var.internal_api_secret },
156+
{ name = "app.env.CRON_SECRET", value = var.cron_secret },
157+
]
158+
}
159+
```
160+
161+
<Callout type="warn">
162+
The example values files ship literal placeholders such as `your-secure-production-auth-secret-here`. The chart only rejects empty values and its own `CHANGE-ME` strings, so a deployment that inherits those placeholders installs cleanly with a publicly known session-signing secret. Override every secret, or use External Secrets and set none of them inline.
163+
</Callout>
164+
165+
Because the chart is local, `version` does nothing — what pins it is the git ref you vendor from, and `helm/sim/Chart.yaml` tells you which chart release that ref carries. Pin that ref, and pin the image tags separately, or an unplanned `terraform apply` can move Sim to a new release with new migrations. See [Upgrades](/platform/self-hosting/upgrades).
166+
167+
Once the infrastructure exists, follow [Kubernetes](/platform/self-hosting/kubernetes) for the install itself, then the [pre-launch checklist](/platform/self-hosting/security) and the [verification checklist](/platform/self-hosting/verify).

‎apps/sim/lib/copilot/generated/docs-manifest.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -392,6 +392,7 @@ export const DOCS_MANIFEST: readonly string[] = [
392392
'platform/self-hosting/observability.mdx',
393393
'platform/self-hosting/platforms.mdx',
394394
'platform/self-hosting/redis.mdx',
395+
'platform/self-hosting/reference-architectures.mdx',
395396
'platform/self-hosting/scaling.mdx',
396397
'platform/self-hosting/security.mdx',
397398
'platform/self-hosting/troubleshooting.mdx',

0 commit comments

Comments
 (0)