Skip to content

feat(supply-chain): sign published images, add a chart image inventory, and correct the self-hosting docs - #7432

Closed
waleedlatif1 wants to merge 14 commits into
stagingfrom
feat/supply-chain-attestations
Closed

feat(supply-chain): sign published images, add a chart image inventory, and correct the self-hosting docs#7432
waleedlatif1 wants to merge 14 commits into
stagingfrom
feat/supply-chain-attestations

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • Sign every published GHCR image with cosign keyless signing, and attach SLSA provenance and a Syft SBOM as registry referrers. Runs after create-ghcr-manifests because buildx's own attestations break the imagetools create retagging the deploy gate depends on.
  • Add helm/sim/images.yaml — the complete set of images a full install pulls, generated from the rendered chart and drift-checked in the helm workflow. Derived from the render rather than values.yaml because one image (the NVIDIA device plugin) is written directly into a template and no values key names it.
  • Correct the self-hosting and enterprise docs against the source. An audit of 27 pages found claims that made documented paths fail outright, and claims that understated what ships.

What was wrong

  • openssl rand -hex 32 prints 64 hex characters, not the "32 hex chars" five entries claimed.
  • API_ENCRYPTION_KEY was missing from the whole Kubernetes path and marked required elsewhere. It is optional, and unset means API keys are stored in plain text behind one warning.
  • Bucket fallback behavior was wrong for all three object-storage providers, in both directions.
  • The outbound-proxy section said no platform traffic can be proxied. The server runs on Bun, whose fetch honors the proxy variables — replaced with a verified per-path table.
  • Telemetry is off by default on Helm, not on, and enabling the collector collects nothing until NEXT_TELEMETRY_DISABLED is cleared.
  • The CronJob count was 18 in two places; there are 22.
  • Every Settings → Enterprise/Security → X path named nav groups that do not exist, across eleven pages.
  • The permission-group table was stale throughout, named a deployment key that does not exist, and inverted the checkbox semantic.
  • Retention deletion is scheduled by nothing in the chart, so a deployment that followed the docs deleted nothing.

Also documents ~25 environment variables nothing described, the migration advisory lock and its operational knobs, and sim-setup doctor. Moves the sandbox base-image runbook out of the enterprise enablement page into its own self-hosting page rather than losing it. Removes FAQ blocks that restated their own page body and four drifting copies of the same tables.

Type of Change

  • New feature (image signing + inventory)
  • Documentation

Testing

bun run check:audits (45 audits), bun run lint, bun run docs-manifest:check, bun run images:check, actionlint, and the script suite (271 tests) all pass. Chart image list verified against helm template. Every action pinned by SHA and resolved against the GitHub API.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

🤖 Generated with Claude Code

https://claude.ai/code/session_015BwsJTEQRzWJaY4BRCkPZt

@waleedlatif1
waleedlatif1 requested a review from a team as a code owner September 3, 2026 06:09
@vercel

vercel Bot commented Sep 3, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Sep 3, 2026 4:26pm UTC

Request Review

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@greptile-apps

greptile-apps Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR adds keyless signing, provenance, and architecture-specific SBOM publication for GHCR images, introduces a rendered Helm image inventory with CI drift checks, and updates self-hosting and enterprise documentation.

  • Resolves published image tags into a dynamic digest matrix before signing and attestation.
  • Adds a generated inventory covering images rendered by the Helm chart.
  • Makes the GPU device-plugin image configurable through the chart’s shared image helper.
  • Revises deployment, security, storage, telemetry, and enterprise documentation.

Confidence Score: 2/5

The PR does not appear safe to merge until the disconnected-registry path mismatch and the successful-stale-read gap in latest-image attestation are fixed.

The inventory directs operators to mirror Sim-owned images beneath <registry>/ghcr.io/simstudioai/* while the chart pulls <registry>/simstudioai/*, and the attestation resolver still skips newly published latest subjects when GHCR successfully returns an older digest.

Files Needing Attention: scripts/generate-image-manifest.ts, helm/sim/images.yaml, helm/sim/templates/_helpers.tpl, .github/workflows/ci.yml

Important Files Changed

Filename Overview
.github/workflows/ci.yml Adds digest-based subject discovery and a matrix that signs images and publishes provenance and per-architecture SBOM attestations.
scripts/generate-image-manifest.ts Renders the Helm chart, extracts unique container references, and generates or verifies the checked-in image inventory.
helm/sim/templates/gpu-device-plugin.yaml Moves the NVIDIA device-plugin image through the shared configurable image-rendering helper.
helm/sim/values.yaml Adds configurable device-plugin repository, tag, pull policy, and digest values.
helm/sim/images.yaml Records the complete rendered image set for disconnected-install preparation.
.github/workflows/helm.yml Adds the generated image-inventory drift check to Helm CI and expands workflow path triggers.

Sequence Diagram

sequenceDiagram
  participant Build as Architecture builds
  participant Publish as create-ghcr-manifests
  participant Resolve as attest-subjects
  participant Attest as attest-images
  participant Release as create-release
  Build->>Publish: Immutable architecture tags
  Publish->>Publish: Create SHA, version, and eligible latest indexes
  Publish->>Resolve: Publication job succeeds
  Resolve->>Resolve: Resolve tags and deduplicate digests
  Resolve->>Attest: Dynamic subject matrix
  Attest->>Attest: Generate per-architecture SBOM
  Attest->>Attest: Publish provenance and signatures
  Attest->>Release: Successful attestation gate
Loading

Reviews (10): Last reviewed commit: "fix(supply-chain): scan the plain manife..." | Re-trigger Greptile

Comment thread .github/workflows/ci.yml Outdated
Comment thread .github/workflows/ci.yml
Comment thread scripts/generate-image-manifest.ts
Comment thread scripts/generate-image-manifest.ts Outdated
Comment thread apps/docs/content/docs/platform/self-hosting/docker.mdx

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 43 files

Re-trigger cubic

Comment thread apps/docs/content/docs/platform/self-hosting/security.mdx Outdated
Comment thread apps/docs/content/docs/platform/enterprise/data-drains.mdx Outdated
Comment thread .github/workflows/ci.yml Outdated
Comment thread helm/sim/images.yaml Outdated
Comment thread apps/docs/content/docs/platform/self-hosting/upgrades.mdx
Comment thread apps/docs/content/docs/platform/self-hosting/verify.mdx Outdated
Comment thread apps/docs/content/docs/platform/enterprise/whitelabeling.mdx
Comment thread apps/docs/content/docs/platform/enterprise/whitelabeling.mdx
Comment thread apps/docs/content/docs/platform/enterprise/data-retention.mdx Outdated
Comment thread apps/docs/content/docs/platform/self-hosting/upgrades.mdx Outdated
@waleedlatif1
waleedlatif1 force-pushed the feat/supply-chain-attestations branch from d1e8fee to 4820548 Compare September 3, 2026 07:06
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

Comment thread .github/workflows/ci.yml Outdated
Comment thread scripts/generate-image-manifest.ts Outdated
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 43 files

Re-trigger cubic

Comment thread apps/docs/content/docs/platform/enterprise/sso.mdx
Comment thread .github/workflows/ci.yml Outdated
Comment thread apps/docs/content/docs/platform/enterprise/custom-blocks.mdx Outdated
Comment thread apps/docs/content/docs/platform/self-hosting/architecture.mdx Outdated
Comment thread apps/docs/content/docs/platform/self-hosting/troubleshooting.mdx Outdated
Comment thread apps/docs/content/docs/platform/self-hosting/upgrades.mdx Outdated
Comment thread apps/docs/content/docs/platform/enterprise/index.mdx
Comment thread apps/docs/content/docs/platform/enterprise/index.mdx
Comment thread apps/docs/content/docs/platform/enterprise/self-hosted.mdx Outdated
Comment thread apps/docs/content/docs/platform/enterprise/data-drains.mdx Outdated
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

Comment thread .github/workflows/ci.yml Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 46 files

Re-trigger cubic

Comment thread .github/workflows/ci.yml Outdated
Comment thread apps/docs/content/docs/platform/enterprise/access-control.mdx Outdated
Comment thread apps/docs/content/docs/platform/enterprise/data-drains.mdx Outdated
Comment thread apps/docs/content/docs/platform/enterprise/data-drains.mdx Outdated
Comment thread apps/docs/content/docs/platform/self-hosting/troubleshooting.mdx Outdated
Comment thread .github/workflows/ci.yml Outdated
Comment thread apps/docs/content/docs/platform/enterprise/self-hosted.mdx Outdated
Comment thread apps/docs/content/docs/platform/enterprise/self-hosted.mdx Outdated
Comment thread apps/docs/content/docs/platform/self-hosting/security.mdx
Comment thread apps/docs/content/docs/platform/self-hosting/security.mdx
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

Comment thread scripts/generate-image-manifest.ts Outdated
Comment thread .github/workflows/ci.yml Outdated
waleedlatif1 and others added 13 commits September 3, 2026 09:08
…e inventory

Enterprise security reviews ask for three artifacts Sim did not publish: a
signature proving who built an image, SLSA provenance describing how, and an
SBOM listing what is inside. Add all three, plus the image inventory an
operator needs to mirror Sim into a disconnected registry.

attest-images runs after create-ghcr-manifests rather than inside the build.
buildx's own provenance/sbom attestations stay off because the extra manifests
they add to an index break the imagetools retagging promote-images depends on;
attaching attestations to the finished index leaves it untouched, since they
are stored as separate referrer manifests. The subject is the sha index digest
— imagetools create is deterministic, so the version and latest indexes built
from the same per-arch manifests share that digest and one attestation covers
every tag a customer can pull.

The digest comes from `{{json .Manifest}}` piped through jq rather than
`{{.Manifest.Digest}}`: the latter resolves the index to the runner's own
platform, which would attest one architecture and leave the pulled index
unsigned.

helm/sim/images.yaml is derived from the rendered chart, not from values.yaml,
because one image is written directly into a template — the NVIDIA device
plugin — and a values-derived list misses it in the case that is hardest to
notice, where the mirror succeeds and one pod still pulls from the internet.
Tags stay unresolved: digests belong to a release, and the chart's sim.image
helper already accepts a per-image digest for pinning at install time.

The check runs in the helm workflow, which has Helm set up; check:audits does
not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015BwsJTEQRzWJaY4BRCkPZt
An audit of the 27 self-hosting and enterprise pages checked every concrete
claim against the code it describes. This fixes what it found.

Errors that made a documented path fail:

- `openssl rand -hex 32` prints 64 hex characters, not the "32 hex chars" five
  entries claimed. Only ENCRYPTION_KEY and API_ENCRYPTION_KEY are length-
  validated; the rest are free-form minimums, so the fix differs per variable.
- API_ENCRYPTION_KEY was absent from the whole Kubernetes path and listed as
  required elsewhere. It is optional, and unset means API keys are stored in
  plain text behind one warning — a silent failure documented nowhere.
- The per-purpose S3 fallback was described backwards: knowledge-base, chat,
  copilot and profile-picture buckets resolve to an empty string, not a literal.
  The GCS table carried the mirror-image claim, and Azure had no warning at all.
- Local uploads go to /app/uploads, not /uploads, and no compose file or chart
  mounts a volume there.
- The remote-sandbox table omitted the immutable template and snapshot refs, so
  the documented configuration could not work.
- The GKE BackendConfig snippet set a values key no template renders.
- Retention deletion is scheduled by nothing in the chart, so a deployment that
  followed the docs deleted nothing.
- Every "Settings → Enterprise/Security → X" path named nav groups that do not
  exist, across eleven enterprise pages.

Claims that understated what ships:

- The outbound-proxy section said no platform traffic can be proxied. The server
  runs on Bun, whose fetch honors the proxy variables, so provider SDK and
  Resend traffic already proxies; guarded egress, SMTP, object storage and OTLP
  do not. Replaced with a per-path table, including the constraint that the
  per-request proxyUrl refuses a private address.
- Telemetry is off by default on Helm, not on, and enabling the collector
  collects nothing until NEXT_TELEMETRY_DISABLED is cleared.
- The CronJob count was 18 in two places; there are 22 in both the chart and the
  crontab.
- The permission-group table was stale throughout, named a deployment key that
  does not exist, and inverted the checkbox semantic. About fifteen governed
  toggles were undocumented.

Also documents ~25 environment variables nothing described, the migration
advisory lock and its operational knobs, the CloudWatch metrics that activate on
any deployment with AWS credentials, and sim-setup doctor.

Redundancy removed where a page restated itself: FAQ blocks that repeated their
own body, four drifting copies of the same tables, and three cloud tabs
identical but for a filename.

The sandbox base-image runbook moves from the enterprise enablement page, where
it was out of place, to its own self-hosting page rather than being lost.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015BwsJTEQRzWJaY4BRCkPZt
A second round of verification checked every added and removed line against
source. It found errors the first pass introduced, which matter more than the
ones it fixed — a wrong new claim is worse than the stale one it replaced.

Corrections to the corrections:

- The GCS bucket table was inverted. Every GCS context falls back to
  GCS_BUCKET_NAME; the rewrite claimed none did. The pre-existing text was
  right and was replaced with a falsehood.
- S3 and Azure each have three fallback exceptions, not one: og-images and
  workspace-logos fall back to the general bucket, and execution-files falls
  back to a literal.
- The proxy table had four wrong rows. Azure OpenAI, Azure Anthropic and vLLM
  are pinned only when the endpoint is user-supplied, so an operator-configured
  endpoint does proxy. Azure Blob and GCS honor the proxy variables; S3 and
  Bedrock do not. SES is the AWS SDK over HTTPS, not raw TCP. Adds the on-prem
  hazard that matters most here: Ollama, LiteLLM and vLLM are dialed over the
  default fetch, so NO_PROXY is required.
- The CloudWatch warning described a bill that cannot happen — every hosted-key
  metric call site is gated on the deployment being Sim Cloud.
- networkPolicy.egressExceptCidrs: [] keeps the metadata defaults rather than
  removing them, since Helm's `default` treats an empty list as unset. Verified
  by rendering the chart. The security guidance said the opposite.
- The migrator retries far more than two error classes, so a database that is
  simply down backs off rather than failing immediately.
- /api/v1/audit-logs authenticates with an API key, not a session, and the
  Enterprise-plan requirement does not apply self-hosted — the exact deployment
  the new paragraph addressed.
- The local upload path is /app/apps/sim/uploads: the standalone server chdirs
  into its own directory.
- The update feed answers 200, not 302. KB_OPENAI_MODEL_NAME is optional.
  OCR_PROVIDER inference is current behavior, not a legacy path.

Restores six things the cleanup dropped that existed nowhere else: version-skip
guidance, the workspace and last-member rejection rules, the gemma3:4b starter
model, kubectl top for OOMKill diagnosis, the grandfathered-workspace case, and
an explicit note that Compose telemetry is on by default.

Also resolves contradictions only visible across pages: the sandbox
prerequisites disagreed on three pages, and authentication.mdx kept a settings
path the enterprise pages had corrected.

The image inventory drops chartVersion. It made every chart-version bump fail a
check whose message says an image changed, and appVersion already pins the tags.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015BwsJTEQRzWJaY4BRCkPZt
helm/sim/** changed, so the chart-version gate requires a SemVer bump. Patch:
images.yaml is an added inventory file, with no template or values change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015BwsJTEQRzWJaY4BRCkPZt
…eview

Review found that the attestation covered only the multi-arch index while
create-ghcr-manifests also publishes single-arch tags (:version-amd64,
:latest-arm64, and so on) that resolve to different digests. Those were
pullable but unsigned, so verification failed for anyone who pinned one.

The matrix now attests all three subjects per image — the index and both
per-arch manifests. That also makes the SBOMs truthful: the amd64 and arm64
images contain different packages, and one SBOM on the index cannot describe
both. create-release now requires attest-images, so a release can no longer
advertise images whose signing failed.

The helm workflow's path filter now includes the inventory generator, which
could previously change without the check that validates it ever running.

The inventory header named global.imageRegistry alone as the redirect. That
only rewrites the simstudioai/* images; third-party ones also need
global.useRegistryForAllImages, and the NVIDIA device plugin is hardcoded in a
template and cannot be redirected by any values key at all.

Docs corrections from the same round:

- The copilot workloads cannot be covered by networkPolicy.egress — it is only
  rendered into policies the chart creates, and it creates none for them.
- Data drains and retention cleanup need Trigger.dev. Without it the work is
  claimed and enqueued to the database, but nothing drains that job type.
- MIGRATION_DATABASE_URL has no chart values key: the migrations init container
  builds DATABASE_URL itself and reads only the database Secret.
- Finished CronJob Jobs are deleted after ten minutes by ttlSecondsAfterFinished,
  so the history limits do not mean what the page implied.
- lock_timeout is 5s for ordinary DDL, but a migration can set it to 0 for
  statements like CREATE INDEX CONCURRENTLY.
- Losing Redis interrupts in-flight execution streams and cancellation, which
  "drops live updates and nothing else" understated.
- NEXT_PUBLIC_BRAND_BACKGROUND_COLOR sets no background; it only tells Sim
  whether the brand ground is dark.
- Pi uses its own image, not the Function base.
- doctor skips the Postgres, migration, and Redis probes on the production
  Compose layout, and does not shape-check optional secrets.
- The Daytona snapshot build needs the E2B parity manifest to exist first.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015BwsJTEQRzWJaY4BRCkPZt
Reading the final state of all 34 pages as prose, rather than diffing them,
surfaced seams that per-claim review cannot see.

The worst was mine: an earlier replacement produced "with a configured
Trigger.dev project, and a configured Trigger.dev project" in three files.

Also fixed:

- A dangling colon in troubleshooting where the list it introduced had been
  removed, and a lead-in promising two deployment options above a list of three.
- "Before the UI walkthrough" on a section that now sits after it, and a
  reference to "steps 3 and 4 below" that are above.
- A rollback snippet whose comment said to set the previous tag but whose
  commands never did, and an LM Studio section telling operators to recreate a
  service using the development compose file.
- A scaling page that recommended the 8 Gi default two lines above warning that
  peaks reach 12 GB.
- An ADFS example setting SSO_ISSUER to Sim's own URL, which the callout four
  lines earlier says it must not be.
- Table cells that were sentence fragments, prose glued onto the end of an
  <Image> tag, and a comma splice.
- Paragraphs restating an earlier paragraph on the same page verbatim, in
  access-control, self-hosted, whitelabeling, object-storage and email.
- "canvas" used for the visual builder in five places, competitor name-dropping,
  and two marketing-shaped passages on operator pages.
- Title Case headings in troubleshooting, the only page not using sentence case.
- type="warning" normalized to type="warn". Both resolve identically in
  fumadocs, so this is consistency rather than a rendering fix.

Every in-scope link and anchor still resolves after the heading renames.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015BwsJTEQRzWJaY4BRCkPZt
…mage mirrorable

Review was right on both blocking points, and both needed code rather than
wording.

`imagetools create` always writes an index, even from a single manifest — so
`:<version>-amd64` is a single-entry index whose digest differs from the
`:<sha>-amd64` manifest it wraps. Verified against a local registry: pushing a
manifest at bd44eb13 and retagging it produced e857a74f. Attesting the manifest
therefore left the tag people actually pin unverifiable, and a static matrix
could not fix it: resolving the live tags for one image returns five distinct
digests, and which of them exist depends on whether the run is a release and
whether the latest guard passed.

A new attest-subjects job resolves the published tags to digests, de-duplicates,
and feeds attest-images as a dynamic matrix. Tags this run did not publish are
skipped rather than failing it.

The NVIDIA device plugin was hardcoded in the template, so the previous
guidance — mirror it to the same path, or patch the DaemonSet — was not durable:
copying to another registry does not redirect the pull, and a patch is reverted
by the next helm upgrade. It now takes its image from
ollama.gpu.devicePlugin.image and goes through the shared helper, so it honors
global.imageRegistry with useRegistryForAllImages like every other third-party
image. Default renders byte-identical; chart minor bumped for the new key.

Docs corrections from the same round, several of them regressions from my own
previous commit:

- The ADFS example set SSO_ISSUER to the IdP identifier, and SSO_SAML_AUDIENCE
  defaults to SSO_ISSUER — so it silently broke audience validation. The example
  now sets both.
- Compose does not pass MIGRATION_DATABASE_URL through from .env: its migrations
  service declares an explicit environment list.
- CLI authentication requires Redis; it does not fall back, and redis.mdx
  already said so.
- The rollback snippet assigned SIM_VERSION in a subshell that Compose never saw.
- Custom blocks hide their internals only when tracing is off.
- A new workspace is still governed by the organization's default group.
- doctor is not a Kubernetes tool, so it cannot be the first step for Helm users.
- ENTERPRISE_ENABLED needs its NEXT_PUBLIC twin.
- The 100 MB response cap does not cover MCP's standalone SSE stream.
- Generating CRON_SECRET in a shell configures nothing by itself.

Renaming two headings broke published anchors that an in-repo library article
links to; both now point at the new slugs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015BwsJTEQRzWJaY4BRCkPZt
Retention's cleanup dispatcher runs its chunks inline when Trigger.dev is not
configured, so deletion completes on a default self-host. Only data drains have
no inline path. The previous wording told operators both would stay pending.

Docker Compose also schedules the drain endpoint hourly through its cron
service, so 'outside Helm, schedule it yourself' pointed operators at a
redundant scheduler.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015BwsJTEQRzWJaY4BRCkPZt
The resolver added `latest` unconditionally. When create-ghcr-manifests' guard
holds those tags back — it does that when the branch has moved on — they still
point at an older build, so the job would have put this run's signature and
provenance on an image it did not produce. A moving alias is now included only
when it resolves to the index this run published; the per-arch aliases ship in
the same guarded block, so one comparison gates all three.

`digest_of` also aborted the step instead of reporting an absent tag: under
`set -e` a failing inspect inside a command substitution takes the step down
before the caller can skip it. Confirmed against a real registry, then fixed
with an explicit `if !` rather than swallowing the status.

Docs, all verified against source:

- The General tab does not buffer everything: workspace scope, the default
  switch, and member changes apply immediately, so Discard does not undo them.
- Both data-drain screenshots were declared at the wrong aspect ratio when I
  converted them to <Image>; they rendered distorted.
- LM Studio needs VLLM_API_KEY when its API authentication is on.
- The rollback snippet hardcoded a version that does not exist.
- Docker Compose schedules the drain endpoint too, so the table implied Compose
  operators had to add their own.
- Generating CRON_SECRET in a shell configures neither the app nor the caller.
- SANDBOXES_ENABLED is required beyond the provider and Function base on a
  billing-free self-host, and Pi runs on its own image and fails closed without
  E2B_PI_TEMPLATE_ID or DAYTONA_PI_SNAPSHOT_ID.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015BwsJTEQRzWJaY4BRCkPZt
…ipping it

`digest_of` treated every inspect failure as "tag absent", so a transient
registry error would drop a published digest from the matrix while both jobs
still went green — an unsigned image that verification would later reject, with
nothing in the run to show for it. It now retries, reports absent only when the
registry says the manifest is unknown, and fails the step otherwise. The three
sha tags are required outright, and an alias group that is current must resolve
all three of its tags.

SBOMs no longer attach to index subjects. Syft resolves an index to a single
platform, so the SBOM described amd64 while the index also serves arm64. The
per-arch subjects each carry an accurate SBOM; the index keeps its signature and
provenance.

Docs:

- The member-directory restriction does not apply to organization owners and
  admins — the roster routes exempt those roles.
- Python's execution does not hinge on NEXT_PUBLIC_SANDBOXES_ENABLED; that flag
  gates the controls, the server-side provider and image decide whether it runs.
- The security checklist demanded sslMode: require on a Compose database that
  ships without TLS, so it could not be satisfied as written.
- CERT_HAS_EXPIRED is usually an expired certificate, an incomplete chain, or a
  drifted clock. Sending operators straight to NODE_EXTRA_CA_CERTS fixes none of
  those, and the section also said the same thing twice.
- Four endpoints, not "both".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015BwsJTEQRzWJaY4BRCkPZt
… for a missing one

digest_of accepted "manifest unknown" on the first attempt. GHCR can report a
just-published alias as unknown for a moment, and taking that at face value
would skip a tag this run did publish — leaving it unsigned while the workflow
went green, which is the failure the retry existed to prevent. Absence is now
retried like any other failure and only accepted when the registry still says so
on the final attempt.

Docs:

- The LM Studio steps recreated services with docker-compose.ollama.yml, a
  development stack a production install does not run. They now use the file
  that started the installation.
- Retention is not external work: its cleanup runs inline in the app process,
  and only data drains hand off to a background job. The endpoints start both,
  which is what the sentence should have said.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015BwsJTEQRzWJaY4BRCkPZt
… index

`latest` is allowed to lag — the monotonic guard holds it back when the branch
has moved on, and it then belongs to an older run that already attested it. A
version tag has no such carve-out: this run published it, so a mismatch now
fails rather than skipping, and a release can no longer ship a version image
nothing has attested.

The device-plugin mirror path was ambiguous. `sim.image` prepends the registry
to the repository verbatim, so the plugin lands at
`<registry>/nvcr.io/nvidia/k8s-device-plugin` — an operator who mirrored it
without the embedded host would get ImagePullBackOff. The inventory now states
the destination rule and offers the direct repository override for registries
that cannot nest that path.

Docs:

- doctor needs no root .env from a source checkout; that guidance was
  Compose-specific.
- The LM Studio commands hardcoded the production Compose file after the last
  round, which breaks an install started from the Ollama stack. They now take
  the file as a variable.
- NODE_EXTRA_CA_CERTS does fix an incomplete chain by supplying the missing
  intermediate; only expiry and clock drift are beyond it. The section had
  ruled it out for all three.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015BwsJTEQRzWJaY4BRCkPZt
…t all

The SBOM step could not have succeeded. Half the per-arch subjects are
single-entry indexes — `imagetools create` writes an index even from one
manifest — and Syft resolves an index against the runner's platform, so an
arm64-only index fails outright on an amd64 runner with "no child with platform
linux/arm64". That is 5 of 25 legs failing on every push to main, taking
attest-images red and, through its gate, preventing any release from being
created.

Scanning by the `<sha>-<arch>` tag fixes it: that is the plain manifest the
alias index wraps, so no platform resolution happens. It also halves the pulls,
since both subjects for an architecture now share one scan of identical content.

An empty subject list would also have opened the release gate — a matrix built
from an empty include produces no legs and still reports success — so the gate
now checks a published count rather than trusting a sibling job's guard.

Two smaller corrections: `digest_of` no longer sleeps after its final attempt,
which was costing 18 seconds per legitimately absent tag, and the alias
comment now says what the code proves (content identity with this run's index)
rather than implying provenance.

The release gate's comment also now says what it gates. CodePipeline fires from
the ECR tags that promote-images moves, upstream of attestation, and only the
GHCR mirrors are signed — so this gates the GitHub release, not the deploy.

Docs, each verified against source:

- Restricted MCP, custom-tool and skill types fail the block; they are not
  silently skipped, and the same page already said so 180 lines later.
- NO_PROXY needs every internal destination, not just model endpoints — the app
  calls realtime, Presidio, and itself over the same default fetch.
- The proxy table omitted Google/Gemini and every other OpenAI-compatible
  provider, and omitted Azure Communication Services from the email row.
- activeDeadlineSeconds caps a failing CronJob at five minutes, so it gets about
  one attempt rather than six retries.
- Docker Compose does not schedule the retention endpoints either; the callout
  blamed Helm alone while the table beside it was already right.
- Import-free JavaScript does leave the local VM when a block selects a sandbox
  or references a file path.
- The GCS CORS list named headers that do not exist; the real key is sent
  server-side and never by the browser.
- CRON_SECRET is optional, and nothing enforces the 32-character floor at
  runtime because env validation is skipped.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015BwsJTEQRzWJaY4BRCkPZt
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1
waleedlatif1 force-pushed the feat/supply-chain-attestations branch from f7ab61d to 2dc5476 Compare September 3, 2026 16:08
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found and verified against the latest diff

Confidence score: 3/5

  • In scripts/generate-image-manifest.ts, the fallback for registries that cannot nest nvcr.io can prepend the global registry twice, causing the GPU device-plugin image pull to fail; document nvidia/k8s-device-plugin as the required override.
Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="scripts/generate-image-manifest.ts">

<violation number="1" location="scripts/generate-image-manifest.ts:110">
P2: When an operator follows this fallback for a registry that cannot nest `nvcr.io`, the chart prepends the global registry again and the GPU device-plugin pull fails. Document `nvidia/k8s-device-plugin` as the override while global rewriting remains enabled, then regenerate `helm/sim/images.yaml`.</violation>
</file>

You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Fix all with cubic | Re-trigger cubic

Comment thread helm/sim/images.yaml Outdated
Comment thread .github/workflows/ci.yml Outdated
Comment thread scripts/generate-image-manifest.ts Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 46 files

Requires human review: Auto-approval blocked because this review re-detected 2 unresolved issues already reported by Cubic.

Re-trigger cubic

Comment thread apps/docs/content/docs/platform/enterprise/data-retention.mdx Outdated
Comment thread apps/docs/content/docs/platform/enterprise/access-control.mdx Outdated
Comment thread apps/docs/content/docs/platform/self-hosting/environment-variables.mdx Outdated
Comment thread apps/docs/content/docs/platform/self-hosting/networking.mdx
Comment thread apps/docs/content/docs/platform/self-hosting/sandboxes.mdx

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4 issues found across 46 files

Confidence score: 3/5

  • apps/docs/content/docs/platform/self-hosting/networking.mdx contains two operationally significant examples: the ingress snippet omits copilot.enabled: true, and the timeout command assumes the default Helm release name, so copied guidance can leave Copilot unrouted or preserve GKE's 30-second timeout. Update the examples to show enablement and resolve the realtime Service by Helm labels.
  • apps/docs/content/docs/platform/self-hosting/index.mdx inaccurately promises HTTP 422 when a custom sandbox lacks a configured provider; the route returns 503 before file handling, so revise the documented failure behavior.
  • apps/docs/content/docs/platform/self-hosting/networking.mdx uses contradictory wording about which providers bypass the proxy, leaving operators uncertain about model-call routing; explicitly identify the listed providers as exceptions or examples.
Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="apps/docs/content/docs/platform/self-hosting/index.mdx">

<violation number="1" location="apps/docs/content/docs/platform/self-hosting/index.mdx:116">
P2: When a user selects a custom sandbox without a configured provider, the function route returns HTTP 503 before file handling; this sentence incorrectly promises a 422 for that case. Say that sandbox selection fails with 503 and reserve 422 for file mounts or exports.</violation>
</file>

<file name="apps/docs/content/docs/platform/self-hosting/networking.mdx">

<violation number="1" location="apps/docs/content/docs/platform/self-hosting/networking.mdx:86">
P2: Copying this snippet without separately setting `copilot.enabled: true` renders no Copilot route, even though the example appears to configure one. State the required enablement alongside the ingress example.</violation>

<violation number="2" location="apps/docs/content/docs/platform/self-hosting/networking.mdx:228">
P2: When the chart is installed under any release name other than the documented default, this command annotates a nonexistent Service and GKE keeps the 30-second timeout. Resolve the realtime Service from its Helm labels or show the release-derived name instead of hard-coding `sim-realtime`.</violation>

<violation number="3" location="apps/docs/content/docs/platform/self-hosting/networking.mdx:310">
P2: The “except the three below” wording contradicts the provider list and the `Yes` result, so operators cannot tell which model calls use the proxy. Replace it with wording that identifies the listed providers as examples, or enumerate the actual exceptions explicitly.</violation>
</file>

Requires human review: Auto-approval blocked because this review re-detected 1 unresolved issue already reported by Cubic.
You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Re-trigger cubic

Comment thread apps/docs/content/docs/platform/self-hosting/background-jobs.mdx Outdated
Comment thread apps/docs/content/docs/platform/enterprise/access-control.mdx Outdated
Comment thread apps/docs/content/docs/platform/self-hosting/object-storage.mdx Outdated
Comment thread apps/docs/content/docs/platform/enterprise/data-retention.mdx Outdated
Comment thread apps/docs/content/docs/platform/self-hosting/sandboxes.mdx
Comment thread .github/workflows/ci.yml
Comment thread apps/docs/content/docs/platform/self-hosting/index.mdx
Comment thread apps/docs/content/docs/platform/self-hosting/networking.mdx
Both `ingress` and `ingressInternal` accept an optional `copilot` block, commented out in `values.yaml` because the `copilot` service is off by default. It follows the same host-sharing rule:

```yaml
ingress:

@cubic-dev-ai cubic-dev-ai Bot Sep 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Copying this snippet without separately setting copilot.enabled: true renders no Copilot route, even though the example appears to configure one. State the required enablement alongside the ingress example.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apps/docs/content/docs/platform/self-hosting/networking.mdx, line 86:

<comment>Copying this snippet without separately setting `copilot.enabled: true` renders no Copilot route, even though the example appears to configure one. State the required enablement alongside the ingress example.</comment>

<file context>
@@ -56,6 +55,42 @@ Both hostnames need DNS records and TLS certificates.
+Both `ingress` and `ingressInternal` accept an optional `copilot` block, commented out in `values.yaml` because the `copilot` service is off by default. It follows the same host-sharing rule:
+
+```yaml
+ingress:
+  copilot:
+    host: sim.yourdomain.com
</file context>
Suggested change
ingress:
copilot:
enabled: true
ingress:
Fix with cubic

annotations:
cloud.google.com/backend-config: '{"default": "sim-realtime-backendconfig"}'
```bash
kubectl annotate service sim-realtime -n simstudio \

@cubic-dev-ai cubic-dev-ai Bot Sep 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: When the chart is installed under any release name other than the documented default, this command annotates a nonexistent Service and GKE keeps the 30-second timeout. Resolve the realtime Service from its Helm labels or show the release-derived name instead of hard-coding sim-realtime.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At apps/docs/content/docs/platform/self-hosting/networking.mdx, line 228:

<comment>When the chart is installed under any release name other than the documented default, this command annotates a nonexistent Service and GKE keeps the 30-second timeout. Resolve the realtime Service from its Helm labels or show the release-derived name instead of hard-coding `sim-realtime`.</comment>

<file context>
@@ -187,24 +222,22 @@ spec:
-    annotations:
-      cloud.google.com/backend-config: '{"default": "sim-realtime-backendconfig"}'
+```bash
+kubectl annotate service sim-realtime -n simstudio \
+  cloud.google.com/backend-config='{"default": "sim-realtime-backendconfig"}'

</file context>


</details>

```suggestion
kubectl annotate service "$(kubectl get service -n simstudio -l app.kubernetes.io/component=realtime -o jsonpath='{.items[0].metadata.name}')" -n simstudio \
Fix with cubic

…e for it

The mirror instruction was wrong for the five first-party images. The inventory
lists what you pull — `ghcr.io/simstudioai/app` — but `ghcr.io` is the chart's
default registry, so a redirected install pulls `<registry>/simstudioai/app`.
Following "mirror to <registry>/<reference as listed>" put them at
`<registry>/ghcr.io/simstudioai/...` and every Sim image failed to pull. The
device plugin is the opposite case: its `nvcr.io/` really is part of the
repository and survives the rewrite.

Rather than write that rule out and hope it stays true, the generator now
renders the chart a second time against a sentinel registry and strips it back
off, so each entry carries both the source reference and the exact path the
chart resolves to. Verified by rendering with a real mirror and diffing: all 13
mirror paths match.

Also retries the moving-alias read. A mismatch meant "the guard held this tag
back", but it can equally mean GHCR is still serving the previous digest a
moment after this run wrote it — and that reading dropped three subjects from
the matrix without a word.

Docs, each verified against source:

- Chat drops a disallowed tool while generating a workflow and records the
  omission; a run that reaches one fails. Both statements were needed, and the
  page previously asserted one or the other in different places.
- A quickly-failing CronJob pod is retried six times by Kubernetes; the
  five-minute deadline only preempts one that hangs.
- Selecting a sandbox without a provider is a 503; the 422 is the file path.
- Chat's public code tool is run_function, not the internal function_execute.
- The parity step needs SANDBOX_PROVIDER=e2b, or an exported daytona provider
  makes step 1 exit before writing the manifest.
- PII_URL is auto-wired by the chart and needs a service URL on Compose.
- NO_PROXY should use the cluster suffix; the Service names depend on the
  release name.
- The bucket loops now fail loudly instead of silently iterating nothing when
  SIM_BUCKETS is unset in a fresh shell.
- Generating CRON_SECRET beside the curl suggested it was the token to use.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015BwsJTEQRzWJaY4BRCkPZt
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

Closing in favour of #7437, which combines this with the other self-hosting PR against current staging and folds in a pre-review audit pass. Everything here is carried over — nothing dropped.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant