Skip to content

fix(tls): distrust the roots behind the compromised eMudhra intermediates - #9248

Merged
waleedlatif1 merged 2 commits into
stagingfrom
fix/emudhra-distrust
Oct 20, 2026
Merged

waleedlatif1 merged 2 commits into
stagingfrom
fix/emudhra-distrust

Conversation

@waleedlatif1

@waleedlatif1 waleedlatif1 commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

The private keys of eight eMudhra intermediate CAs were reported compromised (disclosure). Browsers block them through CRLSet and OneCRL. Non-browser clients don't read those, so Node, Bun, curl and Python in our images still trusted any certificate issued under them. Each intermediate is reachable two ways: through emSign ECC Root CA - G3, or through Sectigo's AAA Certificate Services cross-sign.

  • @sim/security/tls-trust removes the four roots that the disclosure's roots.json lists from the process's default CA set. It runs first in the app and realtime bootstraps and in Trigger.dev init.
    • Mozilla's store already excludes all four, so the result matches what Firefox trusts today and doesn't block anything more.
    • The CA set is replaced only when one of those roots is present. Once a runtime ships its own fix, the call does nothing.
    • Certificates that fail to parse are kept, and any failure leaves trust as it was. The function never throws.
    • Under Bun, the set it rebuilds combines the system and extra roots with default. Bun's node:tls verifies against all of them, but Bun reports only its built-in list as default, so rebuilding from that alone would drop the OS roots.
  • App and cron images remove the same two roots from the OS store. A build-time grep matches a signature line of each root, so the build fails if the edit ever stops removing them.
  • Electron 43.5.0 → 43.8.1, which includes the upstream SPKI blocklist (fix: apply the certificate SPKI blocklist to Node and add entries (43-x-y) electron/electron#55130). The lockfile pins it, so bun install (frozen or not, cold cache) installs it without a minimumReleaseAgeExcludes entry and the 7-day gate stays intact.
  • If the CA set can't be replaced (unsupported runtime or an error), startup logs at error, so the fail-open state shows at the default production log level.

Verification

Run in Docker against all 12 disclosure test hosts (6 intermediates × 2 chains) plus 10 control endpoints: Google, GitHub, npm, OpenAI, Anthropic, Slack, Google OAuth, Microsoft login, Stripe, AWS STS.

Runtime / client Before After Controls
Node 24 (fetch + node:https) 6/12 trusted 0/12 10/10
Bun 1.4.2 node:https (pg, ioredis, AWS SDK, ws) 12/12 (Debian) 0/12 10/10
Bun 1.4.2 native fetch, app image 12/12 6/12 (AAA path closed) 10/10
curl / python3, app image 12/12 0/12 OK
curl, cron image 6/12 0/12 OK

Removing the sed line from either Dockerfile fails its build at the guard.

The trusted set loses only the compromised root: Bun 155 → 154 and Node 121 → 120, nothing added. A custom root passed with NODE_EXTRA_CA_CERTS, as used behind a corporate TLS proxy, is kept in both runtimes.

Not covered here

  • Bun native fetch uses its compiled-in root store and ignores tls.setDefaultCACertificates. The emSign ECC Root CA - G3 path stays open until a Bun release includes crypto: update root certificates oven-sh/bun#46159 (merged 2026-10-20, not released yet). Follow-up: bump Bun once that release is out.
  • The Oracle worker is a Node child process using Node's bundled store. It's covered when the Node release with tls: reject chains containing blocked public keys nodejs/node#66871 ships.
  • Realtime and migration images have no OS-store patch. On Alpine the only affected root also ships in Bun's built-in store, so patching the OS store there changes nothing.

Test plan

  • CI green
  • Staging deploy: app and realtime log Distrusted compromised CA roots at startup; integrations, OAuth, and webhooks behave normally
  • Desktop build on 43.8.1 launches and connects

…ates

The private keys of eight eMudhra intermediate CAs were reported compromised
(https://emudhra.we-have-a-problem.com). Browsers block them through
CRLSet/OneCRL, but Node, Bun, and the OS stores in our images still trusted
chains through them via emSign ECC Root CA - G3 and the Sectigo AAA
Certificate Services cross-sign.

- @sim/security/tls-trust drops the four roots the disclosure lists from the
  process default CA set at startup (app and realtime bootstraps, Trigger.dev
  init). It replaces the set only when one of them is present, so it becomes a
  no-op once runtimes ship their own fix, and never throws.
- The app and cron images distrust the same roots in the OS store, with a
  build-time check that fails if the edit stops removing them.
- Electron 43.8.1 carries the upstream SPKI blocklist (electron/electron#55130).

Bun's native fetch keeps its compiled-in store and ignores the process CA set,
so the emSign path there closes with the Bun release carrying oven-sh/bun#46159.
@waleedlatif1
waleedlatif1 requested a review from a team as a code owner October 20, 2026 17:26
@vercel

vercel Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Oct 20, 2026 5:48pm UTC

Request Review

Comment thread docker/app.Dockerfile
@greptile-apps

greptile-apps Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[High impact] The PR appears safe to merge within its stated coverage.

Summary

The PR removes compromised CA roots from supported TLS clients, updates the app and cron trust stores, and pins Electron to 43.8.1.

  • App and worker startup removes compromised roots from its TLS trust list.
  • App and cron images stop trusting two compromised roots.
  • The desktop app now uses Electron 43.8.1.

Reviews (2) · Last reviewed commit: "fix(tls): log the fail-open trust state ..." · Reviewed by Greptile

Comment thread apps/sim/bootstrap.ts Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 10 files

This PR changes authentication, authorization, or input validation. Ultrareviews find 2.4x more serious bugs than standard reviews. Comment @cubic-dev-ai ultrareview to run one.

Turn on auto-fix | Re-trigger cubic

Comment thread bunfig.toml Outdated
Comment thread apps/sim/bootstrap.ts Outdated
Comment thread apps/realtime/src/bootstrap.ts Outdated
Comment thread packages/security/src/tls-trust.ts
…-age gate for electron

The default production log level is ERROR, so a warn-level report of a runtime
that cannot replace its CA set was invisible. The electron exclusion was
unnecessary: the lockfile pins 43.8.1 and the age gate only applies when a
version is resolved.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 9 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1
waleedlatif1 merged commit 583e0ea into staging Oct 20, 2026
47 of 48 checks passed
@waleedlatif1
waleedlatif1 deleted the fix/emudhra-distrust branch October 20, 2026 17:52
@jake-jenkins

Copy link
Copy Markdown

Thanks very much for sorting this out so promptly you beauties!

This branch was successfully deployed

1 active deployment
Preview — 8f945d0d Deployed Oct 20, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants