Skip to content

feat(deploy): one provisioning path, a prompting DigitalOcean installer, and a domain that gets a certificate (#2380) - #2707

Merged
vybe merged 15 commits into
devfrom
feature/2380-do-provision
Sep 13, 2026
Merged

vybe merged 15 commits into
devfrom
feature/2380-do-provision

Conversation

@obasilakis

@obasilakis obasilakis commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Summary

The DigitalOcean provisioning branch that v0.9.5-rc1 / rc2 were cut from, previously #2682 (closed unmerged). It now includes #2683's installer fixes and two firewall guards restored after the port-list rewrite. Merges cleanly into dev.

What changes

  • One installer, three callers. scripts/deploy/start.sh --provision owns machine setup (Docker, pinned Caddy, ufw, the firewall unit) and site setup (the droplet's IP, Caddyfile, certificate, .env keys). The Packer bakery (--machine-only), the 1-Click first boot (--site-only --provenance do-marketplace) and the new installer all call it. The copies under packer/ are deleted.
  • Firewall with no port list. DOCKER-USER drops everything entering a container from off-box, with RETURNs only for replies to container-initiated connections and for Docker's own bridges. A link-local 169.254.0.0/16 DROP sits ahead of the bridge RETURNs, so containers cannot read the instance metadata service. On a script install, user-data carries the admin password and the Claude token.
  • A prompting DigitalOcean installer. scripts/deploy/trinity-do-create.sh runs on the operator's machine. It asks for the admin password and a subscription token (read -rs), creates a stock Ubuntu droplet whose user-data runs start.sh --provision, attaches the account's SSH keys, and waits for HTTPS. fix(deploy): snap doctl cannot read /tmp, and an apostrophe kills first boot (#2380) #2683 made it portable: GNU mktemp, a snap doctl's private /tmp, secrets shell-quoted, a refusal before the prompts, and bash 3.2 with no SSH keys.
  • Adding a domain is a Settings field. Caddy uses on-demand TLS with an ask gate, GET /api/public/tls-allowed. It is unauthenticated because Caddy holds no credential. It matches the exact host of the saved public URL (parsed, not substring-matched) and fails closed.
  • do-script provenance. The hardening guide's eligibility is a separate flag (hardening_guide_eligible), not a widening of marketplace_install.

Tests

  • test_2380_provision_single_source.py: single implementation, firewall rule ordering, the metadata-service DROP, the ask gate executed with exact-host and fail-closed cases, and the restored guards (ufw vs iptables-persistent; firewall re-applied on every boot).
  • test_2380_install_provenance.py, test_2380_installer_portability.py (runs the real installer against a stub doctl), test_2380_installer_release_pin.py.

For reviewers

Test plan

  • pytest tests/unit/test_2380_*.py
  • A 1-Click snapshot built from a branch stacked on this one (v0.9.5-rc3) was booted on a fresh droplet: first boot finished, the IP certificate validated, all containers healthy. The domain flow was not verified end to end.

Refs #2380
Fixes #2593

🤖 Generated with Claude Code

https://claude.ai/code/session_01PdRMazCDYVoxqLFw6uKm3a

obasilakis and others added 12 commits September 10, 2026 14:48
… it (#2380)

Provisioning a bare cloud VM for Trinity existed in three copies: the Packer
bakery, the Packer first-boot script, and a hand-written script pasted into the
DigitalOcean deploy doc. They had already drifted — one carried a DOCKER-USER
DROP list missing 8081, so the login page answered plain HTTP past the
certificate and past the http->https redirect, on the image whose headline
design note is that everything reaches users through Caddy on 80/443.

`start.sh --provision --cloud <name>` is now the only implementation, in two
phases because a snapshot-based image splits them: `--machine-only` (packages,
pinned Caddy with the IP-certificate floor asserted, ufw, the firewall unit) is
bakeable; `--site-only` (the instance's own IP, the .env keys, the Caddyfile,
the certificate) is per-instance. The bakery calls the first, first boot calls
the second and continues into the install, and a doc-driven install calls both.
firstboot.sh drops from 230 lines to 118 and 01-provision.sh from 176 to 99.

It is off by default and refuses unless it is root on Linux with a cloud
metadata service answering, so it stays inert on a developer laptop. An
ADMIN_PASSWORD in the environment is now written through to .env, so an install
script does not have to hand-roll its own .env writer.

The firewall rule is inverted rather than enumerated. Everything entering a
container from off-box is dropped whatever the port, with two RETURNs ahead of
it: replies to connections a container opened (without which agents lose
outbound internet), and traffic from Docker's own bridges. Naming what is inside
rather than which interface is outside also covers DigitalOcean's private eth1.
There is no list left to drift, so the test that policed the list is replaced by
one that pins the ordering and the absence of a fourth copy.

Also widens the first-run hardening guide to doc-driven DigitalOcean installs,
which #2380's author amended their own acceptance criterion to require. The
installer records `do-script` — honest, because it refuses to run at all unless
DO's metadata service answers — and `hardening_guide_eligible` is a separate
gate from `marketplace_install` rather than a widening of it, so the guide
reaches these installs without anyone claiming a vendor listing they never came
from. Deliberately still provenance and never TLS state: the managed fleet runs
plain HTTP behind a tunnel with no domain and a 100.x address, so a posture-based
gate would fire on every paying client forever.

The https-ip copy gains the renewal caveat, hedged the way that block's own
acceptance criterion requires: a ~6-day certificate renewed only while the
machine runs is a property of the profile such an install is known to use, so it
is stated as what happens after a long shutdown, never as a claim about this
instance's live certificate.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QvfFoqB2v2zPyXJXeyBGbP
…2380)

The deploy doc's flow was eight manual steps in DigitalOcean's web console,
one of which is "paste this script into a textarea after editing three lines".
Seven of those eight can be a script, and the one that cannot — open the URL and
sign in — is the human's job anyway. A console click-path also cannot satisfy
this work's own acceptance criterion that every command in the doc has been
executed verbatim at least once: nobody can execute "tick Advanced Options",
so the most error-prone step is the one that ships permanently untested.

It prompts rather than shipping a file to edit. "Download this and change four
lines" fails the audience the doc exists for, and it puts two secrets in a file
on disk; `read -rs` keeps both off the terminal, out of shell history, and out
of every file but the droplet's own user-data. Checks come before questions, so
a missing doctl is never discovered after collecting two credentials. The
password is asked twice because it is the credential for a server that does not
exist yet — a typo is not recoverable by retrying, it is a rebuild — and the
paid resource is confirmed before it is created.

Prompts still read from a pipe, so the whole flow stays drivable in a test:

    printf 'pw\npw\ntoken\n\n\ny\n' | bash scripts/deploy/trinity-do-create.sh

The droplet-side half is unchanged and stays one copy: clone, then
`start.sh --provision --cloud digitalocean --hosted --unattended`, the same
installer the Packer image runs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QvfFoqB2v2zPyXJXeyBGbP
)

A cloud instance serves its own user-data verbatim from link-local, for the life
of the machine, to anything that can reach it — and on a script-installed
droplet that user-data carries the Trinity admin password and the operator's
Claude subscription token. Agent containers could reach it. An agent running
untrusted model-authored code is precisely the case that must not be able to
read the credential that owns the box.

Dropped outbound from containers as the RFC 3927 range rather than the
well-known 169.254.169.254 host: the property being blocked is "link-local,
host-adjacent, not routable", and every cloud publishes its metadata endpoint
somewhere in that range, so the single address would have been a DigitalOcean-
shaped fix to a general problem.

Position is load-bearing and is what the test pins. The DROP sits after the
conntrack RETURN (so only the opening packet of a metadata connection is ever
evaluated, which is enough — it never establishes) and BEFORE the two bridge
RETURNs. Behind them it would be decoration: container-originated traffic
RETURNs out of the chain before reaching it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QvfFoqB2v2zPyXJXeyBGbP
…ed (#2380)

The hardening card's step one told the operator to point DNS at the server and
set the Public URL in Settings, then said "whatever terminates TLS in front of
it picks up the name". Nothing does. `public_chat_url` is a display and
webhook-base setting, and the card's own source comment says so — no code reads
it and reconfigures a proxy, a listener or a certificate. On every install this
card is shown to, Caddy is holding a single `https://<IP>` site block written at
provision time.

So the sequence the card produced was: operator sets the URL, install_tls_posture
flips to `https-domain` by string-parsing that setting, the card treats step one
as complete and advances to the tunnel step, and the domain serves a certificate
error because the web server still answers only to the IP. The card retired on a
state it had helped break, and the instance was worse off than before it gave
the advice.

`scripts/deploy/set-domain.sh` does the job the card was describing. It refuses
unless DNS already points at this machine — that check is first, before anything
is touched, because issuance validates over the name and a stale A record is
what turns a working instance into a broken one. Then it rewrites the Caddyfile
for the name (no `shortlived` profile: a real name earns an ordinary ~90-day
certificate, which is the upgrade being made), validates, reloads, waits for the
certificate against the real hostname with the system trust store, and only then
tells Trinity its new address. Every failure after the rewrite restores the
previous config and reloads, so a bad run ends where it started.

The bare IP is kept as a redirect rather than dropped, with its short-lived
certificate, so links handed out before the domain existed keep working — a
redirect that cannot complete a handshake is not a redirect.

It runs on the server because it has to: Trinity is in a container with no host
privileges, so it can neither write /etc/caddy/Caddyfile nor reload Caddy. The
card now says that, names the command, and keeps every honest claim the previous
copy made — it only moves the actor from "whatever is out there, somehow" to
something the operator can actually run. That constraint is the same one that
already keeps the tunnel step to prose, so the card is now consistent about it.

`set_env_key` moves to scripts/deploy/env-file.sh, sourced by both writers: a
.env writer that disagrees with itself corrupts credentials silently, and this
branch exists because two copies of provisioning logic drifted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QvfFoqB2v2zPyXJXeyBGbP
Walking the journey a real operator takes found the previous fix had covered
the wrong half. The card said the right thing in its disclosure and still
pointed its primary button at the trap.

The operator who HAS a domain — exactly who the button was for — clicked "Add a
domain", landed on Settings → General → Public URL (a bare input, a Save button,
no guard, placeholder `https://your-domain.com`), typed their domain and saved.
That sets the name Trinity hands out and reconfigures nothing: Caddy still held
only the `https://<IP>` site written at provision time, so the domain served a
certificate error while the old IP link kept working and the instance looked
fine. `install_tls_posture` then flipped to `https-domain` by string-parsing
their own input, the card advanced to the tunnel stage, and the address section
carrying the command that would have fixed it — `v-if` on that stage —
disappeared. The UI congratulated them on a step it had just helped them break,
and removed the way back.

So the face carries the command now, not a link to the field. There is no
navigation button because there is nowhere useful to navigate: `set-domain.sh`
sets the Public URL itself, and setting it by hand is not enough on its own.
`select-all` on the block so one click takes the whole line. The disclosure
keeps the reasoning, including the sentence saying explicitly that the settings
field alone does not do it.

The three tests that pinned the old shape are updated rather than deleted, each
carrying why the assertion inverted — "links to Settings → General" was a true
description of a defect.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QvfFoqB2v2zPyXJXeyBGbP
#2380)

The last two attempts at this both ended with an operator needing a root shell
on the droplet, which a non-engineer following a deploy guide does not have. The
constraint that forced it was real — Trinity is containerised and can neither
rewrite /etc/caddy/Caddyfile nor reload Caddy — but it was the wrong thing to
work around. Inverting the direction removes it entirely: Caddy asks Trinity
whether a hostname is allowed, Trinity answers from what an admin saved, and no
privilege moves into the container.

The provisioned Caddyfile gains a catch-all site with on-demand TLS and an `ask`
gate at /api/public/tls-allowed. Save a domain in Settings, point its A record
here, and the first visit obtains the certificate. That is the whole step, which
is what this card claimed from the beginning.

The gate carries the entire security model, so it allows exactly one name: the
host of the saved public_chat_url, parsed with urlparse and compared for
equality. Substring matching would hand `evil-example.com` a certificate request
from a server configured for `example.com`. It fails closed on an unset URL and
on a failed settings read — `on_demand` without a working gate turns the instance
into a certificate requester for anyone who points DNS at its address, until the
ACME account is rate-limited and the operator's own renewals start failing.
Executed rather than grepped in the tests: a static check cannot tell an exact
match from a substring one, and that difference is the whole of it.

set-domain.sh is deleted. It was written for this job two commits ago and is now
a second way to do something that works from the browser; keeping it as a
"fallback" would have been keeping a path nobody tests for a case that no longer
exists.

Caddyfile syntax verified against the 2.11.x docs rather than memory: `interval`
and `burst` are removed from on_demand_tls, `ask` receives ?domain= and
authorises on 2xx.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QvfFoqB2v2zPyXJXeyBGbP
…2380)

Without this the droplet has no key on it, DigitalOcean emails a root password
to the account owner, and the only way to a shell is the browser console. That
is fine until something needs looking at, which on a first deploy is exactly
when it happens.

Attaches keys that already exist on the account — the operator's own, on the
operator's account, going onto the operator's server. Nothing is created,
uploaded or generated, and an account with no keys behaves exactly as before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QvfFoqB2v2zPyXJXeyBGbP
`mktemp -t trinity-user-data` takes a bare prefix on macOS/BSD, but GNU
coreutils reads the same argument as a template and requires the trailing
X's — so on Ubuntu (i.e. every Linux operator) the installer died with
"mktemp: too few X's in template" immediately after collecting both
secrets and confirming the create.

Use a full path template instead: portable on both, and it keeps the
umask-077 0600 mode the user-data file needs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LgRv6koYJKzor7ZHoRkGYS
#2680 landed two build-standard changes on the files this branch restructured:

- 01-provision.sh: the pre-install `apt-get full-upgrade` (DO's img-check
  fails on pending security updates) is kept, now between `apt-get update`
  and the checkout's own package install.
- firstboot.sh: the body is this branch's `start.sh --provision --site-only`
  call; the `X-DO-MARKETPLACE` header #2680 added to the Caddyfile moves to
  `provision_site` in start.sh, where the Caddyfile is written now, keyed on
  the do-marketplace provenance since the same path serves the doc install.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015owqMKD5QDjzZrUTF2Joht
…ddyfile is written now

#2680's guard pinned the header inside firstboot.sh's Caddyfile heredoc;
#2380 moved that heredoc into `start.sh --provision --site-only`, so the
merge left the test asserting a file that no longer writes a Caddyfile.
Point it at start.sh and also pin that the header is keyed on the
do-marketplace provenance, since the same function serves the doc install.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015owqMKD5QDjzZrUTF2Joht
…ropped (#2380)

6104b0e deleted test_2281_firstboot_port_exposure.py because the
DOCKER-USER rule no longer enumerates ports, so the tests that policed the
list had nothing left to check. Two of its five tests were never about the
list, and they went with the file:

- ufw and iptables-persistent are never both installed. ufw Breaks
  iptables-persistent, so installing it to persist the rules makes apt
  remove ufw, and the install dies later at `ufw --force reset`.
- the firewall rules are re-applied on every boot. Without
  iptables-persistent, trinity-docker-firewall.service is the only thing
  that survives a reboot; without it every Docker-published port reopens
  after the first restart (#2281 review I1).

Both are back in test_2380_provision_single_source.py, pointed at where the
code lives now: start.sh --provision installs ufw and writes the unit as a
heredoc, so the tests read that heredoc instead of a shipped unit file. Each
was checked against a mutation of start.sh (iptables-persistent added, the
enable line removed, After=docker.service removed) and fails on each.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PdRMazCDYVoxqLFw6uKm3a
…st boot (#2380) (#2683)

* fix(deploy): snap doctl cannot read /tmp, and an apostrophe kills first boot (#2380)

Two fixes to the DigitalOcean installer, found QA'ing it across macOS and Linux.
Neither is visible on the machine it was written on.

**A snap-installed doctl has a private /tmp.** On most Linux distributions doctl
comes from snap, and a snap runs in its own mount namespace — a file written to
the real /tmp is not there when doctl opens it, so it fails on a file that
demonstrably exists:

    Error: open /tmp/trinity-user-data.XXXXXX: no such file or directory

Snap's `home` interface can read non-hidden paths under $HOME, so a snap doctl
gets the file there instead; everyone else keeps TMPDIR. The name stays visible
because that interface denies dotfiles too, `umask 077` still makes it 0600, and
the EXIT trap still removes it. Reported by dolho, who hit it on Linux. The
`mktemp -t` half of his finding is already on this branch (aaa3de5).

A snap doctl with no usable $HOME now fails at that point with an explanation,
rather than at the droplet-create call with a path error, after both prompts.

**A single quote in either secret breaks the droplet's first boot.** Both are
interpolated into single-quoted assignments in the user-data, so a `'` inside one
closes the string early:

    export ADMIN_PASSWORD='Tr0ub4dor's!Horse'
    bash: unexpected EOF while looking for matching `''

Not a hypothetical input — the password rules require a special character and `'`
is one, so that password passes this script's own check and the backend's. The
failure is expensive out of proportion to the typo: the droplet is created and
billing before first boot runs, the operator watches a 15-minute progress bar end
in a timeout, and the cause is invisible without reading the install log. Both
secrets now go through `_shquote` (the portable `'\''` idiom).

Verified on both platforms rather than reasoned about — macOS bash 3.2 and
ubuntu:24.04 bash 5.2 / GNU coreutils 9.4, driving the real prompt flow with a
stubbed doctl and diffing the generated user-data:

- `mktemp -t trinity-user-data`  → macOS OK, Linux `too few X's in template`
- the full template              → both OK, mode 0600
- snap-detection truth table     → identical on both
- password round-trip            → 7/7 both platforms, including apostrophes,
                                   backticks, backslashes, tabs and non-ASCII;
                                   the unfixed script fails the apostrophe cases
                                   with a syntax error

`tests/unit/test_2380_installer_portability.py` pins all three. The round-trip
guard is executed, not pattern-matched — it runs the script's own `_shquote` and
lets a shell parse the result back, so it catches any breakage rather than the
one spelling that was wrong.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011sj48trT3XmyaSus4GUtXB

* test(deploy): the installer's default tag must track VERSION (#2380)

`trinity-do-create.sh` hardcodes the release it installs:

    TRINITY_IMAGE_TAG="${TRINITY_IMAGE_TAG:-v0.9.5-rc2}"

That default IS what operators get — the guide tells them to run the script
straight off a tag with no environment set. So on the day `v0.9.5` is cut, a
script fetched from the v0.9.5 tag still clones and pulls `v0.9.5-rc2` unless
someone remembers this one line.

Nothing catches it today. It is not a syntax error, the rc2 images still exist
and still pull, and the install SUCCEEDS — it just installs the previous release
candidate. The operator cannot tell, and neither can CI: the release checklist's
"VERSION match" step compares the VERSION file to the tag, not this script to
either.

The guard ties them together. It passes now (VERSION `0.9.5-rc2`, default
`v0.9.5-rc2`) and fails the moment VERSION is bumped for the cut — which is
exactly when someone needs telling. Verified by mutation: setting VERSION to
`0.9.5` produces

    AssertionError: trinity-do-create.sh installs v0.9.5-rc2 but VERSION says 0.9.5.

A second case pins the `v` prefix. One string feeds both `git clone --branch`
(needs the git ref) and `docker pull` (published under both spellings), so only
the `v` form works for both — the #2471 failure, where `v0.9.5-rc1` shipped
without its v-prefixed image alias and left no tag value able to build the
marketplace snapshot.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011sj48trT3XmyaSus4GUtXB

* fix(deploy): refuse before the prompts, quote the tag, and test the installer by running it (#2380)

Addresses review I1-I3 on #2683.

I1: the snap-doctl-without-$HOME refusal ran after both secret prompts,
breaking the script's own "checks first, questions second" rule. The
USER_DATA_DIR decision moves into the pre-flight block beside
`doctl account get`; mktemp stays where it was.

I2: TRINITY_IMAGE_TAG was the third single-quoted interpolation in the
user-data heredoc and the only unquoted one. It now goes through _shquote,
and the static guard asserts the rule (every '${...}' in the heredoc is a
_Q value) instead of two spellings of it.

I3: the guards grepped for the quoted spelling and never ran the script.
New tests run the real installer against a stub doctl (package and snap
paths, apostrophes and shell metacharacters in the password, token and
tag), then `bash -n` the captured user-data and evaluate its real export
lines and subscription call in context, and assert a snap doctl without
$HOME is refused before any question.

Running it found a fourth defect: under `set -u`, bash < 4.4 (macOS's
/bin/bash is 3.2) treats an empty "${SSH_ARGS[@]}" as unbound, so an
account with no SSH keys died at "Creating the droplet...", after both
secrets were typed. Expanded as ${SSH_ARGS[@]+"${SSH_ARGS[@]}"}.

Each fix was checked by reverting it: the tests fail on every revert.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PdRMazCDYVoxqLFw6uKm3a

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
@obasilakis
obasilakis requested a review from dolho September 11, 2026 14:12
obasilakis and others added 2 commits September 11, 2026 16:40
… TLS gate (#2380)

/validate-pr on #2707 found the branch shipping new capability without the
requirements and feature-flow updates CLAUDE.md requires. Written from the
code at this commit, not from the PR body.

Requirements (docs/memory/requirements/infrastructure.md):
- Corrected: HOST-010 and the section 8.10 description, PROV-006 (flags now
  carry hardening_guide_eligible), PROV-009 (the second path has been a
  Cloudflare Tunnel since #2564, not a VPN), PROV-010 (a two-stage card
  gated on hardening_guide_eligible, with a domain field), PROV-011 (start.sh
  --provision now writes TRINITY_INSTALL_SOURCE).
- Added: PROV-012 one provisioning implementation, three callers; PROV-013
  the portless container firewall and the metadata-service block; PROV-014
  the prompting installer and do-script provenance; PROV-015 the on-demand
  TLS ask gate.

Feature flows: install-provenance (two-stage card, do-script,
hardening_guide_eligible, the domain field and ask gate; fixes both stale
claims in #2593), hosted-install (a Provision Layer section), telemetry-sharing
(the IntersectionObserver gate from #2593), and their index rows. Stale
sentences in agent-lifecycle.md, backend.md, api-endpoints.md and
DEPLOYMENT.md corrected.

Fixes #2593

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PdRMazCDYVoxqLFw6uKm3a
…ts (#2380)

0423e6c replaced scripts/deploy/set-domain.sh with a Settings field, but
two comments still named it as env-file.sh's second caller, and the
provisioning test kept an unused path constant pointing at it. start.sh is
now the only script that sources env-file.sh; the comments say so.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PdRMazCDYVoxqLFw6uKm3a
obasilakis added a commit that referenced this pull request Sep 11, 2026
Brings #2683's installer fixes, the #2707 docs (PROV-012..015) and the
set-domain.sh cleanup under the onboarding work. Where the two sides
described the hardening guide differently, the merged docs describe the
state once both land: the guide is the first-run overlay's `secure` step,
gated on `hardening_guide_eligible` (marketplace sources plus do-script),
and a saved domain gets its certificate through the on-demand TLS ask gate.
trinity-do-create.sh's default tag moves to v0.9.5-rc3 so it matches
VERSION, as test_2380_installer_release_pin requires.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PdRMazCDYVoxqLFw6uKm3a
@vybe

vybe commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

merge-train 2026-09-11: validated READY, held back from this train for maintainer acknowledgement — not for a defect. Lane C validation found no criticals: /cso --diff has nothing at the gate, the new unauthenticated GET /api/public/tls-allowed is a fail-closed exact-hostname yes/no for Caddy's on-demand ask (404 on unset URL, failed read, or mismatch — discloses nothing DNS doesn't), no new setting is written, secrets are never echoed, .env is 0600 before any write, no curl | bash, Caddy pinned. 185 PR tests + 78 standing guards pass locally; merge-tree is clean against dev.

Five items are the maintainer's to accept or push back on, in one line each, before it lands — they're decisions, not code:

  1. AC amendment on feat: record install provenance (DO Marketplace) + first-run HTTPS/VPN hardening guide #2380 — the issue still reads "renders only when provenance is a marketplace value" and "no new endpoint"; this PR widens the guide to do-script and adds /api/public/tls-allowed, citing an amendment recorded nowhere on the issue.
  2. Firewall unit bound to the checkout pathstart.sh:248,259 points the unit's ExecStart at /opt/trinity/... and deletes the checkout-independent copy under /opt/trinity-firstboot/. Move or re-clone the checkout and the next reboot leaves TRINITY-FW unpopulated while Caddy keeps the site up — every 0.0.0.0-published port (8000, 8080, 8081 plain-HTTP login, 8686, 4317/4318/8889/13133) internet-reachable. Nothing re-asserts or monitors it. The feat: DigitalOcean Marketplace 1-Click Droplet listing for Trinity (vendor application + Packer snapshot, AI Agents category) #2281-C1 class; the documented upgrade path is in-place so it takes an operator deviation, but it's the residual to accept or close (install to /usr/local/sbin re-introduces the second copy the design rejects).
  3. User-data secret lifetimetrinity-do-create.sh:173-210: admin password + Claude token ride in cloud-init user-data, which the metadata service serves to any host process (the caddy user included) for the droplet's lifetime; docker-firewall.sh blocks containers only. PROV-013/014 acknowledge lifetime and content but frame the container side.
  4. Metadata miss now aborts first bootstart.sh:274,404 provision_dies on one 10 s attempt; the deleted firstboot.sh warned and still started Trinity over HTTP. Behaviour change on the 1-Click path.
  5. The title feature is unverified live — only the IP-cert 1-Click path was booted; provision_site / the on-demand domain flow are unexecutable by tests and weren't run.

Mechanical, can ride whenever: test_2380_provision_single_source.py:235 "on_demand" in body is satisfied by the global and doesn't pin the https:// { tls { on_demand } } site block — deleting it passes every test; no FastAPI-level test reaches /tls-allowed (route registration + no-auth are grepped); Settings.vue:2168 INSTALL_SOURCE_LABELS lacks 'do-script'; test_2380_installer_release_pin.py fails the moment VERSION is bumped, so /release must bump trinity-do-create.sh:24 in the same commit.

With the five answered on this thread it merges alone, or on the next train.

…ough FastAPI (#2380)

Review follow-ups on #2707, all mechanical.

The Caddyfile guard asserted `"on_demand" in body`, which the global
`on_demand_tls` option already satisfies: deleting the catch-all
`https:// { tls { on_demand } }` site — the entire "add a domain" feature —
passed every test. It now extracts the heredoc and asserts that site as a
block, and that the bare-IP site still asks for a short-lived certificate.
Checked by deleting the block: the test fails.

Nothing reached GET /api/public/tls-allowed through FastAPI. The decision
(exact hostname, fail closed) is proved by exec-slicing the function, which
cannot see the route's path, its query parameter, or whether it gained an
auth dependency — and Caddy holds no credential, so a 401 there would refuse
every certificate with the failure visible only in a live handshake. A new
test mounts the real router and calls the real URL.

Settings' install-source label map had no entry for `do-script`, so a
script-installed droplet showed the raw value.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PdRMazCDYVoxqLFw6uKm3a
@vybe

vybe commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

merge-train 2026-09-12: the five items from 09-11 are accepted as-is, plus one new one — merging after today's train lands.

Re-validated at a1ff1314 (lane C, /review + /cso --diff): 0 critical; the on-demand site block and the FastAPI-level ask-gate test you added close the mechanical items. merge-tree against dev is clean; the 12 dev commits since your merge-base overlap only in docs, disjoint hunks.

Maintainer decisions, recorded here so they're findable:

  1. AC amendment on feat: record install provenance (DO Marketplace) + first-run HTTPS/VPN hardening guide #2380 — accepted: the guide renders for do-script too, and GET /api/public/tls-allowed is the new endpoint (fail-closed exact-hostname yes/no for Caddy's ask).
  2. Firewall unit ExecStart bound to the checkout path — accepted as the documented residual: the upgrade path is in-place; a moved/re-cloned checkout is an operator deviation.
  3. Secrets in cloud-init user-data for the droplet's lifetime — accepted as-is (PROV-013/014); requires prior host compromise. Hardening (scrub user-data post-boot, or drop link-local from OUTPUT for non-root) is a follow-up, not a blocker.
  4. Metadata miss aborts first boot — accepted: fail loud on a 1-Click boot beats silently starting over HTTP.
  5. Domain / on-demand-TLS flow unverified live — accepted; the IP-cert 1-Click path was booted, the domain path is covered by test_2380_tls_ask_endpoint.py through a real TestClient and the pinned Caddyfile block.
  6. (new) Agent SSH ports 2222+ now dropped off-box on provisioned hosts — correct tightening (they were internet-reachable before); the SSH panel still hands out a host:port, so a one-line note in the hardening guide / get_agent_ssh_access doc is a follow-up.

Note for /release: test_2380_installer_release_pin.py pins trinity-do-create.sh:24 to VERSION — bump both in the same commit.

@vybe vybe left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

merge-train: validated alongside train/20260913-0705 (#2745), merged individually off the post-train dev — lane C, /validate-pr + /review + /cso --diff clean; the six maintainer-intent items are accepted as recorded in the thread. Reminder for /release: bump trinity-do-create.sh:24 with VERSION in the same commit.

@vybe
vybe merged commit 2c5cfe0 into dev Sep 13, 2026
29 checks passed
obasilakis added a commit that referenced this pull request Sep 14, 2026
#2707 landed on dev as a squash commit while this branch carried its
original commits, so the same content arrived twice and every file both
touched conflicted. dev also moved 26 commits on, including the user-docs
and feature-flow syncs that rewrote the same pages.

Resolved toward dev wherever dev was ahead on #2380 (the Caddyfile site-block
assertions, the FastAPI-level ask-gate test, the do-script label) and toward
this branch wherever the onboarding work supersedes it: HardeningGuide.vue,
FinishSetupCard.vue, FrontDeskPanel.vue and OnboardingWizard.vue stay deleted,
and the docs describe the overlay's secure step rather than the card.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PdRMazCDYVoxqLFw6uKm3a
vybe pushed a commit that referenced this pull request Sep 14, 2026
…ials without a terminal (trinity-enterprise#580, #581, #582) (#2715)

* feat(deploy): one installer provisions the machine, three callers use it (#2380)

Provisioning a bare cloud VM for Trinity existed in three copies: the Packer
bakery, the Packer first-boot script, and a hand-written script pasted into the
DigitalOcean deploy doc. They had already drifted — one carried a DOCKER-USER
DROP list missing 8081, so the login page answered plain HTTP past the
certificate and past the http->https redirect, on the image whose headline
design note is that everything reaches users through Caddy on 80/443.

`start.sh --provision --cloud <name>` is now the only implementation, in two
phases because a snapshot-based image splits them: `--machine-only` (packages,
pinned Caddy with the IP-certificate floor asserted, ufw, the firewall unit) is
bakeable; `--site-only` (the instance's own IP, the .env keys, the Caddyfile,
the certificate) is per-instance. The bakery calls the first, first boot calls
the second and continues into the install, and a doc-driven install calls both.
firstboot.sh drops from 230 lines to 118 and 01-provision.sh from 176 to 99.

It is off by default and refuses unless it is root on Linux with a cloud
metadata service answering, so it stays inert on a developer laptop. An
ADMIN_PASSWORD in the environment is now written through to .env, so an install
script does not have to hand-roll its own .env writer.

The firewall rule is inverted rather than enumerated. Everything entering a
container from off-box is dropped whatever the port, with two RETURNs ahead of
it: replies to connections a container opened (without which agents lose
outbound internet), and traffic from Docker's own bridges. Naming what is inside
rather than which interface is outside also covers DigitalOcean's private eth1.
There is no list left to drift, so the test that policed the list is replaced by
one that pins the ordering and the absence of a fourth copy.

Also widens the first-run hardening guide to doc-driven DigitalOcean installs,
which #2380's author amended their own acceptance criterion to require. The
installer records `do-script` — honest, because it refuses to run at all unless
DO's metadata service answers — and `hardening_guide_eligible` is a separate
gate from `marketplace_install` rather than a widening of it, so the guide
reaches these installs without anyone claiming a vendor listing they never came
from. Deliberately still provenance and never TLS state: the managed fleet runs
plain HTTP behind a tunnel with no domain and a 100.x address, so a posture-based
gate would fire on every paying client forever.

The https-ip copy gains the renewal caveat, hedged the way that block's own
acceptance criterion requires: a ~6-day certificate renewed only while the
machine runs is a property of the profile such an install is known to use, so it
is stated as what happens after a long shutdown, never as a claim about this
instance's live certificate.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QvfFoqB2v2zPyXJXeyBGbP

* feat(deploy): a prompting DigitalOcean installer, not a file to edit (#2380)

The deploy doc's flow was eight manual steps in DigitalOcean's web console,
one of which is "paste this script into a textarea after editing three lines".
Seven of those eight can be a script, and the one that cannot — open the URL and
sign in — is the human's job anyway. A console click-path also cannot satisfy
this work's own acceptance criterion that every command in the doc has been
executed verbatim at least once: nobody can execute "tick Advanced Options",
so the most error-prone step is the one that ships permanently untested.

It prompts rather than shipping a file to edit. "Download this and change four
lines" fails the audience the doc exists for, and it puts two secrets in a file
on disk; `read -rs` keeps both off the terminal, out of shell history, and out
of every file but the droplet's own user-data. Checks come before questions, so
a missing doctl is never discovered after collecting two credentials. The
password is asked twice because it is the credential for a server that does not
exist yet — a typo is not recoverable by retrying, it is a rebuild — and the
paid resource is confirmed before it is created.

Prompts still read from a pipe, so the whole flow stays drivable in a test:

    printf 'pw\npw\ntoken\n\n\ny\n' | bash scripts/deploy/trinity-do-create.sh

The droplet-side half is unchanged and stays one copy: clone, then
`start.sh --provision --cloud digitalocean --hosted --unattended`, the same
installer the Packer image runs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QvfFoqB2v2zPyXJXeyBGbP

* fix(deploy): containers cannot read the instance metadata service (#2380)

A cloud instance serves its own user-data verbatim from link-local, for the life
of the machine, to anything that can reach it — and on a script-installed
droplet that user-data carries the Trinity admin password and the operator's
Claude subscription token. Agent containers could reach it. An agent running
untrusted model-authored code is precisely the case that must not be able to
read the credential that owns the box.

Dropped outbound from containers as the RFC 3927 range rather than the
well-known 169.254.169.254 host: the property being blocked is "link-local,
host-adjacent, not routable", and every cloud publishes its metadata endpoint
somewhere in that range, so the single address would have been a DigitalOcean-
shaped fix to a general problem.

Position is load-bearing and is what the test pins. The DROP sits after the
conntrack RETURN (so only the opening packet of a metadata connection is ever
evaluated, which is enough — it never establishes) and BEFORE the two bridge
RETURNs. Behind them it would be decoration: container-originated traffic
RETURNs out of the chain before reaching it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QvfFoqB2v2zPyXJXeyBGbP

* feat(deploy): make "add a domain" actually work, not just be advertised (#2380)

The hardening card's step one told the operator to point DNS at the server and
set the Public URL in Settings, then said "whatever terminates TLS in front of
it picks up the name". Nothing does. `public_chat_url` is a display and
webhook-base setting, and the card's own source comment says so — no code reads
it and reconfigures a proxy, a listener or a certificate. On every install this
card is shown to, Caddy is holding a single `https://<IP>` site block written at
provision time.

So the sequence the card produced was: operator sets the URL, install_tls_posture
flips to `https-domain` by string-parsing that setting, the card treats step one
as complete and advances to the tunnel step, and the domain serves a certificate
error because the web server still answers only to the IP. The card retired on a
state it had helped break, and the instance was worse off than before it gave
the advice.

`scripts/deploy/set-domain.sh` does the job the card was describing. It refuses
unless DNS already points at this machine — that check is first, before anything
is touched, because issuance validates over the name and a stale A record is
what turns a working instance into a broken one. Then it rewrites the Caddyfile
for the name (no `shortlived` profile: a real name earns an ordinary ~90-day
certificate, which is the upgrade being made), validates, reloads, waits for the
certificate against the real hostname with the system trust store, and only then
tells Trinity its new address. Every failure after the rewrite restores the
previous config and reloads, so a bad run ends where it started.

The bare IP is kept as a redirect rather than dropped, with its short-lived
certificate, so links handed out before the domain existed keep working — a
redirect that cannot complete a handshake is not a redirect.

It runs on the server because it has to: Trinity is in a container with no host
privileges, so it can neither write /etc/caddy/Caddyfile nor reload Caddy. The
card now says that, names the command, and keeps every honest claim the previous
copy made — it only moves the actor from "whatever is out there, somehow" to
something the operator can actually run. That constraint is the same one that
already keeps the tunnel step to prose, so the card is now consistent about it.

`set_env_key` moves to scripts/deploy/env-file.sh, sourced by both writers: a
.env writer that disagrees with itself corrupts credentials silently, and this
branch exists because two copies of provisioning logic drifted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QvfFoqB2v2zPyXJXeyBGbP

* fix(onboarding): the card's one action is the one that works (#2380)

Walking the journey a real operator takes found the previous fix had covered
the wrong half. The card said the right thing in its disclosure and still
pointed its primary button at the trap.

The operator who HAS a domain — exactly who the button was for — clicked "Add a
domain", landed on Settings → General → Public URL (a bare input, a Save button,
no guard, placeholder `https://your-domain.com`), typed their domain and saved.
That sets the name Trinity hands out and reconfigures nothing: Caddy still held
only the `https://<IP>` site written at provision time, so the domain served a
certificate error while the old IP link kept working and the instance looked
fine. `install_tls_posture` then flipped to `https-domain` by string-parsing
their own input, the card advanced to the tunnel stage, and the address section
carrying the command that would have fixed it — `v-if` on that stage —
disappeared. The UI congratulated them on a step it had just helped them break,
and removed the way back.

So the face carries the command now, not a link to the field. There is no
navigation button because there is nowhere useful to navigate: `set-domain.sh`
sets the Public URL itself, and setting it by hand is not enough on its own.
`select-all` on the block so one click takes the whole line. The disclosure
keeps the reasoning, including the sentence saying explicitly that the settings
field alone does not do it.

The three tests that pinned the old shape are updated rather than deleted, each
carrying why the assertion inverted — "links to Settings → General" was a true
description of a defect.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QvfFoqB2v2zPyXJXeyBGbP

* feat(deploy): adding a domain is a settings field, not a shell command (#2380)

The last two attempts at this both ended with an operator needing a root shell
on the droplet, which a non-engineer following a deploy guide does not have. The
constraint that forced it was real — Trinity is containerised and can neither
rewrite /etc/caddy/Caddyfile nor reload Caddy — but it was the wrong thing to
work around. Inverting the direction removes it entirely: Caddy asks Trinity
whether a hostname is allowed, Trinity answers from what an admin saved, and no
privilege moves into the container.

The provisioned Caddyfile gains a catch-all site with on-demand TLS and an `ask`
gate at /api/public/tls-allowed. Save a domain in Settings, point its A record
here, and the first visit obtains the certificate. That is the whole step, which
is what this card claimed from the beginning.

The gate carries the entire security model, so it allows exactly one name: the
host of the saved public_chat_url, parsed with urlparse and compared for
equality. Substring matching would hand `evil-example.com` a certificate request
from a server configured for `example.com`. It fails closed on an unset URL and
on a failed settings read — `on_demand` without a working gate turns the instance
into a certificate requester for anyone who points DNS at its address, until the
ACME account is rate-limited and the operator's own renewals start failing.
Executed rather than grepped in the tests: a static check cannot tell an exact
match from a substring one, and that difference is the whole of it.

set-domain.sh is deleted. It was written for this job two commits ago and is now
a second way to do something that works from the browser; keeping it as a
"fallback" would have been keeping a path nobody tests for a case that no longer
exists.

Caddyfile syntax verified against the 2.11.x docs rather than memory: `interval`
and `burst` are removed from on_demand_tls, `ask` receives ?domain= and
authorises on 2xx.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QvfFoqB2v2zPyXJXeyBGbP

* fix(deploy): attach the account's SSH keys to the droplet it creates (#2380)

Without this the droplet has no key on it, DigitalOcean emails a root password
to the account owner, and the only way to a shell is the browser console. That
is fine until something needs looking at, which on a first deploy is exactly
when it happens.

Attaches keys that already exist on the account — the operator's own, on the
operator's account, going onto the operator's server. Nothing is created,
uploaded or generated, and an account with no keys behaves exactly as before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QvfFoqB2v2zPyXJXeyBGbP

* fix(deploy): portable mktemp in the DigitalOcean installer (#2380)

`mktemp -t trinity-user-data` takes a bare prefix on macOS/BSD, but GNU
coreutils reads the same argument as a template and requires the trailing
X's — so on Ubuntu (i.e. every Linux operator) the installer died with
"mktemp: too few X's in template" immediately after collecting both
secrets and confirming the create.

Use a full path template instead: portable on both, and it keeps the
umask-077 0600 mode the user-data file needs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LgRv6koYJKzor7ZHoRkGYS

* test(deploy): the X-DO-MARKETPLACE guard reads start.sh, where the Caddyfile is written now

#2680's guard pinned the header inside firstboot.sh's Caddyfile heredoc;
#2380 moved that heredoc into `start.sh --provision --site-only`, so the
merge left the test asserting a file that no longer writes a Caddyfile.
Point it at start.sh and also pin that the header is keyed on the
do-marketplace provenance, since the same function serves the doc install.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015owqMKD5QDjzZrUTF2Joht

* test(deploy): restore the two firewall guards the port-list rewrite dropped (#2380)

6104b0e deleted test_2281_firstboot_port_exposure.py because the
DOCKER-USER rule no longer enumerates ports, so the tests that policed the
list had nothing left to check. Two of its five tests were never about the
list, and they went with the file:

- ufw and iptables-persistent are never both installed. ufw Breaks
  iptables-persistent, so installing it to persist the rules makes apt
  remove ufw, and the install dies later at `ufw --force reset`.
- the firewall rules are re-applied on every boot. Without
  iptables-persistent, trinity-docker-firewall.service is the only thing
  that survives a reboot; without it every Docker-published port reopens
  after the first restart (#2281 review I1).

Both are back in test_2380_provision_single_source.py, pointed at where the
code lives now: start.sh --provision installs ufw and writes the unit as a
heredoc, so the tests read that heredoc instead of a shipped unit file. Each
was checked against a mutation of start.sh (iptables-persistent added, the
enable line removed, After=docker.service removed) and fails on each.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PdRMazCDYVoxqLFw6uKm3a

* feat(onboarding): browser admin claim, one first-run overlay, credentials without a terminal

Abilityai/trinity-enterprise#580 — marketplace admin claim
A DigitalOcean 1-Click first boot no longer generates an admin password. With
no user-data password, ADMIN_PASSWORD stays blank and ADMIN_PASSWORD_SOURCE=
browser tells start.sh that is deliberate, so the first browser visitor creates
the admin at /setup (email, password, product-updates consent). The MOTD prints
the URL to claim, never a password. An operator-supplied password (cloud-init
user-data, including trinity-do-create.sh) still pre-provisions the admin and
keeps the wizard closed. The prod/hosted compose files move from `:?` to `?`:
an unset password still refuses to render; an explicitly blank one is the
claim path. The backend already handled a blank password; only docstrings
change. The accepted risk (the window before the first visit) is written down
in DEPLOYMENT.md -> Security Recommendations.

Abilityai/trinity-enterprise#581 — one first-run overlay
Replaces the dashboard card ladder (HardeningGuide, FinishSetupCard,
FrontDeskPanel, OnboardingWizard) with one blocking, teleported overlay: a
rail over a conditional step registry (firstRunSteps.js, pure and
unit-tested), steps secure / email / claude / keys / agent / sharing, welcome
and done panels, the schematic illustrations, and TrinityMark.vue shared with
/setup. Completion is derived from existing state, and only skips and "Finish
later" persist. Unlike the spec, `keys` and `agent` never open the overlay on
their own, so an established fleet does not see it on every new browser.
ActivationChecklist stays inline. /setup logs the operator in after the claim.
Re-runnable from Settings -> General and ?onboarding=1.

Abilityai/trinity-enterprise#582 — credentials inside first-run
The Claude step (the only required step) takes a subscription token or an
API key, validates it with Anthropic before saving, and names the fix on bad
input. The first credential is connected to the agents that had none, so the
starter fleet can run immediately. Optional GitHub, Resend and Gemini keys
persist through the encrypted secret-settings path (ent#435) with admin-only
endpoints, and runtime reads resolve the saved setting before the env.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PdRMazCDYVoxqLFw6uKm3a

* fix(onboarding): close the claim paths review found outside the marketplace

Pre-landing review and a diff-scoped security audit of the previous commit.

Claim path (Abilityai/trinity-enterprise#580):
- docker-compose.prod.yml is back to `${ADMIN_PASSWORD:?}`. Neither start.sh
  nor the marketplace uses it, so relaxing it only let a source build that
  forgot its password boot with an open /setup.
- docker-compose.hosted.yml passes ADMIN_PASSWORD_SOURCE (default `unset`)
  to the backend, and /api/setup/admin-password refuses a blank-password
  claim unless the source is `browser`. The #2381 existing-admin check
  still runs first; the dev compose (variable absent) keeps its wizard.
- start.sh decides a blank password with env_value, so `ADMIN_PASSWORD=""`,
  `''` or a trailing space no longer count as set, and writes a generated
  password with set_env_key.

First credential (Abilityai/trinity-enterprise#582):
- Agents come from the DB rather than a Docker listing that swallows
  errors. Agents with a successful execution are skipped, so fleets that
  authenticate per agent keep their own key. Agents with a running
  execution are not restarted. The responses carry an int
  `connected_agents`, and seeding re-runs the idempotent connect so agents
  created during the save are not missed.
- Gemini-runtime agents resolve the saved Gemini key before the env.
- subscriptions router gains its `# mcp:` header.

Overlay (Abilityai/trinity-enterprise#581):
- Auto-login after /setup uses the claimed email, not a hardcoded 'admin'.
- The Claude step reports the real connected count and says a re-paste
  replaces the saved credential.
- setup_started is recorded only when the overlay opens on its own and the
  claude or agent step applies.
- Exports orphaned by the absorbed components are deleted; docs that still
  described them now point at FirstRunOverlay / firstRunSteps.js.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PdRMazCDYVoxqLFw6uKm3a

* chore(release): 0.9.5-rc3

Release candidate cut from feature/ent580-582-first-run so the
marketplace browser-claim, the first-run overlay and the in-browser
credential steps (Abilityai/trinity-enterprise#580, #581, #582) can be
tested on a real 1-Click snapshot before their PR opens. rc1 and rc2 were
cut from feature/2380-do-provision the same way; rc2 stays immutable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PdRMazCDYVoxqLFw6uKm3a

* fix(deploy): snap doctl cannot read /tmp, and an apostrophe kills first boot (#2380) (#2683)

* fix(deploy): snap doctl cannot read /tmp, and an apostrophe kills first boot (#2380)

Two fixes to the DigitalOcean installer, found QA'ing it across macOS and Linux.
Neither is visible on the machine it was written on.

**A snap-installed doctl has a private /tmp.** On most Linux distributions doctl
comes from snap, and a snap runs in its own mount namespace — a file written to
the real /tmp is not there when doctl opens it, so it fails on a file that
demonstrably exists:

    Error: open /tmp/trinity-user-data.XXXXXX: no such file or directory

Snap's `home` interface can read non-hidden paths under $HOME, so a snap doctl
gets the file there instead; everyone else keeps TMPDIR. The name stays visible
because that interface denies dotfiles too, `umask 077` still makes it 0600, and
the EXIT trap still removes it. Reported by dolho, who hit it on Linux. The
`mktemp -t` half of his finding is already on this branch (aaa3de5).

A snap doctl with no usable $HOME now fails at that point with an explanation,
rather than at the droplet-create call with a path error, after both prompts.

**A single quote in either secret breaks the droplet's first boot.** Both are
interpolated into single-quoted assignments in the user-data, so a `'` inside one
closes the string early:

    export ADMIN_PASSWORD='Tr0ub4dor's!Horse'
    bash: unexpected EOF while looking for matching `''

Not a hypothetical input — the password rules require a special character and `'`
is one, so that password passes this script's own check and the backend's. The
failure is expensive out of proportion to the typo: the droplet is created and
billing before first boot runs, the operator watches a 15-minute progress bar end
in a timeout, and the cause is invisible without reading the install log. Both
secrets now go through `_shquote` (the portable `'\''` idiom).

Verified on both platforms rather than reasoned about — macOS bash 3.2 and
ubuntu:24.04 bash 5.2 / GNU coreutils 9.4, driving the real prompt flow with a
stubbed doctl and diffing the generated user-data:

- `mktemp -t trinity-user-data`  → macOS OK, Linux `too few X's in template`
- the full template              → both OK, mode 0600
- snap-detection truth table     → identical on both
- password round-trip            → 7/7 both platforms, including apostrophes,
                                   backticks, backslashes, tabs and non-ASCII;
                                   the unfixed script fails the apostrophe cases
                                   with a syntax error

`tests/unit/test_2380_installer_portability.py` pins all three. The round-trip
guard is executed, not pattern-matched — it runs the script's own `_shquote` and
lets a shell parse the result back, so it catches any breakage rather than the
one spelling that was wrong.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011sj48trT3XmyaSus4GUtXB

* test(deploy): the installer's default tag must track VERSION (#2380)

`trinity-do-create.sh` hardcodes the release it installs:

    TRINITY_IMAGE_TAG="${TRINITY_IMAGE_TAG:-v0.9.5-rc2}"

That default IS what operators get — the guide tells them to run the script
straight off a tag with no environment set. So on the day `v0.9.5` is cut, a
script fetched from the v0.9.5 tag still clones and pulls `v0.9.5-rc2` unless
someone remembers this one line.

Nothing catches it today. It is not a syntax error, the rc2 images still exist
and still pull, and the install SUCCEEDS — it just installs the previous release
candidate. The operator cannot tell, and neither can CI: the release checklist's
"VERSION match" step compares the VERSION file to the tag, not this script to
either.

The guard ties them together. It passes now (VERSION `0.9.5-rc2`, default
`v0.9.5-rc2`) and fails the moment VERSION is bumped for the cut — which is
exactly when someone needs telling. Verified by mutation: setting VERSION to
`0.9.5` produces

    AssertionError: trinity-do-create.sh installs v0.9.5-rc2 but VERSION says 0.9.5.

A second case pins the `v` prefix. One string feeds both `git clone --branch`
(needs the git ref) and `docker pull` (published under both spellings), so only
the `v` form works for both — the #2471 failure, where `v0.9.5-rc1` shipped
without its v-prefixed image alias and left no tag value able to build the
marketplace snapshot.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011sj48trT3XmyaSus4GUtXB

* fix(deploy): refuse before the prompts, quote the tag, and test the installer by running it (#2380)

Addresses review I1-I3 on #2683.

I1: the snap-doctl-without-$HOME refusal ran after both secret prompts,
breaking the script's own "checks first, questions second" rule. The
USER_DATA_DIR decision moves into the pre-flight block beside
`doctl account get`; mktemp stays where it was.

I2: TRINITY_IMAGE_TAG was the third single-quoted interpolation in the
user-data heredoc and the only unquoted one. It now goes through _shquote,
and the static guard asserts the rule (every '${...}' in the heredoc is a
_Q value) instead of two spellings of it.

I3: the guards grepped for the quoted spelling and never ran the script.
New tests run the real installer against a stub doctl (package and snap
paths, apostrophes and shell metacharacters in the password, token and
tag), then `bash -n` the captured user-data and evaluate its real export
lines and subscription call in context, and assert a snap doctl without
$HOME is refused before any question.

Running it found a fourth defect: under `set -u`, bash < 4.4 (macOS's
/bin/bash is 3.2) treats an empty "${SSH_ARGS[@]}" as unbound, so an
account with no SSH keys died at "Creating the droplet...", after both
secrets were typed. Expanded as ${SSH_ARGS[@]+"${SSH_ARGS[@]}"}.

Each fix was checked by reverting it: the tests fail on every revert.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PdRMazCDYVoxqLFw6uKm3a

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(deploy): describe the provisioning path, installer, firewall and TLS gate (#2380)

/validate-pr on #2707 found the branch shipping new capability without the
requirements and feature-flow updates CLAUDE.md requires. Written from the
code at this commit, not from the PR body.

Requirements (docs/memory/requirements/infrastructure.md):
- Corrected: HOST-010 and the section 8.10 description, PROV-006 (flags now
  carry hardening_guide_eligible), PROV-009 (the second path has been a
  Cloudflare Tunnel since #2564, not a VPN), PROV-010 (a two-stage card
  gated on hardening_guide_eligible, with a domain field), PROV-011 (start.sh
  --provision now writes TRINITY_INSTALL_SOURCE).
- Added: PROV-012 one provisioning implementation, three callers; PROV-013
  the portless container firewall and the metadata-service block; PROV-014
  the prompting installer and do-script provenance; PROV-015 the on-demand
  TLS ask gate.

Feature flows: install-provenance (two-stage card, do-script,
hardening_guide_eligible, the domain field and ask gate; fixes both stale
claims in #2593), hosted-install (a Provision Layer section), telemetry-sharing
(the IntersectionObserver gate from #2593), and their index rows. Stale
sentences in agent-lifecycle.md, backend.md, api-endpoints.md and
DEPLOYMENT.md corrected.

Fixes #2593

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PdRMazCDYVoxqLFw6uKm3a

* chore(deploy): drop references to set-domain.sh, which no longer exists (#2380)

0423e6c replaced scripts/deploy/set-domain.sh with a Settings field, but
two comments still named it as env-file.sh's second caller, and the
provisioning test kept an unused path constant pointing at it. start.sh is
now the only script that sources env-file.sh; the comments say so.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PdRMazCDYVoxqLFw6uKm3a

* fix(subscriptions): run the narrowed first-credential connect before the #2572 sweep

The merge put dev's #2572 credential-less sweep ahead of ent#582's
connect_agents_to_first_credential. The sweep has no notion of an agent that
already authenticates some other way, so running first it would assign and
restart agents ent#582 deliberately skips (any agent with a successful
execution, and any agent mid-execution), and leave the first-run step
reporting `connected_agents: 0` because nothing was left to connect.

Swapped: the narrow pass runs first and reports its count, and the sweep then
picks up whatever is genuinely credential-less.

Also rewrites the first-run section of the setup guide, which #2746 synced
while describing the card stack ent#581 retires ("at most one first-run card",
"Not now snoozes for two weeks").

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PdRMazCDYVoxqLFw6uKm3a

* fix(ci): linear from-address parser (CodeQL ReDoS) and stub get_gemini_api_key in telegram backfill test

- platform_keys_service.from_address_domain: replace the single regex that
  CodeQL flagged as py/polynomial-redos with partition + single-class
  fullmatch; same accept/reject set, pinned by a parametrized test and a
  hostile-input timing check.
- test_telegram_webhook_backfill: the settings_service stub lacked
  get_gemini_api_key, which routers/settings.py now imports (ent#582).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011y53gTcB3fTAHhJc3xxpkx

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Oleksii Dolhov <oleksii.dolhov@gmail.com>
dolho added a commit that referenced this pull request Sep 15, 2026
… 903 dev lines into the split packages

The modify/delete conflicts on `routers/settings.py` and
`services/git_service.py` are resolved by DELETING dev's monolith copies and
re-porting every hunk dev added to them since the fork into the file that
now owns it, symbol by symbol, with each function's body checked equal to
dev's modulo package qualification:

git_service (one dev commit, ent#615 / #2757 — the fleet-PAT fix):
  - `_AUTH_PATTERNS` marker            -> conflicts.py
  - `_git_remote_url` removed, `_remote_seturl_subcommand` docstring,
    `_credentialless_remote_url`, `rebind_origin_and_push` (root push +
    credential in the exec env), `update_remote_pat` (env write, not URL)
                                        -> remotes.py
  - the credential-helper install + embedded-token sweep block
    (`write_container_github_pat`, both alarms, `scrub_git_remote_tokens`,
    the fleet sweep, `spawn_git_remote_token_scrub`, all `_SCRUB_*`)
                                        -> NEW token_scrub.py (remotes.py
    would otherwise sit at 821 lines, over the threshold the split exists for)
  - `_agent_can_push`, `_agent_has_write_credentials` docstring,
    `sync_to_github`, `reset_to_main_preserve_state`   -> sync.py
  - `initialize_git_in_container` (seeds before writing a remote)
                                        -> provisioning.py
  Package `__init__` re-exports every new name; the duplicate
  `REBIND_PUSH_TIMEOUT_S` the hunk would have introduced is dropped.

settings (five dev commits — #2715, #2619, #2707, #2741, #2739):
  - 11 changed routes replaced in place across flags/credentials/
    integrations/generic
  - 11 new symbols placed beside their dev-order predecessors; the #2715
    Resend/Gemini routes + their two helpers go to NEW provider_keys.py
    (credentials.py would otherwise reach 1,045 lines), included on the
    package router right after `credentials` and before `generic`
  - `_ANTHROPIC_KEY_ALIASES` / `_adopt_after_instance_key_removed` reached
    from generic.py through the sibling module object, per the package rule

ops: `_format_model_name`'s #2739 `claude-fable-5-1` entry lands in
`ops_costs_service.py`, where the split moved the function; the #2726
test imports from there.

Dev's tests that patch monolith attributes are re-pointed the way the
split re-pointed every earlier one: the ent#615 exec recorder is installed
on each execing sibling and `_detect_git_dir` on `gitignore`; #2572's `db`
fake on `credentials` and `generic`; ent#553's source read on `flags`;
#1677's emitter allowlist and the ent#615 source reads on `token_scrub`.
`_PRE_SPLIT_ROUTES`' post-split allowlist records the six #2715 routes; the
git_service import-surface pin drops `_git_remote_url` (gone by design) for
its ent#615 replacements.

Content conflicts: `backend.md` (dev's facts under the package names),
`test_ent123_tokenless_clone.py` (dev's helper patch, on `gs.sync`).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VpvcfgWkmQPD7DrDLmATTf
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants