Skip to content

image: SSH host keys from the boot's first dispatch; generators and inert units measured and left out - #247

Closed
aledbf wants to merge 2 commits into
mainfrom
cold-boot-batch
Closed

aledbf wants to merge 2 commits into
mainfrom
cold-boot-batch

Conversation

@aledbf

@aledbf aledbf commented Nov 13, 2026

Copy link
Copy Markdown
Member

Three image-side changes for the cold boot to the first SSH login, measured one by one and together against the image as built, judged on the first login over vsock.

Fleet (task boot:ssh-first, lab run 43483372488, release v20261113.02, 20 boots of each, interleaved, 2 vCPUs)

row sshd accepts p50/p95 first login p50/p95 accept to login p50/p95
as built 309/334 489/506 177/194
keys early 315/325 468/494 154/168
generators masked 306/314 489/501 180/192
inert units masked 306/316 484/501 178/191
batch (all three) 311/321 469/490 156/173

No row lost a login (0/20 each). Keys early alone had already measured 486 to 468 and 177 to 155 in lab run 43463472762. Only the keys moved the login, so this PR ships them and leaves the other two out.

Per item

  1. Keys early: shipped. spin-machine-sshd-keygen.service now has DefaultDependencies=no, Conflicts=/Before=shutdown.target and WantedBy=sysinit.target. optimize-systemd.sh enables it with a link in sysinit.target.wants. The sshd@.service.d/keygen.conf ordering stays. build.sh asserts the link and DefaultDependencies=no on the finished filesystem. In the built image, the keygen starts before sysinit.target and is done by then, and no unit fails. boot:ssh-first and boot:logind pass on it locally. One consequence for consumers: a guest checkpointed after boot now always carries host keys, so the restorer has to hand each VM its own. That was already true whenever anything had logged in before the checkpoint. image/README.md says this.
  2. Generators: left out. The image runs four system generators: sshd-socket-generator, systemd-getty-generator, systemd-run-generator and systemd-ssh-generator. The rest are already masked. sshd-socket-generator is the slowest: it loads libcrypto and writes nothing, because sshd_config is at its defaults. In a guest it took 3.5-23.5 ms, against about 1.5 ms for each systemd generator. The generators run in parallel, though. Masking it and systemd-run-generator took the generators phase from 5.3 to 3.7 ms p50 (laptop, 8 boots each), and on the fleet the login did not move (489). The fleet's generators phase is 6.5 ms. The ~44 ms before it is PID 1's own setup, not generators.
  3. The ~29-32 ms gap: explained, not removable. After UnitsLoadFinish, no unit runs for about 32 ms p50 on the fleet (lab run 43361486792). That time is PID 1 itself. systemd stamps UnitsLoadFinish after it enumerates mounts, devices and slices, and only then loads the boot's units: coldplug, then Activating default unit: default.target. That parses every unit file default.target reaches, plus a stub for every name an After=/RequiresMountsFor= gives. It then builds the initial transaction (64 jobs) and garbage-collects the stubs before it dispatches the first job (systemd-ask-password.socket). This comes from task boot:trace with systemd.log_level=debug, which ends in "Loaded units and determined initial transaction". The gap only shrinks along with the transaction. Masking the 11 jobs whose condition fails on every boot here (pcr*, kmod-static-nodes, apparmor, the distribution's sshd-keygen, getty-static, getty@tty1) did not move the login (484). The explanation is now a comment in boot/waits_probe_test.go.

boot/ssh_first_test.go keeps one row, "as this branch", which applies keys early to older releases too. The experiment rows are gone and their results are in its comment. The probe gains an accept-to-login column.

🤖 Generated with Claude Code

aledbf and others added 2 commits November 13, 2026 15:30
…three together (experiment)

Each image change of this branch as an overlay edit beside the image as built, and an
accept-to-login column: what the first login costs once sshd is listening.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
spin-machine-sshd-keygen.service is wanted by sysinit.target without default dependencies,
so the keys exist before sshd-vsock.socket listens and the first login's sshd@ no longer
waits for them before its banner. On the bench fleet, 20 boots of each interleaved (task
boot:ssh-first, lab run 43483372488): the first SSH login 489/506 -> 468/494 ms p50/p95,
accept to login 177 -> 154 ms. build.sh asserts the want and DefaultDependencies=no.

Measured beside it and left out, the conclusions kept by the probes: masking
sshd-socket-generator and systemd-run-generator (generators 5.3 -> 3.7 ms, login 489), and
masking the eleven initial-transaction units whose condition always fails here (login 484);
all three together 469. The ~32 ms with no unit running after the unit load is PID 1 loading
the default target's units and building the initial transaction.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@aledbf

aledbf commented Nov 13, 2026

Copy link
Copy Markdown
Member Author

Superseded by #248 (same change, rebased onto main after #245).

@aledbf aledbf closed this Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant