[samgbafa]

TC-887. Third PR of the refreshed at-rest stack (base: #514). Replaces #327 and adds a root from the environment.

From #327 (c1a040e), rebased onto main

• at_rest config: mode, escrow, escrow_none.
• Keyring setup after migrations, and the gate at boot.
• The key mirror (_tc/at-rest/ in the block bucket, {datadir}/at-rest on disk). The S3 mirror is ported to aws-sdk-s3 1.x.
• The /at-rest/* routes and the /info at_rest block.
• docs/at-rest-escrow.md.
• The ceremony integration tests.

New

• at_rest.root_source = "escrow" | "env" (default escrow).
  • escrow is feat(TC-887): node wiring for at-rest keys and escrow recovery #327's path unchanged: the node root, with the escrow gate.
  • env seals the ARK under a 32-byte key from TINYCLOUD_AT_REST_ROOT_KEY (standard base64, 44 characters, canonical), with no escrow.
  • Everything below the ARK is the same in both modes (data keys, the DekId in TCE1 headers, object names, mirrors).
• TINYCLOUD_AT_REST_PREVIOUS_ROOT_KEY: the root the stored rows were sealed under. The keyring re-seals them under the current root (open_with_previous_root, feat(TC-887): at-rest keyring, key store and migration #514). This covers both changing an env key and moving from an env root to the node root. Data keys and data are not rewritten.
• Fail-closed boot. The node refuses to start on:
  • a missing, malformed or one-byte-repeated key (the error never contains the value);
  • a root key set while root_source = "escrow";
  • a previous root equal to the current one, or set while at-rest is off;
  • an env root combined with any escrow setting;
  • an env key that doesn't open the stored keys. There is no escrow to recover with, so the node refuses rather than booting locked.
• TEE checks for an env root. The node refuses to start unless:
  • the measured compose sets TINYCLOUD_AT_REST__ROOT_SOURCE to the literal env;
  • no root key is a literal in the compose;
  • the measured app-compose does not allow DSTACK_AUTHORIZED_KEYS;
  • the guest agent reports a pinned production dstack OS image hash (PRODUCTION_OS_IMAGE_HASHES, dstack-0.5.12 today; dev images are refused).
• Fingerprint. hex(SHA-256("tinycloud/at-rest/root-fingerprint/v1" ‖ key))[..16] goes in the boot log and in /at-rest/status (root_source, root_fingerprint). It can be computed offline; the key itself never appears.
• tinycloud_core::at_rest::{AtRestMode, AtRestView, AtRestSource}. Storage reads a fresh view per operation, and AtRestRuntime implements it.
• Two seal paths. encrypt/strict stay refused until both KV blocks and SQL artifacts declare a seal path (SealPath::{Blocks, SqlArtifacts}), so this PR alone can't turn encryption on.
• Deploy preflight. --validate-config runs the same root-key checks as boot, without reading the node root, so a missing or malformed key fails the preflight.
• Config loading. The root keys are read with std::env only and never enter Config. TINYCLOUD_AT_REST_ROOT_KEY uses a single underscore on purpose: figment's __ form would map into at_rest and fail deny_unknown_fields.

Tests

• tests/at_rest_env_root.rs (one process, real app). It covers:
  • refusals;
  • provisioning, with the fingerprint shown in status;
  • a wrong key is refused;
  • re-sealing under a new key, after which the old key no longer opens the rows;
  • moving to the node root.
• Unit tests:
  • key parsing (errors never echo the value);
  • root resolution;
  • measured-compose and OS-image checks, including dstack-dev-0.5.12;
  • os_image_hash parsing from Info.
• tests/at_rest_env.rs checks the root-key variables don't reach the config loader.

Verification

• Real binary: --validate-config rejects a missing key, a malformed key, and a key set with root_source = escrow. A boot with an env key serves /at-rest/status with the fingerprint you get from computing it offline. A restart with another key refuses to start and names that key's fingerprint, and no key bytes appear in the log.
• cargo test --locked -p tinycloud-node --lib -- at_rest dstack
• cargo test --locked -p tinycloud-node --test at_rest_env --test at_rest_ceremony --test at_rest_env_root
• cargo clippy --locked -p tinycloud-node -p tinycloud-core --all-targets with -Dwarnings, and cargo fmt --all -- --check: clean.