Skip to content

Add resumable storage re-encryption - #541

Draft
samgbafa wants to merge 7 commits into
mainfrom
skgbafa/tc-2713-resumable-column-rekey
Draft

samgbafa wants to merge 7 commits into
mainfrom
skgbafa/tc-2713-resumable-column-rekey

Conversation

@samgbafa

@samgbafa samgbafa commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Adds an injectable second storage-root generation and resumable, fenced storage re-encryption with durable bounded verification. Production root acquisition and serving-key configuration remain unchanged.

Deployment restriction: this PR ships ONLY as part of the TC-2713 rotation event. Do not deploy it standalone to a node holding v1 network-key or seed rows. A production-integrated runner, verified manifest and custody record, and an offline migration window must exist first; migrate and verify before resuming serving. Unmigrated v1 operations return HTTP 503 with an actionable migration_required response.

One canonical, separately hash-pinned manifest covers network and named-key identities from a dated trusted backup. VerifiedManifest validates the file hash, snapshot date, custody metadata, canonical encoding and identities; raw maps cannot bypass it. Network ceremonies are additional consistency checks. Includes a read-only local-snapshot exporter, manifest format/procedure documentation, and an offline library runner that verifies the file and pin before database access or fence activation. Production root wiring and stopping/resuming serving remain rotation-integration prerequisites.

Preflight reports missing/mismatched identities only as blocking counts. It checks both directions for snapshot network and seed identities, including allocated public identities with NULL seeds. Networks retain their exact version, public key and stored seal; seeds retain their exact network, path, version and public key. Missing snapshot rows block activation. Existing deterministic or deleted key rows may retain schema-valid NULL seeds. Post-snapshot identities require a fresh trusted snapshot; no target-row fallback exists. Activation and step helpers are crate-private; integrations use the preflighted offline runner. Client-wrap protocols are unchanged.

The exporter requires a lowercase PostgreSQL URI prefix, a single explicit local destination, a validated port and no query overrides. The designated rotation custodian separately records the digest in TC-2713 and TinyCloud Secrets for independent operator retrieval and comparison. The procedure requires restoring the exact checked archive to an isolated local database, with no tunnels/proxies/forwarding. Archive-to-PostgreSQL correspondence remains a custodian-verified operational prerequisite.

CI integration repair at 5ca6ea7a70c05c0aecb73e769e149aa1506f8ada:

  • The reported red run 42802370963 tested the synthetic merge of 18696d02 with main 23969d80. That main history introduced BarrierKeyBackend with five old trait signatures, causing E0050/E0061. The failing checks were not rerun to confirm the report.
  • Merged only prerequisite main commit 23969d80 without rebasing or force-pushing. No unrelated S0.6 tinycloud-threshold: durable share store (candidate persistence, positive-proof retirement) #539 threshold-store changes were imported. Full delta from 18696d02: three files, 396 insertions/6 deletions, primarily the existing network-create conflict handling and race regressions from Return conflict for duplicate encryption-network creation #540.
  • Updated the test adapter to forward network ID, key path and expected public key through the current backend contract. Its generation barrier and assertions remain unchanged; no test suppression or protocol fallback was added.

Current exact-source local validation:

  • cargo test -p tinycloud-core encryption_network --lib --offline: 242 passed, including SQLite/PostgreSQL network-create races and cleanup-failure behavior.
  • cargo test -p tinycloud-core reencrypt --lib --offline: 50 passed, including SQLite/PostgreSQL migration and exporter integration.
  • No failed/ignored tests or PostgreSQL skips; only owned loopback PG14 with TINYCLOUD_ALLOW_LOCAL_PG=1.
  • cargo check -p tinycloud-node --lib --offline, cargo fmt --all --check, and git diff --check: passed.
  • Private HOME/TC_HOME, approved isolated /tmp/tc2713-ci-target; owned PostgreSQL stopped and task artifacts cleaned. Shared target untouched.

Exact-head CI run 42850460988 is in progress as last independently checked by the node lead; no CI-green/readiness claim is made from the local results. Exact-head delta review is coordinated through the existing node lead.

Earlier validation at 18696d02: five Python exporter guard tests passed; at de92d462: 253 core encryption and 101 node encryption/HTTP tests passed. Those earlier broader suites were not rerun locally for this focused CI repair.

Draft; no standalone merge, deployment, restart, or production access performed.

Details: TC-2713

@samgbafa samgbafa mentioned this pull request Nov 10, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant