Repository navigation
Conversation
With at_rest.mode encrypt or strict, each database_artifact checkpoint and WAL delta is stored as TCE1 ciphertext bound to its service, space, name and plaintext content (and, for a delta, its revision). Mode off stores exactly what it did before. - Migration m20261107_010000_artifact_payload_encoding adds nullable payload_encoding and delta_encoding (NULL = legacy plaintext). - Sizes and content hashes stay those of the plaintext, so quota and the growth check are unchanged. - Writes seal with the space's current key, prepared before the SQLite writer lock, or fail (503); never a plaintext fallback. - Reads decide legacy vs sealed from the column, open the stored bytes one 64 KiB segment at a time (at_rest::SegmentOpener) and hash the plaintext. Integrity, unsupported format and legacy-in-strict are 500. - The node's artifact repository reads the at-rest runtime and declares the SQL seal path. DuckDB gains an Unavailable (503) error. - The offline fingerprint tool refuses sealed rows without keys.
…oding columns A local snapshot from before m20261107_010000_artifact_payload_encoding has no payload_encoding/delta_encoding columns; read its parts as plaintext instead of failing the query.
… both seal The filesystem-node encrypt tests run again, the ceremony test checks that a verified escrow turns encryption on in encrypt and strict, and INTERFACES records that both seal paths are in.
samgbafa
force-pushed
the
skgbafa/tc-887-r7-sql-sealed
branch
from
November 8, 2026 01:06
4172c08 to
b69f29a
Compare
samgbafa
changed the base branch from
skgbafa/tc-887-r3-node-wiring
to
skgbafa/tc-887-r6-s3-sealed
November 8, 2026 01:06
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TC-887: seal SQL and DuckDB artifacts at rest.
What and why
Every durable SQL and DuckDB byte is a
database_artifactrow:payload(checkpoint) anddelta_payload(WAL delta). This change stores both as TCE1 ciphertext whenever the node runsat_rest.mode = "encrypt"or"strict", and declares the SQL seal path the node needs before it accepts those modes. The node still refuses them until KV block storage seals too (separate stack).m20261107_010000_artifact_payload_encoding: nullable SMALLINTpayload_encodinganddelta_encoding(SQLite and PostgreSQL).NULL= legacy plaintext, so existing rows read exactly as before.ArtifactMetaandload_metacarry them.SqlSizes, quota and the growth check are unchanged. They are part of the AEAD identity (SqlCheckpoint,SqlWalwith the revision the delta is written at).save,save_delta): modeoffstores plaintext exactly as before. Other modes seal withprovider.current_key(space)or fail withDatabaseArtifactError::AtRestUnavailable, never plaintext.current_keyruns before the SQLite writer lock (it may create the key, which writes the key store); sealing runs inspawn_blocking. Each part's encoding column is written with that part.read_partslices the stored length (ciphertext_len(size)) in the existing 4 MiBsubstrslices and opens it one 64 KiB segment at a time (new push-styleat_rest::SegmentOpener). The blake3 hash check runs on the plaintext, and sinks get plaintext, soCacheFileSinkstill checks the SQLite/DuckDB header. Memory stays bounded to one segment beyond what the sink keeps.SqlError::Unavailable/ newDuckDbError::Unavailable(503). Integrity failures, unknown keys,Unsupported(header or column 2..=9) and legacy-in-strict→ 500. Never a plaintext fallback, never re-sealed or skipped.node_artifact_repositorytakes theArc<AtRestRuntime>(AtRestSource), wires it in withwith_at_rest, and declaresSealPath::SqlArtifacts.database_migration::offline_inventorytakes an optional provider.tinycloud-sql-identity offline-fingerprintpasses none, so it refuses sealed rows instead of fingerprinting ciphertext. A snapshot taken before the encoding columns existed (the tool never migrates) is read as plaintext.docs/at-rest/INTERFACES.md§8 (columns, plaintext metadata, identity, stored length, errors, rollback). CHANGELOG[Unreleased].This is new work; it replaces no earlier PR. The KV block-store files are untouched, so this can be rebased onto the KV stack. Out of scope: KV blocks, the harness RED→GREEN flip, and the inert
sql_store.Notes for review
at_rest/keyring/tests.rs::migration_down_refuses_to_drop_keysassumed the key-table migration was the newest one. It now rolls back down to and including that migration, however many come after it.metadata_and_absence_never_select_payload_columnsnow checks the quoted"payload"/"delta_payload"identifiers, sincepayload_encodingis a metadata column.Rollback: a build without this change ignores the encoding columns. It cannot hydrate a sealed part (the header check fails), and a blind full save it makes over a sealed row (DuckDB
import_db) would leavepayload_encoding = 1on plaintext. Covered in INTERFACES §8.tests/sql_identity_cli.rsinserts its pre-migration fixture row without the new columns (exec_without_returning), as a snapshot of that age has none.Verification (run locally)
PostgreSQL tests ran against a private local PostgreSQL 14 (
TINYCLOUD_TEST_POSTGRES_URL,TINYCLOUD_ALLOW_LOCAL_PG=1).Results: all pass.
routes::tests::tc1799_two_proof_sql_invocation_is_refusedfailed once in the filtered node run (a process-wide counter moved by a concurrent test) and passes on its own; it is unrelated to this change.New tests:
database_artifacts::tests::sealed_parts_hold_no_plaintext_and_hydrate_exactly(SQLite): TCE1 bytes with no SQLite/WAL header or marker, correct columns, plaintext sizes and hashes, an 8 MiB+ checkpoint read in 4 MiB slices across segment boundaries back to the exact bytes through a header-checking cache sink, sealed parts still read inoffandstrict, empty database.database_artifacts::tests::postgres_sealed_parts_round_trip_and_refuse_a_moved_part(PostgreSQL): the same round trip, plus a checkpoint moved from another space refused as integrity.database_artifacts::tests::off_writes_plaintext_and_legacy_parts_read_unless_strict:offstores the bytes as given, legacy reads inencrypt, a sealed delta on a legacy checkpoint,strictrefuses legacy, and a full save seals or un-seals by mode.database_artifacts::tests::without_unlocked_keys_writes_fail_and_change_nothing: gate closed, no keyring or locked keys →AtRestUnavailableand the row is unchanged; sealed reads with locked or missing keys refuse.database_artifacts::tests::tampered_or_moved_sealed_parts_are_integrity_errors: flipped body or header byte, a part moved from another space or name, an older revision's WAL, encoding 2 (unsupported), 0 (invalid), andNULLon sealed bytes. All are errors and nothing stays staged.sql::service::tests::sealed_sql_database_hydrates_a_cold_cacheandduckdb::service::tests::sealed_duckdb_artifact_hydrates_a_cold_cache: end-to-end through the services with a cold cache; locked keys →Unavailable.database_migration::tests::offline_fingerprint_opens_sealed_rows_only_with_keys,at_rest::tests::pushed_slices_open_like_the_whole_buffer.Restacked onto the KV stack (base now #520). With both seal paths in, the node boots
encryptandstrict:tests/at_rest_fs_node.rsrun again (they were ignored in feat(TC-887): the FileSystem block store seals KV blocks #519);at_rest_ceremonynow checks that a pinned, verified escrow turns encryption on inencryptandstrict, and thatencryptwithout escrow orescrow_noneis refused by the gate;Local, after the restack:
cargo test --locked -p tinycloud-node --test at_rest_ceremony --test at_rest_fs_node --test at_rest_fs_store --test at_rest_env_root --test at_rest_env --test did_resolution_single_entryall pass. The at-rest harness withTC887_AT_REST_MODE=encrypt TC887_AT_REST_ROOT_SOURCE=envreports KV and SQL sealed (see #522).