release(0.5.107): verify-FFI fold (#232) + canonical boot-prime (#238) — adopt persist v15.1.2 / verify v9.0.2 / edge v10.1.2 - #240
Merged
Conversation
… — adopt persist v15.1.2 / verify v9.0.2 / edge v10.1.2 Substrate triple: persist v15.1.1->v15.1.2, verify v9.0.1->v9.0.2, edge v10.1.1->v10.1.2. ## CIRISServer#232 — the agent now RIDES US for verify ciris-verify-ffi is FOLDED into `ciris_server._native.so`, so the ~84 `ciris_verify_*` C symbols live in THIS wheel's .so — there is no separate libciris_verify_ffi.so. The folded FFI tag == the bundled substrate tag, so version skew is impossible by construction. Unblocks CIRISAgent#917 dropping the standalone `ciris-verify` pin (kills the v9.0.0-vs-v5.1.3 wire/canonical-hash skew). Mechanically this needed two upstream pieces, both now shipped: - verify v9.0.1 (#187/#188) added `rlib` to ciris-verify-ffi's crate-type, so we can link it at all. - verify v9.0.2 (#189) added `ciris_verify_ffi_link_anchor()`. Nothing in our Rust calls the FFI (the agent reaches it via ctypes at runtime), so the linker's --gc-sections would dead-strip every symbol out of the cdylib — PER-PLATFORM- SILENTLY, the classic cargo-green-but-artifact-broken trap. We reference the anchor from a `#[used]` static + a black_box call in `_native` init; the anchor takes the address of every export, which the compiler cannot resolve without keeping the symbol. Gated behind the `python` feature (only the wheel needs it, not the headless bin). `verify_ffi_path()` (python/ciris_server/__init__.py) returns `_native.__file__` for the agent's ffi_bindings ctypes loader. VERIFIED ON THE ACTUAL ARTIFACT (never trust cargo-green): `nm -D` the release cdylib lists all 85 symbols — anchor + jcs_canonicalize, kex x4, wrap/unwrap_dek, self_enc x3, locale_merkle, attestation, create_federation_identity. A new Verify-FFI fold smoke runs on all three desktop wheel legs (linux/macOS/windows): it pip-installs the built wheel, ctypes-CDLLs verify_ffi_path(), and asserts the symbols resolve — so a platform-specific strip fails the wheel job instead of shipping silently. ## CIRISServer#238 — canonical boot-prime (no more waiting on a slow announce) prime_trusted_peers primes from Rooted transport_destination ROWS, but the baked canonical seed carries only a KeyRecord + an IP dial hint and has no such row — so the canonical was NEVER boot-primed and rooted only via its slow/unreliable announce (~130-200s, or never). An explicit-hash peer needs just (dest_hash, ed25519), and BOTH are deterministically derivable from the fed Ed25519 pubkey the seed already carries: dest_hash = reticulum_destination_for_pubkey(fed_ed25519) = sha256(fed_ed25519)[..16] signing_key = that same fed Ed25519 — transport and federation share the Ed25519 signing half (the v10.1.0 split was the X25519 half, which priming never uses; inject_rooted_peer_for_test takes no X25519 at all). New compose::prime_canonical_bootstrap_peers looks up each canonical's KeyRecord (lookup_public_key), derives both, and inject_rooted_peer_for_test -> deterministic root at boot, no announce dependency, no seed change. Runs in BOTH the standalone node and the agent fold. CIRISPersist#428 closed as wrong-premise (I first thought this needed the transport Ed25519 baked into the seed; it does not — the key is derivable from the fed key). ## Build-memory guardrail `cargo test` links the ~32 integration-test binaries in tests/*.rs, each statically linking the FULL substrate. At the dev/test default of full DWARF that is ~2-3 GB peak RSS per link x `nproc` (32) parallel jobs = tens of GB over the 31 GiB of RAM; the kernel swap-THRASHES rather than OOM-killing, and the machine goes unresponsive. This actually happened. [profile.dev]/[profile.test] now pin `debug = "line-tables-only"` — keeps file:line in panic backtraces while cutting per-link RSS ~5-10x. CI-safe (lighter/faster links, parallelism unchanged), unlike a `[build] jobs` cap which would over-subscribe a 4-core runner. Debugger override is documented inline. Full suite 309/309 (0 failed, 13 ignored, 35 suites); clippy (default + python), fmt, and the release-gate pin gate green at the new triple. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018tDoby6hToRTt7of4y4pcG
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
…l), not just at release build-wheels.yml is workflow_call-only — invoked by publish-pypi at TAG time — so the #232 fold smoke added there would have first executed during the PyPI publish. That is the worst place to discover a bug in the smoke script itself. The PR/push wheel leg actually lives in conformance.yml, so mirror the assertion there: every PR now pip-installs the built linux wheel, ctypes-CDLLs verify_ffi_path(), and asserts the ciris_verify_* surface resolves. build-wheels.yml keeps the same check for the macOS/Windows legs (exercised on a main push via warm-release-cache, i.e. still BEFORE any tag), which is where a platform-specific --gc-sections strip would show up. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018tDoby6hToRTt7of4y4pcG
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Substrate triple: persist v15.1.1→v15.1.2 / verify v9.0.1→v9.0.2 / edge v10.1.1→v10.1.2.
CIRISServer#232 — the agent now rides us for verify
ciris-verify-ffiis folded intociris_server._native.so, so the ~84ciris_verify_*C symbols live in this wheel's.so— there is no separatelibciris_verify_ffi.so. The folded FFI tag == the bundled substrate tag, so version skew is impossible by construction. Unblocks CIRISAgent#917 dropping the standaloneciris-verifypin (kills the v9.0.0-vs-v5.1.3 wire/canonical-hash skew).Two upstream pieces made it mechanically possible, both now shipped:
rlibtociris-verify-ffi's crate-type, so we can link it at all.ciris_verify_ffi_link_anchor(). Nothing in our Rust calls the FFI (the agent reaches it viactypesat runtime), so--gc-sectionswould dead-strip every symbol out of the cdylib — per-platform-silently, the classic cargo-green-but-artifact-broken trap. We reference the anchor from a#[used]static + ablack_boxcall in_nativeinit; the anchor takes the address of every export, which the compiler cannot resolve without keeping the symbol.Gated behind the
pythonfeature (only the wheel needs it, not the headless bin).verify_ffi_path()returns_native.__file__for the agent'sffi_bindingsctypes loader.Verified on the actual artifact (never trust cargo-green):
nm -Don the release cdylib lists all 85 symbols — anchor +jcs_canonicalize,kex×4,wrap/unwrap_dek,self_enc×3,locale_merkle,attestation,create_federation_identity. A new Verify-FFI fold smoke runs on all three desktop wheel legs (linux/macOS/windows): it pip-installs the built wheel,ctypes.CDLLsverify_ffi_path(), and asserts the symbols resolve — so a platform-specific strip fails the wheel job instead of shipping silently.CIRISServer#238 — canonical boot-prime
prime_trusted_peersprimes from Rootedtransport_destinationrows, but the baked canonical seed carries only a KeyRecord + an IP dial hint and has no such row — so the canonical was never boot-primed and rooted only via its slow/unreliable announce (~130–200s, or never).An explicit-hash peer needs just
(dest_hash, ed25519), and both are deterministically derivable from the fed Ed25519 pubkey the seed already carries:Transport and federation share the Ed25519 signing half — the v10.1.0 split was the X25519 half, which priming never uses (
inject_rooted_peer_for_testtakes no X25519 at all).New
compose::prime_canonical_bootstrap_peerslooks up each canonical's KeyRecord, derives both, and injects a Rooted peer → deterministic root at boot, no announce dependency, no seed change. Runs in both the standalone node and the agent fold. CIRISPersist#428 closed as wrong-premise (I first thought this needed the transport Ed25519 baked into the seed; it does not).Build-memory guardrail
cargo testlinks the ~32 integration-test binaries intests/*.rs, each statically linking the full substrate. At the dev/test default of full DWARF that's ~2–3 GB peak RSS per link ×nproc(32) parallel jobs = tens of GB over 31 GiB of RAM — the kernel swap-thrashes rather than OOM-killing, and the machine goes unresponsive. This actually happened during this work.[profile.dev]/[profile.test]now pindebug = "line-tables-only"— keepsfile:linein panic backtraces while cutting per-link RSS ~5–10×. CI-safe (lighter/faster links, parallelism unchanged), unlike a[build] jobscap which would over-subscribe a 4-core runner. Debugger override documented inline.QA
nm -Dfold check on the real release cdylib — 85/85 symbols presentCloses #232. Closes #238.