Skip to content

release(0.5.107): verify-FFI fold (#232) + canonical boot-prime (#238) — adopt persist v15.1.2 / verify v9.0.2 / edge v10.1.2 - #240

Merged
emooreatx merged 2 commits into
mainfrom
release/0.5.107-verify-ffi-fold-canonical-prime
Jul 12, 2026
Merged

release(0.5.107): verify-FFI fold (#232) + canonical boot-prime (#238) — adopt persist v15.1.2 / verify v9.0.2 / edge v10.1.2#240
emooreatx merged 2 commits into
mainfrom
release/0.5.107-verify-ffi-fold-canonical-prime

Conversation

@emooreatx

Copy link
Copy Markdown
Contributor

Substrate triple: persist v15.1.1→v15.1.2 / verify v9.0.1→v9.0.2 / edge v10.1.1→v10.1.2.

CIRISServer#232 — the agent now rides us for verify

ciris-verify-ffi is folded into ciris_server._native.so, so the ~84 ciris_verify_* C symbols live in this wheel's .so — there is no separate libciris_verify_ffi.so. The folded FFI tag == the bundled substrate tag, so version skew is impossible by construction. Unblocks CIRISAgent#917 dropping the standalone ciris-verify pin (kills the v9.0.0-vs-v5.1.3 wire/canonical-hash skew).

Two upstream pieces made it mechanically possible, both now shipped:

Gated behind the python feature (only the wheel needs it, not the headless bin). verify_ffi_path() returns _native.__file__ for the agent's ffi_bindings ctypes loader.

Verified on the actual artifact (never trust cargo-green): nm -D on the release cdylib lists all 85 symbols — anchor + jcs_canonicalize, kex ×4, wrap/unwrap_dek, self_enc ×3, locale_merkle, attestation, create_federation_identity. A new Verify-FFI fold smoke runs on all three desktop wheel legs (linux/macOS/windows): it pip-installs the built wheel, ctypes.CDLLs verify_ffi_path(), and asserts the symbols resolve — so a platform-specific strip fails the wheel job instead of shipping silently.

CIRISServer#238 — canonical boot-prime

prime_trusted_peers primes from Rooted transport_destination rows, but the baked canonical seed carries only a KeyRecord + an IP dial hint and has no such row — so the canonical was never boot-primed and rooted only via its slow/unreliable announce (~130–200s, or never).

An explicit-hash peer needs just (dest_hash, ed25519), and both are deterministically derivable from the fed Ed25519 pubkey the seed already carries:

dest_hash   = reticulum_destination_for_pubkey(fed_ed25519) = sha256(fed_ed25519)[..16]
signing_key = that same fed Ed25519

Transport and federation share the Ed25519 signing half — the v10.1.0 split was the X25519 half, which priming never uses (inject_rooted_peer_for_test takes no X25519 at all).

New compose::prime_canonical_bootstrap_peers looks up each canonical's KeyRecord, derives both, and injects a Rooted peer → deterministic root at boot, no announce dependency, no seed change. Runs in both the standalone node and the agent fold. CIRISPersist#428 closed as wrong-premise (I first thought this needed the transport Ed25519 baked into the seed; it does not).

Build-memory guardrail

cargo test links the ~32 integration-test binaries in tests/*.rs, each statically linking the full substrate. At the dev/test default of full DWARF that's ~2–3 GB peak RSS per link × nproc (32) parallel jobs = tens of GB over 31 GiB of RAM — the kernel swap-thrashes rather than OOM-killing, and the machine goes unresponsive. This actually happened during this work.

[profile.dev]/[profile.test] now pin debug = "line-tables-only" — keeps file:line in panic backtraces while cutting per-link RSS ~5–10×. CI-safe (lighter/faster links, parallelism unchanged), unlike a [build] jobs cap which would over-subscribe a 4-core runner. Debugger override documented inline.

QA

  • Full suite 309/309 (0 failed, 13 ignored, 35 suites)
  • clippy (default + python), fmt, release-gate pin gate — green at the new triple
  • nm -D fold check on the real release cdylib — 85/85 symbols present

Closes #232. Closes #238.

… — adopt persist v15.1.2 / verify v9.0.2 / edge v10.1.2

Substrate triple: persist v15.1.1->v15.1.2, verify v9.0.1->v9.0.2, edge v10.1.1->v10.1.2.

## CIRISServer#232 — the agent now RIDES US for verify

ciris-verify-ffi is FOLDED into `ciris_server._native.so`, so the ~84 `ciris_verify_*`
C symbols live in THIS wheel's .so — there is no separate libciris_verify_ffi.so.
The folded FFI tag == the bundled substrate tag, so version skew is impossible by
construction. Unblocks CIRISAgent#917 dropping the standalone `ciris-verify` pin
(kills the v9.0.0-vs-v5.1.3 wire/canonical-hash skew).

Mechanically this needed two upstream pieces, both now shipped:
  - verify v9.0.1 (#187/#188) added `rlib` to ciris-verify-ffi's crate-type, so we
    can link it at all.
  - verify v9.0.2 (#189) added `ciris_verify_ffi_link_anchor()`. Nothing in our Rust
    calls the FFI (the agent reaches it via ctypes at runtime), so the linker's
    --gc-sections would dead-strip every symbol out of the cdylib — PER-PLATFORM-
    SILENTLY, the classic cargo-green-but-artifact-broken trap. We reference the
    anchor from a `#[used]` static + a black_box call in `_native` init; the anchor
    takes the address of every export, which the compiler cannot resolve without
    keeping the symbol.

Gated behind the `python` feature (only the wheel needs it, not the headless bin).
`verify_ffi_path()` (python/ciris_server/__init__.py) returns `_native.__file__` for
the agent's ffi_bindings ctypes loader.

VERIFIED ON THE ACTUAL ARTIFACT (never trust cargo-green): `nm -D` the release
cdylib lists all 85 symbols — anchor + jcs_canonicalize, kex x4, wrap/unwrap_dek,
self_enc x3, locale_merkle, attestation, create_federation_identity. A new
Verify-FFI fold smoke runs on all three desktop wheel legs (linux/macOS/windows):
it pip-installs the built wheel, ctypes-CDLLs verify_ffi_path(), and asserts the
symbols resolve — so a platform-specific strip fails the wheel job instead of
shipping silently.

## CIRISServer#238 — canonical boot-prime (no more waiting on a slow announce)

prime_trusted_peers primes from Rooted transport_destination ROWS, but the baked
canonical seed carries only a KeyRecord + an IP dial hint and has no such row — so
the canonical was NEVER boot-primed and rooted only via its slow/unreliable announce
(~130-200s, or never).

An explicit-hash peer needs just (dest_hash, ed25519), and BOTH are deterministically
derivable from the fed Ed25519 pubkey the seed already carries:
  dest_hash   = reticulum_destination_for_pubkey(fed_ed25519) = sha256(fed_ed25519)[..16]
  signing_key = that same fed Ed25519 — transport and federation share the Ed25519
                signing half (the v10.1.0 split was the X25519 half, which priming
                never uses; inject_rooted_peer_for_test takes no X25519 at all).

New compose::prime_canonical_bootstrap_peers looks up each canonical's KeyRecord
(lookup_public_key), derives both, and inject_rooted_peer_for_test -> deterministic
root at boot, no announce dependency, no seed change. Runs in BOTH the standalone
node and the agent fold. CIRISPersist#428 closed as wrong-premise (I first thought
this needed the transport Ed25519 baked into the seed; it does not — the key is
derivable from the fed key).

## Build-memory guardrail

`cargo test` links the ~32 integration-test binaries in tests/*.rs, each statically
linking the FULL substrate. At the dev/test default of full DWARF that is ~2-3 GB
peak RSS per link x `nproc` (32) parallel jobs = tens of GB over the 31 GiB of RAM;
the kernel swap-THRASHES rather than OOM-killing, and the machine goes unresponsive.
This actually happened. [profile.dev]/[profile.test] now pin
`debug = "line-tables-only"` — keeps file:line in panic backtraces while cutting
per-link RSS ~5-10x. CI-safe (lighter/faster links, parallelism unchanged), unlike a
`[build] jobs` cap which would over-subscribe a 4-core runner. Debugger override is
documented inline.

Full suite 309/309 (0 failed, 13 ignored, 35 suites); clippy (default + python), fmt,
and the release-gate pin gate green at the new triple.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018tDoby6hToRTt7of4y4pcG
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

…l), not just at release

build-wheels.yml is workflow_call-only — invoked by publish-pypi at TAG time — so the
#232 fold smoke added there would have first executed during the PyPI publish. That is
the worst place to discover a bug in the smoke script itself.

The PR/push wheel leg actually lives in conformance.yml, so mirror the assertion there:
every PR now pip-installs the built linux wheel, ctypes-CDLLs verify_ffi_path(), and
asserts the ciris_verify_* surface resolves. build-wheels.yml keeps the same check for
the macOS/Windows legs (exercised on a main push via warm-release-cache, i.e. still
BEFORE any tag), which is where a platform-specific --gc-sections strip would show up.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018tDoby6hToRTt7of4y4pcG
@emooreatx
emooreatx merged commit 6619cfb into main Jul 12, 2026
8 checks passed
@emooreatx
emooreatx deleted the release/0.5.107-verify-ffi-fold-canonical-prime branch July 12, 2026 19:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment