Conversation
session-deps is a new repo that consolidates how we handle loading and doing static bundle builds of various common external dependencies across Session projects.
Carrying a libsodium fork is too much of a nuissance as updating it to the latest version is non-trivial. This drops the libsodium-internal fork in favour of using tweenacl's implementation *just* for the X25519 -> Ed25519 pubkey conversion, and using a stock libsodium for everything else. This also bumps the libsodium requirement up to 1.0.21: that version will be required for SHAKE support in future commits on this branch.
- Blake2b hashing:
- Add nicer hash::blake2b functions for simpler has computations where
you can just pass a bunch of spannables and get the hash over them,
rather than needing to do a bunch of manual C API update calls.
- Add a ""_b2b_pers for a compile-time validated blake2b
personalisation string
- Drop make_blake2b32_hasher().
- TODO: convert these to make full use of hash::blake2b(...), as
above.
- Change cleared_array to take a Char type instead of forcing unsigned
char.
- Add cleared_uchars (and cleared_bytes) for the old force-unsigned char
typedef.
- Make `to_span` work for any input convertible to a span
- Tighten up various functions taking fixed length values (like session
ids, pubkeys) to take compile-time-fixed spans instead of dynamic
extent spans. This allows these generic functions to not have to
worry about length checking.
- Add a generic random::fill(s) function that fills some spannable type
s with random bytes.
This switches to the PR branches for session-router and libquic to use session-deps. (This was required under ninja builds, in particular, to get the deduplication handling for gmp and nettle via new session-deps code to deal with that).
- Refactor manual libsodium blake2b hash calls to use simpler hash::blake2b functions instead. - Unify usage of array_uc32/33/64 and uc32/33/64: now we have just uc32/33/64 and cleared_uc32/33/64 (i.e. the "array_" prefix is gone). - Add a unsigned char array literal, which is quite useful for static hash keys.
- Including raw integer bytes would break the hashes on a non-little endian arches; this adds a helper than ensures we are always hashing the little-endian value. - Make the remaining manual hash use blake2b_pers. This didn't get autoconverted before because the `if` around part of the hash, but that if is actually completely unnecessary: if the value is empty, a blake2b hash update does nothing, and so it can just be always included (and when empty, it is still the right thing).
oxenc has buggy "constexpr" overloads that just break if invoked, and they get invoked here with a uint8_t value + unsigned char array. The issue is fixed in oxenc dev, but switching to a byte here works around it for current and older oxenc versions.
The recent commit to unify the types wasn't applied to the test suite.
hash::blake2b (and related) now take integers directly, writing the integer bytes (with byte swapping applied, if necessary), which further simplies the hashing API.
Make way for account keys to be in here as well.
If two devices rotate account keys at approximately the same time, both might end up with inconsistent "active" keys. This commit adds deterministic tie breaking (always prefering the later, with fallback to seed ordering).
Adds a "needs push" concept, along with more tracking fields to let us distinguish between various possible states.
`session::AdjustedClock` now carries an adjustable static offset and when `now()` is called it returns standard system clock timepoints with the adjustment applied. The networking code had a similar adjustment already, although it was per-network-object: this change replaces that, and makes it now global across all uses of the clock anywhere, instead of per-Network instance.
(PFS) Drop redeemed_at/redeemed_ts and the "unredeemed" status
The backend renamed the out-of-band grant provider from "rangeproof" to
"stf", reflecting that such issuances come from the Session Technology
Foundation, not the inactive Rangeproof dev house. Follow the wire/slug
value ("stf") and the C/C++ constant names to match.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(PFS) Rename payment provider rangeproof -> stf
Forward-port of #109 (dev commits c3f6951..b08630b) onto pfs. session_id_matches_blinded_id() read blinded_id[1] before checking the value's length, and never checked that it was hex. It also used the predicate `blinded_id[1] != '5' && (blinded_id[0] != '1' || blinded_id[0] != '2')`, whose right-hand side is always true (a char cannot differ from both '1' and '2' only when it equals one of them, so the || is tautological), leaving the check as just `blinded_id[1] != '5'` -- so any X5-prefixed value, including a plain 05 session id, was accepted as a blinded id. Validate length and hex before indexing, and replace the prefix tests with starts_with("15")/starts_with("25"). Applies to pfs unmodified: line-for-line identical to the dev change.
Records dev up to 4a47113 as merged, keeping pfs's tree unchanged (-s ours). Everything in the range is already applied to pfs: - #104 (refund-requested-config) via #106 refund-requested-config-pfs - #107 (drop-redeemed-at) via #108 drop-redeemed-at-pfs - #110 (rename-rp-to-stf) via #111 rename-rp-to-stf-pfs - #109 (fix-blinded-id-validation) forward-ported in the preceding commit The first three were rewritten for pfs and so have different patch-ids, which is why `git cherry` still reports their commits as absent; the content was verified present by comparing the identifiers each PR added or removed across both branches.
The backend is dropping this field and libsession follows. It carried no actionable information: a single count of "some number of backend errors happened at some unspecified past time", with no what, no when, and nothing a client could do in response. Parsing it only added a way for the response to fail. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the manual skip_until + consume_string_view + explicit size checks with require<>/require_span<>/maybe<>, which fold presence and size/type validation into the accessor. Build the verification buffer with reserve+insert instead of resize+memcpy (identical bytes; also drops the spurious -Wstringop-overflow warning on the memcpy).
(PFS) Drop error_report from get_pro_status
Records dev up to 947d72a as merged, keeping pfs's tree unchanged (-s ours). The only PR in the range, #112 (drop-error-report), is already applied to pfs via #113 drop-error-report-pfs; nothing needed forward-porting. #113 is a faithful adaptation rather than a straight pick, so its commits have different patch-ids: it carries the same six files, differing only in pfs's unsigned char -> std::byte types, the resulting drop of an ed_pk.first<32>() now that require_span is fixed-extent, and each branch removing its own pre-existing form of the two size-error messages. Verified by outcome: error_report is absent from both branches, both parse decrypt_group_message through the bt_dict_consumer require/maybe helpers, and seed_payment.py is identical on both.
The proof/status/payment/revocation parsers threaded a std::vector<std::string> of errors through every helper and made the caller check it afterwards -- C-style error handling in C++. Replace it with a `parse_error` exception (new, public in pro_backend.hpp): the JSON helpers and each parser throw on a malformed reply, while a well-formed backend *failure* (envelope status fail/error) is still returned normally with status/error_code set. The C entry points catch once and translate to the invalid_response header, now carrying the real diagnostic rather than a fixed "out-of-memory" string. Along the way: - json_require<double> accepts any JSON number (is_number(), since an integer is a valid float value) rather than is_number_float(); the now-redundant json_require_number helper is removed. - json_require<integral> uses is_number_integer() so a fractional wire value is rejected rather than silently truncated by get_to. - rename json_require_fixed_bytes_from_hex -> json_require_hex. - C-layer allocations use make_unique + release() rather than a raw new with a manual delete in the catch; this also closes a leak in the error-translation path where a throw between the new and the pointer assignment orphaned the object. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
pro_renewal_target returned nullopt ("never (re)fetch") whenever there was
no proof credential unless a prepaid purchase marker was set. But `s` (the
credential) and `E` (the access expiry) are independent config keys, so an
account can be genuinely entitled -- `E` still in the future -- while
holding no proof (e.g. `s` was dropped or merge-lost). That state should
fetch a proof, not sit idle forever. Return `now` when the access expiry is
still in the future and there's no proof.
Also rename the local `pro` -> `pro_config`: it is the full ProConfig
credential (rotating key + proof), not a boolean "are we pro", and the old
name misled at least one reader into misdiagnosing this very path.
Relies on the client keeping `E` synced to the backend's reported horizon
and clearing it on not_subscribed (E does not self-age); communicated
separately to the clients.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(PFS) pro_backend: throw on parse errors instead of an error vector
(PFS) user_profile: fetch a proof when entitled but holding none
Forward-port of #120 (dev commit b63b4b0) onto pfs; applies unmodified. GetProRevocationsCResponse / GetPaymentDetailsCResponse are aggregates built by the *_parse functions via std::make_unique from a parsed base response. make_unique initializes with parentheses, which only aggregate-initializes under P0960R3 (C++20): GCC implements it, the Apple Clang on the macOS CI runners does not, so it looks for a constructor and finds none. Give each an explicit base-slice constructor; call sites are unchanged. pfs has the same structs and the same make_unique call sites, so it has the same latent macOS break.
Forward-port of #114 (dev commit 89c6acb) onto pfs. Most of #114 is already here: the byte refactor (7662405) had independently given the binary API fixed-extent spans, so sign/verify/pubkey already reject wrong sizes at compile time, the string overloads already validate, and the blinding / group-keys call sites already pass sized views. Two things were missing. First, pfs's verify(string_view) returned false for a wrong-sized signature or pubkey where #114 throws std::invalid_argument, so it conflated "you passed a malformed argument" with "this signature does not verify". Align with dev: letting pfs keep return-false would silently revert #114's behaviour when pfs eventually lands on dev. Nothing in the tree calls the string overloads -- every call site uses the span overloads -- and the return-false came in incidentally with 7662405 rather than as a considered choice. Second, port #114's regression test for the rejected sizes. The three string overloads all validated a length and then narrowed to a fixed span; that is now one require_bytes<N> helper, which also gives the arguments #114's exception wording.
Records dev up to 2b27d27 as merged, keeping pfs's tree unchanged (-s ours). All four PRs in the range are now applied to pfs: - #116 (this-is-not-c) via #115 this-is-not-c-pfs - #118 (renewal-target-no-proof-fetch) via #119 renewal-target-no-proof-fetch-pfs - #120 (macOS C response holders) forward-ported in e682e6b - #114 (fixed-size XEd25519 spans) forward-ported in b03709d Verified for the two that were already applied: #119's user_profile.cpp change is line-identical to #118's, and for #116/#115 the `errs` error-vector is gone from both branches with identical parse_error usage.
Clients sometimes need to know whether a Pro subscription is terminal or auto-renewing (e.g. "renews on X" vs "expires on X"). Store the backend's `auto_renewing` (from get_pro_status) as a presence-only config flag `A`: 1 when auto-renewing, absent otherwise (terminal / unknown / not Pro). Deliberately not tri-state: unlike blinded_msgreqs `M`, this is backend- derived fact, not a defaulted client preference, so there's no upgrade- default edge case that a distinct "unset" would guard. And no t/T bump -- it's synced pro state like E/I/R, not a user-initiated profile edit. Exposes get_/set_pro_auto_renewing (C++ bool; C 0/1) with unit + C-API coverage. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Forward-port of #125 (dev commit d9406eb) onto pfs, where the same code was present verbatim. encrypt_for_multiple_simple's decoy padding was sized from the plaintext rather than the ciphertext, so a padding entry was encrypt_multiple_message_overhead bytes shorter than every real entry and could be picked out by length alone -- defeating the point of padding the list to a fixed count. The `if (int pad_size = ...)` form also made an empty first message pad to zero, i.e. falsy, which skipped the padding loop entirely: a list of empty messages got no decoys at all. Test both, the sizes and the count.
pfs has broken a fair bit of the API (the std::byte span refactor, the crypto header reorganization, the C API changes), and together with core and the coming Client code this is a substantially different release, so it takes a major bump rather than following dev's 1.x line. This deliberately diverges from dev, which is at 1.8.0 (#128): pfs has been on its own version line since before 1.7.0, and the successive `-s ours` merges have been quietly masking dev's bumps. Setting 2.0.0 makes pfs's version unambiguously ahead, so a future merge into dev carries the version forwards rather than dragging it back.
I kept dev's line wrapping when porting #125, but pfs's encryption::XCHACHA20_ABYTES is shorter than dev's crypto_aead_xchacha20poly1305_ietf_ABYTES, so the CHECK now fits on one line and clang-format joins it. Missed because I did not run utils/format.sh before committing.
Forward-port of #126 (dev commit 4143882) onto pfs; the five files dev touches are line-for-line identical to dev's change. A 421 means the account we asked about is not in the queried node's swarm, so recovery requires re-resolving *that account's* swarm. The old code used failed_node.swarm_pubkey(), which was compute_x25519_pubkey(remote_pubkey) -- the X25519 pubkey of the node we had just been rejected by, not of the account -- so it re-resolved the wrong swarm entirely. Requests now carry the account they address in Request::swarm_pubkey (swarm_pubkey_hex in the C params), the 421 path uses that, and a request without one is failed rather than misdirected again. service_node::swarm_pubkey() goes with it: its only caller was that wrong line, and its name invited exactly this mistake. Beyond dev's diff: dev only populates the field in the C API, but pfs has three internal swarm-addressed request sites in core.cpp (the namespace batch poll, the PFS-key prefetch, and the DM store) which dev does not. Left unset those would now fail on their first 421 instead of recovering, so all three go through a swarm_request() helper that carries the pubkey alongside the node -- one place to get right, and a fourth site cannot silently omit it.
Forward-port of #127 (dev commits 724939a..4b582a5) onto pfs. A refresh can succeed and still yield too few nodes -- most easily when the multi-request intersection finds little in common -- and the result was written over the existing cache unconditionally, leaving nothing to route through until the next refresh landed. An empty result wiped it entirely. Treat an under-sized result as a failed refresh instead: keep the cache we have and retry after the usual backoff. The retry path already existed for an unparseable response, so it is lifted out of the catch block into a discard_and_retry lambda holding a weak self ref and reused for both cases. Also carried along from the same branch: seed_payment.py inlines the day-rounding it needs, since the backend moved base.round_datetime_to_next_day into its own test scaffolding. pfs adaptations: `_hexbytes` -> `_hex_b`; the new test helper is static, as pfs builds with -Werror=missing-declarations; and the byte cast when copying the remote key is dropped, view_remote_key() already returning std::byte here.
Forward-port of dev commit 0a23f70; applies unmodified. Aligns the revocationTag column and corrects expiryUnixTs's comment, which claimed milliseconds where the field carries seconds.
…-renewing flag Forward-port of #124 (dev commits 332b3e6..059e529) plus dev commit 7ef8385 onto pfs. pfs already had the auto-renewing half of this (config key `A`, ported as jagerman/pro-auto-renewing-config-pfs), so what lands here is the rest: - The grace period, config key `G`: how much longer the account is served past `E`, so any linked device can compute when coverage actually ends at `E + G`. Deliberately a plain duration rather than an optional -- the backend sends zero when the subscription is not auto-renewing, so "unset" and "zero" describe the same account. - Clearing `E` now clears `G` and `A` with it. A grace that outlived its expiry would silently pair with the *next* `E` write, and that write is typically a proof outcome, which carries no grace to correct it with. - The backend's grace and renewal flags are read off the proof response and off a `subscription_expired` failure. An absent field means "not applicable" and clears any stale cached value rather than preserving it. - The parse helpers move out of pro_backend.cpp: parse_error grows a hierarchy (parse_error_key/_missing/_type) in the new public session/parse_error.hpp, and the JSON readers move to session::detail in the new src/json_parser.hpp, gaining json_maybe and a wire_seconds concept so call sites stop writing sys_seconds{seconds{get<int64_t>()}}. json_require_hex moves there too, per 7ef8385. pfs adaptations: json_require_hex takes std::span<std::byte>; pro_backend.hpp keeps pfs's clock/ed25519 includes and does not gain dev's `using namespace oxenc::literals` (pfs uses session's `_hex_b` here); and the auto-renewing accessors dev's diff re-adds are dropped as pfs already has them.
pfs-only; not going upstream to dev. Every function in the newly-imported json_parser.hpp carried a `json_` prefix -- C-style namespacing by naming convention, in a codebase that has namespaces. Put them in `session::json` and drop the prefix, so call sites read `json::require<T>(obj, "key")`, `json::maybe<T>(...)`, `json::parse(...)`, `json::require_hex(...)`, and internally `json::is<T>` / `json::extract<T>`. `session::detail` is left to what it already held (ProFlags in config/internal.hpp); these helpers were only put there by dev because that is where the extraction landed, not because they belong beside it. One consequence: a parameter named `json` shadows the new namespace, breaking `json::` lookup inside such a function. That only affects definitions, so the four `parse_*` definitions and `read_envelope` take `json_in`, while the public declarations keep `json` -- there the name is doing real work, saying the value is JSON rather than generic input. The C wrappers keep `const char* json` throughout: they only forward it and never qualify with `json::`.
pfs-only, on top of the session::json rename. json_require_hex was misnamed and under-specified: what these fields hold is a binary value -- a pubkey, a signature, a tag -- and hex is merely one encoding the backend may send it in. Renamed to require_binary and taught to accept base64 as well, padded or unpadded. The encoding is chosen by the encoded length, not by inspecting the alphabet, which cannot work: every hex string is also valid base64. That makes the destination length load-bearing, so the buffer must be fixed-size and at least 5 bytes, both enforced by static_assert via CTAD on the span. Below 5 the encodings collide -- 1 byte is 2 chars either way, 2 and 4 bytes are 4 and 8 chars as hex or as padded base64 -- and from 5 up 2N exceeds both 4*ceil(N/3) and ceil(4N/3), so the lengths never coincide again. Dropped the 0x/0X prefix stripping. These are binary values rather than numbers, so the prefix means nothing here, and worse, `0`, `x` and `X` are all valid base64 characters: the strip would have silently corrupted any base64 value beginning that way. Sizes now come from oxenc::to_hex_size and to_base64_size rather than being recomputed inline. Tested through parse_pro_proof: hex, padded base64 and unpadded base64 all decode to the same bytes, and a length matching no encoding is rejected as a parse_error_key naming the field.
Records dev up to 7ef8385 as merged, keeping pfs's tree unchanged (-s ours). Everything in the range is now applied to pfs: - #125 (multi-encrypt padding size) -> 0763aff - #126 (re-resolve swarm after 421) -> 3699813 - #127 (undersized snode refresh) -> ec91a11 - 0a23f70 (proto formatting/comment) -> f8ffbd3 - #124 (pro grace period + tristate) and 7ef8385 (json_require_hex move) -> 85483fe #128 (dev's bump to 1.8.0) is deliberately NOT taken: pfs went to 2.0.0 in db19773 instead, for the API breaks it already carries. Taking dev's 1.8.0 would move pfs's version backwards. Two follow-ups on top are pfs-only and are not going to dev: 4003a13 moves the JSON helpers to session::json without the json_ prefix, and 3706900 reworks json_require_hex into require_binary, which accepts hex or base64.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This branch (
pfs) is now actively maintained alongsidedevand will eventually replace thedevbranch.This adds perfect forward secrecy support with post-quantum encryption; a "Core" component with a database layer to track state; direct-QUIC file server support; and a host of code cleanups.
For extensive details, see the description of PR #90.