Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,483 changes: 64 additions & 1,419 deletions CHANGELOG.md

Large diffs are not rendered by default.

183 changes: 120 additions & 63 deletions CLAUDE.md

Large diffs are not rendered by default.

14 changes: 14 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,20 @@ See the [docs/](docs/) directory for detailed guides:
- [Telemetry](docs/telemetry.md)
- [iOS Integration](docs/ios-integration.md) / [Android Integration](docs/android-integration.md)

Reference material for anyone implementing against the protocol or changing its
behaviour:

- [Protocol Specification](docs/spec/README.md), the wire and behaviour contract,
independent of this implementation
- [Threat Model](docs/security/threat-model.md), including the residual risks
stated plainly
- [State Machines](docs/state-machines/README.md) for delivery, retries,
sessions, groups, and transports
- [Architecture Decision Records](docs/adr/README.md), why the non-obvious
choices are what they are
- [Bridge Contracts](docs/bridges/README.md) for Swift, Kotlin, Python, and
TypeScript

## Development

```bash
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ object MeshWakePolicy {
* A drift here fails silently in the worst way: React Native logs "No task
* registered for key" to the device log and resolves nothing, the app sees
* an opt-in that does nothing, and both sides still compile. Pinned by
* `react_native_mesh_wake_task_key_matches_native` in the uniffi crate.
* `react_native_mesh_wake_wiring_is_present` in the uniffi crate.
*/
const val TASK_KEY = "OfflineProtocolMeshWake"

Expand Down
2 changes: 1 addition & 1 deletion bindings/react-native/src/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ export const ONE_SHOT_EVENT_TYPES = [
* silently and in the worst possible way — React Native logs "No task
* registered for key" to the device log, the app sees an opt-in that does
* nothing, and both sides still compile — so it is pinned by a Rust guard
* (`react_native_mesh_wake_task_key_matches_native` in
* (`react_native_mesh_wake_wiring_is_present` in
* `crates/offline-protocol-uniffi`).
*/
export const MESH_WAKE_TASK_KEY = 'OfflineProtocolMeshWake';
Expand Down
15 changes: 12 additions & 3 deletions crates/offline-protocol-core/src/address.rs
Original file line number Diff line number Diff line change
Expand Up @@ -93,9 +93,18 @@ pub enum AddressError {
/// The two orders differ: the bech32 charset (`qpzry9x8gf2tvdw0s3jn54khce6mua7l`)
/// is not monotonic in ASCII — value 4 renders as `y` (0x79) and value 5 as
/// `9` (0x39) — and a string comparison would also weigh the checksum
/// characters. Protocol tiebreakers (both-create session ownership, leave
/// election, admin auto-promotion, fork leader) must therefore compare
/// `Address` values, never their `Display` output.
/// characters.
///
/// **Which order a tiebreaker uses is fixed per site, and the sites disagree.**
/// Both-create session ownership compares `Address` values (hash bytes); group
/// leave election, admin auto-promotion and fork leader election sort rendered
/// strings. Each converges on its own, because every peer running that
/// tiebreaker sorts the same way.
///
/// Do not "harmonize" one site onto the other order. Peers that changed and
/// peers that did not would then elect different winners from identical input,
/// with no way to detect the disagreement locally. Prefer hash-byte order for
/// anything new. See `docs/adr/0003-self-certifying-addresses.md`.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)]
pub struct Address {
hash: [u8; Address::HASH_LEN],
Expand Down
2 changes: 1 addition & 1 deletion crates/offline-protocol-router/src/dors.rs
Original file line number Diff line number Diff line change
Expand Up @@ -868,7 +868,7 @@ impl TransportSelector {
//
// The Internet baseline intentionally exceeds the 0–100 range of
// offline transports so that the gap comfortably exceeds the default
// switch hysteresis (10). Increasing hysteresis beyond ~20 may
// switch hysteresis (15.0). Increasing hysteresis beyond ~20 may
// prevent DORS from switching *to* Internet even when prefer_online
// is set.
//
Expand Down
3 changes: 2 additions & 1 deletion crates/offline-protocol/src/group_mesh.rs
Original file line number Diff line number Diff line change
Expand Up @@ -839,7 +839,8 @@ pub(crate) struct PendingGroupMessage {
pub(crate) buffered_at: Instant,
/// Transport the frame arrived on, recorded so the drain can send the
/// deferred delivery ACK directly on it once the message finally decrypts
/// (see the deferred-ACK atom in CLAUDE.md). `None` for the relay path
/// (see the deferred-acknowledgement atom in
/// `docs/state-machines/delivery-and-acks.md`). `None` for the relay path
/// (the relay sender is not ACK-gated) and for transport-less test enqueue
/// — in both cases the drain ACK is a correct no-op.
pub(crate) received_via: Option<TransportType>,
Expand Down
6 changes: 4 additions & 2 deletions crates/offline-protocol/src/protocol/decryption_queue.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,10 @@ pub(crate) struct PendingDecryptMessage {
pub(crate) sequence: u64,
pub(crate) message: Message,
/// Transport the frame arrived on, when the caller knew it. Recorded so the
/// drain can send the deferred delivery ACK directly (see the deferred-ACK
/// design in CLAUDE.md) instead of waiting for the sender's next resend to
/// drain can send the deferred delivery ACK directly (see the
/// deferred-acknowledgement atom in
/// `docs/state-machines/delivery-and-acks.md`) instead of waiting for the
/// sender's next resend to
/// hit the duplicate re-ACK path. `None` when the message was enqueued from
/// a context with no transport (tests, or a re-queue by the handler during a
/// drain, which re-processes with no live arrival transport) — the drain
Expand Down
4 changes: 2 additions & 2 deletions crates/offline-protocol/src/protocol/pending_queue.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ impl OfflineProtocol {
/// [`Self::enqueue_pending_decryption`] with the transport the frame arrived
/// on, when the caller knows it. The transport is recorded on the pending
/// entry so the drain can send the deferred delivery ACK directly instead of
/// relying on the sender's next resend (see the deferred-ACK design in
/// CLAUDE.md).
/// relying on the sender's next resend (see the deferred-acknowledgement
/// atom in `docs/state-machines/delivery-and-acks.md`).
pub(super) fn enqueue_pending_decryption_via(
&mut self,
sender: &str,
Expand Down
3 changes: 2 additions & 1 deletion crates/offline-protocol/src/protocol/session.rs
Original file line number Diff line number Diff line change
Expand Up @@ -632,7 +632,8 @@ impl OfflineProtocol {
/// (delayed delivery, never lost). Closing that needs a signed
/// epoch-corroboration exchange before teardown; a liveness-only probe does
/// not work, since a healthy peer answers and we would tear down anyway.
/// Also see the CLAUDE.md "Crypto-failure recovery" note.
/// Also see `docs/state-machines/session-lifecycle.md` ("Desync and heal")
/// and `docs/security/threat-model.md` (residual risk R2).
pub(super) fn schedule_session_rekey(&mut self, peer_id: &str) {
let now = Utc::now();
if let Some(due_at) = self.rekey_due_at.get(peer_id) {
Expand Down
5 changes: 3 additions & 2 deletions crates/offline-protocol/src/protocol/types.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1273,8 +1273,9 @@ pub(crate) enum InternalMessageResult {
/// In the latter three, recovery is the sender's *resend* rather than this
/// frame: Tier 2 re-seals each resend of an encrypted DM against a live
/// generation, and a message that stays undeliverable settles as an honest
/// `MessageFailed` instead of a lying "delivered". See the deferred-ACK and
/// crypto-failure-recovery designs in CLAUDE.md.
/// `MessageFailed` instead of a lying "delivered". See
/// `docs/state-machines/delivery-and-acks.md` for the deferred-acknowledgement
/// atom and the decrypt-failure classification.
Deferred,
/// Message was decrypted, here's the plaintext.
Decrypted(String),
Expand Down
77 changes: 73 additions & 4 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,19 @@
# Documentation

Two kinds of document live here, and they answer different questions.

**Guides** answer "how do I use this". Start with the Quick Start and the
integration guide for your platform.

**Reference** answers "what is the contract" and "why is it like this". Read it
before changing behaviour, not before using the SDK.

## Getting Started

| Guide | Description |
|-------|-------------|
| [Quick Start](../QUICKSTART.md) | Get started in 5 minutes (React Native, iOS, Android) |
| [Upgrading](UPGRADING.md) | **Breaking changes and required app-side work for the storage-split release** |
| [Upgrading](UPGRADING.md) | **Breaking changes and required app-side work** |
| [React Native Integration](react-native-integration.md) | Full SDK integration guide with complete API reference |
| [iOS Integration](ios-integration.md) | Native iOS (Swift) setup and usage |
| [Android Integration](android-integration.md) | Native Android (Kotlin) setup and usage |
Expand All @@ -31,7 +39,69 @@
| [Telemetry](telemetry.md) | Wire up a telemetry sink for metrics, routing decisions, and MLS lifecycle |
| [Transport Architecture](transport-architecture.md) | Transport abstraction layer and how to add new transports |
| [Reticulum Transport](reticulum.md) | Reticulum mesh transport setup, architecture, and platform integration |
| [Nostr Transport](nostr.md) | Nostr relay transport — censorship-resistant routing over WebSockets |
| [Nostr Transport](nostr.md) | Nostr relay transport, censorship-resistant routing over WebSockets |

## Protocol specification

The wire and behaviour contract, independent of this implementation. A second
implementation written against these documents should interoperate.

| Document | Scope |
|----------|-------|
| [Specification index](spec/README.md) | Layering, conformance language, the two overriding invariants |
| [Identity and addressing](spec/identity.md) | Address derivation, canonical form, session and group identifiers |
| [Message model and wire format](spec/wire-format.md) | The abstract message, the JSON floor, binary v1, the extension TLV registry |
| [Control messages](spec/control-messages.md) | Reserved prefix registry, control-plane signing, the two exemption classes |
| [Encryption envelopes](spec/encryption-envelopes.md) | MLS envelope forms, media chunk envelope, sealed rich payload |
| [Group protocol](spec/group-protocol.md) | Group frames, membership commits, leaf identity binding, relay broadcast |
| [Capability negotiation](spec/capability-negotiation.md) | What peers advertise, what it gates, what absence means |

## Security

| Document | Scope |
|----------|-------|
| [Threat model and trust boundaries](security/threat-model.md) | Assets, adversary classes, controls, and the residual risks stated plainly |
| [Security Policy](../SECURITY.md) | Vulnerability reporting and safe harbor |

## State machines

| Document | Governs |
|----------|---------|
| [Overview](state-machines/README.md) | The invariant that spans all five |
| [Delivery and acknowledgements](state-machines/delivery-and-acks.md) | What happens to an inbound frame, and when a receiver acknowledges |
| [Outbox and retries](state-machines/outbox-and-retries.md) | An outbound message from send to terminal state |
| [Session lifecycle](state-machines/session-lifecycle.md) | 1:1 MLS establishment, confirmation, desync, and heal |
| [Group message lifecycle](state-machines/group-message-lifecycle.md) | A group message through fan-out, buffering, and drain |
| [Transport lifecycle](state-machines/transport-lifecycle.md) | Transport availability, scoring, switching, escalation |

## Decisions

| Document | Scope |
|----------|-------|
| [ADR index](adr/README.md) | Fifteen decisions that are expensive to reverse or easy to undo by accident |

If something in the codebase looks redundant or over-engineered, check here
before simplifying it.

## Bridge contracts

What each language binding owes the core, and what the core owes it. Every rule
in here fails **silently** when violated.

| Document | Scope |
|----------|-------|
| [Shared contract](bridges/README.md) | The ten rules every binding shares |
| [Swift](bridges/swift.md) | iOS native and the React Native iOS bridge |
| [Kotlin](bridges/kotlin.md) | Android native and the React Native Android bridge |
| [Python](bridges/python.md) | Desktop and tooling |
| [TypeScript](bridges/typescript.md) | The React Native JavaScript surface |

## Release history

| Resource | Description |
|----------|-------------|
| [CHANGELOG](../CHANGELOG.md) | Unreleased changes and the current release |
| [Changelog archive](changelog/README.md) | Older releases, one file per minor series |

## Examples

Expand All @@ -46,11 +116,10 @@
| Resource | Description |
|----------|-------------|
| [Contributing Guide](../CONTRIBUTING.md) | Development setup, code quality standards, and PR process |
| [Security Policy](../SECURITY.md) | Vulnerability reporting and security design |

## Licensing

| Resource | Description |
|----------|-------------|
| [Licensing FAQ](licensing-faq.md) | The dual license in practice app stores, the AGPL's reach, commercial licensing |
| [Licensing FAQ](licensing-faq.md) | The dual license in practice: app stores, the AGPL's reach, commercial licensing |
| [Export Control Notice](../EXPORT.md) | Encryption export status of the SDK and what app teams must handle themselves |
66 changes: 66 additions & 0 deletions docs/adr/0001-json-as-permanent-wire-floor.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# 0001. JSON is the permanent wire floor; compact encodings are additive

**Status:** Accepted
**Shipped in:** 0.14.0 (binary wire codec), 0.14.0 (compact MLS envelope)

## Context

JSON message framing is expensive on a Bluetooth LE link. A 36-character
hyphenated message identifier, verbose field names, and a ciphertext rendered as
a decimal integer array cost roughly 3 to 4 times what a compact encoding does,
which translates directly into fragment counts and airtime.

The obvious move is to replace JSON with a compact encoding. Two things make
that wrong here:

1. The fleet upgrades gradually and mesh peers meet arbitrary strangers. There
is no coordinated flag day.
2. The internet relay and every persisted record are on the same code path.
Changing the encoding changes what old records deserialize as.

## Decision

Compact encodings are **additive**. JSON remains a permanent obligation:

- every receiver decodes JSON, unconditionally and forever,
- a compact encoding is emitted only to a peer that advertised it,
- persistence and the internet relay transport stay JSON unconditionally (the
Nostr transport is relay-mediated but uses the negotiated codec like any other
peer-to-peer path),
- decoding of compact encodings is always on, independent of whether emitting
them is.

Three layers carry this full shape, each with its own switch and its own
capability, because they are independent: the hop-local wire codec
(`binary_wire_enabled` / `wire_versions`), the end-to-end MLS envelope
(`compact_envelope_enabled` / `env_versions`), and the sealed rich payload
(`rich_payload_enabled` / `rich_versions`).

The media chunk envelope is **not** a fourth instance and should not be
described as one. It has no JSON form to fall back to and no switch: its payload
is always the compact encoding, and only the choice between its v1 and v2 forms
is negotiated, riding the rich-payload capability rather than one of its own.

## Consequences

**Good.** A mixed fleet works with no coordination. Rollback is a configuration
change, not a migration. A peer that mis-advertises costs a delivery failure to
itself, not a fleet-wide outage.

**Cost.** Two encoders and two decoders per layer, forever. The JSON path can
never be deleted, so it must stay tested. Size wins only materialize once both
ends have upgraded.

**Cost.** Detection is by first byte, which constrains the magic byte to a range
that cannot begin valid JSON or valid UTF-8. That range holds eleven values, of
which v1 spends one, leaving ten for future versions. That is plenty, but it is
finite.

## What would undo this

Making decode of a compact form conditional on local configuration. That turns a
kill switch into a compatibility break: peers that were told we are capable
start sending a form we then refuse.

Adding a compact form to persistence "since we already have the codec". Stored
records outlive every negotiation.
66 changes: 66 additions & 0 deletions docs/adr/0002-frozen-dto-with-extension-tlv.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# 0002. The binary encoding uses a frozen positional DTO with an extension TLV

**Status:** Accepted
**Shipped in:** 0.14.0

## Context

The binary encoding uses postcard, which is positional and non-self-describing.
That is where the size win comes from, and it is also the hazard: reordering,
removing, retyping, or inserting a field silently corrupts decoding on peers
running the previous layout. There is no error, only wrong values.

Serializing the domain message type directly compounds the problem. That type
carries defaulting rules, skip-when-absent rules, and validation-on-deserialize
behaviour that a non-self-describing format cannot honour field for field. The
skip-when-absent rules in particular mean the field count varies by content,
which a positional format cannot express.

## Decision

Encode through a separate flat DTO with a **frozen** field order, and convert
back to the domain type through the **validating** constructors.

Evolution is constrained to three rules:

1. Existing fields never change, in order or in type.
2. Additive data goes into a trailing `(tag, bytes)` extension list that old
decoders read and ignore.
3. Anything that cannot be expressed as an extension entry takes a new magic
byte and is negotiated.

The numeric enum mappings are frozen on the same terms.

## Consequences

**Good.** The security checks the JSON path enforces (identifier caps, logical
clock clamps) apply identically on the binary path, because both go through the
same constructors.

**Good.** Rich, rarely-present structures ride as embedded JSON blobs, so they
keep evolving through their own additive rules without touching the frozen
surface.

**Cost.** A field added to the domain type does not automatically appear on the
binary wire. Someone must decide, per field, whether it warrants an extension
tag.

## The constraint the first extension tag imposes

Extension tag 1 (the base64 content tail) shipped in the **first** release of
wire v1, so advertising the version implies understanding it. A decoder that
ignored it would reconstruct a truncated content field.

That is only safe because no v1 decoder without tag-1 support ever shipped, and
it fixes the rule for every future tag: **a tag whose absence changes meaning
cannot be added to v1.** It needs a new wire version. Tags may be added to v1
only when ignoring them costs efficiency or optional context.

Tag 2 (quoted-reply context) satisfies that test: skipping it delivers the
message without its reply preview, which is exactly what a legacy JSON receiver
does with an unknown field.

## What would undo this

Adding a field to the DTO "at the natural place" rather than at the end, or
adding an extension tag whose absence changes what the message means.
Loading
Loading