fix(zero-swift): harden the client against crashes, data loss, and JS divergences by tmm · Pull Request #38 · wevm/zero

tmm · 2026-10-26T23:41:03Z

Hardening the Swift client against what the JS client survives: about 30 fixes, found by writing failing tests whose expected values come from running the JS client (the conformance harness, temporary vitest runs, or the JS source where noted), plus end-to-end chaos tests against a real zero-cache and Postgres.

The commits are grouped for review: each area's tests come first (failing), then its fixes.

Crashes (a trap kills an iOS app; JS throws, logs, or disconnects)

From server data or the local database: an Int column holding -2^63; huge or unsafe last mutation IDs (a tampered stored ID crashed every launch); huge or infinite backoff and timer delays; update row ops that broke the in-memory sources; a one relationship that gets two rows; the partitioned-take and "node does not exist" errors during a poke or mutation.
From app input: limit(-1), limit(Int.max), limit/orderBy on a junction relationship, related or start inside whereExists, a flipped exists over a limited subquery, and ZeroVirtual's minPageSize: .max. Each now reaches the app as JS's error from materialize, run, liveQuery, or preload.

Data loss in storage

The database collector could delete a database a client had just opened, and deleted files of a newer layout version.
A client of a group that is behind could write an older snapshot over a newer one.
Writes that found the database busy were lost on close; flush()/close() now write them (at most ~10 s) or report a storage failure.
Undecodable stored data left the client on memory forever; it now drops the database and starts fresh, as Replicache does.
A second drop could delete a replacement client's new file; logout left the memory fallback.

Client behavior

A zero.mutate inside a mutator shared its mutation ID and the outer one was never pushed; it now waits for the running mutator, as JS waits for Replicache's write lock.
View listeners are notified in JS's order when a listener mutates, removes, or adds listeners (one could end on stale data). A mutate from inside a listener is deferred until the notification ends, as JS's async mutators are.
desiredQueriesPatches apply in the server's order; the pong handler is set before the ping; a timed-out connect aborts its handshake instead of leaking a socket; falsy app error details are dropped; decoding errors use valita's wording.

Differences from JS, documented in JSParity.md

A poke with an unsafe last mutation ID disconnects (JS accepts it).
A value of the wrong type for its column is sorted with the others. JS fails that mutation and every later one, and a poke with it disconnects at every reconnect.
A server frame that isn't JSON shows Swift's own parse message, not V8's.
Lone surrogates become U+FFFD; a failed hydration doesn't leave its query registered.

Verification (Linux)

Full Swift suite, 286 tests including the end-to-end and chaos tests against zero-cache, passes (on a fresh database).
ThreadSanitizer: no races. IVM differential fuzz: ~34k cases match JS; JSON, downstream-message, and binary-chunk fuzz: ~30k cases, no crashes.
Checked-in vectors are fresh; tsc, oxlint, oxfmt, and swift format lint --strict are clean; DocC builds without warnings.
Benchmarks roughly unchanged (1,000-row optimistic update ~21.5 → ~23 µs).
macOS and iOS run in CI only.

Known leftovers

Opening a database another connection has locked blocks Zero.init for ~5 s.
1–3 IVM fuzz cases per seed where JS throws "Unexpected early return" and Swift returns rows (no crash).
Swift doesn't send deleteClients after collecting a database.

Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

…et the pong handler before the ping A poke's desiredQueriesPatches were decoded into a dictionary and applied sorted by client ID, so a clear for one client could run after a put for another and wipe it, and the next initConnection sent the put again. They now keep the order of the object's keys, as JS iterates them. The ping sent before setting onPong, so a pong delivered from inside send reached the old handler and the ping timed out. As in JS, the handler is set first. Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

…pp error details A mutator that called zero.mutate ran the inner mutation nested inside the outer one: both got the same mutation ID, the pusher skipped the outer one, and the outer commit overwrote the inner one's writes in the store. As in JS, where a mutation waits for Replicache's write lock, a mutation that starts while a mutator runs (for a mutation or a poke's rebase) now runs after it, in order. An app error from the server keeps its details only when they are truthy, and ApplicationError(details: .null) has none, as in JS. Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

Server data and stored state reached traps that crash the app, where the JS client throws and disconnects: - A poke whose lastMutationIDChanges holds a number that isn't a safe non-negative integer is now a poke error (the client disconnects). Mutation IDs become Ints for Transaction.mutationID; a huge one trapped in the next mutate. Stored mutations and last mutation IDs with such IDs are dropped, with an error logged, when the client opens, instead of trapping in every launch's replay. The remaining Double-to-Int conversions of mutation IDs are guarded. - A poke whose update ops leave a row under a key that isn't its primary key (a merge that drops or changes the key's columns) is now a poke error, checked before the IVM sources see it. It used to trap in MemorySource's "Row already exists" / "Row not found" preconditions; JS asserts there and disconnects. - Timer delays follow setTimeout: NaN, negative, and delays past 2^31 - 1 ms fire at once, in SystemScheduler and ManualScheduler. A huge minBackoffMs from zero-cache or an infinite pingTimeoutMs trapped making a Duration. - An Int column holding -2^63 fails to read instead of overflowing abs(Int.min). - The disconnect message of a non-finite or huge negative disconnectTimeoutMs no longer converts it to an Int. JSParity.md notes the mutation ID rejection, and that JSON lone surrogates read as U+FFFD (a deliberate difference). Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

Builder misuse that JS's query builder throws for was a precondition that crashed the app. A negative limit, a limit past Int32.max, and a limit or orderBy in a junction relationship now leave that step out and make the query invalid; materializing, running, or preloading it throws JS's message as a PipelineError (a preload rejects cached and complete, and logs it). The error carries through related and exists subqueries. Pipeline building throws "EXISTS subqueries must not have start / related" instead of trapping, and rejects a limit that isn't a non-negative integer. The crash tests now also check what the client does instead. Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

minPageSize: .max overflowed rounding it up to an even number. The page size is now at most 2^30, from minPageSize and from the viewport height over the estimated row height alike, so a page query's limit fits. Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

A client of a group that is behind another (two clients of one group in one process) wrote every poke's snapshot through, so applying 01->02 after another client applied 02->03 left cookie 02 with a mix of both snapshots' rows on disk, a state the server never had. The database now keeps the cookie of each group's stored snapshot and, as Replicache's persistDD31 does, writes a snapshot only when its cookie is newer. An older one still deletes the mutations its last mutation IDs confirm. Both the SQLite and the memory database go through this. Writes that found the database busy were kept for the next write and lost when the client closed before one, with no storage failure. flush() and close() now write them, trying once more (waiting out the 5 s busy timeout, so close() blocks for at most about 10 s); if they still fail, the database stops writing and reports a storage failure. The engine now releases the database before it removes its write-failure listener, so a failure at close reaches the closing client's onStorageFailure. Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

The database collector decided on the main actor which files were open, then read their heartbeats and unlinked them later on the storage queue. A client that opened one of those databases in between kept writing to deleted files, and its pending mutations were gone at the next launch with no storage failure. Now the queue only finds the databases no client used for two days; on the main actor, the files opened or closed in this process since the collection started are kept, and the others are renamed out of the way (a cheap rename, on the main actor) and deleted in the background. A client opening that path afterwards starts a new file. Files left renamed by an earlier run are deleted by the next collection. flush() finishes the collections found so far. The collector also matched every zero-*.sqlite file, so an older binary sharing the directory deleted a newer layout's database. As gatherDatabaseInfoForCollect never collects a newer format, it now only considers files of this layout version or an older one that hold their name in meta, with a Replicache format this client knows. Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

Stored data that doesn't decode (a mutation's args, a group's mutator names) was reported as a storage failure, so every launch ran on memory and persisted nothing, forever. As Replicache does for an invalid ref count, it is now lost client state: the client deletes the database, starts a new one that persists, and calls onClientStateNotFound if the app set it. Unlike JS, which reloads by default, the client is already on the new database, which is what replacing it would do (JSParity.md). SQLite errors stay storage failures and fall back to memory as before. zero.delete() and dropAllDatabases now also drop the memory databases clients of that directory fell back to after their SQLite store failed, as JS's dropTargets drops the fallback stores. The memory databases of .memory clients are left alone. Dropping a database that was already dropped (a second client of the group getting InvalidConnectionRequestBaseCookie after the first was replaced) unlinked the file the replacement had started, with its pending mutations. A database is now dropped once, and a database whose file another opened since owns is closed without deleting it. Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

When a connection attempt timed out before the WebSocket handshake finished, close() only marked the socket closed: the task waiting in Transport.connect kept waiting, so each hung attempt against a black-holed server leaked a socket and a task. close() now cancels that task, which cancels the transport's connect, as closing a connecting JS WebSocket aborts it. Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

… unpartitioned A flipped whereExists over a subquery with a limit (for example `issue.whereExists('comments', q => q.limit(1), {flip: true})`) fetches the limited child without a constraint. JS's Take asserts there, and the error comes out of materialize; Swift stopped the process on a precondition. Take.fetch now reports the error to a PipelineFailure capture and yields nothing; hydration (ArrayView in materialize, DeferredInput.attach for views made before the store loads) runs inside a capture and throws it. So materialize and liveQuery throw "Partitioned take does not allow unpartitioned fetches", run rejects with it, and a view deferred until the store loads becomes an error view, as in JS. A fetch outside a capture (a push that refetches) still traps; JS throws there out of the push, which Swift can't surface yet. This is done at hydration rather than at build time: whether the child is fetched unpartitioned depends on the shape of the flipped join above it, and a build-time check would have to reproduce that exactly. Hydration-time detection matches JS on every hydration error in the hardening fuzz. Unlike JS, a materialize that fails leaves nothing registered with the server: there's no view to destroy. Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

…re-enters A mutate from a view listener committed synchronously, inside the notification: its flush ran before the listeners after the current one had heard the earlier change, so they then got the earlier, stale rows last, and other views skipped a state JS shows. JS mutators are async, so a mutation from a listener commits once the notification is done. Engine.mutate now defers a mutation made while listeners are notified to the end of the current task (EventLoop.afterCurrentTask, before the task's microtasks; as a microtask when no task runs). Its outcomes settle then. Outside listeners nothing changes: mutate still commits at once and views are current when it returns. Listeners are iterated as JS iterates a live Set: one removed during the pass is skipped, one added is called in the same pass, in ArrayView and in MaterializedView. A view destroyed by one of its listeners still notifies its other listeners of the change under way, as JS does. Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

…view can't take a change Applying a change could stop the process where JS throws from the push: a limited flipped whereExists reached by a new row ("Partitioned take does not allow unpartitioned fetches"), two rows for a `one` relationship ("Singular relationship ... should not have multiple rows", reachable with ordinary server data), or a view that lost track of a row ("node does not exist"). These operators now report the error to the PipelineFailure capture and leave the view as it was. QueryContext.processChanges pushes inside a capture, and the source stops where JS's throw would unwind: no later connections, no write to the source, no later changes of the diff. The views flush what they got, as JS's finally does, and processChanges throws. As in JS (`#advanceError` in ivm-branch.ts), the branch is unusable from then on: every later push, materialize, and mutation fails with the same error. - A poke whose push fails is a poke error: the connection goes to the error state with "Poke processing error: Error: ...". - A mutation whose push fails stays committed and pending but isn't pushed now; its client outcome is the app error. Later mutations fail on the client and the server, run rejects, and materialize throws, without registering the query (JS registers it first). Hydration already ran in a capture. With every client path covered, an operator that reports outside a capture (a pipeline driven directly) logs the error at .error and yields nothing instead of trapping. Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

An unknown literal, such as an error kind the client doesn't know, reads 'Expected literal value "A", "B" or "C" at <path> Got "x"', and a primary key that isn't a string, number, or boolean 'Expected string, number or boolean. Got undefined', a TypeError in a poke error and in a mutation's details, as in JS. Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

…ffer from JS JS breaks on a value of the wrong type for its column: the mutation that writes it and every later one fail, and a poke with it disconnects at every reconnect. Swift sorts it with the others and keeps working, which JSParity now says. The vectors that asserted JS's broken state, and the one comparing V8's JSON.parse wording, are dropped. Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

tmm force-pushed the swift-hardening branch 3 times, most recently from a793659 to eb4e1b7 Compare October 27, 2026 00:16

ampagent and others added 21 commits October 27, 2026 01:50

test(zero-swift): hardening tests for the connection and protocol

87c5b4d

Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

test(zero-swift): hardening tests for mutations

a0570a3

Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

test(zero-swift): hardening tests for crashes on untrusted input

c354764

Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

test(zero-swift): hardening tests for storage and the client lifecycle

3ff5a03

Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

test(zero-swift): end-to-end chaos tests

b477862

Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

test(zero-swift): hardening tests for queries, views, and IVM

00b92ab

Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

chore(zero-swift): changeset for the client hardening fixes

cb1cfeb

Amp-Thread-ID: https://ampcode.com/threads/T-01a17664-8fd0-76be-8fd5-75ac6e7aa0d8 Co-authored-by: tmm <tmm@tmm.dev>

tmm force-pushed the swift-hardening branch from eb4e1b7 to cb1cfeb Compare October 27, 2026 01:50

tmm merged commit f7e8a23 into main Oct 27, 2026
19 checks passed

tmm deleted the swift-hardening branch October 27, 2026 03:15

tmm mentioned this pull request Oct 27, 2026

docs(zero-swift): storage waits, schemas sharing a table name, and an IVM difference from JS #43

Merged

Navigation Menu

Repository navigation

fix(zero-swift): harden the client against crashes, data loss, and JS divergences - #38

fix(zero-swift): harden the client against crashes, data loss, and JS divergences#38
tmm merged 21 commits into
mainfrom
swift-hardening

tmm commented Oct 26, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

tmm commented Oct 26, 2026

Crashes (a trap kills an iOS app; JS throws, logs, or disconnects)

Data loss in storage

Client behavior

Differences from JS, documented in JSParity.md

Verification (Linux)

Known leftovers

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants