fix(driver-sql): bulkCreate and upsert re-seed a stale autonumber counter instead of burning the whole batch (#6943) - #6999
Merged
Conversation
…nter (#6943) `create()` learned this at #5495; `bulkCreate()` and `upsert()` call the same `fillAutoNumberFields` and did not. Measured, they are not the same defect: - `upsert` is `create()`'s old shape exactly — single row, one burned number per refused call. Its `ON CONFLICT DO UPDATE` absorbs a merge-key conflict only; the tenanted autonumber sits under a different unique index. - `bulkCreate` is worse. Each row reserves in its own committed transaction and the batch goes in as ONE insert, so one colliding row burns every number the batch reserved and fails the whole request — on the path framework#2678 made the common case for seed/import, which is what creates the staleness. Both now reuse #5495's machinery unchanged: `collidingAutoNumberReservations` for the three-state routing, `autoNumberValueExists` as the data-based discriminator, forward-only `resyncSequenceToDataMax`. Batch semantics are unchanged by measurement, not by choice: `insert(rows[])` is a single statement, so the batch was already all-or-nothing and re-issuing it whole preserves the contract exactly. Per-row retry was rejected — it would have to split the statement and invent partial success. Re-issue is per counter, not per row: a batch straddling the seeded range would otherwise regenerate only its low rows and hand them numbers above the kept ones, an intra-batch duplicate. Retry stays confined to the no-caller-transaction case, as at #5495. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DZrSGPUVrFqYCKc3ELYRqR
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
Contributor
📓 Docs Drift CheckThis PR changes 1 package(s): 9 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:
|
…try passes A retry regenerates only the fields it cleared, so `fillAutoNumberFields` reports only those. A value kept from the previous pass (drawn from a counter that did NOT go stale) is still a live reservation, and dropping it from `reservationsPerRow` meant a second collision on that counter would find nothing to route and rethrow blind. Carry the per-row reservation list across attempts, replacing only what was re-issued. Safe direction either way — the old shape rethrew the driver's own error, which is today's behaviour — but the invariant is now what the loop claims it is. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DZrSGPUVrFqYCKc3ELYRqR
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #6943.
Two paths, two different shapes — measured, not inherited
The card said
bulkCreateandupsert"still burn a number per collision, likecreate()". Only one of them does. Re-derived onmain@c8ff269, on a fresh DB with seeded rows above the counter (#5495's repro constraint: the storm is one-time per database, and on a DB already ground past the seeds everything passes and the defect is invisible).upsertiscreate()'s pre-#5495 shape exactly. Single row, so a stale counter costs it one burned number per call —last_valuewalked 1 → 2 → 3 across two refused upserts. Worth stating because it is not obvious:ON CONFLICT (mergeKeys) DO UPDATEabsorbs a conflict on the merge key only, and the tenanted autonumber lives under a different unique index, so its violation is still raised and still reaches the caller.bulkCreateis a sharper shape. Each row reserves its number in its own committed transaction, then the whole batch goes in as one insert. So one colliding row does not burn one number — it burns every number the batch reserved and fails the entire request:bulkCreate, counter at 10, rows 11–39 already presentlast_valueAnd the exposure is the worst available:
bulkCreate's own docstring records that framework#2678 made it the common case for seed/import — and seed/import is precisely what creates the staleness, since anisSystemreplay or apreserveAuditimport keeps its explicit numbers and never entersfillAutoNumberFields(#5495/#5503). The path most likely to meet a stale counter was the one with no recovery at all.Batch semantics are unchanged, and that is a measurement
This card was split out of #5495 because per-row retry inside a batch was expected to force a decision about partial success, transaction boundaries, and whether a failed row rolls back its siblings.
There is no such decision to make.
insert(rows[])is a single statement (… select … union all select …on SQLite, multi-rowVALUESelsewhere), so the batch is already all-or-nothing — the failed 3-row batch above left the table at exactly the 31 rows it started with. Re-issuing and retrying the whole batch therefore preserves the existing contract byte for byte: no partial success is introduced, no transaction is opened, and the sibling-rollback question never arises because siblings already fail together.Rejected, with reasons, in
bulkCreate's own comment:getNextSequenceValuecommits on purpose, which is what makes a forward-only re-seed meaningful), and on SQLite withpool max = 1is the deadlockensureSequencesTablealready documents.The one thing the batch may not borrow from
create()create()keeps a reservation that did not collide, to avoid burning a second number. A batch cannot, and this is the non-obvious correctness point of the change.A batch that straddles the seeded range — counter at 10, rows 11–39 present, batch reserves 11–70 — has its low rows collide and its high rows not. Re-issuing only the collided ones hands them numbers above the kept ones: an intra-batch duplicate the driver would have manufactured itself. So re-issue is per counter: every row drawn from a counter that went stale is re-issued, and counters that did not go stale keep their values, so a co-tenant's rows in the same batch are undisturbed. Both halves are pinned by tests.
Machinery reused, not rebuilt
collidingAutoNumberReservations(three-state routing),autoNumberValueExists(the data-based discriminator — the conflicting column is never determinable for a tenanted autonumber, so routing cannot rest on the error text), and the forward-onlyresyncSequenceToDataMaxare all #6932's, unchanged. No fifth dialect word-list: the judgement isisUniqueViolationError/uniqueViolationColumnfrom@objectstack/types(Prime Directive #12).One helper is new —
autoNumberCounterKey, which reuses the existingsequenceKeyHash.create()never needed it because one row draws at most one reservation per field; a batch has many rows sharing one counter, and both of the batch's decisions (re-seed once, re-issue all) are per counter.Retry stays confined to the no-caller-transaction case, as at #5495. Inside a caller's transaction the sequence
UPDATErolls back with the refusedINSERT, so nothing is burned and there is nothing to repair — re-measured on both paths here — and on Postgres a constraint failure aborts the transaction outright. The caller owns that retry.Three faces
SqlDriverSqliteWasmDriverTursoDriverlocal/replicasuper; own testTursoDriverremoteRemoteTransport.bulkCreatebuilds its own INSERT, never entersfillAutoNumberFieldsTurso needed saying out loud more than at #5495: it overrides
bulkCreate/upsertrather than merely inheriting, so "the base class was fixed" is not on its own an answer about that face. The remote face's absence is pinned as an assertion, not a comment, so wiring autonumber into that transport later (#6944) cannot silently inherit this file's green.driver-memory/driver-mongodbare untouched and inside the #5499 freeze — neither declaressupports.autonumber, so both use the engine fallback (#6806's surface). A real absence, not a skipped row.Out of scope, recorded not fixed
upsertburns a number on every merge, and overwrites the record's autonumber. Independent of any stale counter:fillAutoNumberFieldsruns before the statement knows whether it will insert or merge, and the autonumber column is inmergeColumns. Measured on a correct counter — creatingCASE-00001then upserting the same row by id twice yieldedCASE-00002thenCASE-00003, one row throughout,last_value1 → 2 → 3. That is an unbounded burn on the success path, and it mutates a business identifier rather than merely leaving a gap. It is not this card's defect (no staleness involved), and fixing it is its own semantics decision — should an upsert that updates keep the old number, and should it reserve at all? — of exactly the kind that got this card split from #5495. Recorded for triage; deliberately not fixed here. Guarded only to the extent that the retry loop cannot turn a merge into an insert.🤖 Generated with Claude Code
https://claude.ai/code/session_01DZrSGPUVrFqYCKc3ELYRqR
Generated by Claude Code