1.0 hardening: fix @transaction early-return composition - #7
Merged
Conversation
The return-rewrite threw an untagged TransactionReturn marker, so the dynamically nearest @transaction expansion always intercepted it: - a return inside a NESTED @transaction committed only the inner savepoint, and the inner expansion's own plain `return` then skipped every enclosing commit — all levels' work was silently rolled back and the connection was left inside the outer transaction - a user try/catch inside the body swallowed the marker and returned the catch's value instead of the intended return value, silently - a return inside Threads.@Spawn / @async in the body was rewritten too, so the task threw the marker instead of producing its value Each expansion now tags its markers with a compile-time token. A catch that receives a foreign marker commits its own level and keeps unwinding to the owning expansion, so an early return commits every enclosing level and returns exactly once. User catch blocks get a guard injected that rethrows the marker (a private type no handler can mean to catch). Task-forming macros are excluded from the rewrite, matching the existing exclusion of closures. break/continue — which bypass both the commit and any catch — now commit via a finally, making every non-exceptional exit consistent: only a thrown exception rolls back. Documented in the docstring. Regression tests cover nested return, both catch shapes, @Spawn, break, continue, and recursive re-entry of the same expansion; removing the fix fails six of them plus downstream testsets poisoned by the stuck-open transaction. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
quinnj
commented
Aug 7, 2026
h(x) = ... parses as :(=) with a call-shaped left-hand side, not as :function, so the rewrite's closure exclusion missed it: a return inside a local short-form helper defined in the @transaction body was rewritten into a transaction-return marker. Calling such a helper silently early-returned the ENCLOSING function with the helper's internal value (committing on the way out), and a helper that escaped the block threw a raw TransactionReturn at its caller with no expansion active to catch it. All short-form shapes are skipped (plain, ::T return-type, where-clauses, qualified names), while ordinary assignments whose right-hand side contains a return are still rewritten. Also adds @spawnat to the task-macro skip list — same bug class as @spawn/@async, verified to wrap the marker in a RemoteException instead of producing the task's value. Live test: a short-form helper with an internal early return, used inside the block and after it escapes. Unit pins for every definition shape, the task macros, and the ordinary-assignment counter-cases. Removing the skip fails six of them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
quinnj
commented
Aug 7, 2026
`return` anywhere inside a comprehension or generator — body or iterator expression — is a lowering error in plain Julia. The rewrite turned it into a legal `throw` of the transaction-return marker, silently accepting code that would stop compiling the moment the @transaction wrapper is removed, and giving it early-return semantics it never legitimately had. Comprehension, typed-comprehension, generator, and flatten heads are now left untouched so the construct errors exactly as it does everywhere else. Nothing valid is lost: a legal comprehension cannot contain a bare `return`, and nested closures inside one were already excluded. Unit pins cover all four syntactic shapes plus the counter-case that a `return` inside an ordinary `for` loop is still rewritten. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
quinnj
commented
Aug 7, 2026
Distributed.@fetch and @fetchfrom wrap their body in a remotely-executed thunk whose return is the fetched value, exactly like @spawnat — but they were missing from _TASK_MACROS, so a return inside one was rewritten into a transaction-return marker. Verified live: the block then throws a RemoteException wrapping the marker and rolls back, where plain Julia returns the value. Also corrects the comprehension-skip rationale in comments: a return in a comprehension/generator BODY is a lowering error (which the rewrite must not legalize), while the iterator-expression shapes lowering does accept behave correctly un-rewritten — they exit the block non-exceptionally and commit through the expansion's finally, as verified live. And documents at the token comparison that unconditional returning would be observationally equivalent today only because every enclosing expansion's finally also commits; the token check stays as the semantic guarantee. Independent adversarial verification of the three @transaction commits (61 live scenarios, plain-Julia baselines, 6 mutations against the full suite) found no other behavioral gaps. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…nally Review of the rewrite approach (PR #7 threads) proved its task-macro allowlist structurally insufficient: any third-party macro that wraps its body in a task or closure — reproduced with a minimal @local_task — had its internal returns rewritten into transaction-return markers, throwing TaskFailedException(TransactionReturn) instead of producing the task's value. No finite list of standard macros covers user-defined ones. The expansion's finally already gives plain `return` the intended semantics with no rewriting at all: a return unwinds through every enclosing expansion's finally, each committing its level exactly once, innermost first. User catches cannot intercept a plain return, closures and task macros keep their ordinary meaning untouched, and the flattened-iterator form that plain lowering accepts behaves identically wrapped or not. The marker struct, the AST walker, the try-guard injection, and both skip lists are deleted. The finally also now handles its own commit failure: it rolls back the current level before propagating (commit at savepoint depth leaves depth unchanged on failure), so every enclosing level — macro expansion or plain catch — unwinds its own. Previously a RELEASE SAVEPOINT failure during a break out of a nested level (savepoint aborted by a swallowed server error) escaped past the enclosing macro's ability to clean up, leaving the outer transaction open with its work pending. Behavioral regressions replace the deleted unit AST pins: a third-party @local_task macro, Distributed @spawnat/@fetch/@fetchfrom run locally on worker 1, plain-vs-wrapped flattened-iterator equivalence, and the nested break with an aborted savepoint (asserts the server error surfaces and nothing stays open client- or server-side; removing the finally rollback fails five assertions). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Member
Author
|
READY/CLEAN on exact head Independent PostgreSQL 16 retests passed: standard and third-party task macros preserve plain return semantics and commit; a failed |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Continuation of the 1.0 readiness work from #5, on a fresh branch against merged main: independent adversarial review rounds against a live PostgreSQL, fixing what they surface, until a round comes back clean.
The problem:
@transactionearly-returnwas unsound under compositionRound 13 (the first review of #5's merged tail) found that the early-return support rewrote
return xinto an untagged thrown marker, so the dynamically nearest@transactionexpansion always intercepted it. Verified live: nested@transaction+returnsilently rolled back all levels and left the connection stuck in a transaction; a usertry/catchswallowed the marker and returned its own fallback value;returninside task macros threw the marker instead of producing the task's value;break/continueleft the transaction open.The evolution of the fix (rounds 14–16 + external review)
Commits 1–4 fixed this incrementally: per-expansion tokens, guards injected into user catches, and a growing skip list of closure/task-forming constructs (short-form defs,
@spawnat,@fetch/@fetchfrom, comprehensions). Each round's reviewer found the next hole in the allowlist.External review (codex) then proved the endpoint of that trajectory: no finite allowlist can cover third-party task macros (reproduced with a minimal
@local_task), and — the key insight — the expansion'sfinallyalready gives plainreturnthe intended semantics with no rewriting at all.Final design (commit 5, net −140 lines)
The marker struct, AST walker, try-guard injection, and both skip lists are deleted. The macro is now just try/catch/finally:
return,break,continue. Areturnunwinds through every enclosing expansion'sfinally, each committing its level exactly once, innermost first. Only a thrown exception rolls back.return; closures,do-blocks, comprehensions, and any task-forming macro (standard or third-party) keep their ordinary meaning untouched.finallyhandles its own commit failure (the second P1 from external review): it rolls back the current level before propagating —commitat savepoint depth leaves the depth unchanged on failure, so each enclosing level unwinds its own. Previously aRELEASE SAVEPOINTfailure during abreakout of an aborted nested level escaped cleanup and left the outer transaction open with its work pending.Verification
Threads.@spawn, a third-party@local_taskmacro,Distributed.@spawnat/@fetch/@fetchfrom(run locally on worker 1), short-form helpers escaping the block,break/continue, recursion re-entering the same expansion, plain-vs-wrapped flattened-iterator equivalence, and the nested-break-with-aborted-savepoint commit-failure case (asserts the server error surfaces and nothing stays open client- or server-side).🤖 Generated with Claude Code