Skip to content

fix(replay): repair-transaction lifecycle — keep-alive, no-partial-emit, close-as-lifecycle, atomic publish#1235

Open
thymikee wants to merge 10 commits into
mainfrom
feat/repair-transaction-lifecycle
Open

fix(replay): repair-transaction lifecycle — keep-alive, no-partial-emit, close-as-lifecycle, atomic publish#1235
thymikee wants to merge 10 commits into
mainfrom
feat/repair-transaction-lifecycle

Conversation

@thymikee

@thymikee thymikee commented Jul 13, 2026

Copy link
Copy Markdown
Member

Summary

Implements the four "repair transaction" lifecycle fixes for ADR-0012 Decision 6 (agent-supervised re-record repair, #1228), closing real end-to-end gaps found in live testing.

Unifying model: replay <file>.ad --save-script opens a multi-invocation repair TRANSACTION (ARMED → COMPLETE → COMMITTED). It is committed (the healed .ad atomically published) on any teardown of the session — explicit close, idle-reap, or daemon shutdown — but only when the transaction is COMPLETE; a teardown while INCOMPLETE never publishes a prefix. Continuation keys off persisted transaction state, so replay --from continues without repeating --save-script. Everything below serves that model.

  • Fix 1 (Q1)cleanupDaemonAfterRequest (src/daemon/client/daemon-client-lifecycle.ts) no longer tears down a cold, owned-ephemeral daemon when a repair-armed replay returns a held REPLAY_DIVERGENCE. Keep-alive keys off the daemon's resume.repairSessionHeld signal (markRepairSessionHeldIfArmed, set from persisted armed-uncommitted session state — not the per-request --save-script flag, so a replay --from continuation that itself diverges stays held) via isHeldRepairDivergence; it is independent of resume.allowed (plan-resumability). Because an owned/ephemeral state dir (fs.mkdtempSync) is otherwise unaddressable by the agent's next command, the response carries a hint pointing at --state-dir <path> (attachRepairSessionAddressHint, src/daemon/client/daemon-client.ts).
  • Fix 2 (Q2a)SessionScriptWriter.write (src/daemon/session-script-writer.ts) publishes a repair-armed session's (saveScriptBoundary set) healed .ad only when the transaction is COMPLETE (isRepairArmedWriteBlocked gates on saveScriptComplete, idempotent once saveScriptCommitted). Commit happens at any teardown of a COMPLETE transaction — explicit close, idle-reap, and daemon shutdown all route through finalizeRepairTeardown (src/daemon/session-store.ts) → the writer. A teardown while INCOMPLETE publishes nothing (never a prefix): a plain close discards; an idle-reap/shutdown leaves a REPAIR_SESSION_EXPIRED tombstone (surfaced by request-router), and hasReapBlockingOpenSessions (src/daemon/handlers/daemon-idle-reap.ts) lets an uncommitted repair session reap rather than pinning the daemon forever.
  • Fix 3 (Q2b) — the source plan's own terminal close is skipped (isRepairArmedTerminalClose, src/daemon/handlers/session-replay-runtime.ts) while a repair is armed, instead of dispatching as an ordinary step and diverging on lifecycle. Verification already exempted unannotated actions from target-binding divergence kinds (verifyReplayActionTarget); the fix removes the remaining action-failure misfire by never dispatching it at all during repair.
  • Fix 4 (Q2c) — the healed script publishes atomically via create-exclusive linkSync with the temp file in the target's own directory (publishHealedScriptAtomically — same-filesystem rename, race-safe no-clobber even for a cross-mount explicit --save-script=<path>) and carries a # agent-device:heal-complete sentinel; assertNoCompleteHealedClobber refuses to overwrite only a complete prior artifact — a stale/partial file (no sentinel) is silently overwritable.

Files touched

  • src/daemon/client/daemon-client-lifecycle.tscleanupDaemonAfterRequest (+isResumableRepairDivergence, attachRepairSessionAddressHint)
  • src/daemon/client/daemon-client.tssendToDaemon (+withRepairSessionAddressHintIfOwned)
  • src/daemon/session-script-writer.tsSessionScriptWriter.write, isRepairArmedWriteBlocked, HEAL_COMPLETE_SENTINEL, publishHealedScriptAtomically, assertNoCompleteHealedClobber
  • src/daemon/session-store.tsfinalizeRepairTeardown (commit-or-tombstone at teardown)
  • src/daemon/handlers/session-replay-runtime.tsisRepairArmedTerminalClose, markRepairSessionHeldIfArmed, createReplaySaveScriptArmer, preflightReplayAgainstActiveRepair, sets saveScriptComplete at plan completion
  • src/daemon/request-router.tsREPAIR_SESSION_EXPIRED for a reaped uncommitted repair session (tombstone)
  • src/daemon/handlers/daemon-idle-reap.tshasReapBlockingOpenSessions
  • src/daemon/types.tsSessionState.saveScriptComplete/saveScriptCommitted/saveScriptBoundary; resume.repairSessionHeld

New tests

  • src/utils/__tests__/daemon-client-lifecycle.test.ts: keeps an owned daemon alive + hints --state-dir on a resumable divergence; still tears down on a non-resumable divergence; still tears down without --save-script.
  • src/daemon/__tests__/session-script-writer.test.ts: discards an unfinalized repair-armed write; ordinary recording unaffected; never sentinel-tags ordinary recordings; overwrites a stale incomplete .healed.ad; refuses to clobber a complete one; atomic publish leaves no stray temp file.
  • src/daemon/handlers/__tests__/session-replay-repair-loop.test.ts: terminal close skipped while armed; ordinary replay still dispatches its close; only the terminal close is skipped (mid-plan close still dispatches); a --from resume landing on the terminal close completes instead of diverging.
  • src/daemon/handlers/__tests__/session-replay-repair-transaction.test.ts (new): end-to-end chain — cold divergence stays alive (session never deleted) → corrective press + --from resume runs to completion (terminal close skipped) → close --save-script commits a complete, atomically-published healed .ad with no bare @refs → a separate diverged-and-abandoned repair (close without --save-script) leaves no partial file on disk.

Test plan

  • npx vitest run — full suite: 467 files / 4097 tests, all green
  • npx tsc --noEmit — clean
  • npx oxlint --deny-warnings — clean
  • npx oxfmt --check — clean on all touched files (37 pre-existing unrelated files were already failing on origin/main before this change)
  • pnpm check:fallow --base origin/main — no issues in 11 changed files

No live simulator was run (per instructions — a human will re-run the live prevent-remove heal separately). No device-lease semantics were touched beyond the session staying open (hasOpenSessions) for the repair's lifetime.

@github-actions

github-actions Bot commented Jul 13, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 1.7 MB 1.7 MB +8.1 kB
JS gzip 542.7 kB 545.3 kB +2.5 kB
npm tarball 653.4 kB 655.9 kB +2.6 kB
npm unpacked 2.3 MB 2.3 MB +8.1 kB

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 28.3 ms 27.6 ms -0.7 ms
CLI --help 59.3 ms 58.7 ms -0.7 ms

Top changed chunks:

Chunk Raw diff Gzip diff
dist/src/internal/daemon.js +5.0 kB +1.5 kB
dist/src/session.js +2.1 kB +621 B
dist/src/agent-device-client.js +587 B +236 B

thymikee added a commit that referenced this pull request Jul 13, 2026
- C1: R7 keep-alive keys off a DISTINCT resume.repairSessionHeld signal, not
  resume.allowed (which means plan-resumability and fires for every divergence).
- C2: define the ARMED -> COMPLETE -> COMMITTED commit state machine. close
  before COMPLETE = abort (publish nothing); close at COMPLETE = atomic commit;
  close after COMMITTED = idempotent teardown, no re-publish. No auto-commit.
- C4: precise terminal-source-close contract (last source action == close) —
  SKIPPED (not dispatched) under armed repair so the session is not deleted;
  regression required (added to migration step 9).
- C5a: REPAIR_SESSION_EXPIRED backed by a bounded tombstone keyed by session key
  (owner + expiry), cleared by a fresh replay --save-script.
- C5b: atomic publication temp file in the target's own directory; race-safe
  no-clobber via create-exclusive/rename-if-absent.
- C6: Status block corrected — Decisions 1-6 base MERGED (#1228 et al.); R7 +
  commit machine UNIMPLEMENTED, tracked by #1235. Migration split into step 8
  (merged) and step 9 (#1235). Validation extended for all of the above.
thymikee added a commit that referenced this pull request Jul 13, 2026
…tion

Resolves the two contract ambiguities blocking merge (aligns with #1235):

- Completion model = TEARDOWN-COMMITS (not explicit-close-only). A repair-armed
  session stays addressable until the transaction ends; ANY teardown (explicit
  close, idle-reap, daemon shutdown) commits the healed .ad atomically iff the
  transaction is COMPLETE, else aborts with no publish (never a prefix). An
  incomplete reap/shutdown leaves the REPAIR_SESSION_EXPIRED tombstone; an
  explicit close of an incomplete tx just discards. Kept the
  ARMED->COMPLETE->COMMITTED machine and idempotent post-COMMITTED teardown;
  removed the "no auto-commit / commit only on explicit close" language.
- Continuation by PERSISTED transaction state, not the per-request flag:
  replay --from <n> --plan-digest <sha> resumes on the persisted repair-armed
  session WITHOUT repeating --save-script; --save-script appears only on the
  transaction opener. Implementation MUST key keep-alive/continuation off
  persisted state. Decision 4 repairSessionHeld updated to match.

Edited: R7, Decision 4 resume signal, terminal-close, Emitting, commit state
machine, tombstone, migration step 9, and validation.
@thymikee thymikee force-pushed the feat/repair-transaction-lifecycle branch from c3829c4 to d72ec31 Compare July 13, 2026 10:24
@thymikee

Copy link
Copy Markdown
Member Author

Coordinator review at d72ec310a: prior completion-gate and persisted-state keep-alive defects are fixed, but four lifecycle/publication blockers remain despite 22 green checks.

  1. A reaped replay --from becomes REPLAY_DIVERGENCE before the router can translate the tombstone to REPAIR_SESSION_EXPIRED; current coverage tests only close.
  2. Teardown commit failures can be invisible: generic filesystem errors become {written:false}, replay success is already returned, explicit close still deletes the session after no-clobber failure, and output does not positively report the healed artifact path/commit result.
  3. No-clobber is still racy for an existing partial target: two writers can both pass the sentinel check and the later renameSync can overwrite the first complete artifact. Add a competing-writer regression.
  4. A two-step script containing open then terminal close never arms the transaction: pre-open arming is a no-op, then close is dispatched because no boundary exists. Cover and satisfy ARMED-before-step-1 semantics.

Ordering remains: resolve and merge governing ADR #1234 first, then rebase #1235 onto it and align the stale PR description. Practical shipped-CLI simulator evidence is still required for held divergence/continuation, commit and fresh replay, expiry recovery, clobber/unwritable/concurrent output, and cleanup. No fixer dispatched; readiness remains withheld.

thymikee added a commit that referenced this pull request Jul 13, 2026
- C1: R7 keep-alive keys off a DISTINCT resume.repairSessionHeld signal, not
  resume.allowed (which means plan-resumability and fires for every divergence).
- C2: define the ARMED -> COMPLETE -> COMMITTED commit state machine. close
  before COMPLETE = abort (publish nothing); close at COMPLETE = atomic commit;
  close after COMMITTED = idempotent teardown, no re-publish. No auto-commit.
- C4: precise terminal-source-close contract (last source action == close) —
  SKIPPED (not dispatched) under armed repair so the session is not deleted;
  regression required (added to migration step 9).
- C5a: REPAIR_SESSION_EXPIRED backed by a bounded tombstone keyed by session key
  (owner + expiry), cleared by a fresh replay --save-script.
- C5b: atomic publication temp file in the target's own directory; race-safe
  no-clobber via create-exclusive/rename-if-absent.
- C6: Status block corrected — Decisions 1-6 base MERGED (#1228 et al.); R7 +
  commit machine UNIMPLEMENTED, tracked by #1235. Migration split into step 8
  (merged) and step 9 (#1235). Validation extended for all of the above.
thymikee added a commit that referenced this pull request Jul 13, 2026
…tion

Resolves the two contract ambiguities blocking merge (aligns with #1235):

- Completion model = TEARDOWN-COMMITS (not explicit-close-only). A repair-armed
  session stays addressable until the transaction ends; ANY teardown (explicit
  close, idle-reap, daemon shutdown) commits the healed .ad atomically iff the
  transaction is COMPLETE, else aborts with no publish (never a prefix). An
  incomplete reap/shutdown leaves the REPAIR_SESSION_EXPIRED tombstone; an
  explicit close of an incomplete tx just discards. Kept the
  ARMED->COMPLETE->COMMITTED machine and idempotent post-COMMITTED teardown;
  removed the "no auto-commit / commit only on explicit close" language.
- Continuation by PERSISTED transaction state, not the per-request flag:
  replay --from <n> --plan-digest <sha> resumes on the persisted repair-armed
  session WITHOUT repeating --save-script; --save-script appears only on the
  transaction opener. Implementation MUST key keep-alive/continuation off
  persisted state. Decision 4 repairSessionHeld updated to match.

Edited: R7, Decision 4 resume signal, terminal-close, Emitting, commit state
machine, tombstone, migration step 9, and validation.
@thymikee thymikee force-pushed the feat/repair-transaction-lifecycle branch from d72ec31 to d55395b Compare July 13, 2026 11:21
@thymikee

Copy link
Copy Markdown
Member Author

Additional documentation finding: this PR description still presents close --save-script as the only commit trigger, contradicting the amended ADR and current implementation where teardown commits a COMPLETE transaction. Please update the PR description.

The previously reported implementation blockers remain unchanged; this comment does not supersede them.

@thymikee

Copy link
Copy Markdown
Member Author

Addressed: PR description updated to match the amended ADR + implementation — commit is teardown-gated on COMPLETE (not close --save-script-only): finalizeRepairTeardown commits a COMPLETE transaction on any teardown, incomplete teardown publishes nothing (reap → tombstone). Also refreshed Fix 1/2/4 + the file list to the current symbols.

thymikee added a commit that referenced this pull request Jul 13, 2026
… semantics) (#1234)

* docs: ADR-0012 Decision 6 — repair-transaction lifecycle (R7 + commit semantics)

Frames --save-script as a multi-invocation repair TRANSACTION committed only on
completion:
- R7 (new normative rule): a repair-armed replay returning resume.allowed:true
  must keep its daemon/session live until close; heal/--from target that same
  session (strengthens R2). Plain close/teardown/idle-reap while armed = abort/
  discard. Bounded-expiry must surface REPAIR_SESSION_EXPIRED, not bare
  SESSION_NOT_FOUND. Persistent-daemon precondition rejected: fail-fast before
  step 1, never a later SESSION_NOT_FOUND.
- Decision 4 resume: one sentence noting the session is kept addressable so
  resume.allowed:true is not misleading.
- Commit semantics: healed .ad committed only on full-plan completion or explicit
  close --save-script; never on divergence-only exit, teardown, or idle-reap;
  atomic temp->publish. R6 defines the slice, this defines when it is complete.
- Terminal lifecycle steps: non-target steps (incl. source close, unannotated
  steps) are already exempt from target-binding divergence per decision 3
  (clarification, not a change); prefer SKIPPING the source terminal close while
  armed and finalize via close --save-script.
- Clobber P2: no-clobber guards a COMPLETE (heal-complete sentinel) artifact only;
  partials are overwritable; auto-versioned names out of scope.
Validation + migration step 8 extended accordingly.

* docs: ADR-0012 R7 — fix 6 architecture blockers (transaction contract)

- C1: R7 keep-alive keys off a DISTINCT resume.repairSessionHeld signal, not
  resume.allowed (which means plan-resumability and fires for every divergence).
- C2: define the ARMED -> COMPLETE -> COMMITTED commit state machine. close
  before COMPLETE = abort (publish nothing); close at COMPLETE = atomic commit;
  close after COMMITTED = idempotent teardown, no re-publish. No auto-commit.
- C4: precise terminal-source-close contract (last source action == close) —
  SKIPPED (not dispatched) under armed repair so the session is not deleted;
  regression required (added to migration step 9).
- C5a: REPAIR_SESSION_EXPIRED backed by a bounded tombstone keyed by session key
  (owner + expiry), cleared by a fresh replay --save-script.
- C5b: atomic publication temp file in the target's own directory; race-safe
  no-clobber via create-exclusive/rename-if-absent.
- C6: Status block corrected — Decisions 1-6 base MERGED (#1228 et al.); R7 +
  commit machine UNIMPLEMENTED, tracked by #1235. Migration split into step 8
  (merged) and step 9 (#1235). Validation extended for all of the above.

* docs: ADR-0012 R7 — teardown-commits model + persisted-state continuation

Resolves the two contract ambiguities blocking merge (aligns with #1235):

- Completion model = TEARDOWN-COMMITS (not explicit-close-only). A repair-armed
  session stays addressable until the transaction ends; ANY teardown (explicit
  close, idle-reap, daemon shutdown) commits the healed .ad atomically iff the
  transaction is COMPLETE, else aborts with no publish (never a prefix). An
  incomplete reap/shutdown leaves the REPAIR_SESSION_EXPIRED tombstone; an
  explicit close of an incomplete tx just discards. Kept the
  ARMED->COMPLETE->COMMITTED machine and idempotent post-COMMITTED teardown;
  removed the "no auto-commit / commit only on explicit close" language.
- Continuation by PERSISTED transaction state, not the per-request flag:
  replay --from <n> --plan-digest <sha> resumes on the persisted repair-armed
  session WITHOUT repeating --save-script; --save-script appears only on the
  transaction opener. Implementation MUST key keep-alive/continuation off
  persisted state. Decision 4 repairSessionHeld updated to match.

Edited: R7, Decision 4 resume signal, terminal-close, Emitting, commit state
machine, tombstone, migration step 9, and validation.
@thymikee

Copy link
Copy Markdown
Member Author

Live CLI evidence — repair transaction end-to-end (iOS iPhone 17 Pro, RN prevent-remove)

Captured on the shipped CLI built from this branch (6beb13f5f), against the drifted prevent-remove fixture (recorded Push Article label renamed → selector-miss).

1. Held divergence + keep-alive (cold and warm). A replay <drifted>.ad --save-script returns REPLAY_DIVERGENCE (kind: selector-miss, screen.state: available) with resume.repairSessionHeld: true. The owning daemon is not torn down — verified both for a warm (open-first) session and a cold --save-script (no prior open): the daemon persists and a subsequent snapshot -i on the same state-dir succeeds (session addressable), where before #1235 a cold divergence returned SESSION_NOT_FOUND.

2. Continuation to completion (terminal-close no longer diverges). Corrective press @<ref> on the real Push Article from screen.refs, then replay --from 5 --plan-digest <sha> (no --save-script repeat — persisted-state continuation) → Replayed 8 steps, success. The source plan's terminal close is skipped while armed instead of diverging — the exact failure this PR fixes.

3. Commit → complete healed artifact. close --save-script=<out> commits atomically. The healed .ad:

  • step 5 healed to the correct label="Push Article" (the drifted "Push Article RENAMED" is gone — it recorded the real press);
  • all 12 steps + a self-contained terminal close;
  • 8 fresh target-v1 annotations (recomputed, not copied);
  • trailing # agent-device:heal-complete sentinel;
  • zero bare @ref.

4. Fresh-session replay (acceptance). Replaying the committed healed .ad in a clean state-dir passes 12/12. (Note: the prevent-remove modal transition at the 2nd "Discard and go back" is timing-flaky — one retry failed at that step, the next passed 12/12; the healed script itself is valid.)

The close response reports the committed path (saveScript field), and the session is torn down after the successful commit.

Not captured live here: REPAIR_SESSION_EXPIRED recovery requires the ~5-min idle-reap to mint a tombstone (covered by the new router/store unit regressions); the competing-writer no-clobber race is likewise unit-covered (linkSync create-exclusive).

@thymikee thymikee force-pushed the feat/repair-transaction-lifecycle branch from 6beb13f to 1a09ed1 Compare July 13, 2026 14:02
@thymikee

Copy link
Copy Markdown
Member Author

Coordinator + independent re-review at 1a09ed1b3: two prior blockers remain, and one additional teardown artifact defect is now explicit. Readiness remains withheld.

  1. The partial-target no-clobber race is still present. After linkSync sees an existing target, two writers can both classify that file as incomplete and then renameSync over each other. The new competing-writer test is sequential complete-vs-complete, so it does not exercise the reported race. Add a regression with both writers starting against the same partial target and make the atomic primitive decide the winner.
  2. COMPLETE repair commit failures during idle-reap/daemon-shutdown are still invisible and non-recoverable. finalizeRepairTeardown ignores { written:false, error }, then daemon teardown deletes the session and exposes only a generic expiry tombstone. Preserve/report the failed commit outcome instead of losing it.
  3. Idle-reap/shutdown auto-commit does not record the skipped terminal source close; explicit close does. The ADR requires the committed healed artifact to be self-contained, but the auto-commit test checks only existence/sentinel. Assert parsed actions and fresh replayability for this path.

The replay --from tombstone translation and [open, terminal close] arming defects are fixed. The supplied shipped-CLI evidence is useful for held divergence, continuation, explicit close commit, and fresh replay, but it intentionally does not cover expiry or the concurrent partial-target case above.

@thymikee thymikee force-pushed the feat/repair-transaction-lifecycle branch 2 times, most recently from 5d4d999 to fb44cf4 Compare July 13, 2026 18:03
@thymikee

Copy link
Copy Markdown
Member Author

Addressed all three blockers from the 1a09ed1b3 re-review (rebased onto current main, 0 behind / 4 ahead; new HEAD fb44cf46c).

1. Partial-target no-clobber race — fixed. publishHealedScriptAtomically (session-script-writer.ts) no longer decides overwritability with a isCompleteHealedScript read followed by an unconditional renameSync. The default healed-sibling path now runs publishNoClobberAtomically, where every winner-deciding step is an atomic primitive:

  • tryExclusiveLink (linkSync) is the only way to win outright — EEXIST iff a file is at the path at that instant.
  • On EEXIST, grabAndDiscardPartialTarget atomically renameSyncs whatever currently sits at the target into a private, uniquely-named quarantine and inspects completeness there — never on the shared path — so no two writers can classify the same partial concurrently. ENOENT (a competitor already grabbed it) just re-evaluates from the top.
  • A COMPLETE grab is restored (best effort) and the publish is refused; a genuine partial is discarded and the exclusive link retried (bounded, 16 attempts).

Every interleaving converges on exactly one winning linkSync; every other writer sees a definitive thrown "already exists". New regression BLOCKER 1: two writers racing on the SAME pre-existing PARTIAL target … (session-script-writer.test.ts) forces writer B's entire publish to run inside writer A's grab via a renameSync spy (genuine interleaving, not timing luck) and asserts exactly one {written:true} / one thrown loss + byte-exact winner. Verified to fail against the pre-fix code.

2. Invisible COMPLETE-commit failures on idle-reap/shutdown — fixed. finalizeRepairTeardown (session-store.ts) now captures writeSessionLog's {written, error} instead of discarding it. A COMPLETE transaction whose commit genuinely fails writes a distinct commit-failure tombstone (RepairSessionTombstone.commitFailure = {code, message}) carrying the real cause, and request-router.ts returns a new REPAIR_COMMIT_FAILED error (real cause + re-run guidance) rather than folding it into the generic REPAIR_SESSION_EXPIRED "reaped before finalized" message. Covered by a session-store test (prior artifact byte-identical + tombstone carries the error) and a request-router-repair-expired test (router surfaces REPAIR_COMMIT_FAILED, not expiry) — both fail without the fix.

3. Auto-commit now records the skipped terminal close. The explicit-close path recorded a synthetic finalize close via commitRepairBeforeClose, but the idle-reap/shutdown auto-commit through finalizeRepairTeardown did not, so its .ad was not self-contained. New recordRepairFinalizeCloseIfCommitting mirrors that recording (same command/positionals/flags/result), firing only when the transaction is actually about to commit (COMPLETE, not yet COMMITTED). The auto-commit test now parses the committed script and asserts ['open','click','close'] with an empty-positional terminal close and zero bare @refs (fresh-replayable), not just existence + sentinel; plus a direct session-store unit test for the same self-contained-artifact guarantee.

Checks: tsc / oxlint --deny-warnings / oxfmt --check (touched files) / fallow --base origin/main all clean; full vitest green except the documented CPU-contention flakies (runtime-hints.test.ts Android adb, runner-client.test.ts concurrent-build) in untouched files, which pass 100% re-run in isolation.

@thymikee

Copy link
Copy Markdown
Member Author

Current-head review at fb44cf46c found a remaining no-clobber race in src/daemon/session-script-writer.ts:300-305. refuseCompleteClobber assumes renameSync(quarantinePath, scriptPath) fails if another writer repopulated scriptPath, but POSIX rename replaces an existing destination. Interleaving: writer A quarantines an existing COMPLETE artifact; writer B links its new COMPLETE artifact into the now-empty target and returns success; writer A then renames the quarantine back over B. B is marked committed even though its published bytes were silently replaced.

Please make the restore/publish protocol mutually exclusive with competing publishers (an atomic lock/claim protocol is one viable shape) and add a deterministic regression where writer B publishes after writer A quarantines a COMPLETE target but before A restores it. Assert no writer reports success unless its complete bytes remain at the target, and that the pre-existing COMPLETE artifact is never silently clobbered. No fixer was dispatched.

@thymikee

Copy link
Copy Markdown
Member Author

Independent exact-head review confirmed the publication race above and found two additional close-path issues:

  1. commitRepairBeforeClose records a successful terminal close and publishes before dispatchTargetedPlatformClose runs. If the platform close then fails, the session is deleted and the healed artifact falsely contains a successful close, contradicting the existing failed-close lifecycle contract. Add a regression with a COMPLETE repair plus a failing targeted/web close; no committed artifact may claim a successful close unless the platform close succeeded.
  2. buildRepairCloseFailureResponse preserves the session specifically for retry but emits details.retriable: false; this must be true (with a contract assertion) or the recovery guidance is machine-contradictory.

Readiness remains withheld. No fixer was dispatched.

thymikee added 7 commits July 13, 2026 21:20
Agent-supervised re-record repair lifecycle, rebased onto #1225's
failure-isolated close teardown. Consolidated from the earlier iterative
rounds into the final teardown-commits model:

- R7 keep-alive keyed off PERSISTED transaction state (repairSessionHeld
  signal), so a `replay --from` continuation without --save-script is still
  held on divergence.
- Commit gated on transaction COMPLETION (saveScriptComplete/saveScriptCommitted),
  never on `close` alone — no prefix is ever published.
- Single commit path: `commitRepairBeforeClose` runs before #1225's
  `runSessionCloseTeardown` destructive steps; a repair-armed session skips the
  teardown's ordinary writeSessionLog, non-repair keeps it. Idle-reap/shutdown
  commit-on-completion or tombstone via `finalizeRepairTeardown`.
- BLOCKER fixes: reaped `replay --from` -> REPAIR_SESSION_EXPIRED; commit
  failures surfaced (not swallowed) and keep the session for retry (with
  healed-path reporting on success); race-safe atomic no-clobber publish;
  minimal `[open, close]` arms the transaction.

Integrated with #1225: keeps runSessionCloseTeardown's failure-isolated
cleanup + preserved platform-close error; repair commit happens first so a
failed commit keeps the session addressable.
…race winner

BLOCKER 1 (coordinator re-review): after linkSync saw an existing target,
the no-clobber publish fell back to an unconditional renameSync once the
target was classified "incomplete" — two concurrent writers could both read
the SAME pre-existing partial as overwritable and both renameSync over it,
each returning success with no signal to the loser. A silent, undetectable
clobber.

publishNoClobberAtomically now makes every winner decision an atomic
primitive:
- linkSync is the only way to win outright (EEXIST iff a file is at the
  target at that instant).
- On EEXIST, the existing file is grabbed via an atomic renameSync into a
  private, uniquely-named quarantine path *before* it is inspected, so the
  completeness check never races the shared path. A competing writer's own
  grab racing ours surfaces as ENOENT, and we re-evaluate from the top
  instead of trusting a stale read.
- A COMPLETE quarantined file is restored (best effort) and the publish is
  refused; a genuinely partial one is discarded and the exclusive linkSync
  is retried.

Every interleaving converges on exactly one winning linkSync and every other
writer observing a definitive, thrown "already exists" — never two silent
successes, never a torn file.

Adds a regression (session-script-writer.test.ts) with both writers starting
against the SAME pre-existing partial target, using a renameSync spy to force
a genuine interleaving (writer B's whole publish runs inside writer A's grab
step) instead of the existing competing-writer test's sequential
complete-vs-complete scenario, which never exercised this race.
…a generic expiry

BLOCKER 2 (coordinator re-review): finalizeRepairTeardown ignored the
writer's { written: false, error } outcome, so a COMPLETE transaction whose
commit failed at idle-reap/daemon-shutdown teardown (no-clobber refusal,
bare-@ref, or a filesystem error) was silently swallowed. Daemon teardown
then deleted the session and left a generic "reaped before it was finalized"
REPAIR_SESSION_EXPIRED tombstone — losing the only record that a commit was
even attempted, let alone why it failed.

finalizeRepairTeardown now captures the writer's result. On a real commit
failure it writes a distinct commit-failure tombstone (RepairSessionTombstone
gains an optional commitFailure: { code, message }); request-router's
repairExpiredIfTombstoned surfaces that as a new REPAIR_COMMIT_FAILED error
carrying the real cause instead of folding it into REPAIR_SESSION_EXPIRED.

Adds a new AppErrorCode REPAIR_COMMIT_FAILED (kernel/errors.ts) with its own
hint, a session-store regression proving finalizeRepairTeardown preserves the
failure (and leaves the prior complete artifact untouched), and a
request-router regression proving the router translates a commit-failure
tombstone to REPAIR_COMMIT_FAILED rather than the generic expiry.
…auto-commit

BLOCKER 3 (coordinator re-review): the source plan's terminal `close` is
skipped-while-armed (Fix 3), so it never lands in session.actions. The
explicit `close --save-script` path accounts for this by recording a
synthetic finalize close (commitRepairBeforeClose) before committing, but
finalizeRepairTeardown's auto-commit at idle-reap/daemon-shutdown never runs
that handler — its committed healed .ad was missing its own terminal close,
so the ADR's "self-contained, fresh-replayable artifact" requirement didn't
hold for this path even though the existing auto-commit test only checked
existence + the completeness sentinel.

finalizeRepairTeardown now calls a new recordRepairFinalizeCloseIfCommitting
before writeSessionLog, mirroring commitRepairBeforeClose's recording exactly
(same command/positionals/flags shape), but only when the transaction is
actually about to be committed (COMPLETE, not yet COMMITTED) — an aborted
transaction's write is a no-op regardless.

Strengthens the existing auto-commit test (session-replay-repair-transaction
.test.ts) to use a source plan with a real terminal close, and to parse the
committed script and assert it ends with ['open', 'click', 'close'] with no
bare @ref — not just sentinel/existence. Adds a session-store.ts unit
regression exercising finalizeRepairTeardown directly for the same
self-contained-artifact assertion.
…usive lock

BLOCKER 1 (review follow-up): publishNoClobberAtomically's inspect/restore/
publish sequence was atomic per-step but not exclusive as a whole. Writer A
could quarantine an existing COMPLETE target, and — before A restored it —
writer B could linkSync its own COMPLETE artifact into the now-empty target
and return success, only for A's restore (renameSync, which replaces an
existing destination per POSIX) to silently stomp B's freshly published
bytes.

Wrap the whole decide-and-act sequence in an exclusive publish lock
(acquireNoClobberLock/releaseNoClobberLock, an atomic linkSync claim over a
PID-stamped lock file) so a competing writer for the same scriptPath cannot
begin its own decision until the lock holder's sequence has finished and
released it. A lock whose PID is provably dead is reclaimed immediately; a
lock held by a live process is never stolen, only waited on with a bounded
backoff before failing loudly.

Adds a deterministic regression that forces the exact reported interleaving
via a renameSync spy (mirroring the existing BLOCKER 1 test's technique) and
confirms the pre-existing COMPLETE artifact is never clobbered. Also relaxes
the older PARTIAL-race test's loser-message assertion, since a losing writer
may now fail via lock contention instead of the no-clobber-specific message,
depending on interleaving timing.
BLOCKER 3: buildRepairCloseFailureResponse preserves the session specifically
so the agent can retry close/close --save-script, but reported
details.retriable: false — machine-consistent recovery guidance requires
retriable: true whenever the session was kept addressable for a retry.

Extends the existing BLOCKER 2b/2c no-clobber-failure test with an assertion
on this contract.
BLOCKER 2: commitRepairBeforeClose recorded a successful terminal `close`
and published the healed artifact BEFORE dispatchTargetedPlatformClose ran.
If the platform close then failed, the session was torn down and the
committed .ad falsely contained a successful close — contradicting the
existing failed-close lifecycle contract (a failed close is never recorded
as Closed).

For a repair-armed session, dispatch the targeted platform close first; only
on success does the commit (record + publish) proceed. On failure, return
without touching the session at all — same as the existing commit-failure
contract, so the agent can fix the cause and retry. runSessionCloseTeardown
gains a skipPlatformClose flag so the already-confirmed-successful close is
never dispatched a second time during teardown.

Adds a regression: a COMPLETE repair whose targeted platform close rejects
must not commit a healed .ad, must not record a close action, and must keep
the session addressable for retry; a subsequent successful retry then
commits cleanly with exactly one terminal close.
@thymikee thymikee force-pushed the feat/repair-transaction-lifecycle branch from fb44cf4 to 65d6679 Compare July 13, 2026 19:49
@thymikee

Copy link
Copy Markdown
Member Author

Addressed the fb44cf46c re-reviews (rebased onto current main, 0 behind / 7 ahead; new HEAD 65d6679ac).

1. No-clobber restore race — fixed with an exclusive lock. The prior grab/restore made each step atomic but not the sequence, so A could quarantine a COMPLETE target, B win the emptied slot, and A's renameSync restore silently stomp B (POSIX rename replaces the destination). The decide-and-act sequence now runs under an exclusive linkSync claim on <scriptPath>.lock (acquireNoClobberLock/releaseNoClobberLock): a holder keeps the lock through grab → inspect → restore/discard, so a competitor's linkSync gets EEXIST and cannot begin its own decision until the holder releases. A lock whose recorded PID is provably dead (isProcessAlive) is reclaimed; a live holder is never stolen — the waiter backs off (bounded) then fails loudly. New deterministic regression forces B's publish after A quarantines a COMPLETE target but before A restores (via a renameSync spy) and asserts no writer reports success unless its bytes remain + the pre-existing COMPLETE artifact is never clobbered — fails against the old code, passes now.

2. Successful close no longer committed before the platform close runs. For a repair-armed session, handleCloseCommand now dispatches dispatchTargetedPlatformClose first; only on success does it commitRepairBeforeClose (runSessionCloseTeardown gained skipPlatformClose so it isn't dispatched twice). A failing platform close returns buildRepairCloseFailureResponse and touches nothing — no committed .ad can claim a successful close unless the close succeeded. Regression: COMPLETE repair + rejected platform close asserts no artifact, no recorded close, session addressable; then a clean retry commits with exactly one terminal close.

3. retriable: true. buildRepairCloseFailureResponse (reached only when the session is deliberately preserved for retry) now reports retriable: true, with a contract assertion.

Checks: tsc / oxlint / oxfmt / fallow clean; full vitest green (only the documented Android runtime-hints / runner-client contention flakies, pass in isolation). I traced the lock interleaving and confirmed the close-ordering + double-dispatch guard myself. CI green.

@thymikee

Copy link
Copy Markdown
Member Author

Blocking findings on 65d6679a:

  1. Stale-lock reclamation is still racy. Two waiters can both classify the old lock PID as dead; waiter B removes it and acquires a new live lock, then waiter A executes its stale rmSync(lockPath) and deletes B's lock before acquiring its own. Both can enter the supposedly exclusive publish/restore section. Avoid stealing by pathname after a separate read, or use an ownership/CAS protocol and add the two-reclaimer interleaving regression.
  2. buildRepairCloseFailureResponse places retriable under error.details, but the public contract and router/client path use top-level error.retriable. It also discards platform details, diagnosticId, and logPath. Preserve the normalized error contract and test through the router/client boundary.
  3. When targeted platform close succeeds and the subsequent script commit fails, the session is retained for retry but no state records that close already succeeded. Retrying close --save-script=<other> dispatches the platform close again; a non-idempotent backend can now fail and make the commit unrecoverable. Persist/consume the successful-close phase or otherwise make the retry skip that completed operation.

CI is green, but these keep the PR from readiness.

thymikee added 3 commits July 14, 2026 08:36
…me-CAS

Two waiters could both observe the same dead-PID publish lock and both
decide to reclaim it. If one waiter's reclaim (remove + re-acquire with
its own LIVE lock) completed inside the other's decision window, the
first waiter's stale rmSync(lockPath) deleted the SECOND waiter's live
lock by pathname (not the dead one it actually inspected), letting both
enter the exclusive publish section at once.

Reclaim is now a rename-based compare-and-swap: renameSync(lockPath,
uniquePath) is the atomic claim (only one racer's rename of a given
source ever succeeds; the loser gets ENOENT and retries). Only the
winner inspects what it actually grabbed at the private claim path — if
genuinely dead, discard it; if the claim raced with someone else's fresh
reclaim and grabbed their live lock instead, restore it untouched and
back off. The live holder's lock is never stolen.

Regression drives the exact two-reclaimer interleaving deterministically
via a readFileSync spy (writer B reclaims+re-acquires live, inside
writer A's reclaim window) and confirms it fails against the prior
rmSync-based reclaim.
…he wire top level

buildRepairCloseFailureResponse hand-rolled its response shape instead
of going through normalizeError, so it put retriable under
error.details.retriable — a location neither the router's
enrichDaemonError nor the client reads (both read the top-level
DaemonError.retriable) — and silently dropped the underlying platform/
commit error's details, diagnosticId, and logPath entirely.

Now routes through normalizeError like every other AppError ->
DaemonResponse conversion in this codebase, preserving the underlying
error's details/diagnosticId/logPath, with retriable forced true at the
top level (the session is retained specifically for retry, which must
never be contradicted by the underlying error's own classification).

Also fixes a companion gap in finalizeDaemonResponse: it rebuilds every
handler-RETURNED (non-thrown) failure response into a fresh AppError
before re-normalizing, but only carried hint/diagnosticId/logPath
through that reconstruction, not retriable/supportedOn — so even a
handler setting them correctly at the top level still lost them at this
step. Both are now carried through the same way, discovered only by
verifying the close fix through the actual router boundary as
requested.

Regression: an updated handler-level test confirms diagnosticId/
logPath/details survive a repair-close failure, and a new router-level
test (through createRequestHandler, not just the raw builder) confirms
retriable:true and the platform error's diagnosticId/logPath/details
all survive to the client. Both fail against the pre-fix code.
…form close

When a repair-armed close's targeted platform close SUCCEEDED but the
subsequent script commit FAILED (no-clobber refusal, a bare-@ref
failure, or an fs error), the session was correctly retained for retry
-- but nothing recorded that the platform close had already happened.
A retry (close --save-script=<other>) dispatched
dispatchTargetedPlatformClose again, so a non-idempotent backend could
fail or wedge recovery on a second close of an already-closed target.

SessionState now carries repairPlatformCloseSucceeded, set the moment
the platform close returns success. A subsequent repair close consumes
it and skips straight to the commit instead of re-dispatching; it is
cleared once the transaction's outcome (commit or abort) is settled, so
it never lingers past a single close attempt.

Regression: platform close succeeds, commit fails (no-clobber),
session is retained; a retry does not re-invoke
dispatchTargetedPlatformClose (asserted via call count) and still
commits cleanly to the retry path. Fails against the pre-fix code
(dispatch called twice).
@thymikee

Copy link
Copy Markdown
Member Author

Blocking finding on b1e039dc4:

The revised stale-lock reclaim still breaks exclusion under a three-writer interleaving. In reclaimDeadLock, waiter A can read an old dead PID, waiter B can then reclaim it and acquire a live lock, and A can rename B's live lock from lockPath to claimPath. A eventually detects the live PID and restores it, but while the canonical lockPath is absent, waiter C can successfully linkSync its own lock and enter the publish critical section concurrently with B. A's restoring renameSync can then replace C's lock, and the unconditional pathname-based release can remove a successor's lock as well.

The new regression covers preservation of B's live lock with only A and B; it does not introduce C during the rename/restore gap, so it misses the remaining race. The ownership primitive must never make a live holder's canonical claim disappear, and release must remove only the caller's own claim. The error-contract and already-succeeded platform-close findings are otherwise addressed.

Readiness remains withheld. No fixer was dispatched.

@thymikee

Copy link
Copy Markdown
Member Author

Additional independent-review blockers on b1e039dc4:

  1. A successful owned one-shot replay --save-script can still lose a teardown commit failure. The replay marks COMPLETE and returns success before publication; daemon shutdown later calls finalizeRepairTeardown, which can only persist a REPAIR_COMMIT_FAILED tombstone. The client cleanup path then removes an owned ephemeral state directory regardless, so the user can receive success while the commit failure and its recovery evidence are deleted. Add a client/daemon boundary regression that forces a shutdown-time commit failure and surfaces it before cleanup.
  2. repairPlatformCloseSucceeded is session-wide, not bound to the close request that succeeded. An untargeted close can perform no platform operation, still set the marker, then fail commit; a retry with a target or changed flags skips the required platform close and commits the new close request as if it ran. Persist the completed operation's identity or reject changed retries, and cover untargeted-to-targeted plus changed-target retries.
  3. ADR 0012 describes race-safe no-clobber for the final target and says only incomplete/partial files are overwritable. The implementation bypasses that guard for every explicit --save-script=<path> and the test asserts replacement even when that explicit target is a complete sentinel-marked healed artifact. Either enforce complete-artifact protection for explicit targets or explicitly amend the ADR/contract to permit this overwrite.

These are in addition to the live-lock rename gap in the preceding comment. No fixer was dispatched.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant