SamPlvs · SamPlvs · Aug 17, 2026 · Aug 17, 2026 · Aug 17, 2026
@@ -27,5 +27,8 @@ PY="python3"
 
 # Pre-set ZO_REPO_ROOT wins (lets tests point the handlers at a sandbox).
 export ZO_REPO_ROOT="${ZO_REPO_ROOT:-$REPO_ROOT}"
+# Wall-clock stamp of the hook event (WS-C heartbeat; cheap, informational —
+# the heartbeat handler stamps its own UTC time and does not require this).
+export ZO_HOOK_EVENT_TS="$(date -u +%s)"
 PYTHONPATH="$REPO_ROOT/src${PYTHONPATH:+:$PYTHONPATH}" "$PY" -m zo.hookkit "$EVENT" || exit 0
 exit 0
@@ -81,6 +81,16 @@
             "timeout": 5
           }
         ]
+      },
+      {
+        "matcher": "*",
+        "hooks": [
+          {
+            "type": "command",
+            "command": "bash .claude/hooks/zo-hookkit.sh heartbeat 2>/dev/null || exit 0",
+            "timeout": 5
+          }
+        ]
       }
     ],
     "Stop": [

@@ -32,6 +32,12 @@ logs/
 # Per-project memory (keep ZO platform memory, ignore everything else)
 memory/*
 !memory/zo-platform/
+# Runtime control-plane state under the platform memory root is never
+# tracked (WS-B ledger/contracts, WS-C heartbeats). These MUST come after
+# the !memory/zo-platform/ re-include above.
+memory/zo-platform/heartbeats/
+memory/zo-platform/plan-ledger.json
+memory/zo-platform/contracts.json
 
 # Design source files contain project-specific examples (client-identifying)
 docs/source-design/

@@ -10,7 +10,7 @@
 <br/>
 
 [![Status](https://img.shields.io/badge/status-validated-D87A57?style=flat-square&labelColor=12110F)](#status)
-[![Tests](https://img.shields.io/badge/tests-854_passing-D87A57?style=flat-square&labelColor=12110F)](#status)
+[![Tests](https://img.shields.io/badge/tests-1053_passing-D87A57?style=flat-square&labelColor=12110F)](#status)
 [![Agents](https://img.shields.io/badge/agents-21_defined-D87A57?style=flat-square&labelColor=12110F)](#agent-teams)
 [![Docs](https://img.shields.io/badge/docs-zerooperators.com-D87A57?style=flat-square&labelColor=12110F)](https://docs.zerooperators.com)
 
@@ -526,7 +526,7 @@ delivery-repo/
 | 1.0.2 | Platform-aware Docker scaffold + reference-project end-to-end demos | Done |
 | 1.0.2-post | `--low-token` cost-saving preset (two-tier model routing, per-phase trims) + `ZOTrainingCallback` hard gate enforcement | Done |
 
-780 platform tests. ruff clean (`src/zo/`). 21 agents. 24 slash commands. Measured benchmarks tracked in [docs/reference/cost-benchmark.mdx](docs/reference/cost-benchmark.mdx).
+1053 platform tests. ruff clean (`src/zo/`). 21 agents. 24 slash commands. Measured benchmarks tracked in [docs/reference/cost-benchmark.mdx](docs/reference/cost-benchmark.mdx).
 
 ---
 

@@ -22,7 +22,7 @@ Launch an agent team to execute a plan. Parses the plan, shows a phase review, a
 zo build plans/project.md [--gate-mode supervised|auto|full-auto] [--no-tmux]
                           [--bypass-permissions] [--low-token]
                           [--lead-model opus|sonnet|haiku] [--max-iterations N]
-                          [--no-headlines]
+                          [--no-headlines] [--no-watchdog]
 ```
 
 **Cost-saving options:**
@@ -34,6 +34,9 @@ zo build plans/project.md [--gate-mode supervised|auto|full-auto] [--no-tmux]
 **Permission prompts:**
 - `--bypass-permissions`: auto-approve **every** Claude Code tool-call prompt (Bash, Edit, Write, Read, ...) for the run. Independent of `--gate-mode` (which gates ZO's phases, not individual tool calls); implied by `--gate-mode full-auto`. Off by default. Works in tmux (temporary `.claude/settings.local.json` overlay, auto-restored on exit) and headless (`--dangerously-skip-permissions`) modes. See `docs/cli/build.mdx`.
 
+**Anti-stall watchdog:**
+- `--no-watchdog`: disable the anti-stall watchdog for this run (no stall detection, nudges, or rate-limit pause). Equivalent to `ZO_WATCHDOG=0`. On by default; policy comes from the `watchdog:` block in `.zo/config.yaml` (see `specs/watchdog.md`).
+
 ### zo continue
 
 Resume a paused project. Shorthand for `zo build` with an existing plan -- finds the plan by project name and picks up from the current phase.
@@ -42,13 +45,13 @@ Resume a paused project. Shorthand for `zo build` with an existing plan -- finds
 zo continue [project-name] [--repo PATH] [--gate-mode supervised|auto|full-auto]
                            [--bypass-permissions] [--low-token]
                            [--lead-model opus|sonnet|haiku] [--max-iterations N]
-                           [--no-headlines]
+                           [--no-headlines] [--no-watchdog]
 ```
 
 **Options:**
 - `project-name`: optional if cwd contains `.zo/config.yaml` (auto-detected)
 - `--repo PATH`: path to delivery repo (overrides target file lookup)
-- `--low-token`, `--lead-model`, `--max-iterations`, `--no-headlines`, `--bypass-permissions`: same semantics as `zo build`
+- `--low-token`, `--lead-model`, `--max-iterations`, `--no-headlines`, `--bypass-permissions`, `--no-watchdog`: same semantics as `zo build`
 
 ### zo draft
 

@@ -137,6 +137,7 @@ While `zo build` is running, you have several windows into what the team is doin
 | `--max-iterations N` | `10` (or `2` if `--low-token`) | Hard cap on Phase-4 experiment iterations. Wins over plan and preset. |
 | `--no-headlines` | (off) | Skip the end-of-session Haiku bullet summary (~1 Haiku call per run, ~$0.0002). |
 | `--bypass-permissions` | (off) | Auto-approve Claude Code tool-call prompts. Implied by `--gate-mode full-auto`. See [Permission prompts](#permission-prompts-bypass-permissions). |
+| `--no-watchdog` | (off) | Disable the anti-stall watchdog for this run (no stall detection, nudges, or rate-limit pause). Equivalent to `ZO_WATCHDOG=0`. Policy otherwise comes from the `watchdog:` block in `.zo/config.yaml`. |
 
 ## Examples
 

@@ -65,6 +65,17 @@ architecture — features from different repos interlock into single mechanisms.
 | 2 | **Watchdog from proven parts** — heartbeat-file liveness, a taxonomy of stops that must never be fought (context-limit, rate-limit, auth, user abort), bounded nudge budgets, and rate-limit wait-and-resume for overnight runs | oh-my-claudecode + ruflo | P0 |
 | 6 | **Fresh-context-per-subtask execution loop** — a new agent per iteration re-derives state from the ledger, experiment lineage, and a curated priors digest; git commits are the checkpoints; context rot and compounding hallucination are eliminated structurally | ralph + oh-my-claudecode | P1 |
 
+**Status (Phase 3, in progress):** the watchdog has shipped — an external
+checker in the `LifecycleWrapper` poll loop (not an LLM monitor, not a cron)
+that reads hook-written heartbeats, classifies never-fight stops before any
+nudge, nudges the lead pane at most three times through a pane-ready guard,
+pauses on the usage-limit banner until the parsed reset time and verifies the
+resume by real progress, then escalates. It is on by default (`watchdog:` block
+in `.zo/config.yaml`, `zo build --no-watchdog` or `ZO_WATCHDOG=0` to disable);
+design and integration points live in `specs/watchdog.md`. The fresh-context
+loop (feature 6) is next and consumes the watchdog's `STALLED` /
+`RATE_LIMITED` outcomes.
+
 ### Workstream D — Self-learning & platform oracle
 
 *The self-evolution loop gets the same rigor ZO applies to ML work.*

@@ -1277,3 +1277,37 @@ The `--no-headlines` flag is preserved (not removed) for backwards compatibility
 **Scope deferrals (deliberate, to Phase 3):** `evaluate_loop_state` keeps its ExperimentRegistry input (already oracle-derived; signature change touches ~20 test call sites and belongs to the fresh-context loop rework); session-state restore still reads STATE.md (cutover rides the fresh-context substrate to avoid regressing the PR-036 GATED-precedence resume fix).
 
 **Outcome:** 908 → 929 passed / 7 skipped (+21: 13 ledger, 5 stories/lint, 2 nonce-flip, 1 status), ruff clean, validate-docs green. Oracle checks 8-10 have passing seeded tests: status renders from the ledger; builder ledger-write denied while the oracle flip lands; vague story rejected. Cascade: specs/plan.md (§10 Stories + lint), specs/workflow.md (state-tracking line), docs/COMMANDS.md (status control plane). Branch `claude/v2-phase2-control-plane`.
+
+## Decision: 2026-08-17T09:00:00Z
+**Type:** SCOPE + ARCHITECTURE
+**Title:** v2 Phase 3 sequencing and four build-shaping decisions (Sam), after recon found the oracle-owned gate path has no runtime caller
+
+**Decision:** Phase 2 (PR #108) merged to main at session start; Phase 3 (WS-C) built on `claude/v2-phase3-substrate` off `1faf53a`. A read-only recon swarm (7 mappers + synthesis, `memory/zo-platform/research/2026-08-17-phase3-recon/`) established, with `file:line` evidence, that `Orchestrator.advance_phase()` and `mark_subtask_complete()` have zero runtime callers: `zo build` launches one lead session per phase and calls `end_session()`; the automated gate, `_auto_iterate_if_needed`, the WS-B `mark_phase_passed` flip and the ONLY gate-nonce mint (`orchestrator.py:788`) are unreachable in production. Phases have only ever advanced through hand-edited STATE.md (the PR-036/037 prod-001 incident). Sam decided: (1) **the driver evaluates gates for ALL phases** — after any session exit the Python side calls `advance_phase()` mechanically (COMPLETED → next phase; GATED → mint nonce, stop; ITERATE → relaunch fresh); only Phase 4 gets fresh headless spawns per iteration until check 13 says extend; (2) **two PRs**: A = watchdog (checks 11–12, buildable here), B = driver + fresh-context loop + the two Phase-2 deferrals; (3) **restore cutover: ledger wins over STATE.md** with a loud warning, plus a sanctioned `zo phase set <phase> <status>` override logged to DECISION_LOG (replaces the hand-edit lever); (4) **tmux nudges default ON** behind a pane-ready / no-permission-dialog guard.
+
+**Rationale:** The fresh-context loop's driver is not a Phase-4 cost feature — it is the missing runtime caller that makes gates, nonces, ledger flips and `zo gates approve` real (PR-009's own rule, "built ≠ wired", applied to `advance_phase`). Two PRs de-risk the substrate change and let the watchdog merge first. Ledger-wins is the plan's dual-plane doctrine (control plane decides, STATE.md is the projection). Nudges on: the 38-hour stall was an idle lead; a nudge is the resume mechanism after a rate-limit reset too — the guard (idle prompt visible, no `awaiting_input` dialog) prevents Enter from approving a permission prompt.
+
+**Alternatives considered:** Phase-4-only driver (rejected: leaves five phases on self-asserted completion); one PR (rejected: substrate risk); STATE.md wins / refuse-to-start (rejected: STATE.md is hand-editable and clobbered by `end_session`); nudges log-only in tmux (rejected: no auto-resume after rate-limit reset).
+
+**Outcome:** PR-A shipped this session (next entry); PR-B queued with the exact seams in the integration map §2–3.
+
+## Decision: 2026-08-17T12:30:00Z
+**Type:** FEATURE + ARCHITECTURE
+**Title:** v2 Phase 3 PR-A — watchdog from proven parts (heartbeats + never-block taxonomy + bounded nudges + rate-limit wait-and-resume + PID/start-time identity)
+
+**Decision:** Implemented WS-C's watchdog per plan Phase 3 and the build contract (`memory/zo-platform/research/2026-08-17-phase3-recon/pr-a-build-contract.md`): pure policy module `zo.watchdog` (+ `_watchdog_models/_watchdog_text/_proc`), stdlib-only heartbeat writer `zo._hook_heartbeat` on a `PostToolUse *` hook (+ Stop/PreCompact/SubagentStop/SessionEnd stamps), `WatchdogRunner` ticked from BOTH wrapper loops before the liveness reads, config in `ProjectConfig.watchdog` + `--no-watchdog` + `ZO_WATCHDOG=0`. `specs/watchdog.md` rewritten: the RFC's cron-scheduled orchestrator-owned tick and respawn/reroute ladder are superseded by the plan's external checker in the wrapper poll loop (a non-LLM process cannot itself stall); respawn moves to PR-B's driver.
+
+**Design choices worth the record:** (a) three-state freshness — `unknown` (no/unreadable heartbeat, unknown pid identity, EPERM) is never a stall verdict; (b) taxonomy runs before any nudge in both loops, ported from OMC (MIT) with ZO changes: no bare `429`/`overloaded` (0.4291, step 4290, "GPU overloaded"), tiered rate-limit patterns (banner / prose / loose-only-with-vocabulary), `awaiting_input` added (menu cursor `❯ 1.` required — plain numbered lists must not disable nudging), bare "interrupt" excluded but `⎿ Interrupted by user` included; (c) evidence = heartbeat tick deltas (pre-existing files baselined) + normalized text digest (spinner/counters stripped, first observation ≠ progress) + progress-path mtimes + **process-tree CPU time** (added by the fix round so a silent 40-minute training call inside one Bash tool is not a false stall/kill; a CPU-spinning hung process is bounded by `--timeout`); (d) rate-limit = paused state evaluated per poll, never a blocking sleep; reset time parsed in the operator's local tz; a static banner (the real TUI never clears the line) gets a bounded resume nudge after `paused_until`; resume is verified by heartbeat/file progress, never asserted; timeout excludes paused time, capped at an in-pause escalation; (e) escalation once per stall; tmux never kills a human-facing pane (logs blocking + `STALLED` at exit), headless kills by default (`kill_headless_on_escalate`) because it has no other lever until the driver; (f) the headless retry-with-backoff loop is REMOVED — exit while rate-limited → `RATE_LIMITED` + `resume_at` for the PR-B driver; (g) heartbeats live under the per-project memory root, are sealed against agent Write/Edit, and gitignored on the platform root and in the delivery `.zo/` templates (they were tracked before — recon caught it).
+
+**Correction of the 2026-08-12T15:30 entry:** `PostToolUseFailure` DOES fire on nonzero-exit Bash — `logs/comms/failures-2026-08-17.jsonl` captured this session's own `git checkout` exit-1. The "infrastructure errors only; add PostToolUse error inspection in WS-D" caveat is withdrawn.
+
+**Live evidence:** the new hook fired in this session for the lead and for workflow subagents (`agent_type=workflow-subagent`, `tick_count` 44 → `shutdown`) — PostToolUse carries `agent_id`/`agent_type` for subagents.
+
+**Verification method:** contract-first build (4 concurrent builders on disjoint files) → integrator → 3 adversarial verifier lenses (semantics / wiring+sealing / test quality: 19 findings, 2 high — banner reset times parsed in UTC; static banner could never resume) → fixer (11 applied with regression tests, 4 rejected with reasons). 929 → 1131 passed / 7 skipped, ruff clean, validate-docs 0 failures. Seeded tests for checks 11 and 12 on both loops.
+
+**Follow-ups (not done, recorded):** `_watchdog-ticks.jsonl` unbounded growth; `wrapper.py` 1404 lines (split `_wrapper_tmux.py`); verify the CPU-evidence idle threshold on a real tmux session; sealed-prefix symlink resolution in hookkit; `is_interrupt` from the failure feed not yet fed to `evaluate()`.
+
+## Decision: 2026-08-17T13:05:00Z
+**Type:** FAILURE + FIX
+**Title:** PR #109 CI red on 3.11/3.12 (green locally on 3.14) — CPU-evidence probe spawned `ps -A` under a global `time.sleep` mock
+
+**Failure:** `test_running_process_with_rate_limit_text_pauses_without_backoff` saw CPython's `Popen.wait` doubling back-off sleeps in `mock_sleep.call_args_list` because the watchdog's CPU probe ran the real `ps -A` (`subprocess.run(timeout=5)`) and `mock.patch("zo.wrapper.time.sleep")` patches the global `time` module. Race-dependent reap timing → Linux red, macOS green. **Root cause:** `missing_rule` — no rule required neutralising new process-spawning helpers in unit tests. **Fix:** autouse fixture `_no_real_cpu_probe` in `tests/unit/test_wrapper.py` (probe → `None`); proven with a counting `Popen` spy (2 spawns before, 0 after). **Prior:** PR-049 (five rules incl. "raising traps are swallowed by fail-open code — use counting spies" and "default-arg binding defeats late patching"). Suite 1131 / 7 skipped, ruff clean; pushed to #109.