feat(cloud-agent-next): add durable control-plane diagnostics

[eshurakov]

Summary

• Add verbose, sanitized Worker/DO and wrapper diagnostics for heartbeats, event forwarding, task phases, deadlines, provider operations, and durable message transitions so stalled sessions can be investigated beyond the final failure reason.
• Archive bounded structured wrapper batches in the existing R2 bucket using allocation-scoped upload-only grants. Add internal authenticated list/download routes that remain usable after the sandbox disappears, and document the operator workflow.
• Keep diagnostics best-effort: exclude credentials and message/tool content, bound buffering and shutdown flushing, stop uploads after authentication rejection, and limit optional signing-secret lookup to one second. Preserve existing session lifecycle and allocation fencing.
• Trace shared-worktree chat admission/reconciliation, workspace reuse versus session restoration, ownership evidence, and staged deletion/fencing. Keep worktree and chat identities distinct, and distinguish runtime resource cleanup from complete worktree deletion.
• Trace accepted-runtime reconciliation before generic failure/cleanup: sync stage, timeout, runtime identity, scoped interaction counts, and superseded results. Split wrapper status/question/permission query outcomes and preserve pending-query evidence at the shared abort, without changing recovery policy or parallelism.

Verification

No manual/live-provider end-to-end validation or deployment was performed. Live-provider verification remains outstanding; automated coverage is listed below.

Visual Changes

N/A

Reviewer Notes

Based on main at 9398fade1, with three focused commits covering core diagnostics, worktree lifecycle, and accepted-runtime reconciliation. The logging follow-ups use existing behavioral tests; no diagnostic-only tests were added.

Automated checks passed on the final changes:

• Repository formatting check and git diff --check.
• Cloud Agent lint, Worker/wrapper typecheck, and production wrapper build.
• Reconciliation/service regressions: pnpm exec vitest run src/sandbox-session src/sandbox-control/log-routes.test.ts — 331 passed.
• Wrapper regressions, run separately: bun test src/control — 373 passed; bun test src/kilo-api.test.ts — 82 passed.
• Workers-runtime checks, run serially — 1 root-scoped reconnect/sync test and 20 worktree-deletion tests passed.

An initial combined wrapper run hit the known 500ms child-startup fixture flake; the fixture and both suites passed separately. A parallel filtered Workers run hit a console-reporter teardown error; serial reconnect/full-deletion suites passed. No test workaround was added.

A complete Workers control-suite run remains unverified: earlier runs timed out during teardown after printing test passes. The focused runtime suites above pass, but this is not full integration or live-provider coverage. No runner workaround was added.

Verbose metadata logging is enabled by default. Upload grants expire after four hours without renewal. R2 retention remains governed by the existing external bucket policy; this PR does not change retention or add historical-data processing.

[kilo-code-bot]

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

Incremental review of the Workers test-pool pin (@cloudflare/vitest-pool-workers 0.18.6) and lockfile update found no high-confidence bugs.

Files Reviewed (2 files)

• pnpm-lock.yaml
• services/cloud-agent-next/package.json

Previous Review Summaries (5 snapshots, latest commit 129e53c)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 129e53c)

Status: No Issues Found | Recommendation: Merge

Executive Summary

Incremental review of archive-read endpoint removal in five files found no high-confidence bugs; upload-only log routes and R2-direct retrieval docs remain consistent.

Files Reviewed (5 files)

• services/cloud-agent-next/DEBUG.md
• services/cloud-agent-next/src/sandbox-control/log-routes.test.ts
• services/cloud-agent-next/src/sandbox-control/log-routes.ts
• services/cloud-agent-next/src/server.test.ts
• services/cloud-agent-next/src/server.ts

Previous review (commit 2ed6865)

Status: No Issues Found | Recommendation: Merge

Executive Summary

Incremental review of diagnostic-volume reduction (heartbeat peak counters and suppressed successful message.part.delta Worker logs) in four files found no high-confidence bugs; failure and lifecycle logging remain verbose.

Files Reviewed (4 files)

• services/cloud-agent-next/DEBUG.md
• services/cloud-agent-next/src/persistence/SandboxControl.ts
• services/cloud-agent-next/src/sandbox-control/diagnostics.ts
• services/cloud-agent-next/src/sandbox-control/socket.ts

Previous review (commit 97f0e1a)

Status: No Issues Found | Recommendation: Merge

Executive Summary

Incremental review of accepted-runtime reconciliation diagnostics in four files found no high-confidence bugs; existing sync/fail-delivery behavior is preserved.

Files Reviewed (4 files)

• services/cloud-agent-next/DEBUG.md
• services/cloud-agent-next/src/sandbox-session/SandboxSession.ts
• services/cloud-agent-next/src/shared/control-diagnostics.ts
• services/cloud-agent-next/wrapper/src/control/sandbox-control-handlers.ts

Previous review (commit 3e42b05)

Status: No Issues Found | Recommendation: Merge

Files Reviewed (32 files)

• services/cloud-agent-next/DEBUG.md
• services/cloud-agent-next/src/persistence/SandboxControl.ts
• services/cloud-agent-next/src/router/handlers/session-worktree.ts
• services/cloud-agent-next/src/router/handlers/worktree-deletion.ts
• services/cloud-agent-next/src/sandbox-control/cloudflare-provider.ts
• services/cloud-agent-next/src/sandbox-control/diagnostics.ts
• services/cloud-agent-next/src/sandbox-control/lifecycle.test.ts
• services/cloud-agent-next/src/sandbox-control/log-routes.test.ts
• services/cloud-agent-next/src/sandbox-control/log-routes.ts
• services/cloud-agent-next/src/sandbox-control/log-upload-grant.test.ts
• services/cloud-agent-next/src/sandbox-control/log-upload-grant.ts
• services/cloud-agent-next/src/sandbox-control/socket.ts
• services/cloud-agent-next/src/sandbox-control/vercel-provider.ts
• services/cloud-agent-next/src/sandbox-control/worktree-deletion.ts
• services/cloud-agent-next/src/sandbox-control/worktree-ownership.ts
• services/cloud-agent-next/src/sandbox-control/wrapper-launch-env.test.ts
• services/cloud-agent-next/src/sandbox-control/wrapper-launch-env.ts
• services/cloud-agent-next/src/sandbox-session/SandboxSession.ts
• services/cloud-agent-next/src/server.test.ts
• services/cloud-agent-next/src/server.ts
• services/cloud-agent-next/src/shared/control-diagnostics.ts
• services/cloud-agent-next/src/shared/runtime-environment.ts
• services/cloud-agent-next/wrapper/src/control/apply-attach.ts
• services/cloud-agent-next/wrapper/src/control/delete-worktree.ts
• services/cloud-agent-next/wrapper/src/control/diagnostics.test.ts
• services/cloud-agent-next/wrapper/src/control/diagnostics.ts
• services/cloud-agent-next/wrapper/src/control/main.ts
• services/cloud-agent-next/wrapper/src/control/sandbox-control-client.ts
• services/cloud-agent-next/wrapper/src/control/sandbox-control-handlers.test.ts
• services/cloud-agent-next/wrapper/src/control/sandbox-control-handlers.ts
• services/cloud-agent-next/wrapper/src/control/sandbox-control-runtime.ts
• services/cloud-agent-next/wrapper/src/control/worktree-runtime.ts

Previous review (commit 60ee4a3)

Status: No Issues Found | Recommendation: Merge

Files Reviewed (25 files)

• services/cloud-agent-next/DEBUG.md
• services/cloud-agent-next/src/persistence/SandboxControl.ts
• services/cloud-agent-next/src/sandbox-control/cloudflare-provider.ts
• services/cloud-agent-next/src/sandbox-control/diagnostics.ts
• services/cloud-agent-next/src/sandbox-control/lifecycle.test.ts
• services/cloud-agent-next/src/sandbox-control/log-routes.test.ts
• services/cloud-agent-next/src/sandbox-control/log-routes.ts
• services/cloud-agent-next/src/sandbox-control/log-upload-grant.test.ts
• services/cloud-agent-next/src/sandbox-control/log-upload-grant.ts
• services/cloud-agent-next/src/sandbox-control/socket.ts
• services/cloud-agent-next/src/sandbox-control/vercel-provider.ts
• services/cloud-agent-next/src/sandbox-control/wrapper-launch-env.test.ts
• services/cloud-agent-next/src/sandbox-control/wrapper-launch-env.ts
• services/cloud-agent-next/src/sandbox-session/SandboxSession.ts
• services/cloud-agent-next/src/server.test.ts
• services/cloud-agent-next/src/server.ts
• services/cloud-agent-next/src/shared/control-diagnostics.ts
• services/cloud-agent-next/src/shared/runtime-environment.ts
• services/cloud-agent-next/wrapper/src/control/diagnostics.test.ts
• services/cloud-agent-next/wrapper/src/control/diagnostics.ts
• services/cloud-agent-next/wrapper/src/control/main.ts
• services/cloud-agent-next/wrapper/src/control/sandbox-control-client.ts
• services/cloud-agent-next/wrapper/src/control/sandbox-control-handlers.ts
• services/cloud-agent-next/wrapper/src/control/sandbox-control-runtime.ts
• services/cloud-agent-next/wrapper/src/control/worktree-runtime.ts

---

_{Reviewed by grok-4.6 · Input: 95.3K · Output: 4.1K · Cached: 123.6K}

_{Review guidance: REVIEW.md from base branch main}