Skip to content

feat(snapshot): the save response takes optional stats, and agent-env logs each save - #697

Open
andrewpark-scaleai wants to merge 3 commits into
mainfrom
andrew-park/rel-snapshot-save-telemetry
Open

andrewpark-scaleai wants to merge 3 commits into
mainfrom
andrew-park/rel-snapshot-save-telemetry

Conversation

@andrewpark-scaleai

@andrewpark-scaleai andrewpark-scaleai commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Why

The goal is that every failure of an agent run leaves enough in the logs to debug it, and that code heals what it can. When a snapshot_agent_state save is slow, agent-env's logs cannot say why. The agent logs its archive's file count and size, but that line stays in the sandbox. agent-env logs nothing about the save on success, and only the exception on a timeout. So a slow save cannot be told apart as "the workspace is big", "archiving is slow" or "the upload is slow" without reaching into a live sandbox.

What changed

  • ObjectSnapshotSaveResponse gains an optional stats field: file_count, archive_seconds and upload_seconds (SnapshotSaveStats).
  • capture_workspace logs event=snapshot_save_completed with agent_name, context_id, elapsed_seconds, timeout_seconds, workspace_archive_bytes (from the uploaded object) and the three stats.
  • On a timeout it logs event=snapshot_save_timeout with the elapsed time, the budget and the workspace grant's cap, then re-raises.
  • When the caller cancels the save, it logs event=snapshot_save_cancelled with the elapsed time and the budget, then re-raises. A periodic capture's outer wait_for shares the save's budget and starts first, so a stalled periodic save ends here, not in httpx's timeout.
  • Agent cards built against this protocol list stats under the save response's optional fields, so the card tests for claude-code, codex and gemini expect it.

Design decisions

  • The claude-code change moved to a follow-up, feat(claude-code): report a snapshot save's file count and phase durations #711. An agent image installs the protocol from PyPI, not from this tree, and CONTRIBUTING.md says an agent's floor rises only once the protocol it names is published. The first version of this PR filled stats in the claude-code agent and raised its floor in the same change, so the image could not find a protocol with the field and every claude-code integration test errored. The agent validates its own answer against its installed protocol, which rejects an unknown field, so the agent change cannot work against a published protocol without a shim. This PR now ships the protocol field and agent-env's logging. The tradeoff is one release of lag: until the follow-up merges, claude-code saves log their size and elapsed time with null stats.
  • Stats ride on the response. The worker already parses the save response, and the archive's size is already in it. A separate size call would add protocol surface for a number the agent has at hand.
  • Optional and ignorable. A worker parses the response with unknown fields ignored, so an older worker accepts a newer agent's answer. An agent that sends no stats, which is every agent image until the follow-up, logs them as null. test_a_completed_save_from_an_agent_that_reports_no_stats_logs_its_size pins that.
  • A timeout carries no stats. The agent has not answered yet. Seeing progress during a save needs a streamed or polled save, a larger protocol change left for later.
  • Fields are log-only. Ids and sizes are unbounded, so none become metric tags.

Scope

  • No agent reports stats yet. Does not change any timeout or retry.

Blast Radius

Adds one log line per save, or one on a timeout or cancel, and one optional response field. It changes the advertised snapshot/v1 card (an added optional field). No control flow changes.

Verification

  • uv run --extra dev pytest tst/unit/task_step/test_agent_state_capture.py tst/unit/agents/claude_code/test_card_contract.py tst/unit/agents/claude_code/test_workspace_snapshot.py tst/unit/agents/claude_code/test_object_transfer.py tst/unit/agents/claude_code/test_image_contract.py tst/unit/agents/codex/test_card_contract.py tst/unit/agents/codex/test_object_transfer.py tst/unit/agents/gemini/test_card_contract.py packages/agentenv-protocol/tests/test_a2a_agent.py -q: 224 passed.
  • After rebasing onto main: tst/unit/task_step/test_agent_state_capture.py, the tst/unit/task_step/*snapshot*.py files, the three card tests and test_a2a_agent.py: 399 passed. tst/unit/agents/: 1023 passed.
  • After deploy: search agent-env-worker logs for @event:snapshot_save_completed, @event:snapshot_save_timeout and @event:snapshot_save_cancelled.

RetriggerView in GreptileConfidence Score: 5/5

This PR appears safe to merge.

Summary

Adds optional snapshot-save statistics and logs save completion, HTTP timeouts, and caller cancellation without changing the save budget or retries.

  • Snapshot save responses can include archive counts and timings.
  • Agent-env logs the size and timing of each snapshot save.

The earlier missing cancellation-log issue is addressed. No new actionable issues found. Agents in this PR do not report statistics yet; Claude Code reporting remains in follow-up PR #711.

Reviews (5) · Last reviewed commit: "fix(snapshot): log a save the caller's d..." · Reviewed by Greptile

@andrewpark-scaleai
andrewpark-scaleai force-pushed the andrew-park/rel-snapshot-save-telemetry branch from 0081850 to 5515f7b Compare November 16, 2026 18:05
@andrewpark-scaleai andrewpark-scaleai changed the title feat(snapshot): report a save's file count and phase durations, and log each save feat(snapshot): the save response takes optional stats, and agent-env logs each save Nov 16, 2026
@andrewpark-scaleai
andrewpark-scaleai force-pushed the andrew-park/rel-snapshot-save-telemetry branch from 5515f7b to c98261a Compare November 18, 2026 18:20
@andrewpark-scaleai
andrewpark-scaleai marked this pull request as ready for review November 18, 2026 18:37
@andrewpark-scaleai
andrewpark-scaleai requested a review from a team as a code owner November 18, 2026 18:37
@andrewpark-scaleai
andrewpark-scaleai force-pushed the andrew-park/rel-snapshot-save-telemetry branch from c98261a to 35f6dc0 Compare November 18, 2026 18:53
Ubuntu and others added 3 commits November 18, 2026 23:09
…s in agent-env's logs

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
… logs each save

ObjectSnapshotSaveResponse gains an optional stats field: the archived file count and
the seconds the agent spent archiving and uploading. capture_workspace logs
snapshot_save_completed with the workspace archive's size and those stats, or
snapshot_save_timeout with how long it waited. An agent that sends no stats still
logs its size, with the stats fields empty.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
A periodic capture's outer wait_for shares the save's budget and starts
first, so a stalled save is cancelled before httpx times out and never
logged snapshot_save_timeout. capture_workspace now logs
snapshot_save_cancelled with the elapsed time and budget, then re-raises.
@andrewpark-scaleai
andrewpark-scaleai force-pushed the andrew-park/rel-snapshot-save-telemetry branch from 35f6dc0 to bd5d058 Compare November 18, 2026 23:11

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant