Repository navigation
[FD-5916] feat(agents): the Codex image follows the env's clock - #765
williamloo-scale wants to merge 54 commits into
Conversation
…5916) Prototype of the Agents on the Env Clock design, to size the change. The gateway takes followers declared after it starts; every AgentEnvAgent in an image that runs the clock follower serves clock/v1 sync_time; sync_env_clock and deploy_agent declare each agent as a follower and hand it its config; the Claude Code image runs the computer base's follower, egress proxy and libfaketime, copied in until FD-5700; get_time no longer calls the machine's clock unsynced. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…kip unless require_agent_clock The image's changes move to wloo/claude-code-clock-image, along with the computer base's egress options only it uses: on CI the image's Claude Code timed out reaching the test's model endpoint. What stays serves sync_time for an agent whose image runs the follower and is otherwise inert. An agent that can't follow is now skipped even when tolerate_missing_sync_time is false, which stays about the env's servers; require_agent_clock fails the step instead. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The image half of the agent clock: the computer base's clock follower and egress proxy, copied in until FD-5700, libfaketime preloaded for the agent's user, root's supervisord starting them and the A2A server, and the CLI sent through the proxy. Needs #714 and an agentenv-protocol release with CLOCK_V1. Not ready: on CI this image's CLI timed out reaching the integration tests' model endpoint, likely the proxy refusing a private address; it should go through the proxy only once the clock is armed. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…gent, retire on teardown - An agent's sync_time takes only a status of the config it just wrote (its URL and name, written after it), so what the follower showed of an earlier gateway or token never passes for success. - An agent follows one env's clock; another env's is skipped as follows_another_env, as its one follower can't watch two gateways. deploy_agent's late follows are recorded with the sync's, so both see it. - teardown_sandboxes retires a torn-down agent's follower (DELETE /clock/followers, retire_follower), so later writes don't wait for it or answer 504 naming it. A declaration no longer replaces a follower the gateway started with. - Logs say an outcome's synced, reason and follower, never the agent's answer; the timeouts, token size and error length are named constants. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…tirement wakes waiting writes - Which env an agent follows moves from clock_configurations' items to metadata.agent_clock_following, one key per agent, so late deploy_agent steps that save at once never overwrite each other's record and teardown still finds it. - Retiring a follower notifies writes waiting for it, and a watch by a follower retired meanwhile acknowledges nothing. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… retire however its sandbox goes - Agent names are percent-encoded into their metadata keys, so team.solver and team_solver never share a record. - An agent follows the clock of the first env it's registered with, fixed by the agent alone, so two steps arming two of its envs at once never both take its one follower. - teardown_sandboxes retires the follower of every agent whose sandbox it takes down, by agent, sandbox or env name. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…d on local Docker (FD-5916) Deploys the image with an env, runs sync_env_clock with require_agent_clock, and checks that the agent's user reads the env's time (date and datetime.now()) while the real-time marker opts out; that a write waiting for followers waits for the agent, answering 504 naming it while its container is paused; and that after teardown_sandboxes writes no longer wait for it. Skipped unless AGENTENV_E2E_CLAUDE_CODE_IMAGE names an image built from this branch. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…e survives a server restart; get_time keeps its sentence - follow retires the follower it declared when the agent's sync_time fails, by an HTTP error or a transport one, so the gateway isn't left waiting on every later write for a follower that never watches. - sync_time saves the task's zone in the follower's config, and the A2A server takes it back when it starts, so a server supervisord restarts mid-run keeps the zone as the follower keeps the clock. - get_time's description keeps "the clock on the machine you are running on is not synchronized": most agents won't follow the clock for a while yet. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…through the proxy only on an armed clock - The agent's libfaketime takes Edgar's init guard: a clock call made while ftpl_really_init runs on that thread, as the CLI's allocator makes one when dlsym allocates, reads the kernel's clock instead of waiting on pthread_once, which deadlocked: claude --version never returned. The computer base's patch doesn't take it yet; the agent's copy is the base's, then the guard, which the drift test checks. - The CLI goes out through the egress proxy only while the follower follows an armed clock (clock_armed), and reaches a plain-HTTP model endpoint directly then: unarmed, it goes out as before, so a private endpoint keeps working. - egress/ is the base's again after main moved it. - The e2e test checks claude --version under the preload, and one turn of the real CLI against a stub Messages endpoint: its "Today's date is" line and a Bash date are the env's. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… down alone Their records stay in deployed_agents after teardown, and calling their gone servers failed the step. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…browser base's egress copy takes boot.go - test_agent_env_clock_local skips as agentenv-capability-missing: agent_clock_image, which the slow integration job allows, as no CI job builds agent images. - ubuntu-24.04-browser's egress/ is ubuntu-24.04's again, with this PR's AGENTENV_SESSION_USER and AGENTENV_EGRESS_NSS. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…tocol's tests write it Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…lag, not the follower's status The page is what libfaketime reads, and it keeps the last armed clock while the follower reconnects to a restarted gateway or restarts itself, when its status says connected false or armed false. A CLI started then reads the env's time, so it must go through the proxy. Unit tests pin both windows. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ds show An agent whose card serves clock/v1 sync_time is now checked end to end after the validation DAG: the validation env's clock is armed frozen in America/Chicago with require_agent_clock, the agent runs `date` through its harness and must show that instant and zone, a write that waits for followers must be acknowledged, and `date` must show the time it moved to. A synced follower alone doesn't pass: an image whose preload is missing, or whose harness can't run under it, follows and still fails. An agent whose card serves no sync_time is skipped. The result is recorded under validated_a2a_extensions[urn:agentenv:clock/v1] and verifications["a2a_clock"]. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ength Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ed with the env An agent in an image that follows the env's clock reads the env's time, so "is not synchronized, so do not use it" is false for it. "May not be" holds for every agent, following or not, and get_time answers what a following agent's clock already shows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he other agent extension clients It drives an agent's clock/v1 sync_time and the gateway's follower declarations for the steps that call it (sync_env_clock, deploy_agent, teardown_sandboxes), as object_transfer does for snapshot transfers; it defines no step. The name mirrors agentenv_protocol.a2a_agent.clock, the agent's side of the same extension. The persisted metadata key agent_clock_following is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t.clock, where #714 moved it Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…st with an agent's clock/v1 sync_time 0.1.575 is the release that carries #714 (agentenv_protocol.a2a_agent.clock): before it, the image's protocol serves no clock/v1, so the agent advertises none and sync_env_clock skips it as not_advertised. The image is built again only when this folder changes, so the floor is what makes it take the release. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… gateway refuses at once A watch that fails fast (connection refused) answers before the follower's quarter-second poll sees a changed config, and the retry backoff ends early on that change, so the follower kept retrying the old URL in a tight loop and never followed the new one: an agent re-synced to another gateway after its first one went away stayed on the old clock, burning a core, and every write that waited for it answered 504. It now checks the config after each failed watch. The base's, the browser base's and the Claude Code image's copies stay identical. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ct; name the egress probe's error limit - egress_env merges the hosts NO_PROXY and no_proxy already name with the direct ones, so a private registry the agent was deployed to reach directly isn't sent to the proxy, which refuses it, once the clock is armed. - egress/upstream.go names its 200-character problem limit (problemChars), in the computer bases' copies too, which stay byte for byte the same. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ge, not Claude Code's alone - tst/unit/agents/test_agent_clock_images.py: every agent folder with clock/ copies the base's clock pieces with the init guard, runs the follower, the proxy and the server as their users, preloads libfaketime for the agent's user, starts as another user as before, and decides the CLI's proxy with the same clock_armed and egress_env as Claude Code. - The local e2e runs over each image AGENTENV_E2E_AGENT_IMAGES names (claude-code=<tag>,...), with a CLI turn for an agent whose CLI the stub endpoint speaks to. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… brings as in the build context Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… armed clock; name the clock page's limits - model_direct_hosts: the model endpoint's host is direct on an armed clock when it's plain HTTP or isn't a public address (an IP literal, or a name that resolves only to non-public ones), which the egress proxy refuses. Resolved only while the clock is armed. Shared, with _global_host, by every clocked bridge. - clock_armed's retry count, delay and check salt are named (_PAGE_READ_TRIES, _PAGE_RETRY_S, _PAGE_CHECK_SALT). - The e2e's cleanup finds a stopped or missing agent container without raising, and always closes the env. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… armed clock; name the clock page's limits The bridge takes #726's shared helpers (model_direct_hosts, _global_host, the named page limits). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…hout a lookup leaving the machine Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…esolver can't stall the A2A server The lookup runs on a thread of its own and waits _RESOLVE_TIMEOUT_S (2 s) at most, once per host per process; a host that doesn't resolve in time is left to the egress proxy. Shared by every clocked bridge. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
# Conflicts: # src/agent_env/agents/README.md # src/agent_env/agents/agents/claude-code/claude_bridge.py # src/agent_env/agents/agents/claude-code/root.conf # tst/integration/task_step/test_agent_env_clock_local.py # tst/unit/agents/claude_code/test_env_clock_image.py # tst/unit/agents/test_agent_clock_images.py
…xy reaches private networks, a plain-HTTP model endpoint alone is direct The bridge takes #726's final helpers (model_direct_hosts without a lookup), and root.conf gives the egress proxy AGENTENV_EGRESS_PRIVATE=allow, so a model endpoint over TLS at a private address goes through it at any date. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…servers through the egress proxy - start-agent adds the bundles SSL_CERT_FILE, REQUESTS_CA_BUNDLE, NODE_EXTRA_CA_CERTS and CURL_CA_BUNDLE name (Sail's, which it signs model requests under) to agent-env's bundles, which root.conf points the server at, and to the egress proxy's roots, which carry those requests once the clock is armed. - Only plain-HTTP MCP hosts are reached directly on an armed clock: an HTTPS one goes through the proxy, which checks its certificate on real time, where the CLI's own TLS would check it against the env's date. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The Codex CLI verifies with rustls, which refuses any certificate dated before 1970 as badly encoded, so it rejected every leaf the proxy minted (dated from the CA's 1950) once the clock was armed. The CA keeps its window; only its leaves start at the epoch. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
| SerialNumber: serial(), | ||
| Subject: pkix.Name{CommonName: truncate(host, commonNameLimit)}, | ||
| NotBefore: by.cert.NotBefore.Add(LeafInset), | ||
| NotBefore: latest(by.cert.NotBefore.Add(LeafInset), LeafFrom), |
There was a problem hiding this comment.
Historical clocks break HTTPS. This changes every shared proxy certificate to start in 1970, not just certificates used by Codex. A computer or Claude Code agent following a supported date such as 1960 now receives a certificate that is not yet valid when it opens a live HTTPS site. Those clients previously accepted the certificate starting in 1950. Keep the earlier window for existing clients rather than applying Codex’s date restriction to every image.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agent_env/utils/egress_proxy/leaves.go
Line: 89
Comment:
**Historical clocks break HTTPS.** This changes every shared proxy certificate to start in 1970, not just certificates used by Codex. A computer or Claude Code agent following a supported date such as 1960 now receives a certificate that is not yet valid when it opens a live HTTPS site. Those clients previously accepted the certificate starting in 1950. Keep the earlier window for existing clients rather than applying Codex’s date restriction to every image.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.|
🤖 Automated Claude routine — Env Pod PR Context Review. This is a bot, not Edgar Arakelyan; not a human review. Context from a sibling PR rather than from this diff. #818 (open, ready) moves the clock-egress rule out of the per-agent bridges into one
One behavioural note on the same seam: #818 was opened for an env whose Generated by Claude Code |
# Conflicts: # tst/integration/task_step/test_agent_env_clock_local.py
|
Hi @williamloo-scale, this is Claude, working with Edgar. Please don't merge this yet. It needs #818 merged a single egress rule for every clocked agent:
This branch still has its own Why it matters beyond the test: on an armed clock, the CLI checks TLS certificates against the virtual date. An env on Modal whose Codex specifically: only the model endpoint goes direct here, so MCP is already proxied. This one is just the migration. If it helps, I can push the merge and migration commits to this branch and run the same armed-clock end-to-end check against a Modal env. Just say so here or on Slack. |
The bridge imports clock_armed and egress_env from agent_clock's agent_egress, which the image and an install copy beside it with clockpage, and hands it the model endpoint's URL; its own copy of the rule goes. mcp-remote, which codex hands only the trust variables, is still not given the proxy. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ss proxy on an armed clock Codex hands mcp-remote only the variables its env_vars names, so on an armed clock it went to an HTTPS MCP server directly and checked the server's certificate against the env's date. It now gets the proxy's variables too, and the bridge hands the egress rule every MCP server's URL, so only plain-HTTP ones go direct, as #818 has for Claude Code and OpenCode. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
Thanks, done, with one correction: codex's MCP wasn't already proxied. Codex hands a stdio server only what @greptileai review |
|
Claude again, working with Edgar. Thanks for the update. The migration onto
I agree with Greptile's open P1 on
It would help to say in the PR why Codex needs 1970. My guess is that its TLS stack rejects a Two codex-scoped options:
I can run the armed-clock check against a Modal env for Codex once this settles, if that helps. |
What and why
The Codex image gets the same env clock as the Claude Code image in #726, now merged, which this follows. Once a task sets the clock, the Codex CLI and every command the model runs read the env's time, including
dateanddatetime.now().What changes
shared_sources.tomltakes the clock library and the egress proxy from agent-env's one copy inutils(utils/faketime_clock,utils/egress_proxy, [FD-5700] refactor(computers): the clock library and the egress proxy live once, in utils, and each image takes them #763), as the Claude Code image and the computer bases do, libfaketime's init guard included.start-agentandroot.conf: root's supervisord runs the clock follower, the egress proxy and the A2A server. The A2A server runs asagentwith the real-time marker and the same uvicorn arguments asentrypoint.sh. Started as any other user, the image runsentrypoint.shas before.root.confgives the egress proxyAGENTENV_EGRESS_PRIVATE="allow", as [FD-5916] feat(agents): the Claude Code image follows the env's clock #726 does, so a model endpoint over TLS at a private address goes through the proxy at any date. Only a plain-HTTP model endpoint is reached directly.agentonly, noUSER, andCMD ["/usr/local/bin/start-agent"].requirements.txt: the protocol floor is 0.1.575, the first release whose agents offersync_time.codex_bridge.cli_env: with the clock set, the CLI goes through the egress proxy. It reaches the hostsNO_PROXYnames and a plain-HTTP model endpoint directly.mcp-remotegets only codex's trust variables from it, so MCP traffic stays direct either way.tst/unit/agents/codex/test_env_clock.pycovers the proxy routing.test_agent_clock_images.pynow covers codex too.codexrow.Testing
codex --versionreturnscodex-cli 0.147.0under the preload.AGENTENV_E2E_AGENT_IMAGES=codex=<tag>, 63 s):dateanddatetime.now()show the env's time, and the marker opts out;agent-solverwhile its container is paused;SSL_CERT_FILE) for HTTPS while the clock is set.🤖 Generated with Claude Code
Do not merge until the existing HTTPS failure for clocks before 1970 is addressed.
Fix with agent prompt
Summary
Codex now follows the environment’s clock while its A2A server keeps real time.
No new findings. Previous finding #1 remains outstanding: the shared proxy still rejects HTTPS connections at supported dates before 1970. The provider-certificate merge and shared-helper fixes are present.
Reviews (7) · Last reviewed commit: "fix(codex): mcp-remote reaches the env's..." · Reviewed by Greptile