Repository navigation
[FD-5916] feat(agents): the Letta Code image follows the env's clock #769
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
williamloo-scale
wants to merge
50
commits into
main
Choose a base branch
from
wloo/letta-code-clock-image
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+233
−13
Open
Changes from all commits
Commits
Show all changes
50 commits
Select commit
Hold shift + click to select a range
ae1d4be
feat(agents): agents follow the env's clock the way computers do (FD-…
williamloo-scale eea933c
refactor(agents): leave the Claude Code image to its own PR; agents s…
williamloo-scale 608a93d
Merge origin/main
williamloo-scale 182cef0
feat(agents): the Claude Code image follows the env's clock (FD-5916)
williamloo-scale bce00d4
fix(agents): agent clock review: new config's status, one clock per a…
williamloo-scale 84381b8
fix(agents): keep each agent's clock record on a key of its own; a re…
williamloo-scale 56dac45
fix(agents): agent clock keys can't collide; one clock env per agent;…
williamloo-scale c726743
chore: drop test run output committed by mistake
williamloo-scale 5e8c756
Merge #714
williamloo-scale 3b6f499
test(agents): the Claude Code image follows the env's clock end to en…
williamloo-scale cecc686
fix(agents): a failed agent sync retires its follower; the task's zon…
williamloo-scale afe55b4
Merge origin/main
williamloo-scale d523a6e
Merge #714
williamloo-scale ac0a39e
fix(agents): libfaketime's init can't deadlock the CLI; the CLI goes …
williamloo-scale c8ed6e1
fix(agents): a later clock sync leaves agents teardown_sandboxes took…
williamloo-scale b5d3b4b
fix(ci): the agent clock e2e skips as a declared capability gap; the …
williamloo-scale 25af21a
chore: drop test-run output committed again, and ignore where the pro…
williamloo-scale 7216c7c
fix(agents): route the CLI through egress by the clock page's armed f…
williamloo-scale fc3134b
Merge #714
williamloo-scale f8bd9ff
feat(a2a-agent): validate clock/v1 by the time the agent's own comman…
earakely-scale f9af1b6
refactor(a2a-agent): name the clock validation's timeouts and error l…
williamloo-scale 43fdb97
fix(gateway): get_time says the machine's clock may not be synchroniz…
earakely-scale 8bf8e4c
refactor(a2a-agent): move agent_clock to a2a_agent/clock.py, beside t…
earakely-scale 810f42e
Merge #714
williamloo-scale 686fdf1
test(agents): the agent clock e2e imports FOLLOWING_KEY from a2a_agen…
williamloo-scale 9d7d6c0
Merge origin/main (#714 merged)
williamloo-scale 1ff90bd
fix(agents): the Claude Code image installs protocol 0.1.575, the fir…
earakely-scale ca97b37
fix(computers): the clock follower follows a changed config while its…
earakely-scale 0875e06
fix(agents): an armed clock keeps the agent's own NO_PROXY hosts dire…
williamloo-scale bb4f07f
test(agents): the agent clock checks run over every clocked agent ima…
williamloo-scale 7980e27
feat(agents): the Letta Code image follows the env's clock (FD-5916)
williamloo-scale a3f9e34
Merge main into #726: the clock and egress proxy now live in utils (#…
earakely-scale 3c60fda
refactor(agents): the Claude Code image takes the clock and egress pr…
earakely-scale b916e31
Merge #726: the clock and egress proxy come from utils
williamloo-scale 150c563
test(agents): Claude Code's contract test takes the folders shared_so…
williamloo-scale a7e2a69
refactor(agents): the image takes the clock and egress proxy from uti…
williamloo-scale 4ddfca5
Merge #726
williamloo-scale aed27ee
fix(agents): a private HTTPS model endpoint is reached directly on an…
williamloo-scale 1fa4e62
fix(agents): a private HTTPS model endpoint is reached directly on an…
williamloo-scale 4b76192
Merge #726
williamloo-scale a6f3720
fix(agents): the model host's lookup is bounded and kept, so a slow r…
williamloo-scale 6b0e01f
fix(agents): the model host's lookup is bounded and kept, as #726's is
williamloo-scale 1b71135
Merge #726
williamloo-scale 34e2f5a
Merge main (#726 merged)
williamloo-scale 733a2f3
refactor(agents): follow the merged Claude Code image (#726): the pro…
williamloo-scale 36a2293
fix(agents): keep a provider's CA bundle trusted, and send HTTPS MCP …
williamloo-scale 1cb1094
Merge main: the control token (#780, #784)
williamloo-scale 6beca46
Merge remote-tracking branch 'origin/main' into wloo/letta-code-clock…
williamloo-scale 7be2292
Merge remote-tracking branch 'origin/main' into wloo/letta-code-clock…
williamloo-scale e7bc8c0
refactor(letta-code): take the shared egress rule (#818)
williamloo-scale File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| ; Root's supervisord (start-agent, FD-5916): the clock follower and the egress proxy, each as its own user, on real | ||
| ; time, and the A2A server as the agent's user. The server starts with the real-time marker, which agent-env's | ||
| ; libfaketime takes out of its environment, so the server's own records keep real timestamps and everything it starts, | ||
| ; the CLI and the model's commands, reads the env's time. Its socket and logs are root's alone, so the agent's user can | ||
| ; neither stop nor restart any of them. The follower follows nothing until the server writes its config, which | ||
| ; sync_env_clock has it do, then picks it up within a quarter second. | ||
| [supervisord] | ||
| nodaemon=true | ||
| user=root | ||
| logfile=/var/log/agentenv-root/supervisord.log | ||
| pidfile=/run/agentenv-root/supervisord.pid | ||
| childlogdir=/var/log/agentenv-root | ||
|
|
||
| [unix_http_server] | ||
| file=/run/agentenv-root/supervisor.sock | ||
| chmod=0700 | ||
|
|
||
| [rpcinterface:supervisor] | ||
| supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface | ||
|
|
||
| [supervisorctl] | ||
| serverurl=unix:///run/agentenv-root/supervisor.sock | ||
|
|
||
| [program:clock] | ||
| command=/usr/local/bin/python3 -I /usr/local/lib/agentenv-clock/clock-follower | ||
| user=agentenv-clock | ||
| directory=/ | ||
| priority=3 | ||
| autorestart=true | ||
| startsecs=1 | ||
| startretries=1000 | ||
| redirect_stderr=true | ||
| stdout_logfile=/var/log/agentenv-root/clock.log | ||
| environment=HOME="/nonexistent",USER="agentenv-clock",LOGNAME="agentenv-clock" | ||
|
|
||
| [program:egress] | ||
| command=/usr/local/lib/agentenv-egress/egress serve | ||
| user=agentenv-egress | ||
| directory=/ | ||
| priority=3 | ||
| autorestart=true | ||
| startsecs=1 | ||
| startretries=1000 | ||
| redirect_stderr=true | ||
| stdout_logfile=/var/log/agentenv-root/egress.log | ||
| environment=HOME="/nonexistent",USER="agentenv-egress",LOGNAME="agentenv-egress",AGENTENV_EGRESS_PRIVATE="allow" | ||
|
|
||
| ; uvicorn itself, not entrypoint.sh: the marker holds for the process it's given to alone, and a shell that execs | ||
| ; uvicorn would hand uvicorn an environment without it. HOME is the image's, /data, under which the CLI's state lives. | ||
| [program:agent] | ||
| command=/usr/local/bin/python3 -m uvicorn a2a_server:app --host 0.0.0.0 --port %(ENV_A2A_PORT)s --timeout-graceful-shutdown 5 | ||
| user=agent | ||
| directory=/app | ||
| priority=10 | ||
| autorestart=true | ||
| stopasgroup=true | ||
| killasgroup=true | ||
| stopwaitsecs=30 | ||
| redirect_stderr=true | ||
| stdout_logfile=/dev/stdout | ||
| stdout_logfile_maxbytes=0 | ||
| environment=HOME="/data",USER="agent",LOGNAME="agent",AGENTENV_REAL_TIME="1",AGENTENV_EGRESS_PROXY="http://127.0.0.1:3128",NODE_EXTRA_CA_CERTS="/etc/agentenv/ca.pem",SSL_CERT_FILE="/etc/agentenv/ca-bundle.pem",REQUESTS_CA_BUNDLE="/etc/agentenv/ca-bundle.pem" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| # What this image's build context takes from agent-env's own sources, at the name its Dockerfile copies it from: one | ||
| # copy of each, shared with the other images that take it (utils/docker/context.py). Paths are under src/agent_env. | ||
| clock = "utils/faketime_clock" | ||
| egress = "utils/egress_proxy" | ||
| agent_clock = "utils/agent_clock" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| #!/bin/sh | ||
| # The image's start (FD-5916). As root, it makes what the agent's user must not touch, the clock page and file (the | ||
| # clock user's from their first byte) and the egress proxy's CA (its key the egress user's alone, trusted in the | ||
| # system store, agent-env's trust files and every certifi bundle), then starts root's supervisord (root.conf), which | ||
| # runs the clock follower and the egress proxy, each as its own user on real time, and the A2A server as the agent's | ||
| # user on real time, everything it starts reading the env's time from the page. Until sync_env_clock has the agent | ||
| # follow a clock, the page holds real time and the proxy relays TLS untouched. Started as another user, the image runs | ||
| # the A2A server alone, as before, on real time. | ||
| set -eu | ||
| if [ "$(id -u)" != 0 ]; then | ||
| exec /opt/agent/entrypoint.sh | ||
| fi | ||
| install -d -o root -g root -m 0700 /var/log/agentenv-root /run/agentenv-root | ||
| install -d -o agentenv-clock -g agentenv-clock -m 0755 /run/agentenv-clock | ||
| python3 -I /usr/local/lib/agentenv-clock/clockpage.py create agentenv-clock | ||
| AGENTENV_SESSION_USER=agent AGENTENV_EGRESS_NSS=off \ | ||
| /usr/local/lib/agentenv-egress/egress boot > /var/log/agentenv-root/egress-boot.log 2>&1 \ | ||
| || { cat /var/log/agentenv-root/egress-boot.log >&2; echo "the egress proxy's boot failed" >&2; exit 2; } | ||
| # A provider's own CA bundle stays trusted: Sail, for one, points SSL_CERT_FILE and the like at the bundle it signs | ||
| # model requests under, and root.conf points the server's trust at agent-env's bundles instead, so the provider's goes | ||
| # into them, and into the egress proxy's roots, which carry those requests once the clock is armed. | ||
| for bundle in "${SSL_CERT_FILE:-}" "${REQUESTS_CA_BUNDLE:-}" "${NODE_EXTRA_CA_CERTS:-}" "${CURL_CA_BUNDLE:-}"; do | ||
| case "$bundle" in ""|/etc/agentenv/*) continue ;; esac | ||
| [ -r "$bundle" ] || continue | ||
| for target in /etc/agentenv/ca-bundle.pem /etc/agentenv/ca.pem /usr/local/lib/agentenv-egress/roots.pem; do | ||
| grep -qxF "# agentenv: $bundle" "$target" 2>/dev/null && continue | ||
| { printf '\n# agentenv: %s\n' "$bundle"; cat "$bundle"; } >> "$target" | ||
| done | ||
| done | ||
| exec supervisord -c /etc/agentenv/root.conf |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,73 @@ | ||
| """Letta Code on the env's clock: its bridge hands the shared egress rule (tst/unit/agents/test_agent_egress.py) every | ||
| MCP server's URL, which the CLI connects to itself, so on an armed clock the CLI goes out through the image's egress | ||
| proxy, and to loopback, the model relay's among it, the hosts NO_PROXY names and plain-HTTP hosts directly; unarmed, it | ||
| goes out as before. What every clocked image shares is checked in tst/unit/agents/test_agent_clock_images.py.""" | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import subprocess | ||
|
|
||
| import pytest | ||
|
|
||
| from tst.unit.agents.letta_code.agent import AGENT_DIR, agent_egress | ||
| from tst.unit.agents.letta_code.agent import letta_code_bridge as bridge | ||
|
|
||
| PROXY = "http://127.0.0.1:3128" | ||
|
|
||
|
|
||
| @pytest.fixture(autouse=True) | ||
| def _sessions(tmp_path, monkeypatch): | ||
| monkeypatch.setattr(bridge, "SESSIONS_ROOT", str(tmp_path)) | ||
|
|
||
|
|
||
| def _env(monkeypatch, environ: dict[str, str], *, urls=(), armed: bool) -> dict[str, str]: | ||
| for name in (agent_egress.EGRESS_PROXY_ENV, "NO_PROXY", "no_proxy", "HTTPS_PROXY", "https_proxy", "HTTP_PROXY", | ||
| "http_proxy", "ALL_PROXY", "all_proxy"): | ||
| monkeypatch.delenv(name, raising=False) | ||
| for name, value in environ.items(): | ||
| monkeypatch.setenv(name, value) | ||
| return bridge._context_env("ctx", urls, armed=armed) | ||
|
|
||
|
|
||
| def test_on_an_armed_clock_the_cli_goes_out_through_the_proxy(monkeypatch): | ||
| env = _env(monkeypatch, {agent_egress.EGRESS_PROXY_ENV: PROXY, "NO_PROXY": "registry.internal"}, armed=True) | ||
|
|
||
| assert {env[name] for name in ("HTTP_PROXY", "HTTPS_PROXY", "ALL_PROXY", "https_proxy")} == {PROXY} | ||
| # The model relay the CLI is pointed at is on loopback. | ||
| assert {"localhost", "127.0.0.1", "host.docker.internal", "registry.internal"} <= set(env["NO_PROXY"].split(",")) | ||
| assert env["NODE_USE_ENV_PROXY"] == "1" | ||
|
|
||
|
|
||
| @pytest.mark.parametrize("environ, armed", [({}, True), ({agent_egress.EGRESS_PROXY_ENV: PROXY}, False)]) | ||
| def test_without_the_proxy_or_an_armed_clock_the_cli_goes_out_directly_as_before(monkeypatch, environ, armed): | ||
| assert not {"HTTPS_PROXY", "NO_PROXY"} & set(_env(monkeypatch, environ, armed=armed)) | ||
|
|
||
|
|
||
| def test_the_envs_plain_http_mcp_servers_are_reached_directly_and_https_ones_through_the_proxy_on_an_armed_clock( | ||
| monkeypatch): | ||
| """Node checks an HTTPS server's certificate against the env's date, which the proxy, on real time, doesn't.""" | ||
| urls = ["http://10.0.0.5:8080/mcp", "https://abc-123.w.modal.host/mcp"] | ||
| direct = set(_env(monkeypatch, {agent_egress.EGRESS_PROXY_ENV: PROXY}, urls=urls, armed=True)["NO_PROXY"].split(",")) | ||
|
|
||
| assert "10.0.0.5" in direct and "abc-123.w.modal.host" not in direct | ||
|
|
||
|
|
||
| def test_start_agent_keeps_a_providers_ca_bundle_trusted_by_the_server_and_the_proxy(tmp_path): | ||
| """Sail points SSL_CERT_FILE and the like at the bundle it signs model requests under; root.conf points the server | ||
| at agent-env's bundles, so start-agent adds the provider's to them and to the proxy's roots, once each.""" | ||
| start = (AGENT_DIR / "start-agent").read_text() | ||
| merge = start[start.index('for bundle in "${SSL_CERT_FILE'):start.index("exec supervisord")] | ||
| targets = {"/etc/agentenv/ca-bundle.pem": tmp_path / "ca-bundle.pem", "/etc/agentenv/ca.pem": tmp_path / "ca.pem", | ||
| "/usr/local/lib/agentenv-egress/roots.pem": tmp_path / "roots.pem"} | ||
| for real, fake in targets.items(): | ||
| fake.write_text("SYSTEM ROOTS\n") | ||
| merge = merge.replace(real, str(fake)) | ||
| sail = tmp_path / "sailbox-ca-bundle.crt" | ||
| sail.write_text("-----BEGIN CERTIFICATE-----\nSAIL\n-----END CERTIFICATE-----\n") | ||
|
|
||
| subprocess.run(["sh", "-c", merge], check=True, env={"PATH": "/usr/bin:/bin", "SSL_CERT_FILE": str(sail), | ||
| "NODE_EXTRA_CA_CERTS": str(sail), "REQUESTS_CA_BUNDLE": "/etc/agentenv/ca-bundle.pem"}) | ||
|
|
||
| for fake in targets.values(): | ||
| text = fake.read_text() | ||
| assert text.startswith("SYSTEM ROOTS\n") and text.count("SAIL") == 1, (fake.name, text) |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.