You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
refactor(harnesses): docker-* solver, sandbox and probe on the process runner - #104
The docker-* harnesses (docker-oracle, docker-claude[-host][-skill:], docker-probe[-host]) were the last spawn sites outside dokimi/process.py. Now every docker step, both claude launches and both isolation probes go through the harness's runner. create_harness(runner=) and TaskSuiteRunner(runner=) reach them as they reach the other harnesses.
DockerSolverHarness.run (+ runner= field)
_docker(args) # 43 call sites: 22 in docker_solver, 21 in ContainerSandbox
- subprocess.run(timeout=None | budget) # most steps unbounded+ runner(Env.inherit, capture_limit=None, timeout=budget or 600 s)
_build_image docker build build_timeout_sec (unchanged)
_solve_oracle docker exec solve.sh
- no limit+ agent budget -> reward 0, rc 124
_solve_claude (host) run_solver
- subprocess.run(env, timeout) # kills claude only+ runner(Env.only((), env), capture_limit=None) # kills claude's whole tree
_solve_claude_in_container
run_solver(docker exec ... claude) same path; agent env still in a 0600 --env-file
_solve_probe
ContainerSandbox.run_probe
- subprocess.run(timeout=120)+ self.runner(Env.only((), client_env), timeout=120)
run_probe (host) already on the runner; now gets the harness's
_verify docker exec test.sh verifier timeout (unchanged)
everything else cp, inspect, exec mkdir/chmod/rm, run -d, rm -f, network *, sidecar create/start/logs, recompile
- no limit+ 600 s
Output is kept whole (capture_limit=None), in memory, as subprocess.run kept it. The runner's default 1 MiB head and tail would cut solver.jsonl, the tool counts and the sidecar's decision log. Writing it to a spool file would put the stream on disk before it is redacted.
Tests drive the pipeline through DockerCLI (tests/docker_cli_fake.py), a FakeRunner that answers like a daemon with one healthy task container and stands in for claude. They no longer patch _docker, subprocess.run or the private steps.
Evidence
Tree kill, before/after. A stand-in claude (a real process) starts a tool process that sleeps, then hangs past a 1.5 s budget:
run_solver(["claude", ...], timeout=1.5) # same script against both trees
main: FAILED AssertionError: the agent's tool process outlived the cell
branch: 1 passed
In the repo: test_a_timed_out_host_agent_leaves_none_of_its_tools_running runs it through docker-claude-host (cell rc 124, reward 0, tool gone).
Budgets.test_every_step_runs_with_a_budget[docker-oracle|docker-claude|docker-claude-host] checks every call's timeout:
step
main
now
docker build
build_timeout_sec
same (overrun: errored)
solve.sh (oracle)
none
agent budget (overrun: reward 0, rc 124)
claude host / in container
agent budget
same (host: whole tree killed)
test.sh
verifier timeout
same
container probe / host probe
120 s / 60 s
same
every other docker step
none
600 s
test_solve_or_verify_timeout_is_a_failed_cell[solve.sh] (reward 0, rc 124, partial output in run.log) and test_a_docker_step_without_a_budget_that_hangs_errors_the_cell (docker cp failed (timed out after 600s), env is the caller's).
Whole stream.test_a_stream_over_the_capture_limit_is_kept_whole: a 1.5 MiB stream with 3 tool_use events and a 1 MiB verifier log. With the runner's default capture: tool_calls == {}, so it fails. Now: {"Bash": 3}, solver.jsonl is byte-identical and the log's middle is in run.log.
Fake CLI instead of patches. Across test_docker_solver_harness, test_container_agent, test_agent_sandbox, test_docker_task_md, test_task_formats and test_skill_treatment: 92 monkeypatches of _docker / subprocess.run / private steps on main, 0 now. test_nothing_else_in_the_package_spawns_a_process fails on any new subprocess/os.system/asyncio spawn outside process.py. test_a_hung_container_probe_is_untested_not_sealed covers the probe's timeout.
No real CLI ran in tests. The suite ran under a spawn guard. It is a pytest plugin that raises on any spawn that resolves to an installed claude/codex/harbor/bench/docker, plus PATH shims for grandchildren. Two random seeds: 5187 passed, 2 skipped each, blocked=0, 0 shim hits. The timing-sensitive files (process runner, sandbox, container agent, docker solver, task.md; 313 tests) ran 3x in parallel: all passed.
Local CI.make ci-local --offline: 14 passed, 0 failed (ruff, mypy strict with no new exemption, tests, schema, docs build, changelog fragment). Coverage of changed lines: 100% (73 lines).
CI. All checks pass, including Tests on py3.11 to 3.14 and the external catalog smoke test, which runs a real uxlfoundation/skills task through docker-oracle and docker-probe against Docker.
Merge Danger
Door: two-way. No schema, artifact field or persisted format changes. Reverting restores the old spawn code.
Blast Radius: task-cells
docker-oracle: a solve.sh that overruns the agent budget (agent.timeout_sec, else 600 s) is now a failed cell, reward 0, rc 124, and the verifier does not run. Before, it waited with no limit. Oracle pass rates can drop for tasks whose oracle is slower than their own agent budget.
Steps that had no budget now get 600 s:
One that must succeed and runs out (copy-in, mkdir, chmod, docker run -d, the sidecar's create, which may pull its pinned image) errors the cell: docker <step> failed (timed out after 600s).
One whose failure was already tolerated treats running out as that failure: rm -f and network rm are ignored, the recompile lands in recompile_failed, and the reward-file copy falls back to the verifier's exit code.
A killed step's partial output now goes to run.log. Before, only the TIMEOUT line was written.
docker-claude-host: a timeout kills claude's whole process group, and so does its exit. A background process the agent started no longer outlives its cell. Ctrl-C, or SIGTERM to the dokimi CLI, kills a running docker step's or claude's group.
docker and claude output is decoded as UTF-8 with replacement. Before, it was decoded in the locale's encoding.
Library API: DockerSolverHarness has a runner field, and ContainerSandbox.open takes runner=. solver_stream.run_solver takes keyword-only timeout/env/cwd/runner and returns process.Completed, which record_solver and container_agent.Docker now use too.
Security properties unchanged:
The in-container agent stays on the --internal network with egress only through the sidecar to the LLM hosts.
Its env stays in a 0600 --env-file, never on argv. The docker client gets exactly its old env (Env.only((), client_env)).
The host agent gets exactly workspace.env(...), the allow-list. The docker CLI keeps the caller's env (Env.inherit).
Order of steps: the agent uid is reaped, then the plugin and the credential files are removed, then /tests is copied in. The verifier's proxy credentials are still stripped, and the proxy is still stopped before a public task joins bridge.
run.log and solver.jsonl are redacted before they are written. Output is never spooled raw to disk.
Isolation probes: a hung probe is still untested and reward 0, with the same 120 s and 60 s.
These are covered by the existing launch, egress and isolation tests, ported from patches to DockerCLI.
Checklist
Tests added/updated and pytest passes
ruff, mypy, and schema checks pass
Docs updated (or N/A) — see the docs-drift checklist in CONTRIBUTING.md
No ad-hoc run artifacts committed
New files carry the SPDX license header (or are exempt in .github/.licenserc.yaml)
✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.31%. Comparing base (964c87f) to head (aca2f9f).
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
The reason will be displayed to describe this comment to others. Learn more.
Fixed in c653714: both callbacks (the close test and the Ctrl-C test) now return reply(), and reap/stop_proxy read timed_out from the runner's Completed.
…s runner
DockerSolverHarness takes runner= like the other harnesses, and
create_harness passes it on (the pass-through for harnesses without one
is gone). Every docker step, the host and the in-container claude, and
both isolation probes go through it. Nothing in the package spawns
outside dokimi/process.py now, and a test keeps it that way.
- _docker(): a step with its own budget (build, solve, verify) still
raises TimeoutExpired, so the caller decides the cell. Any other step
gets 600 s (before: none). Running out errors the cell where the step
must succeed. A killed step's partial output is written to run.log.
- docker-oracle: solve.sh gets the agent budget (before: unbounded). An
overrun is a failed cell, reward 0, like an agent timeout.
- docker-claude-host: a timeout kills claude's whole process tree, so
the tool processes it started no longer outlive the cell.
- run_solver() and _docker() keep their output whole (capture_limit=None).
The runner's 1 MiB head-and-tail capture would cut solver.jsonl, the
tool counts and the sidecar's decision log. The output stays in memory,
as with subprocess.run, so nothing reaches disk before it is redacted.
- run_solver(): exactly the env it is given (Env.only), stdin closed.
ContainerSandbox carries the runner for the in-container probe.
Tests drive the harness through DockerCLI (tests/docker_cli_fake.py), a
FakeRunner standing for the docker CLI and claude. They no longer patch
_docker, subprocess.run or the private pipeline steps (_build_image,
_start_container, _solve_*, _verify, _recompile, ContainerSandbox.open).
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Three critical timeout handling fixes ensure failed cleanup operations or
timeouts don't allow task scoring to proceed with stale state:
1. stop_proxy timeout (line 235): Raise if sidecar removal times out, preventing
verification from running with credentials still exposed. close() retries
removal as best-effort but doesn't re-raise.
2. reap timeout (line 450): Raise if process reap times out, preventing verification
from running while backgrounded agent processes may still access test files.
3. recompile timeout (line 577): Raise if compilation times out, preventing stale
or partially-built binaries from being verified, and stopping resource consumption.
All three changes ensure the container is removed without running verification
when these critical pre-verification operations timeout.
Test coverage added for all three scenarios.
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
Isolate provider environment and settings in lifecycle tests
dokimi/tests/test_cleanup_timeouts.py:27
These tests still read the developer's environment and Claude settings through operator_source_env(). Importing _fixture_task and _pipeline does not activate their module's _hermetic fixture. For example, ANTHROPIC_BASE_URL=http://localhost:4000 makes all four tests fail in preflight before reaching cleanup. Isolate the provider/proxy environment and settings here, as the other Docker lifecycle tests do.
The reason will be displayed to describe this comment to others. Learn more.
Review at aca2f9f. The runner migration itself is sound: budgets, env handling and the credentials-off-argv path hold up. The problems are the cleanup steps the PR left on check=False. Two of them can now score a cell as a pass when a step times out. I'd fix those before merge.
make ci-local with docker: 23 passed, 0 failed, 3 skipped. Changed-line coverage gate passed.
Reproductions for the code findings through the DockerCLI fake, no real docker, claude or LLM.
Targeted mutants of the new code (timeout dropped from the oracle, build, verify and recompile calls, the timed_out check in _recompile, redaction, credentials=True for the verifier, the FILES_DIR removal). Of those I have results for, two survive: the oracle's 600 s default changed to 6000 s, and the [killed after ...] line dropped from run.log.
Documentation
Documentation does not cover the reap and sidecar-removal timeouts. I agree with @copilot-pull-request-reviewer on the recompile line in the changelog. Two more cases: reap() (container_agent.py:223) and stop_proxy() (container_agent.py:337) also raise on a timeout, and run() turns that into reward 0, rc 124. The changelog lists rm -f under steps whose failure is tolerated, and coding-harnesses.md:881-890 lists only the agent, verifier and oracle overruns as reward 0. A slow daemon will read as a model failure in a pass-rate table.
Checked, no issues
Env: the docker client gets Env.inherit only for the CLI; claude and the probe get Env.only((), env) with the same dict as before. Agent env stays in the 0600 --env-file.
Callers of run_solver / record_solver: both call sites in docker_solver.py pass timeout, env and runner; record_solver only uses fields Completed has.
Sidecar: a stop_proxy timeout leaves _stopped false, close() retries, and test.sh never runs after it.
Registry:create_harness(runner=) reaches all docker-* adapters.
Repo rules: changelog fragment present, SPDX headers on new files, no generated artifacts, no nav change needed.
…xt copy timeouts
Add tests that verify credential-file removal timeout aborts the cell before
verification, preventing access to leftover credential files. Also add test
for reward.txt copy timeout to ensure the task fails rather than proceeding
with incorrect assumptions.
Update docstrings to use behavior descriptions instead of thread numbers.
Fix assertion to check for exactly 3 removal attempts (open + stop_proxy + close).
Update changelog to clarify that critical cleanup steps abort the cell on
timeout to prevent security issues and stale binary scoring.
Addresses PR intel#104 review threads.
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
…ting the verifier run
The pre-verifier reap, the plugin and credential-file removals, the reward.txt
copy and the recompile now raise on a timeout (reward 0, rc 124). The rest of the
cleanup still runs after a reap timeout, and the solve is recorded before it.
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
The reason will be displayed to describe this comment to others. Learn more.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
The reason will be displayed to describe this comment to others. Learn more.
No change: every adapter create_harness can return (CommandHarness, DockerSolverHarness, BenchflowHarness) declares a runner field, so the replace cannot hit an adapter without one.
Prevent zero verdicts when conflict coverage is incomplete
dokimi/dokimi/gate/novelty_gate.py:182
Nonempty categories do not establish that conflict detection completed. With zero harmful atoms and conflict_coverage={"pairs_compared": 0, "pairs_total": 45}, this now reports measured=True, passed=True, and value 0. Partial scans, malformed coverage, and capped samples can likewise appear cleared. Restore the coverage and cap guards together with persisted coverage metadata: incomplete scans should withhold a zero verdict, while conflicts actually found should still be reported.
Restore whole capped-set batching for cross-chunk conflict detection
dokimi/dokimi/novelty/analyze.py:273
Removing the batch override restores chunks of 40. At the default 120-atom cap, only 2,340 of 7,140 pairs share a call, so contradictions between chunks cannot be detected even when every call succeeds. Restore the whole capped-set batch to preserve cross-chunk conflict detection.
The reason will be displayed to describe this comment to others. Learn more.
Review at c653714. All six of my earlier threads are fixed, and the cleanup logic in docker_solver.py and container_agent.py now does what the changelog says. The problem is a different one: commit 36d2e48 (test(cleanup-timeouts): add tests ...) also reverts three merged changes that have nothing to do with this PR. I'd fix that before merge.
make ci-local with docker: 23 passed, 0 failed, 3 skipped. The changed-line coverage gate passed. schema_compat prints a warning for novelty.v1.json (see item 1).
Reproductions for the code findings through the DockerCLI fake, no real docker, claude or LLM.
The base's tests for the reverted features run against the PR head.
17 mutants of the cleanup code against test_cleanup_timeouts.py, test_container_agent.py and test_docker_solver_harness.py: 14 killed, 4 survive (see the inline comments and the last item).
Findings that cannot be anchored inline
The PR reverts #72 (conflict_coverage) and #84 (provider-aware judge default), and 36d2e48 hides it. The commit message names cleanup-timeout tests, but the commit changes 30 files and removes 556 lines. Both changes are on the PR's base (964c87f, d9f8c5a). Looks like a stale tree or a bad conflict resolution. Effects at this head:
evaluate --product x --judge-provider openai now pairs openai with claude-sonnet-5-5 (base: gpt-6.1-sol). --judge-provider google no longer exits with "has no default judge model; pass --judge-model". parse_args is gone from dokimi.cli; base's tests/test_judge_provider_default.py fails at collection with ImportError: cannot import name 'parse_args' from 'dokimi.cli'. The same applies to the 12 commands the #84 fragment lists. @copilot-pull-request-reviewer raised the CLI part; New evidence: the openai and google results above and the collection failure.
harmful_atoms reports a cleared check on an unfinished conflict scan. With zero harmful atoms and conflict_coverage={"pairs_compared": 0, "pairs_total": 45} the gate gives measured=True, passed=True, value=0 (base: measured=False, "compared 0 of 45 atom pairs"). Base's tests/test_novelty_gate.py run against the PR head: 14 failed, 35 passed. analyze is back to batches of 40 (about a third of the pairs at the default cap of 120) and no longer records conflict_coverage.
schemas/novelty.v1.json loses the conflict_coverage property with no version bump. Malformed values ({"pairs_compared": -1}, string counts) now validate: base's test_a_negative_or_string_pair_count_fails_the_schema fails with DID NOT RAISE ArtifactValidationError. ci-local prints schema_compat: property 'conflict_coverage' removed from an open object as a warning, not a failure.
The two changelog fragments (judge-provider-default, partial-conflict-not-cleared) are deleted, and docs/skill-novelty.md and docs/benchmarking-and-comparison.md go back to the old text ("chunks of 40", 33% of pairs).
The PR's own suites pass only because the tests for the reverted behaviour were deleted along with it.
Suggestion: rebase on the current base and drop every hunk outside the docker-solver change.
The #90 dependency bump is reverted too..agent-eval/Dockerfile:13 goes from uv==0.13.0 to uv==0.11.9, and dokimi/requirements.lock.txt:17 from anthropic==1.13.0 to anthropic==1.12.0. Both pins were last set by 6e695ce. requirements.txt has anthropic~=1.12, so nothing fails loudly. Same fix as item 1.
I agree with @copilot-pull-request-reviewer on the unhermetic test_cleanup_timeouts.py. New evidence: with ANTHROPIC_BASE_URL=http://localhost:4000 set by an autouse fixture, all 8 tests fail with ValueError: http://localhost:4000 is on the host's loopback from ContainerSandbox.open. Importing _fixture_task and _pipeline does not bring in the _hermetic fixture.
Checked, no issues
Earlier threads: the credential and plugin removals, the reward.txt copy and the reap each end the cell with reward 0 / rc 124 and test.sh does not run. The work tree, solver.jsonl and the cost survive a reap timeout (record_solver runs before the cleanup). Mutants that drop the note() calls, the raise stuck, the pulled.timed_out raise or the work copy are all killed.
Exception order: when the solve itself times out, the finally runs _stop_container_agent and the original TimeoutExpired wins.
Registry:create_harness(id, runner=...) works for every registered id (codex, claude-code, terminal-bench, docker-, benchflow, command). Every adapter has a runner field, so the unconditional dataclasses.replace is safe at this head.
Host path: a timed-out copy-back in docker-claude-host raises through check=True and test.sh does not run. The rm -f count of 3 in test_stop_proxy_timeout_retried_in_close is meaningful: dropping the stop_proxy call in close() fails it.
Docs: the changelog and coding-harnesses.md match the code for the timeouts that end a cell with reward 0, rc 124 (no metrics.error). Repo rules: SPDX headers, no generated artifacts, not stacked, no nav change needed.
A fourth surviving mutant: removing and stuck is None in note() (first timeout wins) passes all 137 tests. Only the argv in the raised exception changes, so I did not make it a finding.
The reason will be displayed to describe this comment to others. Learn more.
A timed-out docker logs of the sidecar leaves an empty egress record and the cell keeps its pass.
logs has check=False and timed_out is not read. self.log becomes parse_log(""), then the rm -f succeeds and _stopped is set, so close() never reads the log again. Through DockerCLI with docker-claude, answering the second logs call with reply(timed_out=True): returncode 0, reward 1.0, metrics["egress"] == {'allowed': [], 'denied': [], 'errors': [], 'denied_entries': []}, no error recorded. That reads as "the agent made no requests", which is what the sidecar's decision log exists to rule out. This predates the PR as check=False, but the 600 s budget is what makes a hang end instead of block.
Suggestion: treat a timed-out logs like the timed-out rm -f (raise TimeoutExpired, not stopped), or mark the egress summary as unknown.
The reason will be displayed to describe this comment to others. Learn more.
Not changed in this push, needs a decision: close() swallows a sidecar teardown timeout by design, so raising in stop_proxy alone would still pass the cell with an empty egress record. Options: fail the cell on a timed-out logs call, or mark the egress summary as unknown. Leaving this open.
The refactor branch dropped parse_args and JUDGE_MODEL_FOR_PROVIDER, so
--judge-provider openai picked the Claude default judge and failed at the
first call. Take main's cli, defaults and command registrations back;
none of them is part of the docker-* change.
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
An unknown image WORKDIR fell back to /app on a timeout, so the agent
ran in a tree the image never declared and the cell was billed. Raise
the timeout instead, and tighten the cleanup tests: the reap matcher
matches only kill -9 (rm -rf /opt/abskill contains kill), and close()
is asserted to save egress.jsonl and remove the network after a timed
out sidecar removal.
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
They start every child through dokimi.process now, so the spawn
baseline shrinks to nothing; a stale entry would fail the ratchet.
Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
This branch has not been deployed
No deployments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
antonwolfy
left a comment
Part of #4.
Follow-up to #29 and #79.
Summary
The
docker-*harnesses (docker-oracle,docker-claude[-host][-skill:],docker-probe[-host]) were the last spawn sites outsidedokimi/process.py. Now every docker step, bothclaudelaunches and both isolation probes go through the harness'srunner.create_harness(runner=)andTaskSuiteRunner(runner=)reach them as they reach the other harnesses.Output is kept whole (
capture_limit=None), in memory, assubprocess.runkept it. The runner's default 1 MiB head and tail would cutsolver.jsonl, the tool counts and the sidecar's decision log. Writing it to a spool file would put the stream on disk before it is redacted.Tests drive the pipeline through
DockerCLI(tests/docker_cli_fake.py), aFakeRunnerthat answers like a daemon with one healthy task container and stands in forclaude. They no longer patch_docker,subprocess.runor the private steps.Evidence
Tree kill, before/after. A stand-in
claude(a real process) starts a tool process that sleeps, then hangs past a 1.5 s budget:In the repo:
test_a_timed_out_host_agent_leaves_none_of_its_tools_runningruns it throughdocker-claude-host(cell rc 124, reward 0, tool gone).Budgets.
test_every_step_runs_with_a_budget[docker-oracle|docker-claude|docker-claude-host]checks every call'stimeout:docker buildbuild_timeout_secsolve.sh(oracle)claudehost / in containertest.shtest_solve_or_verify_timeout_is_a_failed_cell[solve.sh](reward 0, rc 124, partial output inrun.log) andtest_a_docker_step_without_a_budget_that_hangs_errors_the_cell(docker cp failed (timed out after 600s), env is the caller's).Whole stream.
test_a_stream_over_the_capture_limit_is_kept_whole: a 1.5 MiB stream with 3tool_useevents and a 1 MiB verifier log. With the runner's default capture:tool_calls == {}, so it fails. Now:{"Bash": 3},solver.jsonlis byte-identical and the log's middle is inrun.log.Fake CLI instead of patches. Across
test_docker_solver_harness,test_container_agent,test_agent_sandbox,test_docker_task_md,test_task_formatsandtest_skill_treatment: 92monkeypatches of_docker/subprocess.run/ private steps on main, 0 now.test_nothing_else_in_the_package_spawns_a_processfails on any newsubprocess/os.system/asynciospawn outsideprocess.py.test_a_hung_container_probe_is_untested_not_sealedcovers the probe's timeout.No real CLI ran in tests. The suite ran under a spawn guard. It is a pytest plugin that raises on any spawn that resolves to an installed
claude/codex/harbor/bench/docker, plus PATH shims for grandchildren. Two random seeds:5187 passed, 2 skippedeach,blocked=0, 0 shim hits. The timing-sensitive files (process runner, sandbox, container agent, docker solver, task.md; 313 tests) ran 3x in parallel: all passed.Local CI.
make ci-local --offline: 14 passed, 0 failed (ruff, mypy strict with no new exemption, tests, schema, docs build, changelog fragment). Coverage of changed lines: 100% (73 lines).CI. All checks pass, including Tests on py3.11 to 3.14 and the external catalog smoke test, which runs a real uxlfoundation/skills task through
docker-oracleanddocker-probeagainst Docker.Merge Danger
Door: two-way. No schema, artifact field or persisted format changes. Reverting restores the old spawn code.
Blast Radius: task-cells
docker-oracle: asolve.shthat overruns the agent budget (agent.timeout_sec, else 600 s) is now a failed cell, reward 0, rc 124, and the verifier does not run. Before, it waited with no limit. Oracle pass rates can drop for tasks whose oracle is slower than their own agent budget.mkdir,chmod,docker run -d, the sidecar'screate, which may pull its pinned image) errors the cell:docker <step> failed (timed out after 600s).rm -fandnetwork rmare ignored, the recompile lands inrecompile_failed, and the reward-file copy falls back to the verifier's exit code.run.log. Before, only theTIMEOUTline was written.docker-claude-host: a timeout kills claude's whole process group, and so does its exit. A background process the agent started no longer outlives its cell. Ctrl-C, or SIGTERM to thedokimiCLI, kills a running docker step's or claude's group.dockerandclaudeoutput is decoded as UTF-8 with replacement. Before, it was decoded in the locale's encoding.DockerSolverHarnesshas arunnerfield, andContainerSandbox.opentakesrunner=.solver_stream.run_solvertakes keyword-onlytimeout/env/cwd/runnerand returnsprocess.Completed, whichrecord_solverandcontainer_agent.Dockernow use too.--internalnetwork with egress only through the sidecar to the LLM hosts.--env-file, never on argv. The docker client gets exactly its old env (Env.only((), client_env)).workspace.env(...), the allow-list. The docker CLI keeps the caller's env (Env.inherit)./testsis copied in. The verifier's proxy credentials are still stripped, and the proxy is still stopped before a public task joins bridge.run.logandsolver.jsonlare redacted before they are written. Output is never spooled raw to disk.untestedand reward 0, with the same 120 s and 60 s.DockerCLI.Checklist
pytestpassesruff,mypy, and schema checks pass.github/.licenserc.yaml)🤖 Generated with Claude Code