fix: the agent loop screens what a shell runs, and stops when the money does - #217
Merged
Conversation
…ey does
Eight wiring bugs in this adapter were found by live runs. A ninth arrived the
same way: a real rdpapp item came back `LimitsExceeded` after 40 model calls
with 15 of the 40 turns refused by `CommandGuard`. The loop was working — it
created files — and it ran out of steps being told no.
`_segments` split a shell line by replacing its separators, which is not a
shell. Two consequences, opposite in direction and both real:
* a heredoc *body* was chopped into argv and screened as commands, so
writing a source file — the most common thing a coding agent does — was
refused for containing `&&`, `|`, or a bare `/` (division resolves to `/`
and trips path confinement). Over the shapes of that run: 6 of 9
file-writes refused before, 0 after;
* and it could not see `$(...)`, backticks, `env`/`xargs`/`nohup`/`timeout`,
`find -exec` or a subshell, so `echo $(rm -rf .)` ran with a refusal list
naming `rm`. 3 of 7 genuinely dangerous lines refused before, 7 of 7 after.
The guard is not widened. A heredoc fed to an interpreter is still screened
line by line; a redirection outside the worktree is still refused — the
refusal now names the tree so the next turn can comply instead of guessing.
Six more, each with a test that fails before it:
* `HarnessModel.cost` was never written to, so the loop's own spend ceiling
could not fire and a per-item `Budget` reached nothing that runs long. Cost
comes from `usage_for`, the same reading the audit uses; unpriced calls
stay unpriced, so the ceiling is a lower bound and never fires early.
* `subprocess.TimeoutExpired` was unhandled: one hung command ended the item
with a traceback rather than a turn.
* Output truncation kept the last 32k, unmarked — throwing away the head of
every compiler error, and the finish marker itself when the agent said it
was done and then printed anything.
* The assistant's own turns reached the wire empty, because the action lived
only in the `tool_calls` this layer strips: thirty outputs, no way to tell
which command produced which.
* A gateway answering HTTP 200 with an error body was reported as the model
failing to format an action, three calls later.
* And the correction handed back when no tool call arrives was the *text*
protocol's — "provide EXACTLY ONE action in triple backticks" — which is
bug #8 surviving in the one message that only appears when things have
already gone wrong. It, and the observation template, now come from the
same config the prompts do, so they cannot drift apart again.
A refusal is also recorded now: the rule that fired, in the trajectory, and
through an optional callback a deployment can point at its event stream.
Fifteen refusals in a real run existed only in a list that died with it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Aug 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Eight wiring bugs in
adapters/minisweagent.pywere found by live runs, severalat ten minutes and real money each. A ninth arrived the same way, and this PR is
that one plus six found without a network.
The measured one
A real rdpapp item:
status: LimitsExceeded, 40 model calls, 15 of the 40turns refused by
CommandGuard. The loop was working — it created files — andit ran out of steps being told no.
_segmentssplit a shell line by replacing&&,||,;,|and newlineswith a delimiter and calling
shlex.spliton the pieces. That is not a shell,and it was wrong in both directions at once.
False positives — the expensive ones. A heredoc body was chopped into argv
and screened as commands, so writing a source file was refused for containing
&&,|, or a bare/. That last one is the dominant mechanism and it isworth stating plainly:
let mean = total / count;tokenises to a/, whichresolves to the filesystem root, which is outside the worktree. Ordinary
arithmetic was reaching outside the tree.
False negatives. The same splitter could not see
$(...), backticks,env/xargs/nohup/timeout,find -exec, or a subshell — soecho $(rm -rf .)andenv rm -rf .both ran with a refusal list namingrm.Measured over a corpus reconstructed from the shapes in that run (the 15
commands themselves were not in the report, so this is the shapes, not the log):
The guard is not widened to get there. A heredoc fed to an interpreter
(
bash <<EOF) is still screened line by line; only a body going into a file istreated as data. A redirection outside the worktree is still refused — the
refusal message now names the tree, states that the rule is permanent, and
counts the refusals so far, so the next turn can comply rather than retry
variants. That is message quality, not policy.
Six more, each with a test that fails before the fix
HarnessModel.costwas never written to. The loop reads a per-call costoff
extraand stops itself atcost_limit; nothing ever put one there, socostwas0.0for the life of every run and the only bound on an agent wasits step count. The cost now comes from
client.usage_for, the same readingthe audit rollup uses, and
build()takes aBudget— so an item's declaredwall-clock and spend ceilings reach the one stage long enough to need them.
budgets.py's rule is kept: an unpriced call is counted as unpriced, not asfree, so the ceiling is a lower bound that can fail to fire and can never fire
early.
subprocess.TimeoutExpiredwas unhandled. It leftexecute, and the loopdoes not catch it:
DefaultAgent.runrecords an exit message and re-raises.One hung test command ended the whole item with a traceback and no submission.
The agent is now told, as return code 124 with
exception_info, and pickssomething cheaper.
command — a compiler puts its first error at the top, a test runner puts its
summary at the bottom — so both ends are kept with a count of what went. And
the submit marker is looked for in the whole output: a command that said it
had finished and then printed anything sizeable had its own marker truncated
away, which is bug D5: GUI stack #5 reached by a different road.
tool_calls, which_for_the_wirestrips (correctly — atoolreply needsthe id it strips). Nothing replaced it, so a model replying with a tool call
and no prose saw thirty of its own turns as empty strings and could not tell
which output answered which command. The command is folded back into the
assistant's content.
_message_ofswallowedevery exception and returned an empty message, which is indistinguishable from
a model that would not format an action: the loop asked again, got the same
error, and ended as
RepeatedFormatErrorafter three calls against a brokenendpoint. It now says what arrived, redacted, on the first call.
model on the tool-call path to "provide EXACTLY ONE action in triple
backticks" and discarded
{{error}}— bug D8: Gate plugin interface #8 surviving in the one messagethat only appears when things have already gone wrong. It and the observation
template now come from the same config the prompts do, so they cannot drift
apart again.
finish_reasonis passed with it, because their template uses itto tell a format mistake from a token-limit truncation.
A refusal is also durable now: the rule that fired goes into the trajectory, and
an optional
on_refusalcallback lets a deployment point it at an event streamwithout core learning this adapter's name. Fifteen refusals in a real run
existed only in a list that died with the process.
What was checked and found sound
_for_the_wiredroppingtool_callsdoes not orphan atool_call_id—no
toolrole is ever sent, so no provider is left waiting for one. The bugwas the lost content, not the lost pairing.
echo x > /etc/passwd) was alreadyrefused, by the path boundary, incidentally and correctly.
VAR=x cmdstripping, the unparseable-line fallback,2>&1, and theworktree-relative path checks all behave as documented.
semantics (answer, don't terminate) are deliberate and unchanged.
Verification
uv run ruff format .,uv run ruff check .,uv run mypy .,uv run pytestall pass. Every new test was run against the pre-fix adapter and fails there.
🤖 Generated with Claude Code