Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,20 @@ new version heading in the same commit.

## [Unreleased]

## [0.448.2] - 2026-09-18
### Fixed
- **The waiting note no longer teaches a pattern the harness blocks.** It told every agent to wait by
polling with `sleep 5` in a bounded loop, and argued for it from a ~5-minute prompt-cache TTL. Both
premises are gone: claude-code now REFUSES a foreground `sleep` (`Blocked: sleep …`, adding "do not
chain shorter sleeps to work around this block") and cache writes on these runs are essentially all
1-hour entries (measured: instapods 8,793 `ephemeral_1h` vs 0 `ephemeral_5m`; expresstech ~20k vs 12),
so the "one long wait costs more than twenty short polls" argument no longer holds. The note now points
at the two supported waits — `Bash` with `run_in_background: true` on a command that exits when the
condition holds (read with `TaskOutput`), and `Monitor` with a bounded `timeout_ms` and a filter that
also matches failures — and keeps what still stands: the ~2-minute foreground cut-off, "make each wait
earn its turn", and "this is not an argument to batch less". The old TTL reasoning is recorded in the
source comment so it is not reinstated from memory.

## [0.448.1] - 2026-09-18
### Fixed
- **`Monitor` was an ungoverned shell.** claude-code's `Monitor` tool runs an arbitrary shell command
Expand Down
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "agent-os",
"version": "0.448.1",
"version": "0.448.2",
"description": "A generic, governed operating system for running autonomous agents safely across brands. Ships with a local web console.",
"license": "MIT",
"type": "commonjs",
Expand Down
5 changes: 4 additions & 1 deletion scripts/context-injection-test.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,10 @@ async function main() {
// snake_case, or a known bare-word tool. Anything else is prose and is skipped deliberately.
const cited = [...new Set((prose.match(/`([a-z][a-z0-9_]{2,})`/g) || []).map((x) => x.slice(1, -1)))]
.filter((x) => x.includes('_') || ['recall', 'remember', 'revise', 'forget', 'report', 'update', 'ask', 'publish', 'notify', 'schedule', 'unschedule', 'stop'].includes(x));
const unknown = cited.filter((c) => !alwaysNames.includes(c) && !eNames.includes(c) && !rNames.includes(c));
// Backticked snake_case that is a harness TOOL FIELD, not an OS tool. Kept explicit (and short) so
// the scanner stays strict: a typo'd tool name must still fail here.
const FIELDS = ['timeout_ms', 'run_in_background', 'blocked_on', 'claude_session_id'];
const unknown = cited.filter((c) => !alwaysNames.includes(c) && !eNames.includes(c) && !rNames.includes(c) && !FIELDS.includes(c));
assert(unknown.length === 0, 'every tool named in the prompt is actually exposed', unknown.join(', '));
}

Expand Down
20 changes: 12 additions & 8 deletions scripts/waiting-brief-test.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -27,20 +27,24 @@ const check = (name, cond) => { if (cond) pass++; else failures.push(name); };
const B = String(WAITING_BRIEF || '');

check('the brief exists and is substantial', B.length > 800);
check('names the ~2 minute Bash kill', /2 minutes?/.test(B) && /kill/i.test(B));
check('names sleep 240 as the concrete loss', /sleep 240/.test(B));
check('says an until/while loop is not the fix either', /until/.test(B) && /while/.test(B));
check('names the ~5 minute cache TTL', /5 minutes?/.test(B) && /cach/i.test(B));
check('explains that one long wait can cost MORE than many short polls', /MORE/.test(B));
check('gives a concrete upper bound on a single wait', /60-90 seconds|60–90 seconds/.test(B));
check('shows a bounded early-exit poll, not a bare sleep', /break/.test(B) && /seq 1/.test(B));
check('tells the agent to do work between polls', /earn its turn/i.test(B));
check('names the ~2 minute foreground Bash cut-off', /2 minutes?/.test(B) && /(kill|cut off)/i.test(B));
check('names the harness sleep BLOCK (the rule that replaced the 2-min-loss framing)', /Blocked: sleep/.test(B));
check('says a sleep-only until/while loop is not the fix either', /until/.test(B) && /while/.test(B));
// The 5-minute cache TTL was the brief's central claim until v0.447.3 and is FALSE for these runs
// (measured: ~100% `ephemeral_1h` cache writes across three tenants). Assert it does not come back —
// an argument from a cliff that isn't there teaches agents to avoid waits that now cost nothing extra.
check('does NOT argue from a 5-minute prompt-cache TTL', !/5[- ]minute/i.test(B) && !/cache/i.test(B));
check('names run_in_background as the one-notification wait', /run_in_background/.test(B));
check('names Monitor as the per-occurrence wait, bounded', /Monitor/.test(B) && /timeout_ms/.test(B));
check('shows a condition command that EXITS, not a bare sleep', /until grep/.test(B) && /run_in_background: true/.test(B));
check('tells the agent to do work while it waits', /earn its turn/i.test(B));

// The property most likely to be lost in a later edit: this must not contradict the batching advice
// that produced the largest measured win.
check('explicitly does NOT read as "batch less"', /batch less/i.test(B));
check('reaffirms pushing loops into scripts', /300/.test(B) && /still right|still the largest/i.test(B));
check('scopes the rule to a SINGLE call, not to batching', /SINGLE/.test(B));
check('says the wait commands are themselves governed', /governed/.test(B));

// Lane independence: it must not be phrased as an unattended-only rule, and must not duplicate the
// turn-boundary framing that already misses the interactive case.
Expand Down
76 changes: 42 additions & 34 deletions src/edge/background-work.ts
Original file line number Diff line number Diff line change
Expand Up @@ -169,47 +169,55 @@ export function pendingBackgroundWork(transcript: string | undefined): PendingBa
* legitimate and unavoidable (a CDN rule propagating, a 300-domain sweep, a build) — the question is
* how, not whether.
*
* Both numbers here are enforced by something outside the agent's control, which is why they are
* stated as limits rather than advice:
* The limits here are enforced by the harness, not by taste, which is why they are stated as rules:
*
* - A `Bash` tool call is killed at ~2 minutes. A `sleep 240` does not wait 4 minutes; it loses the
* whole call. Measured: cost a watchdog run 120 s for nothing, and the agent's workaround was an
* `until` loop, which dodges the timeout while keeping every second of the wall clock.
* - Prompt caching has a ~5-minute TTL. A single wait past it forces a full re-write of the run's
* context. Measured across three watchdog runs: consolidating into one 600 s wait cut tool calls
* 40 -> 29 and RAISED cost $8.37 -> $11.83, with cache_write tripling 0.27M -> 0.73M. Wall clock
* did not improve either (862.7 s -> 892.5 s).
* - A foreground `sleep` is REFUSED by the Bash tool ("Blocked: sleep N followed by …"), whose own
* message adds "Do not chain shorter sleeps to work around this block" and points at
* `run_in_background` / `Monitor`. Fleet transcripts show agents hitting that block repeatedly
* while this brief was still telling them to poll with `sleep 5`.
* - A foreground `Bash` call ends at ~2 minutes — killed on older builds, auto-backgrounded on newer
* ones. Both shapes appear in live transcripts.
*
* That second measurement is the reason this brief exists and the reason it has to be explicit. "Fewer
* turns is cheaper" is true right up to the cache TTL and false past it, and an agent optimising turn
* count in good faith will sail straight through the crossover. Telling it to poll without telling it
* why reads as a contradiction of the batching advice it was already given.
* HISTORY, so the old argument is not reinstated from memory: until v0.447.3 this brief's central claim
* was a ~5-minute prompt-cache TTL, with a measurement behind it (one 600 s wait cut tool calls 40 -> 29
* yet RAISED cost $8.37 -> $11.83, cache_write 0.27M -> 0.73M). That premise no longer holds for these
* runs: cache writes are now essentially all 1-hour entries — instapods 8,793 `ephemeral_1h` vs 0
* `ephemeral_5m` over ~8.8k responses, expresstech ~20k vs 12, instawp all 1h — so a wait of a few
* minutes no longer re-writes the context at full price, and an argument built on the 5-minute cliff was
* telling agents something false. The reason not to idle is now the harness limits above plus the plain
* opportunity cost of a turn spent sleeping. If a future release changes the TTL again, re-measure from
* `ephemeral_*_input_tokens` in the transcripts rather than restating either number.
*/
export const WAITING_BRIEF =
'# Waiting: short polls, never one long sleep\n\n' +
'# Waiting: hand the wait to the harness, never to a sleep\n\n' +
'Some work genuinely takes minutes and is not yours to speed up — a rule propagating to an edge, a ' +
'sweep over hundreds of hosts, a build. Waiting for it is fine. Waiting for it in ONE long call is ' +
'not, and two hard limits decide that for you:\n\n' +
'- **A `Bash` call is killed at about 2 minutes.** `sleep 240` does not wait four minutes — it loses ' +
'the entire call and everything after it in that command. An `until`/`while` loop evades the kill but ' +
'keeps the full wall clock, so it is not the fix either.\n' +
'- **Prompt caching expires after about 5 minutes.** One wait longer than that re-writes your whole ' +
'context at full price. This is why a single 10-minute wait can cost MORE than twenty short polls ' +
'despite being far fewer turns.\n\n' +
'**So: no single wait longer than ~60-90 seconds.** Start the slow thing in the background, then poll ' +
'in short bounded steps that exit as soon as it is done:\n\n' +
'sweep over hundreds of hosts, a build, CI. Waiting for it is fine. Waiting for it by sitting in a ' +
'`Bash` call is not, and the harness enforces that for you:\n\n' +
'- **A foreground `sleep` is BLOCKED.** The tool refuses it (`Blocked: sleep …`) and says so in the ' +
'same words this note does: do not chain shorter sleeps to get around it. A `while`/`until` loop that ' +
'only sleeps is the same idling with extra steps.\n' +
'- **A foreground `Bash` call is cut off at about 2 minutes** — killed, or moved to the background ' +
'mid-way. Either way the wait is not where your work should be.\n\n' +
'**The two supported ways to wait**, both of which let you keep working (or stop) while it runs:\n\n' +
'- **One notification, when a condition comes true** — `Bash` with `run_in_background: true` and a ' +
'command that EXITS once it holds. You are told when it exits, and `TaskOutput` reads what it printed. ' +
'No trailing `&`.\n' +
'```bash\n' +
'long-running-thing & \n' +
'# ...then, once per turn:\n' +
'for i in $(seq 1 12); do [ -f "$DONE_SENTINEL" ] && break; sleep 5; done\n' +
'```\n\n' +
'**Make each poll earn its turn.** A poll that only sleeps converts model time into wall time and ' +
'saves nothing. Read partial results, write the rows that are ready, update the sheet or the ticket, ' +
'check the log for the failure you can already act on. If a run of yours is mostly `sleep`, the work ' +
'was serialised behind a wait that did not need to block it.\n\n' +
'# run_in_background: true\n' +
'until grep -q "Ready in" dev.log; do sleep 0.5; done\n' +
'```\n' +
'- **One notification per occurrence** (each new CI check, each ERROR line) — `Monitor`, with a ' +
'bounded `timeout_ms` and a filter that matches the FAILURE signatures too, not just the happy path: ' +
'a monitor that greps only for success is silent through a crash, and silence reads as "still ' +
'running". Re-arm it if it expires while you still care. Both of these run real commands, so they are ' +
'governed like any other shell call.\n\n' +
'**Make each wait earn its turn.** A poll that only sleeps converts model time into wall time and ' +
'saves nothing. While the slow thing runs, read partial results, write the rows that are ready, update ' +
'the sheet or the ticket, act on the failure already in the log. If a run of yours is mostly waiting, ' +
'the work was serialised behind something that did not need to block it.\n\n' +
'**Do not read this as "batch less".** Pushing a loop over 300 items into one script instead of 300 ' +
'turns is still right, and still the largest win available. The rule is narrower: do not let any ' +
'SINGLE call sit idle past the limits above. Batch the work; poll the wait.';
'turns is still right, and still the largest win available. The rule is narrower: no SINGLE call sits ' +
'idle waiting. Batch the work; let the harness watch the wait.';

export const UNATTENDED_TURN_BRIEF =
'# This run ends when your turn ends\n\n' +
Expand Down
Loading