_history grows every round and is re-sent on every LLM call of every subsequent
round. Worse, tool output goes in raw — loop.py:580:
self._note(ctx, f"{approved}. Result: {output}")
A Zendesk ticket dump or a Stripe charge object is several KB, it never leaves, and
it's paid for on every call thereafter. Cost grows roughly with
rounds × payload × calls-per-round.
This is a cost bug today, before any long-running-task story.
Would adopting a harness solve it?
No — and the reason is worth stating, because it looks like it should.
deepagents, DeepSeek Harness and friends all ship context management. But their
compaction operates inside one session, and a Charter session is one round.
Every gate, every rejection, every approved action ends the round and the harness's
state goes with it. We'd rebuild the prompt from a growing _history regardless.
So it splits cleanly:
| context |
who can compact it |
why |
within a round — the message list in run_step |
a harness |
it owns that loop |
across rounds — _history |
only Charter |
no harness knows this structure exists |
Charter's is the half that actually grows without bound, because a task can span
hours and a human. A harness can't help with it at any price, and the
harness seam doesn't
change that.
(It does become relevant the other way around: if we ever persist harness
checkpoints across rounds, harness-side compaction starts applying to our timeline
too. That's the pluggable-checkpointing idea, and it's downstream of this.)
What to do
1. Stop putting raw tool output in history. The round immediately after a call
needs the full result; every round after that needs to know it happened. Keep it
verbatim once, then reduce to a line. Cheapest fix, biggest win, no model calls.
2. Compact old history. Past some number of lines, summarise the oldest into
one entry and keep the recent ones intact. Needs a model call, so it needs a budget
and a decision about whose budget it comes out of — probably the task's, which
means it must be cheap enough not to eat the thing it's protecting.
3. Make the cost visible. charter validate should report the assembled prompt
size, and charter status should show input tokens alongside cost. Right now
nothing tells you your history is the reason a task costs what it does.
Do (1) first and measure. It may be most of the problem — a summarised tool result
is usually an order of magnitude smaller than the raw one, and unlike (2) it costs
nothing to produce.
Related
- #2 — instruction files ride on
every call too, which tightens the same budget.
- The
memory.from_audit block also prepends to every round; it's capped by count
today but not by size.
_historygrows every round and is re-sent on every LLM call of every subsequentround. Worse, tool output goes in raw —
loop.py:580:A Zendesk ticket dump or a Stripe charge object is several KB, it never leaves, and
it's paid for on every call thereafter. Cost grows roughly with
rounds × payload × calls-per-round.This is a cost bug today, before any long-running-task story.
Would adopting a harness solve it?
No — and the reason is worth stating, because it looks like it should.
deepagents, DeepSeek Harness and friends all ship context management. But their
compaction operates inside one session, and a Charter session is one round.
Every gate, every rejection, every approved action ends the round and the harness's
state goes with it. We'd rebuild the prompt from a growing
_historyregardless.So it splits cleanly:
run_step_historyCharter's is the half that actually grows without bound, because a task can span
hours and a human. A harness can't help with it at any price, and the
harness seam doesn't
change that.
(It does become relevant the other way around: if we ever persist harness
checkpoints across rounds, harness-side compaction starts applying to our timeline
too. That's the pluggable-checkpointing idea, and it's downstream of this.)
What to do
1. Stop putting raw tool output in history. The round immediately after a call
needs the full result; every round after that needs to know it happened. Keep it
verbatim once, then reduce to a line. Cheapest fix, biggest win, no model calls.
2. Compact old history. Past some number of lines, summarise the oldest into
one entry and keep the recent ones intact. Needs a model call, so it needs a budget
and a decision about whose budget it comes out of — probably the task's, which
means it must be cheap enough not to eat the thing it's protecting.
3. Make the cost visible.
charter validateshould report the assembled promptsize, and
charter statusshould show input tokens alongside cost. Right nownothing tells you your history is the reason a task costs what it does.
Do (1) first and measure. It may be most of the problem — a summarised tool result
is usually an order of magnitude smaller than the raw one, and unlike (2) it costs
nothing to produce.
Related
every call too, which tightens the same budget.
memory.from_auditblock also prepends to every round; it's capped by counttoday but not by size.