Skip to content

Compact task history — it grows every round and carries raw tool output #3

Description

@arjunvlama

_history grows every round and is re-sent on every LLM call of every subsequent
round. Worse, tool output goes in raw — loop.py:580:

self._note(ctx, f"{approved}. Result: {output}")

A Zendesk ticket dump or a Stripe charge object is several KB, it never leaves, and
it's paid for on every call thereafter. Cost grows roughly with
rounds × payload × calls-per-round.

This is a cost bug today, before any long-running-task story.

Would adopting a harness solve it?

No — and the reason is worth stating, because it looks like it should.

deepagents, DeepSeek Harness and friends all ship context management. But their
compaction operates inside one session, and a Charter session is one round.
Every gate, every rejection, every approved action ends the round and the harness's
state goes with it. We'd rebuild the prompt from a growing _history regardless.

So it splits cleanly:

context who can compact it why
within a round — the message list in run_step a harness it owns that loop
across rounds — _history only Charter no harness knows this structure exists

Charter's is the half that actually grows without bound, because a task can span
hours and a human. A harness can't help with it at any price, and the
harness seam doesn't
change that.

(It does become relevant the other way around: if we ever persist harness
checkpoints across rounds, harness-side compaction starts applying to our timeline
too. That's the pluggable-checkpointing idea, and it's downstream of this.)

What to do

1. Stop putting raw tool output in history. The round immediately after a call
needs the full result; every round after that needs to know it happened. Keep it
verbatim once, then reduce to a line. Cheapest fix, biggest win, no model calls.

2. Compact old history. Past some number of lines, summarise the oldest into
one entry and keep the recent ones intact. Needs a model call, so it needs a budget
and a decision about whose budget it comes out of — probably the task's, which
means it must be cheap enough not to eat the thing it's protecting.

3. Make the cost visible. charter validate should report the assembled prompt
size, and charter status should show input tokens alongside cost. Right now
nothing tells you your history is the reason a task costs what it does.

Do (1) first and measure. It may be most of the problem — a summarised tool result
is usually an order of magnitude smaller than the raw one, and unlike (2) it costs
nothing to produce.

Related

  • #2 — instruction files ride on
    every call too, which tightens the same budget.
  • The memory.from_audit block also prepends to every round; it's capped by count
    today but not by size.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions