fix: eight things that stopped a second repository getting through the loop - #151
Merged
Merged
Conversation
This was referenced Aug 4, 2026
Open
thedancingdeveloper
changed the base branch from
fix/validator-rejects-valid-patches
to
main
August 4, 2026 23:03
Found importing rdpapp, whose gateway is one 612 KB source file. The planner named it correctly; the 60,000-character context budget could not hold it; the selector recorded "named target exceeds remaining content budget" and moved on, and the fallback then filled the whole budget with unrelated files — migration SQL and a vendored patch directory. The implementer was called anyway and asked to change a file it had never seen. It would have answered, because a model short of evidence writes a plausible diff rather than refusing, and the failure would have surfaced as a patch that did not apply. Two changes, and they are separate ideas: - A named target that cannot be supplied now stops the item BEFORE the implementer is called: escalated / context_unavailable, blocked, and costing no attempt, because no attempt could succeed and retrying will not make the file smaller. The message names the file, its size, the budget and the flag. - The budget is configurable at last — --context-budget and $HARNESS_CONTEXT_BUDGET. It was a constant, so a repository with large files could not be worked on at all, and no single number is right for every project. The omission reasons become constants, because the executor now has to tell the two apart rather than only print them. Also corrects the reason-kind list in the API schema and USAGE, which had fallen four kinds behind the taxonomy. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The reviewer's prompt has always carried `Checks: {checks}`. The
implementer's carried nothing, so the model writing the change was graded by
gates it was never shown.
Measured on rdpapp. The implementer produced exactly the right change — the
validation function in the right module, wired into main before serving, with
a test for the rejection and a test for the case that must keep working — and
`cargo fmt --all -- --check` refused it over line wrapping. That is one
attempt and two model calls spent discovering something the harness knew
before it asked.
Naming the commands does not weaken the gate: it still runs, and it still
refuses. It stops the gate being a secret.
Empty when a project configured no checks, so a run without them reads exactly
as it did before — asserted, because a prompt that grows a stray blank section
is how a fix like this quietly changes every other run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The reviewer is asked whether a change is wired in where it should be, and whether anything unrelated moved, and was given only the change. Measured on rdpapp. A correct diff — the right helper, called from main before serving, with tests for both the rejection and the case that must keep working — was rejected, and two of the three reasons were of this form: "The diff does not show whether main's returned error is surfaced in the intended way, whether the rest of startup avoids side effects before this check … or that there are no other startup routes bypassing the check." No diff can show any of that. And the prompt lists "the task cannot be judged from what you were given" as grounds to reject — so a diff-only reviewer rejects for the shape of its own prompt rather than for a defect in the work. Meanwhile the implementer had been shown 700,000 characters of the same repository. The touched files now follow the diff, at their post-change state, bounded by the same context budget. A file too large to include is named as absent: a reviewer that does not know its view is partial will treat it as complete, which is worse than one that knows. This strengthens the gate rather than weakening it. The reviewer's answer stays its own, and nothing about what it may reject has changed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A retry re-planned from the brief with no memory of what had just happened, so it repeated the same mistake blind. Measured on rdpapp: three of four attempts at one item were refused by `cargo fmt --all -- --check`, each one unaware the previous had been, and each costing a planner call and an implementer call against a 612 KB file. The material was already there. `requeue` deliberately keeps `last_error` — "it is the only record of why the item failed" — and `WorkRecord` carries it to the executor. Nothing passed it to the model doing the work. **This is not a resumption**, and the prompt says so in as many words. D11 and the attempt machinery are untouched: a decided attempt still ends resumability, a retry still re-plans against the current brief, and nothing here is treated as progress. What changes is only that the new attempt knows what the last one was refused for, which is what a person retrying would read first. Bounded at 4,000 characters, because a check's output can be a whole build log and the useful part is where the tool says what it objected to. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…n scope Three of the four fixes on this branch change what a model is given, which is what it is judged on. That belongs in USAGE next to the other guarantees rather than only in the commits that made it true. The last paragraph is the one that cost the most to learn and is not a harness behaviour at all: a brief that does not say what is out of scope collects rejections that are each individually reasonable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Supplying the touched files was half the fix. The prompt still framed the answer as "anything the diff claims that the diff alone does not show", so the reviewer went on writing that section as though the diff were all it had. Measured after the previous commit landed. The reviewer received 637,317 characters — the whole of the 612 KB file the change touched — and rejected with: "I cannot verify there isn't another snippet read/query path elsewhere that should also have been extended … I cannot verify that the new row type is fully compatible with all existing callers; the diff does not show the surrounding type signatures." Every one of those was answerable from the file it was holding. So section 2 now says *what you were given* rather than *the diff alone*, and says plainly that the files are there and must be read before claiming something could not be verified. The reject-if list keeps "the task cannot be judged from what you were given" — it is a real ground and it should stay — with the two qualifications that make it honest: after looking, and not for evidence the task never asked for. Nothing about the reviewer's authority changed. It may still reject anything; it is asked to do so on what is in front of it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The planner's entire job is to name the files an item will change. It was given the brief and nothing else, and told — correctly — not to invent a path. So it could only ever name a path the brief had already quoted. That held up for three items whose briefs quoted paths, and broke on the first one that described a surface instead: "I do not have the repository tree or actual file paths, so naming …" with an empty targets list, which is the honest answer to an impossible question. Target-first context selection then had no target, the implementer received relevance-guessed files — a terminal adapter and a vendored Kerberos patch — and wrote a diff against a stylesheet it had not been shown. It did not apply. The tracked paths now follow the brief. Paths only: the planner needs to know what exists, and reading it is the implementer's job, which has its own budget. A listing that will not fit says how many paths are missing rather than stopping silently at an arbitrary letter of the alphabet. Bounded by the same context budget, so a repository with a very large tree cannot turn the cheapest call in the pipeline into the most expensive one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The guard added earlier blocked an item when any named target did not fit. On a repository with more than one large file that is too strict, and it cost a real item: FAIL R3: the planner's target(s) src/db.rs (144480 bytes), docs/rdpappv2/current-state.md (21829 bytes) do not fit … The file the change actually belonged in — 634 KB — had been supplied. What did not fit was a supporting file and a design document, neither of which the item was going to edit. Blocking there loses the work to protect a file nobody was touching. The planner is asked to order its targets by importance, so the first usable one is the file the work is in. Losing that is still fatal and still escalates, for the original reason: the implementer would be asked to change a file it has not been shown, and it will answer rather than refuse. Losing a supporting file is not fatal — it is named in the prompt, with an instruction not to change it and to say so if the task cannot be done without it. Which keeps the property that mattered: nothing is ever silently asked to edit a file it has not read. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An item that added two constants to one 5.6 KB file was rejected. The reviewer's own section 1 confirmed the change was correct, correctly named, correctly grouped and within scope. It rejected anyway, for two reasons: "The diff does not show whether web/src/main.tsx remains unchanged in the actual working tree beyond what was provided here." "I cannot verify that the file still type-checks, because no build output was provided beyond the claim 'Checks: passed.'" Both are answerable, and the harness was the one withholding the answers. A diff produced from the repository IS the whole change — a file absent from it is unchanged, and that is a property of how the diff was made, not a claim by whoever wrote the code. The prompt now says so. And `Checks: passed` reads exactly like the author asserting their own work is fine, which a reviewer told to assume the work is wrong will discount — as this one did, in as many words. The harness ran those commands, on that tree, after the change and before the reviewer. The prompt now names them and says who ran them. A project with no checks is told that too, plainly, rather than left to infer from silence that there were some. Neither is a softening. Both remove uncertainty that was never real, so the reviewer can spend its scepticism on the work. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An item declared two constants in a contract file and said, in a paragraph headed "Scope, stated so it can be judged", that replacing the corresponding literals in a 272 KB file was deliberately out of scope and why. The reviewer verified every stated requirement — the right file, the right names, the right values, the right grouping, no other file touched, the typecheck passing — quoted the exclusion back in its own answer, and rejected: "the broader problem statement ties those constants to repeated 14 literals in web/src/main.tsx, and that code remains unchanged, so the repository still has the inconsistency the task described … the implementation itself is fine" That is the reviewer overruling the plan author about what the item was for. The gate exists to catch work that does something adjacent to what was asked, or claims more than it did; rejecting work for doing exactly what was asked is the same failure inverted, and it is more expensive, because the answer is to write the item again identically. So the prompt gains its first "do not reject if": where a task states what is out of scope, judge the change against that. Disagreeing with the scope is a note for a person, and a note is the right size for it. Nothing else about the reviewer's authority moves. It may still reject work that is wrong, incomplete against what the task did ask, or overreaching. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
thedancingdeveloper
force-pushed
the
fix/context-budget-and-starved-target
branch
from
August 4, 2026 23:07
ed7e195 to
5e2aa6e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Eight fixes from one exercise: importing
rdpapp— a Rust/TypeScript productwith a 634 KB gateway file and a 272 KB browser entry point — and driving items
through the loop until they landed. Closes #150, #154, #156, #157, #159.
Based on
fix/validator-rejects-valid-patches(#145), notmain, because thecode these change only exists there.
Every one was found by watching a real item fail. Two items now go end to
end that could not before.
Context: what each role could see
1. The planner had never seen the repository (the most consequential).
Its entire job is naming the files an item will change, and it was given the
brief and nothing else — so it could only name a path the brief had already
quoted. The first item that described a surface rather than a path replied
"I do not have the repository tree or actual file paths" with an empty target
list, and the implementer then wrote a diff against a stylesheet it had not
been shown. It now gets the tracked paths, bounded by the same budget.
2. A target too large to supply was dropped silently (#150). The
implementer was asked to change a file it had never seen. Now the item stops
before that call is paid for —
escalated/context_unavailable,blocked,costing no attempt — and
--context-budget/$HARNESS_CONTEXT_BUDGETexist,because the budget was a constant with a
context_policynothing ever passed.3. …but only a missing primary target is fatal. The first version of
that guard blocked whenever any named target did not fit, which on a
repository with two large files lost an item whose real target had been
supplied. The planner orders targets by importance; supporting files are now
named in the prompt with an instruction not to change them.
4. The reviewer got the diff and nothing else (#156). Two of three reasons
in a rejection were "the diff does not show whether…", which no diff can —
while the implementer had 700,000 characters of the same repository. It now
gets the touched files, and a file too large to include is named as absent.
Instructions: what each role was told
5. The reviewer was still told to reason about "the diff alone." Supplying
the files was only half of it — with 637,317 characters in hand including the
whole target file, it still wrote "I cannot verify there isn't another path…".
6. The implementer was not told which checks judge it (#154). A correct
change was refused by
cargo fmt --all -- --check, a command it had neverseen.
7. A retry did not know why the last attempt failed (#157). Three of four
attempts at one item were refused by the same formatter, each unaware the
previous had been.
requeuekeepslast_errordeliberately; nothing passed iton. Not resumption — D11 is untouched, no prior diff is fed back, and the
prompt says so in as many words.
8. The reviewer rejected for things that were not the work (#159). It could
not confirm "no other file changed" (a diff is the whole change) or that the
tree type-checked (
Checks: passedreads as the author's claim; the harnessran them). And it overruled an explicit scope statement — verifying every
requirement, quoting the exclusion back, and rejecting because the wider
problem remained. The prompt now states both facts and gains its first "do not
reject if": scope is the task's to set.
Evidence
Two items completed
plan → implement → apply → checks → commit → review → done, each on its own branch. The second is the direct test of #8 — samemodel, same diff, third attempt:
What these do not solve is #158: 8 of 23 model diffs did not apply, and the
rate tracks target file size (0/1 against 258 bytes, 8/22 against 272 KB and
634 KB). That is a protocol question under D10 and is deliberately untouched.
Also corrects the reason-kind list in
schemas.pyandUSAGE.md, four kindsbehind
outcomes.py, and documents what each role is shown.Gates
All four from the repository root with
TMPDIRon a fast volume:pytestgreen,
ruff check,ruff format --check,mypy(full-project strict) clean.Fourteen new tests, each asserting behaviour rather than a flag: that the
implementer role is never called when its primary target is missing, that a
supporting one is named instead of fatal, that the planner gets paths and not
contents, that a project with no checks tells its reviewer it is the only gate.
🤖 Generated with Claude Code