test: drive the direct executor end to end, and fix four rendering defects - #216
Merged
Conversation
…fects `tests/test_executor_e2e.py` drives the executor as `run` drives it — claim, plan, implement, apply, check, review, commit — against a real temporary git repository, a real `git apply`, real check commands that are shell scripts and a scripted model transport, asserting on the git state and the queue row. It found four bugs, all in the seam D10 opened: the implementer returns edit blocks and `edits.to_diff` renders them into a unified diff. `git apply` compares bytes, so a diff that is wrong about the file is refused with no rung of the ladder able to help. Each of these was an edit the model got right, lost between matching the text and applying the patch. 1. A file whose last line has no newline rendered as `-beta+gamma` on one line: `corrupt patch at line 6`. Now rendered with the `\ No newline at end of file` marker. 2. A CRLF file could never be edited at all. `read_text` translates on the way in so SEARCH matched, then the LF diff was refused with `patch does not apply` — on every attempt, with nothing the model could have done. The file's line ending is now read off disk and the diff rendered in it. 3. An empty file that exists was rendered as `--- /dev/null`, which git refuses with `already exists in working directory` — while `plan_edits` deliberately permits an empty SEARCH against exactly that file. `fromfile` now asks whether the file exists, not whether it has content. 4. The diff was computed against the working tree, which still holds the previous item's branch. `select_repo_context` reads through `git show base:path` precisely so that cannot happen; `to_diff` read the tree, so the second item to touch any file was told that text it had been shown "does not occur in the file". The tree is now moved to the base before the implementer is asked, and no branch is created there. No gate is weakened. Three of the four make the rendered diff more faithful to the file on disk, which is what lets the strictest rung accept it; the fourth removes a disagreement between two parts of the harness about which tree an item is written against. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Aug 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
tests/test_executor_e2e.pydrives the direct executor the wayrundrivesit — claim, plan, implement, apply, check, review, commit — against a real
temporary git repository, a real
git apply, real check commands thatare shell scripts, and a scripted model transport. It asserts on the resulting
git state and queue row, never on internals. Same pattern as
tests/test_agent_loop_e2e.py, pointed at the other executor.It found four bugs, all in the seam D10 opened on 2026-08-05: the implementer
now returns edit blocks and
edits.to_diffrenders them into a unified diff.The rendering is new; everything downstream of it is not.
git applycomparesbytes, so a diff that is wrong about the file is refused and no rung of the
tolerance ladder can help — which is the good news and the bad news. Every
one of the four is an edit the model got right, lost between matching the
text and applying the patch.
The four
A file whose last line has no newline produced a corrupt patch.
difflibsignals a missing final newline by emitting a line that does notend in one. Concatenated straight into the diff,
-betaand+gammaarrive as
-beta+gammaand git reportscorrupt patch at line 6. Nowrendered with
\ No newline at end of file, and the file keeps its shapebyte for byte.
A CRLF file could never be edited at all.
read_texttranslates on theway in, so the SEARCH block matches happily; the diff was then rendered in
LF and refused with
patch does not apply. Nothing the model could writewould have changed that — the item failed identically on every attempt
until its ladder ran out. The file's line ending is now read off disk and
the diff is rendered in it.
An empty file that exists was rendered as a creation.
fromfilewaschosen on whether the content was empty, so an empty
__init__.pygot--- /dev/nulland git refused withalready exists in working directory— while
plan_editsdeliberately permits an empty SEARCH against exactlythat file. It now asks whether the file exists.
The diff was computed against the working tree, which holds the previous
item's branch.
select_repo_contextalready refuses to read the tree forthis reason and reads through
git show base:pathinstead;to_diffreadthe tree. So the second item to touch any file named text it had been
shown, that is present on the base its branch will be cut from, and was
told
the SEARCH text does not occur in the file. The model was right andthe harness blamed it — and the better the previous item did, the more
certain the next one was to fail. The tree is now moved to the base before
the implementer is asked. No branch is created there, so an item that
produces no usable diff still leaves none behind.
No gate is weakened
Nothing here makes anything more tolerant. Three of the four fixes make the
rendered diff more faithful to the file on disk, which is what lets the
strictest rung — plain
git apply— accept it; the fuzzy rung stays off and--unidiff-zerois not relied on. The fourth removes a disagreement betweentwo parts of the harness about which tree an item is being written against.
What the tests also pin, and found sound
Checks run before the reviewer is paid (asserted by the reviewer never being
called); a rejected review keeps its checkpoint commit; nothing lands on the
default branch; a refused edit costs an attempt and leaves the tree clean with
no branch; ambiguous SEARCH text is refused rather than applied to the first
match; edit blocks are applied in order so a later one may depend on an
earlier one's result; a unified diff is still accepted as a fallback; edits
change nothing is reported as no change rather than as a broken patch; #152's
ceiling blocks an item without consuming an attempt and without the
implementer ever seeing an unseen target; and #155's declared fix clears its
gate, reaches the commit, and is stated to the reviewer — while a fix that
does not clear its gate still refuses the item.
🤖 Generated with Claude Code