Skip to content

test: drive the direct executor end to end, and fix four rendering defects - #216

Merged
thedancingdeveloper merged 1 commit into
mainfrom
test/executor-e2e
Aug 6, 2026
Merged

test: drive the direct executor end to end, and fix four rendering defects#216
thedancingdeveloper merged 1 commit into
mainfrom
test/executor-e2e

Conversation

@thedancingdeveloper

Copy link
Copy Markdown
Contributor

tests/test_executor_e2e.py drives the direct executor the way run drives
it — claim, plan, implement, apply, check, review, commit — against a real
temporary git repository, a real git apply, real check commands that
are shell scripts, and a scripted model transport. It asserts on the resulting
git state and queue row, never on internals. Same pattern as
tests/test_agent_loop_e2e.py, pointed at the other executor.

It found four bugs, all in the seam D10 opened on 2026-08-05: the implementer
now returns edit blocks and edits.to_diff renders them into a unified diff.
The rendering is new; everything downstream of it is not. git apply compares
bytes, so a diff that is wrong about the file is refused and no rung of the
tolerance ladder can help
— which is the good news and the bad news. Every
one of the four is an edit the model got right, lost between matching the
text and applying the patch.

The four

  1. A file whose last line has no newline produced a corrupt patch.
    difflib signals a missing final newline by emitting a line that does not
    end in one. Concatenated straight into the diff, -beta and +gamma
    arrive as -beta+gamma and git reports corrupt patch at line 6. Now
    rendered with \ No newline at end of file, and the file keeps its shape
    byte for byte.

  2. A CRLF file could never be edited at all. read_text translates on the
    way in, so the SEARCH block matches happily; the diff was then rendered in
    LF and refused with patch does not apply. Nothing the model could write
    would have changed that — the item failed identically on every attempt
    until its ladder ran out. The file's line ending is now read off disk and
    the diff is rendered in it.

  3. An empty file that exists was rendered as a creation. fromfile was
    chosen on whether the content was empty, so an empty __init__.py got
    --- /dev/null and git refused with already exists in working directory
    — while plan_edits deliberately permits an empty SEARCH against exactly
    that file. It now asks whether the file exists.

  4. The diff was computed against the working tree, which holds the previous
    item's branch.
    select_repo_context already refuses to read the tree for
    this reason and reads through git show base:path instead; to_diff read
    the tree. So the second item to touch any file named text it had been
    shown, that is present on the base its branch will be cut from, and was
    told the SEARCH text does not occur in the file. The model was right and
    the harness blamed it — and the better the previous item did, the more
    certain the next one was to fail. The tree is now moved to the base before
    the implementer is asked. No branch is created there, so an item that
    produces no usable diff still leaves none behind.

No gate is weakened

Nothing here makes anything more tolerant. Three of the four fixes make the
rendered diff more faithful to the file on disk, which is what lets the
strictest rung — plain git apply — accept it; the fuzzy rung stays off and
--unidiff-zero is not relied on. The fourth removes a disagreement between
two parts of the harness about which tree an item is being written against.

What the tests also pin, and found sound

Checks run before the reviewer is paid (asserted by the reviewer never being
called); a rejected review keeps its checkpoint commit; nothing lands on the
default branch; a refused edit costs an attempt and leaves the tree clean with
no branch; ambiguous SEARCH text is refused rather than applied to the first
match; edit blocks are applied in order so a later one may depend on an
earlier one's result; a unified diff is still accepted as a fallback; edits
change nothing is reported as no change rather than as a broken patch; #152's
ceiling blocks an item without consuming an attempt and without the
implementer ever seeing an unseen target; and #155's declared fix clears its
gate, reaches the commit, and is stated to the reviewer — while a fix that
does not clear its gate still refuses the item.

🤖 Generated with Claude Code

…fects

`tests/test_executor_e2e.py` drives the executor as `run` drives it — claim,
plan, implement, apply, check, review, commit — against a real temporary git
repository, a real `git apply`, real check commands that are shell scripts and
a scripted model transport, asserting on the git state and the queue row.

It found four bugs, all in the seam D10 opened: the implementer returns edit
blocks and `edits.to_diff` renders them into a unified diff. `git apply`
compares bytes, so a diff that is wrong about the file is refused with no rung
of the ladder able to help. Each of these was an edit the model got right,
lost between matching the text and applying the patch.

1. A file whose last line has no newline rendered as `-beta+gamma` on one
   line: `corrupt patch at line 6`. Now rendered with the
   `\ No newline at end of file` marker.
2. A CRLF file could never be edited at all. `read_text` translates on the way
   in so SEARCH matched, then the LF diff was refused with `patch does not
   apply` — on every attempt, with nothing the model could have done. The
   file's line ending is now read off disk and the diff rendered in it.
3. An empty file that exists was rendered as `--- /dev/null`, which git
   refuses with `already exists in working directory` — while `plan_edits`
   deliberately permits an empty SEARCH against exactly that file. `fromfile`
   now asks whether the file exists, not whether it has content.
4. The diff was computed against the working tree, which still holds the
   previous item's branch. `select_repo_context` reads through
   `git show base:path` precisely so that cannot happen; `to_diff` read the
   tree, so the second item to touch any file was told that text it had been
   shown "does not occur in the file". The tree is now moved to the base
   before the implementer is asked, and no branch is created there.

No gate is weakened. Three of the four make the rendered diff more faithful to
the file on disk, which is what lets the strictest rung accept it; the fourth
removes a disagreement between two parts of the harness about which tree an
item is written against.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant