Repository navigation
loopguard: an edit that applied is not an error output (fixes #151) - #158
Closed
Vu Linh Truong (vulinh1704) wants to merge 1 commit into
Closed
Vu Linh Truong (vulinh1704) wants to merge 1 commit into
Vu Linh Truong (vulinh1704) wants to merge 1 commit into
Conversation
Claude Code's Edit result names no outcome: it is the file's text (filePath, oldString, newString, originalFile, structuredPatch, ...), and a Write's carries the whole file as `content`. With no exit status, the guard read that text with ERR_RE, so edits to a document that mentions `assert`, `failed` or an `...Error` counted as error outputs and were tagged [err#...] in the judge's digest. An edit that does not apply (text not found, file not read first) comes as PostToolUseFailure, so an Edit, MultiEdit or Write that came back as PostToolUse succeeded: claude_success_code now gives it 0, as it gives a covered shell call (microsoft#128). The history import's envelope replay does the same. A failed edit and a failing run still count; ERR_RE and SpiralState are unchanged.
Collaborator
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #151.
Claude Code's Edit result names no outcome. It is the file's text (
filePath,oldString,newString,originalFile,structuredPatch, ...), and a Write result carries the whole file ascontent. A success has no exit status, so the guard read that text withERR_RE. Edits to a document that mentionsassert,failedor an...Errorcounted as error outputs and were tagged[err#...]in the judge's digest (#151).An edit that does not apply (text not found, file not read first) comes as
PostToolUseFailure. So an Edit, MultiEdit or Write that came back asPostToolUsesucceeded, andclaude_success_codenow gives it 0, the same way it gives 0 to a covered shell call since #128. The history import's envelope replay uses the same function, so it reads edits the same way.ERR_RE,SpiralStateandresponse_textare unchanged.I did not take the fix suggested in the issue (return
""fromresponse_textfor a dict with no known key). It covers Edit, but not Write, whosecontentis a known key and is the whole file. It would also change what every host folds for an unknown dict.The tests are in a new
tests/test_loopguard_edit_results.py, next totest_loopguard_productive_turns.py. They send Edit, MultiEdit and Write results whose file text containsassert,failedandAssertionError, shaped as Claude Code 2.1 logs Edit and Write results. A failed Edit and a failing test run still count as error outputs.Verified on Ubuntu 24.04 (WSL), Python 3.12, on
mainata71684c:error_outputs == 4.python -m pytest tests -rfEs --tb=linereports 739 passed and 16 skipped.python -m pytest tests experiments/testsreports 1132 passed and 20 skipped. Each skip needs a tool the machine does not have (a real Codex, Hermes or opencode CLI, aider) or Windows.Risk: only Claude Code edits reported as successes change, and they could not have failed. A failed edit still arrives as
PostToolUseFailureand is read by its error text, as before.