test: adapt the messages max_tokens test to anthropic 1.1's terminal-stop policy - #434
Merged
Conversation
anthropic 1.1.0 replaced the runner's refusal special case with an explicit stop-reason table: every non-tool_use stop is terminal and its tool_use blocks are never executed (1.0 executed a max_tokens turn's complete blocks). The envelope already keys on the runner's history, so the product adapts by design; only the test had the 1.0 behavior baked in. It now asserts the version-appropriate shape on both sides, and the run_messages comment describing the old behavior is reworded to name the policy split. Verified: 434 green on anthropic 1.1.0 (CI's failing config) and on 0.120.2 (the pre-1.1 branch); no-frameworks collection stays clean.
… numbers The rationale to preserve is that execution policy belongs to the vendor and only the runner's history is ground truth; the exact 1.0/1.1 switch point lives in the previous commit's message.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The three with-frameworks CI legs went red on
test_messages_tool_use_cut_by_max_tokens_not_duplicatedafter anthropic 1.1.0 shipped (main was green on 1.0.0 yesterday; CI installs latest).Root cause: 1.0.0 special-cased only
refusalas terminal, so amax_tokensturn with complete tool_use blocks was executed and appended. 1.1.0 introduces an explicit stop-reason table (_STOP_REASON_STEPS) where every non-tool_use stop is terminal and its tool_use blocks are never executed.The product needs no change:
run_messageskeys the appendable history on the runner's own params rather than on stop_reason, so on 1.1+ the cut turn is stripped exactly like the refusal case and the continuation invariant holds. Only the test had the 1.0 vendor behavior baked into its assertions; it now asserts the version-appropriate shape on both sides of 1.1. Therun_messagescomment that described the old behavior as timeless is reworded to name the policy split.Verified: 434 passed on anthropic 1.1.0 (the failing CI config, previously red) and on 0.120.2 (the pre-1.1 branch); collection stays clean with anthropic and agents blocked (the without-frameworks legs).