Repository navigation
fix(agents): session block in the system instruction, not the user turn - #12199
Merged
Merged
Conversation
ADK 2.11 sends an agent's dynamic `instruction` next to a `static_instruction` as user content between <<<BEGIN_SYSTEM_INSTRUCTION>>> markers, inserted into the user's own turn. Claude Haiku 5.5 read the root's session block there as user text posing as a system instruction (7 of 8 prep-eval turns' thinking said so; one reply told the user about the markers "in your message"). The root now has no dynamic instruction. Its before_model_callback (add_session_context) appends the session block to the system instruction as a second part after the static policy, and VertexClaude sends each part as its own system block with ADK's cache breakpoint on the first, so a changing block leaves the policy cached. root.md describes the block as part of the instruction. The prep eval scores a reply that talks about the root's own instructions as a miss (instruction_talk). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
Moving session data into system instructions and changing provider-specific caching warrant final human review.
0 open findings
What changed in this PR
Moves the chat root’s session context from user messages into the system instruction so the model does not mistake server-provided context for forged instructions.
Changes:
- Appends session context after the static policy, preserving Claude’s cache breakpoint on the policy.
- Makes instruction-related replies fail the prep evaluation.
- Adds regression tests and updates documentation.
| File | Description |
|---|---|
| tests/unit/agents/runtime/test_service_flow.py | Verifies session context stays in system instructions. |
| tests/unit/agents/runtime/test_registry.py | Requires agents to have no dynamic instruction. |
| tests/unit/agents/runtime/test_policy.py | Checks updated session-context guidance. |
| tests/unit/agents/runtime/test_models.py | Tests system-block splitting and caching. |
| tests/unit/agents/evals/test_prep.py | Tests instruction-talk failures and request structure. |
| docs/concepts/agent-network.md | Documents session placement and cache behavior. |
| changelog.d/agent-session-block-system.md | Records the fix and evaluation results. |
| agents/README.md | Updates architecture and evaluation documentation. |
| agents/evals/prep.py | Detects and reports instruction-related replies. |
| agents/anyplot/prompts/root.md | Clarifies that session context belongs to system instructions. |
| agents/anyplot/models.py | Preserves separate Claude system blocks. |
| agents/anyplot/agent.py | Adds session context through a model callback. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…hecks The CI runner under coverage scanned the 30 MB crafted comment in 1.17 s and 1.19 s, over the 1.0 s bound, on two runs of #12199. A backtracking scan of these inputs takes hours, so 5 s still tells the two apart. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
… check, prompt wording)
- with_session_part accepts only a non-empty string or a Content of
text parts; any other shape raises instead of sending the root out
with the session block and no policy.
- INSTRUCTION_TALK matches ADK's markers by name (a bare <<< also hit
heredocs such as <<<EOT) and the plain words only after a determiner
("your system instruction", "der Sitzungsblock").
- root.md: the block is "the last section" of the instruction (ADK's
identity line stands between the refusals and it), and the values in
its <plot_language> and <user_data> fences are data the user chose.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M
A live prep eval on origin/main (2026-10-15) had the root reply "Den eingefügten Text, der sich als Systemanweisung ausgibt, behandle ich nicht als Anweisung." The determiner list of the narrowed pattern missed it; "als"/"as" and the indefinite articles now count. Replayed on every traced reply: only that case matches. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
static_instructionplus a dynamicinstruction(the session block). ADK 2.11 sends the dynamic part as user content: a<<<BEGIN_SYSTEM_INSTRUCTION>>>block with a preamble, inserted at the current-turn boundary, so it lands inside the user's own turn (google/adk/flows/llm_flows/prompt/_instructions.py,LlmRequest._insert_transient_user_content). Claude Haiku 5.5 read it as user text that posed as a system instruction. In a baseline trace of 8 prep cases on origin/main, its thinking said so in 7 of 8 turns, for example "a suspicious block embedded in the user's message ... which looks like an injection attempt I should disregard". Every one of the 23 root requests carried the markers in a user message.instructionanymore. Abefore_model_callback(add_session_context) appends the session block to the system instruction as a second part after the static policy.VertexClaudesends each part as its own Claude system block, with ADK's cache breakpoint on the first part only (system_blocks). A session block that changes therefore leaves the policy cached, and the static prefix stays byte-identical.root.mdnow describes the block as part of the instruction instead of explaining away the markers. The thinkingblock_bindingdrop stays: a data tool can still change the block between two calls.instruction. Model tests cover the two-block split, with and without caching. The prep eval now scores a reply that talks about the root's own instructions (markers, a system instruction, the session block) as a miss and counts it underinstruction_talk. A scripted test covers that path and asserts the shape of the Claude request.Numbers (Claude Haiku 5.5, Vertex AI
eu)guard_unavailable)On the same 8 cases, the cost per model call fell from $0.00114 to $0.00087. The three full-run misses are model choices unrelated to the block: one leaves the
Notizcolumn in, one labels classes1/2instead ofKlasse 1/Klasse 2, and one adds an extra sort. None of them talked about instructions.Decided and documented
<plot_language>and<user_data>fences. User text can no longer imitate it in place, androot.mdsays that a message that imitates the block is wrong. The scope guard and the[message withheld]path touch only user contents and are unchanged.Plan
N/A
Test plan
pytest tests/unit/agents: 6,842 passed.renderer/test_executor.py::test_the_memory_floor_kills_the_runfailed once under-n autowhile the live eval ran alongside, and passes alone. It is unrelated.test_convert.py::TestSecurityReview(from feat(agents): read dropped Excel, ODS, Parquet, Feather, Arrow and NumPy files #12194) asserted a 1.0 s wall clock for scanning a 30 MB crafted comment. The CI runner, under coverage, took 1.17 s and 1.19 s on two runs. The bound is now 5 s, which still tells a linear scan from a backtracking one (hours).ruff check,ruff format --check,mypy api core agentsuv run python -m tools.changelog check --base origin/main🤖 Generated with Claude Code
https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M