Skip to content

feat(prompt_agent): let a post-turn hook give the user sim a note - #828

Open
kaihackney-scale wants to merge 1 commit into
mainfrom
kai/post-turn-hook-user-note
Open

kaihackney-scale wants to merge 1 commit into
mainfrom
kai/post-turn-hook-user-note

Conversation

@kaihackney-scale

@kaihackney-scale kaihackney-scale commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

What and why

A post-turn hook can already read the envs after each agent turn and decide what happens next, but it can't tell the
LLM user sim what it found. A hook that checks the agent's work (say, a sheet the agent claims to have made) needs the
user sim to react as the person would after looking: "I don't see the sheet yet". Without a channel, the sim judges
the work from the agent's own description of it.

turn.ask_user(note=...) adds that channel. The note goes to the user sim with the agent's reply, as something only
the sim sees: what the person found when checking the work themselves. The agent never sees it.

  • A peer user sim (a deployed agent named as the step's user) gets the note as one more text part after the reply.
    It is recorded under metadata["user_side_notes"].
  • The built-in user sim (deploy_user_sim_agent) gets it in its read as user_note, rendered as its own section. It
    is kept in the stored read.
  • A human peer gets no note.
  • ask_user() with no note behaves exactly as before. An empty or non-text note is refused.

How it was tested

  • New unit tests in test_prompt_agent_post_turn_hooks.py: the note reaches a peer user sim after the reply, is
    recorded, never reaches the agent, a human peer gets none, and a bad note is refused. In
    test_prompt_agent_built_in_user_sim.py: the built-in user sim's read carries it and the prompt shows it.
  • pytest tst/unit/task_step -k "prompt_agent or user_sim or post_turn": 820 passed. make lint: passed.
  • End to end: a local agent-env task run of a multi-turn task on deployed sandboxes, with a peer LLM user sim and a
    post-turn hook that checked the agent's work in the env after each turn and passed a note when something was
    missing. The notes reached the user sim, it raised them in its own words in its next message, the agent then
    fixed the missing item, and no note text appeared in anything the agent received.

🤖 Generated with Claude Code

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge.

Summary

Post-turn hooks can now give user simulators a private note about the agent’s work.

  • Post-turn hooks can give user simulators a private note.

No actionable issues found.

Reviews (1) · Last reviewed commit: "feat(prompt_agent): let a post-turn hook..." · Reviewed by Greptile

turn.ask_user(note=...) shows the note to an LLM user sim with the agent's reply,
as what the person found checking the work themselves. The agent never sees it.
A peer user sim gets it as one more text part after the reply; a built-in user
sim gets it in its read as user_note. A human peer gets none. The note is
recorded under metadata user_side_notes (peer) or in the stored read (built-in).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kaihackney-scale
kaihackney-scale requested a review from a team as a code owner November 19, 2026 22:17

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant