Skip to content

feat(prompt_agent): an opt-in gate on user-sim messages - #597

Open
kaihackney-scale wants to merge 11 commits into
mainfrom
kai/usersim-gate
Open

kaihackney-scale wants to merge 11 commits into
mainfrom
kai/usersim-gate

Conversation

@kaihackney-scale

@kaihackney-scale kaihackney-scale commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

What

prompt_agent and always_on get an opt-in user_sim_gate. When the user side is an LLM user-sim, each of its messages is
checked before the agent receives it. That covers a peer user-sim and the built-in one deploy_user_sim_agent deploys:

  • patterns (free): phrases and shapes that belong to the assistant's side, such as a report of work
    done, a formatted deliverable, offers to do the work, or service phrases.
  • judge_model (optional): one model call over the last turns and the candidate. It decides only whose
    voice the message is in. On its own it fails only messages of judge_min_chars or more; shorter ones
    also need a pattern.

A message that fails isn't sent. The user-sim is told why and asked to rewrite, up to max_rewrites
times. If it still fails, on_fail decides what happens:

  • fail (the default) ends the conversation and fails the step, after the PromptResponse is recorded;
  • end ends the conversation only;
  • send records the verdicts and sends the message anyway.

Every verdict is recorded in context.metadata["user_sim_gate"][step_id], and an ended run is recorded in
context.metadata["user_sim_invalid"].

The built-in user sim answers in JSON (outcome, message, used), so its rewrite is parsed as its own answer:

  • a reply goes on to the gate's next check;
  • a done with text ends the conversation after the agent answers it, as before;
  • a no_reply or an empty done ends the conversation, as it would have.

The read's record (user_sim_reads) keeps what the user sim first answered, for anything grading the user sim, and lists each rewrite under gate_rewrites.

In always_on each answer the user sim would send is checked at its read, before it's held for its response time:

  • a rewrite is asked for in the user sim's own session;
  • the read's timeline event records the verdicts, the rewrites and the first message;
  • an answer that still fails is never delivered, and the run ends with user_sim_invalid. With on_fail: fail it also sets error_type, so verifiers skip the run, as for a user-sim error.

Her last read's answer, which never reaches the agent, isn't checked.

A step type that subclasses PromptAgentTaskStep but runs its own conversation would otherwise accept user_sim_gate and ignore it. Instead it refuses the setting at construction, unless it applies the gate and sets USER_SIM_GATE_SUPPORTED = True, as always_on now does. For that, the check-and-rewrite loop is user_sim_gate.run_gate(gate, text, history, rewrite): rewrite(note) returns the user-sim's new message, or None if the user side ended. prompt_agent uses the same function.

{"user_sim_gate": {"judge_model": "<model>", "max_rewrites": 2, "on_fail": "fail"}}

Why

An LLM playing the user can slip into the assistant's role. It answers its own question, reports work as
done, or offers to do the agent's job. After that, the rest of the run no longer tests the agent. Checking
each message, with a rewrite, keeps those messages out of the conversation, whether or not the user-sim
cooperates.

How it was tested

  • Unit tests:
    • tst/unit/task_step/test_user_sim_gate.py: the patterns, judge parsing, the rendered history and the
      rewrite note.
    • tst/unit/task_step/test_prompt_agent_user_side.py: 8 new tests over the real _UserSide request flow:
      • pass, rewrite and recover;
      • fail after the rewrites, both fail and end;
      • the judge failing a message the patterns pass;
      • a judge error falling back to the patterns;
      • a short message not failed by the judge alone;
      • dict round-trip and validation;
      • run_gate on its own (rewrite, give up, send, stop);
      • a subclass with its own loop refusing the gate unless it supports it.
    • tst/unit/task_step/test_prompt_agent_built_in_user_sim.py: the built-in user sim held back and rewritten, failing
      after its rewrites, and a rewrite that says nothing.
    • tst/unit/task_step/test_always_on_user_sim_gate.py: rewrite and deliver, fail (an error end), end, a pass with
      no rewrite, and no gate.
  • Offline, against labels of real user-sim messages (two independent model labelers, with an adjudicator for
    disagreements, checked against known cases):
    • every labeled out-of-role message is caught;
    • under 0.5% of in-role messages are failed;
    • on held-out messages the patterns alone flag 0.22% of in-role messages.
  • Rewrite recovery: at real out-of-role points, asked to rewrite, the user-sim recovered every time,
    with the next scripted request.
  • E2E: gated and ungated runs of the same multi-turn tasks, through the full task (deploy, conversation,
    snapshot).
    • Every run completed.
    • The gate recorded a verdict for every user-sim message.
    • No out-of-role message reached the agent, judged separately by a grader with the scenario.
    • The rewrite path was not triggered live, because nothing failed; the unit tests and the offline
      replay cover it.
  • E2E on the built-in user sim in always_on: two runs of a multi-day task, 4 wakes each, on the gate's
    patterns plus a judge. One used the persona as written; the other's persona was told to answer like a status report.
    • Both completed.
    • Every reply the agent would get was checked, and passed.
    • The persona change didn't make the user sim write as the assistant, so the rewrite and fail paths again ran
      only in the unit tests.
    • Every sandbox was torn down.

ruff check is clean.

No screenshots: the change has no UI.

Off by default: a step without user_sim_gate behaves exactly as before, and to_dict leaves the field out.

🤖 Generated with Claude Code

RetriggerView in GreptileConfidence Score: 5/5

The latest routing changes appear safe to merge.

Summary

Adds an opt-in check that holds back assistant-like user-sim messages and asks for rewrites before delivery.

  • The shared gate checks user-sim messages and asks for rewrites when needed.
  • Prompt-agent conversations check peer and built-in user-sim messages before sending them.
  • Always-on runs check user-sim answers before holding them for delivery.

Reviews (10) · Last reviewed commit: "fix(user_sim_gate): route the judge call..." · Reviewed by Greptile

prompt_agent's new user_sim_gate checks each LLM user-sim message before the agent receives it: phrase
patterns for the assistant's side of a conversation, and optionally a small judge model that decides only
whose voice the message is in. A message that fails is held back and the user-sim is asked to rewrite it,
up to max_rewrites times; on_fail then fails the step, ends the conversation, or sends it anyway. Every
verdict is recorded in context.metadata["user_sim_gate"].

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e its cutoffs

`user_sim_gate={}` now enables the gate with its defaults; only None leaves it off. The patterns' length and
count cutoffs, and how much of a judge's reason and error a verdict keeps, are named class constants.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale

Copy link
Copy Markdown
Collaborator

🤖 Automated Claude routine — Env Pod PR Context Review. This is a bot, not Edgar Arakelyan; not a human review.

Context from outside this diff, in case it's useful: there's a second open PR adding a step type that subclasses PromptAgentTaskStep and filters its inherited fields by an explicit list, so user_sim_gate would land on it as a silently-inert field.

  • This PR: src/agent_env/task_step/task_steps/prompt_agent.py:181 adds user_sim_gate as a constructor kwarg on PromptAgentTaskStep, serialized at :309-310 / :348, and enforced inside _UserSide (:1063, :1130).
  • feat(always_on): a step that runs an assistant over days of task time, with a simulated user (FD-5364) #566 (feat(always_on): a step that runs an assistant over days of task time, with a simulated user, head 974a6aba): src/agent_env/task_step/task_steps/always_on.py:201 is class AlwaysOnTaskStep(PromptAgentTaskStep). Its __init__ takes **prompt_fields and pops a fixed drop list at :217 (max_conversation_turns, user_agent_name, user_a2a_url, post_turn_hooks, user_output_format, user_model, snapshot_config), then forwards the rest to super().__init__. Its to_dict at :303-304 starts from TaskStep.to_dict(self) — not the parent's — and copies only _PROMPT_FIELDS (:252) plus _OWN.

So in either merge order, an always_on step authored with user_sim_gate is accepted at construction (it isn't in the drop list), dropped on to_dict (it isn't in _PROMPT_FIELDS), and never consulted: always_on.py has its own _Run loop and contains no reference to _UserSide, _converse or user_sim_gate. The :588 warning only fires on the prompt_agent path, so there'd be no signal either. Deciding where it goes — a third list entry, the drop list, or out of scope — is probably cheaper now than after both land.

Related, same direction: #567 addresses the same failure mode from the prompt side — user_sim_versions.py builds a system prompt around "You are not the AI assistant … Don't do your assistant's work: never look things up for it or hand over facts it should find", and user_sim.md reports a measured effect (done collapsed into the same message 9/10 → 1/10). Worth a look if you want the gate's pattern list and that prompt to agree on what counts as the assistant's voice.


Generated by Claude Code

t and others added 2 commits November 10, 2026 13:52
…te where it would be ignored

The check-and-rewrite loop moves to user_sim_gate.run_gate, with a rewrite callback, so a step type that runs
its own conversation can apply the gate; prompt_agent's user side now calls it. A PromptAgentTaskStep subclass
that replaces prompt_agent's conversation refuses user_sim_gate at construction, instead of accepting and
ignoring it, unless it sets USER_SIM_GATE_SUPPORTED = True.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main reworked prompt_agent's user side (the built-in user sim, end_after). The gate keeps its
history wrapper in ask(), main's end_after short-circuit opens _answer(), and the gate still
checks peer user-sim messages. A gate set with the built-in user sim is warned about, as with a
non-user-sim peer, since those messages aren't checked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
t and others added 4 commits November 12, 2026 17:18
…p errors fall back to the patterns

- AlwaysOnTaskStep.from_dict passes user_sim_gate on, so a saved step is refused as one built in
  code is, rather than running with the setting dropped.
- A judge whose setup fails (its key or endpoint) no longer ends the conversation: like a failed
  judge call, the patterns' verdict stands and the error is recorded on each verdict.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ps checking

The fallback rebuilt the gate without its judge, which a judge-only gate (patterns off) refuses,
so its setup error ended the conversation. The gate's check now takes judge_unavailable instead:
the judge isn't called, the patterns' verdict stands (a pass with patterns off), and the reason is
recorded on the verdict, as for a judge call that fails. run_gate passes it only when set, so a
check override with the old signature still works.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…agent and always_on

prompt_agent applies user_sim_gate to the user deploy_user_sim_agent deployed: a held-back answer is rewritten in its session and parsed as its own answer, and the read's record keeps the first answer and lists each rewrite. always_on applies it at each read before an answer is held: one that still fails is never delivered and ends the run (user_sim_invalid; an error with on_fail fail).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
t and others added 2 commits November 16, 2026 13:07
…ite counts toward max_replies

A conversation the gate cut off with on_fail fail now records error_type user_sim_invalid on its PromptResponse, so verifiers skip it as they skip a user-sim error. A built-in user sim's rewrite replaces the held answer in the reply count, so a done rewritten into a reply counts, and max_replies ends the conversation after it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…i/usersim-gate

# Conflicts:
#	src/agent_env/task_step/task_steps/prompt_agent.py
…es its judge

Through an endpoint, a Claude judge now drops the base's trailing /v1 (its client appends /v1/messages), and a bare
name litellm can't place goes as openai/<name>. Before, both failed every call, and the gate fell back to its
patterns alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants