Skip to content

feat(agents): parallel chat windows, a per-user cap on sandbox runs - #12220

Merged
MarkusNeusinger merged 3 commits into
mainfrom
feat/agent-parallel-windows
Oct 21, 2026
Merged

MarkusNeusinger merged 3 commits into
mainfrom
feat/agent-parallel-windows

Conversation

@MarkusNeusinger

Copy link
Copy Markdown
Owner

Summary

  • Owner decision 2026-10-21: chat windows (sessions) of one user work in parallel. Before, a plot rendering in window A refused window B's paste at once with 409 run_active, which read as "a run or another paste is in progress". Now paste, chat turns, data steps, file conversion and the theme toggle run in every session immediately. Only sandbox work (plot_pipeline, run_data_code) goes through the existing run queue, with its queued, position N status. Within one session it stays one thing at a time.
  • Per-user cap: at most AGENT_USER_MAX_RUNS sandbox runs (default 3, owner's choice) may be queued or running per user, across all sessions. A 4th run ends with the new stream error run_limit, shown as "you already have 3 plots or data runs in progress in other windows; wait until one finishes, then try again". The cap lives in RunQueue.submit (QueueUserLimit). Queue capacity and ordering are unchanged.
  • Locks are keyed by session id: _active(runtime, sid, *, paste=True) is checked on every route, _data_route_free is folded into it, and Runtime.pasting and Runtime.converting hold session ids. Each per-session protection is kept, including the paste-versus-data-route race inside one session. Only the cross-session locks are gone.
  • Budget overlap: counts that are checked and booked without an await in between (messages, pastes, conversions, pipeline runs) cannot overshoot. The daily token budget can overshoot by the model calls already in flight in parallel turns. The daily data-run count can overshoot by at most 2. Code comments document both.
  • Plumbing:
    • run_limit is added to telemetry ErrorCode, to the BFF allowlist _ERROR_CODES (otherwise it would reach the browser as internal), to STREAM_ERROR_CODES and to the page texts.
    • The run_active texts now say "in this window".
    • Docs: docs/reference/api.md, docs/concepts/agent-network.md and agents/README.md are updated.

Plan

N/A. These are owner decisions from the session of 2026-10-21, implemented by a delegate. Gemini 4 Argon reviewed it first; its verdict is in a comment.

Test plan

  • uv run pytest tests/unit tests/integration -n 4: 12,239 passed, 3 skipped.
  • Queue tests:
    • per-user cap: the 4th entry is refused while another user's still queues;
    • the cap is checked before the queue length;
    • finished, withdrawn and expired entries free their place.
  • Flow tests:
    • window B pastes and chats while window A renders;
    • A's own second turn still gets 409;
    • B's plot queues;
    • the 4th sandbox run ends with run_limit and no pipeline model call, while another user is unaffected;
    • the toggle blocks only its own session;
    • two sessions convert at once.
  • BFF: run_limit passes through. Frontend: event parsing, hook and page texts.
  • ruff, mypy api core, tools.changelog check; frontend lint, fm:check, type-check, test (1,166) and build.
  • After deploy, live check: start a plot in window A, then paste and chat in window B. B's plot should show queued.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M

Owner decision of 2026-10-21: the admin agent chat's browser windows work
in parallel, and only sandbox work queues.

- Every exclusion in the agents service is per session now. `_active`
  checks the session's registry entry (`Runtime.active`) and, unless the
  route passes `paste=False` (the language chip), the session's paste in
  flight; `Runtime.pasting` and `Runtime.converting` hold session ids, not
  user ids. `_data_route_free` folds into `_active`. A turn, a paste, a
  theme toggle and the data routes still exclude each other within one
  session, and the user's other sessions never block them.
- `RunQueue.submit` refuses a user's entry past `AGENT_USER_MAX_RUNS`
  (default 3, waiting and running together, across sessions) with the new
  `QueueUserLimit`, checked before the queue's length, so a user at the cap
  learns that from their own runs. `TurnSlot.take` fails the turn with
  `run_limit` (stream `error{code:"run_limit"}`), `run_data_code` answers
  the same code, and the `chat_turn` telemetry's `ErrorCode` gains
  `run_limit`. The queue's capacity and order are unchanged.
- BFF: `run_limit` joins the pass-through allowlist of stream error codes.
- Page: `run_limit` has its own text ("you already have 3 plots or data
  runs in progress in other windows; ..."), and the `run_active` texts now
  name this window.

Decisions:
- Why the cross-session paste exclusion existed: a paste holds the view it
  read before its judges ran and writes the next dataset from it, so a
  version stored meanwhile would be lost. That hazard is per session (the
  stores are keyed by session id, no ADK state is user-scoped), and the
  guards are keyed by session id, so they hold whoever sends the request,
  also two tabs sharing one id. The page opens one session per window and
  keeps its id in memory only, so its tabs never share one.
- Conversions: one per session; each waits for the render slot at most
  `AGENT_CONVERT_SLOT_WAIT_S`, and the per-user conversion counter still
  bounds them.
- Budgets under parallel turns (comment in `post_message`): message, paste,
  conversion and pipeline-run counts are checked and booked with no
  suspending await between, so they never overshoot. The daily token
  budget is checked before every model call and booked after, so parallel
  turns overshoot it by the calls in flight. Daily data runs are checked
  before the queue wait and counted after it, so they can overshoot by at
  most `AGENT_USER_MAX_RUNS` - 1.
- `drop_stale`, `finish` and the idle sweep work per registry entry and
  release each entry's own ticket, so several entries per user need no
  change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M
Copilot AI balanced review requested due to automatic review settings October 20, 2026 22:52
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Session concurrency and shared-budget behavior require final human validation, including the pending live multi-window check.

No open issues 🎉

What changed in this PR

Enables parallel agent-chat sessions while limiting each user’s queued and running sandbox work.

Changes:

  • Makes activity locks session-specific.
  • Adds a configurable per-user sandbox cap, defaulting to three, with run_limit reporting.
  • Updates concurrency tests, browser messages, and documentation.
File Description
tests/​unit/​api/​test_agent_router.py Tests run_limit forwarding.
tests/​unit/​agents/​test_settings.py Checks the default cap.
tests/​unit/​agents/​runtime/​test_turn_limits.py Tests session-specific paste protection.
tests/​unit/​agents/​runtime/​test_service_flow.py Tests independent sessions.
tests/​unit/​agents/​runtime/​test_run_queue.py Tests cap enforcement and release.
tests/​unit/​agents/​runtime/​test_run_queue_flow.py Tests parallel windows and queue limits.
tests/​unit/​agents/​runtime/​test_data_code_tool.py Tests data-run limit errors.
tests/​unit/​agents/​runtime/​test_convert_route.py Tests parallel file conversions.
docs/​reference/​api.md Documents session locks and errors.
docs/​concepts/​agent-network.md Updates concurrency and resource bounds.
changelog.d/​agent-parallel-windows.md Records the behavior change.
app/​src/​sections/​agent-chat/​messages.ts Adds actionable limit messages.
app/​src/​sections/​agent-chat/​DataPanel.test.tsx Updates busy-message expectations.
app/​src/​sections/​agent-chat/​ChatThread.test.tsx Tests session and cap messages.
app/​src/​lib/​agent.ts Accepts run_limit events.
app/​src/​lib/​agent.test.ts Tests error parsing.
app/​src/​hooks/​useAgentSession.ts Updates session-lock commentary only.
app/​src/​hooks/​useAgentSession.test.ts Tests limit-error completion.
api/​routers/​agent.py Allows run_limit through the proxy.
agents/​stream.py Documents the new stream error.
agents/​README.md Documents parallel sessions and configuration.
agents/​main.py Applies session locks and handles cap errors.
agents/​anyplot/​tools/​session.py Propagates data-run cap errors.
agents/​anyplot/​theme_render.py Updates session-exclusion documentation.
agents/​anyplot/​telemetry.py Recognizes run_limit telemetry.
agents/​anyplot/​settings.py Defines the configurable user cap.
agents/​anyplot/​run_queue.py Enforces per-user queue limits.

Review effort: Balanced

@codecov

codecov Bot commented Oct 20, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Gemini 4 Argon review of #12220: a run of the user that goes stale
during another turn's root call would still count toward
AGENT_USER_MAX_RUNS until the next request swept it. TurnSlot.take now
sweeps right before it submits. A flow test pins that a stale run in
another window never causes run_limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M
@MarkusNeusinger

Copy link
Copy Markdown
Owner Author

First review: Gemini 4 Argon

Verdict: sound with changes. Gemini confirmed that every single-session lock is kept, that the counts which are checked and booked without an await (messages, pastes, conversions) cannot overshoot, and that the cap is checked before the queue length. (Its file reads were refused, so it reviewed from the diff and marked unverified claims; I verified the ones below.)

Applied (d7718da12):

  • TurnSlot.take now sweeps stale runs right before submit. A run of the user that goes stale during another turn's root call no longer counts toward AGENT_USER_MAX_RUNS until the next request sweeps. I added a flow test for a stale run in another window. It covers the sweep at the request's start; the take-time sweep covers a window of seconds that the test cannot reach.

Checked, not a problem:

  • "plot_pipeline may attribute capacity on run_limit." The pipeline maps no code. On a refused slot it returns, and the route's TurnSlot._fail(code) sets the ledger error (agents/anyplot/pipeline.py:1321, agents/main.py:466). The flow test asserts error run_limit and chat_turn.error == run_limit.

Accepted for now (admin-only; listed for the public launch):

  • Daily token budget: a user's parallel turns can overshoot it by the calls in flight. The rate limits (10 messages/min, 10 pastes/min) bound this, and a code comment documents it.
  • Refused runs still count: a run_limit (like a capacity) refusal still counts as a daily pipeline run, because BudgetPlugin books the run before the slot, as it already did for capacity.
  • Theme toggles: they are no longer limited to one per user. They wait in the render slot's toggle lane behind pipeline renders, give up after 120 s, and cost no tokens.
  • The hardcoded 3 in the page text matches the other page texts that quote server limits.

Fable was not involved: no finding was serious, and the chat is admin-only. Copilot is still to come.

@MarkusNeusinger
MarkusNeusinger merged commit 3fc9560 into main Oct 21, 2026
15 checks passed
@MarkusNeusinger
MarkusNeusinger deleted the feat/agent-parallel-windows branch October 21, 2026 06:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants