fix: scope socket events and queue UI per user in multiuser mode#9358
Open
lstein wants to merge 10 commits into
Open
fix: scope socket events and queue UI per user in multiuser mode#9358lstein wants to merge 10 commits into
lstein wants to merge 10 commits into
Conversation
In multiuser mode, invocation events were broadcast to the whole queue room, leaking every user's activity into everyone's client. Now: - invocation_progress goes to the owner's user room only - invocation_started/complete/error emit once to [owner room, admin room] - model load events carry the owner's user_id and follow the same routing - queue_items_retried emits the full event to each affected owner and to admins, plus a sanitized companion (empty id lists, user_id lists) to the rest of the queue room so other users' badge totals refresh; _owner_and_admin_sids accepts a collection because a retry batch may span multiple users' items Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
SessionQueueStatus gains user_pending and user_in_progress, populated for the requesting user (admins included). The global pending/in_progress counts include other users' generations in multiuser mode, so user-facing indicators need counts scoped to the current user. Null in contexts with no requesting user (socket event payloads); frontend consumers fall back to the global counts, which is correct in single-user mode. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two modules to be wired into the socket listeners next:
- eventScope.ts: one canonical classifier for socket event ownership
('own' | 'foreign' | 'sanitized'), replacing the near-duplicate
isOwnEvent/isOwnItem/redacted predicates scattered across handlers
- onQueueItemStatusChanged.tsx: the queue_item_status_changed handler
extracted from setEventListeners.tsx, split into an own-event path
(optimistic cache updates, failure toast, progress clear) and a
non-owner path (queue tag invalidation only), mirroring the
buildOnInvocationComplete pattern
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Admin clients receive every user's invocation and queue item events via the "admin" socket room; these must not drive personal UI state. Route events by ownership at the socket listener, so each handler only ever sees the events it owns: - foreign invocation_started/progress/error are dropped (backend already routes progress owner-only; this is defense in depth) - foreign invocation_complete downgrades to tag invalidation only (buildOnForeignInvocationComplete): no DTO fetches, no optimistic cache edits, no auto-switch, no progress clear. RTK Query only refetches invalidated queries with active subscribers, so an admin actively viewing another user's board stays fresh while idle clients do no work - non-owner queue_item_status_changed (sanitized companion or admin-room copy) only invalidates queue tags With ownership decided upstream, buildOnInvocationComplete and the own-event queue status handler lose their per-user gating and the workflow execution coordinator never records foreign items. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rent user The progress bar, favicon badge, and invoke buttons used the global queue counts, so another user's generations made them light up. They now prefer the per-user counts (falling back to the global counts in single-user mode), and the invoke buttons spin whenever the current user has queued sessions - feedback that their session is enqueued even while another user's generation occupies the processor. Queue status events carry null per-user counts, so cached statuses retain the last fetched per-user counts until the accompanying tag invalidation refetches them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
cancel_all_except_current and delete_all_except_current update rows in a single SQL statement and emit no per-item queue_item_status_changed, so only the initiating client (via its own mutation's tag invalidation) learned that pending items left the queue. When an admin canceled other users' items, those users' badge counts stayed stale until an unrelated event refetched them. Add a queue_items_canceled event, emitted by both bulk operations with the affected item ids grouped by owner. Routing mirrors queue_items_retried: each owner receives the full event scoped to their own item ids, admins receive all of them, and a sanitized companion (no ids, no owners) reaches the rest of the queue room so every subscriber's queue list and badge counts refetch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 16, 2026
The generated OpenAPI schema lists component schemas alphabetically; QueueItemsCanceledEvent was hand-spliced after QueueItemsRetriedEvent, so the openapi-checks and typegen-checks CI jobs failed on the ordering. Regenerated both files so they byte-match CI's generator output. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Correctness: - Guard the ImageViewer's direct queue_item_status_changed and invocation_progress listeners by ownership, so sanitized companions and foreign full events no longer blank another user's live progress preview. - Classify events as foreign while multiuser auth is hydrating (token present, user not yet loaded); previously other users' admin-room events ran full own-event handling during the /me window after a page reload. - Restore CurrentSessionQueueItem/NextSessionQueueItem/InvocationCacheStatus/ BatchStatus/QueueCountsByDestination invalidations for non-owner status changes, so admin UI (cancel-current button, canvas queue counts) stays fresh on other users' activity. - Emit queue_items_canceled from all bulk mutations (cancel_by_batch_ids, cancel_by_destination, delete_by_destination, cancel_by_queue_id), not just cancel/delete_all_except_current. - Embed the owner's per-user counts in queue_item_status_changed events so personal UI (progress bar, spinner, favicon) updates from the event instead of stalling on stale counts until a refetch lands. - Skip admin sids on bulk-event owner-room emits so an admin owner (and every single-user socket) receives exactly one copy. - Skip intermediate images in the foreign invocation-complete handler; they never appear in the gallery, so the invalidation was pure waste. - Fail closed in _handle_queue_event's default branch: unrouted events that carry user identity go to owner+admin rooms, never the whole queue room. Cleanup: - Share one bulk-event routing helper in sockets.py, one collect-and-emit helper in the session queue, and one tag-invalidation helper in the socket listeners (retried/canceled were verbatim copies). - Extract getUserScopedQueueCounts as the single source of the per-user count fallback (progress bar, favicon, spinner), and one InvokeButtonIcon shared by both invoke buttons. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- defer the socket connection until auth.user hydrates from /me so the user's own events cannot be misclassified as foreign during that window, silently dropping one-shot side effects (progress, node execution states, gallery auto-switch, failure toast) that never replay - emit queue_item_status_changed, batch_enqueued, and generic queue item events to the owner+admin rooms in a single call so a socket in both rooms (admin owners; every client in single-user mode) receives one copy - scope useIsGenerationInProgress to the current user's own counts so the progress workspace tabs stop reacting to other users' generations - exclude the separately-canceled current item from the bulk queue_items_canceled event in delete_by_destination; it already emits its own per-item status event - include board write access in the invoke button spinner's disabled state so the spinner never renders on a disabled button - share the queue invalidation tag core across the five queue event handlers, and use id-scoped BatchStatus/QueueCountsByDestination tags for admin-room full events instead of type-level over-invalidation Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…mment - derive socketOptions from the redux auth token instead of a one-time localStorage read, making the token a dependency of the connect effect: an in-tab logout or session expiry now disconnects the socket instead of leaving it connected with the old user's room membership (and private events) until the next full page reload; also reset $socket to null on teardown - correct the SessionQueueStatus invalidation comment: the per-user counts now arrive in the event's embedded queue_status, and the refetch is kept for the processor status and event-drift reconciliation, not the counts Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Supersedes #9314 and #9353, which fixed overlapping parts of the same problem: in multiuser mode, socket events for one user's generations leaked into other users' (especially admins') personal UI — hijacking the gallery's auto-switch, the progress bar, the favicon badge, and queue counts. This PR combines both fixes, resolves their conflicting handling of
invocation_completewith the invalidate-only "middle ground" agreed with @JPPhoto, and restructures the frontend event handling so the per-user routing logic is explicit and readable (addressing @JPPhoto's review feedback on legibility).It also fixes a bug found while smoke-testing this branch: bulk cancels (
cancel_all_except_current/delete_all_except_current) emitted no events at all, so other clients' queue badges went stale when an admin canceled their items.How multiuser event handling works after this PR
Backend: room-based routing (
invokeai/app/api/sockets.py)On connect (
_handle_connect), each authenticated socket joins its personal roomuser:<user_id>; admin sockets additionally joinadmin. In single-user mode the lone client joinsuser:systemandadmin. Queue subscribers also join the queue room (default)._handle_queue_eventroutes each queue event by type:invocation_progressinvocation_started/complete/errorqueue_item_status_changed,batch_enqueueduser_id="redacted", identifiers/errors cleared)queue_items_retried,queue_items_canceled(new)queue_cleared(user-scoped)queue_cleared(unscoped)Every owner+admin delivery is a single emit to
[user room, admin]— python-socketio dedups recipients across a room list, so a socket in both rooms (an admin who owns the item, or the single-user client, which is always in both) receives exactly one copy instead of processing the event twice.Sanitized companions exist so non-owners' queue lists and badge totals refetch without seeing whose items changed or what they were.
_owner_and_admin_sids()computes theskip_sidlist so owners and admins never receive a redacted second copy that would clobber their full-fidelity caches.Progress events are owner-only because no admin UI consumes other users' progress — this is the primary guard against progress-bar hijacking; the frontend filter below is defense in depth.
Frontend: ownership decided at the listener
getEventScope(getState, data)(src/services/events/eventScope.ts) is the single classifier used everywhere. It returns:'own'— the current user's event (in single-user mode there is no authenticated user, so every event is'own'and behavior is identical to before);'foreign'— another user's full event, which only admin clients ever receive (via theadminroom);'sanitized'— a redacted companion (user_id === "redacted").It replaces the three near-duplicate predicates (
isOwnEvent,isOwnItem, inline'redacted'checks) that were previously scattered mid-handler.Classification requires
auth.user, so in multiuser modeuseSocketIOdefers the socket connection until it has hydrated from/me— see the review-fixes section below.setEventListeners.tsxroutes each event at thesocket.oncallback, before any state is touched, so downstream handlers only ever see events they should act on:invocation_started/invocation_progress/invocation_error: dropped unless'own'. Foreign events must not reach theworkflowExecutionCoordinator, node execution states, or$lastProgressEvent.invocation_complete:'own'→ the coordinator →buildOnInvocationComplete; otherwise →buildOnForeignInvocationComplete.queue_item_status_changed:'own'→buildOnQueueItemStatusChanged; otherwise →buildOnNonOwnerQueueItemStatusChanged.queue_items_canceled(new): tag invalidation for queue caches, plus per-item tags when the event names ids.Key handlers (each now free of per-user conditionals, since scope is decided upstream):
buildOnInvocationComplete(onInvocationComplete.tsx) — own completions only: fetches result image DTOs, optimistically updates gallery caches (board totals, image name lists), auto-switches the gallery to the new image, clears the progress indicator.buildOnForeignInvocationComplete(same file) — the middle ground for admins observing other users. No DTO fetches, no optimistic cache edits, no auto-switch, no progress clear: it dispatches a singleinvalidateTags(FOREIGN_GALLERY_REFRESH_TAGS)(ImageNameList,BoardImagesTotal, board list,VirtualBoards). RTK Query only refetches invalidated queries with active subscribers, so an admin actively viewing another user's board stays fresh within one round-trip while idle admin clients do no work at all.buildOnQueueItemStatusChanged/buildOnNonOwnerQueueItemStatusChanged(onQueueItemStatusChanged.tsx, extracted from a ~120-line inline handler) — the own path does the optimistic queue cache updates, failure toast, and progress clear; the non-owner path (sanitized companion or admin-room copy) only invalidates queue tags.Per-user queue counts.
SessionQueueStatusgainsuser_pending/user_in_progress, populated for the requesting user.ProgressBar, the favicon badge (useSyncFaviconQueueStatus), and the invoke buttons use them viauseUserHasActiveQueueItems, falling back to the global counts in single-user mode. The invoke buttons now spin while the user has queued sessions, so enqueueing gives feedback even when another user's generation occupies the processor. A fullqueue_item_status_changedevent embeds the owner's fresh per-user counts, so the owner's personal UI updates from the event without waiting for a refetch; the sanitized companion nulls them, andwithRetainedUserCounts(queueStatusEvents.ts) retains the last known values for statuses that carry none until the accompanying invalidation refetches them.Bulk cancel notification (new).
cancel_all_except_currentanddelete_all_except_currentpreviously updated rows in one SQL statement and emitted nothing — only the initiating client (via its own mutation) refreshed. They now emitqueue_items_canceledwith the affected item ids grouped by owner, routed likequeue_items_retried(see table above), so every client's badge and queue list update immediately.Resolution of the #9314 / #9353 conflict
The two PRs disagreed on foreign
invocation_complete: #9353 kept admins' gallery caches live via optimistic updates; #9314 (per review) ignored foreign events entirely, which left an admin's queue tab showing an item completed while the gallery beside it never showed the image. The middle ground adopted here — invalidate-only — gets the freshness of the former at nearly the simplicity of the latter, with no per-image fetches and no partial gating inside handlers.Code-review fixes (ce65777, 274374c)
A review pass over the finished branch surfaced eight issues, fixed in ce65777:
/mepopulatedauth.user— so a reload during the user's own running generation misclassified their events as'foreign'in that window, silently dropping one-shot side effects (progress, node execution states, gallery auto-switch, the failure toast) that never replay.useSocketIOnow defersconnect()untilauth.userhas hydrated whenever a token is present; the'foreign'fallback ingetEventScoperemains as defense in depth.queue_item_status_changed,batch_enqueued, and the generic queue-item fallback still used two separate emits (user room, then admin room), so a socket in both rooms received and processed every event twice — in single-user mode, that was every client on every status change. They now use the same singleemit(room=[user_room, "admin"])as the invocation events.useIsGenerationInProgressis now user-scoped, so the Dockview progress/canvas tabs no longer react to other users' generations.delete_by_destinationno longer double-signals the current item: it is canceled with its own per-item status event, so it is excluded from the bulkqueue_items_canceledevent (still deleted and counted).queue.isDisabled || !canWriteImages), so it can no longer animate on a disabled button.QUEUE_CHANGED_TAGS, spread by all five queue event handlers), preventing the hand-copied lists from drifting apart.BatchStatus/QueueCountsByDestinationtags when the event carries real ids, reserving type-level invalidation for the sanitized companion — so an admin client on a busy queue no longer refetches every batch/destination query on every foreign status change.A second adversarial review pass over the updated branch (including the fixes above) confirmed the fixes held up and surfaced two more items, fixed in 274374c:
socketOptionsis now derived from the redux auth token instead of a one-timelocalStorageread, making the token a dependency of the connect effect: logout re-runs it, disconnecting the old socket. This also removes the dual token source (redux vs. mount-timelocalStorage).SessionQueueStatusinvalidation: the per-user counts now arrive in the event's embeddedqueue_status; the refetch is kept because it refreshes.processor(which the optimistic write never touches) and reconciles event drift — the comment previously suggested it existed for the counts, inviting removal as "redundant".Testing
tests/app/test_invocation_event_socketio.py— backend routing for invocation events.tests/app/routers/test_multiuser_authorization.py— sanitized-companion routing for status changes, retries, cancels, and clears (full events stay private; companions carry no identifying data; owners/admins skipped; full events delivered in a single emit).tests/app/services/session_queue/test_session_queue_bulk_cancel_events.py— bulk cancel/delete emitqueue_items_canceledgrouped by owner (regression test for the stale-badge bug), anddelete_by_destinationexcludes the separately-canceled current item.tests/app/services/session_queue/test_session_queue_status_user_scoping.py— per-user counts.eventScope.test.ts,onInvocationComplete.test.ts(own path auto-switches; foreign path invalidates only, never fetches DTOs, never touches selection/progress),setEventListeners.test.ts(cross-user isolation: foreign events don't reach the coordinator/node states/progress; sanitized and foreign status changes are invalidate-only with id-scoped tags for admin full events; single-user mode unaffected),queueStatusEvents.test.ts.Follow-ups (out of scope)
sanitize_queue_item_for_userplus themodel_copy(update=...)companion blocks insockets.py) into.sanitized()methods on the event models, so a newly added sensitive field cannot silently leak from a missed site.workflowExecutionCoordinator's per-item owner map is now mostly a guard (foreign events are filtered upstream); it could be simplified in a later PR — note thequeue_clearedlistener is deliberately not scope-filtered, which is what keeps the map load-bearing today.🤖 Generated with Claude Code