Repository navigation
Stop dropping messages sent after their session was deleted - #67
Draft
sohanmaheshwar wants to merge 1 commit into
Draft
sohanmaheshwar wants to merge 1 commit into
sohanmaheshwar wants to merge 1 commit into
Conversation
… name
A channelsd session's name is derived from (channel, key), and its
memory scope is keyed by that name, not the CR's UID. The transcript
outlives the CR on purpose (append-only, signed audit evidence). So
when a session was deleted, by hand or by the archive sweep followed by
session GC, the next message on the same key minted a session into its
predecessor's transcript. The runner read that as a resume: turn 0
already existed, so the new message (spec.prompt) was never placed. A
predecessor that ended on the agent's reply parked without a word; one
that ended on a tool result ran the old conversation again. Either way
the message was lost, and the only trace was the runner's "cold-start:
skipped" log line; Slack had already shown "starting". Seen live as a
Slack DM that got no reply after the user's DM session was deleted.
The old conversation also came back without the rules that govern
continuing one. While a session's CR exists,
lifecycle.ContinuationDisposition refuses an inbound to a policy-halted
or held session, and a finished session's follow-up starts a new
session that inherits only under the takeover rules. Those rules read
the CR. Once it is gone, nothing applied them.
This fix chooses a fresh start, which the design leaves open. While the
CR exists, continuity is intended: an archived session resumes, and a
DM is one ongoing session per user. But everything tied to a session
incarnation (lifecycle.ForIncarnation, sweepBeforeMint) treats a reused
name as a new session, and the continuation rules cannot run without
the CR. So a message after deletion starts a new session. The cost: a
DM user whose session was garbage-collected gets a session with no
memory of the old conversation. Before this, they got no reply.
The fresh-session path now picks its name with freshSessionName. It
walks makeSessionName's base, then base-g2, base-g3, ... (at most 512),
and takes the first generation whose memory scope holds no turns.
- The walk is deterministic, so concurrent replicas still collapse onto
one session through AlreadyExists.
- Each step reads turns before the CR. A replica that sees another
replica's just-written turns therefore also sees its CR and joins it.
- A session that is being deleted counts as a predecessor.
- Follow-ups still route by the channel labels, which the successor
carries.
- A memory read error fails closed (InternalError, no session minted),
like sweepBeforeMint, and runs after the thread-ownership guard.
- The longest name stays within the 63-byte Job-name budget
("ws-apply-<session>").
Nothing in memory is deleted or rewritten. Audit chains and `oap audit
verify` are unchanged.
The bronze bundle recreated-session-starts-fresh covers it end to end:
user turn, session deleted, user turn on the same key. The second
session's first request opens on the new message, and the first
session's tool output appears nowhere in its context. Supporting bundle
format additions:
- userTurns[].deleteSessionFirst
- expect.requestNotContains, which steelthread's expectAssertsNothing
now counts as an assertion
- Harness.DeleteSession
Webhook triggers are covered: they enter through the same Deliver path.
Not covered:
- A message that arrives while a session is being deleted (its
finalizer is running) still finds it: correlateSessions matches by
label and does not check DeletionTimestamp, so the message is
delivered into a session that is about to vanish.
- Sessions created outside channelsd can still start on a
predecessor's transcript. That includes kubectl/oap, scheduled goal
sessions (named from the occurrence digest, goals.SessionName), and
fork, inherit and takeover children, whose names are deterministic.
A runner-side fix should tie turn-0 placement to the CR's UID;
lifecycle.ForIncarnation (pkg/memory/kinds/lifecycle/order.go:45-77)
already separates incarnations in a name-keyed scope. The controller
should also stop adopting a predecessor's per-session Secret (check
IsControlledBy): today a fast delete and apply hands the successor
the old memory token and audit seed until owner-reference GC runs.
|
Deployment failed for project openagentprimitives with the following error: View Documentation: https://vercel.com/docs/accounts/team-members-and-roles |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
In Slack, a user's DM session was deleted, and their next DM got no reply.
channelsd names a session from its (channel, key), so a DM always maps to the same session name. The memory scope is keyed by that name, not by the CR's UID, and the transcript outlives the CR on purpose (append-only, signed audit evidence). So the next message on the same key minted a new session into its predecessor's transcript. The runner saw that turn 0 already existed, treated the session as a resume, and never placed the new message. The only trace was the runner's
cold-start: skipped (resume — initial prompt already in memory)log line. Slack had already shown "starting…". If the old conversation ended on a tool result rather than a reply, the old conversation ran again instead.Repro: DM the agent,
kubectl delete agentsession <name>, DM it again. Deleting the session by hand and the archive sweep followed by session GC both end the same way.That breaks the no-silent-drop rule in CLAUDE.md. It also has a security side, which is narrower than "data leaks to someone else". In a DM the key is the same user, and OAP already lets a thread be handed to another user in some cases. What is lost is the set of rules for continuing a conversation. While a session's CR exists,
lifecycle.ContinuationDispositionrefuses an inbound to a policy-halted or held session, and a finished session's follow-up starts a new session that inherits only under the takeover rules. Those rules read the CR. Once it was gone, nothing applied them.Summary of changes
freshSessionName. It walksmakeSessionName's base, thenbase-g2,base-g3and so on, up to 512. It takes the first name whose memory scope holds no turns.sweepBeforeMint. The check runs after the thread-ownership guard.ws-apply-<session>).oap audit verifyare unchanged.recreated-session-starts-fresh, plus bundle-format additions it needs:userTurns[].deleteSessionFirstexpect.requestNotContains, which steelthread'sexpectAssertsNothingnow counts as an assertionHarness.DeleteSessionA design choice for reviewers
This picks fresh start for a message that arrives after its session was deleted. The design pulls both ways:
lifecycle.ForIncarnation,sweepBeforeMint) treats a reused name as a new session, and the continuation rules cannot run without the CR.The cost: a DM user whose session was garbage-collected gets a session with no memory of the old conversation. Before this PR, they got no reply at all.
Alternatives considered
lifecycle.ForIncarnationalready separates incarnations in a name-keyed scope). It would cover every session creator, not just channelsd, but it is a larger change to the runner's resume logic, so it is left as a follow-up.Not covered (follow-ups)
Webhook triggers are covered: they enter through the same
Deliverpath. Not covered:correlateSessionsmatches by label and does not checkDeletionTimestamp. The message is delivered into a session that is about to vanish.oap, scheduled goal sessions (named from the occurrence digest,goals.SessionName), and fork, inherit and takeover children, whose names are deterministic.IsControlledBy). A fast delete-and-apply hands the successor the old memory token and audit seed until owner-reference GC runs.Provenance
claude-opus-5-5), directed by Sohan MaheshwarShip gate
mage test:unitmage test:integrationmage test:e2eResults (branch head
b3f9a9e):Regeneration
+kubebuilder:rbacmarker or a CRD-shaping field: n/aconfig/**: n/amage fmt:checkcovers only Markdown and TypeScript, and this PR changes neither (the bundle is JSON).gofmt -lon the changed Go files is clean.Coverage
test/e2e/bronzethread/testdata/recreated-session-starts-fresh/. Its assertions are on what the second session's model request contains, not on the agent's reply. With the fix reverted it fails: the second session's request opens on the first session's message ("list the companies") instead of the new one ("list the companies again"), and the first session's tool output ("Acme Payroll", "acquisition target") reaches the model.mage test:integration+mage test:e2eare green.Before requesting review