Skip to content

Stop dropping messages sent after their session was deleted - #67

Draft
sohanmaheshwar wants to merge 1 commit into
mainfrom
fix/recreated-session-drops-message
Draft

sohanmaheshwar wants to merge 1 commit into
mainfrom
fix/recreated-session-drops-message

Conversation

@sohanmaheshwar

Copy link
Copy Markdown

Motivation

In Slack, a user's DM session was deleted, and their next DM got no reply.

channelsd names a session from its (channel, key), so a DM always maps to the same session name. The memory scope is keyed by that name, not by the CR's UID, and the transcript outlives the CR on purpose (append-only, signed audit evidence). So the next message on the same key minted a new session into its predecessor's transcript. The runner saw that turn 0 already existed, treated the session as a resume, and never placed the new message. The only trace was the runner's cold-start: skipped (resume — initial prompt already in memory) log line. Slack had already shown "starting…". If the old conversation ended on a tool result rather than a reply, the old conversation ran again instead.

Repro: DM the agent, kubectl delete agentsession <name>, DM it again. Deleting the session by hand and the archive sweep followed by session GC both end the same way.

That breaks the no-silent-drop rule in CLAUDE.md. It also has a security side, which is narrower than "data leaks to someone else". In a DM the key is the same user, and OAP already lets a thread be handed to another user in some cases. What is lost is the set of rules for continuing a conversation. While a session's CR exists, lifecycle.ContinuationDisposition refuses an inbound to a policy-halted or held session, and a finished session's follow-up starts a new session that inherits only under the takeover rules. Those rules read the CR. Once it was gone, nothing applied them.

Summary of changes

  • channelsd's fresh-session path picks its name with freshSessionName. It walks makeSessionName's base, then base-g2, base-g3 and so on, up to 512. It takes the first name whose memory scope holds no turns.
    • The walk is deterministic, so concurrent replicas still converge on one session through AlreadyExists.
    • Each step reads turns before the CR, so a replica that sees another replica's just-written turns also sees its CR and joins it.
    • A session that is being deleted counts as a predecessor.
    • A memory read error fails closed (InternalError, no session minted), like sweepBeforeMint. The check runs after the thread-ownership guard.
    • The longest name fits the 63-byte Job-name budget (ws-apply-<session>).
  • Follow-ups still route by the channel labels, which the successor carries.
  • Nothing in memory is deleted or rewritten. Audit chains and oap audit verify are unchanged.
  • New bronze bundle recreated-session-starts-fresh, plus bundle-format additions it needs:
    • userTurns[].deleteSessionFirst
    • expect.requestNotContains, which steelthread's expectAssertsNothing now counts as an assertion
    • Harness.DeleteSession

A design choice for reviewers

This picks fresh start for a message that arrives after its session was deleted. The design pulls both ways:

  • Continuity: while the CR exists, continuity is intended. An archived session resumes ("The user is still in the thread and expects to continue"), and a DM is one ongoing session per user.
  • Fresh start: everything tied to a session incarnation (lifecycle.ForIncarnation, sweepBeforeMint) treats a reused name as a new session, and the continuation rules cannot run without the CR.

The cost: a DM user whose session was garbage-collected gets a session with no memory of the old conversation. Before this PR, they got no reply at all.

Alternatives considered

  • Place the new message as an inbox turn on the old transcript, to keep continuity. It would skip the continuation rules above, since there is no CR for them to read.
  • Delete or rewrite the old scope. The transcript and audit kinds are append-only evidence by design.
  • Random or UID-based session names. Deterministic names are what make concurrent channelsd replicas converge on one session.
  • A runner-side fix that ties turn-0 placement to the CR's UID (lifecycle.ForIncarnation already separates incarnations in a name-keyed scope). It would cover every session creator, not just channelsd, but it is a larger change to the runner's resume logic, so it is left as a follow-up.

Not covered (follow-ups)

Webhook triggers are covered: they enter through the same Deliver path. Not covered:

  • Messages during deletion. A message that arrives while a session is being deleted (its finalizer is running) still finds that session: correlateSessions matches by label and does not check DeletionTimestamp. The message is delivered into a session that is about to vanish.
  • Sessions created outside channelsd can still start on a predecessor's transcript: kubectl/oap, scheduled goal sessions (named from the occurrence digest, goals.SessionName), and fork, inherit and takeover children, whose names are deterministic.
  • The predecessor's per-session Secret. The controller adopts it (it should check IsControlledBy). A fast delete-and-apply hands the successor the old memory token and audit seed until owner-reference GC runs.

Provenance

  • Author (person, or model and version): Claude Opus 5.5 (claude-opus-5-5), directed by Sohan Maheshwar
  • Harness or tooling, with version: Claude Code 2.1.311
  • Person who read the diff: not yet. This PR stays a draft until someone has.

Ship gate

  • mage test:unit
  • mage test:integration
  • mage test:e2e

Results (branch head b3f9a9e):

mage test:unit         exit 0  567 packages ok, 0 failed
mage test:integration  exit 0  506 packages ok, 0 failed
mage test:e2e          exit 0   64 packages ok, 0 failed

The first integration run had one failure, TestLiveOpenFrameCarriesTheCurrentView
("i/o timeout"), on a machine at load ~12. The test is in a
package this PR does not touch. It passed 3/3 when run alone, and a full
integration rerun at the same head was green (above).

Regeneration

  • Changed a +kubebuilder:rbac marker or a CRD-shaping field: n/a
  • Edited anything under config/**: n/a
  • Changed a cobra command or a CRD schema: n/a
  • Formatting: mage fmt:check covers only Markdown and TypeScript, and this PR changes neither (the bundle is JSON). gofmt -l on the changed Go files is clean.

Coverage

  • This changes user-visible behavior, and a bronzethread bundle exercises it: test/e2e/bronzethread/testdata/recreated-session-starts-fresh/. Its assertions are on what the second session's model request contains, not on the agent's reply. With the fix reverted it fails: the second session's request opens on the first session's message ("list the companies") instead of the new one ("list the companies again"), and the first session's tool output ("Acme Payroll", "acquisition target") reaches the model.
  • This touches session creation, not the SpiceDB schema or the approver model. mage test:integration + mage test:e2e are green.

Before requesting review

  • The PR holds a single change.
  • A person has read every line of the diff.

… name

A channelsd session's name is derived from (channel, key), and its
memory scope is keyed by that name, not the CR's UID. The transcript
outlives the CR on purpose (append-only, signed audit evidence). So
when a session was deleted, by hand or by the archive sweep followed by
session GC, the next message on the same key minted a session into its
predecessor's transcript. The runner read that as a resume: turn 0
already existed, so the new message (spec.prompt) was never placed. A
predecessor that ended on the agent's reply parked without a word; one
that ended on a tool result ran the old conversation again. Either way
the message was lost, and the only trace was the runner's "cold-start:
skipped" log line; Slack had already shown "starting". Seen live as a
Slack DM that got no reply after the user's DM session was deleted.

The old conversation also came back without the rules that govern
continuing one. While a session's CR exists,
lifecycle.ContinuationDisposition refuses an inbound to a policy-halted
or held session, and a finished session's follow-up starts a new
session that inherits only under the takeover rules. Those rules read
the CR. Once it is gone, nothing applied them.

This fix chooses a fresh start, which the design leaves open. While the
CR exists, continuity is intended: an archived session resumes, and a
DM is one ongoing session per user. But everything tied to a session
incarnation (lifecycle.ForIncarnation, sweepBeforeMint) treats a reused
name as a new session, and the continuation rules cannot run without
the CR. So a message after deletion starts a new session. The cost: a
DM user whose session was garbage-collected gets a session with no
memory of the old conversation. Before this, they got no reply.

The fresh-session path now picks its name with freshSessionName. It
walks makeSessionName's base, then base-g2, base-g3, ... (at most 512),
and takes the first generation whose memory scope holds no turns.
- The walk is deterministic, so concurrent replicas still collapse onto
  one session through AlreadyExists.
- Each step reads turns before the CR. A replica that sees another
  replica's just-written turns therefore also sees its CR and joins it.
- A session that is being deleted counts as a predecessor.
- Follow-ups still route by the channel labels, which the successor
  carries.
- A memory read error fails closed (InternalError, no session minted),
  like sweepBeforeMint, and runs after the thread-ownership guard.
- The longest name stays within the 63-byte Job-name budget
  ("ws-apply-<session>").

Nothing in memory is deleted or rewritten. Audit chains and `oap audit
verify` are unchanged.

The bronze bundle recreated-session-starts-fresh covers it end to end:
user turn, session deleted, user turn on the same key. The second
session's first request opens on the new message, and the first
session's tool output appears nowhere in its context. Supporting bundle
format additions:
- userTurns[].deleteSessionFirst
- expect.requestNotContains, which steelthread's expectAssertsNothing
  now counts as an assertion
- Harness.DeleteSession

Webhook triggers are covered: they enter through the same Deliver path.

Not covered:
- A message that arrives while a session is being deleted (its
  finalizer is running) still finds it: correlateSessions matches by
  label and does not check DeletionTimestamp, so the message is
  delivered into a session that is about to vanish.
- Sessions created outside channelsd can still start on a
  predecessor's transcript. That includes kubectl/oap, scheduled goal
  sessions (named from the occurrence digest, goals.SessionName), and
  fork, inherit and takeover children, whose names are deterministic.
  A runner-side fix should tie turn-0 placement to the CR's UID;
  lifecycle.ForIncarnation (pkg/memory/kinds/lifecycle/order.go:45-77)
  already separates incarnations in a name-keyed scope. The controller
  should also stop adopting a predecessor's per-session Secret (check
  IsControlledBy): today a fast delete and apply hands the successor
  the old memory token and audit seed until owner-reference GC runs.
@vercel

vercel Bot commented Nov 12, 2026

Copy link
Copy Markdown

Deployment failed for project openagentprimitives with the following error:

You don't have permission to create a Preview Deployment for this Vercel project: openagentprimitives.

View Documentation: https://vercel.com/docs/accounts/team-members-and-roles

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant