Skip to content

docs(pipecat-cloud): clarify that sequential sessions reuse the same process (T-2660) - #1049

Draft
jamsea wants to merge 1 commit into
mainfrom
docs/pcc-session-isolation-t2660
Draft

docs(pipecat-cloud): clarify that sequential sessions reuse the same process (T-2660)#1049
jamsea wants to merge 1 commit into
mainfrom
docs/pcc-session-isolation-t2660

Conversation

@jamsea

@jamsea jamsea commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Found while working Daily support ticket T-2660: a customer saw memory grow call after call on a warm Pipecat Cloud instance.

The gap

Nothing public said that a warm instance which picks up a new session is running the same process that handled the previous session. Worse, the Session API page said the opposite:

Different sessions are separate pods and don't share state.

That sentence is about routing, and it is true for two sessions running at the same time. But it reads as a promise of a fresh process per session, which is not what happens for sequential sessions on a reused warm instance. The scaling page says an instance "is returned to the pool and can immediately serve another session" and never says the process survives.

What changed

pipecat-cloud/guides/session-api.mdx

Split the "Session scope" bullet in two. The first keeps the real guarantee: two sessions running at the same time are on separate instances, so a request for one never reaches the other. The second says plainly that being routed separately is not the same as getting a fresh process, and links to the new scaling section.

pipecat-cloud/fundamentals/scaling.mdx

New subsection under "Updating scaling configuration" covering:

  • A min-agents or max-agents change only resizes the pool. It does not replace the instances you already have.
  • An instance that keeps running keeps its process, so state the previous session did not release is still in memory.
  • The practical fix: release per-session references when the session ends (set module-level references back to None, close clients, stop tasks and threads).
  • How to get a genuinely fresh set of instances: pipecat cloud deploy [agent-name] --force, with a heads-up that in-flight sessions can be cut off.
  • A pointer to the readyz() override for taking one instance out of rotation, and a note that it does not restart the process.

Verified against source

Checked on current main of each repo rather than from memory:

  • The base image runs one long-lived server and dispatches each session into that same process as a background task. Nothing exits or restarts the process when a session ends, and module-level state persists.
  • A deploy that changes only the autoscaling values takes the version-only path: no new ReplicaSet, no pod replacement.
  • The CLI --force flag sets the force-redeploy input, which creates a new deployment and therefore replaces pods.
  • Evidence from the customer's namespace: one instance started once, never restarted its container, and served 7 sessions across roughly 2.5 days.

Note on overlap with #1028

Draft PR #1028 (T-3090) covers the same underlying finding from the instance-lifecycle and readyz() angle, and adds an "Instance reuse and memory" section to scaling.mdx. These edits touch different sections of that file and do not conflict textually, but there is real overlap in what they explain. The two should be deduped before either one lands. Landing #1028 first and rebasing this onto it is probably cleanest.

Draft on purpose: not ready for review, pending that dedupe decision.

…process (T-2660)

Found while working Daily support ticket T-2660: a customer saw memory
grow call after call on a warm Pipecat Cloud instance.

Two gaps, both about session isolation:

1. The Session API page said "Different sessions are separate pods and
   don't share state." That is true for two sessions running at the same
   time, but it reads as a promise of a fresh process per session. It is
   not. Split the bullet so the concurrent-session guarantee stays and
   the sequential-session behavior is stated plainly.
2. The scaling page did not say how to get fresh instances. Added a
   subsection under "Updating scaling configuration": a min-agents or
   max-agents change only resizes the pool and does not replace running
   instances, `pipecat cloud deploy <agent> --force` does, and readyz()
   takes an instance out of rotation without restarting its process.
   Also spells out releasing per-session references.

Note: draft PR #1028 (T-3090) covers the same underlying finding from the
instance-lifecycle angle and adds an "Instance reuse and memory" section
to the same page. These edits touch different sections and do not
conflict textually, but the two should be deduped before either lands.
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

🔍 Mintlify preview for this branch: https://daily-docs-pcc-session-isolation-t2660.mintlify.site

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant