Python: Move SessionStore to core and persist Foundry Responses sessions

[eavanvalkenburg]

Motivation & Context

Core agents need a reusable way to persist complete AgentSession state, while Foundry Responses hosting currently restores transcript history without restoring middleware state, provider state, or service continuation handles. This change moves session storage into core and uses it in Foundry hosting so state survives Responses API calls with explicit per-user isolation.

Description & Review Guide

• What are the major changes? Move the concrete, overridable SessionStore into core; add an experimental msgspec-backed FileSessionStore with JSON and MessagePack formats, atomic replacement, opaque keys encoded to portable filenames, explicit provider-owned custom-state registration, and non-destructive schema/version/decoder failures; make msgspec the FileHistoryProvider default while deprecating custom dumps/loads; and restore/save sessions throughout the Foundry Responses lifecycle. Hosted Responses servers default to the experimental FoundrySessionStore at the session file API path /.sessions/<user-id>/<session-id>.json (stored under $HOME/.sessions), local servers default to the in-memory SessionStore, and session_store= is the override for persisting MAF snapshots outside Foundry. The change also updates hosting imports, samples, package metadata, tests, and ADR-0033.
• What is the impact of these changes? Regular Foundry-hosted agents can retain framework session state across streaming and non-streaming Responses calls, including process restarts. A Foundry session governs platform compute and filesystem lifetime, while a MAF AgentSession carries framework context state; the shared identifier correlates them without equating their semantics. Workflow checkpoints and function approvals retain their existing paths and raw-user partitioning. Consumers of the alpha hosting package now import SessionStore from agent_framework, and core gains a msgspec runtime dependency. Foundry deployment validation used deterministic middleware that increments AgentSession.state["turn_count"]: two calls sharing one hosted session returned SESSION_TURN=1 then SESSION_TURN=2; after stopping the hosted compute while preserving its filesystem, the resumed call returned SESSION_TURN=3. The session file API also exposed the snapshot under /.sessions/<user-id>/<session-id>.json, confirming the intended user partition and filename.
• What do you want reviewers to focus on? The concrete overridable store API, typed serialization and dynamic state registry, Foundry persistence lifecycle on success/failure/cancellation, and the hosted session path and isolation boundaries.

Related Issue

No issue is linked for this draft, as requested.

Contribution Checklist

• The code builds clean without any errors or warnings
• All unit tests pass, and I have added new tests where possible
• The PR follows the Contribution Guidelines
• This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
• This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

[github-actions]

Python Test Coverage Report •

File	Stmts	Miss	Cover	Missing
packages/core/agent_framework
_feature_stage.py	189	10	94%	109, 148, 164, 219, 230, 251, 289, 328, 378, 406
_sessions.py	820	59	92%	157, 169–170, 219, 230, 244, 268, 313, 318, 320, 330, 362, 374, 384, 493, 562–563, 696, 979–983, 998, 1028, 1065–1066, 1080, 1082, 1101, 1103, 1186, 1192, 1226, 1297, 1301, 1311, 1525, 1557–1558, 1563, 1578, 1654–1655, 1657, 1740, 1756, 1835, 1906, 1935, 1954, 1957, 1965–1966, 1978–1979, 1991, 2001, 2031
packages/core/agent_framework/_harness
_memory.py	774	117	84%	75–76, 92, 99, 151, 164, 171–173, 196, 216, 225–229, 292, 298, 333, 414, 422, 424, 428, 431, 435, 530, 542, 560–561, 572–573, 584, 699, 716, 725, 740, 768, 771–773, 783, 805–808, 857, 867, 870, 872, 892, 902, 907, 910, 922, 992, 994, 996, 998, 1000, 1002, 1072, 1089, 1117, 1146, 1154–1156, 1158, 1212, 1238–1241, 1247, 1402, 1405, 1416–1417, 1421, 1424–1426, 1432, 1436, 1441, 1445, 1450, 1454, 1462–1463, 1514, 1517–1520, 1527–1528, 1556, 1560–1568, 1585, 1611–1612, 1615–1616, 1622, 1624, 1629, 1634, 1640
packages/foundry_hosting/agent_framework_foundry_hosting
_invocations.py	45	1	97%	59
_request_context.py	15	0	100%
_responses.py	939	98	89%	203–206, 256, 264–265, 322, 327, 351, 406–407, 421, 424–425, 478, 483–484, 494, 559, 644, 654, 658, 687, 701, 755–756, 790, 793, 799, 801, 878, 965–966, 1084, 1314, 1327, 1818–1820, 1822, 1869–1870, 1872–1873, 1875–1876, 1878–1879, 1884, 1893, 1896–1898, 1900, 1910, 1914, 1927, 2018–2019, 2034–2035, 2037, 2042–2046, 2048, 2055–2056, 2058–2059, 2065, 2067–2071, 2078, 2084, 2106, 2112, 2118, 2120, 2122–2125, 2133, 2135, 2196–2198, 2212–2213, 2216–2217
_session_store.py	26	0	100%
packages/hosting/agent_framework_hosting
_state.py	104	9	91%	137, 150–152, 230, 260, 273–275
TOTAL	46019	4498	90%

Python Unit Test Overview

Tests	Skipped	Failures	Errors	Time
9504	34 💤	0 ❌	0 🔥	2m 24s ⏱️

[Copilot]

Pull request overview

Moves Python session persistence primitives into agent-framework-core and updates Foundry Responses hosting to persist/restore full AgentSession state (not just transcript history), including a new msgspec-backed file store and explicit per-user storage isolation for hosted scenarios.

Changes:

• Introduces experimental SessionStore (in-memory) and FileSessionStore (JSON/MessagePack, atomic replace) in core, plus a stricter session-state type registry via register_state_type.
• Switches FileHistoryProvider default JSON codec to msgspec (and adds optional MessagePack), deprecating custom JSON dumps/loads hooks.
• Updates Foundry Responses hosting to persist sessions across calls, partition durable storage by resolved authenticated identity, and harden isolation/scoping across sessions, checkpoints, and approvals.

Reviewed changes

Copilot reviewed 34 out of 35 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
python/uv.lock	Adds msgspec to the locked dependency set.
python/samples/04-hosting/af-hosting/README.md	Updates sample documentation to reflect core-owned SessionStore.
python/samples/04-hosting/af-hosting/local_telegram/README.md	Updates Telegram session-id format description (no :).
python/samples/04-hosting/af-hosting/local_responses/README.md	Documents use of core FileSessionStore for session snapshots.
python/samples/04-hosting/af-hosting/local_responses/app.py	Wires AgentState to a FileSessionStore for local persistence.
python/samples/02-agents/conversations/file_history_provider.py	Removes optional orjson path; documents msgspec default.
python/samples/02-agents/conversations/file_history_provider_conversation_persistence.py	Removes optional orjson path; documents msgspec default.
python/samples/02-agents/context_providers/simple_context_provider.py	Demonstrates explicit register_state_type for persisted custom state.
python/packages/hosting/tests/hosting/test_state.py	Updates hosting tests for core-owned SessionStore and new ID validation.
python/packages/hosting/README.md	Documents new core session store APIs and ID restrictions.
python/packages/hosting/agent_framework_hosting/_state.py	Removes hosting-owned store implementation; uses core SessionStore and validation.
python/packages/hosting/agent_framework_hosting/init.py	Stops re-exporting SessionStore.
python/packages/hosting-telegram/tests/hosting_telegram/test_parsing.py	Updates tests for new Telegram session-id format.
python/packages/hosting-telegram/README.md	Updates docs to refer to core session stores and new ID format.
python/packages/hosting-telegram/agent_framework_hosting_telegram/_parsing.py	Changes telegram_session_id output to use underscore-delimited IDs.
python/packages/hosting-responses/README.md	Documents session store location in core.
python/packages/foundry_hosting/tests/test_responses.py	Adds/updates tests for session persistence and isolation behavior.
python/packages/foundry_hosting/README.md	Documents hosted persistence behavior, default storage layout, and isolation resolver.
python/packages/foundry_hosting/agent_framework_foundry_hosting/_session_isolation.py	Adds isolation resolver and scoped file session store wrapper.
python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses.py	Implements session persistence + isolation scoping across Responses lifecycle.
python/packages/foundry_hosting/agent_framework_foundry_hosting/init.py	Exposes new isolation-related public types.
python/packages/core/tests/core/test_sessions.py	Adds comprehensive tests for state registry + session/file stores + msgspec history changes.
python/packages/core/tests/core/test_harness_memory.py	Removes custom JSON codec plumbing now that msgspec is default.
python/packages/core/tests/core/test_feature_stage.py	Adds test ensuring internal subclasses don’t warn under experimental features.
python/packages/core/pyproject.toml	Adds msgspec runtime dependency.
python/packages/core/AGENTS.md	Updates package map docs for new session persistence APIs.
python/packages/core/agent_framework/_sessions.py	Adds session stores + registry hardening + msgspec-based history/session serialization.
python/packages/core/agent_framework/_harness/_memory.py	Marks history codec overrides as deprecated.
python/packages/core/agent_framework/_feature_stage.py	Avoids experimental warnings for internal framework subclasses.
python/packages/core/agent_framework/init.pyi	Exports SessionStore/FileSessionStore typing surface.
python/packages/core/agent_framework/init.py	Exports SessionStore/FileSessionStore runtime surface.
python/PACKAGE_STATUS.md	Adds SESSION_STORE feature-stage entry and updates staged-API listing.
python/AGENTS.md	Adds “Changelog Ownership” guidance for Python PRs.
python/.github/skills/agent-framework-py-release/SKILL.md	Documents that release workflow owns python/CHANGELOG.md.
docs/decisions/0033-python-session-store-serialization.md	Adds ADR for session store ownership + msgspec serialization strategy.

[github-actions]

Automated Code Review

Reviewers: 4 | Confidence: 84% | Result: All clear

Reviewed: Correctness, Security Reliability, Test Coverage, Design Approach

---

Automated review by eavanvalkenburg's agents

[github-actions]

Automated Code Review

Reviewers: 4 | Confidence: 85%

✓ Correctness

The implementation is solid overall. The session store, serialization registry, and feature-stage changes are well-tested and handle edge cases properly. One low-severity behavioral asymmetry exists in SessionStore where set() stores by reference while get() deep-copies, creating different mutation semantics than FileSessionStore.

✓ Security Reliability

The implementation is well-structured with proper security controls: path traversal protection (validate_session_id + resolve + is_relative_to), atomic file writes via os.replace, and corruption quarantining. The serialization pipeline correctly chains exceptions from custom decoders into ValueError for quarantine handling. The in-memory SessionStore stores direct references on write (no deep-copy) while deep-copying on read — this is asymetric but explicitly documented ('Reads return independent working copies'). The main reliability note is that SessionStore.set() stores a direct reference, meaning external mutation after set() silently corrupts the stored value, creating a behavioral difference from FileSessionStore which inherently provides write isolation through serialization.

✓ Test Coverage

The test coverage for this PR is generally strong. The new SessionStore, FileSessionStore, register_state_type enhancements, and FileHistoryProvider msgpack support all have meaningful tests with good edge-case coverage (corruption quarantine, concurrent writes, non-finite floats, reserved Windows filenames). Two minor test gaps exist: (1) no test for register_state_type raising when only one of encoder/decoder is provided, and (2) no test for the FileSessionStore quarantine-failure path where the OSError branch is hit. Both are low-priority given the scope of existing coverage. The test coverage for the new session persistence lifecycle in Foundry hosting is solid overall, with tests for success, failure, streaming, multi-turn, file store isolation, and corrupt file recovery. However, there's a notable gap: the finally clause in _handle_inner_agent (line 826-828) that saves the session when the async generator is abandoned mid-stream (client disconnect/cancellation) has no corresponding test. Additionally, the telegram session ID format change is a breaking change without any migration path or backward-compatibility test.

✗ Design Approach

I found one design-level correctness issue in the new in-memory SessionStore: it presents itself as a session snapshot store, but set() keeps the caller's AgentSession by reference instead of capturing a snapshot. That means post-set mutations to the original session silently alter the stored value, and the in-memory base no longer matches the write-time snapshot semantics of FileSessionStore. I found one design issue: the Telegram helper now returns a different session-ID format, which splits durable conversation state across two unrelated keys after upgrade. Because AgentState passes the caller-selected session ID through unchanged to the configured store (python/packages/hosting/agent_framework_hosting/_state.py:169-175, reinforced by python/packages/hosting/tests/hosting/test_state.py:202-226), changing telegram_session_id() from telegram:... to telegram_... makes existing persisted sessions unreachable with no compatibility path. I found one non-blocking design issue in the new benchmark script: although its CLI exposes workload sizes as free-form integers, the verification path assumes at least one message and one custom class state object, so zero-sized runs crash instead of benchmarking an empty-session case or rejecting the input up front.

Flagged Issues

• python/packages/core/agent_framework/_sessions.py:1379-1384,1420-1434: SessionStore documents snapshot-style behavior and deep-copies on read, but set() stores the caller-owned AgentSession by reference. A caller can await store.set(id, session), mutate session.state afterward, and later get(id) will return the mutated data rather than the snapshot that was supposedly stored—diverging from FileSessionStore which inherently snapshots at write time via serialization.
• Changing telegram_session_id() from telegram:<bot_id>:<chat_id> to telegram_<bot_id>_chat_<chat_id> breaks continuity for any app already persisting Telegram sessions, because the helper output is used directly as the storage key via AgentState (python/packages/hosting/agent_framework_hosting/_state.py:169-175) and there is no migration or fallback lookup path.

Suggestions

• Consider adding a test for the register_state_type validation branch at _sessions.py:308 where providing only encoder without decoder (or vice-versa) raises ValueError('State type encoder and decoder must be provided together.'). This is a user-facing validation that has no direct test.
• Consider adding a test for the FileSessionStore quarantine-failure path (_sessions.py:1539-1543) where _quarantine_corrupt_snapshot raises OSError, producing the 'could not be quarantined' error message. The current tests cover successful quarantine and concurrent-replacement cases but not OS-level quarantine failures.

---

Automated review by eavanvalkenburg's agents

[github-actions]

Flagged issue

python/packages/core/agent_framework/_sessions.py:1379-1384,1420-1434: SessionStore documents snapshot-style behavior and deep-copies on read, but set() stores the caller-owned AgentSession by reference. A caller can await store.set(id, session), mutate session.state afterward, and later get(id) will return the mutated data rather than the snapshot that was supposedly stored—diverging from FileSessionStore which inherently snapshots at write time via serialization.

---

Source: automated DevFlow PR review

[github-actions]

Flagged issue

Changing telegram_session_id() from telegram:<bot_id>:<chat_id> to telegram_<bot_id>_chat_<chat_id> breaks continuity for any app already persisting Telegram sessions, because the helper output is used directly as the storage key via AgentState (python/packages/hosting/agent_framework_hosting/_state.py:169-175) and there is no migration or fallback lookup path.

---

Source: automated DevFlow PR review