Skip to content

DatabaseSessionService raises false StaleSessionError when two events share a timestamp #7276

Description

@rishwanth-th

🔴 Required Information

Describe the Bug:

DatabaseSessionService.append_event raises StaleSessionError ("The session has
been modified in storage since it was loaded") when there is only one writer. It
happens when an event has the same timestamp, to the microsecond, as the
previous event and carries a state change.

Cause: append_event sets storage_session.update_time to the event's timestamp,
and get_update_marker() (that timestamp in microseconds) is the revision marker for
the stale-writer check. The column is also declared
onupdate=func.now() (schemas/v1.py, StorageSession.update_time). When the
new timestamp equals the stored one, SQLAlchemy sees no change to update_time and
leaves it out of the UPDATE. The UPDATE still runs because state changed, so
onupdate fires and the database's now() is written instead: whole seconds on
SQLite (…:37.000000), the transaction start time on Postgres. The marker kept on
the in-memory session (read before commit) no longer matches storage, so the next
append_event on that session fails.

Equal timestamps are common with coarse clocks: Python 3.12 on Windows uses
GetSystemTimeAsFileTime (15.6 ms by default), so a workflow that emits several
events per step hits this in most runs. We hit it running a graph Workflow with
RequestInput pause/resume and ResumabilityConfig(is_resumable=True).

This is the documented setup: the sessions page shows
DatabaseSessionService(db_url="sqlite+aiosqlite:///./my_agent_data.db") as the
persistent-storage example. SqliteSessionService, which does not have the problem,
is not mentioned in the docs.

Steps to Reproduce:

  1. pip install "google-adk[db]==2.9.2" (plus asyncpg for Postgres)
  2. Save the script below as repro.py
  3. python repro.py "sqlite+aiosqlite:///./repro.db" or
    python repro.py "postgresql+asyncpg://user:pass@localhost:5432/db"
  4. The third append_event raises StaleSessionError

Expected Behavior:

Appends from the only writer succeed, whatever the event timestamps.

Observed Behavior:

google.adk.errors._stale_session_error.StaleSessionError: The session has been modified in storage since it was loaded. Please reload the session before appending more events.

The stored update_time after the second append is 2026-09-25T04:37:37.000000
(SQLite) while the session's marker is 2026-09-25T04:37:37.847349.

Environment Details:

  • ADK Library Version: 2.9.2 (also present on main as of 2026-09-25)
  • Desktop OS: Windows 11; the Postgres run also from Linux (Docker)
  • Python Version: 3.12.14
  • SQLAlchemy 2.0.54, aiosqlite 0.22.1, asyncpg 0.31.0, Postgres 17.11

Model Information:

  • Are you using LiteLLM: Yes (not involved in the bug)
  • Which model is being used: Azure OpenAI gpt-5.4 (not involved)

🟡 Optional Information

Minimal Reproduction Code:

import asyncio
import sys
import time

from google.adk.events import Event, EventActions
from google.adk.sessions import DatabaseSessionService


async def main(db_url: str) -> None:
    service = DatabaseSessionService(db_url=db_url)
    session = await service.create_session(app_name="repro", user_id="u")

    ts = time.time()
    await service.append_event(session, Event(author="a", timestamp=ts))
    # Same timestamp as the previous event, with a state change.
    await service.append_event(
        session,
        Event(author="a", timestamp=ts, actions=EventActions(state_delta={"k": 1})),
    )
    # Raises StaleSessionError.
    await service.append_event(session, Event(author="a", timestamp=ts + 1))
    print("no error")


asyncio.run(main(sys.argv[1]))

How often has this issue occurred?:

  • Always (100%) with the script above, on SQLite and Postgres.
  • In a real workflow it depends on the clock. Start + resume cycles of a
    two-step Workflow, with ADK's clock set to a 15.625 ms tick through
    google.adk.platform.time.set_time_provider: 92/100 cycles failed on SQLite and
    82/100 on Postgres 17. On Linux with a 1 ns clock: 0/300 on Postgres.

Additional Context:

Possible fixes:

  • Drop onupdate=func.now() from update_time: append_event always sets it
    explicitly, and create_session already has default.
  • Or mark the attribute modified (sqlalchemy.orm.attributes.flag_modified(storage_session, "update_time"))
    so the explicit value is always written.
  • Document SqliteSessionService for local SQLite use.

User-side workaround: subclass DatabaseSessionService and bump
event.timestamp to one microsecond after session.last_update_time when it is
not strictly later. The comparison must use datetime.fromtimestamp, as
append_event does; comparing round(ts * 1e6) disagrees on half-microsecond
floats.

Related but different: #6943 (SQL Server precision), #5085 (MySQL timezone), #3717.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions