🔴 Required Information
Describe the Bug:
Session services reload a session's events ordered by (timestamp, id). Event ids
are random (Event.new_id()), so when two events carry the same timestamp they
come back in a random-but-stable order, not in the order they were appended. This
happens in SqliteSessionService (ORDER BY timestamp DESC, id DESC, with a
comment saying the id tie-break makes the order stable across reads) and in
DatabaseSessionService (order_by(timestamp.desc(), id.desc())).
For a resumable graph Workflow this breaks resume. ReplayManager builds the
replay sequence barrier from the reloaded events, so a swapped pair gives the
barrier a completion order the workflow can never follow, and the resume fails
after the 15 s barrier timeout:
RuntimeError: Replay divergence detected: Timed out waiting for sequence key 'wait_test_key@1' to be unblocked.
Equal timestamps are common: Python 3.12 on Windows ticks every 15.6 ms by default,
and fast function nodes emit several events per tick on any OS. It is a different
cause from #7027 (re-emitted outputs reordering the sequence in memory, fixed in
#7028): this one is in storage, and the repro below involves no workflow at all.
Steps to Reproduce:
pip install "google-adk[db]==2.9.2" (plus asyncpg for Postgres)
- Save the script below as
repro.py
python repro.py sqlite, or python repro.py "sqlite+aiosqlite:///./r.db", or
python repro.py "postgresql+asyncpg://user:pass@localhost:5432/db"
- The loaded order is the reverse of the append order
Expected Behavior:
get_session returns events in append order, including events with equal
timestamps.
Observed Behavior:
SqliteSessionService append order: ['b-appended-first', 'a-appended-second']
SqliteSessionService loaded order: ['a-appended-second', 'b-appended-first']
OUT OF ORDER
The same for DatabaseSessionService on SQLite and on Postgres 17.
In a resumable Workflow (a question loop: wait → classify → answer → wait,
SqliteSessionService), forcing ties with a 15.625 ms clock through
google.adk.platform.time.set_time_provider made resume fail with the divergence
above at loop 7; the same workflow with a strictly increasing clock passed every
loop.
Environment Details:
- ADK Library Version: 2.9.2
- Desktop OS: Windows 11; also a Linux container against Postgres 17.11
- Python Version: 3.12.14
- SQLAlchemy 2.0.54, aiosqlite 0.22.1, asyncpg 0.31.0
Model Information:
- Are you using LiteLLM: Yes (not involved)
- Which model is being used: none needed for the repro
🟡 Optional Information
Minimal Reproduction Code:
import asyncio
import sys
import tempfile
import time
from pathlib import Path
from google.adk.events import Event
from google.adk.sessions import DatabaseSessionService
from google.adk.sessions.sqlite_session_service import SqliteSessionService
async def main(target: str) -> None:
if target == "sqlite":
service = SqliteSessionService(db_path=str(Path(tempfile.mkdtemp()) / "r.db"))
else:
service = DatabaseSessionService(db_url=target)
session = await service.create_session(app_name="repro", user_id="u")
ts = time.time()
await service.append_event(session, Event(id="b-appended-first", author="a", timestamp=ts))
await service.append_event(session, Event(id="a-appended-second", author="a", timestamp=ts))
loaded = await service.get_session(app_name="repro", user_id="u", session_id=session.id)
order = [e.id for e in loaded.events]
print("append order: ['b-appended-first', 'a-appended-second']")
print("loaded order:", order)
asyncio.run(main(sys.argv[1]))
How often has this issue occurred?:
- Always (100%) with the script above.
- In a workflow, whenever two events share a timestamp; frequent with coarse clocks.
Additional Context:
Possible fixes: order by insertion rather than by a random id on ties (SQLite
rowid, or an autoincrement / sequence column in DatabaseSessionService), or make
event ids monotonic.
User-side workaround: a strictly increasing clock through
google.adk.platform.time.set_time_provider (no two events in a process share a
timestamp). It does not help across processes.
🔴 Required Information
Describe the Bug:
Session services reload a session's events ordered by
(timestamp, id). Event idsare random (
Event.new_id()), so when two events carry the same timestamp theycome back in a random-but-stable order, not in the order they were appended. This
happens in
SqliteSessionService(ORDER BY timestamp DESC, id DESC, with acomment saying the id tie-break makes the order stable across reads) and in
DatabaseSessionService(order_by(timestamp.desc(), id.desc())).For a resumable graph
Workflowthis breaks resume.ReplayManagerbuilds thereplay sequence barrier from the reloaded events, so a swapped pair gives the
barrier a completion order the workflow can never follow, and the resume fails
after the 15 s barrier timeout:
Equal timestamps are common: Python 3.12 on Windows ticks every 15.6 ms by default,
and fast function nodes emit several events per tick on any OS. It is a different
cause from #7027 (re-emitted outputs reordering the sequence in memory, fixed in
#7028): this one is in storage, and the repro below involves no workflow at all.
Steps to Reproduce:
pip install "google-adk[db]==2.9.2"(plusasyncpgfor Postgres)repro.pypython repro.py sqlite, orpython repro.py "sqlite+aiosqlite:///./r.db", orpython repro.py "postgresql+asyncpg://user:pass@localhost:5432/db"Expected Behavior:
get_sessionreturns events in append order, including events with equaltimestamps.
Observed Behavior:
The same for
DatabaseSessionServiceon SQLite and on Postgres 17.In a resumable
Workflow(a question loop: wait → classify → answer → wait,SqliteSessionService), forcing ties with a 15.625 ms clock throughgoogle.adk.platform.time.set_time_providermade resume fail with the divergenceabove at loop 7; the same workflow with a strictly increasing clock passed every
loop.
Environment Details:
Model Information:
🟡 Optional Information
Minimal Reproduction Code:
How often has this issue occurred?:
Additional Context:
Possible fixes: order by insertion rather than by a random id on ties (SQLite
rowid, or an autoincrement / sequence column inDatabaseSessionService), or makeevent ids monotonic.
User-side workaround: a strictly increasing clock through
google.adk.platform.time.set_time_provider(no two events in a process share atimestamp). It does not help across processes.