fix(tracing): serialize concurrent trace finalization - #6713
Conversation
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
AI assistance is disclosed in the PR body. GitHub does not grant this fork contributor permission to set upstream labels; could a maintainer please add the required |
Summary
TraceBatchManager.finalize_batch()performs an irreversible sequence — snapshotthe event buffer, send it, finalize the backend batch, then clear local state —
but only the backend-finalization step ran under a lock. Two overlapping callers
could therefore send the same event snapshot before either one cleared the
buffer.
This change holds
_finalize_lockacross the whole publicfinalize_batch()transaction. The lock becomes an
RLockrather than moving the existing one,because
FirstTimeTraceHandler.handle_execution_completion()calls_finalize_backend_batch()directly and must keep its current serialization.The event bus can execute lifecycle handlers concurrently, while the singleton
trace listener calls
finalize_batch()from terminal crew handlers and itssignal-flush handler. A second finalizer now serializes behind the first,
including its existing backend I/O, instead of racing it.
This fixes duplicate sends from overlapping local finalizers. It does not claim
exactly-once delivery across an ambiguous network response. There is no public
API or dependency change.
Reproduction
lib/crewai/tests/tracing/test_tracing.py::TestTraceListenerSetup::test_finalize_batch_sends_events_once_when_called_concurrentlyraces two public finalizers around a successful event send. On
mainit fails:The test captures worker exceptions, so a thread error cannot let it pass
silently.
Verification
The new test was also run twelve times in isolation with no flake.
AI assistance disclosure
This PR was prepared with AI coding assistance and reviewed by the contributor
before submission. GitHub does not permit this fork contributor to apply labels
in the upstream repository, so maintainer assistance is needed to add the
required
llm-generatedlabel.