Conversation
…n close() TSan caught a data race: Server.close() could return, and the caller could then close monitor sockets, while the monitor's background executor was still mid socket.recv() on the same connection. gc_safe_close() only signals the executor to stop (it must stay non-blocking since it also runs from a GC weakref callback); it never waited for the executor to actually exit before proceeding to reset the pool. Join the executor with the same 1-second bound already used by periodic_executor.py's shutdown path in the three affected close() overrides (MonitorBase, Monitor, _RttMonitor) before touching the pool.
…or join MonitorBase.close()'s join only ever runs for SrvMonitor, which has no pool/socket to race with, so it just added a guaranteed timeout to every mongodb+srv:// client close; drop it and leave the joins in Monitor.close() and _RttMonitor.close(), the classes that actually own the raced socket. Also wake() the executor there before joining so the thread rechecks the stop flag sooner instead of waiting out a full sleep chunk.
…ng GC internals
The TSan task LD_PRELOADed libtsan.so onto a prebuilt, uninstrumented free-threaded interpreter, which reported false races in CPython's own free-threading internals because TSan cannot see synchronization in code compiled without -fsanitize=thread. It now builds CPython 3.14 from source with --with-thread-sanitizer, matching CPython's own CI, so all five suppressions come out. pip keeps build isolation so it resolves hatchling's build dependencies itself; CFLAGS and LDFLAGS are shell env vars, which the isolated build subprocess inherits. The task gets a 7200 second exec timeout because the source build does not fit in the project-wide 3600.
The suppressions file was emptied on the assumption that upstream's Tools/tsan/suppressions_free_threading.txt is empty for an instrumented build. That is true on CPython main but not on the pinned v3.14.0 tag, where it carries 24 entries that upstream's own TSan CI needs. Build a combined file from the cloned tag's copy plus this repo's additions and point TSAN_OPTIONS at that, with handle_segv=0 to match upstream. Also fix the interpreter glob, which missed the real python3.14td name that --disable-gil plus --with-pydebug produces; extend apt-get to CPython's official dependency list; and abort before the 20-30 minute build when the OpenSSL headers pip needs are missing.
Monitor.close() and _RttMonitor.close() no longer block. Joining there burned up to a second while Topology.close() and _update_servers() held the topology lock, stalling every thread in server selection. Instead, extend the existing _monitor_tasks deferral to sync: closed monitors are queued under the lock and joined by cleanup_monitors() once it is released, from Topology.close() and the next select_servers(). Monitor.join() and cleanup_monitors() drop asyncio.gather, which synchro could not translate and which left a broken call in the generated sync monitor.
MongoClient.close() now waits (bounded) for monitor threads to stop before returning, fixing a real race between monitor threads and pool teardown. Document the resulting latency change for users.
A real TSan CI run finished in 16m3s, well under the project's 60 minute default. Drop the 7200s override and the comment justifying it.
… packages Production Evergreen hosts for this task can't run apt-get install. Drop both calls and let the existing OpenSSL header check fail fast when the headers aren't already on the host.
The free-threaded GC's delete_garbage path and a worker thread's deferred refcount merge can both enter _memory_release for the same memoryview. Both stacks are entirely within CPython.
The C-specific tests skip when has_c() is False, so a silently broken rebuild would report a false-green task.
clang's -print-file-name=libasan.so can return a bare name or linker script that ld.so silently refuses to preload, leaving the instrumented extensions unable to import. Resolve real ELF candidates instead, and preload only the ASan runtime: it carries the UBSan handlers, and preloading libubsan_standalone alongside it aborts during init when the two sanitizer_common copies fight over sigaction.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PYTHON-6048
Changes in this PR
Adds a
sanitizers-ubuntu-22Evergreen variant so memory-safety, undefined-behavior, and data-race bugs in the C extensions are caught in CI instead of only in manual runs.RawBSONDocumentand buffer-lifetime edge cases in the C decoder's error paths.Test Plan
test-sanitizer-asanwith the new tests, no sanitizer diagnostics).just lintandjust typingpass.Checklist
Checklist for Author
_memory_releasememoryview race against CPython.Checklist for Reviewer