feat(diskann): Add Windows DiskANN support with asynchronous I/O - #625
Open
iaojnh wants to merge 148 commits into
Open
feat(diskann): Add Windows DiskANN support with asynchronous I/O#625iaojnh wants to merge 148 commits into
iaojnh wants to merge 148 commits into
Conversation
… into refactor/diskann_dlopen
# Conflicts: # tests/db/index/common/db_proto_converter_test.cc
iaojnh
marked this pull request as draft
August 20, 2026 09:02
iaojnh
marked this pull request as ready for review
August 20, 2026 12:08
richyreachy
reviewed
Aug 25, 2026
richyreachy
reviewed
Aug 25, 2026
richyreachy
reviewed
Aug 25, 2026
richyreachy
reviewed
Aug 25, 2026
egolearner
reviewed
Aug 26, 2026
egolearner
reviewed
Aug 26, 2026
egolearner
reviewed
Aug 26, 2026
# Conflicts: # src/db/index/common/schema.cc
egolearner
reviewed
Aug 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR enables DiskANN on 64-bit Windows using aligned, unbuffered asynchronous reads backed by Windows I/O completion ports (IOCP).
It also fixes a Windows file-handle interaction that significantly reduced random-read throughput, hardens asynchronous I/O and resource lifetimes, and ensures vector fetching remains safe after the source streamer or its storage segments are released.
Root cause and fix
DiskANN issues aligned 4 KiB random reads. A standalone IOCP reader reached about 29K IOPS, while the same reader inside a loaded searcher reached only about 3.8K IOPS.
An ABBA test isolated the cause:
FileReadStorageretained an ordinary buffered file handle alongside DiskANN'sFILE_FLAG_NO_BUFFERING | FILE_FLAG_OVERLAPPEDhandles.The fix loads the metadata needed by search, closes the shared buffered handle, releases the storage segments, and then opens DiskANN's aligned IOCP reader. Vector fetching and streamer providers use the same reader without depending on the released storage.
Main changes
FILE_FLAG_NO_BUFFERING | FILE_FLAG_OVERLAPPED.memcpy.windows_overlappedthrough the existing I/O backend APIs.io_uring, dynamically loadedlibaio, or synchronouspreadfallback.Performance validation
Cohere 1M, 768 dimensions, cosine distance, FP32, 1,000 queries, Windows Server 2022, with the same 40,088 hot nodes cached in both runs:
The full FP32/FP16 parameter sweep retained the expected recall range and reached about 27.5K process read operations per second. These are process I/O counters rather than device-wide hardware IOPS.
Compatibility
cache_node_numconfiguration remains unchanged.Test coverage
Tests cover:
The affected mixed-reducer target also builds successfully with warnings treated as errors.