Skip to content

feat(diskann): Add Windows DiskANN support with asynchronous I/O - #625

Open
iaojnh wants to merge 148 commits into
alibaba:mainfrom
iaojnh:feat/diskann-support-windows
Open

feat(diskann): Add Windows DiskANN support with asynchronous I/O#625
iaojnh wants to merge 148 commits into
alibaba:mainfrom
iaojnh:feat/diskann-support-windows

Conversation

@iaojnh

@iaojnh iaojnh commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Summary

This PR enables DiskANN on 64-bit Windows using aligned, unbuffered asynchronous reads backed by Windows I/O completion ports (IOCP).

It also fixes a Windows file-handle interaction that significantly reduced random-read throughput, hardens asynchronous I/O and resource lifetimes, and ensures vector fetching remains safe after the source streamer or its storage segments are released.

Root cause and fix

DiskANN issues aligned 4 KiB random reads. A standalone IOCP reader reached about 29K IOPS, while the same reader inside a loaded searcher reached only about 3.8K IOPS.

An ABBA test isolated the cause: FileReadStorage retained an ordinary buffered file handle alongside DiskANN's FILE_FLAG_NO_BUFFERING | FILE_FLAG_OVERLAPPED handles.

Condition IOPS geometric mean Retention
Buffered handle closed 29,299 100%
Buffered handle held open 3,809 13.0%

The fix loads the metadata needed by search, closes the shared buffered handle, releases the storage segments, and then opens DiskANN's aligned IOCP reader. Vector fetching and streamer providers use the same reader without depending on the released storage.

Main changes

  • Add a Windows aligned reader using FILE_FLAG_NO_BUFFERING | FILE_FLAG_OVERLAPPED.
  • Use an independent file handle and completion port for each I/O context.
  • Validate request alignment, file identity, completion ownership, completed length, duplicate completions, stale batches, and short reads.
  • Cancel and drain outstanding requests before releasing contexts or destination buffers.
  • Release per-operation Windows I/O resources to prevent IOCP cross-thread starvation.
  • Support UTF-8 and Unicode Windows index paths.
  • Allow safe replacement and reopening of index files while preserving file identity isolation.
  • Keep vector providers and iterators valid after the source streamer is closed.
  • Return failures safely when a provider, iterator, or vector fetch produces a null result instead of passing null data to the reformer or memcpy.
  • Batch hot-node cache preloading and reuse nodes already read during BFS.
  • Report windows_overlapped through the existing I/O backend APIs.
  • Keep Linux backend selection using io_uring, dynamically loaded libaio, or synchronous pread fallback.

Performance validation

Cohere 1M, 768 dimensions, cosine distance, FP32, 1,000 queries, Windows Server 2022, with the same 40,088 hot nodes cached in both runs:

Metric Before handle fix After fix
Cache preload 10.36 s 1.06 s
Average latency 26.3 ms 4.3 ms
QPS 38.0 235.1

The full FP32/FP16 parameter sweep retained the expected recall range and reached about 27.5K process read operations per second. These are process I/O counters rather than device-wide hardware IOPS.

Compatibility

  • Existing DiskANN indexes do not require rebuilding.
  • The existing cache_node_num configuration remains unchanged.
  • No new byte-based cache-budget parameter or serialization field is introduced.

Test coverage

Tests cover:

  • Concurrent IOCP contexts and cross-thread progress.
  • Destruction with outstanding requests.
  • Misaligned, truncated, and short reads.
  • Duplicate, stale, and foreign completions.
  • Unicode paths and index-file replacement.
  • Retained reader and storage references.
  • Provider and iterator lifetime after streamer close.
  • Hot-node cache loading and reuse.
  • Null provider, iterator, and vector-data handling in the mixed reducer.

The affected mixed-reducer target also builds successfully with warnings treated as errors.

@iaojnh
iaojnh marked this pull request as draft August 20, 2026 09:02
@iaojnh
iaojnh marked this pull request as ready for review August 20, 2026 12:08
Comment thread src/core/algorithm/diskann/diskann_index_provider.h
Comment thread tests/ailego/parallel/thread_queue_test.cc
Comment thread tools/core/recall_original.cc Outdated
Comment thread src/core/algorithm/diskann/diskann_streamer.h Outdated
Comment thread src/core/algorithm/diskann/diskann_builder_entity.cc Outdated
Comment thread src/core/algorithm/diskann/diskann_holder.h
Comment thread src/core/algorithm/diskann/diskann_index_provider.h
Comment thread src/core/algorithm/diskann/diskann_searcher_entity.cc Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Tracking] Roll out DiskANN support on Windows, Android, and iOS

3 participants