Parallel Write performance improvements

docs/release notes/4.1.0/695.performance.md

@@ -0,0 +1,86 @@
+### Parallel Graph Index Writing via AsynchronousFileChannel
+
+**Description**
+Introduces `OnDiskParallelGraphIndexWriter`, a new writer that parallelizes the serialization
+of Level-0 (L0) node records using Java's `AsynchronousFileChannel`. In the standard
+`OnDiskGraphIndexWriter`, all L0 records are written sequentially by a single thread. At
+large scale — tens of millions of vectors or high-dimensional embeddings — this sequential
+write step becomes the dominant bottleneck after graph construction, negating much of the
+benefit of parallel HNSW build.
+
+The parallel writer divides the L0 ordinal space into tasks, each covering a contiguous range
+of nodes. Two write strategies are used depending on whether inline features have been
+pre-written via `writeInline()`:
+
+- **Fast path** (no pre-written features): all records in the range are packed into a single
+  contiguous `ByteBuffer` and committed with one `channel.write()` call, minimizing syscall
+  overhead.
+- **Legacy path** (features pre-written): each node's non-pre-written byte spans (ordinal,
+  neighbor section) are identified and issued as individual non-blocking async writes. All
+  futures for the task are submitted before any are joined, allowing the OS to schedule the
+  full I/O workload efficiently.
+
+All tasks are submitted to a configurable thread pool before results are collected; the
+`BufferedRandomAccessWriter`'s cache is invalidated after parallel writes complete to prevent
+stale buffered reads.
+
+**How to Enable**
+
+*Programmatic API* — substitute `OnDiskParallelGraphIndexWriter.Builder` for
+`OnDiskGraphIndexWriter.Builder` when writing the index to disk:
+
+```java
+// Serial (existing) path
+var writer = new OnDiskGraphIndexWriter.Builder(graph, outputPath)
+        .withMapper(ordinalMapper)
+        .with(new InlineVectors(dimension))
+        .build();
+
+// Parallel path
+var writer = new OnDiskParallelGraphIndexWriter.Builder(graph, outputPath)
+        .withMapper(ordinalMapper)
+        .with(new InlineVectors(dimension))
+        // Optional tuning:
+        .withParallelWorkerThreads(0)       // 0 = use available processors
+        .withParallelDirectBuffers(false)   // true = off-heap ByteBuffers
+        .build();
+
+writer.write(featureStateSuppliers);
+writer.close();
+```
+
+To supply a shared, externally-managed executor (e.g. to bound total thread count across
+concurrent builds):
+
+```java
+ExecutorService ioPool = Executors.newFixedThreadPool(16);
+var writer = new OnDiskParallelGraphIndexWriter.Builder(graph, outputPath)
+        .withMapper(ordinalMapper)
+        .with(new InlineVectors(dimension))
+        .withExecutor(ioPool)   // caller is responsible for shutdown
+        .build();
+```
+
+*BenchYAML / AutoBenchYAML* — add the following field under `construction` in any
+index-parameter YAML config file:
+
+```yaml
+construction:
+  parallelGraphConstruction: Yes   # default: No
+```
+
+When set to `Yes`, `Grid` uses `OnDiskParallelGraphIndexWriter` for the on-disk build path.
+The field is optional and defaults to `No` (serial writes) if omitted, so existing config
+files require no changes.
+
+**Notes**
+- `OnDiskParallelGraphIndexWriter` is annotated `@Experimental`. Its API may change in a
+  future release.
+- The writer produces output in the same on-disk format as `OnDiskGraphIndexWriter`; indexes
+  written with either class are interchangeable and loaded with `OnDiskGraphIndex.load()`.
+- Write tasks perform blocking file I/O. For best performance supply an I/O-sized thread pool
+  (thread count ≥ logical cores) rather than a compute-sized pool when using `withExecutor()`.
+- The per-thread scratch `ByteBuffer` that previously limited throughput to one record at a
+  time has been removed. Each task now allocates a range-sized buffer, eliminating the
+  sub-record write bottleneck at high core counts.
+- Related issue: #695.

jvector-base/src/main/java/io/github/jbellis/jvector/graph/disk/ParallelGraphWriter.java

@@ -22,7 +22,6 @@
 import io.github.jbellis.jvector.graph.disk.feature.FeatureId;
 
 import java.io.IOException;
-import java.nio.ByteBuffer;
 import java.nio.channels.AsynchronousFileChannel;
 import java.nio.file.Path;
 import java.nio.file.StandardOpenOption;
@@ -67,11 +66,14 @@ class ParallelGraphWriter implements AutoCloseable {
     private final ExecutorService executor;
     private final boolean ownsExecutor;
     private final ThreadLocal<ImmutableGraphIndex.View> viewPerThread;
-    private final ThreadLocal<ByteBuffer> bufferPerThread;
     private final CopyOnWriteArrayList<ImmutableGraphIndex.View> allViews = new CopyOnWriteArrayList<>();
     private final int recordSize;
     private final Path filePath;
     private final int taskMultiplier;
+    // Passed through to NodeRecordTask so each task allocates the right buffer type.
+    // FUTURE IMPROVEMENT: when the legacy path is removed, each task allocates exactly
+    // one range-sized buffer, making the direct/heap distinction more impactful and cleaner.
+    private final boolean useDirectBuffers;
     private static final AtomicInteger threadCounter = new AtomicInteger(0);
 
     /**
@@ -143,6 +145,7 @@ public ParallelGraphWriter(RandomAccessWriter writer,
         this.graph = graph;
         this.filePath = Objects.requireNonNull(filePath);
         this.taskMultiplier = config.taskMultiplier;
+        this.useDirectBuffers = config.useDirectBuffers;
         if (externalExecutor != null) {
             this.executor = externalExecutor;
             this.ownsExecutor = false;
@@ -163,23 +166,17 @@ public ParallelGraphWriter(RandomAccessWriter writer,
             + Integer.BYTES // neighbor count
             + graph.getDegree(0) * Integer.BYTES; // neighbors + padding
 
-        // Thread-local views for safe neighbor iteration
-        // CopyOnWriteArrayList handles concurrent additions safely
+        // Thread-local views for safe neighbor iteration.
+        // CopyOnWriteArrayList handles concurrent additions safely.
         this.viewPerThread = ThreadLocal.withInitial(() -> {
             var view = graph.getView();
             allViews.add(view);
             return view;
         });
-
-        // Thread-local buffers to avoid allocation overhead
-        // Use BIG_ENDIAN to match Java DataOutput specification
-        final int bufferSize = recordSize;
-        final boolean useDirect = config.useDirectBuffers;
-        this.bufferPerThread = ThreadLocal.withInitial(() -> {
-            ByteBuffer buffer = useDirect ? ByteBuffer.allocateDirect(bufferSize) : ByteBuffer.allocate(bufferSize);
-            buffer.order(java.nio.ByteOrder.BIG_ENDIAN);
-            return buffer;
-        });
+        // FUTURE IMPROVEMENT: the old per-thread scratch buffer (bufferPerThread) is removed.
+        // Each task now allocates its own range-sized buffer in the fast path, or per-region
+        // buffers in the legacy path.  The thread-local was sized to a single record and forced
+        // sub-record writes; the new approach eliminates that bottleneck entirely.
     }
 
     /**
@@ -239,23 +236,19 @@ public void writeL0Records(OrdinalMapper ordinalMapper,
                 final int end = endOrdinal;
 
                 Future<Void> future = executor.submit(() -> {
-                    var view = viewPerThread.get();
-                    var buffer = bufferPerThread.get();
-
                     var task = new NodeRecordTask(
-                            start,                    // Start of range (inclusive)
-                            end,                      // End of range (exclusive)
+                            start,              // range start (inclusive)
+                            end,                // range end (exclusive)
                             ordinalMapper,
                             graph,
-                            view,
+                            viewPerThread.get(),
                             inlineFeatures,
                             featureStateSuppliers,
                             recordSize,
-                            baseOffset,               // Base offset (task calculates per-ordinal offsets)
-                            channel,                  // Async file channel for position-based writes
-                            buffer                    // Thread-local buffer
+                            baseOffset,
+                            channel,
+                            useDirectBuffers    // each task allocates its own buffer(s)
                     );
-
                     return task.call();
                 });
 

jvector-examples/src/main/java/io/github/jbellis/jvector/example/AutoBenchYAML.java

@@ -148,15 +148,16 @@ public static void main(String[] args) throws IOException {
 
                     List<BenchResult> datasetResults = Grid.runAllAndCollectResults(ds,
                             config.construction.useSavedIndexIfExists,
-                            config.construction.outDegree, 
+                            config.construction.isParallelGraphConstruction(),
+                            config.construction.outDegree,
                             config.construction.efConstruction,
-                            config.construction.neighborOverflow, 
+                            config.construction.neighborOverflow,
                             config.construction.addHierarchy,
                             config.construction.refineFinalGraph,
-                            config.construction.getFeatureSets(), 
+                            config.construction.getFeatureSets(),
                             config.construction.getCompressorParameters(),
-                            config.search.getCompressorParameters(), 
-                            config.search.topKOverquery, 
+                            config.search.getCompressorParameters(),
+                            config.search.topKOverquery,
                             config.search.useSearchPruning);
                     results.addAll(datasetResults);
 

jvector-examples/src/main/java/io/github/jbellis/jvector/example/BenchYAML.java

@@ -126,6 +126,7 @@ public static void main(String[] args) throws IOException {
 
             Grid.runAll(ds,
                     config.construction.useSavedIndexIfExists,
+                    config.construction.isParallelGraphConstruction(),
                     config.construction.outDegree,
                     config.construction.efConstruction,
                     config.construction.neighborOverflow,