Repository navigation
[diskann-wide] Add byte-vector reinterpretations for u32x8 and u64x8 - #1445
Merged
juchen-ms (partychen) merged 5 commits intoSep 30, 2026
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Replace the test macro with explicit architecture-gated tests and an existing-helper-based byte comparison. Use distinct byte positions and signed-boundary offsets so fixed fixtures expose lane/byte transposition. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep architecture-gated macro call sites and the unary test driver. Compare raw bytes and use distinct byte positions instead of a transpose-symmetric fixture. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Remove special Miri branching and alias imports from the V4 conversion tests. Keep scalar emulation coverage in emulated.rs. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
juchen-ms (partychen)
requested review from
a team
and
a balanced review from Copilot
September 29, 2026 05:23
Contributor
There was a problem hiding this comment.
Copilot review overview
🟢 Approval recommended
The implementations preserve register bits and include targeted byte-order validation across native and emulated paths.
Review effort: Balanced
Findings: None
What changed in this PR
Adds tested SIMD bit reinterpretation support for additional x86 and emulated vector widths.
Changes:
- Adds reusable reinterpretation tests.
- Supports
u32x8andu64x8reinterpretation into signed and unsigned byte vectors. - Adds corresponding emulated implementations.
| File | Description |
|---|---|
diskann-wide/src/test_utils/ops.rs |
Adds the reinterpretation test macro. |
diskann-wide/src/emulated.rs |
Adds emulated reinterpretations and tests. |
diskann-wide/src/arch/x86_64/v3/conversion.rs |
Adds 256-bit reinterpretations and tests. |
diskann-wide/src/arch/x86_64/v4/conversion.rs |
Adds 512-bit reinterpretations and tests. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
juchen-ms (partychen)
changed the title
juchen-ms (partychen)
changed the title
Codecov Report❌ Patch coverage is
❌ Your patch status has failed because the patch coverage (81.25%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #1445 +/- ##
==========================================
+ Coverage 90.58% 90.87% +0.29%
==========================================
Files 568 579 +11
Lines 112680 114587 +1907
==========================================
+ Hits 102069 104132 +2063
+ Misses 10611 10455 -156
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
Mark Hildebrand (hildebrandmw)
approved these changes
Sep 29, 2026
Address PR microsoft#1445 review feedback by covering every V3, V4 and Emulated reinterpretation with the existing test macro. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
juchen-ms (partychen)
enabled auto-merge (squash)
September 30, 2026 02:26
xwj-ox
approved these changes
Sep 30, 2026
Mark Hildebrand (hildebrandmw)
added a commit
that referenced
this pull request
Oct 5, 2026
# DiskANN v0.60.0 ## Major Changes ### `diskann-utils`: Matrix API changes There have been significant changes to the matrix APIs in `diskann-utils`. The previous contents of `diskann_utils::views` have been moved to `diskann_utils::views::rowmajor`. The following type changes have been made | v0.59 | v0.60 | Remarks | |-|-|-| | `views::Matrix<T>` | `views::rowmajor::Owned<T>` | Size changed from 32 bytes to 24 bytes | | `views::MatrixView<'a, T>` | `views::rowmajor::Ref<'a, T>` | Size changed from 32 bytes to 24 bytes | | `views::MatrixViewMut<'a, T>` | `views::rowmajor::Mut<'a, T>` | Size changed from 32 bytes to 24 bytes | | `MatrixBase` inherent methods | `rowmajor::Matrix` and `rowmajor::MatrixMut` trait methods | | The `rowmajor::Matrix` and `rowmajor::MatrixMut` trait methods need to be imported to access most matrix functionality. In addition, the following methods have been renamed and changed: | v0.59 | v0.60 | Remark | |-|-|-| | `row_iter` | `rows` | No longer panics when `ncols == 0` | | `row_iter_mut` | `rows_mut` | No longer panics when `ncols == 0` | | `par_row_iter` | `par_rows` | No longer panics when `ncols == 0` | | `par_row_iter_mut` | `par_rows_mut` | | | `get_row_unchecked` | `row_unchecked` | | | `get_row_unchecked_mut` | `row_unchecked_mut` | | | `as_mut_view` | `as_view_mut` | | | `try_from` constructors | `try_from_data` | | In addition, * The `Index` and `IndexMut` implementations have been removed. Use `Matrix::element` and `Matrix::element_mut` instead. * The `Init` and `Generator` style constructors for `Matrix::new` have been removed. Instead, `rowmajor::Owned::from_fn` and `rowmajor::Owned::from_element` can be used instead. The former's closure has been changed to take an argument `rowmajor::RowCol` indicated the row and column being initialized. Both of these constructors take now take the initializer as the last argument. * Matrix constructors validate against integer overflow and `isize::MAX` allocations via a `rowmajor::Layout<T>` layout. Constructors panic if these validations fail. `try_*` variations of these constructors which return `rowmajor::LayoutError` instead. ### `diskann-utils`: Strided API changes * Structs `StridedBase`, `StridedView`, and `MutStridedView` were collapsed into a single, immutable `Strided<'a, T>`. * `strided::linear_length` was removed, use `strided::Layout::linear_length` instead. * `strided::TryFromErrorLight` was removed. * `strided::TryFromError` changed from a struct to an enum, and `into_inner`/`as_static` were removed. ### `diskann` `InsertStrategy` is no longer a `SearchStrategy` subtrait. Implementations must now provide ```rust type SearchAccessor; type SearchAccessorError; fn insert_search_accessor(...) -> Result<Self::SearchAccessor, Self::SearchAccessorError>; ``` Previously, `insert_search_accessor` had a provided implementation forwarding to `SearchStrategy::search_accessor`. That default is now gone. This change was done to decouple insert element types from search element types. ### `diskann-disk` The publicly re-exported types below were removed in favor of standard buffered I/O: ``` diskann_disk::storage::CachedReader diskann_disk::storage::CachedWriter ``` Consumers should migrate to `std::io::BufReader`/`BufWriter` and the relevant storage-provider reader/writer. `READ_WRITE_BLOCK_SIZE` also changed from `u64` to `usize`. ### `diskann-quantization` These public traits now require `Debug`: ``` spherical::iface::Quantizer spherical::iface::DynQueryComputer spherical::iface::DynDistanceComputer ``` ### `diskann-inmem` (experimental) The old `layers` abstraction was removed and replaced by the `Representation`/`Store`/`Slots` architecture. And support for a spherically quantized backed was added. # Full Change Log * [diskann-inmem] Prepare code for quantization and beyond by @hildebrandmw in #1352 * Add Neon inner product U8xU1, U8xU2 and U8xU4 kernels by @pfoxARM in #1372 * [CI] Check all workspace features and private docs by @wuw92 in #1400 * [CI] Check Rust license headers by @wuw92 in #1401 * [disk] Replace custom cached I/O with standard buffered readers and writers by @wuw92 in #1396 * [dikkann_wide] Integer SIMDMinMax by @suri-kumkaran in #1407 * Add Neon Squared L2 kernels for USlice2 and USlice4 by @pfoxARM in #1398 * Bump the github-actions group with 3 updates by @dependabot[bot] in #1406 * [CI] Check workspace direct dependency versions by @wuw92 in #1410 * Train fresh disk-search PQ codebooks for each index build by @xwj-ox in #1299 * [diskann-quantization] Make most things in `diskann-quantization/spherical/iface.rs` debuggable by @hildebrandmw in #1414 * [diskann] Make `InsertStrategy` independent of `SearchStrategy` by @hildebrandmw in #1393 * Use #[expect] to detect unnecessary lint suppressions by @wuw92 in #1409 * [diskann-garnet] Store attributes during insert, not after by @metajack in #1412 * [diskann-wide] Fix `alias!` by @hildebrandmw in #1417 * [diskann-utils] Simplify `StridedView`. by @hildebrandmw in #1376 * [diskann-wide] More Neon Zip/Unzip impls by @hildebrandmw in #1405 * [diskann-garnet] Deserialize max id for quantization on load by @metajack in #1419 * More ergonomic `spherical::iface::Quantizer` constructors by @hildebrandmw in #1418 * [diskann-bftree] Clamp number of concurrency test parallel workers by @hildebrandmw in #1433 * Tweak behavior of adaptive L by @magdalendobson in #1354 * Bump the github-actions group with 2 updates by @dependabot[bot] in #1436 * Reform Range Search by @magdalendobson in #1337 * PiPNN 2/6: add numerical kernels by @SeliMeli in #1287 * Matrix overhaul 1 of N - Data Accessors by @hildebrandmw in #1415 * Add `assert_contains!` helper by @hildebrandmw in #1421 * Matrix overhaul 2 of N - `Matrix` invariants by @hildebrandmw in #1420 * [CI] Run SDE tests for diskann with all features by @SeliMeli in #1440 * [diskann-wide] Add Neon `vabdq_u8` and `vabdq_s8`. by @hildebrandmw in #1437 * [diskann-inmem] Add support for spherical quantization by @hildebrandmw in #1432 * [benchmark-runner] Bump edition to 2024 by @hildebrandmw in #1444 * [diskann-wide] Add byte-vector reinterpretations for u32x8 and u64x8 by @partychen in #1445 * Propagate invalid matrix layout errors in production paths by @arrayka in #1453 * Add sparse vector distance kernels (f32/f16 L2, inner product, cosine) by @mana-agarwal in #1434 * Bump the github-actions group with 3 updates by @dependabot[bot] in #1454 * [diskann-utils] Matrix Update 3 of N (constructors) by @hildebrandmw in #1451 * [diskann-utils] Matrix Update 4 of 4 by @hildebrandmw in #1457 ## New Contributors * @mana-agarwal made their first contribution in #1434 **Full Changelog**: v0.59.0...v0.60.0
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reference Issues/PRs
Extracted from #1394 to land the reusable diskann-wide support independently.
What does this implement/fix? Briefly explain your changes.
Adds bit-preserving SIMD reinterpretations from word lanes to byte lanes:
u32x8→u8x32andi8x32.u64x8→u8x64andi8x64.Emulatedimplementations for all four conversions.Tests follow the existing conversion-test macro and architecture-check conventions. They compare raw bytes using distinct byte positions, signed-byte boundary patterns, and the existing seeded test driver.
The change is limited to diskann-wide. No MaxSim consumers, custom intrinsics, or unrelated refactors are included.
Any other comments?
Local validation passed:
cargo test -p diskann-widecargo fmt --allcargo clippy --workspace --all-targets -- -D warningscargo +nightly miri test -p diskann-wide --lib reinterpret_