Repository navigation
fix(univmon): reject layer seeds and row geometries that break hash independence - #220
Merged
Merged
Conversation
…ndependence Layer i's Count Sketch hashes at seed index i, and seed indices wrap modulo the 20-entry seed list. So layer 19 hashed at BOTTOM_LAYER_FINDER, the seed whose bits choose which keys reach each layer, and layer 20 reused layer 0's seed. MAX_LAYER_SIZE drops from 64 to BOTTOM_LAYER_FINDER (19): layers 0..=18 use 19 distinct seeds, none of them the finder's. A const assertion pins BOTTOM_LAYER_FINDER < SEEDLIST.len(). CountL2HH slices every row's column and sign from one 128-bit hash. Once rows * (ceil_log2(cols) + 1) > 128, deep rows' column bits overlap shallow rows' sign bits, and from rows * ceil_log2(cols) >= 128 the shift runs past the hash. The wire doors already refused these shapes, but the constructor built them silently. Now the constructor, the OctoSketch L2hhWorkerSketch (which checked column bits only) and both wire doors share check_l2hh_hash_bits. Shapes that were valid hash exactly as before, so no wire bytes or goldens change.
GordonYuanyc
approved these changes
Oct 15, 2026
GordonYuanyc
left a comment
Collaborator
There was a problem hiding this comment.
This makes perfect sense. LGTM!
This was referenced Oct 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
UnivMon's error bounds (documented in #221) treat every hash a key meets as an independent random draw. Two construction-time gaps break that. This PR rejects both at construction. Valid shapes hash exactly as before, and no wire bytes change.
Fixed
CountL2HH. Rowrreads its column from bits[r·b, (r+1)·b)of one 128-bit hash (b = ⌈log2 cols⌉) and its sign from bit127 − r.rows·(b+1) > 128, a deep row's column bits overlap the shallow rows' sign bits.rows·b ≥ 128, the shift runs past the hash.CountL2HH::with_dimensions_and_seed(and soinit_univmonand theUnivMonPyramidconstructor) now panics on them. So does the OctoSketchL2hhWorkerSketch, which used to check onlyrows·b ≤ 128and so accepted the sign overlap (for example 11 × 2048).check_l2hh_hash_bits, now serves the constructor, the worker and both wire doors.ihashes at seed indexi, and seed indices wrap modulo the 20-entrySEEDLIST.BOTTOM_LAYER_FINDER(19), the same seed whose bits decide which keys reach which layer. Layer 20 reused layer 0's seed.MAX_LAYER_SIZEdrops from 64 toBOTTOM_LAYER_FINDER(19). Layers0..=18use 19 distinct seeds, none of them the finder's. Aconstassertion pinsBOTTOM_LAYER_FINDER < SEEDLIST.len().UnivMonPyramid, and both UnivMon wire decoders throughcheck_layer_size, share the cap.Why reject instead of drawing more hash
seed_list). Extending it would change the bytes of every sketch in the crate and every golden. Re-mapping layer seeds would break the wire contract that layerihashes at seed indexi.L ≥ ⌈log2(F0_max/k)⌉ + 2, 19 layers coverF0up tok·2^17, about 4M distinct keys atk = 32.Behaviour change
init_univmonand theCountL2HHconstructor panic on the shapes above.layer_size > 19, where they used to accept up to 64. Such a pyramid's layer 19 hashed at the finder's seed.MAX_LAYER_SIZEis public and its value changes.Tests (all fail on
main)countl2hh_rejects_column_bits_that_reach_the_sign_bits(11 × 2048 = 132 bits).init_univmon_rejects_column_bits_that_reach_the_sign_bits.a_univmon_worker_refuses_column_bits_that_reach_the_sign_bits.layer_seeds_are_distinct_and_never_the_layer_finder.init_univmon_rejects_a_layer_on_the_finder_seed.Also updated:
the_widest_accepted_univmon_geometry_still_addresses_distinct_rowsgoes from 11 × 2048 (which overlaps) to 10 × 2048.countl2hh_accepts_the_widest_disjoint_geometryis new.l2hh_wireencoder-rejection tests now build their out-of-budget sketch directly, because the constructor no longer can.Golden files
None change. No golden uses more than 19 layers or a shape over the hash budget, and
asapv1_goldenpasses unchanged.Sequencing
This PR is independent of #221; they touch different functions and different sections of
api_univmon.mdanddocs/tests.md, so they merge in either order.#176 and #198
(ceil_log2(cols)+1)·rows ≤ 128. It states no UnivMon layer bound, so the doc is left alone here so as not to collide with refactor(univmon)!: ASAPv1 only — drop the proto envelope field, add goldens #176's spec edits.layer_size ≤ 19check.Verification
cargo fmt --check.-D warningsandcargo test --lockedfor each of the four CI feature sets.RUSTDOCFLAGS="-D warnings" cargo doc.