Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions datasketches/src/bloom/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,8 @@
//! filter.insert(42_u64);
//!
//! // Check membership
//! assert!(filter.contains(&"apple")); // true - definitely inserted
//! assert!(!filter.contains(&"grape")); // false - never inserted (probably)
//! assert!(filter.contains(&"apple")); // true - possibly present (and known to be inserted here)
//! assert!(!filter.contains(&"grape")); // false - definitely not present
//!
//! // Get statistics
//! println!("Capacity: {} bits", filter.capacity());
Expand Down
4 changes: 2 additions & 2 deletions datasketches/src/bloom/sketch.rs
Original file line number Diff line number Diff line change
Expand Up @@ -66,8 +66,8 @@ impl BloomFilter {
/// let mut filter = BloomFilterBuilder::with_accuracy(100, 0.01).build();
/// filter.insert("apple");
///
/// assert!(filter.contains(&"apple")); // true - was inserted (probably)
/// assert!(!filter.contains(&"grape")); // false - never inserted
/// assert!(filter.contains(&"apple")); // true - possibly present (and known to be inserted here)
/// assert!(!filter.contains(&"grape")); // false - definitely not present
/// ```
pub fn contains<T: Hash>(&self, item: &T) -> bool {
if self.is_empty() {
Expand Down
8 changes: 4 additions & 4 deletions datasketches/src/codec/decode.rs
Original file line number Diff line number Diff line change
Expand Up @@ -123,28 +123,28 @@ impl SketchSlice<'_> {
Ok(i32::from_be_bytes(buf))
}

/// Reads a 16-bit unsigned integer from the slice in little-endian byte order.
/// Reads a 64-bit unsigned integer from the slice in little-endian byte order.
pub fn read_u64_le(&mut self) -> io::Result<u64> {
let mut buf = [0u8; 8];
self.read_exact(&mut buf)?;
Ok(u64::from_le_bytes(buf))
}

/// Reads a 16-bit unsigned integer from the slice in big-endian byte order.
/// Reads a 64-bit unsigned integer from the slice in big-endian byte order.
pub fn read_u64_be(&mut self) -> io::Result<u64> {
let mut buf = [0u8; 8];
self.read_exact(&mut buf)?;
Ok(u64::from_be_bytes(buf))
}

/// Reads a 16-bit signed integer from the slice in little-endian byte order.
/// Reads a 64-bit signed integer from the slice in little-endian byte order.
pub fn read_i64_le(&mut self) -> io::Result<i64> {
let mut buf = [0u8; 8];
self.read_exact(&mut buf)?;
Ok(i64::from_le_bytes(buf))
}

/// Reads a 16-bit signed integer from the slice in big-endian byte order.
/// Reads a 64-bit signed integer from the slice in big-endian byte order.
pub fn read_i64_be(&mut self) -> io::Result<i64> {
let mut buf = [0u8; 8];
self.read_exact(&mut buf)?;
Expand Down
10 changes: 6 additions & 4 deletions datasketches/src/frequencies/mod.rs
Comment thread
tisonkun marked this conversation as resolved.
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,9 @@
//! in Data Streams"](https://arxiv.org/abs/1705.07001) by Daniel Anderson, Pryce Bevan, Kevin Lang,
//! Edo Liberty, Lee Rhodes, and Justin Thaler.
//!
//! This sketch is useful for tracking approximate frequencies of items of type `T` that implements
//! [`FrequentItemValue`], with optional associated counts (`T` item, `u64` count) that are members
//! of a multiset of such items. The true frequency of an item is defined to be the sum of
//! associated counts.
//! This sketch tracks approximate frequencies of items of type `T` that implement [`Eq`] and
//! [`Hash`](std::hash::Hash), with optional associated counts (`T` item, `u64` count) that are
//! members of a multiset. The true frequency of an item is the sum of its associated counts.
//!
//! This implementation provides the following capabilities:
//! * Estimate the frequency of an item.
Expand Down Expand Up @@ -89,6 +88,9 @@
//!
//! # Serialization
//!
//! The built-in serialization methods are available when the item type implements
//! [`FrequentItemValue`].
//!
//! ```
//! use datasketches::frequencies::FrequentItemsSketch;
//!
Expand Down
20 changes: 11 additions & 9 deletions datasketches/src/hll/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -42,24 +42,26 @@
//!
//! # HLL Types
//!
//! Three target HLL types are supported, trading precision for memory:
//! The three target HLL types are isomorphic representations of the same registers. Given the
//! same `lg_k` and input, they produce identical estimates and error distributions; they trade
//! memory layout and update performance, not statistical precision:
//!
//! * [`HllType::Hll4`]: 4 bits per bucket (most compact)
//! * [`HllType::Hll6`]: 6 bits per bucket (balanced)
//! * [`HllType::Hll8`]: 8 bits per bucket (highest precision)
//! * [`HllType::Hll6`]: 6 bits per bucket (fixed-size middle ground)
Comment thread
tisonkun marked this conversation as resolved.
//! * [`HllType::Hll8`]: 8 bits per bucket (largest and simplest representation)
//!
//! # Union Operations
//!
//! The [`HllUnion`] type enables combining multiple HLL sketches into a unified estimate.
//! It maintains an internal "gadget" sketch that accumulates the union of all input sketches
//! and automatically handles:
//! It accumulates the distinct values represented by all input sketches and automatically handles:
//!
//! * Sketches with different `lg_k` precision levels (resizes/downsamples as needed)
//! * Sketches in different modes (List, Set, or Array)
//! * Sketches with different `lg_k` configurations (resizes/downsamples as needed)
//! * Sketches with different target HLL types
//!
//! The union operation preserves cardinality estimation accuracy while enabling distributed
//! computation patterns where sketches are built independently and merged later.
//! The result's accuracy is determined by its final effective `lg_k`. Merging sketches with
//! different configurations may reduce this value. Once reduced, it remains lower until the union
//! is reset. A lower effective `lg_k` widens the error distribution; converting among HLL target
//! types does not change accuracy.
//!
//! # Serialization
//!
Expand Down
12 changes: 8 additions & 4 deletions datasketches/src/hll/union.rs
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,13 @@ use crate::hll::mode::Mode;

/// An HLL Union for combining multiple HLL sketches.
///
/// The union maintains an internal sketch (the "gadget") that accumulates
/// the union of all input sketches. It automatically handles sketches with
/// different configurations and modes.
/// The union accumulates the distinct values represented by all input sketches and automatically
/// handles sketches with different configurations.
///
/// Merging sketches with different configurations may reduce the union's effective `lg_k`. Once
/// reduced, it remains lower until [`reset`](Self::reset), so estimates and bounds reflect the
/// reduced register count. The requested [`HllType`] changes only the result representation, not
/// its statistical accuracy.
///
Comment thread
tisonkun marked this conversation as resolved.
/// See the [module level documentation](super) for more.
#[derive(Debug, Clone)]
Expand Down Expand Up @@ -114,7 +118,7 @@ impl HllUnion {
/// Update the union with another sketch
///
/// Merges the input sketch into the union's internal gadget, handling:
/// * Sketches with different lg_k values (resizes/downsamples as needed)
/// * Sketches with different `lg_k` values (resizes/downsamples as needed)
/// * Sketches in different modes (List, Set, Array4/6/8)
/// * Sketches with different target HLL types
///
Expand Down
5 changes: 3 additions & 2 deletions datasketches/src/thetafamily/theta/jaccard_similarity.rs
Original file line number Diff line number Diff line change
Expand Up @@ -63,8 +63,9 @@ impl ThetaJaccardSimilarity {
///
/// # Errors
///
/// Returns an error if either non-empty sketch was built with a seed different from this
/// operator's configured seed.
/// Returns an error if both sketches are logically non-empty and either was built with a seed
/// different from this operator's configured seed. Empty-input fast paths return an exact
/// result without checking seed compatibility.
pub fn compute<'a, 'b>(
&self,
sketch_a: impl Into<ThetaSketchView<'a>>,
Expand Down
5 changes: 3 additions & 2 deletions datasketches/src/thetafamily/tuple/jaccard_similarity.rs
Original file line number Diff line number Diff line change
Expand Up @@ -68,8 +68,9 @@ impl TupleJaccardSimilarity {
///
/// # Errors
///
/// Returns an error if either non-empty sketch was built with a seed different from this
/// operator's configured seed.
/// Returns an error if both sketches are logically non-empty and either was built with a seed
/// different from this operator's configured seed. Empty-input fast paths return an exact
/// result without checking seed compatibility.
pub fn compute<'a, 'b, S, T>(
&self,
sketch_a: impl Into<TupleSketchView<'a, S>>,
Expand Down