[zzylol]

Why

In asap_sketchlib 0.2.2, HHHeap::update rebuilt its key index after every accepted update. That made CMSHeap/CSHeap inserts O(k) in the heap capacity. 0.3.0 (asap_sketchlib#107) patches the index through each sift, so inserts no longer depend on capacity. Since #182 sizes top-k heaps to m·k, the 0.2.2 cost grew with every merged window and would have dominated the top-k cost table.

CMSHeap 3×256 FastPath, 1e6 Zipf(1.1) inserts over 1e4 keys, measured on clnode138:

heap	0.2.2	0.3.0
32	537 K/s	12.4 M/s
128	123 K/s	12.2 M/s
512	26 K/s	11.8 M/s
2048	5 K/s	10.7 M/s

What

• Pin asap_sketchlib = 0.3.0 (aqpbm-planeval already used it).
• Hydra wrappers, adapted only so they compile. 0.3.0 fixes a grid's key columns at construction (with_schema) and queries positionally (&[Option<&str>]).
  • build_hydra_* take the stream's label-column count (its first record's), and new_hydra builds the schema once.
  • update keeps every ;-joined label in its column, empty labels included.
  • query constrains the probe's leading columns (the scored subpopulations group by column 0) and leaves the rest open.
  • A record or probe of another width is refused, naming the variant and both widths. A zero label count is a BuildError.
• DDSketch::merge now returns a Result; the wrappers expect it.
• Top-k memory now counts what 0.3's HHHeap holds per resident, via one helper aqpbm_core::hh_heap_bytes(capacity, size_of::<HHItem>()): capacity · (HHItem + 8 B digest) + hash_buckets(capacity) · 33 B, where hash_buckets is the next power of two of ⌈capacity/7⌉·8 and each bucket holds a u64 digest, a SmallVec<[usize; 2]> and a control byte. Used by cms_heap, cs_heap, UnivMon and aqpbm-planeval (which adds string keys' own bytes).
• aqpbm-planeval takes the workspace asap_sketchlib.
• A Hydra stream whose records have different label widths is a BuildError naming the variant and both widths.

Known behavior changes (Hydra, not measured in the optimizer)

• 0.3.0 Hydra keys subpopulations by column. The Polars subpopulation baselines still match an equal value in any label column, so the two differ when label columns share a value domain.
• 0.3.0's query_key also queries the empty template on every probe, so Hydra's query cost changes.
• Hydra's cost and accuracy rows need re-measuring before anyone uses them again.
• A label containing ; splits into extra columns, and the record is refused.
• Probes fill the leading columns, so group_columns other than 0 would be misread.
• Splitting each record into labels happens inside the timed insert.
• Hydra-UnivMon's footprint still counts its cell heaps as HHItems only.

Verification

• Workspace cargo clippy --all-targets -D warnings and cargo fmt --check are clean.
• cargo test passes for sketch-bench (92), aqpbm-core (94), aqpbm-cli, rqe-optimizer (74) and aqpbm-datagen (40), including the Hydra wrappers' own tests and new ones for a record of another width and an empty label.
• The Python tests (test_study_saturation, test_export_autosketch_eval_table) pass.

Follow-up

Rerun study_saturation.py --phase optimizer-cost on an idle machine to price top-k at heaps 32/128/512/2048 on 0.3.0.

🤖 Generated with Claude Code

[zzylol]

@milindsrivastava1997 FYI