[zzylol]

Stacked on #180.

Why

A merged top-k answer loses keys: each window's heap keeps only its own top k, so a key in the global top k can be missing from every window's heap. #179 reads merge curves for that, but the synthetic workload's top-k instances hold 1.8e9 to 1.7e11 items, far past any measurable merge curve. The heap was also tied to the graded k by a compile-time constant (CMS_HEAP_TOP_K = 32).

Approved design: a deployment that merges m windows keeps a heap of m·k, and such a merged answer reads as one sketch.

What

sketch-bench

• heap= is a runtime config param on cms-heap (CmsHeapParams) and on a new countsketch-heap vocabulary (CsHeapParams). The default and minimum is TOPK_K = 32. asap_sketchlib's CMSHeap::new/CSHeap::new already take the capacity, so no upstream change is needed.
• The answered and graded k stays TOPK_K: a larger heap answers its heaviest k (top_k).
• Memory counts heap × HHItem. UnivMon and the CLI's TopkGT use TOPK_K.

rqe-optimizer

• Heap per deployment: has_heap, heap_capacity, TOPK_K. A heap top-k deployment answering from m merged windows keeps m·k (m = 1 gives k).
• Candidates: one heap per distinct m among a window's RAQEs. A RAQE is eligible only where heap ≥ (L/x)·k.
• Costs: from the cost table's heap rows, linear in the heap between and past the measured sizes (at_heap). With only one measured size, the smallest row that holds the heap serves at its own size and cost.
• Accuracy: with heap ≥ m·k, a merged answer reads as unmerged: the plain curve, then rqe-optimizer: fall back to each sketch's guarantee where the study measured nothing #180's guarantee fallback with one window. This assumes the merged heaps still hold the true top k; it isn't a guarantee.
  • SaturationCurves::load requires merge curves only for KLL.
  • Curves match configs ignoring heap; they're measured at a heap of k.
  • check_cost_table skips heaps other than k (cost-only rows).
• TOPK_ENTRIES (query output) = TOPK_K.

study_saturation.py

• --phase optimizer-cost measures top-k at TOPK_HEAPS = (32, 128, 512, 2048).
• Fix: --resume over a narrower grid (--families, --points-from) used to rewrite saturation.csv with only the current points, while their curves were kept, so SaturationCurves::load failed. It now keeps the other points' rows. This hit the rqe-optimizer: one study for sketch cost and accuracy (#174) #178 rerun on clnode109.

Decisions

• The param is named heap. top_k is still refused by name: k is not a knob.
• Interpolation is linear per op (memory, insert, merge, query) and extrapolates linearly from the two nearest measured sizes. Accuracy fields come from the smallest heap's row.
• Accuracy curves stay at heap = k, the default grid configs.

Verification

• cargo test: sketch-bench (90, including heap below k refused, a larger heap answers its heaviest k, and heap memory), aqpbm-core, aqpbm-cli, rqe-optimizer (72, including interpolation and extrapolation, the single-row fallback, m·k candidates and eligibility, heap ≥ m·k reading the plain curve, load without top-k merge curves, and heap rows in check_cost_table).
• python3 -m unittest test_study_saturation (30): heap rows in optimizer-cost, and the --resume summary fix, which fails on the old script.
• cargo clippy --all-targets -D warnings and fmt are clean. approxbench sketchbench --config "rows=3 cols=1024 heap=128" runs for both variants; heap=8 is refused.

Experiments it needs

• The top-k cost table at heaps 32/128/512/2048: study_saturation.py --phase optimizer-cost, timed, alone on the machine.
• No top-k merge curves. Top-k accuracy curves at heap = k are unchanged.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FhcWJExZcmVqS6r6rEtjis