[milindsrivastava1997]

Closes #156.

Sketch accuracy now comes from the saturation study's error-vs-N curves (#130, #140), read at the number of items a deployment's answer covers per group, at the workload's fitted data shape. Before, it came from the cost table's single measurement at N = 1e6. Exact accumulators keep the cost table's value.

Design decisions (please approve each)

This is a short version. The full questions, the options offered and the answers are in #156 under "Design decisions (Q&A)".

1. Q1. Which N: the merged total (series per group × scrapes per lookback), not each window's count. CMS, CountSketch, HLL and DDSketch merge exactly.
2. Q2. KLL and top-k merge lossily: also require every pane to be saturated amended: no pane check; like every sketch, they read the curve at the lookback's item count (after merging) (see the discussion below). Their merge penalty is rqe-optimizer: use merged accuracy curves for KLL and top-k #158.
3. Q3. Where the saturation points and the curves come from: the workload's fitted data parameters. Not the cost table's measurement point, and not the grid's worst case.
4. Q4. Cost-table configs with no curve: the export now sweeps grid configs (KLL k 500 → 800, CMS-heap cols 2048 → 1024/4096). A sketch config with no curve is never eligible.
5. Q5b. Filter or lookup: lookup. Accuracy is the curve's error at the lookback's item count, replacing the cost table's value for sketches. (Q5, on never-saturating points, is superseded by this.)
6. Q6. Reading the curve: between checkpoints, the worse neighbour. Below the first checkpoint, not eligible. Past the last, the plateau if the point saturated, otherwise not eligible.
7. Q7. Data shape: a new fitted input per (metric, grouping), MetricFacts.data_shape (zipf skew, distinct keys, tail index), taken as the worst case over windows and groups. zipf skew describes whichever insertion mode the engine uses.
8. Q8. Between grid points: the worst error over every bracketing grid point. Outside the grid: not eligible.
9. Q9. Metric: only the curve's error_metric (precision_at_k, mean_rank_err, relative_error; DDSketch moves to mean_relative_value_error with Report mean_relative_error for DDSketch (paper's relative-accuracy metric) #162/Report DDSketch relative value error #164). A mismatch means not eligible. Exact rows keep the cost table's 0.
10. Q10. Input: --saturation-dir containing out_grid_1e7_cost/ and out_1e9/ (saturation.csv and saturation_curve.csv), parsed in Rust. The 1e9 data wins.
11. Q11. KLL/top-k under lookup: when merging, each pane must have each window's item count ≥ the saturation point amended: read the curve at the lookback's item count whatever the window size, the same as every other sketch.
12. Q12. AutoSketch-Adapted: uses the same lookup with no merging through its accuracy oracle (SaturationCurves::autosketch_accuracy).

Changes

• rqe-optimizer/src/saturation.rs (new): DataShape, plus SaturationCurves::{load, accuracy, autosketch_accuracy}.
• rqe-optimizer/src/lib.rs:
  • adds MetricFacts.data_shape;
  • adds the Accuracy oracle type and table_accuracy;
  • removes Raqe::accuracy_ok_for.
• candidates.rs, enumerate.rs, milp.rs: the oracle is passed through is_eligible, eligible_deployments_for, prune_dominated_candidates, build_all_candidates, unservable, brute_force, for_each_mapping(_while) and minimize. Tests pass table_accuracy.
• examples/small_problem.rs: requires --saturation-dir and gives each grouping a data shape.
• scripts/export_rqe_optimizer_costs.sh: sweeps grid configs (Q4).
• Cost table vs. curves (comment):
  • aqpbm-core: MeasuredShape (Zipf skew and distinct keys, or Pareto tail index) and MeasuredAt::data_shape(). A cost-table row and a saturation grid point now share this key.
  • SaturationCurves::check_cost_table reads each sketch row's curve at its measured_at. A row is mismatched if its value is off by more than 3 seed SEs + 5%, its config is off the grid, or it lacks the curve's metric. A row is unchecked if the study has no point at its shape or N (KLL: Zipf ranks vs. Pareto).
  • small_problem refuses to plan on a mismatch.
• docs/rqe_sketch_deployment_v1.md: documents the new eligibility rule.

Not in this PR

• The export's accuracy pass for sketch rows stays. The optimizer never reads those values for sketches, but check_cost_table uses them as the cross-check.
• The CSVs aren't committed. They're the clnode109 outputs whose SHA-256 values are recorded in rqe-optimizer/data/autosketch-eval/README.md.
• The cost table needs a re-export with the new configs. Until then, a table with cols=2048 top-k rows leaves top-k RAQEs unservable.

Testing

After rebasing onto main (table_accuracy keeps #154's merge-count bracketing for exact accumulators):

• cargo test -p rqe-optimizer: 58 passed. cargo clippy -p rqe-optimizer -p aqpbm-core --all-targets -D warnings: clean. cargo check --workspace --all-targets: clean.
• small_problem --cost-dir <2026-10-06 table> --saturation-dir <clnode109 curves> panics, as intended. Mismatched: KLL k=500 and CMS-heap cols=2048, which are off the grid. Unchecked: KLL, DD and HLL, because the study has no point at Zipf 1.1 / distinct keys 1e5. So the table still needs the re-export.

Before the rebase:

• cargo test -p rqe-optimizer: 46 passed, including 7 new tests in saturation.
• cargo clippy --all-targets -D warnings: clean.
• small_problem with --saturation-dir . and the real cost table out_10_6_26_1342:
  • As is: top_endpoints_1h is unservable, because its top-k rows are cols=2048, which isn't in the grid.
  • On a scratch copy relabelled to cols=1024 (checks the plumbing only): --milp serves all 8 RAQEs. After the Q2/Q11 amendment, KLL's latency_p99_* RAQEs have 12–36 eligible deployments, up from 1.

🤖 Generated with Claude Code

[milindsrivastava1997]

Code vs. the proposed saturation logic (to resolve)

Proposed logic, for each (RAQE, deployment) pair:

• No merge needed: use the empirical error from the sketch-bench cost export for the config. The saturation study isn't needed.
• Merge needed, and (sketch, data distribution) saturates: pick any config whose item count after merging is ≥ the saturation point, and use the saturation study's error.
• Merge needed, and it doesn't saturate: pick the config that matches the item count after merging, and use the error from the cost export and/or the saturation study.

Sketch behaviour should be about the same across configs.

Where the code matches

• The curves: accuracy comes from the saturation study's map (sketch, config, data shape, N) → error (rqe-optimizer/src/saturation.rs).
• Merged, saturating point: the code reads the curve at the lookback's item count (after merging). Once the lookback's item count ≥ the saturation point, that's the plateau, so the error comes from the saturation study.
• Merged, never-saturating point: the code reads the curve at the lookback's item count (after merging). Past the largest measured N there's no value, and the deployment isn't eligible.

Where the code differs

1. No merge (window = lookback). The proposal uses the cost-table number. The code reads the curve at the lookback's item count instead (Q5b and Q9 in Candidate generation ignores saturation: deployments whose instances never reach N_sat are planned #156). The cost-table value was measured at N = 1e6, zipf skew 1.1, distinct keys 1e5, so it only applies to workloads that look like that. Only exact accumulators still use the cost table.
2. Lossy merges (KLL, top-k). The proposal checks only the merged count. The code also requires every pane to have each window's item count ≥ the saturation point (Q2, Q11). On the small_problem workload, this rules out every merged KLL deployment, because KLL k=200 at tail index 1.5 never saturates.
3. The same behaviour across configs. The code keys the saturation point and the curves per config and doesn't assume they match. The data says they don't: CMS-heap top-k's worst-case saturation point is 1.78e6 at cols=16384 but 1.78e8 at cols=1024.

Ambiguous

• A saturating point with the lookback's item count < the saturation point. "Pick any config whose item count ≥ the saturation point" could mean such a config is rejected. The code allows it and uses the curve's value at the lookback's item count, which isn't stable yet.

[milindsrivastava1997]

Resolution

1. No merge uses the curve, not the cost table. This stays as is here. A separate fix comes first: the cost export doesn't record the data distribution or N it measured at (zipf skew 1.1, distinct keys 1e5 and N = 1e6 are fixed constants in scripts/export_rqe_optimizer_costs.sh, and AtomicCostEntry has no field for them). That fix should merge before this PR.
2. Lossy merges. Changed to the proposed version in be40f0d. There's no pane check any more: KLL and top-k read the curve at the lookback's item count (after merging), like every other sketch. Q2 and Q11 are amended in Candidate generation ignores saturation: deployments whose instances never reach N_sat are planned #156 and in the PR description. On small_problem, the KLL latency_p99_* RAQEs now have 12–36 eligible deployments, up from 1.
3. Behaviour across configs. Resolved by Candidate generation ignores saturation: deployments whose instances never reach N_sat are planned #156 (comment). Saturation is decided per (sketch, config, data distribution), which is how the code already works: n_sat lives on each config's own grid point. No change.

Still open: the ambiguous case, a saturating point with the lookback's item count < the saturation point. The code currently reads the curve's (not yet stable) value at the lookback's item count rather than rejecting the deployment.

[zzylol]

On Where the code differs, 1 (no merge, x = T): I think this comes from the cost export and the saturation study being keyed differently, not from two genuinely different numbers. They should be one lookup.

The data already lines up.

• This branch starts at f130b23 (rqe-optimizer: small_problem rejects unknown arguments #153). It predates atomic-costs: record the measurement conditions on each cost-table row #155, which added measured_at to AtomicCostEntry.
• On main, every sketch row carries measured_at {items_per_instance: 1e6, keys_per_instance: 1e5, distribution: zipf s=1.1, population 1e5}.
• The export's sketch rows use --dataset zipf --zipf-s 1.1 --cardinality 1e5 (one column, both key and value). That is the same approxbench generator and flags the saturation study uses (--dataset zipf --zipf-s θ --cardinality K).
• So a cost-table row is one point on the saturation curve (sketch, config, θ = 1.1, K = 1e5), at N = 1e6.

Proposal:

1. Rebase onto main. This picks up measured_at (atomic-costs: record the measurement conditions on each cost-table row #155), merge_accuracy (rqe-optimizer: check accuracy at the query's merge count #154), deny_unknown_fields (rqe-optimizer: split SumOrCount and TopK; deny unknown AtomicCostEntry fields #163) and the DDSketch metric (Report DDSketch relative value error #164).
2. Move DataShape into aqpbm-core and add MeasuredAt::data_shape() (zipf → θ and K; pareto → a). SaturationCurves::load parses each row's (dist, param, cardinality) into the same type. Both structs are then keyed by (sketch, config, DataShape, N).
3. Sketch accuracy comes only from the curves. The cost table's query_accuracy becomes a check: at load time, read the curve at each sketch row's measured_at. It is an error if the curve's value differs from the row's by more than the seed spread, or if the curve has no matching config or metric. This catches the configs that are off the grid (cols=2048) and metric mismatches (Report mean_relative_error for DDSketch (paper's relative-accuracy metric) #162/Report DDSketch relative value error #164).
4. So the export's accuracy pass for sketch rows stays (it was going to be removed under "Not in this PR"). It is now the cross-check.

With this, x = T needs no special case. Where the workload looks like the cost-table measurement, the curve at n(T) is the cost-table value, and the check enforces that.

Not covered:

• KLL can't be checked yet. The cost table measures it on Zipf ranks; the study uses Pareto a, so there is no shared point. It is skipped and reported.
• Differences 2 and 3, and the ambiguous case, are semantic choices, not out-of-sync data. They still need a separate decision.

[zzylol]

@milindsrivastava1997 I force-pushed this branch (with a lease on be40f0d):

• 6f3067e, 8d4fa36: your two commits, rebased onto main. Conflicts with rqe-optimizer: check accuracy at the query's merge count #154/feat(rqe-optimizer): family properties, shared fixed-size sketches and DeltaSet key tracker #160/rqe-optimizer: split SumOrCount and TopK; deny unknown AtomicCostEntry fields #163/feat(rqe-optimizer): plan output with active deployments and instance counts #168 are resolved. Accuracy is threaded next to facts. table_accuracy keeps rqe-optimizer: check accuracy at the query's merge count #154's merge-count bracketing.
• 11e4e03: implements points 2–4 of the comment above (MeasuredShape and check_cost_table). The PR body is updated.

Please pull with git reset --hard origin/<branch> before pushing again.

[milindsrivastava1997]

autosketch_accuracy has no caller. autosketch::plan itself has no non-test caller, and its tests still pass the table-based oracle, so the Q12 claim (AutoSketch-Adapted uses the curve lookup) isn't wired up or tested. Either hook it into the AutoSketch baseline and add a test, or drop it until it's needed.

[milindsrivastava1997]

load() keys rows by their raw param/cardinality strings, while scripts/study_saturation.py's point_key compares them numerically (pandas round-trips turn "1000" into "1000.0"). If the two CSVs, or the two runs, format a value differently, you get either curve row for a point not in saturation.csv, or duplicate points: bracketing_points then sees more matches than shape_params.len()*keys.len(), returns None, and silently makes the config ineligible. Parse these fields to numbers (or normalize them) before keying.

[milindsrivastava1997]

This detects exact accumulators by the "exact-" name prefix, but lib.rs already has a family registry (family_properties / the exact-* family list). Using the registry keeps "which families are exact" defined in one place.

[milindsrivastava1997]

value.is_finite() && accuracy_ok(value) is also duplicated in autosketch.rs:153. Make it a single Raqe method (e.g. Raqe::meets_sla(Option<f64>)) so the MILP and the AutoSketch baseline can't drift apart on eligibility.