Skip to content

rqe-optimizer: workload_scenarios example (lookback/interval, grouping, type and SLA sweeps) - #177

Merged
milindsrivastava1997 merged 3 commits into
mainfrom
rqe-optimizer-quantile-scenarios
Oct 9, 2026
Merged

milindsrivastava1997 merged 3 commits into
mainfrom
rqe-optimizer-quantile-scenarios

Conversation

@milindsrivastava1997

@milindsrivastava1997 milindsrivastava1997 commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Adds rqe-optimizer/examples/workload_scenarios.rs: sweeps that show how the MILP's optimal deployment moves as the query workload changes. Each sweep steps one knob of a base workload; each step is solved with the MILP and printed as the deployments it runs, the queries each one serves, and per-query merges, latency and achieved accuracy vs SLA.

cargo run --release -p rqe-optimizer --example workload_scenarios -- \
  --saturation-dir <study dir with out_grid_1e7_cost/, out_1e9/ and optimizer_cost/> \
  [--sweep lookback-interval|mixed-grouping|mixed-type|latency-sla|accuracy-sla] \
  [--w-cpu X --w-mem Y] [--allow-undeployable-families]

Sweeps

Sweep Workload Knob
lookback-interval 10 p99 queries on 10 distinct metrics (lookback, interval) ∈ {5m,1h,6h,1d,7d} × {15s,1m,5m}, shared by all 10
mixed-grouping 1h p99 by (service) and by (service, endpoint) each alone, then both
mixed-type 1h rate, p99, distinct count, top-k each alone, then together
latency-sla p99 over 1h/6h/1d of one metric p99_1h latency SLA ∈ {none, 5, 2, 1, 0.5, 0.2} ms
accuracy-sla p99 over 1h/6h/1d; distinct count over 1h/6h/1d 1h query's SLA: rank error ∈ {0.05…0.001}; relative error ∈ {0.1…0.01}

Results on out_10_6_26_1342 (w_cpu=1, w_mem=0), before rebasing onto #173/#174

That export no longer parses (rows now need accuracy_metric). The example is rebased and builds against current main but has not been rerun; these results will be refreshed from a study_saturation.py --phase optimizer-cost run.

  • lookback-interval: always KLL k=200; window x grows with lookback (1m at 5m → 35m at 7d), slide = interval, memory ∝ lookback/interval (7d @ 15s ≈ 129 GB for 10 queries).
  • latency-sla: p99_1h/6h/1d share one KLL deployment throughout. Tightening p99_1h does not split it off; the shared window grows instead (15m → 30m at 1 ms → 1h at 0.5 ms), raising CPU 1.45e-3 → 3.18e-3. 0.2 ms is infeasible.
  • accuracy-sla, distinct count: shared HLL lg_k=12 until distinct_1h's relative-error SLA reaches 0.01; then distinct_1h splits onto its own HLL lg_k=14 and 6h/1d stay on lg_k=12.
  • accuracy-sla, p99: KLL k=200 gives rank error 0.0026 at 1h; at ≤ 0.002 p99_1h is unservable with this export (no k=800 row).
  • mixed-type: top-k is unservable with this export (cms-heap cols=2048 is not in the saturation grid).

Not done

  • Rerun every sweep on a current study directory (curves + optimizer_cost/) and refresh the results above.
  • Cost-table vs saturation-curve mismatches still warn instead of panic; switch to a panic like small_problem now that both come from one study.
  • After rqe-optimizer: let mergeable finer-group deployments serve coarser RAQEs #187 (finer groupings serve coarser RAQEs), mixed-grouping stops being a baseline.

🤖 Generated with Claude Code

@milindsrivastava1997 milindsrivastava1997 changed the title rqe-optimizer: quantile_scenarios example sweeping lookback and interval rqe-optimizer: workload_scenarios example (lookback/interval, grouping, type and SLA sweeps) Oct 7, 2026
milindsrivastava1997 and others added 3 commits October 9, 2026 11:16
Each scenario is 10 quantile_over_time RAQEs on distinct metrics sharing
one (lookback, interval); the example solves each with the MILP and prints
one row per scenario.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… sweeps

Renames quantile_scenarios to workload_scenarios and adds sweeps over mixed
groupings, mixed query types, a latency SLA and an accuracy SLA. Each step
prints the deployments the MILP runs and the queries each one serves.
--sweep NAME runs one sweep.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… study's cost table

Raqe no longer names an accuracy metric, so each query keeps only its SLA
and the output prints the chosen family's metric. The cost table is read
from the saturation directory, so --cost-dir is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@milindsrivastava1997
milindsrivastava1997 force-pushed the rqe-optimizer-quantile-scenarios branch from df2ee52 to c80789b Compare October 9, 2026 15:21
@milindsrivastava1997
milindsrivastava1997 marked this pull request as ready for review October 9, 2026 15:39
@milindsrivastava1997
milindsrivastava1997 merged commit 1c860e6 into main Oct 9, 2026
4 checks passed
@milindsrivastava1997
milindsrivastava1997 deleted the rqe-optimizer-quantile-scenarios branch October 9, 2026 15:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant