Stage 3: memory, disk and latency trade-offs per candidate, and cost models beyond cloud billing #660

Why

A deployment that runs on its own cluster does not pay per CPU-second or per GB-month. It has a fixed amount of memory and disk and a latency guarantee to meet. To choose a plan it needs the trade-off: how much memory and how much disk each candidate requires, and what latency each one guarantees. Today Stage 3 collapses every candidate into one operational cost, priced like a cloud bill. That is one useful cost model, but not the only one a deployment needs.

Current state (top of the #528 stack, crates/plan-selection/src/lib.rs)

• One cost model (analytical-cost-v2, design in docs/design_docs/proposals/stage3-cost-model.md, Stage 3 per-second cost model, memory pricing and calibration #594). A candidate's cost per second of wall time is CPU operations, scanned bytes and retained byte-seconds, each times a price (Stage3Calibration::ILLUSTRATIVE, explicitly illustrative). Selection takes the cheapest valid candidate.
• Constraints: DeploymentCapabilities.memory_budget_bytes rejects a candidate that retains more state; each query's LatencyRequirement is checked against the query-time work in one evaluation (latency_ms_per_cost_unit).
• Memory and disk are not separated. retained_bytes adds up ingestion-time state, query-time panes kept across evaluations and raw data retained for query-time scans (priced as memory, Q48). There is no disk budget.
• Only the scalar cost is reported per candidate; the resource amounts behind it are not.

Proposal

1. A resource profile per candidate, in absolute units and without prices:
   • memory: state held in memory (peak and retained);
   • disk: retained raw data and persisted summary or pane state;
   • latency: estimated response time per query, against its bound;
   • CPU: ingestion-time work (per second of ingestion) and query-time work (per evaluation).
2. Several cost models over the same profile, chosen by the deployment:
   • Cloud operational cost (today's model): price each dimension, minimize the total.
   • Own cluster: memory capacity, disk capacity and the latency guarantee are hard constraints; among the feasible candidates, choose by a declared objective (for example lowest latency, or least memory then least disk).
   • Trade-off report: return the Pareto frontier of the valid candidates over (memory, disk, latency) instead of a single choice, so the user picks.
3. Deployment inputs: add a disk budget next to memory_budget_bytes, and say where each kind of state lives (memory or disk) so the two can be checked separately. The latency guarantee is already per query (LatencyRequirement).
4. Output: per candidate, the resource profile and its cost under each model, plus the frontier, in PlanOutput, the devtools and the DAG viewer. Evaluation figures show each candidate's absolute memory, disk and latency, not ratios.

Open questions

• What decides whether a summary or a pane is held in memory or on disk: the deployment (a storage tier per state kind), or the plan?
• Where latency estimates come from: today's analytical factor, or empirical evidence (cost model: validate analytical estimates against empirical benchmark evidence #347, Feed empirical sketch error, resource usage, and data distributions into planner estimates #322).
• Selection is a dynamic program for one scalar objective (exhaustive only up to 64 combinations). A frontier needs a multi-objective version or enumeration.
• Peak memory versus retained memory: which one a capacity constraint should check.

Related

#525 (deployment inputs bundle), #526 (latency bounds and loss reasons), #323 (resource estimation from algorithm complexity), #436 (CostModel responsibilities), #347 / #322 (empirical evidence), #604 (Stage 2 materialization, which decides what is retained).