Skip to content

refactor: take native candidates from PlanSpace; drop forests and rate placement - #795

Open
zzylol wants to merge 1 commit into
feat/lifecycle-placementfrom
feat/planspace-compile-once
Open

zzylol wants to merge 1 commit into
feat/lifecycle-placementfrom
feat/planspace-compile-once

Conversation

@zzylol

@zzylol zzylol commented Sep 30, 2026

Copy link
Copy Markdown
Contributor

Stacked on #794.

Why

Startup built one whole-workload forest per per-query Planner alternative, called SketchAlgorithmStrategy directly on identity-typed roots, derived fixed-window/query-time Rate variants itself (rate_placement.rs), and compiled every forest for quotes. The 44-query issue workload missed its 30 s readiness budget.

What

  • Native realizations come from PlanSpace: planner_selection::enumerate_workload_candidates (search_workload_with_targets + enumerate_candidate_dags_for_root) over the identity-typed root (row-level realizations) and the canonical root (whole-root realizations such as a current-series heap). Direct strategy calls and rate_placement.rs are deleted.
  • A state built from retained readouts (heap or grouped Sum over per-series Rate) is placed by the backend lifecycle costs of the previous PR. Planner binds the choice (SummaryMaintenanceLifecycleCandidates::select); its execution_timed_dag() feeds Planner's fixed-window compiler when the state is maintained, and the same timing is written into the root for backend state selection. Rebuilt per query, it uses Planner's query-time ranking over readouts. Only the chosen placement of each logical candidate is compiled; no per-placement variants are enumerated.
  • Per-root forests of other alternatives are no longer built: Planner's global selection (with the backend ControlPlaneCostModel) picks the computation, and per-state Ephemeral placement replaces per-root raw substitution.

Before / After

44-query issue workload, debug build, process start → /api/v1/health:

#793 previous PR this PR
candidates 164 84 4
startup 69.3 s 38.6 s 3.1 s

sum by(job)(rate(requests_total[1m])): before, both the maintenance and query-time placements were separate quote candidates; now store_per_byte_second = 0 installs the maintenance program and 1.0 (local-only) rebuilds the Sum per query.

Behaviour differences

  • Other sketch families of a root (e.g. KLL vs DDSketch) are ranked in the Planner selection trace but no longer compiled and quoted as separate workloads. Deployments that relied on quotes to override Planner's family choice lose that; the ERP/quote path still prices the remaining whole-workload candidates by exact manifest equality.
  • planner.physical_candidate trace events are kept (physical_dag / physical_candidate / unsupported / rejected); the logical trace drops candidate_assembly_rejections and computation_search_scope.
  • Native heap/Rate tests now select placement by store price instead of expecting both placements in the inventory. Per-forest decisions are traced as deployment.native_candidate_placement.
  • Known limit: a root with nested readout-built states (a readout-built state over another) has no native candidate.

Validation

  • cargo fmt --all -- --check, cargo clippy --workspace --all-targets --locked -- -D warnings
  • cargo test --workspace --locked --lib, cargo test -p control_plane --locked --tests
  • cargo test -p data_plane --locked --test asapquery_compatibility_process_e2e -- --test-threads=1: 26/26, including issue_workloads_execute_warm_at_successive_evaluations at scale 1 with the unchanged 30 s budget.

🤖 Generated with Claude Code

…cycle

The compiler built one whole-workload forest per per-query Planner
alternative, called SketchAlgorithmStrategy directly on identity-typed
roots, and derived fixed-window/query-time Rate variants itself
(rate_placement.rs). Every forest was then fully compiled for quotes.

Native realizations now come from PlanSpace (search_workload_with_targets
and enumerate_candidate_dags_for_root) over the identity-typed and the
canonical root. A state built from retained readouts is placed by the
backend's lifecycle costs: Planner binds the choice, its lifecycle-timed
DAG selects the fixed-window program, and the timing is written into the
root. Other per-root alternatives are no longer compiled as forests:
Planner's global selection picks the computation and per-state lifecycle
placement replaces per-root raw substitution.

For the 44-query issue workload the inventory drops from 84 to 4
candidates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol

zzylol commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by the #818 → #822 stack (draft). The backend no longer selects plans: it executes the PhysicalASAPDAG that ASAPPlanner selects (#509), with the Planner pinned at one stack revision. Leaving this open for now; close it when the new stack is accepted.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant