Repository navigation
Conversation
This was referenced Oct 4, 2026
zzylol
force-pushed
the
stack/509-d1-stage-pipeline-facade
branch
from
October 5, 2026 06:21
8a986cc to
b9a779e
Compare
zzylol
force-pushed
the
stack/509-d2-stage1-without-cost
branch
from
October 5, 2026 06:21
af65096 to
07c4e47
Compare
zzylol
force-pushed
the
stack/509-d1-stage-pipeline-facade
branch
from
October 8, 2026 16:47
b9a779e to
5684cac
Compare
zzylol
force-pushed
the
stack/509-d2-stage1-without-cost
branch
2 times, most recently
from
October 10, 2026 14:32
3f6ba16 to
ac43130
Compare
Move the cost- and recurrence-dependent selection over `CandidateLogicalASAPDAGs` (cost ranking, recurrence profiles, global selection, selected-DAG assembly, runtime support evidence) and its tests from `replacement` into `plan_selection::candidate_selection`. The code is unchanged; only visibility and imports were adjusted. `plan_selection.rs` becomes `plan_selection/mod.rs`. `recurrence_profiles_from_workload` is deleted rather than moved: it has no caller anywhere in the workspace. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t model `realizations_for_intent` lists sketch candidates in `summary_candidates` order and sizes them with the analytical `accuracy::estimators::size_params`, which is what `CostModel::size_params` defaulted to. Extension intents stay PassThrough, the previous default. - `CandidatePlanningInputs` drops its `cost` field; `realize_child` drops its cost argument. - `ASAPStrategies` and `HydraGroupingStrategy` are built with `default()` or `new_with_planning_inputs*` without a cost model; `ASAPStrategies::new`, the `default_cost_model` constructors and `default_strategies_with` are removed, and `default_strategies_with_evidence` takes only the evidence. - `ExactCompositionStrategy` is a unit struct and no longer checks runtime support; selection already requires positive support evidence. - `CostModel` loses `size_params`, `realize_extension` and `evaluation_extension`. Tests that exercised only the removed hooks are deleted; call sites across the workspace use the new constructors. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Scans the Stage 1 modules' non-test code and fails on any import of `cost_model` or `recurrence`, including names `lib.rs` re-exports from them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Update the developer docs and the `CostModel` trait doc: strategies take no cost model, sketch sizing is analytical, extension intents stay pass-through, and the cost model is consulted only at selection. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rage Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol
force-pushed
the
stack/509-d2-stage1-without-cost
branch
from
October 11, 2026 17:11
ac43130 to
7eadcd5
Compare
3 tasks
zzylol
added a commit
that referenced
this pull request
Oct 13, 2026
…e MajorPass (#674) * feat(planner): run the planning stages as the planner pipeline; retire MajorPass plan_stages runs logical ASAP-aware optimization (Pass 1 and composition of every combination, capped), physical ASAP-aware optimization and plan selection over the frontends' roots, and returns every stage's output. StagePipeline runs it behind OptimizationPass and is the facade default; the stage_pipeline devtool uses it (its fixture is unchanged). The combination count saturates instead of overflowing. summary_sharing: sharing tests are ignored until Pass 2 (#580); the UnivMon legacy-search test is deleted; scalar and combined-quantile tests run on plan_stages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test(planner): logical and physical ASAP-aware optimization import no cost model From #582: only plan selection prices plans, so the Pass 1 / composition and physical ASAP-aware optimization modules may not import cost_model, recurrence, analytical_cost, physical_operator_statistics or plan_selection (paths or re-exported names) outside tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * fix(planner): compose a per-series avg over rows that carry the series identity The per-series sum keeps the input's labels, so with $promql_series_identity after the sample value its last column is the identity, and composition rejected the only exact option ("non-Float64 per-series avg"). The planner pipeline then had no candidate for avg_over_time. Read the summed value by its role instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test(executor): take plans and candidates from the planner pipeline From #629/#633: executor tests take the plan plan_stages selects (selected_dag) or its composed logical candidates (logical_candidates) instead of the legacy search and global selection. - deployment_computation, precompute_candidates, the candidates.rs fixture: ported. - planspace_series_identity_heap: keeps the selection check; the four tests of the legacy whole-root proposals are deleted. - weighted_topk_binding: ported to Pass 1 heap candidates (Count Sketch only over rates: no sign proof). The tests of legacy-only candidate generation are deleted. Ignored, with reasons: the grouped weighted binding (whole-expression heap, #589) and the two rate-heap executions (top-k output schema, #672). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test(frontend-promql): take plans and candidates from the planner pipeline From #629/#633, on support helpers over plan_stages (plan_stages, selected_dag, logical_candidates, logical_plan): - count_planning: exact counts select a Count accumulator; count updates are read from the composed Count-Min / Count Sketch candidates. The Hydra test is ported and ignored: Pass 1 offers no Hydra for a grouped count yet (#605). - univmon_candidates: the synthetic accuracy model is gone; UnivMon candidates come from Pass 1 (the shared count at an approximate target), the uncalibrated ones carry no guarantee and plan selection does not select them. - promql_corpus, metrics_observability: totality over Pass 1 and composition (logical_plan) instead of the legacy strategies; the metrics corpus reports composition errors under METRICS_OBSERVABILITY_REPORT. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test(frontend-sql): take corr's exact plan from the planner pipeline From #633: the plan plan_stages selects replaces retain_exact. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test(planner): the batch SUM example reads one exact accumulator per query Under the planner pipeline each query's SUM is replaced by its exact accumulator state; sharing identical producers is Pass 2 (#580). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * refactor(planner): stage identifiers carry number and name; enumerate every candidate User decisions (2026-10-13): - stage modules stage2_physical_asap_optimization and stage3_plan_selection; tests stage1_logical_asap_optimization.rs and stage2_physical_asap_optimization_sql.rs; stage_pipeline keys stage0_query_language_frontends, stage1_logical_asap_optimization, stage2_physical_asap_optimization, stage3_plan_selection (fixture regenerated: only the keys change); - enumeration and selection are never capped: plan_stages enumerates every combination; the devtool's --max-displayed (was --max-candidates) caps only the listing; - code docs say "query-language frontends". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test(integration): take plans and candidates from the planner pipeline From #629/#633, on asap_integration_tests::planning (plan_stages, selected_dag, logical_candidates): - sql_to_physical, operator_sharing, materialization_e2e, precompute_raw_samples, summary_coverage_e2e, promql_numeric_regressions: ported. - cse.rs: deleted; it pinned the legacy search's memo groups (sharing identical sub-DAGs is Pass 2, #580). - Ignored, with reasons: the UNION ALL summary test (plan selection picks the exact plan until materialization, #604), arithmetic over one selector (#580), the unit-weight heap over raw samples (#589). - precompute_raw_samples no longer requires a Count Sketch heap over raw samples (#589); materialization_e2e takes the quantile's sketch candidate, since plan selection picks the exact plan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * refactor(devtools): sketch_coverage reports Pass 1's summary candidates From #633: per lowered query, the summary candidates Pass 1 offers, instead of the legacy explain_replacements. The CSE-shareable metric is dropped (sharing is Pass 2). Over every corpus at epsilon 0.01 and a 1 s ingestion interval: 145/2183 lowered queries get a summary candidate, and Pass 1 rejects none. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test(frontend-promql): the Kubernetes mixin aggregates over topk compose (ignored, #672) Both corpus queries that do not compose aggregate over a topk output, which the frontend declares as [keys.., topk_k] with a Utf8 placeholder and no value column; the outer sum's accumulator has no value to read. They compose once the top-k output carries the ranked value (#672). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * refactor(devtools): run_planning_stages runs the library's plan_stages One pipeline implementation: the devtool's run_planning_stages (#577) validates the frontends' roots and every composed candidate, and otherwise is plan_stages with the built-in models. Its candidate types are the library's. The Example 1 fixture is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
Closing as part of the upper-stack consolidation (G1): its cost-independence test is in #674; its legacy-selection move is superseded because the legacy search is deleted (G2) instead of moved. Closed with the author's approval. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebased on main d4869a7 (DF 54).
Why
#572 ("Update (2026-10-04)") records decision Q36(a): following #509, only Stage 3 uses the cost model. After #581 the planner facade already runs Stage 1 → Stage 2 → Stage 3, but the Stage 1 modules still imported
cost_modelandrecurrence: candidate generation asked aCostModelhow to order and size sketches, and the legacy whole-workload selection lived insidereplacement.rs. That coupling also blocks moving Stage 1 into its own crate (Phase B, step B2).What
Stage 1 no longer depends on the cost model. Planner facade behavior is unchanged: the #581 tests pass and
stage_pipelineregenerates the Example 1 fixture byte for byte.Before this PR (production imports):
After this PR:
A new test,
stage1_cost_independence, fails if a Stage 1 module importscost_modelorrecurrenceoutside#[cfg(test)].How
Cost-free candidate generation.
realizations_for_intent(intent)lists sketch candidates insummary_candidatesorder and sizes them withaccuracy::estimators::size_params.CostModel::size_paramsdefaulted to the same function. Extension intents stayPassThrough, which was the previous default.CandidatePlanningInputsdropscost, andrealize_childdrops its cost argument.Cost-free strategies. Strategies are now built without a cost model:
ASAPStrategies::default()andHydraGroupingStrategy::default();ExactCompositionStrategy, now a unit struct;new_with_planning_inputs*, without the cost argument;default_strategies_with_evidence(evidence).The strategy-level runtime-support check is dropped, because selection already requires positive support evidence before it commits a composition.
Pure move. The cost- and recurrence-dependent selection block moves from
replacement.rsto the newplan_selection/candidate_selection.rs(plan_selection.rsbecomesplan_selection/mod.rs), together with its tests. Only visibility and imports change. Its deletion is tracked in Pass 2 sharing and Stage 1 coverage parity with the retired MajorPass #580.Call sites. Call sites in integration-tests, asap-physical-operators, frontend-promql, planner and devtools use the new constructors, and
docs/develop_docsno longer describes the removed hooks.Moved to
plan_selection::candidate_selection(1,718 production lines and 22 tests)ReplacementSubDAG::runtime_support_evidenceCandidateLogicalASAPDAGs::{cost_sorted, cost_sorted_with_recurrence, recurrence_profiles, global_selection, global_selection_with_recurrence}RecurrenceProfileMap,RankedTargetSubDAGCandidates,rank_group,cse_preference,realize_one,sketch_kind_of,summary_groupingTargetSubDAGSelection,CompositionDecision,GlobalSelection(withassemble_*),composition_options,is_automatically_selectable,decide_*,multiplier,ReferenceDAG,topological_order, and their helpersThese are still used by tests,
devtools dag_export --post-asapandintegration-tests, so they are moved rather than deleted. The public types are still re-exported at the crate root.Deleted
CandidateLogicalASAPDAGs::recurrence_profiles_from_workloadCostModel::{size_params, realize_extension, evaluation_extension}ASAPStrategies::{new, default_cost_model},HydraGroupingStrategy::{new, default_cost_model},ExactCompositionStrategy::{new, default_cost_model},default_strategies_withdefault_strategies_with(m)equalsdefault_strategies()once the cost model is gone)Ten tests were deleted, each because it exercised only a removed hook:
cost_model::size_params_default_body_matches_default_size_paramssize_paramscost_model::custom_cost_model_can_override_sizing_independently_of_rankingsize_paramsreplacement::extension_intent_realizes_via_custom_cost_modelrealize_extensionexact_composition::a_runtime_without_the_capability_gets_no_candidateintegration-tests exact_composition::a_runtime_without_mixed_execution_gets_no_composition_candidatesunknown_runtime_capability_keeps_candidate_but_prevents_selection)replacement::realize_with_custom_cost_model_overrides_default_summary_choicerealize_childreplacement::custom_cost_model_still_enumerates_every_candidate_not_just_its_own_pickapproximate_quantile_enumerates_every_summary_candidate)grouping::custom_cost_model_cannot_enable_unproven_hydra_kllquantile_has_no_hydra_candidate_without_a_modeled_error_bound)explanation::custom_cost_model_changes_the_reported_sketch_kindempirical_cost::public_cost_model_changes_replacement_order_without_changing_guaranteessize_paramsOne test was adapted:
replacement::enumerating_the_targets_candidates_does_not_leak_into_a_nested_aggregatenow expects the inner aggregate's static order[Kll, DDSketch]instead of a custom cost model's order.Behavior note (legacy search path only). A custom cost model no longer reorders or resizes candidates during generation. It still ranks them at selection time through
rank_candidates,candidate_costand the other hooks. The in-tree production models (PhysicalPlanCostModel, thedag_exportexport model) already used the identity ranking.Remaining B2 blockers (moving Stage 1 into a
logical-optimizercrate)Stage 1 imports Stage 3:
replacement.rs:383importsplan_selection::candidate_selection::{GlobalSelection, TargetSubDAGSelection}.enumerate_candidate_dags*builds them atreplacement.rs:4240,replacement.rs:4249andreplacement.rs:4258(assemble_target) to assemble the unpriced candidate inventory.candidate_selection.rsadds inherent impls on Stage 1 types:impl ReplacementSubDAG(candidate_selection.rs:34) andimpl CandidateLogicalASAPDAGs(:53,:199,:1211). These are not allowed across crates. They also use Stage 1 internals that are onlypub(crate):groups,orderandcomposition_plans;PreparedComposition;cse_candidate_pair,direct_child_countsandrealize_child(candidate_selection.rs:28-29).Resolved by Pass 2 sharing and Stage 1 coverage parity with the retired MajorPass #580.
The shared
accuracymodule is used by both Stage 1 and Stage 3, and it imports Stage 1:accuracy/mod.rs:31importsexact_composition::ExactOperation;accuracy/estimators/mod.rs:160usesreplacement::accuracy_budget;accuracy/composition.rs:453uses the privatefunction_rules.It has to move with Stage 1, or be split.
cost_model.rs:52-53,:858and:875call thepub(crate)replacement::realize_child. The direction is fine, but this needs a public API once Stage 1 is a separate crate.No Stage 1 module imports Stage 2 (
physical_candidates) or the facade (pass).Gate
cargo fmt --all --check: passes.cargo clippy --workspace --all-targets --all-features -- -D warnings: passes.cargo test --workspace: 1,524 passed and 12 ignored. That is the 1,533 from feat(planner): run the #509 stage pipeline with tree-DP selection; retire MajorPass #581, less the 10 deleted tests, plus the new guard.python3 -m unittest discover -s tools/dag-viewer -p test_render.py: 29 tests OK, 6 skipped.stage_pipeline --example planner-layering-1regeneratestools/dag-viewer/examples/planner-layering-example1.jsonbyte for byte.Part of #509. Follows #572 (decision Q36(a)). Follow-up: #580.
🤖 Generated with Claude Code