Repository navigation
Conversation
zzylol
force-pushed
the
stack/509-x9-pass1-coverage
branch
from
October 4, 2026 20:18
31ecc84 to
7c0019e
Compare
zzylol
force-pushed
the
stack/509-x9b-hydra-pass1
branch
from
October 4, 2026 20:18
79f0dff to
b89974e
Compare
This was referenced Oct 4, 2026
zzylol
force-pushed
the
stack/509-x9b-hydra-pass1
branch
2 times, most recently
from
October 8, 2026 16:47
70a252c to
3f3c394
Compare
zzylol
force-pushed
the
stack/509-x9-pass1-coverage
branch
2 times, most recently
from
October 10, 2026 14:32
36bc40a to
cb917b6
Compare
zzylol
force-pushed
the
stack/509-x9b-hydra-pass1
branch
from
October 10, 2026 14:32
3f3c394 to
ba09d63
Compare
A count with `by` groups and an approximate target gets one more alternative (#580 W7): one shared Count-Min grid for every group, as the SummaryAgg contract of #600 states. The update is a unit weight per row, hashed by a non-null Utf8/Int64/Bool column (a grouping column first, else the PromQL series identity). Grouping is a new per-alternative axis on LocalLogicalTarget, carried through tumbling forms. Stage 3's analytical guarantee is now grouping-aware: for HydraCms it is pass1::grouping::hydra_guarantee over the inner Count-Min, with collision term e/shared_columns and failure probability e^-shared_rows (relative to the whole input's weight). Pass 1 sizes the inner sketch and the grid for ε/2 and δ/2 each so the sum meets the target. Other shared groupings have no model. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#604 gave RootDemand a latency_ms and a logical candidate several physical candidates. The Hydra count test sets no latency bound and checks the all-query-time physical candidate, which comes first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol
force-pushed
the
stack/509-x9b-hydra-pass1
branch
from
October 11, 2026 17:11
ba09d63 to
e2d4f2a
Compare
3 tasks
zzylol
added a commit
that referenced
this pull request
Oct 12, 2026
The dev doc describes per-consumer HAVING entries, the adjacent-form scope and that TopK Hydra is added in #605. planner-layering says a filter or HAVING does not prevent accumulator or summary candidates, and Example 1 says Sum has no summary family rather than pruned ones. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
3 tasks
zzylol
added a commit
that referenced
this pull request
Oct 13, 2026
The dev doc describes per-consumer HAVING entries, the adjacent-form scope and that TopK Hydra is added in #605. planner-layering says a filter or HAVING does not prevent accumulator or summary candidates, and Example 1 says Sum has no summary family rather than pruned ones. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol
added a commit
that referenced
this pull request
Oct 13, 2026
The dev doc describes per-consumer HAVING entries, the adjacent-form scope and that TopK Hydra is added in #605. planner-layering says a filter or HAVING does not prevent accumulator or summary candidates, and Example 1 says Sum has no summary family rather than pruned ones. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol
added a commit
that referenced
this pull request
Oct 13, 2026
The dev doc describes per-consumer HAVING entries, the adjacent-form scope and that TopK Hydra is added in #605. planner-layering says a filter or HAVING does not prevent accumulator or summary candidates, and Example 1 says Sum has no summary family rather than pruned ones. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol
added a commit
that referenced
this pull request
Oct 13, 2026
) * feat(planner): enumerate phase-free local logical alternatives * docs(planner): say local candidates have no execution timing assigned Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(planner): call the promoted PromQL workload lowering The PromQL frontend's unified module was promoted to the crate root later in the stack; asap_frontend_promql::unified no longer exists. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(planner): validate local candidates with validate_structure after the export removal Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * fix(planner): Pass 1 local candidates per #509 Summary candidates are unsized (LocalCandidate::Summary carries only the algorithm); sizing happens after Pass 2 sharing. Every pruned candidate is recorded on its target with a PruneReason (RequiresExact, NullableAvgInput). Each measure of an aggregate with a realization besides exact execution is its own target; input containing ASAP operators is rejected. avg is rewritten as exact sum/count accumulator states. The Example 1 test runs Q1 as exact, and the dev doc places composition in Pass 1. - One exact option per target: the exact accumulator (ExactAccumulator, or AvgAsSumCount for avg) when it can be built, otherwise plain Exact (NullableAvgInput prune, or no accumulator form). The plain form would duplicate "accumulator, not materialized"; Stage 2/3 decide materialization and sharing. - Per-measure filters and HAVING keep every candidate. The target records the measure's filter (applied to the state as SummaryAgg.filter) and a HavingPlacement for the aggregate's having or a Filter directly above it (the SQL frontend's HAVING form): BelowAggregate when it reads only group keys (pushed below as a row predicate), else AboveOutput (a Filter above the finalize/estimate output; approximate candidates carry guarantee = None). Composition (#575) applies them; PruneReason::FilteredAggregate is removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * fix(planner): per-consumer HAVING entries, global HAVING above, stricter Pass 1 input checks LocalLogicalTarget.having is now one HavingEntry (consumer Filter, predicate, placement) per HAVING. A consumer Filter's key-only predicate is pushed below only when every consumer of the aggregate applies it, and a global aggregate's HAVING always stays above the output. Only the adjacent form is recorded; value predicates elsewhere get guarantee None via #575, multi-level push-down is #664. Inputs with assigned materialization are rejected too (AssignedPhysicalAnnotation). Approximate accuracy needs epsilon and delta in (0, 1), and InvalidAccuracy names the root and measure. The avg nullability check runs only for avg. Tests cover SQL HAVING placement, a shared aggregate, a global constant HAVING and #509 Example 1 Q2. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * docs: Pass 1 HAVING placement and filters keep candidates The dev doc describes per-consumer HAVING entries, the adjacent-form scope and that TopK Hydra is added in #605. planner-layering says a filter or HAVING does not prevent accumulator or summary candidates, and Example 1 says Sum has no summary family rather than pruned ones. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * fix(planner): only a Filter over the aggregate is its HAVING A Filter that reads an aggregate only from a scalar expression (PromQL scalar(), subqueries) stays a plain consumer instead of a HavingEntry; filters above are deduplicated by pointer. Print the InvalidAccuracy root once, add tests for Project/scalar consumers, mixed SQL HAVING, without/PerEntity and annotated non-root nodes, and rewrap the doc. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * refactor(planner): name Pass 1 per-computation candidates as in the planning-stages doc Follow the review on #561: use the terms of planner-layering.md (Stage 1, Pass 1: per-computation candidate generation) instead of "local". - Module logical_candidates -> per_computation_candidates; LocalCandidate -> ComputationCandidate, LocalLogicalTarget -> ComputationTarget, LocalChoices -> ComputationChoices (field alternatives -> candidates), LocalLogicalCandidates -> PerComputationCandidates, enumerate_local_logical_candidates -> generate_per_computation_candidates, local_candidates_for_intent -> computation_candidates_for_intent. - The module doc states the stage and pass and carries the rules that were in docs/develop_docs/local-logical-candidates.md, which is removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
zzylol
added a commit
that referenced
this pull request
Oct 13, 2026
The tests run Pass 1 and composition, optimize_physical_asap and select_plan through a shared tests/planner_layering_common pipeline, and follow the decided behavior: three Pass 1 candidates (Q2 exact, CMS+heap, CountSketch+heap), every sub-DAG NotMaterialized, P1 selected, CMS+heap invalid (weights not proven non-negative) and CountSketch+heap invalid (a Top-K output has no guarantee and no deployment evidence). Hydra (#605), Pass 2 sharing (#588) and the two runtime gaps (#593, #579) stay ignored with the PR that delivers them. Tests are named after the stages; the compile check no longer uses the PromQL fallback. The spec lists where the doc and the planner differ. The expected-only viewer fixture is removed: the generated four-stage fixture replaces it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
3 tasks
zzylol
added a commit
that referenced
this pull request
Oct 13, 2026
The tests run Pass 1 and composition, optimize_physical_asap and select_plan through a shared tests/planner_layering_common pipeline, and follow the decided behavior: three Pass 1 candidates (Q2 exact, CMS+heap, CountSketch+heap), every sub-DAG NotMaterialized, P1 selected, CMS+heap invalid (weights not proven non-negative) and CountSketch+heap invalid (a Top-K output has no guarantee and no deployment evidence). Hydra (#605), Pass 2 sharing (#588) and the two runtime gaps (#593, #579) stay ignored with the PR that delivers them. Tests are named after the stages; the compile check no longer uses the PromQL fallback. The spec lists where the doc and the planner differ. The expected-only viewer fixture is removed: the generated four-stage fixture replaces it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol
added a commit
that referenced
this pull request
Oct 13, 2026
…stages (#577) * test: acceptance spec and pending tests for planner-layering Example 1 Define MVP acceptance for #509 Example 1 before the Phase C stage APIs exist: 1 -> 6 -> 6 -> 1 candidates (Pass 1 x identical-expression sharing; physical operator implementation only, no materialization). - Spec with per-stage candidate tables, invariants and doc ambiguities. - Integration tests against todo!() stage stubs, ignored until the stages land; workload and Stage 0 frontend-shape tests run now. - Expected-only asap-stage-pipeline/v1 fixture for the DAG viewer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: run the Example 1 acceptance tests against the real stages Replace the todo!() stubs with adapters over Stage 0-3, un-ignore the 11 tests that pass, and give each still-ignored test the precise difference between the implementation and the spec. Add runtime capability checks: the selected plan compiles in the physical planner; compiling every candidate stays ignored with the two runtime gaps it finds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: read Example 1 logical stages as flat DAGs after the export removal Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test: Example 1 acceptance on the composed stages (1 -> 3 -> 3 -> 1) The tests run Pass 1 and composition, optimize_physical_asap and select_plan through a shared tests/planner_layering_common pipeline, and follow the decided behavior: three Pass 1 candidates (Q2 exact, CMS+heap, CountSketch+heap), every sub-DAG NotMaterialized, P1 selected, CMS+heap invalid (weights not proven non-negative) and CountSketch+heap invalid (a Top-K output has no guarantee and no deployment evidence). Hydra (#605), Pass 2 sharing (#588) and the two runtime gaps (#593, #579) stay ignored with the PR that delivers them. Tests are named after the stages; the compile check no longer uses the PromQL fallback. The spec lists where the doc and the planner differ. The expected-only viewer fixture is removed: the generated four-stage fixture replaces it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test: port #590's Example 2-4 acceptance specs and tests The specs and tests from #590 run the current stages through the shared planner_layering_common pipeline (Pass 1 and composition, optimize_physical_asap, select_plan); tests are named after the stages. Materialization is read from the physical DAG (#667); window forms and retention stay pending adapters. Behavior that works passes (29 tests); every expectation that needs a feature not in this stack is ignored with the PR that delivers it (29 tests): Pass 2 sharing (#588, #595, #596), window composition (#601, #628, #634), materialization enumeration and MatAtQuery (#604, #625), UnivMon sizing and accuracy (#621), SQL recognition and COUNT(*) composition (#598, #603) and plan selection row estimates (#632). Each spec's "Today" column states the current output. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * feat(devtools): stage_pipeline examples for #509 Example 3 Ported from #590: --example planner-layering-3a (Pattern A, ad hoc batch of five p99 reports) and planner-layering-3b (Pattern B, repeating 5-min p99), the Example 3 acceptance tests' workloads. Example 1's output is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test: follow #576's per-consumer charging and exact top-k columns Plan selection charges a NotMaterialized node once per consuming query, so the charging helper checks NodeCost.multiplier and the Example 4 A3 ambiguity is recorded as decided. Example 1 checks that exact Q2 sorts by the ranked value and returns [job, topk_10]. The stage viewer guide's tab 1 row now describes the actual all-exact result. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * refactor(devtools): one planning pipeline for stage_pipeline and the acceptance tests run_planning_stages and the #509 example workloads move into the asap-devtools library; stage_pipeline serializes the run and the planner-layering tests adapt it. The shared data workload drops ingestion_rate for data at rest, which validate() requires. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * fix(devtools): select over every candidate; --max-displayed limits only the document stage_pipeline capped enumeration at 64, so Example 3a (243 candidates) selected P41 instead of the cheapest P122 (KLL for all five queries). The pipeline now enumerates, optimizes and prices every candidate; the document shows the first --max-displayed (default 64) plus the selected one, with `combinations` and `capped`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * docs: Example 2's SQL candidates compose; the SQL test waits only on #598 Running the ignored SQL test composes every candidate and fails only on Q2's options (per-source counts until #598), so #603 is not a blocker. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test: name stages by number and name; say query-language frontends The frontends' Run field, id and locals become stage0_query_language_frontends, and test names carry stage0..stage3 with the stage name. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test: charging helper follows plan selection's per-consumer rule With a materialized node the helper accepted any multiplier >= 1. It now computes each node's computations as plan selection does (walks from query roots and materialized nodes' inputs, stopping at materialized nodes), checked on a hand-built candidate with a materialized node. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test: Example 1 picks Q2 candidates by (option, no shared input) physical_with and the invalid-candidate check no longer assume one candidate per Q2 option, so #588's shared-input variants need no change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * docs: Example 3 Pattern B charges each node once per consuming query Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * docs(stage-viewer): rows 2, 3a and 3b describe the planner's current output Read from uncapped runs: Example 2's sketch plans are rejected for having no (ε, δ) guarantee, 3a selects P122 (first of 108 tied cheapest plans, nothing shared yet) and 3b selects query-time KLL; panes and the latency check come with the window-summary PRs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * docs(stage-viewer): row 2 from the SQL run; 3a ties 32 plans, not 108 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test: Example 1 counts only unshared candidates; stage3_ charging-helper names The 3-logical and 3-physical counts now hold when #588 adds shared-input variants. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * refactor(devtools): PlanningStagesRun fields carry stage number and name Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test(devtools): unique temp file per stage_pipeline run cargo test runs the tests as threads of one process, so two runs of the same example shared a file name and one deleted the other's output. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
3 tasks
zzylol
added a commit
that referenced
this pull request
Oct 13, 2026
…e MajorPass (#674) * feat(planner): run the planning stages as the planner pipeline; retire MajorPass plan_stages runs logical ASAP-aware optimization (Pass 1 and composition of every combination, capped), physical ASAP-aware optimization and plan selection over the frontends' roots, and returns every stage's output. StagePipeline runs it behind OptimizationPass and is the facade default; the stage_pipeline devtool uses it (its fixture is unchanged). The combination count saturates instead of overflowing. summary_sharing: sharing tests are ignored until Pass 2 (#580); the UnivMon legacy-search test is deleted; scalar and combined-quantile tests run on plan_stages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test(planner): logical and physical ASAP-aware optimization import no cost model From #582: only plan selection prices plans, so the Pass 1 / composition and physical ASAP-aware optimization modules may not import cost_model, recurrence, analytical_cost, physical_operator_statistics or plan_selection (paths or re-exported names) outside tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * fix(planner): compose a per-series avg over rows that carry the series identity The per-series sum keeps the input's labels, so with $promql_series_identity after the sample value its last column is the identity, and composition rejected the only exact option ("non-Float64 per-series avg"). The planner pipeline then had no candidate for avg_over_time. Read the summed value by its role instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test(executor): take plans and candidates from the planner pipeline From #629/#633: executor tests take the plan plan_stages selects (selected_dag) or its composed logical candidates (logical_candidates) instead of the legacy search and global selection. - deployment_computation, precompute_candidates, the candidates.rs fixture: ported. - planspace_series_identity_heap: keeps the selection check; the four tests of the legacy whole-root proposals are deleted. - weighted_topk_binding: ported to Pass 1 heap candidates (Count Sketch only over rates: no sign proof). The tests of legacy-only candidate generation are deleted. Ignored, with reasons: the grouped weighted binding (whole-expression heap, #589) and the two rate-heap executions (top-k output schema, #672). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test(frontend-promql): take plans and candidates from the planner pipeline From #629/#633, on support helpers over plan_stages (plan_stages, selected_dag, logical_candidates, logical_plan): - count_planning: exact counts select a Count accumulator; count updates are read from the composed Count-Min / Count Sketch candidates. The Hydra test is ported and ignored: Pass 1 offers no Hydra for a grouped count yet (#605). - univmon_candidates: the synthetic accuracy model is gone; UnivMon candidates come from Pass 1 (the shared count at an approximate target), the uncalibrated ones carry no guarantee and plan selection does not select them. - promql_corpus, metrics_observability: totality over Pass 1 and composition (logical_plan) instead of the legacy strategies; the metrics corpus reports composition errors under METRICS_OBSERVABILITY_REPORT. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test(frontend-sql): take corr's exact plan from the planner pipeline From #633: the plan plan_stages selects replaces retain_exact. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test(planner): the batch SUM example reads one exact accumulator per query Under the planner pipeline each query's SUM is replaced by its exact accumulator state; sharing identical producers is Pass 2 (#580). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * refactor(planner): stage identifiers carry number and name; enumerate every candidate User decisions (2026-10-13): - stage modules stage2_physical_asap_optimization and stage3_plan_selection; tests stage1_logical_asap_optimization.rs and stage2_physical_asap_optimization_sql.rs; stage_pipeline keys stage0_query_language_frontends, stage1_logical_asap_optimization, stage2_physical_asap_optimization, stage3_plan_selection (fixture regenerated: only the keys change); - enumeration and selection are never capped: plan_stages enumerates every combination; the devtool's --max-displayed (was --max-candidates) caps only the listing; - code docs say "query-language frontends". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test(integration): take plans and candidates from the planner pipeline From #629/#633, on asap_integration_tests::planning (plan_stages, selected_dag, logical_candidates): - sql_to_physical, operator_sharing, materialization_e2e, precompute_raw_samples, summary_coverage_e2e, promql_numeric_regressions: ported. - cse.rs: deleted; it pinned the legacy search's memo groups (sharing identical sub-DAGs is Pass 2, #580). - Ignored, with reasons: the UNION ALL summary test (plan selection picks the exact plan until materialization, #604), arithmetic over one selector (#580), the unit-weight heap over raw samples (#589). - precompute_raw_samples no longer requires a Count Sketch heap over raw samples (#589); materialization_e2e takes the quantile's sketch candidate, since plan selection picks the exact plan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * refactor(devtools): sketch_coverage reports Pass 1's summary candidates From #633: per lowered query, the summary candidates Pass 1 offers, instead of the legacy explain_replacements. The CSE-shareable metric is dropped (sharing is Pass 2). Over every corpus at epsilon 0.01 and a 1 s ingestion interval: 145/2183 lowered queries get a summary candidate, and Pass 1 rejects none. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test(frontend-promql): the Kubernetes mixin aggregates over topk compose (ignored, #672) Both corpus queries that do not compose aggregate over a topk output, which the frontend declares as [keys.., topk_k] with a Utf8 placeholder and no value column; the outer sum's accumulator has no value to read. They compose once the top-k output carries the ranked value (#672). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * refactor(devtools): run_planning_stages runs the library's plan_stages One pipeline implementation: the devtool's run_planning_stages (#577) validates the frontends' roots and every composed candidate, and otherwise is plan_stages with the built-in models. Its candidate types are the library's. The Example 1 fixture is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
40 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack: Wave 2 chain: #599 → #601 → #600 → #604 → #606 → #603 → #605
Rebased on main d4869a7 (DF 54). At this head
cargo fmt --all --check,cargo clippy --workspace --all-targets -- -D warningsandcargo test --workspacepass (1642 passed, 0 failed, 23 ignored).Why
#580 item F / W7: Hydra for count and point queries first. Part of #509. #600 added the executor's HydraCms kernel and stated the IR the planner must emit, but Stage 1 never offered Hydra. Stage 3's analytical check also ignored grouping, so it would have certified a shared grid with a per-group sketch's guarantee.
What
SELECT src_ip, COUNT(*) FROM flows GROUP BY src_ipat ε = 0.1, throughplan_stages:Countacc, CMS, CountSketch, UnivMonSummaryAgg{Sketch(Cms), SharedMultiSubpopulation{HydraCms}}→SummaryEstimate{PointCount{SampleValue, None}}. It passes Stage 3's accuracy check, is priced at 0.0157 (the exact accumulator costs 0.0073), compiles, and executes toa=3, b=2, c=1.Selection does not change in these examples:
Countaccumulator, is still selected.How
logical_candidates::add_hydra_alternatives): for a singleCountwith an approximate target andbygroups (notwithout), it emits executor: HydraCms kernel for count and point queries #600's contract:Constant(1.0), with theUnitCountproof.width/depthequal across the family and the grid.LocalLogicalTarget.groupingsis parallel toalternatives, likewindowsandabsorbs. Tumbling forms copy it, since HydraCms merges per executor: HydraCms kernel for count and point queries #600. The capability rule resets it; count targets are never keyed.hydra_guaranteeadds the grid's collision term to the inner sketch's error. So the inner sketch and the grid are each sized for ε/2 and δ/2 (default_size_params+default_hydra_params). At ε = δ = 0.01 that gives a 544 × 6 inner sketch in a 6 × 544 grid.estimators::local_guaranteeis now grouping-aware. For HydraCms it returns the existingpass1::grouping::hydra_guaranteeover the inner Count-Min, with collision bounde/shared_columnsand failure probabilitye^-shared_rows. HydraCountSketch and HydraKll getNone, which means uncertified.stage_pipelinelabels these candidates "HydraCms".pass1/grouping.rsHydraGroupingStrategy, whose guarantee composition is reused.Review points
e·N·(1/shared_columns + 1/width). A small group's count can therefore be dominated by the error, whereas a per-group sketch's bare count is exact. The target is read the same way for both.shared_rows × shared_columnscells, each the size of one per-group sketch. At ε = 0.01 that exceeds the executor's default memory limit, so the execution test uses ε = 0.1.Tests
All four failed or did not exist on the parent:
pass1_sql_coverage::grouped_count_offers_a_priced_executable_hydra_plan(written first; it failed with "a Hydra candidate is generated") checks the contract, Stage 3 pricing, compilation and execution.logical_candidates::grouped_approximate_count_offers_hydrachecks that Hydra is offered only forbyplus an approximate target, and checks the sizes.estimators::cms::hydra_guarantee_adds_the_shared_grid_term: a grid sized like one per-group sketch misses ε, and one sized for the ε/2 split meets it.stage_pipeline_selection::sql_hydra_count_dp_equals_exhaustive: the DP picks the exhaustive minimum over the 18 combinations.Not in this PR
PointCount{Named(item), Some(v)}): noAggIntentasks for one. A filtered count,COUNT(*) FILTER (WHERE service = 'checkout') GROUP BY job, could use it, so it belongs with Pass 2 sharing and Stage 1 coverage parity with the retired MajorPass #580 J (filtered aggregates).Gates
-D warnings): pass.cargo test --workspace: 1,642 passed, 23 ignored. fix(planner): SQL COUNT(*) candidates count rows, not a sample value #603 had 1,638, and this PR adds 4 tests.Stacked on #603. Integration with #604:
fix: integrate with #604setslatency_ms: Nonein the Hydra test's demand and checks the all-query-time physical candidate, since a logical candidate now has several. The numbers above are unchanged on the rebased chain.🤖 Generated with Claude Code