Skip to content

feat(planner): Pass 1 offers HydraCms for grouped approximate counts - #605

Draft
zzylol wants to merge 2 commits into
stack/509-x9-pass1-coveragefrom
stack/509-x9b-hydra-pass1
Draft

zzylol wants to merge 2 commits into
stack/509-x9-pass1-coveragefrom
stack/509-x9b-hydra-pass1

Conversation

@zzylol

@zzylol zzylol commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Stack: Wave 2 chain: #599 → #601 → #600 → #604 → #606 → #603 → #605

Rebased on main d4869a7 (DF 54). At this head cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings and cargo test --workspace pass (1642 passed, 0 failed, 23 ignored).

Why

#580 item F / W7: Hydra for count and point queries first. Part of #509. #600 added the executor's HydraCms kernel and stated the IR the planner must emit, but Stage 1 never offered Hydra. Stage 3's analytical check also ignored grouping, so it would have certified a shared grid with a per-group sketch's guarantee.

What

SELECT src_ip, COUNT(*) FROM flows GROUP BY src_ip at ε = 0.1, through plan_stages:

candidates Hydra
Before this PR 5: pass-through, Count acc, CMS, CountSketch, UnivMon none
After this PR 6 P6: SummaryAgg{Sketch(Cms), SharedMultiSubpopulation{HydraCms}} → SummaryEstimate{PointCount{SampleValue, None}}. It passes Stage 3's accuracy check, is priced at 0.0157 (the exact accumulator costs 0.0073), compiles, and executes to a=3, b=2, c=1.

Selection does not change in these examples:

  • Examples 1, 3a and 3b have no grouped count; their fixtures and DP tests are unchanged.
  • Example 2's design queries go from 400 to 480 candidates; P2, the exact Count accumulator, is still selected.
  • I did not tune costs.

How

  • Pass 1 (logical_candidates::add_hydra_alternatives): for a single Count with an approximate target and by groups (not without), it emits executor: HydraCms kernel for count and point queries #600's contract:
    • item: a non-null Utf8/Int64/Bool column. A grouping column comes first; PromQL labels are nullable, so PromQL uses the series identity.
    • weight: Constant(1.0), with the UnitCount proof.
    • width/depth equal across the family and the grid.
  • New axis: LocalLogicalTarget.groupings is parallel to alternatives, like windows and absorbs. Tumbling forms copy it, since HydraCms merges per executor: HydraCms kernel for count and point queries #600. The capability rule resets it; count targets are never keyed.
  • Sizing: hydra_guarantee adds the grid's collision term to the inner sketch's error. So the inner sketch and the grid are each sized for ε/2 and δ/2 (default_size_params + default_hydra_params). At ε = δ = 0.01 that gives a 544 × 6 inner sketch in a 6 × 544 grid.
  • Stage 3 accuracy: estimators::local_guarantee is now grouping-aware. For HydraCms it returns the existing pass1::grouping::hydra_guarantee over the inner Count-Min, with collision bound e/shared_columns and failure probability e^-shared_rows. HydraCountSketch and HydraKll get None, which means uncertified.
  • Devtool: stage_pipeline labels these candidates "HydraCms".
  • Legacy origin: pass1/grouping.rs HydraGroupingStrategy, whose guarantee composition is reused.

Review points

  • Error basis: the collision term is relative to the whole input's weight, not one group's. This matches executor: HydraCms kernel for count and point queries #600's e·N·(1/shared_columns + 1/width). A small group's count can therefore be dominated by the error, whereas a per-group sketch's bare count is exact. The target is read the same way for both.
  • Memory: the grid holds shared_rows × shared_columns cells, each the size of one per-group sketch. At ε = 0.01 that exceeds the executor's default memory limit, so the execution test uses ε = 0.1.

Tests

All four failed or did not exist on the parent:

  • pass1_sql_coverage::grouped_count_offers_a_priced_executable_hydra_plan (written first; it failed with "a Hydra candidate is generated") checks the contract, Stage 3 pricing, compilation and execution.
  • logical_candidates::grouped_approximate_count_offers_hydra checks that Hydra is offered only for by plus an approximate target, and checks the sizes.
  • estimators::cms::hydra_guarantee_adds_the_shared_grid_term: a grid sized like one per-group sketch misses ε, and one sized for the ε/2 split meets it.
  • stage_pipeline_selection::sql_hydra_count_dp_equals_exhaustive: the DP picks the exhaustive minimum over the 18 combinations.

Not in this PR

  • Point frequency (PointCount{Named(item), Some(v)}): no AggIntent asks for one. A filtered count, COUNT(*) FILTER (WHERE service = 'checkout') GROUP BY job, could use it, so it belongs with Pass 2 sharing and Stage 1 coverage parity with the retired MajorPass #580 J (filtered aggregates).
  • Top-k (W7 defers it), HydraCountSketch, and grids sized smaller than the inner sketch.

Gates

Stacked on #603. Integration with #604: fix: integrate with #604 sets latency_ms: None in the Hydra test's demand and checks the all-query-time physical candidate, since a logical candidate now has several. The numbers above are unchanged on the rebased chain.

🤖 Generated with Claude Code

zzylol and others added 2 commits October 11, 2026 16:22
A count with `by` groups and an approximate target gets one more
alternative (#580 W7): one shared Count-Min grid for every group, as the
SummaryAgg contract of #600 states. The update is a unit weight per row,
hashed by a non-null Utf8/Int64/Bool column (a grouping column first, else
the PromQL series identity). Grouping is a new per-alternative axis on
LocalLogicalTarget, carried through tumbling forms.

Stage 3's analytical guarantee is now grouping-aware: for HydraCms it is
pass1::grouping::hydra_guarantee over the inner Count-Min, with collision
term e/shared_columns and failure probability e^-shared_rows (relative to
the whole input's weight). Pass 1 sizes the inner sketch and the grid for
ε/2 and δ/2 each so the sum meets the target. Other shared groupings have
no model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#604 gave RootDemand a latency_ms and a logical candidate several
physical candidates. The Hydra count test sets no latency bound and checks
the all-query-time physical candidate, which comes first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol
zzylol force-pushed the stack/509-x9b-hydra-pass1 branch from ba09d63 to e2d4f2a Compare October 11, 2026 17:11
zzylol added a commit that referenced this pull request Oct 12, 2026
The dev doc describes per-consumer HAVING entries, the adjacent-form scope
and that TopK Hydra is added in #605. planner-layering says a filter or
HAVING does not prevent accumulator or summary candidates, and Example 1
says Sum has no summary family rather than pruned ones.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 13, 2026
The dev doc describes per-consumer HAVING entries, the adjacent-form scope
and that TopK Hydra is added in #605. planner-layering says a filter or
HAVING does not prevent accumulator or summary candidates, and Example 1
says Sum has no summary family rather than pruned ones.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 13, 2026
The dev doc describes per-consumer HAVING entries, the adjacent-form scope
and that TopK Hydra is added in #605. planner-layering says a filter or
HAVING does not prevent accumulator or summary candidates, and Example 1
says Sum has no summary family rather than pruned ones.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 13, 2026
The dev doc describes per-consumer HAVING entries, the adjacent-form scope
and that TopK Hydra is added in #605. planner-layering says a filter or
HAVING does not prevent accumulator or summary candidates, and Example 1
says Sum has no summary family rather than pruned ones.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 13, 2026
)

* feat(planner): enumerate phase-free local logical alternatives

* docs(planner): say local candidates have no execution timing assigned

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(planner): call the promoted PromQL workload lowering

The PromQL frontend's unified module was promoted to the crate root later
in the stack; asap_frontend_promql::unified no longer exists.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(planner): validate local candidates with validate_structure after the export removal

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(planner): Pass 1 local candidates per #509

Summary candidates are unsized (LocalCandidate::Summary carries only the
algorithm); sizing happens after Pass 2 sharing. Every pruned candidate is
recorded on its target with a PruneReason (RequiresExact,
NullableAvgInput). Each measure of an aggregate with a realization besides
exact execution is its own target; input containing ASAP operators is
rejected. avg is rewritten as exact sum/count accumulator
states. The Example 1 test runs Q1 as exact, and the dev doc places
composition in Pass 1.

- One exact option per target: the exact accumulator (ExactAccumulator,
  or AvgAsSumCount for avg) when it can be built, otherwise plain Exact
  (NullableAvgInput prune, or no accumulator form).
  The plain form would duplicate "accumulator, not materialized"; Stage
  2/3 decide materialization and sharing.
- Per-measure filters and HAVING keep every candidate. The target records
  the measure's filter (applied to the state as SummaryAgg.filter) and a
  HavingPlacement for the aggregate's having or a Filter directly above it
  (the SQL frontend's HAVING form): BelowAggregate when it reads only group
  keys (pushed below as a row predicate), else AboveOutput (a Filter above
  the finalize/estimate output; approximate candidates carry guarantee =
  None). Composition (#575) applies them; PruneReason::FilteredAggregate
  is removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(planner): per-consumer HAVING entries, global HAVING above, stricter Pass 1 input checks

LocalLogicalTarget.having is now one HavingEntry (consumer Filter,
predicate, placement) per HAVING. A consumer Filter's key-only predicate is
pushed below only when every consumer of the aggregate applies it, and a
global aggregate's HAVING always stays above the output. Only the adjacent
form is recorded; value predicates elsewhere get guarantee None via #575,
multi-level push-down is #664.

Inputs with assigned materialization are rejected too
(AssignedPhysicalAnnotation). Approximate accuracy needs epsilon and delta
in (0, 1), and InvalidAccuracy names the root and measure. The avg
nullability check runs only for avg. Tests cover SQL HAVING placement, a
shared aggregate, a global constant HAVING and #509 Example 1 Q2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: Pass 1 HAVING placement and filters keep candidates

The dev doc describes per-consumer HAVING entries, the adjacent-form scope
and that TopK Hydra is added in #605. planner-layering says a filter or
HAVING does not prevent accumulator or summary candidates, and Example 1
says Sum has no summary family rather than pruned ones.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(planner): only a Filter over the aggregate is its HAVING

A Filter that reads an aggregate only from a scalar expression (PromQL
scalar(), subqueries) stays a plain consumer instead of a HavingEntry;
filters above are deduplicated by pointer. Print the InvalidAccuracy
root once, add tests for Project/scalar consumers, mixed SQL HAVING,
without/PerEntity and annotated non-root nodes, and rewrap the doc.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* refactor(planner): name Pass 1 per-computation candidates as in the planning-stages doc

Follow the review on #561: use the terms of planner-layering.md (Stage 1,
Pass 1: per-computation candidate generation) instead of "local".

- Module logical_candidates -> per_computation_candidates; LocalCandidate
  -> ComputationCandidate, LocalLogicalTarget -> ComputationTarget,
  LocalChoices -> ComputationChoices (field alternatives -> candidates),
  LocalLogicalCandidates -> PerComputationCandidates,
  enumerate_local_logical_candidates -> generate_per_computation_candidates,
  local_candidates_for_intent -> computation_candidates_for_intent.
- The module doc states the stage and pass and carries the rules that were in
  docs/develop_docs/local-logical-candidates.md, which is removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 13, 2026
The tests run Pass 1 and composition, optimize_physical_asap and
select_plan through a shared tests/planner_layering_common pipeline, and
follow the decided behavior: three Pass 1 candidates (Q2 exact, CMS+heap,
CountSketch+heap), every sub-DAG NotMaterialized, P1 selected, CMS+heap
invalid (weights not proven non-negative) and CountSketch+heap invalid (a
Top-K output has no guarantee and no deployment evidence). Hydra (#605),
Pass 2 sharing (#588) and the two runtime gaps (#593, #579) stay ignored
with the PR that delivers them. Tests are named after the stages; the
compile check no longer uses the PromQL fallback. The spec lists where the
doc and the planner differ. The expected-only viewer fixture is removed:
the generated four-stage fixture replaces it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 13, 2026
The tests run Pass 1 and composition, optimize_physical_asap and
select_plan through a shared tests/planner_layering_common pipeline, and
follow the decided behavior: three Pass 1 candidates (Q2 exact, CMS+heap,
CountSketch+heap), every sub-DAG NotMaterialized, P1 selected, CMS+heap
invalid (weights not proven non-negative) and CountSketch+heap invalid (a
Top-K output has no guarantee and no deployment evidence). Hydra (#605),
Pass 2 sharing (#588) and the two runtime gaps (#593, #579) stay ignored
with the PR that delivers them. Tests are named after the stages; the
compile check no longer uses the PromQL fallback. The spec lists where the
doc and the planner differ. The expected-only viewer fixture is removed:
the generated four-stage fixture replaces it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 13, 2026
…stages (#577)

* test: acceptance spec and pending tests for planner-layering Example 1

Define MVP acceptance for #509 Example 1 before the Phase C stage APIs
exist: 1 -> 6 -> 6 -> 1 candidates (Pass 1 x identical-expression
sharing; physical operator implementation only, no materialization).

- Spec with per-stage candidate tables, invariants and doc ambiguities.
- Integration tests against todo!() stage stubs, ignored until the
  stages land; workload and Stage 0 frontend-shape tests run now.
- Expected-only asap-stage-pipeline/v1 fixture for the DAG viewer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: run the Example 1 acceptance tests against the real stages

Replace the todo!() stubs with adapters over Stage 0-3, un-ignore the 11
tests that pass, and give each still-ignored test the precise difference
between the implementation and the spec. Add runtime capability checks:
the selected plan compiles in the physical planner; compiling every
candidate stays ignored with the two runtime gaps it finds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: read Example 1 logical stages as flat DAGs after the export removal

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test: Example 1 acceptance on the composed stages (1 -> 3 -> 3 -> 1)

The tests run Pass 1 and composition, optimize_physical_asap and
select_plan through a shared tests/planner_layering_common pipeline, and
follow the decided behavior: three Pass 1 candidates (Q2 exact, CMS+heap,
CountSketch+heap), every sub-DAG NotMaterialized, P1 selected, CMS+heap
invalid (weights not proven non-negative) and CountSketch+heap invalid (a
Top-K output has no guarantee and no deployment evidence). Hydra (#605),
Pass 2 sharing (#588) and the two runtime gaps (#593, #579) stay ignored
with the PR that delivers them. Tests are named after the stages; the
compile check no longer uses the PromQL fallback. The spec lists where the
doc and the planner differ. The expected-only viewer fixture is removed:
the generated four-stage fixture replaces it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test: port #590's Example 2-4 acceptance specs and tests

The specs and tests from #590 run the current stages through the shared
planner_layering_common pipeline (Pass 1 and composition,
optimize_physical_asap, select_plan); tests are named after the stages.
Materialization is read from the physical DAG (#667); window forms and
retention stay pending adapters. Behavior that works passes (29 tests);
every expectation that needs a feature not in this stack is ignored with
the PR that delivers it (29 tests): Pass 2 sharing (#588, #595, #596),
window composition (#601, #628, #634), materialization enumeration and
MatAtQuery (#604, #625), UnivMon sizing and accuracy (#621), SQL
recognition and COUNT(*) composition (#598, #603) and plan selection row
estimates (#632). Each spec's "Today" column states the current output.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* feat(devtools): stage_pipeline examples for #509 Example 3

Ported from #590: --example planner-layering-3a (Pattern A, ad hoc batch of
five p99 reports) and planner-layering-3b (Pattern B, repeating 5-min p99),
the Example 3 acceptance tests' workloads. Example 1's output is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test: follow #576's per-consumer charging and exact top-k columns

Plan selection charges a NotMaterialized node once per consuming query,
so the charging helper checks NodeCost.multiplier and the Example 4 A3
ambiguity is recorded as decided. Example 1 checks that exact Q2 sorts by
the ranked value and returns [job, topk_10]. The stage viewer guide's
tab 1 row now describes the actual all-exact result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* refactor(devtools): one planning pipeline for stage_pipeline and the acceptance tests

run_planning_stages and the #509 example workloads move into the
asap-devtools library; stage_pipeline serializes the run and the
planner-layering tests adapt it. The shared data workload drops
ingestion_rate for data at rest, which validate() requires.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(devtools): select over every candidate; --max-displayed limits only the document

stage_pipeline capped enumeration at 64, so Example 3a (243 candidates)
selected P41 instead of the cheapest P122 (KLL for all five queries).
The pipeline now enumerates, optimizes and prices every candidate; the
document shows the first --max-displayed (default 64) plus the selected
one, with `combinations` and `capped`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: Example 2's SQL candidates compose; the SQL test waits only on #598

Running the ignored SQL test composes every candidate and fails only on
Q2's options (per-source counts until #598), so #603 is not a blocker.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test: name stages by number and name; say query-language frontends

The frontends' Run field, id and locals become stage0_query_language_frontends,
and test names carry stage0..stage3 with the stage name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test: charging helper follows plan selection's per-consumer rule

With a materialized node the helper accepted any multiplier >= 1. It now
computes each node's computations as plan selection does (walks from
query roots and materialized nodes' inputs, stopping at materialized
nodes), checked on a hand-built candidate with a materialized node.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test: Example 1 picks Q2 candidates by (option, no shared input)

physical_with and the invalid-candidate check no longer assume one
candidate per Q2 option, so #588's shared-input variants need no change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: Example 3 Pattern B charges each node once per consuming query

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs(stage-viewer): rows 2, 3a and 3b describe the planner's current output

Read from uncapped runs: Example 2's sketch plans are rejected for having
no (ε, δ) guarantee, 3a selects P122 (first of 108 tied cheapest plans,
nothing shared yet) and 3b selects query-time KLL; panes and the latency
check come with the window-summary PRs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs(stage-viewer): row 2 from the SQL run; 3a ties 32 plans, not 108

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test: Example 1 counts only unshared candidates; stage3_ charging-helper names

The 3-logical and 3-physical counts now hold when #588 adds shared-input
variants.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* refactor(devtools): PlanningStagesRun fields carry stage number and name

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test(devtools): unique temp file per stage_pipeline run

cargo test runs the tests as threads of one process, so two runs of the same
example shared a file name and one deleted the other's output.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 13, 2026
…e MajorPass (#674)

* feat(planner): run the planning stages as the planner pipeline; retire MajorPass

plan_stages runs logical ASAP-aware optimization (Pass 1 and composition
of every combination, capped), physical ASAP-aware optimization and plan
selection over the frontends' roots, and returns every stage's output.
StagePipeline runs it behind OptimizationPass and is the facade default;
the stage_pipeline devtool uses it (its fixture is unchanged). The
combination count saturates instead of overflowing.

summary_sharing: sharing tests are ignored until Pass 2 (#580); the
UnivMon legacy-search test is deleted; scalar and combined-quantile tests
run on plan_stages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test(planner): logical and physical ASAP-aware optimization import no cost model

From #582: only plan selection prices plans, so the Pass 1 / composition
and physical ASAP-aware optimization modules may not import cost_model,
recurrence, analytical_cost, physical_operator_statistics or
plan_selection (paths or re-exported names) outside tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(planner): compose a per-series avg over rows that carry the series identity

The per-series sum keeps the input's labels, so with
$promql_series_identity after the sample value its last column is the
identity, and composition rejected the only exact option ("non-Float64
per-series avg"). The planner pipeline then had no candidate for
avg_over_time. Read the summed value by its role instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test(executor): take plans and candidates from the planner pipeline

From #629/#633: executor tests take the plan plan_stages selects
(selected_dag) or its composed logical candidates (logical_candidates)
instead of the legacy search and global selection.

- deployment_computation, precompute_candidates, the candidates.rs
  fixture: ported.
- planspace_series_identity_heap: keeps the selection check; the four
  tests of the legacy whole-root proposals are deleted.
- weighted_topk_binding: ported to Pass 1 heap candidates (Count Sketch
  only over rates: no sign proof). The tests of legacy-only candidate
  generation are deleted. Ignored, with reasons: the grouped weighted
  binding (whole-expression heap, #589) and the two rate-heap executions
  (top-k output schema, #672).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test(frontend-promql): take plans and candidates from the planner pipeline

From #629/#633, on support helpers over plan_stages (plan_stages,
selected_dag, logical_candidates, logical_plan):

- count_planning: exact counts select a Count accumulator; count
  updates are read from the composed Count-Min / Count Sketch
  candidates. The Hydra test is ported and ignored: Pass 1 offers no
  Hydra for a grouped count yet (#605).
- univmon_candidates: the synthetic accuracy model is gone; UnivMon
  candidates come from Pass 1 (the shared count at an approximate
  target), the uncalibrated ones carry no guarantee and plan selection
  does not select them.
- promql_corpus, metrics_observability: totality over Pass 1 and
  composition (logical_plan) instead of the legacy strategies; the
  metrics corpus reports composition errors under
  METRICS_OBSERVABILITY_REPORT.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test(frontend-sql): take corr's exact plan from the planner pipeline

From #633: the plan plan_stages selects replaces retain_exact.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test(planner): the batch SUM example reads one exact accumulator per query

Under the planner pipeline each query's SUM is replaced by its exact
accumulator state; sharing identical producers is Pass 2 (#580).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* refactor(planner): stage identifiers carry number and name; enumerate every candidate

User decisions (2026-10-13):
- stage modules stage2_physical_asap_optimization and
  stage3_plan_selection; tests stage1_logical_asap_optimization.rs and
  stage2_physical_asap_optimization_sql.rs; stage_pipeline keys
  stage0_query_language_frontends, stage1_logical_asap_optimization,
  stage2_physical_asap_optimization, stage3_plan_selection (fixture
  regenerated: only the keys change);
- enumeration and selection are never capped: plan_stages enumerates
  every combination; the devtool's --max-displayed (was
  --max-candidates) caps only the listing;
- code docs say "query-language frontends".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test(integration): take plans and candidates from the planner pipeline

From #629/#633, on asap_integration_tests::planning (plan_stages,
selected_dag, logical_candidates):

- sql_to_physical, operator_sharing, materialization_e2e,
  precompute_raw_samples, summary_coverage_e2e,
  promql_numeric_regressions: ported.
- cse.rs: deleted; it pinned the legacy search's memo groups (sharing
  identical sub-DAGs is Pass 2, #580).
- Ignored, with reasons: the UNION ALL summary test (plan selection picks
  the exact plan until materialization, #604), arithmetic over one
  selector (#580), the unit-weight heap over raw samples (#589).
- precompute_raw_samples no longer requires a Count Sketch heap over raw
  samples (#589); materialization_e2e takes the quantile's sketch
  candidate, since plan selection picks the exact plan.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* refactor(devtools): sketch_coverage reports Pass 1's summary candidates

From #633: per lowered query, the summary candidates Pass 1 offers,
instead of the legacy explain_replacements. The CSE-shareable metric is
dropped (sharing is Pass 2). Over every corpus at epsilon 0.01 and a 1 s
ingestion interval: 145/2183 lowered queries get a summary candidate, and
Pass 1 rejects none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test(frontend-promql): the Kubernetes mixin aggregates over topk compose (ignored, #672)

Both corpus queries that do not compose aggregate over a topk output,
which the frontend declares as [keys.., topk_k] with a Utf8 placeholder
and no value column; the outer sum's accumulator has no value to read.
They compose once the top-k output carries the ranked value (#672).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* refactor(devtools): run_planning_stages runs the library's plan_stages

One pipeline implementation: the devtool's run_planning_stages (#577)
validates the frontends' roots and every composed candidate, and
otherwise is plan_stages with the built-in models. Its candidate types
are the library's. The Example 1 fixture is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant