feat(planner): enumerate local logical alternatives without execution timing

[zzylol]

Why

#509 Stage 1 starts with Pass 1, per-computation candidate generation. docs/design_docs/proposals/planner-layering.md says:

For each eligible sub-DAG, Pass 1 identifies its computation semantics, applies rewrite rules, and generates every candidate that is not provably unable to meet its accuracy requirement.

A sub-DAG is eligible when its root is an aggregation (an Aggregate operator) whose intent has at least one realization in the summary-family capabilities F: exact accumulator state or a summary family. Nested aggregations are separate targets: in sum by (job) (rate(x[1m])), the per-series rate and the per-job sum are each a target, and their options multiply.

• Pruning removes an invalid candidate. Any stage may prune, but only when the candidate is provably invalid (for example, a summary family that cannot meet the query's accuracy target), and every pruned candidate carries a reason.
• Selection discards valid candidates. Only stage 3 does this, when it picks the cheapest one.

The planner today (MajorPass, #542) builds replacements and ranks them by cost in one search. There is no logical-only step that lists every valid per-computation candidate over the unified IR without a cost model.

What

A new module, asap_aware_mapping::logical_candidates, that enumerates Pass 1 local candidates for a workload of QueryRoots. It takes no cost model, accuracy model, capabilities, storage policy or materialization assignment, and does not rank, size or build replacement sub-DAGs.

• Candidates (LocalCandidate):
  • Exact: plain exact execution of the original sub-DAG.
  • ExactAccumulator { kind, params }: an exact mergeable accumulator state (count, sum, min, max, rate, irate, increase).
  • Summary(SketchAlgorithm): a summary family, unsized. Pass 2 may share a summary and tighten it to the strictest consumer, so sizing happens later.
  • AvgAsSumCount: avg rewritten as exact sum and count accumulator states.
• One exact option per target, listed first: the exact accumulator form (ExactAccumulator, or AvgAsSumCount for avg) when the intent has one and it can be built, otherwise Exact. Offering both would duplicate "accumulator, not materialized"; Stage 2/3 decide materialization and sharing. Summary families follow in stable catalog order, which carries no preference.
• Pruning with reasons (PruneReason): RequiresExact (an exact request does not get approximate summaries) and NullableAvgInput (count counts rows, so avg over a possibly-null input is not rewritten; the target falls back to Exact). LocalChoices { alternatives, pruned } keeps both lists.
• Targets (LocalLogicalTarget { target, measure, choices, filter, having }): one per measure of every reachable Aggregate whose intent has a realization besides plain exact execution (even if all of them are pruned). Each measure of a multi-measure aggregate is its own target. Nested aggregates are separate targets. Producers read by scalar roots and scalar expressions are discovered, and a shared producer is one target (pointer identity).
• Filters and HAVING keep every candidate. The target records how they are applied; composition (feat(planner): Stage 1 whole-workload candidates and the stage_pipeline devtool #575) applies them.
  • filter: the measure's own filter, applied to the state as SummaryAgg.filter.
  • having: one HavingEntry { filter, predicate, placement } per HAVING: the aggregate's own having (no consumer Filter), then one per consumer Filter directly above the aggregate (the form the SQL frontend lowers HAVING to). Only this adjacent Filter is recorded.
  • HavingPlacement::BelowAggregate only when the predicate reads only by(...) group keys and, for a consumer Filter, every consumer of the aggregate applies the same predicate; it is then pushed below as a row predicate. Everything else is AboveOutput: a Filter above the finalize/estimate output. A global aggregate's HAVING is always AboveOutput, since its one output row exists even with no input rows.
  • Approximate candidates under a value predicate get guarantee = None through feat(planner): Stage 1 whole-workload candidates and the stage_pipeline devtool #575; multi-level push-down is Pass 1: push down filters and HAVING across levels to create more candidates (and reuse DataFusion's filter push-down) #664.
• Input checks (LogicalCandidateError): Structure (QueryRoot::validate_structure failed), ContainsAsap (input already has ASAP operators), AssignedPhysicalAnnotation (a node already has an execution timing or a materialization), InvalidAccuracy { root, measure } (an approximate target needs epsilon and delta in (0, 1)).
• Docs: docs/develop_docs/local-logical-candidates.md (new) describes the API. planner-layering.md now says a filter or HAVING does not prevent the accumulator or summary candidates, and Example 1's diagram says Sum has no summary family instead of showing pruned summaries.

How

enumerate_local_logical_candidates(roots) validates each root, walks every operator node reachable from its operator roots (including those referenced by scalar expressions) once, rejects physical annotations and ASAP operators, and records each node's consumers. For each Aggregate it computes the HAVING entries from the consumers, then calls local_candidates_for_intent(intent) per measure, using the existing accuracy_target, accuracy_budget and summary_candidates from replacement.rs. The roots are returned unchanged with the targets.

Public API:

pub fn local_candidates_for_intent(intent: &AggIntent) -> Result<LocalChoices, LogicalCandidateError>;
pub fn enumerate_local_logical_candidates<Id: Debug>(
    roots: Vec<(Id, QueryRoot)>,
) -> Result<LocalLogicalCandidates<Id>, LogicalCandidateError>; // { roots, targets }

Before this PR

No API lists Pass 1 candidates over OperatorNode / QueryRoot. The only candidate generation is the cost-ranked search inside MajorPass.

After this PR

Callers can get, for a lowered workload, every local target with its unranked candidates, its pruned candidates and reasons, and how its filters and HAVING are applied. Nothing in MajorPass changes; the module is not yet called by the planner.

Examples (approximate = EpsilonDelta { 0.05, 0.01 }):

Measure	Alternatives	Pruned
count, approximate	ExactAccumulator{Count}, CMS, Count Sketch, UnivMon	—
distinct count of one column, approximate (#509 Example 2)	Exact, HLL, Theta, KMV, UnivMon	—
distinct count of a column tuple, approximate	Exact, HLL, Theta, KMV (UnivMon not declared)	—
L2 / entropy, approximate	Exact, UnivMon	—
quantile, approximate	Exact, KLL, DDSketch	—
quantile, exact	Exact	KLL, DDSketch (RequiresExact)
TopK, approximate	Exact, CMS with heap, Count Sketch with heap	—
avg over a non-null input	AvgAsSumCount	—
avg over a nullable input	Exact	AvgAsSumCount (NullableAvgInput)
stddev	not a target

#509 Example 1 with exact accuracy: Q1 sum by (job) (rate(http_requests_total[1m])) gives two targets, ExactAccumulator{Sum} and ExactAccumulator{Rate}; Q2 topk by (job) (10, sum_over_time(http_requests_total[1m])) gives Exact for the top-k with both heap summaries pruned RequiresExact, and ExactAccumulator{Sum} for sum_over_time.

Evidence

Execution examples: crates/asap-aware-mapping/tests/logical_candidates.rs and crates/integration-tests/tests/logical_candidates_sql.rs (SQL HAVING src = 'a' → BelowAggregate; HAVING count(*) > 10 → AboveOutput; global HAVING false → AboveOutput). Screenshots and measurements: not applicable.

Verification

• Unit and integration tests: the unit test approximate_count_keeps_one_exact_accumulator_and_universal_choices; in tests/logical_candidates.rs: catalogs (cardinality_keeps_exact_specialized_and_universal_alternatives, frequency_statistics_keep_universal_choices, exact_quantile_stays_exact_and_approximate_keeps_both_families, topk_keeps_both_specialized_heap_choices, summary_candidates_are_unsized, avg_rewrites_to_sum_and_count_states), targets (scalar_root_producers_are_discovered_once, multi_measure_aggregate_has_one_target_per_measure, aggregate_without_realization_is_not_a_target), filters and HAVING (filtered_measure_keeps_candidates_and_records_its_filter, key_only_having_is_pushed_below_and_keeps_candidates, value_having_stays_above_output_and_keeps_candidates, having_on_one_consumer_of_a_shared_aggregate_stays_above, global_aggregate_having_stays_above), input checks (assigned_physical_annotation_asap_input_and_invalid_accuracy_are_rejected).
• End-to-end tests: example_1_exact_q1_keeps_exact_accumulators_only and example_1_exact_q2_prunes_topk_heaps lower docs: propose workload-wide planning, summary sharing, and materialization #509 Example 1 through the PromQL frontend; the three SQL HAVING tests lower through the SQL frontend.
• Other checks on the PR head: cargo fmt --check; workspace all-target Clippy with -D warnings; cargo test --workspace (1,549 passed, 0 failed, 2 ignored); dag-viewer test_render (29 passed); tools/verify_metricsql_external_consumer.sh OK.

Architectural decisions

• No cost model in Pass 1. Only stage 3 discards valid candidates (docs: propose workload-wide planning, summary sharing, and materialization #509), so the API has no cost or accuracy model and returns an unranked list.
• Summaries are unsized here; sizing happens after Pass 2 sharing.
• One exact option per target (accumulator form when buildable, else plain Exact).
• Pruned candidates are kept with a reason, as docs: propose workload-wide planning, summary sharing, and materialization #509 requires for pruning.
• Filters and HAVING are recorded, not applied: they never remove candidates; composition (feat(planner): Stage 1 whole-workload candidates and the stage_pipeline devtool #575) applies them.

Limitations and follow-up

• feat(planner): Stage 1 whole-workload candidates and the stage_pipeline devtool #575: compose each candidate into a replacement sub-DAG, apply the recorded filters and HAVING, and propagate guarantees (including guarantee = None under value predicates).
• feat(planner): Pass 1 offers HydraCms for grouped approximate counts #605: the grouping-level TopK Hydra candidate ("Hydra over all groups").
• Pass 1: push down filters and HAVING across levels to create more candidates (and reuse DataFusion's filter push-down) #664: HAVING push-down through more than one level.
• Rewrite rules beyond avg (for example recognizing entropy / L2 forms) and extra alternatives for opaque Extension intents are not added.
• MajorPass does not call this module yet.

Stack: tracker #528 · main → #542 → #665 → #666 → #543 → #667 → #561.

Human review — do not complete with an agent

• The MVP boundary is correct.
• New conceptual layers or public interfaces are necessary.
• The before/after description matches the intended product behavior.
• Human reviewer:
• Decision and rationale:

🤖 Generated with Claude Code

https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

[zzylol]

Parked as draft until Phase C (see #528). Stage 1 Pass 1 local alternatives belong to the #509 end-to-end work, which comes after finishing #511 (Phase A) and the #572 reorganization (Phase B). It will move into the logical-optimizer crate then.

🤖 Generated with Claude Code

[zzylol]

you can delete this doc.

[zzylol]

Please rename the term "local" to be aligend with what https://github.com/ProjectASAP/ASAPPlanner/blob/docs/planner-layering-problem-definition/docs/design_docs/proposals/planner-layering.md#asapplanner-planning-stages-design use

[zzylol]

you should be clear this is which stage given the layer design doc: https://github.com/ProjectASAP/ASAPPlanner/blob/docs/planner-layering-problem-definition/docs/design_docs/proposals/planner-layering.md#asapplanner-planning-stages-design