Skip to content

docs: define unified operators and SQL/PromQL scalar boundaries - #511

Merged
zzylol merged 50 commits into
mainfrom
docs/operator-flattening-proposal
Oct 2, 2026
Merged

zzylol merged 50 commits into
mainfrom
docs/operator-flattening-proposal

Conversation

@Selvomega

@Selvomega Selvomega commented Oct 1, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Define a common operator graph before and after ASAP optimization, and separate query operators from scalar expressions using explicit SQL/PromQL semantic mappings.

Before this PR

Relational subplans are hidden behind wrappers, and ordinary operations have separate representations above and below summaries. The scalar/operator proposal largely copies the current mixed representation without establishing language coverage.

After this PR

  • Define Operator::NonASAP and Operator::ASAP, with a common OperatorNode carrying result kind, schema, guarantee and timing. Keep one canonical set of operation payloads across both documents; distinguish structure and timing validation from planner accuracy assessment.
  • Define NonASAPOp and ScalarExpr against DataFusion 55.1.0 and Prometheus 3.15.0, with tables for direct mappings, equivalent lowerings and remaining semantic gaps.
  • Remove PromqlScalarBridge: for example, up * 2 uses a projection with a scalar literal. Retain real scalar/vector conversion and SQL subquery semantics, with visible operator dependencies.
  • Specify comparison modes, temporal selection, standalone scalar queries and evaluation context without claiming complete language or runtime support.
  • Show a complete SQL SUM graph before and after an exact-accumulator rewrite, including value/state schema changes.
  • Preserve existing accuracy rules and follow docs: propose workload-wide planning, summary sharing, and materialization #509 for timing and planning responsibilities. Export the common graph without relational wrappers.

Documentation only. The repository still depends on DataFusion 43.0.0 and a PromQL parser declaring 3.8 compatibility; this PR does not upgrade runtime dependencies.

Validation

Reviewed the pinned upstream definitions and language references. Checked both documents' operator lists, scalar placement, links, references, code fences and whitespace. Compiled the combined Rust structure sketches with placeholders for existing payloads; checked that type definitions are unique and operation lists remain complete. No runtime tests run.

Selvomega and others added 8 commits September 28, 2026 02:07
Add a design proposal for merging the pre- and post-ASAP operator sets
into one Operator { Basic(OriginalOp), Ext(ASAPOp) } tree, and list it
in the proposals index.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both proposal documents take main's version: the branch content was
squash-merged as #469 and main then updated the file and function names
in #470.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Comment thread docs/design_docs/proposals/operator-sharing.md Outdated
@Selvomega Selvomega changed the title Docs/operator flattening proposal updated docs: operator flattening proposal updated Oct 1, 2026
@zzylol zzylol changed the title docs: operator flattening proposal updated docs: clarify operator sharing, lifecycle timing, and DAG export Oct 1, 2026
@zzylol zzylol changed the title docs: clarify operator sharing, lifecycle timing, and DAG export docs: explain the unified operator design and its rationale Oct 1, 2026
This was referenced Oct 3, 2026
zzylol added a commit that referenced this pull request Oct 12, 2026
- timing: a SummaryAgg the lifecycle assignment recomputes at query time
  now forces the SummaryAgg consuming it to query time
  (`forces_query_time` reads the assignment), instead of failing with
  EvaluationUnderMaintenance.
- physical export: `PhysicalASAPDAG::validate` requires each node's
  incoming edges to be exactly its operator's inputs with their roles, so
  swapped Left/Right edges cannot reorder `a - b` (EdgeRoleMismatch).
- physical export: a DAG that fails validation (an empty batch, a
  hand-timed plan) is an `InvalidPhysicalDAG` error instead of a panic.
- precompute: only a TimeRange over (a TimeShift over) a time series Scan
  takes the raw-sample input contract, as before the port; a table range
  needs a stored population.
- compile: an input binding below a node the PromQL fallback evaluates
  is an error; the fallback reads raw series slots, so it was silently
  ignored. A binding below another bound input is still superseded by
  it, as in `cut`.
- compile: drop the scalar-literal binary branch; nothing filled its map.
- PromQL fallback: clamp's `NOT (max < min)` is now the IR `Filter`
  #540 emits; the fallback runs a filter on scalars alone, and the
  pointwise projection no longer filters on its own.
- tests: `raw_series_contract_is_explicit` asserts that a consumed bare
  selector is instant selection (its comment said the opposite) and that
  a binding below the fallback is rejected; new `unified_precompute` test.
- PromQL fallback: unary minus drops the metric name like every other
  pointwise projection, following #540 and the Prometheus 3.15 baseline
  (#511); pointwise_projection_names_and_dynamic_parameters expects it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 12, 2026
- timing: a SummaryAgg the lifecycle assignment recomputes at query time
  now forces the SummaryAgg consuming it to query time
  (`forces_query_time` reads the assignment), instead of failing with
  EvaluationUnderMaintenance.
- physical export: `PhysicalASAPDAG::validate` requires each node's
  incoming edges to be exactly its operator's inputs with their roles, so
  swapped Left/Right edges cannot reorder `a - b` (EdgeRoleMismatch).
- physical export: a DAG that fails validation (an empty batch, a
  hand-timed plan) is an `InvalidPhysicalDAG` error instead of a panic.
- precompute: only a TimeRange over (a TimeShift over) a time series Scan
  takes the raw-sample input contract, as before the port; a table range
  needs a stored population.
- compile: an input binding below a node the PromQL fallback evaluates
  is an error; the fallback reads raw series slots, so it was silently
  ignored. A binding below another bound input is still superseded by
  it, as in `cut`.
- compile: drop the scalar-literal binary branch; nothing filled its map.
- PromQL fallback: clamp's `NOT (max < min)` is now the IR `Filter`
  #540 emits; the fallback runs a filter on scalars alone, and the
  pointwise projection no longer filters on its own.
- tests: `raw_series_contract_is_explicit` asserts that a consumed bare
  selector is instant selection (its comment said the opposite) and that
  a binding below the fallback is rejected; new `unified_precompute` test.
- PromQL fallback: unary minus drops the metric name like every other
  pointwise projection, following #540 and the Prometheus 3.15 baseline
  (#511); pointwise_projection_names_and_dynamic_parameters expects it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 12, 2026
…1 candidates (#663)

* docs: align #509/#511/#573 docs with the decided vocabulary and Pass 1 candidates

- operator-sharing.md: ReadPopulation { readout } -> EvaluatePopulation
  { evaluation } (matches crates/types/src/ir/asap.rs); "readout" ->
  "evaluation" where it names reading summary state; add the missing
  `filter` field to the SummaryAgg sketch.
- asap-primitive-schema.md: "readout" -> "evaluation" where it names the
  operation (intent table header, §6.5 heading and its anchor/table row,
  top-k evaluation, guarantee of an evaluation). Verb "read out" kept, as
  in the code comments. Coverage cache text unchanged.
- planner-layering.md: define an eligible sub-DAG at its first use; add
  the exact accumulator state candidate to the Pass 1 table and text;
  recompute Example 1 (Pass 1: 2 x 2 x 2 x 3 = 24, Pass 2:
  24 x 2 x 3 x 3 = 432, Stage 2: 24 x 52 = 1,248).
- planner-layering.md: every aggregation is its own Pass 1 target,
  including nested ones (Q1's per-series rate and per-job sum, Q2's
  per-series sum_over_time and per-job top-k), as in #561's
  enumerate_local_logical_candidates; the Algorithm 1 example and diagram
  show the four sub-DAGs, and Example 1 lists options per target.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: leave multi-measure targets out of the eligible sub-DAG definition

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: one exact option per Pass 1 target

Each Pass 1 target gets a single exact option: exact accumulator state
when the intent has a buildable mergeable exact accumulator, otherwise
the plain exact computation. Example 1 returns to 3 / 54 / 156 with
four targets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: reflow the eligible sub-DAG paragraph

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 12, 2026
- timing: a SummaryAgg the lifecycle assignment recomputes at query time
  now forces the SummaryAgg consuming it to query time
  (`forces_query_time` reads the assignment), instead of failing with
  EvaluationUnderMaintenance.
- physical export: `PhysicalASAPDAG::validate` requires each node's
  incoming edges to be exactly its operator's inputs with their roles, so
  swapped Left/Right edges cannot reorder `a - b` (EdgeRoleMismatch).
- physical export: a DAG that fails validation (an empty batch, a
  hand-timed plan) is an `InvalidPhysicalDAG` error instead of a panic.
- precompute: only a TimeRange over (a TimeShift over) a time series Scan
  takes the raw-sample input contract, as before the port; a table range
  needs a stored population.
- compile: an input binding below a node the PromQL fallback evaluates
  is an error; the fallback reads raw series slots, so it was silently
  ignored. A binding below another bound input is still superseded by
  it, as in `cut`.
- compile: drop the scalar-literal binary branch; nothing filled its map.
- PromQL fallback: clamp's `NOT (max < min)` is now the IR `Filter`
  #540 emits; the fallback runs a filter on scalars alone, and the
  pointwise projection no longer filters on its own.
- tests: `raw_series_contract_is_explicit` asserts that a consumed bare
  selector is instant selection (its comment said the opposite) and that
  a binding below the fallback is rejected; new `unified_precompute` test.
- PromQL fallback: unary minus drops the metric name like every other
  pointwise projection, following #540 and the Prometheus 3.15 baseline
  (#511); pointwise_projection_names_and_dynamic_parameters expects it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 12, 2026
A workload entry whose root is a standalone scalar (`scalar(quantile(0.99,
rate(x[5m])))`, `scalar(sum(up)) + 1`) was copied through unplanned. Per
#511 §2.3 it has no query node and reads operator sub-DAGs by reference;
per operator-sharing §3 those sub-DAGs get the same validation and accuracy
assessment as operator roots.

MajorPass now searches, selects, assembles and lifecycle-plans each
distinct operator sub-DAG a scalar root reads, against the scalar's entry
and accuracy target, alongside the operator roots, and rebuilds the scalar
over the selected sub-DAGs. PlanOutput::scalar_plans holds their lifecycle
plans. PhysicalASAPDAG gains `scalar_roots`: each standalone scalar query
as a `ScalarExpr<PhysicalASAPNodeId>` keyed by its position in the batch,
so query i is the scalar root keyed i, or else the next of `roots`.
Validation checks positions and treats scalar references as roots for
existence, cycles and reachability. Wire version 9.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 12, 2026
…#541)

* feat(ir): restore execution timing assignment for physical plans

The logical export is phase-free. Physical planning still needs the lifecycle
timing expansion; bring it back unchanged except for carrying coverage.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(ir): export timed plans as PhysicalASAPDAG

The logical export carries no execution timing. Physical planning output
needs it: reuse the logical payloads and node ids, and add each node's and
edge's data state plus window compatibility, as the earlier post-ASAP export
did. Coverage is carried through.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(runtime): compile unified operator and scalar graphs

* feat(ir): give physical ASAP DAGs one root per batch query

Match the logical export: a batch is one DAG whose roots are its queries,
with shared sub-DAGs exported once. The runtime compiles all roots.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: integrate with #614 (DataFusion 54): let chrono resolve to DF 54's version

DataFusion 54 requires chrono ^0.4.44, so the runtime crate's exact
=0.4.39 pin no longer resolves; keep 0.4.39 as the minimum.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(runtime): read Operator<NodeId> payloads from the physical ASAP DAG

Follow #537: the physical DAG now carries each node's operator as
Operator<NodeId> (from ir::flat) instead of the removed wire mirror types.
The runtime matches Operator::NonASAP / Operator::ASAP directly, and
restore() rebuilds the in-memory DAG with map_children instead of a
field-by-field conversion. Node ids are plain usize.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(ir): follow derived summary coverage in timing and physical export

Coverage is derived from the node (#646), so timed copies no longer carry
it and PhysicalASAPDAGNode drops its coverage field.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(runtime): address #541 review findings

- timing: a SummaryAgg the lifecycle assignment recomputes at query time
  now forces the SummaryAgg consuming it to query time
  (`forces_query_time` reads the assignment), instead of failing with
  EvaluationUnderMaintenance.
- physical export: `PhysicalASAPDAG::validate` requires each node's
  incoming edges to be exactly its operator's inputs with their roles, so
  swapped Left/Right edges cannot reorder `a - b` (EdgeRoleMismatch).
- physical export: a DAG that fails validation (an empty batch, a
  hand-timed plan) is an `InvalidPhysicalDAG` error instead of a panic.
- precompute: only a TimeRange over (a TimeShift over) a time series Scan
  takes the raw-sample input contract, as before the port; a table range
  needs a stored population.
- compile: an input binding below a node the PromQL fallback evaluates
  is an error; the fallback reads raw series slots, so it was silently
  ignored. A binding below another bound input is still superseded by
  it, as in `cut`.
- compile: drop the scalar-literal binary branch; nothing filled its map.
- PromQL fallback: clamp's `NOT (max < min)` is now the IR `Filter`
  #540 emits; the fallback runs a filter on scalars alone, and the
  pointwise projection no longer filters on its own.
- tests: `raw_series_contract_is_explicit` asserts that a consumed bare
  selector is instant selection (its comment said the opposite) and that
  a binding below the fallback is rejected; new `unified_precompute` test.
- PromQL fallback: unary minus drops the metric name like every other
  pointwise projection, following #540 and the Prometheus 3.15 baseline
  (#511); pointwise_projection_names_and_dynamic_parameters expects it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* refactor(runtime): remove the PromQL fallback evaluator

ASAPPlanner generates candidates and computes their cost, or says why a
query has none. Executing what the native operators cannot run belongs
to the deployment runtime (ASAPQuery backend), not to the Planner.

Remove unified_physical_planner::promql_fallback and its tests, and the
fallback paths in compile: raw-series input slots, the fallback lowering
and the below-fallback binding check. A node without a native lowering
now fails compilation with Error::Invalid.

Reject Planner node IDs above u32::MAX, which could collide with helper
IDs, and error instead of debug-asserting when a node needs more than
2^16 helpers. Document that forces_query_time also forces an unaligned
per-series PromQL BinaryOp to query time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 12, 2026
A workload entry whose root is a standalone scalar (`scalar(quantile(0.99,
rate(x[5m])))`, `scalar(sum(up)) + 1`) was copied through unplanned. Per
#511 §2.3 it has no query node and reads operator sub-DAGs by reference;
per operator-sharing §3 those sub-DAGs get the same validation and accuracy
assessment as operator roots.

MajorPass now searches, selects, assembles and lifecycle-plans each
distinct operator sub-DAG a scalar root reads, against the scalar's entry
and accuracy target, alongside the operator roots, and rebuilds the scalar
over the selected sub-DAGs. PlanOutput::scalar_plans holds their lifecycle
plans. PhysicalASAPDAG gains `scalar_roots`: each standalone scalar query
as a `ScalarExpr<PhysicalASAPNodeId>` keyed by its position in the batch,
so query i is the scalar root keyed i, or else the next of `roots`.
Validation checks positions and treats scalar references as roots for
existence, cycles and reachability. Wire version 9.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 13, 2026
* refactor(planner): plan one shared unified workload DAG

Cut today's planner over to the unified OperatorNode IR. The planner calls
the unified frontends (`asap_frontend_*::unified`) and returns OperatorNode /
QueryRoot; ParsedWorkload keeps scalar roots; Pass 1 (ASAPStrategies), the
identical-sub-DAG sharing, selection, DAG assembly and lifecycle costing all
run on OperatorNode; and PlanOutput exposes the whole workload DAG.
Planned summary states derive their coverage from the node (#646).

The legacy physical planner read the planner's QueryExpr output, so it is
removed here together with its expression compiler and raw sources; every
caller uses the native compiler from #541 (`unified_physical_planner`,
`unified_sources`), which reads the PhysicalASAPDAG export. Operator names
follow the evaluation vocabulary (readout -> evaluation).

The MetricsQL frontend input carries the planning clock and lowers through
`lower_metricsql_query_workload`, so bare selectors get the workload's
ingestion-interval lookback (#540).

The legacy frontend modules and the QueryExpr/SummaryNode IRs stay compiled
but are no longer re-exported from post_asap. The next PR moves the unified
modules to the crate roots; a later one removes the legacy IRs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* feat(planner): plan and export standalone scalar queries

A workload entry whose root is a standalone scalar (`scalar(quantile(0.99,
rate(x[5m])))`, `scalar(sum(up)) + 1`) was copied through unplanned. Per
#511 §2.3 it has no query node and reads operator sub-DAGs by reference;
per operator-sharing §3 those sub-DAGs get the same validation and accuracy
assessment as operator roots.

MajorPass now searches, selects, assembles and lifecycle-plans each
distinct operator sub-DAG a scalar root reads, against the scalar's entry
and accuracy target, alongside the operator roots, and rebuilds the scalar
over the selected sub-DAGs. PlanOutput::scalar_plans holds their lifecycle
plans. PhysicalASAPDAG gains `scalar_roots`: each standalone scalar query
as a `ScalarExpr<PhysicalASAPNodeId>` keyed by its position in the batch,
so query i is the scalar root keyed i, or else the next of `roots`.
Validation checks positions and treats scalar references as roots for
existence, cycles and reachability. Wire version 9.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(planner): plan non-pass-through scalar references exact

A standalone scalar hands its entry's accuracy target to its operator
reference only when it passes that one reference through (`scalar(v)`, or
arithmetic/comparisons with literals). Otherwise each reference is planned
exact: independently approximate references leave a combination such as a
difference of two quantiles unbounded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(planner): check scalar plans in the contract; remove dead code

check_contract now verifies that `scalar_plans` holds one plan per
distinct reference of each scalar root, in order and keyed by its entry.
The scalar_roots doc says keys are batch positions, equal to entry
indices because the contract requires dense indices. A validation test
covers duplicate and descending scalar-root keys. The never-constructed
VectorScalar/ScalarVector operand modes and the unused
ParsedWorkload::new/entries are removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(planner): lower scalar(v) in Project/Filter expressions as scalar inputs

`a / scalar(b)` and `a > scalar(b)` failed costing with
UnsupportedQueryOperator. A Project or Filter now takes one
PromqlVectorToScalar input per distinct operator reference its
expressions read, after its row input; its statistics carry one
`scalar_inputs` edge per such input.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(planner): reuse a top-k only over the same partitions

TopKLimitReuseStrategy offered a global top-10 as the source of a per-job
top-5. It now requires equal `partition_by`, and the maintained
population recognizer requires the limit to partition like its sort.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* refactor(planner): tell logical rewrites apart by provenance

`is_logical_rewrite` inferred a logical rewrite from the replacement's
shape (no guarantee, no ASAP operator). It is now
`ReplacementSubDAG::is_logical_rewrite`, read from the candidate's
provenance (LogicalRewrite, CseShare, CseRecompute,
AccuracyReconciliation); the free function is removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(planner): return an error when search input already holds ASAP operators

The search entry points asserted that no workload root contains an ASAP
operator. They now return `SearchError::AsapInput`; MajorPass reports it
as `OptimizeError::Search`, and the explanation entry points pass it on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(planner): walk operator nodes read by scalar expressions

The explanation report located only relational children, so a target
read through `scalar(...)` in a predicate or projection had no
breadcrumb; it now visits every operator input, as target discovery does.
`frontier_from_timing` also puts an ingestion-time node a standalone
scalar root reads on the frontier, like an operator root.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(ir): do not promote a filtered count ranking to a heavy-hitter TopK

The IR canonicalize pass ignored measure filters, so `COUNT(*) FILTER
(WHERE ...)` or `COUNT(col)` over a nullable column ranked by ORDER BY
... LIMIT became a heap sketch over every row. It now keeps Sort + Limit
for a filtered measure (the legacy #466 guard), and the keyed heap-sketch
input rule refuses a filtered inner aggregate. The legacy
does_not_promote_a_filtered_count_ranking test is ported to the IR and
through the SQL front end.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* Revert "fix(planner): plan non-pass-through scalar references exact"

Combinations of approximate values may read summaries: each scalar
reference is planned against the entry's accuracy target again. Composing
the guarantee of such a combination is left to #575 and Stage 3's
accuracy model.

This reverts commit c44ecda.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(planner): rank top-k only when the Limit partitions like its Sort

A global Limit over a partitioned Sort keeps k rows overall, not k per
group, so neither the heavy-hitter promotion nor the current-series heap
candidates apply to it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(ir): export one ScalarRef edge per distinct referenced node

Cost lowering reads a node referenced several times from one operator's
expressions once; the export now matches, deduping by node id (one id
per pointer).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test(planner): cover scalar inputs, top-k partitions, scalar plan contract, provenance

Project/Filter with two scalar references and one read twice lower and
cost; the maintained population requires the Limit to partition like its
Sort; the keyed heap input refuses a filtered count; check_contract
rejects scalar plans out of order, under another entry, or extra; a
guarantee-less summary realization is not a logical rewrite.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test(planner): a difference of two scalar quantiles reads summaries

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(planner): plan structurally identical scalar references once

Front ends build identical references as separate nodes and planning
merges them, so pointer-distinct references left the rebuilt scalar with
one reference but two scalar plans, failing the contract. References are
now distinct by structure. Document that each reference keeps its summary
candidates and that the combined guarantee is composed later (#669, #575).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test(planner): reject a per-job Limit over a global Sort; doc cleanups

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(planner): read exact-state query roots through their finalize

MajorPass assembled roots with assemble_selected_dag, so a root whose
selected candidate is an exact state came out as a bare SummaryAgg. The
lifecycle plan now finalizes its root after costing (the finalize wraps
the shared state, so sharing and cost evidence stay), scalar references
included since scalar(v) reads values. Query-root callers and tests use
assemble_selected_query.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation promql PromQL front-end lowering (L1→L2) sql SQL front-end lowering (DataFusion → L2)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants