Skip to content

docs: ASAP primitive schema and summary coverage (definition + selection) - #573

Merged
zzylol merged 61 commits into
mainfrom
docs/asap-primitive-schema
Oct 12, 2026
Merged

zzylol merged 61 commits into
mainfrom
docs/asap-primitive-schema

Conversation

@zzylol

@zzylol zzylol commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Why

ASAP primitives (summary states such as KLL, Count-Min or exact accumulators) flow along DAG edges next to raw values. A Schema says what kind of state an edge carries, but not which data the state summarizes. So two states with identical schemas can still be wrong to merge or reuse: a KLL over latency and one over size, overlapping time windows, or the same population counted twice. The design for this was spread across several docs; this PR collects it into one design document.

What

New: docs/design_docs/proposals/asap-primitive-schema.md

Section Content
§1 Goal links to the LogicalDAG (#509) and operator (#511) docs; every summary operator and its status
§2 Terminology relation, column, schema, aggregate state, materialized view, physical layout, and how ASAPPlanner uses each
§3 Schema the schema as the DAG edge; a summary type's levels (family, category, algorithm, parameters); which aggregation intents each type answers
§4 Coverage §4.1 Goldstein & Larson view matching (SIGMOD 2001) with a worked example. §4.2 Summary Coverage = Summary definition + selection: why there are two parts (§4.2.1), how the planner derives the definition and then the selection from the sub-DAG (§4.2.2), and what derivation costs (§4.2.3: linear in the few operators directly under the SummaryAgg, not in the sub-DAG). §4.3 what is left to the downstream deployment runtime
§5 Operators one example per summary operator, drawn as a DAG with the schema on each edge, and a table per operator: what comes out, when it is rejected, its coverage, state or value. SummaryMerge adds allowed and rejected merges, per-family overlap rules, slice and reuse
§6 Code interfaces ordered from the DAG node inward: OperatorNode, ASAPOp, schema and field types, summary-family parameters, update input and readouts, SummaryCoverage. Every field and function is commented, and a diagram shows how they connect

The SDS definition is left as a TODO for a separate doc.

Moved here, leaving a summary and a link behind: operator-sharing.md §2.1, pre-asap-ir.md (fields and column references), asap-aware-mapping-contracts.md (sketch family identity), physical-planning-and-deployment.md (input vs summary semantics).

Link only: decoupling_op_and_expr.md, concepts/post-asap-ir.md, univmon-frequency-summary.md and the proposals index. Links into this doc point at its current sections.

Before / after

  • Before: schema and state-type design is spread over four docs, and none of them says what a summary state covers.
  • After: one design doc defines it end to end, and the other docs point to it.

Related code

PR Status What it implements from this doc
#560 merged SummaryMerge over children with identical schemas
#646 open coverage derived as definition + selection; merges require equal definitions and non-overlapping selections (§4.2, §6.1, §6.6)
#592 open per-family overlap rules (family_merges, merge_relation) (§5.6)
#601 / #604 / #628 open window-composition panes derive their time from TimeRange + TimeShift instead of declaring it

Where the doc describes code that is not on main yet, it says which PR adds it. This PR is documentation only and based on main.

🤖 Generated with Claude Code

https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

One design document for the edge schema of ASAP primitives: fields that carry
summary state, the summary semantics a schema must preserve, coverage, and
per-operator examples. Related docs keep a summary and link to it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 3, 2026
The design document and the docs it consolidates are reviewed separately on
main. This PR keeps code, tests and the ScanSelection rename in docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keep the author's sections 1–4 as written; the operator examples (§5) and
key code interfaces (§6) stay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Follows #579: the top-k sketch readout derives one row per selected item
instead of a packed Utf8 column, matching the exact Sort → Limit shape.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 6, 2026
The design document and the docs it consolidates are reviewed separately on
main. This PR keeps code, tests and the ScanSelection rename in docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…chema

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 6, 2026
* feat(ir): define joint summary observation coverage

* refactor(ir): name represented observations ObservationExtent

* refactor(ir): rename observation extent to SummaryCoverage and trim its fields

Name the metadata coverage to match #560 and the docs; drop the single-variant
multiplicity and deployment-specific revision; rename grouping to reduction to
match SummaryAgg; report failures through SchemaDerivationError::Coverage; revert
the unrelated PaneCoverageError rename.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(ir): describe coverage source as any observation data source

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(ir): require coverage on summary nodes; allow regions without time bounds

validate_structure rejects a SummaryAgg without coverage (CoverageError::Missing).
CoverageRegion time bounds become optional so tabular sources without a time
column can declare coverage. Population stays trusted; #570 tracks checking it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: explain the summary coverage problem with examples

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(ir): keep only time and population in SummaryCoverage

Given required coverage on summary nodes, input and reduction duplicated the
producing SummaryAgg fields; drop them along with ProducerMismatch. Type source
as Source, matching Scan.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(cost): rename SourceCoverage to ScanSelection

SourceCoverage names the rows a physical scan reads for cost comparison, not
which observations a summary state holds; rename it so it is not confused with
SummaryCoverage.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: turn summary coverage into a design document

Move it to docs/design_docs/proposals with problem and motivation,
requirements, design, alternatives and key code interfaces.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: design doc for ASAP primitive schema and summary semantics

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Update asap-primitive-schema.md

* Update asap-primitive-schema.md

* Update asap-primitive-schema.md

* Update asap-primitive-schema.md

* docs: add code interfaces and per-operator examples to the ASAP primitive schema design

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: move the ASAP primitive schema design to #573

The design document and the docs it consolidates are reviewed separately on
main. This PR keeps code, tests and the ScanSelection rename in docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(ir): say coverage time is absolute and population is Utf8 equality

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(ir): carry summary coverage through CSE and flat DAGs

Coverage is part of a summary state's identity: two states over different
observations are never shared, so CSE hashes and compares it, and a flat
node keeps it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
zzylol and others added 2 commits October 7, 2026 14:21
SummaryCoverage records which columns a state summarizes (input,
group_by) as well as which rows. Update the SummaryAgg and SummaryMerge
examples to #560: merges compare coverage columns, carry no coverage
until #646, and merged_coverage is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…Larson view matching

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 7, 2026
Remove the coverage columns (input, group_by), check_columns and the
merge's coverage checks. SummaryMerge now only checks structure: at least
one input, every input is State with one state column and an identical
schema. Whether a structurally valid merge is semantically valid (same
computation, disjoint selections) is decided by summary coverage in #646,
following the design in #573.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol and others added 2 commits October 7, 2026 22:50
…e interface in #646

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 7, 2026
Coverage is derived from the node, never declared. For a SummaryAgg,
predicate conjuncts that lift through Filter, a range TimeRange, a
TimeShift without @ and direct-column Project items, and are a value set
or an interval on one column, form the selection; everything else stays
in the definition (the SummaryAgg with the selection removed). A
TimeRange(w) over TimeShift(s) gives the relative window (-(s+w), -s].

A SummaryMerge is valid only when its inputs have structurally equal
definitions (ignoring timing and guarantee) and pairwise disjoint
selections; OperatorNode::new and validate_structure enforce it. Its
coverage is the shared definition and the union of the selections,
joining adjacent windows and value sets.

OperatorNode.coverage becomes a private cache behind coverage():
ignored by equality, skipped by serde, emptied on clone. with_coverage,
requires_coverage and CoverageError::Missing are removed, as are the
coverage fields of CSE keys and FlatNode.

Design: docs/design_docs/proposals/asap-primitive-schema.md §4 (#573).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
@zzylol
zzylol marked this pull request as ready for review October 8, 2026 13:41
zzylol added a commit that referenced this pull request Oct 9, 2026
* feat(ir): define compatible logical summary merges

* feat(ir): require disjoint coverage for summary merges

* refactor(ir): use SummaryCoverage names in summary merges

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(ir): check merge update/reduction on producers; test documented coverage examples

SummaryCoverage no longer repeats input/reduction, so SummaryMerge compares
them through OperatorNode::summary_update. summary_coverage_examples.rs builds
each example in docs/develop_docs/summary-coverage.md as a SummaryAgg ->
SummaryMerge plan.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(ir): point coverage examples at the design document

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(ir): point coverage examples at the ASAP primitive schema design doc

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(ir): reject summary merges of heap-based sketches

A merged top-k heap can miss an item that is heavy in only one input, so
CmsWithHeap, CountSketchWithHeap and UnivMon states do not merge. Also
correct the merge_disjoint doc: SummaryMerge checks schema, update,
reduction and heap families, not accuracy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(ir): compute merge coverage with SummaryCoverage::of_merge

`ASAPOp::merged_coverage` only applied to SummaryMerge and returned an error
for every other operator. Replace it with `SummaryCoverage::of_merge`, which
takes the merge's inputs; the callers already have them. Also explain what
`OperatorNode::summary_update` returns and why merging compares it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(ir): leave summary merge coverage to #646; allow heap-based sketches

SummaryMerge no longer derives or checks coverage: of_merge,
UnknownInput and MergeOutputMismatch are removed and summary_coverage.rs
matches main. Coverage for all summary nodes will be derived by one
function in #646. The heap-based sketch restriction is also dropped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(ir): rename summary_update to summary_input_data

It reads the producing SummaryAgg's update expression and reduction; it
does not update the summary.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(ir): record summarized columns in SummaryCoverage

SummaryCoverage now records which columns a state summarizes as well as
which rows: `input` (the SummaryAgg update expression) and `group_by`
(its reduction). with_coverage rejects a SummaryAgg declaration whose
columns differ from the node's own (ColumnMismatch), and SummaryMerge
requires coverage on every input (UnknownInput) with identical columns.
merge_disjoint checks columns too. summary_input_data is removed. A
nested SummaryMerge carries no coverage until #646, so it is rejected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(ir): attach the merge_disjoint doc comment to merge_disjoint

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(ir): keep SummaryMerge structural; leave coverage to #646

Remove the coverage columns (input, group_by), check_columns and the
merge's coverage checks. SummaryMerge now only checks structure: at least
one input, every input is State with one state column and an identical
schema. Whether a structurally valid merge is semantically valid (same
computation, disjoint selections) is decided by summary coverage in #646,
following the design in #573.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 9, 2026
Coverage is derived from the node, never declared. For a SummaryAgg,
predicate conjuncts that lift through Filter, a range TimeRange, a
TimeShift without @ and direct-column Project items, and are a value set
or an interval on one column, form the selection; everything else stays
in the definition (the SummaryAgg with the selection removed). A
TimeRange(w) over TimeShift(s) gives the relative window (-(s+w), -s].

A SummaryMerge is valid only when its inputs have structurally equal
definitions (ignoring timing and guarantee) and pairwise disjoint
selections; OperatorNode::new and validate_structure enforce it. Its
coverage is the shared definition and the union of the selections,
joining adjacent windows and value sets.

OperatorNode.coverage becomes a private cache behind coverage():
ignored by equality, skipped by serde, emptied on clone. with_coverage,
requires_coverage and CoverageError::Missing are removed, as are the
coverage fields of CSE keys and FlatNode.

Design: docs/design_docs/proposals/asap-primitive-schema.md §4 (#573).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol and others added 6 commits October 9, 2026 17:49
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
@zzylol
zzylol force-pushed the docs/asap-primitive-schema branch from 5378e7c to 82ba101 Compare October 11, 2026 04:25
zzylol added a commit that referenced this pull request Oct 11, 2026
* feat(ir): derive summary coverage as definition + selection

Coverage is derived from the node, never declared. For a SummaryAgg,
predicate conjuncts that lift through Filter, a range TimeRange, a
TimeShift without @ and direct-column Project items, and are a value set
or an interval on one column, form the selection; everything else stays
in the definition (the SummaryAgg with the selection removed). A
TimeRange(w) over TimeShift(s) gives the relative window (-(s+w), -s].

A SummaryMerge is valid only when its inputs have structurally equal
definitions (ignoring timing and guarantee) and pairwise disjoint
selections; OperatorNode::new and validate_structure enforce it. Its
coverage is the shared definition and the union of the selections,
joining adjacent windows and value sets.

OperatorNode.coverage becomes a private cache behind coverage():
ignored by equality, skipped by serde, emptied on clone. with_coverage,
requires_coverage and CoverageError::Missing are removed, as are the
coverage fields of CSE keys and FlatNode.

Design: docs/design_docs/proposals/asap-primitive-schema.md §4 (#573).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(ir): keep ambiguous columns and untyped-equal literals out of selections

From an independent review of the derivation:
- a column whose (table, name) is not unique in the agg child's schema
  cannot be named in a selection, so its conjuncts stay in the definition;
- value sets compare literals by typed order, so 1 and 1.0 (or NaN) are
  never proven different;
- a partly lifted Scan predicate is rebuilt with only its residual.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test(ir): cover the design doc's worked coverage example

Point the module docs at #573 §4.2 and note that a Project renames
columns in their identity.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* perf(ir): cache a merge's coverage when it is checked

OperatorNode::new and validate_structure derived a SummaryMerge's coverage
to check it and dropped the result, so coverage() derived it again. Store
it in the cache instead. Also correct two comments (Project qualifier,
absolute time needs timestamp literals).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test(ir): worked example follows the SQL query of the design doc

The #573 worked example (§4.2.2) is now a SQL query; build the DAG the SQL
frontend lowers it to (WHERE folded into Scan.predicates, no time window).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* feat(ir): join touching value ranges in a merged selection

Value ranges on one column now join like touching time windows, so every
selection dimension follows Goldstein & Larson's one-range-per-column form.
The joined range stays explicit: neither input took NULL rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(ir): derive coverage on each call and keep renaming nodes

Review of #646:
- The coverage cache could go stale: `operator` and `schema` are public,
  so a node edited in place (e.g. through `Rc::make_mut`) kept the
  coverage of its old value. Drop the cache; `coverage()` derives on each
  call and returns an owned value.
- A `Filter`, `TimeRange` or `TimeShift` whose schema renames columns was
  removed from the definition once everything was lifted out of it, taking
  the rename with it, so `tier = 'eu'` named `region` looked disjoint from
  `region = 'us'`. The walk now stops at such a node, which stays in the
  definition.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol zzylol added the documentation Improvements or additions to documentation label Oct 12, 2026
@zzylol
zzylol merged commit 5cd72f4 into main Oct 12, 2026
1 check passed
@zzylol

zzylol commented Oct 12, 2026

Copy link
Copy Markdown
Contributor Author

@Selvomega @milindsrivastava1997 just marked both of you as reviewers but merging this doc PR for now to make progress. I have reviewed the doc, and the corresponding coding PR #646 has been reviewed by @Selvomega and merged. If we find new problems later, we can revisit the doc.

@zzylol
zzylol deleted the docs/asap-primitive-schema branch October 12, 2026 19:36
zzylol added a commit that referenced this pull request Oct 12, 2026
…1 candidates (#663)

* docs: align #509/#511/#573 docs with the decided vocabulary and Pass 1 candidates

- operator-sharing.md: ReadPopulation { readout } -> EvaluatePopulation
  { evaluation } (matches crates/types/src/ir/asap.rs); "readout" ->
  "evaluation" where it names reading summary state; add the missing
  `filter` field to the SummaryAgg sketch.
- asap-primitive-schema.md: "readout" -> "evaluation" where it names the
  operation (intent table header, §6.5 heading and its anchor/table row,
  top-k evaluation, guarantee of an evaluation). Verb "read out" kept, as
  in the code comments. Coverage cache text unchanged.
- planner-layering.md: define an eligible sub-DAG at its first use; add
  the exact accumulator state candidate to the Pass 1 table and text;
  recompute Example 1 (Pass 1: 2 x 2 x 2 x 3 = 24, Pass 2:
  24 x 2 x 3 x 3 = 432, Stage 2: 24 x 52 = 1,248).
- planner-layering.md: every aggregation is its own Pass 1 target,
  including nested ones (Q1's per-series rate and per-job sum, Q2's
  per-series sum_over_time and per-job top-k), as in #561's
  enumerate_local_logical_candidates; the Algorithm 1 example and diagram
  show the four sub-DAGs, and Example 1 lists options per target.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: leave multi-measure targets out of the eligible sub-DAG definition

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: one exact option per Pass 1 target

Each Pass 1 target gets a single exact option: exact accumulator state
when the intent has a buildable mergeable exact accumulator, otherwise
the plain exact computation. Example 1 returns to 3 / 54 / 156 with
four targets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: reflow the eligible sub-DAG paragraph

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 12, 2026
Lower each example query of the ASAP primitive schema design doc (#573
§4.2.2), plan it with the default cost model and check the coverage of the
selected KLL state: SQL value sets, ranges and renamed columns, a PromQL
offset window as relative time, a matcher below rate kept in the definition,
and a PromQL comparison above rate lifted into the selection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 12, 2026
…initions

`execution_timed_workload_dag` becomes `pub(crate)`: callers export a
planned workload through `PlanOutput::execution_timed_dag`, which also
carries its standalone scalar roots.

The coverage tests now check what each #573 §4.2.2 example's definition
keeps, not only its selection: the range example yields exactly one box and
a scan without the lifted predicate, the matcher below `rate` stays in the
definition's scan, and the definition keeps the `rate` under a lifted
`Filter`. The two missing rows are added: `rate(a[5m]) > rate(b[5m])` (a
`BinaryOp`, nothing selected) and `rate(m[5m]) > bool 0` (a `Project`,
kept in the definition).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 12, 2026
Lower each example query of the ASAP primitive schema design doc (#573
§4.2.2), plan it with the default cost model and check the coverage of the
selected KLL state: SQL value sets, ranges and renamed columns, a PromQL
offset window as relative time, a matcher below rate kept in the definition,
and a PromQL comparison above rate lifted into the selection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 12, 2026
…initions

`execution_timed_workload_dag` becomes `pub(crate)`: callers export a
planned workload through `PlanOutput::execution_timed_dag`, which also
carries its standalone scalar roots.

The coverage tests now check what each #573 §4.2.2 example's definition
keeps, not only its selection: the range example yields exactly one box and
a scan without the lifted predicate, the matcher below `rate` stays in the
definition's scan, and the definition keeps the `rate` under a lifted
`Filter`. The two missing rows are added: `rate(a[5m]) > rate(b[5m])` (a
`BinaryOp`, nothing selected) and `rate(m[5m]) > bool 0` (a `Project`,
kept in the definition).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 12, 2026
The #573 coverage tests no longer depend on which candidate the default
cost model selects: each takes the one KLL SummaryAgg among the search
space's candidates. The SQL tests also check that the definition's
SummaryAgg has no filter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 13, 2026
Lower each example query of the ASAP primitive schema design doc (#573
§4.2.2), plan it with the default cost model and check the coverage of the
selected KLL state: SQL value sets, ranges and renamed columns, a PromQL
offset window as relative time, a matcher below rate kept in the definition,
and a PromQL comparison above rate lifted into the selection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 13, 2026
…initions

`execution_timed_workload_dag` becomes `pub(crate)`: callers export a
planned workload through `PlanOutput::execution_timed_dag`, which also
carries its standalone scalar roots.

The coverage tests now check what each #573 §4.2.2 example's definition
keeps, not only its selection: the range example yields exactly one box and
a scan without the lifted predicate, the matcher below `rate` stays in the
definition's scan, and the definition keeps the `rate` under a lifted
`Filter`. The two missing rows are added: `rate(a[5m]) > rate(b[5m])` (a
`BinaryOp`, nothing selected) and `rate(m[5m]) > bool 0` (a `Project`,
kept in the definition).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 13, 2026
The #573 coverage tests no longer depend on which candidate the default
cost model selects: each takes the one KLL SummaryAgg among the search
space's candidates. The SQL tests also check that the definition's
SummaryAgg has no filter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 13, 2026
Lower each example query of the ASAP primitive schema design doc (#573
§4.2.2), plan it with the default cost model and check the coverage of the
selected KLL state: SQL value sets, ranges and renamed columns, a PromQL
offset window as relative time, a matcher below rate kept in the definition,
and a PromQL comparison above rate lifted into the selection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 13, 2026
…initions

`execution_timed_workload_dag` becomes `pub(crate)`: callers export a
planned workload through `PlanOutput::execution_timed_dag`, which also
carries its standalone scalar roots.

The coverage tests now check what each #573 §4.2.2 example's definition
keeps, not only its selection: the range example yields exactly one box and
a scan without the lifted predicate, the matcher below `rate` stays in the
definition's scan, and the definition keeps the `rate` under a lifted
`Filter`. The two missing rows are added: `rate(a[5m]) > rate(b[5m])` (a
`BinaryOp`, nothing selected) and `rate(m[5m]) > bool 0` (a `Project`,
kept in the definition).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 13, 2026
The #573 coverage tests no longer depend on which candidate the default
cost model selects: each takes the one KLL SummaryAgg among the search
space's candidates. The SQL tests also check that the definition's
SummaryAgg has no filter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 13, 2026
Lower each example query of the ASAP primitive schema design doc (#573
§4.2.2), plan it with the default cost model and check the coverage of the
selected KLL state: SQL value sets, ranges and renamed columns, a PromQL
offset window as relative time, a matcher below rate kept in the definition,
and a PromQL comparison above rate lifted into the selection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 13, 2026
…initions

`execution_timed_workload_dag` becomes `pub(crate)`: callers export a
planned workload through `PlanOutput::execution_timed_dag`, which also
carries its standalone scalar roots.

The coverage tests now check what each #573 §4.2.2 example's definition
keeps, not only its selection: the range example yields exactly one box and
a scan without the lifted predicate, the matcher below `rate` stays in the
definition's scan, and the definition keeps the `rate` under a lifted
`Filter`. The two missing rows are added: `rate(a[5m]) > rate(b[5m])` (a
`BinaryOp`, nothing selected) and `rate(m[5m]) > bool 0` (a `Project`,
kept in the definition).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 13, 2026
The #573 coverage tests no longer depend on which candidate the default
cost model selects: each takes the one KLL SummaryAgg among the search
space's candidates. The SQL tests also check that the definition's
SummaryAgg has no filter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 13, 2026
* feat(planner): export a planned workload as one physical ASAP DAG

PlanOutput::execution_timed_dag times every query root, and every operator
reference of a standalone scalar root, with one shared TimingMemo and
compiles them with compile_physical_asap_workload, so a batch is one DAG
with a root per operator query, a scalar root per standalone scalar query
(keyed by entry) and shared sub-DAGs exported once. Lifecycle phases come
from the deployments of every plan, including the plans of scalar-read
sub-DAGs. The per-plan SummaryMaintenanceLifecyclePlan::execution_timed_dag
is the batch of one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test(integration): derive summary coverage from SQL and PromQL queries

Lower each example query of the ASAP primitive schema design doc (#573
§4.2.2), plan it with the default cost model and check the coverage of the
selected KLL state: SQL value sets, ranges and renamed columns, a PromQL
offset window as relative time, a matcher below rate kept in the definition,
and a PromQL comparison above rate lifted into the selection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(planner): keep the lifecycle export internal; assert coverage definitions

`execution_timed_workload_dag` becomes `pub(crate)`: callers export a
planned workload through `PlanOutput::execution_timed_dag`, which also
carries its standalone scalar roots.

The coverage tests now check what each #573 §4.2.2 example's definition
keeps, not only its selection: the range example yields exactly one box and
a scan without the lifted predicate, the matcher below `rate` stays in the
definition's scan, and the definition keeps the `rate` under a lifted
`Filter`. The two missing rows are added: `rate(a[5m]) > rate(b[5m])` (a
`BinaryOp`, nothing selected) and `rate(m[5m]) > bool 0` (a `Project`,
kept in the definition).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test(integration): read the KLL coverage state from the candidate list

The #573 coverage tests no longer depend on which candidate the default
cost model selects: each takes the one KLL SummaryAgg among the search
space's candidates. The SQL tests also check that the definition's
SummaryAgg has no filter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs(planner): say scalar roots are keyed by batch position; empty workloads

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant