Skip to content

docs: propose workload-wide planning, summary sharing, and materialization - #509

Merged
zzylol merged 56 commits into
mainfrom
docs/planner-layering-contract
Oct 2, 2026
Merged

docs: propose workload-wide planning, summary sharing, and materialization#509
zzylol merged 56 commits into
mainfrom
docs/planner-layering-contract

Conversation

@zzylol

@zzylol zzylol commented Sep 30, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

Propose ASAPPlanner's planning stages, from language-specific frontends through logical and physical optimization to selection of one physical plan for the whole workload. The deployment supplies empirical cost and accuracy models and execution capabilities, then executes the selected plan.

Add docs/design_docs/proposals/planner-layering.md and link it from the proposals index.

Before this PR

The planning boundary needs an explicit contract for workload-wide candidates, summary sharing, materialization, and plan selection.

After this PR

  • Frontends produce CandidateLogicalDAGs; logical optimization produces CandidateLogicalASAPDAGs; physical optimization produces CandidatePhysicalASAPDAGs; selection returns one PhysicalASAPDAG.
  • Logical optimization generates local exact and summary alternatives, then applies ASAP-aware common-subexpression elimination across computations and windows.
  • Physical optimization explores materialization, retention and operator implementations. Outputs may be maintained at ingestion time, retained at query time, or rebuilt without materialization.
  • Selection evaluates complete workload candidates using deployment cost and accuracy models, rejects candidates that fail requirements or capabilities, and chooses the cheapest remaining plan.

Four worked examples cover candidate growth (3 local, 54 logical and 156 physical alternatives), UnivMon sharing across three computations, KLL sharing across windows, and ingestion/query-time materialization choices.

Scope and validation

Documentation-only proposal; no runtime or API changes. Deployment-input types, summary subtract/delete design, and physical parallelism, partitioning and resource management remain TODO.

The branch's file contents have been restored to commit 8c29c2e376800ad0823d4917e121eed3b60ed3ab and verified identical with git diff. No runtime tests run.

zzylol and others added 3 commits September 30, 2026 17:59
@zzylol @claude
docs: define the Planner and deployment layering contract
b400e54 
Add the layering proposal, its proposals-index entry, and an Output layers
section in the input/output/workflow design. Every Planner layer outputs all
legal candidates; the deployment keeps every summary-family candidate and
selects with its own costs. Design DAG names are marked as the target API with
the current main type alongside.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol @claude
docs: treat source binding as a PhysicalDAG execution step
f1c13d4 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol @claude
docs: place summary materialization under physical planning
2bc772f 
Split physical planning into physical design (summary materialization,
workload-level, like materialized-view selection) and physical
implementation (per-node lowering and cutting). The annotated graph stays a
Post-ASAP DAG; materialization is decided in design and realized by the cut.
State node-by-node correspondence and the Fallback exception that the
operator-flattening proposal removes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol zzylol mentioned this pull request Sep 30, 2026
Closed
zzylol and others added 5 commits September 30, 2026 18:27
@zzylol @claude
docs: point per-operator export at #481
6702d3a 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol @claude
docs: state family retention and timing-sensitive lowering plainly
1bb8ec5 
Describe current Planner behaviour (no family pruning) and the backend gap,
and replace the Binary 'exception' with the general rule that a timing-
sensitive node needs one compilation per distinct timing, with an example.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol @claude
docs: drop Binary lowering detail from the layering proposal
1e84c15 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol @claude
docs: drop mixed-placement and compatibility sections from the layeri…
cd6a18d 
…ng proposal

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol @claude
docs: drop the implementation-status section from the layering proposal
aa0b959 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Selvomega
Selvomega self-requested a review September 30, 2026 18:36
@zzylol @claude
docs: leave input-output-workflow unchanged in the layering PR
62ae3e7 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Selvomega
Selvomega requested changes Sep 30, 2026
View reviewed changes
Comment thread docs/design_docs/architecture/input-output-workflow.md Outdated
Comment thread docs/design_docs/architecture/input-output-workflow.md Outdated
Comment thread docs/design_docs/proposals/planner-backend-layering.md Outdated
@zzylol @claude
docs: move selection into ASAPPlanner; deployment supplies prices
3607270 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Selvomega
Selvomega requested changes Sep 30, 2026
View reviewed changes
Comment thread docs/design_docs/proposals/planner-backend-layering.md Outdated
Comment thread docs/design_docs/proposals/planner-backend-layering.md Outdated
Comment thread docs/design_docs/proposals/planner-backend-layering.md Outdated
Comment thread docs/design_docs/proposals/planner-backend-layering.md Outdated
Comment thread docs/design_docs/proposals/planner-backend-layering.md Outdated
Comment thread docs/design_docs/proposals/planner-layering.md Outdated
Comment thread docs/design_docs/proposals/planner-backend-layering.md Outdated
Comment thread docs/design_docs/proposals/planner-backend-layering.md Outdated
Comment thread docs/design_docs/proposals/planner-backend-layering.md Outdated
@zzylol @claude
docs: address review of layering proposal
dc275fd 
Planner takes the query and data workloads plus the deployment's cost
model, accuracy requirements and capabilities, and returns one optimal
PhysicalDAG; candidate sets stay internal. Name the annotated stage
MaterializedPostASAPDAG, tabulate what each DAG encodes, fold timing and
selection into the layer descriptions, rename physical design to summary
materialization, and make the example trace one query through every DAG.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Selvomega
Selvomega requested changes Sep 30, 2026
View reviewed changes
Comment thread docs/design_docs/proposals/planner-backend-layering.md Outdated
Comment thread docs/design_docs/proposals/planner-backend-layering.md Outdated
Comment thread docs/design_docs/proposals/planner-backend-layering.md Outdated
@zzylol @claude
docs: name layer 2 summary lifecycle planning; drop deployment rankin…
a095ce5 
…g row

A lifecycle fixes materialization, timing, maintenance, retention and
window framework together, so the stage and its DAG are named after the
lifecycle (LifecyclePostASAPDAG) rather than one of those aspects. The
deployment row no longer mentions ranking or selection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Sep 30, 2026
@zzylol @claude
docs: match the DAG API docs to the implemented collection boundaries
25d54e9 
Describe the two plain candidate collections, the view-based compile, the
caller-typed physical candidate errors, lifecycle_guarantee, and
PhysicalExecution as an execution handle. Remove APIs the docs said were
removed but never existed (compile_timed_candidates, PhysicalDAGCandidate),
the agent instructions in the alignment proposal's baseline, and the
ingestion-time Binary exception from design docs, where it is an
implementation detail (the developer migration guide keeps it).

Restore #485's statement that candidates do not choose placement and #508's
CandidatePostASAPDAGs<Id> names in input-output-workflow.md, rejoin the
split test table in physical-planning-and-deployment.md, and take
planner-backend-layering.md verbatim from #509.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Sep 30, 2026
@zzylol @claude
docs: sync the layering proposal with #509
7fde378 
Copies docs/design_docs/proposals/planner-backend-layering.md from #509's
head 3607270 so this file merges cleanly after #509.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Sep 30, 2026
@zzylol @claude
refactor!: merge the lifecycle plan into LifecyclePostASAPDAG
c8ff36b 
The layering proposal (#509) names a single annotated DAG,
LifecyclePostASAPDAG: a PostASAPDAG plus its lifecycle assignment. The
code kept a SummaryMaintenanceLifecyclePlan beside the DAG and named the
timed collection CandidatePostASAPDAGsWithTiming.

- Rename SummaryMaintenanceLifecyclePlan to LifecyclePostASAPDAG and its
  error to LifecyclePostASAPDAGError. The type already held the root and
  each state's lifecycle, retention and window framework; per-node timing
  stays derived by execution_assignment as the existing
  PostASAPDAGAssignment overlay, so no timed graph is stored beside it and
  no new type is added.
- Rename CandidatePostASAPDAGsWithTiming to CandidateLifecyclePostASAPDAGs;
  CandidatePostASAPDAGs stays the logical collection.
- Call layer 2 "summary lifecycle planning" and reword comments that had
  the deployment choose or rank; selection is Planner's, over the
  deployment's cost model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Sep 30, 2026
@zzylol @claude
docs: make selection Planner's, over the deployment's cost model
e7453ee 
Several docs still said the deployment or "downstream" selects, ranks or
chooses the plan, and the glossary said physical plans are owned by
downstream systems. Per the layering proposal (#509), Planner compiles
and selects the physical plan using the deployment's cost model; the
deployment supplies prices and executes the selected plan.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Sep 30, 2026
@zzylol @claude
docs: copy the layering proposal from #509 byte for byte
c3f10eb 
#509 carries the proposal; #480's copy now matches its head a095ce5
exactly, so merging either PR first leaves no conflict.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol zzylol mentioned this pull request Sep 30, 2026
Closed
@zzylol
zzylol requested a review from Selvomega September 30, 2026 20:37
Selvomega
Selvomega previously approved these changes Sep 30, 2026
View reviewed changes
Comment thread docs/design_docs/proposals/planner-layering.md Outdated
@zzylol @claude
docs: rename layering proposal to planner-layering.md
a3c6dbc 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Sep 30, 2026
@zzylol @claude
docs: follow #509's rename to planner-layering.md
67d01c9 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol @claude
docs: name the DAG stages LogicalPostASAPDAG and PhysicalPostASAPDAG
41076ca 
Every stage after the frontend is a Post-ASAP DAG; the prefix says how
far planning has gone: Logical, Lifecycle, Physical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 5, 2026
@zzylol @claude
docs: describe the stage pipeline instead of the removed legacy search
64e122d 
Library API, architecture overview, planner pipeline and the accuracy
and cost developer docs now describe the #509 stage pipeline. Docs
that only describe removed APIs (legacy search, CostModel,
explanation, physical-plan cost adapter) are marked historical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol zzylol mentioned this pull request Oct 5, 2026
Draft
2 tasks done
zzylol added a commit that referenced this pull request Oct 6, 2026
@zzylol @claude
refactor(planner): remove the summary maintenance lifecycle
21cc044 
Stage 2 materialization (#509) will decide per sub-DAG whether and when to
materialize, so the Planner no longer chooses a maintenance lifecycle.

- MajorPass now runs search_workload_with_targets -> global_selection ->
  assemble_selected_dag per root -> share_common_sub_dags.
- QueryLifecyclePlan becomes QueryPlan { entry_index, root }; PlanOutput's
  execution_timed_dag times the roots directly. LifecycleInput, the
  lifecycle errors and UserInput's `lifecycle` field are gone (public API
  break).
- Delete summary_maintenance_lifecycle, summary_maintenance_cost,
  summary_maintenance_dag_export, post_asap::{summary_maintenance,
  summary_maintenance_lifecycle} and SummaryWindowFramework (the pane
  primitives stay), the lifecycle CostModel hooks, CandidateCostOverrides
  and EmpiricalEvidenceProvider::lifecycle_cost_inputs.
- Delete the lifecycle e2e test and the viewer's lifecycle-plan UI; rewire
  e2e_plan, summary_sharing, operator_design_examples and
  weighted_topk_binding to the new pipeline.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 6, 2026
@zzylol @claude
refactor(ir): default to query-time materialization
c14c60e 
Rename LifecycleAssignment to MaterializationAssignment and
apply_lifecycle_timings to apply_materialization_timings. The default
assignment is now all_query_time(): nothing is materialized until Stage 2
materialization (#509) decides per sub-DAG. all_ingestion_time() and set()
assign maintenance explicitly.

- validate_default becomes validate_maintained: candidate legality is still
  checked with every summary maintained, so candidate generation is
  unchanged. planned_data_state and fixed_window_rate_candidates use the
  same maintained assumption, and the maintained precompute compilers in
  promql_rows assign ingestion time explicitly.
- PlanOutput::execution_timed_dag, show_post_asap_ir and the default test
  helpers now emit query-time summaries.
- Tests that exercise maintenance assign it explicitly (new `maintained`
  helpers); a new unit test pins the query-time default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 6, 2026
@zzylol @claude
docs: drop the summary maintenance lifecycle and document the query-t…
c20b6cf 
…ime default

Remove the lifecycle APIs, recipes and viewer section from the docs, delete
the workload-demand-and-summary-lifecycle proposal, and point materialization
questions to Stage 2 (#509). Rename LifecycleAssignment/apply_lifecycle_timings/
validate_default to their new names and describe the all-query-time default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 6, 2026
@zzylol @claude
refactor(planner): plan one shared unified workload DAG
166e386 
Cut today's planner over to the unified OperatorNode IR. Frontends return
OperatorNode / QueryRoot, ParsedWorkload keeps scalar roots, Pass 1
(ASAPStrategies), the existing identical-sub-DAG sharing, selection, DAG
assembly and lifecycle costing all run on OperatorNode, and PlanOutput exposes
the whole workload DAG. The native compiler from #541 becomes the canonical
physical_planner and consumes the PhysicalASAPDAG export.

Ported from the earlier #542 (95eef55) without new #509 stage logic. Legacy
QueryExpr/SummaryNode modules stay compiled for their own tests but are no
longer re-exported from post_asap; the cleanup PR removes them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 6, 2026
@zzylol @claude
refactor(planner): remove the summary maintenance lifecycle
20abea6 
Stage 2 materialization (#509) will decide per sub-DAG whether and when to
materialize, so the Planner no longer chooses a maintenance lifecycle.

- MajorPass now runs search_workload_with_targets -> global_selection ->
  assemble_selected_dag per root -> share_common_sub_dags.
- QueryLifecyclePlan becomes QueryPlan { entry_index, root }; PlanOutput's
  execution_timed_dag times the roots directly. LifecycleInput, the
  lifecycle errors and UserInput's `lifecycle` field are gone (public API
  break).
- Delete summary_maintenance_lifecycle, summary_maintenance_cost,
  summary_maintenance_dag_export, post_asap::{summary_maintenance,
  summary_maintenance_lifecycle} and SummaryWindowFramework (the pane
  primitives stay), the lifecycle CostModel hooks, CandidateCostOverrides
  and EmpiricalEvidenceProvider::lifecycle_cost_inputs.
- Delete the lifecycle e2e test and the viewer's lifecycle-plan UI; rewire
  e2e_plan, summary_sharing, operator_design_examples and
  weighted_topk_binding to the new pipeline.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 6, 2026
@zzylol @claude
refactor(ir): default to query-time materialization
69caa82 
Rename LifecycleAssignment to MaterializationAssignment and
apply_lifecycle_timings to apply_materialization_timings. The default
assignment is now all_query_time(): nothing is materialized until Stage 2
materialization (#509) decides per sub-DAG. all_ingestion_time() and set()
assign maintenance explicitly.

- validate_default becomes validate_maintained: candidate legality is still
  checked with every summary maintained, so candidate generation is
  unchanged. planned_data_state and fixed_window_rate_candidates use the
  same maintained assumption, and the maintained precompute compilers in
  promql_rows assign ingestion time explicitly.
- PlanOutput::execution_timed_dag, show_post_asap_ir and the default test
  helpers now emit query-time summaries.
- Tests that exercise maintenance assign it explicitly (new `maintained`
  helpers); a new unit test pins the query-time default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 6, 2026
@zzylol @claude
docs: drop the summary maintenance lifecycle and document the query-t…
5c1560e 
…ime default

Remove the lifecycle APIs, recipes and viewer section from the docs, delete
the workload-demand-and-summary-lifecycle proposal, and point materialization
questions to Stage 2 (#509). Rename LifecycleAssignment/apply_lifecycle_timings/
validate_default to their new names and describe the all-query-time default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 6, 2026
@zzylol @claude
refactor(planner): remove the summary maintenance lifecycle
823e248 
Stage 2 materialization (#509) will decide per sub-DAG whether and when to
materialize, so the Planner no longer chooses a maintenance lifecycle.

- MajorPass now runs search_workload_with_targets -> global_selection ->
  assemble_selected_dag per root -> share_common_sub_dags.
- QueryLifecyclePlan becomes QueryPlan { entry_index, root }; PlanOutput's
  execution_timed_dag times the roots directly. LifecycleInput, the
  lifecycle errors and UserInput's `lifecycle` field are gone (public API
  break).
- Delete summary_maintenance_lifecycle, summary_maintenance_cost,
  summary_maintenance_dag_export, post_asap::{summary_maintenance,
  summary_maintenance_lifecycle} and SummaryWindowFramework (the pane
  primitives stay), the lifecycle CostModel hooks, CandidateCostOverrides
  and EmpiricalEvidenceProvider::lifecycle_cost_inputs.
- Delete the lifecycle e2e test and the viewer's lifecycle-plan UI; rewire
  e2e_plan, summary_sharing, operator_design_examples and
  weighted_topk_binding to the new pipeline.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 6, 2026
@zzylol @claude
refactor(ir): default to query-time materialization
0d3b750 
Rename LifecycleAssignment to MaterializationAssignment and
apply_lifecycle_timings to apply_materialization_timings. The default
assignment is now all_query_time(): nothing is materialized until Stage 2
materialization (#509) decides per sub-DAG. all_ingestion_time() and set()
assign maintenance explicitly.

- validate_default becomes validate_maintained: candidate legality is still
  checked with every summary maintained, so candidate generation is
  unchanged. planned_data_state and fixed_window_rate_candidates use the
  same maintained assumption, and the maintained precompute compilers in
  promql_rows assign ingestion time explicitly.
- PlanOutput::execution_timed_dag, show_post_asap_ir and the default test
  helpers now emit query-time summaries.
- Tests that exercise maintenance assign it explicitly (new `maintained`
  helpers); a new unit test pins the query-time default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 6, 2026
@zzylol @claude
docs: drop the summary maintenance lifecycle and document the query-t…
877e5d1 
…ime default

Remove the lifecycle APIs, recipes and viewer section from the docs, delete
the workload-demand-and-summary-lifecycle proposal, and point materialization
questions to Stage 2 (#509). Rename LifecycleAssignment/apply_lifecycle_timings/
validate_default to their new names and describe the all-query-time default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol zzylol mentioned this pull request Oct 6, 2026
Merged
zzylol added a commit that referenced this pull request Oct 6, 2026
@zzylol @claude
feat(ir): add canonicalization, sub-DAG sharing and flat DAGs on the …
76e713b 
…unified IR (#645)

#536 defined the unified OperatorNode DAG, but canonicalization, common
sub-DAG sharing and a flat, serializable form existed only for the old
split IRs. Add them on OperatorNode:

- ir::canonicalize: heavy-hitter promotion and EXISTS / NOT EXISTS / IN
  subquery lowering, bottom-up and memoized so shared sub-DAGs stay shared.
- ir::cse: hash-consing over a workload batch (the identical-expression
  rule of #509 Pass 2), following scalar-referenced operator nodes too.
- Generic child references: Operator, NonASAPOp, ASAPOp, ScalarExpr,
  Predicate, ProjectItem, SortKey and QueryRoot take the child reference
  as a type parameter (default Rc<OperatorNode>). ir::flat::flatten writes
  a DAG as nodes whose operators are Operator<NodeId>.

Summary coverage (#567) and SummaryMerge (#560) build on this.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 6, 2026
@zzylol @claude
refactor(planner): remove the summary maintenance lifecycle
5c3f5f4 
Stage 2 materialization (#509) will decide per sub-DAG whether and when to
materialize, so the Planner no longer chooses a maintenance lifecycle.

- MajorPass now runs search_workload_with_targets -> global_selection ->
  assemble_selected_dag per root -> share_common_sub_dags.
- QueryLifecyclePlan becomes QueryPlan { entry_index, root }; PlanOutput's
  execution_timed_dag times the roots directly. LifecycleInput, the
  lifecycle errors and UserInput's `lifecycle` field are gone (public API
  break).
- Delete summary_maintenance_lifecycle, summary_maintenance_cost,
  summary_maintenance_dag_export, post_asap::{summary_maintenance,
  summary_maintenance_lifecycle} and SummaryWindowFramework (the pane
  primitives stay), the lifecycle CostModel hooks, CandidateCostOverrides
  and EmpiricalEvidenceProvider::lifecycle_cost_inputs.
- Delete the lifecycle e2e test and the viewer's lifecycle-plan UI; rewire
  e2e_plan, summary_sharing, operator_design_examples and
  weighted_topk_binding to the new pipeline.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 6, 2026
@zzylol @claude
refactor(ir): default to query-time materialization
38ff0e5 
Rename LifecycleAssignment to MaterializationAssignment and
apply_lifecycle_timings to apply_materialization_timings. The default
assignment is now all_query_time(): nothing is materialized until Stage 2
materialization (#509) decides per sub-DAG. all_ingestion_time() and set()
assign maintenance explicitly.

- validate_default becomes validate_maintained: candidate legality is still
  checked with every summary maintained, so candidate generation is
  unchanged. planned_data_state and fixed_window_rate_candidates use the
  same maintained assumption, and the maintained precompute compilers in
  promql_rows assign ingestion time explicitly.
- PlanOutput::execution_timed_dag, show_post_asap_ir and the default test
  helpers now emit query-time summaries.
- Tests that exercise maintenance assign it explicitly (new `maintained`
  helpers); a new unit test pins the query-time default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 6, 2026
@zzylol @claude
docs: drop the summary maintenance lifecycle and document the query-t…
44a7839 
…ime default

Remove the lifecycle APIs, recipes and viewer section from the docs, delete
the workload-demand-and-summary-lifecycle proposal, and point materialization
questions to Stage 2 (#509). Rename LifecycleAssignment/apply_lifecycle_timings/
validate_default to their new names and describe the all-query-time default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 6, 2026
@zzylol @claude
refactor(planner): remove the summary maintenance lifecycle
479cf3a 
Stage 2 materialization (#509) will decide per sub-DAG whether and when to
materialize, so the Planner no longer chooses a maintenance lifecycle.

- MajorPass now runs search_workload_with_targets -> global_selection ->
  assemble_selected_dag per root -> share_common_sub_dags.
- QueryLifecyclePlan becomes QueryPlan { entry_index, root }; PlanOutput's
  execution_timed_dag times the roots directly. LifecycleInput, the
  lifecycle errors and UserInput's `lifecycle` field are gone (public API
  break).
- Delete summary_maintenance_lifecycle, summary_maintenance_cost,
  summary_maintenance_dag_export, post_asap::{summary_maintenance,
  summary_maintenance_lifecycle} and SummaryWindowFramework (the pane
  primitives stay), the lifecycle CostModel hooks, CandidateCostOverrides
  and EmpiricalEvidenceProvider::lifecycle_cost_inputs.
- Delete the lifecycle e2e test and the viewer's lifecycle-plan UI; rewire
  e2e_plan, summary_sharing, operator_design_examples and
  weighted_topk_binding to the new pipeline.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 6, 2026
@zzylol @claude
refactor(ir): default to query-time materialization
597325b 
Rename LifecycleAssignment to MaterializationAssignment and
apply_lifecycle_timings to apply_materialization_timings. The default
assignment is now all_query_time(): nothing is materialized until Stage 2
materialization (#509) decides per sub-DAG. all_ingestion_time() and set()
assign maintenance explicitly.

- validate_default becomes validate_maintained: candidate legality is still
  checked with every summary maintained, so candidate generation is
  unchanged. planned_data_state and fixed_window_rate_candidates use the
  same maintained assumption, and the maintained precompute compilers in
  promql_rows assign ingestion time explicitly.
- PlanOutput::execution_timed_dag, show_post_asap_ir and the default test
  helpers now emit query-time summaries.
- Tests that exercise maintenance assign it explicitly (new `maintained`
  helpers); a new unit test pins the query-time default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 6, 2026
@zzylol @claude
docs: drop the summary maintenance lifecycle and document the query-t…
13628f9 
…ime default

Remove the lifecycle APIs, recipes and viewer section from the docs, delete
the workload-demand-and-summary-lifecycle proposal, and point materialization
questions to Stage 2 (#509). Rename LifecycleAssignment/apply_lifecycle_timings/
validate_default to their new names and describe the all-query-time default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 7, 2026
@zzylol @claude
refactor(planner): plan one shared unified workload DAG
4041003 
Cut today's planner over to the unified OperatorNode IR. Frontends return
OperatorNode / QueryRoot, ParsedWorkload keeps scalar roots, Pass 1
(ASAPStrategies), the existing identical-sub-DAG sharing, selection, DAG
assembly and lifecycle costing all run on OperatorNode, and PlanOutput exposes
the whole workload DAG. The native compiler from #541 becomes the canonical
physical_planner and consumes the PhysicalASAPDAG export.

Ported from the earlier #542 (95eef55) without new #509 stage logic. Legacy
QueryExpr/SummaryNode modules stay compiled for their own tests but are no
longer re-exported from post_asap; the cleanup PR removes them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 7, 2026
@zzylol @claude
refactor(planner): remove the summary maintenance lifecycle
ca3a631 
Stage 2 materialization (#509) will decide per sub-DAG whether and when to
materialize, so the Planner no longer chooses a maintenance lifecycle.

- MajorPass now runs search_workload_with_targets -> global_selection ->
  assemble_selected_dag per root -> share_common_sub_dags.
- QueryLifecyclePlan becomes QueryPlan { entry_index, root }; PlanOutput's
  execution_timed_dag times the roots directly. LifecycleInput, the
  lifecycle errors and UserInput's `lifecycle` field are gone (public API
  break).
- Delete summary_maintenance_lifecycle, summary_maintenance_cost,
  summary_maintenance_dag_export, post_asap::{summary_maintenance,
  summary_maintenance_lifecycle} and SummaryWindowFramework (the pane
  primitives stay), the lifecycle CostModel hooks, CandidateCostOverrides
  and EmpiricalEvidenceProvider::lifecycle_cost_inputs.
- Delete the lifecycle e2e test and the viewer's lifecycle-plan UI; rewire
  e2e_plan, summary_sharing, operator_design_examples and
  weighted_topk_binding to the new pipeline.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 7, 2026
@zzylol @claude
refactor(ir): default to query-time materialization
f483171 
Rename LifecycleAssignment to MaterializationAssignment and
apply_lifecycle_timings to apply_materialization_timings. The default
assignment is now all_query_time(): nothing is materialized until Stage 2
materialization (#509) decides per sub-DAG. all_ingestion_time() and set()
assign maintenance explicitly.

- validate_default becomes validate_maintained: candidate legality is still
  checked with every summary maintained, so candidate generation is
  unchanged. planned_data_state and fixed_window_rate_candidates use the
  same maintained assumption, and the maintained precompute compilers in
  promql_rows assign ingestion time explicitly.
- PlanOutput::execution_timed_dag, show_post_asap_ir and the default test
  helpers now emit query-time summaries.
- Tests that exercise maintenance assign it explicitly (new `maintained`
  helpers); a new unit test pins the query-time default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 7, 2026
@zzylol @claude
docs: drop the summary maintenance lifecycle and document the query-t…
0e5e4f8 
…ime default

Remove the lifecycle APIs, recipes and viewer section from the docs, delete
the workload-demand-and-summary-lifecycle proposal, and point materialization
questions to Stage 2 (#509). Rename LifecycleAssignment/apply_lifecycle_timings/
validate_default to their new names and describe the all-query-time default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 9, 2026
Merged
Merged
zzylol added a commit that referenced this pull request Oct 12, 2026
@zzylol @claude
docs: align #509/#511/#573 docs with the decided vocabulary and Pass …
7723223 
…1 candidates (#663)

* docs: align #509/#511/#573 docs with the decided vocabulary and Pass 1 candidates

- operator-sharing.md: ReadPopulation { readout } -> EvaluatePopulation
  { evaluation } (matches crates/types/src/ir/asap.rs); "readout" ->
  "evaluation" where it names reading summary state; add the missing
  `filter` field to the SummaryAgg sketch.
- asap-primitive-schema.md: "readout" -> "evaluation" where it names the
  operation (intent table header, §6.5 heading and its anchor/table row,
  top-k evaluation, guarantee of an evaluation). Verb "read out" kept, as
  in the code comments. Coverage cache text unchanged.
- planner-layering.md: define an eligible sub-DAG at its first use; add
  the exact accumulator state candidate to the Pass 1 table and text;
  recompute Example 1 (Pass 1: 2 x 2 x 2 x 3 = 24, Pass 2:
  24 x 2 x 3 x 3 = 432, Stage 2: 24 x 52 = 1,248).
- planner-layering.md: every aggregation is its own Pass 1 target,
  including nested ones (Q1's per-series rate and per-job sum, Q2's
  per-series sum_over_time and per-job top-k), as in #561's
  enumerate_local_logical_candidates; the Algorithm 1 example and diagram
  show the four sub-DAGs, and Example 1 lists options per target.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: leave multi-measure targets out of the eligible sub-DAG definition

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: one exact option per Pass 1 target

Each Pass 1 target gets a single exact option: exact accumulator state
when the intent has a buildable mergeable exact accumulator, otherwise
the plain exact computation. Example 1 returns to 3 / 54 / 156 with
four targets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: reflow the eligible sub-DAG paragraph

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol zzylol mentioned this pull request Oct 12, 2026
Open
zzylol added a commit that referenced this pull request Oct 12, 2026
@zzylol @claude
fix(planner): Pass 1 local candidates per #509
9d5ab8f 
Summary candidates are unsized (LocalCandidate::Summary carries only the
algorithm); sizing happens after Pass 2 sharing. Every pruned candidate is
recorded on its target with a PruneReason (RequiresExact,
NullableAvgInput). Each measure of an aggregate with a realization besides
exact execution is its own target; input containing ASAP operators is
rejected. avg is rewritten as exact sum/count accumulator
states. The Example 1 test runs Q1 as exact, and the dev doc places
composition in Pass 1.

- One exact option per target: the exact accumulator (ExactAccumulator,
  or AvgAsSumCount for avg) when it can be built, otherwise plain Exact
  (NullableAvgInput prune, or no accumulator form).
  The plain form would duplicate "accumulator, not materialized"; Stage
  2/3 decide materialization and sharing.
- Per-measure filters and HAVING keep every candidate. The target records
  the measure's filter (applied to the state as SummaryAgg.filter) and a
  HavingPlacement for the aggregate's having or a Filter directly above it
  (the SQL frontend's HAVING form): BelowAggregate when it reads only group
  keys (pushed below as a row predicate), else AboveOutput (a Filter above
  the finalize/estimate output; approximate candidates carry guarantee =
  None). Composition (#575) applies them; PruneReason::FilteredAggregate
  is removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
@zzylol zzylol mentioned this pull request Oct 12, 2026
Open
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Selvomega Selvomega Selvomega left review comments
@milindsrivastava1997 milindsrivastava1997 milindsrivastava1997 left review comments
@GordonYuanyc GordonYuanyc Awaiting requested review from GordonYuanyc

Assignees

@zzylol zzylol

Labels

documentation Improvements or additions to documentation

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@zzylol @milindsrivastava1997 @Selvomega