feat(stage2): query time, kept for tumbling panes (Example 4 B3)

[zzylol]

Stack: #574 → #620 → #618 → #621 → #627 → #625 → #628 → #632 → #634 → #616 → #617 → #622 → #624 → #629 → #630 → #631 → #633 → #635 → #636 → #637

Problem

Stage 2 offered a repeating query's tumbling panes only two ways: rebuilt at query time at every evaluation (B2) or maintained at ingestion time (B1). Example 4's third option, B3 (query time, kept), was missing, so stage2_b_tumbling_kll_has_three_materialization_options and stage2_b_b3_keeps_query_time_windows were ignored, and Stage 3 never priced it.

Changes

Decisions Q56–Q59, Q63.

• IR. PhysicalASAPDAGNode gains kept: bool: a query-time output kept across evaluations. It is serialized only when true, so documents without kept nodes are unchanged; the wire version stays 8. ExecutionTiming stays two-valued (Q58). MaterializationAssignment carries the kept set, and split_shared_by_phase rekeys it.
• Stage 2. MaterializationSpace now gives each unit a three-way choice: not materialized, ingestion time, or kept (Choice, Choices). Only tumbling panes can be kept, and only when every root reaching them repeats at a fixed interval equal to the pane width, so each evaluation builds exactly one new pane. Keeping needs neither arriving data nor predictability. down_closed_sets still keeps the ingestion-time units down-closed, and kept units are unconstrained. It returns None above MAX_PHYSICAL_PER_LOGICAL (16): the greedy search then also tries "kept" moves. With 5 independent pane chains that is 3^5 = 243 choices, which are searched greedily. Labels read e.g. query time, kept: Kll ×5 panes.
• Stage 3.
  • capability_violation rejects any candidate with a kept node when query_time_retention is false, with the reason deployment cannot keep query-time state across evaluations (query time, kept) (Q57). The executor stays query_time_retention: false (Q56).
  • pane_roles also covers kept chains. The newest pane is built per evaluation at the evaluation rate, and it retains (N − 1) · groups · state_bytes bytes at cost_per_retained_byte_second, counted against the memory budget. The older panes and their shifts and ranges cost 0. Through scan_extent_ms, both the scan and the raw retention cover one pane width w (Q59). Per-evaluation work, which also feeds the latency check, is the newest pane plus the merge and the estimate.
• Doc. stage3-cost-model.md moves B3 from out of scope into the memory term and deployment checks. Cold start is not priced.
• Viewer. A kept node's timing reads ⏱ query time, kept, in both the node label and the details panel.
• Fixture. Example 1 legitimately changes. Its 24 logical candidates with Q2's 10-s sum panes each gain a query time, kept: exact Sum ×6 panes plan, which takes it from 112 to 136 physical plans. The executor rejects all of them on capability, so the selection is unchanged: P82, 4.620/s. The fixture's --max-candidates goes from 128 to 160, so every plan is still written.

Tests

• Un-ignored: stage2_b_tumbling_kll_has_three_materialization_options and stage2_b_b3_keeps_query_time_windows (Example 4). The second now also checks that the five panes are kept.
• Replaced: stage2_b_tumbling_kll_has_b1_and_b2, by the three-option test.
• New:
  • stage3_b_executor_rejects_b3: rejected on the executor's capabilities, with the reason.
  • stage3_b_kept_panes_win_when_the_deployment_can_keep_them: 4b-like, 1k series sampled every 7.5 s, 1 h window every 10 min, raw data not kept, query_time_retention: true. B3 is priced, cheaper than B1 and B2, and selected.
  • kept_panes_build_the_newest_and_retain_the_others: a Stage 3 unit test of the price and the rejection.
  • kept_panes_need_one_new_pane_per_evaluation: a Stage 2 unit test.
  • test_kept_nodes_say_so_in_their_timing: a viewer test.
• Deleted as superseded by tumbling panes (Q63):
  • stage2_b_sliding_kll_has_two_materialization_options (Example 4): the three-option tumbling test covers it.
  • stage1_b_window_composition_adds_sliding_and_tumbling_per_option (Example 3): covered by stage1_b_window_composition_adds_tumbling_per_option.
  • stage1_b_merge_only_where_the_window_form_needs_it (Example 3): covered by stage1_b_tumbling_merges_five_panes_before_the_estimate.
  • The test adapter's WindowForm::Sliding variant, and the sliding arm of stage1_b_window_parameters_are_legal.
• Adapters: materialization reads the exported kept flag (QueryTimeKept). retention_ms gives N · w for a kept pane.
• Updated: Tests that took a non-empty materialization to mean "ingestion time" now say so. These are in Example 1, Example 3's stage3_b_tumbling_candidates_are_valid, and plan-selection. Example 1's runtime and capability checks treat kept plans as rejected on capability.

Test plan

• cargo fmt --all --check
• cargo clippy --workspace --all-targets -- -D warnings
• cargo test --workspace: 1684 passed, 0 failed, 16 ignored (re-run after the rebase on main d4869a7)
• python3 -m unittest test_viewer (tools/dag-viewer): 25 OK
• Example 1 fixture regenerated with --max-candidates 160. Its selection is unchanged.

🤖 Generated with Claude Code