Add aggregate-internal submetrics - #25051

Draft

kosiew wants to merge 37 commits into

apache:mainfrom

kosiew:aggmetrics-06-23570

Commits on Sep 9, 2026

feat: add data‑free SQL harness for array_agg_distinct benchmark
kosiew
committed
68c6fd8
chore: adjust benchmark scope from 1M to 100K groups
kosiew
committed
713afe2

Commits on Sep 11, 2026

bench: scale array_agg distinct groups
kosiew
committed
e015a1a
empty commit
kosiew
committed
93f2dae
feat: add optional AggregateMetric API with lazy internal timers and comprehensive tests
kosiew
committed
06bbae4
feat(metrics): safe refactors to avoid redundant allocations and improve performance
kosiew
committed
238ab26
fix: assert both internal_distinct timers are >0 in test
kosiew
committed
0338b63
docs: update metrics.md with identity/cardinality, accumulator timer, partition display, and empty input behavior
kosiew
committed
1a27e68
fix(DistinctArrayAggAccumulator): correct `size()` calculation
kosiew
committed
e3fce28
feat(aggregates): add 2‑partition execution test and fix array_agg distinct identities and timer handling
kosiew
committed
12ec1e1
feat(metrics): add lock‑free OnceLock fast path for AggregateSubMetrics
kosiew
committed
3f9c375
fix: restore Time::add min‑1ns behavior and remove exact‑duration API
kosiew
committed
7444e9a
fix(time): corrected split to preserve exact duration adds and avoid per‑batch 1 ns inflation
kosiew
committed
139b807
feat(metrics): require RefUnwindSafe for AggregateMetric and add compile‑time test for DistinctArrayAggAccumulator
kosiew
committed
16d93d9
test: add legacy grouped `array_agg(DISTINCT)` metric test
kosiew
committed
7e1ac88
fix(array_agg): skip internal DISTINCT timing for small batches and improve per-group metrics
kosiew
committed
31cfdc7
feat(datafusion): add grouped update metric and update_batch_grouped for accumulators
kosiew
committed
5947693
fix(groups_accumulator): improve Instant usage and fix unnecessary Option clone
kosiew
committed
030cf81
refactor: simplify distinct_metric cloning in DistinctArrayAggAccumulator
kosiew
committed
1e91ca3
feat(metrics): add array_agg(DISTINCT) merge, convert_to_state, and metric-count regressions
kosiew
committed
c797453
feat: cache aggregate metric handles for grouped accumulators
kosiew
committed
9d82f56
feat(groups-accumulator-adapter): move slice/filter prep outside timing, bound prep chunks, time only accumulator calls, and emit one metric update per logical batch
kosiew
committed
30f0ce4
feat: convert_to_state uses bounded 64‑row prep chunks with timer‑wrapped batch updates and adds regression test
kosiew
committed
620e660
refactor: simplify metric lookup and consolidate implementation blocks, import AggregateMetrics
kosiew
committed
68945a9
empty commit2-before merge main
kosiew
committed
fd8c4f9
Merge branch 'main' into aggmetrics-06-23570
kosiew
committed
c7fd9f6
fix(metrics): create execution-owned aggregate_sub_metrics helper and propagate to accumulators
kosiew
committed
eab374e
docs: clarify metrics documentation about internal submetrics
kosiew
committed
2c2bae2
perf(aggregate): move size accounting out of timer and improve error handling in GroupsAccumulatorAdapter
kosiew
committed
e6e4a85
test: add regression test for groups_accumulator
kosiew
committed
e7d2d7e
Merge branch 'main' into aggmetrics-06-23570
kosiew
committed
3b9fed4
fix: clear successful groups before propagating later-group error and add retry regression
kosiew
committed
2b8c4cf

Commits on Sep 12, 2026

fix: update array_agg_distinct benchmark group size from 100K to 1000K
kosiew
committed
3e08f38
feat: implement fast path for uninstrumented aggregates
kosiew
committed
b60600b
feat: add benchmarks for GroupsAccumulatorAdapter
kosiew
committed
714ec75

Commits on Sep 14, 2026

fix: adjust comment formatting in groups_accumulator_adapter benchmark
kosiew
committed
6a5c245
Merge branch 'main' into aggmetrics-06-23570
kosiew
committed
2171d62