Skip to content

Observe points without per-point allocation - #34

Merged
zzylol merged 2 commits into
mainfrom
perf/observe-without-allocation
Oct 20, 2026
Merged

zzylol merged 2 commits into
mainfrom
perf/observe-without-allocation

Conversation

@zzylol

@zzylol zzylol commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

The per-point cost of the sketch branch. Base: main (the experiment-results PR below it was taken out of the stack to be rerun).

Why

On the live top-k stack the sketch branch used more CPU than the exact and raw branches (0.16 vs 0.10 and 0.01 cores at 20,000 points/s per source). A per-step timing of the branch's path showed that the Count-Min + heap update was only 0.24 of 4.1 µs per point. The rest went on building owned copies of every point:

  • Decoding OTAP into observations allocated the metric name and every label as new strings.
  • The per-attribute split made another copy of each point (metric, resource labels, value bytes).
  • observe cloned the whole config (including a HashMap<String, f64>), took three locks, and built a new series-key string per point.

What changes

  • codec::decode_each walks the OTAP view and rewrites one reused Observation per data point, passing it to a callback. decode_pdata_to_observations is now built on it, with the same output.
  • matchers::per_attribute writes each split into a reused Observation, again through a callback. write_series_key fills a reused buffer and skips sorting labels that are already sorted (same bytes as before).
  • PrecomputeImpl shares its config through an Arc and keeps window and stats under one lock.
  • asap_sketches and exact_branch both use decode_each, so the sketch and exact pipelines still decode the same way.

Behaviour changes on bad input:

  • A decode error part-way through a batch (an unknown sketch.type or a bad envelope payload) now keeps the points observed before it; previously the whole batch was dropped. The batch's remaining points are dropped and the error is logged.
  • An observe error is logged once per batch instead of once per point.

Before → after

Per point, on one pinned core of a 56-core x86 server (400,000 top-k points: 100,000 Zipf routes, Count-Min 4 × 512 + top-10 heap; three identical runs). The before column comes from the same harness ported to the parent commit, which builds owned observations instead of calling decode_each. In these runs the timed split also cloned every key for the CMS-only timing, which moved about 0.05 µs from observe to the split; the harness now collects those keys in an untimed pass. The total is unaffected:

Step Before After
Decode OTAP into observations 2.02 µs 1.09 µs
Per-attribute split 1.14 µs 0.33 µs
observe, including Count-Min + heap 0.98 µs 0.29 µs
Total 4.14 µs 1.72 µs

On the live top-k stack (2 × 20,000 points/s, 2-minute average per stage), CPU cores per branch:

Before After
sketch branch-a / branch-b 0.156 / 0.160 0.074 / 0.076
exact branch-a / branch-b 0.093 / 0.098 0.063 / 0.066

The sketch branch now costs about 0.01 cores more than the exact branch, roughly the Count-Min + heap update itself. Window completion delay is unchanged (sketch 5.8 s, exact 5.2 s).

Verification

  • cargo test --release --features otap-engine branch_phase_costs -- --ignored --nocapture prints the table above. It is an ignored test in otap/processor.rs.
  • New unit tests: the sorted fast path of the series key gives the same bytes as the sorted walk, and per_attribute with a reused buffer gives the same observations as before.
  • The full acceptance passes 29/29 on the same server with this change, with the same answers (recall@10 1.00, the same classes, drift and novelty events).
  • cargo fmt --check, cargo clippy --all-targets --features otap-engine,stream-benchmark -- -D warnings and cargo test --features otap-engine,stream-benchmark (the CI commands) pass at this PR's head.
  • Review follow-ups in this PR: the split timing no longer collects keys inside the timed run, a new series is inserted with the entry API (one key clone instead of two), and the series-key writers are crate-private.

@zzylol
zzylol force-pushed the feat/experiment-results branch from 016efad to 82a6d7e Compare October 19, 2026 17:33
@zzylol
zzylol force-pushed the perf/observe-without-allocation branch 2 times, most recently from ab6f04b to 74a8594 Compare October 19, 2026 20:15
@zzylol
zzylol force-pushed the feat/experiment-results branch 2 times, most recently from 26ac19d to 123d82f Compare October 19, 2026 20:16
@zzylol
zzylol force-pushed the perf/observe-without-allocation branch from 74a8594 to f3d2ad5 Compare October 19, 2026 20:16
@zzylol
zzylol force-pushed the feat/experiment-results branch from 123d82f to af506b2 Compare October 19, 2026 20:16
@zzylol
zzylol force-pushed the perf/observe-without-allocation branch 2 times, most recently from 971825f to 11092ab Compare October 19, 2026 20:16
@zzylol
zzylol force-pushed the feat/experiment-results branch from af506b2 to 034fe62 Compare October 19, 2026 20:16
@zzylol
zzylol force-pushed the perf/observe-without-allocation branch from 11092ab to 21f5802 Compare October 19, 2026 20:17
@zzylol
zzylol force-pushed the feat/experiment-results branch 2 times, most recently from 21f5da0 to f8e6762 Compare October 19, 2026 20:17
@zzylol
zzylol force-pushed the perf/observe-without-allocation branch 2 times, most recently from 05ea8f8 to bfbdb55 Compare October 19, 2026 20:18
@zzylol
zzylol force-pushed the feat/experiment-results branch 2 times, most recently from 90f57dd to f91377d Compare October 19, 2026 20:18
@zzylol
zzylol force-pushed the perf/observe-without-allocation branch 2 times, most recently from 541925f to 7da0bc7 Compare October 19, 2026 20:18
@zzylol
zzylol force-pushed the feat/experiment-results branch 2 times, most recently from 3a78cd7 to 402cbc3 Compare October 19, 2026 20:19
@zzylol
zzylol force-pushed the perf/observe-without-allocation branch 2 times, most recently from c03faab to 26e7bb2 Compare October 19, 2026 20:19
@zzylol
zzylol force-pushed the feat/experiment-results branch from 402cbc3 to 0e430b8 Compare October 19, 2026 20:19
The sketch branch spent 4.1 us per point before the sketch saw it:
decoding allocated every label as new strings, the per-attribute split
copied each point again, and observe cloned the whole config, took
three locks and built a fresh series-key string.

- codec: decode_each walks the OTAP view and rewrites one observation
  per point; decode_pdata_to_observations is built on it.
- matchers: per_attribute writes each split into a reused observation;
  write_series_key fills a reused buffer and skips sorting labels that
  are already sorted.
- precompute: the config is shared through an Arc; window and stats
  sit under one lock.
- The sketch and exact branches both use decode_each.

Per point on a pinned core (top-k, 100k Zipf routes, CMS 4x512):
decode 2.02 -> 1.09 us, split 1.14 -> 0.33 us, observe 0.98 -> 0.29
us. Live top-k stack: sketch branch 0.158 -> 0.075 cores, exact branch
0.095 -> 0.064 cores.
…es once

The split timing no longer clones every key into a vector inside the timed
run; the keys for the CMS-only timing are collected in a separate untimed
pass. A new series is inserted with the entry API instead of insert then
get_mut, and the series-key writers are crate-private.
@zzylol
zzylol changed the base branch from feat/experiment-results to main October 20, 2026 23:23
@zzylol
zzylol force-pushed the perf/observe-without-allocation branch from 26e7bb2 to 30cc1fb Compare October 20, 2026 23:24
@zzylol zzylol added the enhancement New feature or request label Oct 20, 2026
@zzylol
zzylol merged commit 0b610b9 into main Oct 20, 2026
2 checks passed
@zzylol
zzylol deleted the perf/observe-without-allocation branch October 21, 2026 00:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant