Skip to content

feat(ci): add opt-in raw and corrected wrk histograms - #87

Merged
DPS0340 merged 2 commits into
devfrom
experiment/25-wrk-dual-histogram
Sep 14, 2026
Merged

feat(ci): add opt-in raw and corrected wrk histograms#87
DPS0340 merged 2 commits into
devfrom
experiment/25-wrk-dual-histogram

Conversation

@DPS0340

@DPS0340 DPS0340 commented Sep 13, 2026

Copy link
Copy Markdown
Collaborator

Summary

Related to #25; diagnostic tooling only, not a runtime fix or issue-closure claim.

Add an opt-in wrk4.2.0 source generator and sparse JSON histogram reporter. Existing wrk reports/Lua callbacks receive the histogram after stats_correct; this tool preserves both recorded raw bins and corrected bins in the same run. Reporting brackets correction after worker joins and after runtime is frozen. Normal CI benchmarks and CWIST source/defaults are unchanged.

  • Four files: generator, private wrk C header, offline tests, usage/limitations documentation.
  • Reject unknown source fingerprints. Accept exact upstream or the explicitly fingerprinted CLOCK_MONOTONIC variant used in this investigation.
  • Transform the same bytes that passed hashing; no source-path reopen. Input checkout is not modified by the generator; output must be new.
  • Existing scripts/ci/test_*.py discovery automatically runs the offline checks. No workflow permissions, dependencies or workload flags changed.

Verification

  • Initial missing-exporter tests RED; four final diagnostic tests GREEN.
  • Independent review found a fingerprint/read race. Deterministic concurrent-edit test failed before the buffer-retention fix and passes afterward. Follow-up independent exact-four-file review passes.
  • Entire offline CI discovery: 21 tests pass normally and with Python -O. Native compiler checks included.
  • Real Linux/aarch64 wrk build succeeds. Original source checkout and binary preserved; copied dependency objects/static libraries avoid a mutable shared build cache.
  • Final generator reproduces exactly the already-built C/header manifest, so later generator hardening does not invalidate binary-bound measurements.
  • Public TCP integration: 8 balanced samples, 84,021,040 measured requests, no reported wrk errors, both histogram kinds present in every warmup/measurement, every bin sum reconciled, and raw count equals completed requests in all measured samples. Actual backend/affinity and cleanup checked.

Experiment boundary

Same run-paired CPU-placement screen as documented in #25, pinned server 65632aed0a30e8aabe1519aa9b706e19685c2b4b (not latest dev), two server processes/four worker reactors each. Only wrk CPU affinity differed between shared and separated conditions. Source/build identities and raw logs retained.

Within-pair median separated/shared ratio: recorded raw nearest-rank P99.999 2.135, corrected nearest-rank P99.999 2.063, throughput 1.424. Raw-tail wins 0/4. Therefore the prior direction is not solely a histogram-correction artifact; this does not isolate the effect at equal offered load or prove a production SLO. Do not compare absolute latency values across experiment runs.

Raw means wrk-recorded observations, not an open-loop arrival distribution. Keep errors, gaps, pipeline settings, percentile rank conventions and post-run export overhead explicit. No cache activation, thread-default change, or benchmark result publication workflow change is proposed.

@DPS0340

DPS0340 commented Sep 13, 2026

Copy link
Copy Markdown
Collaborator Author

Head f4c0845 CI completed:8success/1skipped/1neutral, no failures. Independently read back the exact head.

The matched-rate follow-up is recorded at #25 (comment). At target300kRPS, matched within-pair achieved rates (<0.003% difference), separate client CPUs improved planned-send P99.999 by median31.2% across4/4pairs. This does not change this PR’s scope: opt-in instrumentation only, not runtime defaults or an issue-closure claim. Raw/uncorrected and planned-start/corrected metrics must remain explicitly labeled.

@DPS0340
DPS0340 marked this pull request as ready for review September 14, 2026 05:25
@DPS0340
DPS0340 merged commit 608b6fc into dev Sep 14, 2026
10 checks passed
@gg582
gg582 deleted the experiment/25-wrk-dual-histogram branch September 14, 2026 14:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant