fix(app): preserve inherited CPU affinity when pinning workers - #89
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Keep Linux multiprocess worker pinning inside the affinity mask inherited at startup. Related to #25; this does not close the HTTP P99.999 investigation.
The current code counts allowed CPUs and then uses
worker_index % countas a CPU ID. These are not equivalent for nonzero or sparse masks. On a four-CPU Linux VM:taskset -c 1,3: current workers run on CPUs0,1.taskset -c 2,3: current workers run on CPUs0,1.taskset -c 3: both current workers run on CPU0.A taskset mask is not a cgroup cpuset boundary: a later successful
sched_setaffinitycan widen it within the enclosing cpuset. The old unchecked call can therefore defeat operator CPU placement, not merely fail.Changes
make test, and explicit header dependencies for affected builds.Validation
{1,3}/ worker index 0.make test_worker_affinity, normal, NDEBUG, and ASan+UBSan on Linux ARM64. Seven placement cases, three failure paths, and real fork-child affinity isolation. Checks remain active under NDEBUG.0,1,1,3,2,3, and3. All 256 HTTP 200 responses have the expected body. Per-task identities/masks remain stable; owned process groups are removed and port 9091 is closed after each session.The real baseline server is pinned at
65632aed0a30e8aabe1519aa9b706e19685c2b4b; its app startup code is unchanged in this PR's basefbf9638edb5acd4105e903fe517fd7ac7736e4b3. The candidate used an isolated copy of its library with all four changed translation units rebuilt (app plus latest-dev http/reactor/session_manager); source hashes and archive members were checked. The only intervening public-header delta is documentation. This is not a fresh full-tree build.Limits / pending
9114bed276aaad767500c6a71d7b75e149afee71: 8 checks succeeded, 1 report-publish job was skipped, and the CodeQL summary is neutral. ASan/UBSan, CodeQL analysis, HTTP/2 conformance, macOS kqueue, and benchmark jobs succeeded. The ASan/UBSan log includes the new affinity test.cpu_set_tis retained. If the kernel mask does not fit, lookup fails safely with a diagnostic; dynamicCPU_ALLOCsupport is outside this patch.CI follow-up
The first head failed the raw-allocator gate because app.c line numbers shifted. Commit
9114bed276aaad767500c6a71d7b75e149afee71updates only those eight baseline locations; exact call text and entry count are unchanged. Local allocator lint, test-wiring check, and all 17 scripts/ci Python tests pass. The fresh-head CI checks listed above have now completed without failures. This does not establish a latency improvement or approve merge.CPU-placement screen
Full results. Eight runs used the same candidate server and elapsed-clock wrk2 tool at R300000, with client CPUs0-1 fixed. Only the server mask changed from0-1 to2-3. The run order was AB BA BA AB. No BPF or perf capture ran.
This supports the placement experiment, not a code-change speedup, a general SLO, or closure of #25. Old invalid clock datasets remain invalid. Worker and thread defaults remain unchanged.
Source A/B under identical requested masks
PR #89 now has a source A/B result under identical requested CPU masks.
Base
fbf9638edb5acd4105e903fe517fd7ac7736e4b3vs candidate9114bed276aaad767500c6a71d7b75e149afee71. Both requested server CPUs 2-3 and client CPUs 0-1. Base workers actually ran on 0-1; candidate workers respected 2-3.Both app objects were rebuilt and both binaries relinked with identical normalized arguments and common dependencies. Only app.o differs among 77 archive members. This is controlled reuse, not two full clean builds. Both Linux binaries used the same host-backed filesystem.
Eight 40-second runs, AB BA BA AB, R300000/t12/c400/-U with the same elapsed-clock wrk2. No BPF/perf.
Parent raw/source/identity-to-mask/eventfd checks passed. Postflight verified 113 files, no owned survivors, and a closed port. Independent bounded static review found no blocker.
This supports the affinity correctness fix and the planned-arrival result in this configured workload, NOT a universal server-latency improvement. Default CPU placement, the original 64/256/512 matrix, and overall issue #25 remain unresolved. No merge or issue closure is requested by this result.