Make upstream_response_body_filter async - #974
Open
torinnd wants to merge 2 commits into
Open
Conversation
Add a Criterion benchmark for the per-call cost of ProxyHttp::upstream_response_body_filter, covering both the default implementation and a trivial override. This provides a stable baseline for evaluating changes to a per-body-chunk hot path. Run with: cargo bench -p pingora-proxy --bench noop_body_filter
request_body_filter is async so implementors can await throttling and offloaded work without blocking a proxy worker. Give the upstream response body hook the same capability while preserving its existing body mutation, error, and pacing semantics. The signature otherwise stays the same. The Result<Option<Duration>> pacing mechanism is untouched, and the added CTX bound matches the trait's other async methods. The single dispatch site is already in an async function and now awaits the hook. Update the response-body-filter benchmark for the async signature. A focused test calls the real HttpProxy dispatch path and verifies that it awaits the hook, preserves body mutation and pacing, and propagates an error returned after an await.
torinnd
force-pushed
the
async-upstream-response-body-filter
branch
from
August 27, 2026 13:13
13a07cb to
db63a23
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #964.
This PR builds on #973, which adds a Criterion benchmark for
upstream_response_body_filter. Both commits are included so this branch is independently mergeable. If the benchmark PR lands first, the remaining diff is the async-filter commit. I opted to not attempt to wrestle with GitHub's stacked PR functionality, and hope this format is convenient for review and merging.ProxyHttp::upstream_response_body_filteris now async, matchingrequest_body_filterand the response header hooks. This allows implementations to await response-body work without blocking a proxy worker.The method otherwise keeps its existing contract:
Result<Option<Duration>>continues to control pacing;Self::CTX: Send + Syncbound matches the trait’s other async methods.There is one dispatch site, already in an async function, which now awaits the hook. Implementations that override the method need to add
async.The test calls the actual
HttpProxy::upstream_filterdispatch path. It verifies that the hook can await, that body mutation and pacing survive the await, and that an error returned after an await propagates.The Criterion benchmark changes from about 14 ns/call to about 21 ns/call on my machine. The broader h1/h2 measurements and caveats are in #964; I could not resolve an end-to-end difference above the noise on that setup.