http: frame client requests that carry a payload - #65512
Conversation
A client request whose method defaults to no chunked encoding (GET, HEAD, DELETE, OPTIONS, TRACE) skipped both Content-Length and Transfer-Encoding even when a payload was written. The payload was still sent, so the peer read those bytes as the start of the next request, which allows request smuggling. Frame such requests like any other: use Content-Length when the length is known and a chunked Transfer-Encoding when it is not. Server responses and requests with no payload keep the previous behaviour, and CONNECT is left untouched because its payload is tunnelled data that must not be re-framed. Headers can be rendered before anything is written, for example when the headers are passed as an array or an Expect header is set. Track whether a payload write actually happened so that case is not mistaken for a body of unknown length. The two known_issues tests covering this now pass and move to parallel. They also asserted a response body for HEAD, which cannot have one, and compared buffers with strictEqual, so both are corrected in the move. Fixes: nodejs#27880 Refs: nodejs#34066 Refs: nodejs#60718 Signed-off-by: Avocado <ujubongbong@gmail.com>
|
Review requested:
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #65512 +/- ##
==========================================
- Coverage 90.16% 90.15% -0.02%
==========================================
Files 751 751
Lines 253465 253522 +57
Branches 47741 47756 +15
==========================================
+ Hits 228537 228560 +23
- Misses 16197 16212 +15
- Partials 8731 8750 +19
🚀 New features to boost your workflow:
|
A client request that writes a payload is now framed with a chunked Transfer-Encoding. When the body source is destroyed before EOF the request ends without its terminating chunk, so the peer resets the connection instead of seeing a clean close, and the echoed response fails with ECONNRESET rather than ERR_STREAM_PREMATURE_CLOSE. Signed-off-by: Avocado <ujubongbong@gmail.com>
|
CI is red on What happens The test opens a client request, pipes a body into it, and destroys the source before EOF. It then asserts on the error the server side sees: pipeline(req, res, common.mustCall((err) => {
assert.strictEqual(err?.code, 'ERR_STREAM_PREMATURE_CLOSE');
}));Before this PR the request carried neither Why I think the new code is correct Once a request is declared chunked, aborting it mid-body is a transport-level failure, so I ran the full |
A client request whose method defaults to no chunked encoding (
GET,HEAD,DELETE,OPTIONS,TRACE) skipped bothContent-LengthandTransfer-Encodingeven when a payload was written. The payload was still written to the socket, so the peer reads those bytes as the start of the next request.On the wire this produces a message with no framing headers followed by the payload, which the peer parses as a second request.
_storeHeader()skipped header generation wheneveruseChunkedEncodingByDefaultwas false, regardless of whether a payload was actually written.Such requests are now framed like any other:
Content-Lengthwhen the length is known, a chunkedTransfer-Encodingwhen it is not.Three cases deliberately keep the previous behaviour:
method;CONNECT, whose payload is tunnelled data that must not be re-framed;The last one needs more than
_contentLength. Headers can be rendered before anything is written, for example when headers are passed as an array or anExpectheader is set, and at that point the length isnulljust as it is for awrite()of unknown length. AkPayloadPendingflag records whether a payload write actually reached header rendering, so the two are no longer conflated.Tests
test-http-clientrequest-end-contentlengthandtest-http-clientrequest-write-chunkednow pass and move fromknown_issuestoparallel. Both needed fixes unrelated to this change to be usable there: they asserted a response body forHEAD, which cannot carry one, and compared buffers withstrictEqual.test-http-request-method-delete-payloadasserted the old behaviour and is updated to assert the payload arrives as a body.test-http-clientrequest-end-empty-response-bodystays inknown_issues. It covers theContent-Length: 0thatend()adds for methods that do default to chunked encoding, which comes from a different code path and is unchanged here.test/parallel/test-http-*andtest/known_issuespass locally.Fixes: #27880
Refs: #34066
Refs: #60718