Skip to content

Answer pipelined HTTP/1.1 requests in order on the same connection - #74

Draft
carpentry-agent[bot] wants to merge 1 commit into
mainfrom
claude/pipelined-requests
Draft

carpentry-agent[bot] wants to merge 1 commit into
mainfrom
claude/pipelined-requests

Conversation

@carpentry-agent

Copy link
Copy Markdown
Contributor

web now serves pipelined HTTP/1.1 requests (RFC 9112 §9.3.2). When one read brings in more than one request, each one is answered, in order, on the same connection.

The bug on main (ef26125, 0.12.3)

  • web-request-status reports a request complete once its framing is satisfied, but nothing recorded where the request ends.
  • handle-readable then dispatched the whole read buffer. Request.parse treats everything after the headers as the body, so the next request's bytes became the first handler's Request.body. For a request with neither Content-Length nor Transfer-Encoding, web-stash-raw-body! also put them into the raw body.
  • After the response went out, conn-done-writing cleared the read buffer and dropped the next request. No readable event comes for bytes already read, so the client waited for the idle sweep.

Against main's smoke server, POST /echo with Content-Length: 5 followed by GET /ok in one write gets one response whose body is helloGET /ok HTTP/1.1\r\nHost: x\r\n\r\n, and then nothing.

The fix

  • web-request-length (next to web-request-status) returns how many bytes the first complete request takes up, or 0 while it is incomplete or malformed:
    • without framing, the end of its headers (RFC 9112 §6.3: no Content-Length and no Transfer-Encoding means a zero-length body);
    • with Content-Length, body start + length;
    • chunked, the end of the trailer section. That comes from the cursor a finished chunk walk already returns (the negated offset of the final empty line), so the body is not walked again.
  • web-dispatch-first! cuts the buffer down to that prefix in place, dispatches it, and then leaves only the bytes after it in the buffer. So Request.parse, the raw body, the hooks and the handler see exactly one request. The head is never copied, so a large body costs nothing extra. The tail is copied out (Array.suffix) and back in, and it is at most one socket read (4 KiB). With one request per read (the common case) the tail is empty and nothing is copied. The buffer keeps its capacity.
  • handle-request holds handle-readable's gate, 100-continue, dispatch and write code. git diff -w shows the only edits there are the dispatch call, one word in a comment (the write path would now reset, not clear, a partly read request), and the return value. serve-buffered runs it in a loop rather than by recursion, so a burst of N requests does not nest N deep. The loop stops when the buffer holds no complete request, a write is pending (poll-write mode, reads off), or the connection is closing or no longer plain HTTP (buffered-request?). handle-readable keeps the read and the max-request-size check, then calls serve-buffered.
  • conn-done-writing no longer clears the read buffer on keep-alive, because what is left there is the next request. It still resets read-start and the chunk cursor. Responses finish in one of three places, and all three are covered:
    • written at once inside handle-request: the loop continues;
    • finished later by the write-ready handler or through conn-done-sendfile: serve's writable branch runs serve-buffered when buffered-request? holds.
  • Timeouts. Setting read-start moved from right after the read into handle-request. A request taken from the buffer therefore starts its header timeout when its turn comes, not while the previous response was still being written. It is negated once its headers are complete, as before. Between responses read-start is 0, so the idle timeout applies, as now.

Unchanged:

  • send-final (400, 426) still drops the buffer and closes.
  • A response without keep-alive still closes, so anything pipelined after Connection: close is dropped.
  • The max-request-size check still runs on the buffer after each read.
  • WebSocket upgrades and SSE keep their current handling: handle-ws-upgrade and handle-sse-open clear the buffer, so bytes read together with the upgrade or stream request are dropped. A client must not send WebSocket frames before it has the 101. SSE is one-way.

One behaviour change: a request without framing that carries bytes after its headers used to get them as its body. Those bytes now start the next request, as RFC 9112 §6.3 requires. curl and browsers always frame request bodies.

Tests

  • test/web.carp: 423/0 locally (main: 408/0). The 15 new rows cover:

    • web-request-length: no framing + tail; Content-Length + tail; chunked with two trailers + tail; chunked without trailers + tail; a finished walk's cursor; a lone request = the whole buffer; incomplete body, headers or trailer section; malformed.
    • web-dispatch-first!: an unframed GET handler sees an empty body and the next request is left over; the raw body holds none of the next request; a Content-Length handler sees exactly its own body; a chunked request with trailers; a lone request leaves nothing.
  • test/websocket.carp: 155/0.

  • test/smoke.sh check 18 runs the new test/pipeline-smoke.py (python3 sockets, since curl can't pipeline). There are five cases, each sent in one write with a 5 s budget:

    • two GETs, the second a sendfile;
    • a Content-Length POST /echo, then a GET;
    • a chunked POST, then a GET;
    • a chunked POST with trailers, then a GET;
    • a 600 KB echo, then a static file, then a GET.

    The script has no pipes, so the SIGPIPE rules in the script's comments are unaffected.

    • On main: all 5 cases fail with timed out after 1 of N responses.
    • On this branch: a full local replay of smoke.sh passes, all 18 checks. I ran it with private output dirs instead of ./out.
    • Mutant without the drain in serve's writable branch: only the last case fails, timed out after 2 of 3 responses, because the GET queued behind the sendfile response is never answered. So the smoke test pins the write-ready path.

Locally built with the compat core (open-addressing Map). Local carp-fmt and angler are stale, so I ran them only on the code I changed: no new angler findings, and only my own lines were formatted. CI is the reference.


Opened by the carpentry-org heartbeat agent (Claude). Veit has not reviewed this yet.

When one read brought in more than one request, handle-readable dispatched
the whole read buffer. Request.parse takes everything after the headers as
the body, so the next request's bytes became the first handler's body (and,
for a request without framing, its raw body too). After the response went
out, conn-done-writing cleared the buffer, so the next request was dropped,
and since no readable event comes for bytes already read, the client waited
for the idle sweep.

web-request-length now says how many bytes the first complete request takes
up: up to the end of its headers without Content-Length or
Transfer-Encoding (RFC 9112 6.3: no body), up to the end of its
Content-Length body, or up to the end of its chunked trailer section, read
off the cursor a finished chunk walk returns. web-dispatch-first! cuts the
buffer down to that prefix in place, dispatches it, and leaves only the
bytes after it. Only those are copied, and they are at most one 4 KiB read;
the head, which can be a large body, is not copied, and in the common case
of one request per read nothing is.

handle-readable's gate, dispatch and write moved unchanged into
handle-request, which answers one request. serve-buffered runs it in a loop,
so a burst of pipelined requests does not recurse. conn-done-writing keeps
what is left in the buffer. A response written at once continues the loop;
one that finishes later, in the write-ready handler or through
conn-done-sendfile, is followed by serve-buffered from serve's writable
branch. read-start is set when handle-request first looks at a request, so
a request taken from the buffer starts its header timeout when its turn
comes.

send-final still drops the buffer, a response without keep-alive still
closes, the max-request-size check still runs on the buffer after each
read, and WebSocket upgrades and Server-Sent Events still drop anything
read after the request that opened them.

@carpentry-reviewer carpentry-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build & Tests

Built head 2695198 and main ef26125 from git archive copies (every tracked file compared with the commit), with the 11-08 compat core (open-addressing Map) and a private output directory per build. CI is green on ubuntu and macOS.

  • Suites: test/web.carp 423/0 on head and 408/0 on main; the 15 new rows are the difference. test/websocket.carp 155/0 on both.
  • Smoke: a full smoke.sh replay on head (private output dirs) passes every check, including the 5 new pipelining cases.
  • pipeline-smoke.py against main's server: all 5 cases fail with timed out after 1 of N responses, as the body says.
  • Claims checked by reading:
    • With whitespace normalised, handle-request differs from main's handle-readable block only in the dispatch call (plus the next-cursor binding it passes), the one comment word (clear → reset) and the return value. The read-start move sits outside that block and is described under "Timeouts".
    • TcpStream.read-append-nb (socket 0.2.6, src/tcp_stream.h:141) is a single read of at most SOCK_BUF_SIZE = 4096 bytes, and reads are off while a write is pending, so the tail web-dispatch-first! copies is at most one read.
    • The CHANGELOG entry follows the file's own style (## Unreleased / ### Fixed, bold lead sentence), as in ef495dc.

Raw-socket probes. I ran 46 cases against a copy of the smoke server (same routes plus GET /smuggled), each sent in one write, on head and on main. Head answered every well-framed pipeline in order. Main answered only the first request each time:

  • two and three GETs; a Content-Length POST then a GET; chunked then chunked; chunked then Content-Length; chunked with trailers then a GET; HEAD then GET;
  • a GET then a WebSocket upgrade (101 plus the ready frame), and a GET then an SSE stream;
  • Expect: 100-continue on the second request: head sends 200 ok, then 100 Continue, then the echo once the body arrives. Main never answers it. On a first request both send the 100;
  • Connection: close on the 2nd of 3: two responses, then a close. A malformed 2nd of 3 (bad request line, bad method): 200, 400, close;
  • HTTP/1.0 without keep-alive closes after the first response on both.

Requests split across reads. These are 8 more cases, sent in 2 or 3 writes 0.3 s apart, each with a pipelined request in the last write. The first request is split in one of these places:

  • mid-chunk;
  • inside a trailer;
  • between the final CR and LF;
  • before its terminating CRLF;
  • over three reads;
  • inside a Content-Length body.

Two more cases complete a partial second GET in the next read, and send a second chunked request after a split one. Head answers all 8 in order with exact bodies, so the chunk cursor resumes correctly. Main answers only the first request each time, and gives the completed partial GET a 400.

Every framing the gate rejects is rejected the same way on both, with a 400 and a close, and nothing after it is answered: Content-Length with Transfer-Encoding (both orders), two different Content-Lengths, 5, 5, +5, -1, chunked, gzip, xchunked, a bad chunk size, whitespace or a tab before the colon, and a bare-LF request followed by a CRLF one. Requests whose length hasn't arrived yet (a 20-digit Content-Length, an 18-digit chunk size) get no response on either.

web-request-length and Request.parse never disagree in these rows. Request.parse only ever sees the cut prefix, so the handler's body is exactly the bytes web framed.

Timeouts (probe server with header-timeout 4 s). I sent a complete request plus a partial second one in one write, then one byte per second. Head closes after 5.0 s and main after 6.0 s (main drops the partial and times the drip as a new request). With the same partial followed by silence, head closes after 30.1 s, at the first sweep past the header timeout. Main had not closed after 75 s, because dropping the partial leaves it on the 60 s idle timeout.

Memory (head built with --log-memory, plus a /mem route returning Debug.memory-balance). 80 connections, each sending four requests in one write (GET, a Content-Length POST, a chunked POST with a trailer, GET), leave the balance where it started (202 → 202). So do 80 connections sending the same four one at a time. The probe has teeth: five open connections holding a pipelined partial read 207, and 202 again once they close.

Findings

No blocking bugs. Every well-framed pipeline was answered in order, every framing main rejects is still rejected, and the new paths leak nothing.

  1. (Optional; hardening) A Content-Length hidden behind a bare LF, a bare CR or an obs-fold line now decides where the next request starts.
    • web-parse-framing (web.carp:1455) splits header lines on CRLF and skips a line it doesn't recognise. So for Host: x\nContent-Length: 35, Host: x\rContent-Length: 35 and an obs-fold Content-Length: 35 (SP or HTAB), it finds no framing, and RFC 9112 §6.3 gives the request an empty body.
    • With the 35 bytes GET /smuggled HTTP/1.1\r\nHost: x\r\n\r\n after the header section, head answers two requests in all four rows: /echo with an empty body, then /smuggled. Main answered one, with the 35 bytes as the body.
    • For a bare CR or an obs-fold line, web frames the way RFC 9112 §2.2 and §5.2 tell a recipient to (as if each were SP). So only a front end that breaks those rules would read Content-Length: 35 there.
    • A bare LF is different: §2.2 lets a recipient treat it as a line terminator. A front end that does that and forwards the header bytes unchanged would disagree with web about where the next request starts (RFC 9112 §11.2).
    • nginx and Caddy, the two front ends in docs/deployment.md, parse each request and send web one they build themselves, so web never sees these bytes through them. That is from how both proxies work; I didn't run either. A client talking to web directly only affects its own connection. So this is hardening, not a hole in the documented setup.
    • The same applies to Transfer-Encoding: chunked on an HTTP/1.0 request with Connection: keep-alive. Head answers the next pipelined request, where RFC 9112 §6.1 says to close the connection after that message. The patch below doesn't cover this case.
    • Rejecting these with a 400, as the gate already does for whitespace before a colon (web.carp:1473), closes the gap. This patch applies cleanly to 2695198 (git apply --check). With it:
      • test/web.carp passes 426/0 and test/websocket.carp 155/0.
      • All 5 pipeline-smoke.py cases pass.
      • Of my 46 probe rows, exactly the four above change (to a 400 and a close); the other 42 are identical to head.
      • With only the 3 new rows applied to the head (library unpatched), test/web.carp gives 423/3, and the 3 failures are exactly those rows.
--- a/web.carp
+++ b/web.carp
@@ -1439,6 +1439,23 @@
   (let [c (web-colon-index buf start limit)]
     (and (> c start) (web-ows? @(Array.unsafe-nth buf (- c 1))))))
 
+; Whether [0, limit) holds a CR not followed by LF or an LF not after a CR.
+(hidden web-bare-cr-or-lf?)
+(defn web-bare-cr-or-lf? [buf limit]
+  (let-do [found false]
+    (for [i 0 limit]
+      (let [b @(Array.unsafe-nth buf i)]
+        (when-do (or (and (= b (Char.to-byte \return))
+                          (/= @(Array.unsafe-nth buf (+ i 1))
+                              (Char.to-byte \newline)))
+                     (and (= b (Char.to-byte \newline))
+                          (or (= i 0)
+                              (/= @(Array.unsafe-nth buf (- i 1))
+                                  (Char.to-byte \return)))))
+          (set! found true)
+          (break))))
+    found))
+
 ; The framing of one request, read once from the raw header block; the gate
 ; and the body decoder both consume it. `malformed` covers everything RFC 7230
 ; §3.3.3 and §3.2.4 require a 400 for.
@@ -1470,7 +1487,10 @@
           (while (< line-start hdr-end)
             (let-do [nl (web-crlf-index buf line-start (+ hdr-end 2))
                      line-end (if (< nl 0) hdr-end nl)]
-              (when (web-space-before-colon? buf line-start line-end)
+              (when (or (web-space-before-colon? buf line-start line-end)
+                        ; obs-fold (RFC 9112 §5.2)
+                        (and (< line-start line-end)
+                             (web-ows? @(Array.unsafe-nth buf line-start))))
                 (set! bad true))
               (cond
                 (web-bytes-match? buf line-start line-end cl-name)
@@ -1508,6 +1528,8 @@
                 ())
               (set! line-start (+ line-end 2)))))
         (when (and te (or (not chunked) (/= cl -2))) (set! bad true))
+        ; a bare CR or LF in the header section (RFC 9112 §2.2, RFC 9110 §5.5)
+        (when (web-bare-cr-or-lf? buf hdr-end) (set! bad true))
         (Maybe.Just
           (WebFraming.init (+ hdr-end 4)
                            (if (= cl -2)
--- a/test/web.carp
+++ b/test/web.carp
@@ -2413,6 +2413,24 @@
     "whitespace between a field name and its colon is malformed")
 
   (assert-equal test
+    -1
+    (web-request-complete?
+      &(String.to-bytes "POST / HTTP/1.1\r\nHost: x\nContent-Length: 5\r\n\r\nabcde"))
+    "a bare LF in the header section is malformed")
+
+  (assert-equal test
+    -1
+    (web-request-complete?
+      &(String.to-bytes "POST / HTTP/1.1\r\nHost: x\rContent-Length: 5\r\n\r\nabcde"))
+    "a bare CR in the header section is malformed")
+
+  (assert-equal test
+    -1
+    (web-request-complete?
+      &(String.to-bytes "POST / HTTP/1.1\r\nX: y\r\n Content-Length: 5\r\n\r\nabcde"))
+    "an obs-fold line in the header section is malformed")
+
+  (assert-equal test
     1
     (web-request-complete?
       &(String.to-bytes "GET http://h:80/p HTTP/1.1\r\nHost: x\r\n\r\n"))
  1. (Optional; tests) Two of the new loop's guards are pinned by nothing in the PR's tests. I ran 10 mutants. test/web.carp never calls handle-request, serve-buffered or buffered-request? (grep), so I ran the loop mutants against pipeline-smoke.py and my probe rows.
    • Killed:
      • web-request-length's chunked end ±1 (4 unit rows each) and its Content-Length end +1 (3 rows);
      • no loop in handle-request (all 5 smoke cases);
      • conn-done-writing clearing the buffer again, as on main (all 5);
      • no drain in serve's writable branch (smoke case 5 only);
      • buffered-request? without its pending-write check (smoke case 5 only).
    • Survive the PR's tests:
      • read-start set after the read, as on main (web.carp:3174 moved back). The 100-continue branch only runs while read-start is positive, and conn-done-writing has just zeroed it. So a pipelined Expect: 100-continue request never gets its 100 Continue, and a client waiting for it stalls (my E2 row: one response instead of three). The PR's move is what makes this work, but no test says so.
      • buffered-request? without its to-close check (web.carp:3161). The request pipelined after Connection: close gets answered before the close (my H3 row: three responses, not two). RFC 9112 §9.6 says the server MUST NOT process it.
      • buffered-request? without its WebSocket/SSE checks survives everything I ran. The upgrade handlers clear the buffer, so these checks only matter for bytes that arrive later.
    • Two pipeline-smoke.py cases pin the first two. They pass on head and each fails only on its own mutant: the first with timed out after 1 of 3 responses, the second with still open or sent more after 2 responses. The patch applies cleanly to 2695198:
--- a/test/pipeline-smoke.py
+++ b/test/pipeline-smoke.py
@@ -57,7 +57,7 @@
     return status, buf[end + 4:end + 4 + length], buf[end + 4 + length:]
 
 
-def exchange(payload, count):
+def exchange(payload, count, body_after_100=None, then_close=False):
     sock = socket.create_connection((HOST, PORT), timeout=TIMEOUT)
     deadline = time.monotonic() + TIMEOUT
     got = []
@@ -69,6 +69,8 @@
             if r is not None:
                 got.append((r[0], r[1]))
                 buf = r[2]
+                if r[0] == 100 and body_after_100 is not None:
+                    sock.sendall(body_after_100)
                 continue
             left = deadline - time.monotonic()
             if left <= 0:
@@ -81,6 +83,20 @@
             if not chunk:
                 raise Failure("connection closed after %d of %d responses" % (len(got), count))
             buf += chunk
+        while then_close:
+            left = deadline - time.monotonic()
+            if buf or left <= 0:
+                raise Failure("still open or sent more after %d responses" % count)
+            sock.settimeout(left)
+            try:
+                chunk = sock.recv(65536)
+            except socket.timeout:
+                chunk = b"?"
+            except ConnectionResetError:
+                break
+            if not chunk:
+                break
+            buf += chunk
     finally:
         sock.close()
     return got
@@ -106,14 +122,24 @@
     ("a large echo, then a file, then a GET",
      post("/echo", BIG) + get("/static/index.html") + get("/ok"),
      [(200, BIG), (200, INDEX), (200, b"ok")]),
+    ("a GET, then a POST that waits for 100 Continue",
+     get("/ok") + b"POST /echo HTTP/1.1\r\nHost: x\r\nContent-Length: 5\r\n"
+     b"Expect: 100-continue\r\n\r\n",
+     [(200, b"ok"), (100, b""), (200, b"hello")],
+     {"body_after_100": b"hello"}),
+    ("nothing after Connection: close is answered",
+     get("/ok") + b"GET /ok HTTP/1.1\r\nHost: x\r\nConnection: close\r\n\r\n"
+     + get("/ok"),
+     [(200, b"ok"), (200, b"ok")],
+     {"then_close": True}),
 ]
 
 
 def main():
     failed = 0
-    for label, payload, want in CASES:
+    for label, payload, want, *opts in CASES:
         try:
-            got = exchange(payload, len(want))
+            got = exchange(payload, len(want), **(opts[0] if opts else {}))
             if got != want:
                 raise Failure("expected %r, got %r" % (short(want), short(got)))
             print("pipeline: %s: ok" % label)
  1. (Optional; cost) A full read of tiny pipelined requests costs about twice the server CPU per request.
    • Each web-dispatch-first! copies the rest of the read out (Array.suffix, web.carp:1617) and back in byte by byte (:2357). So a 4 KiB read of K requests copies O(K × 4 KiB) bytes.
    • I measured the server's own CPU time (/proc/PID/stat) over 6,500 GET /ok requests (31 bytes each), in three sets of three interleaved runs at load ~2 or below:
      • one request per write: 95.4–100.0 µs per request on head and 90.8–96.9 µs on main, with head 1.7–8.5% above main in every pair (the ticks are 10 ms, about 1.5 µs per request here). That is probably web-request-length (web.carp:1601) running web-request-status and web-parse-framing again for every request; I didn't profile it;
      • 130 per write (one full read): 204.6–209.2 µs on head.
    • Only pipelined bursts pay this, since a read holding one request has an empty tail. Main couldn't answer such bursts at all, so it isn't a regression. It's worth knowing before anyone benchmarks pipelining.

Verdict: merge

Pipelined requests are answered in order on every case I could build, and single requests cost 2–9% more server CPU than on main. Main's rejections all still hold, and the new paths leak nothing. All three findings are optional: header hardening that pipelining makes worth having, two smoke cases for guards nothing pins, and the measured cost of tiny bursts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants