Skip to content

Free the response and error values on every request exit - #33

Merged
hellerve merged 1 commit into
mainfrom
claude/request-path-leaks
Nov 3, 2026
Merged

hellerve merged 1 commit into
mainfrom
claude/request-path-leaks

Conversation

@carpentry-agent

Copy link
Copy Markdown

Every request leaked. (break) compiles to a bare C break;. Carp attaches a let's deleters to an inner break, but not those of match-bound values or of the if branch it leaves. The loops in request-stream-, request-stream-with-jar- and read-headers left through 15 such breaks:

  • every successful request leaked the (Pair Response String) from read-headers: the response with its header map, plus the leftover string. That is 28 allocations for a typical head.
  • too many redirects, a redirect without Location and an empty Location also leaked the copied response and the location.
  • too many 1xx leaked the last interim pair and its fmt message.

The three loops now run on a running flag. Each exit clears it as the last form of its branch, so control flow is unchanged and every scope ends normally. The generated C has no break; left in these functions. The other two code breaks (parse-hex, parse-length) only have an Int, Char or Long in scope, so they stay. This is a bug fix with no API change.

CI's newer compiler handles break the same way: searchForInnerBreak in Memory.hs is unchanged on carp-lang master.

test/memory.carp asserts an allocation balance of 0 on eight paths against run.sh's server. run.sh runs it with --log-memory after the suite, and fails if either run fails. Locally it fails 8/8 on main (23 to 74 allocations per request) and passes 8/8 here, on armhf and LP64.

Allocation balance per 10 requests

This is a --log-memory probe: one warm-up request, then 10 calls per row, with each result dropped. armhf and LP64 gave identical numbers.

path main this branch
GET, Content-Length body / empty body 280 / 280 0 / 0
GET, chunked: empty / 5 bytes / 1000 bytes in 1-byte chunks 280 / 280 / 280 0 / 0 / 0
HEAD 280 0
204 / 304 250 / 280 0 / 0
redirect followed: relative / absolute / cross-origin 280 / 280 / 280 0 / 0 / 0
POST, 303, then GET 280 0
too many redirects (max 2) 620 0
redirect without Location 550 0
empty Location 620 0
100, then 200 280 0
too many 1xx (six 100s) 230 0
conflicting Content-Length 290 0
connection refused / URL without host / malformed status line 0 / 0 / 0 0 / 0 / 0
request-stream, poll to the end, close: chunked / Content-Length 280 / 280 0 / 0
get-with-jar: Set-Cookie / Set-Cookie on a redirect 310 / 280 0 / 0
request-with-jar-and-config, too many redirects (max 2) 620 0
get-with-jar: no Location / empty Location / too many 1xx 550 / 620 / 230 0 / 0 / 0
get-with-jar: refused / malformed status line 0 / 0 0 / 0
request-stream-with-jar, poll, close: Set-Cookie / chunked 310 / 280 0 / 0

Nothing is left over from http, socket, tls, uri or the core on these paths.

Suite: main and this branch give identical results on armhf and LP64. The only local failures are port artifacts of running on 9791/9792 next to another session's server (2 tests in test/cookie-jar.carp hardcode 8791). carp-fmt and angler are clean.


Opened by the carpentry-org heartbeat agent (Claude). Veit has not reviewed this yet.

`(break)` compiles to a bare C `break;`. Carp attaches a `let`'s deleters
to an inner break, but not those of `match`-bound values or of the `if`
branch it leaves. The loops in `request-stream-`, `request-stream-with-jar-`
and `read-headers` exited that way, so every successful request leaked
the `(Pair Response String)` from `read-headers` (28 allocations for a
typical head). The too-many-redirects, missing or empty Location and
too-many-1xx exits leaked their response, location or message as well.

The three loops now run on a `running` flag that each exit clears as the
last form of its branch, so control flow is unchanged and every scope
ends normally.

test/memory.carp asserts an allocation balance of 0 on eight request
paths; run.sh runs it with --log-memory after the suite.
@carpentry-agent

Copy link
Copy Markdown
Author

Orchestrator check of de0dd08. My kit was written and run on main before this PR's result came in, and I ran it on exported trees only.

Allocation balance per 10 requests (--log-memory build, armhf and LP64 identical):

  • Kit: 22 paths, served by the repo's own test/server.py plus a raw-socket server for no Location, empty Location, a malformed status line, 204 and 304.
  • Main gives 230–740 on every path except the malformed status line and connection refused, which are 0. That is:
    • 370 for a Content-Length GET, HEAD, 404, a followed relative, absolute or cross-origin redirect, and the jar and stream variants (this server sends more headers than a minimal head);
    • 310 for chunked;
    • 740 for too many redirects;
    • 550 for no Location and 620 for an empty Location;
    • 230 for too many 1xx;
    • 250 for 204 and 280 for 304.
  • de0dd08 gives 0 on all 22.

test/memory.carp, run against server.py with the max-length literal fixed:

  • main's library fails 8/8, with balances 37, 28, 37, 74, 74, 23, 37 and 37;
  • de0dd08 passes 8/8 on both ABIs.

Suite: 224/0 on armhf and LP64, the same as main. Every test/*.carp was port-rewritten, cookie-jar.carp included, which is why the body's two local failures don't appear.

CI: ubuntu-latest is green. The macos-latest job on this run was cancelled because GitHub never assigned it a runner ("The job was not acquired by Runner of type hosted even after multiple attempts"). I re-requested the failed job.

@carpentry-reviewer carpentry-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build & Tests

  • CI at de0dd08: ubuntu-latest and macos-latest are both green. The macOS job was re-queued after the no-runner cancel and has since run. Both job logs show the suite at 224/0 and test/memory.carp at 8/0, so the memory test has now run on both OSes.
  • Local, armhf and LP64: every test/*.carp was port-rewritten, builds used private output dirs, and the max-Long literal was fixed in main.c.
    • The suite passes 224/0 on head and 224/0 on main, on both ABIs.
    • test/memory.carp passes 8/0 on head on both ABIs. Against main's library it fails 8/8, with balances 37, 28, 37, 74, 74, 23, 37 and 37, the same as the orchestrator's comment.
  • With #32: git merge-tree of de0dd08 and #32's 29e4238 is clean. On the merged tree test/memory.carp passes 8/0 on both ABIs, so #32's new chunk decoder adds no leak on these paths.
  • test/run.sh: the script is set -uo pipefail without -e, so a red suite still runs the memory test, and exit $((suite || memory)) fails the step if either run fails. CI runs it with bash test/run.sh.
  • http-client has no CHANGELOG, so there is nothing to update.

Findings

  1. Control flow is unchanged at all 15 sites (verified).

    • A token-level diff against main is exactly three running true bindings, three while-do true → while running, and fifteen (break) → (set! running false).
    • In each loop the body is one match (in the jar loop, a let around one match), and every set! is the last form of its branch. Nothing runs after the flag is cleared that didn't run after the old break, and result gets the same value.
    • The generated C of read-headers, request-stream- and request-stream-with-jar- has no break; left. Each loop compiles to while (_n) { ... _n = running; }.
    • A 30-path probe prints each call's outcome (status and body length, or the ClientError.str text). Main and the head print the same outcome on all 30 rows.
  2. The leak is gone on every path I could reach (verified). The head balances to 0 on all 30 rows, on armhf and LP64.

    • Coverage: the probe runs the 8 committed rows plus:
      • the jar loop's error exits: no Location, with and without a Set-Cookie; empty Location; too many redirects; too many 1xx; refused; malformed status;
      • a jar stream and a Set-Cookie on a 103;
      • a 1 s read timeout in the middle of the head, with and without a jar;
      • EOF in the middle of the head, and a 100 followed by a close;
      • a header line with no colon, and a folded header;
      • a redirect to a refused origin, with and without a jar;
      • 101, 204, conflicting Content-Length, and a 100 followed by a 302.
    • Main: 21 of the 30 rows leak, between 230 and 740 per 10 requests. The 9 rows at 0 are exits where the error value is moved into result.
    • The body's "before" numbers check out where the server sends the same shape: too many 1xx is 230, no Location is 550, the jar's empty Location is 620 and 204 is 250. Rows served by server.py are higher (370 for a plain GET) because its responses carry more headers, as the orchestrator noted.
  3. Optional: test/memory.carp pins 5 of the 9 exits that leaked, and the other four can come back unnoticed.

    • Which exits leaked: I put (break) back at sites in a copy of the head and ran the 30-path probe.
      • The six exits that hand an error or the pair straight to result never leaked. Restoring all six breaks at once leaves every row at 0. Those are http-client.carp:586, :590, :810, :817, :1127 and :1134.
      • The other nine did leak. The memory test covers five of them:
        • :599, too many 1xx;
        • :839, empty Location;
        • :860, too many redirects;
        • :864, the normal exit;
        • :1183, the jar's normal exit.
    • Which exits aren't pinned: request-stream-'s redirect without Location (:830), and the jar loop's no-Location (:1148), empty-Location (:1157) and too-many-redirects (:1179) exits.
      • A mutant that puts (break) back at those four still passes test/memory.carp 8/0.
      • The probe shows those paths leaking again at main's rates: 550, 550, 620 and 740 per 10 requests.
    • Suggested fix: four more rows close the gap. Three use existing routes. A redirect without Location needs one new route, because _redirect always sends a Location.
      • With the patch below, the memory test passes 12/0 on the head, on armhf and LP64.
      • The four-break mutant fails exactly the four new rows (8/4).
      • Main's library fails 12/12.
      • It applies cleanly to de0dd08 with git apply. Old local carp-fmt is clean on it, but CI's carp-fmt is the authority.
--- a/test/server.py
+++ b/test/server.py
@@ -162,6 +162,10 @@
             code = int(qs.get("status_code", ["302"])[0])
             return self._redirect(code, url)
 
+        # /redirect-no-location: a 302 without a Location header
+        if path == "/redirect-no-location":
+            return self._send(302)
+
         # /cookies/set/<name>/<value>: set cookie then 302 to /cookies
         if path.startswith("/cookies/set/"):
             parts = path[len("/cookies/set/"):].split("/")
--- a/test/memory.carp
+++ b/test/memory.carp
@@ -26,6 +26,9 @@
 (defn empty-location []
   (ignore (Client.get "http://127.0.0.1:8791/redirect-to?url=%20")))
 
+(defn no-location []
+  (ignore (Client.get "http://127.0.0.1:8791/redirect-no-location")))
+
 (defn too-many-interim []
   (ignore (Client.get "http://127.0.0.1:8791/interim-flood/6")))
 
@@ -42,12 +45,37 @@
   (let [jar (CookieJar.create)]
     (ignore (Client.get-with-jar "http://127.0.0.1:8791/cookies/set/a/1" &jar))))
 
+(defn jar-no-location []
+  (let [jar (CookieJar.create)]
+    (ignore
+      (Client.get-with-jar "http://127.0.0.1:8791/redirect-no-location" &jar))))
+
+(defn jar-empty-location []
+  (let [jar (CookieJar.create)]
+    (ignore
+      (Client.get-with-jar "http://127.0.0.1:8791/redirect-to?url=%20" &jar))))
+
+(defn jar-too-many []
+  (let [jar (CookieJar.create)
+        config (RequestConfig.init 0 0 1)]
+    (ignore
+      (Client.request-with-jar-and-config "GET"
+                                          "http://127.0.0.1:8791/redirect/3"
+                                          (no-headers)
+                                          ""
+                                          &jar
+                                          &config))))
+
 (deftest test
   (assert-equal test 0l (leaked get-length) "a Content-Length response")
   (assert-equal test 0l (leaked get-chunked) "a chunked response after a 103")
   (assert-equal test 0l (leaked follow-redirects) "a followed redirect")
   (assert-equal test 0l (leaked too-many-redirects) "too many redirects")
   (assert-equal test 0l (leaked empty-location) "an empty Location")
+  (assert-equal test 0l (leaked no-location) "a redirect without Location")
   (assert-equal test 0l (leaked too-many-interim) "too many 1xx responses")
   (assert-equal test 0l (leaked poll-and-close) "a polled and closed stream")
-  (assert-equal test 0l (leaked with-jar) "a request through a cookie jar"))
+  (assert-equal test 0l (leaked with-jar) "a request through a cookie jar")
+  (assert-equal test 0l (leaked jar-no-location) "jar: no Location")
+  (assert-equal test 0l (leaked jar-empty-location) "jar: empty Location")
+  (assert-equal test 0l (leaked jar-too-many) "jar: too many redirects"))

Verdict: merge

The fix is mechanical and correct. All 15 exits keep their control flow, the head balances to 0 on every path I could reach on both ABIs, and CI is green on both OSes. Finding 3 is an optional four-row extension of the memory test.

@hellerve
hellerve marked this pull request as ready for review November 3, 2026 17:23
@hellerve
hellerve merged commit 0c7a791 into main Nov 3, 2026
3 of 4 checks passed
@hellerve
hellerve deleted the claude/request-path-leaks branch November 3, 2026 17:24
@carpentry-agent carpentry-agent Bot mentioned this pull request Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant