fix(list): preserve keys with empty path segments (//) instead of 503

[alukach]

Fixes #116.

What I'm changing

Listing a prefix that contained a single object whose key has an empty path segment (a //) returned 503 ServiceUnavailable for the entire page — one malformed-looking (but perfectly legal) key made the whole prefix unlistable through the gateway. S3 itself lists these keys fine; keys are opaque byte strings and // is legal.

The root cause is that the LIST path routed every backend key through object_store::path::Path, whose strict parser rejects empty segments and which cannot even represent a // key (Path::from would collapse it). So there was no lenient-parse escape hatch — the fix is to stop routing list responses through object_store::Path at all.

Guiding principle: if a key is listable in S3, it must be listable through multistore. Keys pass through byte-faithfully — nothing is skipped or normalized.

How I did it

• api/list.rs — new BackendListResult / BackendObject (raw String keys), parse_backend_list_xml (lenient serde/quick-xml parse of the backend's ListBucketResult), and build_backend_list_url (maps prefix/start-after/marker into the backend key space, encodes query values with the SigV4-canonical set so space → %20, and deliberately does not forward encoding-type so we don't double-encode). collect_list_entries and the two XML builders now consume the lenient type; directory-marker detection compares on the slash-trimmed raw key instead of an object_store::Path.
• proxy.rs handle_list — fetches one page from the backend via raw signed HTTP (sign_s3_request + send_raw, the same mechanism multipart and batch delete already use), parses the XML leniently, and builds the response. Pagination is still pushed to the backend via the continuation token.
• refactor(backend) (second commit) — ProxyBackend::create_paginated_store and its object_store::PaginatedListStore plumbing are now dead. Removed the trait method, StoreBuilder::build, all impls (cf-workers, server, lambda examples, test mocks), and the cf-workers fetch_connector module (it existed only to inject an HTTP connector into that list store). Presigned URLs are unaffected — they're still built offline via the object_store signer.
• docs/ — updated custom-backend, request-lifecycle, crate-layout, multi-runtime to describe LIST via raw signed HTTP.

Known limitation (out of scope)

GET/PUT/HEAD/DELETE of a // key still returns 400 (validate_key rejects degenerate segments, and object_store's presigned-URL signer can't represent such keys). This PR fixes the reported LIST 503 so the keys are at least visible; making them individually addressable is a larger, separate change to the object-operation signing path. Happy to file a follow-up if wanted.

Test plan

• cargo test -p multistore — 131 lib tests pass, including new coverage: // round-trips through parse_backend_list_xml, survives to the response XML, the backend-URL prefix mapping/encoding, and an end-to-end handle_list regression proving a stray // key returns 200 with the key intact.
• cargo test (full native workspace) — all green.
• cargo clippy (native + --target wasm32-unknown-unknown) — clean.
• cargo check -p multistore-cf-workers --target wasm32-unknown-unknown — compiles.
• cargo fmt.

🤖 Generated with Claude Code

[claude]

Claude finished @alukach's task in 3m 16s —— View job

---

Reviewing PR #117

• Gather context (PR body, changed files, custom instructions)
• Read diff (api/list.rs, proxy.rs, backend/mod.rs, cf-workers changes, docs)
• Correctness / security / best-practice review
• Ponytail over-engineering pass
• Post final review

---

✅ No blocking issues — safe to merge.

Traced the whole flow: handle_list → build_backend_list_url → sign_s3_request/send_raw → parse_backend_list_xml → collect_list_entries → build_list_xml(_v1). A few things I specifically checked and ruled out as non-issues:

• Query param order vs SigV4: build_backend_list_url emits params in a fixed (non-alphabetical) order, but sign_s3_request → canonicalize_query_string (crates/core/src/auth/sigv4.rs:170) re-sorts the query string itself before signing, so URL param order is irrelevant to signature correctness.
• XML injection: raw keys (including //, <, &) flow into ListContents/ListCommonPrefix, which are serialized via quick_xml::se::to_string over #[derive(Serialize)] structs (api/response.rs) — properly escaped, not string-formatted.
• Dead code: confirmed no leftover references anywhere to FetchConnector, create_paginated_store, PaginatedListStore, or PaginatedListOptions after the removal.
• New deps: none — quick-xml and percent-encoding were already workspace deps before this PR.
• raw_resp.status >= 300 in handle_list (proxy.rs) correctly maps backend errors to ProxyError::BackendError rather than silently parsing an error body as a list result.

One non-blocking observation: LastModified is now passed through as the backend's raw string (api/list.rs, RawContents::last_modified) instead of being parsed/reformatted to a fixed %Y-%m-%dT%H:%M:%S%.3fZ pattern as before. For real S3 this is a no-op (identical format), but a non-AWS S3-compatible backend emitting a slightly different (still-valid) ISO8601 variant (e.g. no milliseconds) would now surface that variance to clients. Likely fine given the "keys pass through byte-faithfully" design goal extends naturally to other fields, just flagging since it's a subtle behavior change from before.

Simplify (ponytail)

• crates/core/src/api/list.rs:214 — build_backend_list_url's local QUERY_SET const is byte-identical to crate::backend::multipart::S3_PATH_ENCODE_SET (both NON_ALPHANUMERIC minus -._~). Reuse the existing pub(crate) constant instead of redefining it.

---

💰 Estimated review cost: $1.28 · 3m15s · 34 turns

[github-actions]

📖 Docs preview deployed to https://multistore-docs-pr-117.development-seed.workers.dev

• Date: 2026-07-21T18:03:59Z
• Commit: ead7373

[github-actions]

🚀 Latest commit deployed to https://multistore-proxy-pr-117.development-seed.workers.dev

• Date: 2026-07-21T18:03:59Z
• Commit: ead7373

[alukach]

Code review follow-ups (out of scope for this PR)

A code review of this branch found no Critical issues and no regressions — the //-key fix is correct, SigV4 signing of the hand-built list URL is verified correct (byte-faithful %20-not-+ round-trip), and dead-code removal is complete on native + wasm. The items below are all pre-existing or theoretical; deliberately left out to keep this PR focused on #116. Filing here so they're tracked:

1. Backend LIST errors collapse to 503 (proxy.rs, non-2xx → BackendError). A backend 403 AccessDenied / 404 NoSuchBucket becomes a retryable-looking 503, so client SDKs retry a permanent failure. Pre-existing: the old from_object_store_error path also mapped 403→503 (_ => BackendError). Worth revisiting across all raw paths (multipart/batch-delete share it), but note the tension with multistore's deliberate opaque-backend posture — passing the raw S3 <Error> body through would leak the backend bucket name.
2. V2 KeyCount counts pre-filter (proxy.rs): computed before directory-marker filtering, so a listing at a marker prefix reports a count one higher than the entries returned. Pre-existing; technically non-conforming (most clients count entries).
3. V1 NextMarker is the raw backend key (proxy.rs → build_list_xml_v1): not prefix-stripped/rewritten/encoded, so on a backend_prefix/add_prefix bucket a truncated V1 listing hands back a double-prefixed marker. Pre-existing (base used obj.location.to_string(), same key space); narrow (V1 + truncated + prefixed).
4. Empty <Size></Size> would 503 the page (list.rs, size: u64): a malformed (not spec-legal) empty Size element fails the parse. Theoretical — real S3 always emits a value.