fix: hybrid passthrough for Scylla manifest mid-frame range tail

[ServerSideHannes]

Summary

• 2026.7.15 still streamed because passthrough_blocked_reason=ranged_copy_segment_misaligned: Scylla sends bytes=0-4999341931 (~4767MB) but internal frames are ~8.33MB (50MB upload / 6), so the range ends mid-frame
• Fix: _split_plaintext_range_on_segments → passthrough all complete ciphertext segments + re-encrypt only the trailing suffix (UPLOAD_PART_COPY_PASSTHROUGH_TAIL)
• Logs streaming_tail_mb on passthrough events for diagnosis

Tests

• test_split_plaintext_range_allows_hybrid_tail — split logic
• test_scylla_prod_shape_* — uses prod exact range end 4999341931 + 8.33MB frames; expects 500+ upload_part_copy + 1 upload_part tail
• 12/12 passthrough unit tests pass

Post-deploy

kubectl logs -n s3proxy-python -l app.kubernetes.io/name=s3proxy-python --since=30m \
  | rg 'UPLOAD_PART_COPY_ROUTE|UPLOAD_PART_COPY_PASSTHROUGH'

Expect route=passthrough, passthrough_blocked_reason=null, and streaming_tail_mb ~1MB on .sm_* manifests.

Made with Cursor

[ServerSideHannes]

Pushed 7a0b6a9 addressing why the hybrid passthrough alone wasn't enough: live-cluster analysis showed the failing requests are UploadPartCopy calls that send zero response bytes until the copy completes, while rclone (scylla-manager-agent) hangs up after exactly 300s idle — a 4.7GB part took ~400s streaming, so every attempt was doomed and the proxy kept it running as a zombie afterwards (1,953 streaming copies started vs 232 completed in 9h, plus 18k MEMORY_BACKPRESSURE events from the retry storm).

What's new:

• Keepalive streaming: UploadPartCopy now commits to 200 OK and trickles whitespace (default every 5s, S3PROXY_COPY_KEEPALIVE_INTERVAL) while the copy runs — the AWS S3 long-copy pattern — then emits CopyPartResult or an <Error> document as the body. Removes the client-timeout ceiling for every route, including the streaming fallback.
• Client disconnect cancels the work — no more zombie copies burning the memory budget for minutes after the client gave up.
• Parallel passthrough segments (bounded, default 8, S3PROXY_PASSTHROUGH_SEGMENT_CONCURRENCY) instead of ~570 sequential backend calls.
• MD5 pass overlapped with the segment copies via TaskGroup instead of serially re-reading the whole 4.7GB source afterwards.
• Fixed pre-existing ruff check failures so the Lint workflow passes.

New tests (tests/unit/test_upload_part_copy_keepalive.py) assert the actual failure mode: first response byte arrives while the backend copy is still in flight, whitespace+XML parses as sent on the wire, mid-copy failure yields a parseable <Error> body, disconnect cancels in-flight backend calls, segments copy concurrently, the MD5 pass starts before the last segment finishes, and a mid-frame-range hybrid copy reassembles byte-exact under parallelism.

Post-deploy check:

kubectl logs -n s3proxy-python -l app.kubernetes.io/name=s3proxy-python --since=30m \
  | rg 'UPLOAD_PART_COPY_KEEPALIVE|UPLOAD_PART_COPY_PASSTHROUGH_COMPLETE|UPLOAD_PART_COPY_CLIENT_GONE|FAILED_AFTER_200'

Expect PASSTHROUGH_COMPLETE with elapsed_sec well under 300s and CLIENT_GONE to disappear.

[ServerSideHannes]

Pushed 6d06a3c fixing the two CI integration failures. Both pre-dated the keepalive commit (same failures on ae39244) — they're latent bugs that ae39244's correct error mapping exposed:

1. test_copy_full_encrypted_object (InvalidPart at Complete): the single-segment passthrough copy stored the synthetic plaintext-MD5 as the part etag, so CompleteMultipartUpload forwarded an etag the backend never issued. It "passed" on main only by accident: the backend's InvalidPart used to become a 500, boto3 retried, and the retry recovered via state reconstruction (which reads real etags from list_parts). Once InvalidPart correctly mapped to 400, the retry disappeared and the bug surfaced. Fix: store the backend CopyPartResult etag in part state; _build_s3_parts now uses stored etags instead of the client echo.

2. test_part_number_out_of_range (Abort → raw 400): the new part-range validation rejects UploadPart before reading the request body, leaving the body bytes on the keep-alive connection — the next request on it (the abort) hit uvicorn's Invalid HTTP request received. Fix: the error handler drains small bodies and sends Connection: close for bodies too large to slurp.

Both reproduced locally against real MinIO (the reason they don't show in unit tests: the S3 mock doesn't validate part etags on complete) and verified fixed end-to-end. Regression tests added: test_single_segment_passthrough_complete_presents_backend_etag asserts the etag actually sent to the backend matches the backend part etag, and test_error_body_drain.py covers the connection-poisoning semantics.