Skip to content

Support Large Files#15

Open
philomath213 wants to merge 1 commit into
exploide:mainfrom
philomath213:support-large-files
Open

Support Large Files#15
philomath213 wants to merge 1 commit into
exploide:mainfrom
philomath213:support-large-files

Conversation

@philomath213

Copy link
Copy Markdown

Summary

  1. File uploads previously read the entire file into memory with f.read() before POSTing, causing socket.send() to fail on large files.
  2. Now passes the open file handle directly to requests, which streams the body in chunks.
  3. HMAC-SHA256 signing is preserved by hashing the file incrementally (64 KB chunks) before seeking back to 0.

Test plan

  • pytest passes
  • Upload a large ZIP file (bhcli upload large.zip) and confirm it completes without a socket/buffer error

@exploide

exploide commented Jun 8, 2026

Copy link
Copy Markdown
Owner

Thanks for your improvement. Will merge soon.

Out of interest, how large was your file that caused a problem?

And I'm thinking about the hashing in 64k chunks. The hashing function can easily process much larger chunks and I'm wondering whether thousands of read-64k iterations would cause a certain IO overhead. But probably not that important.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants