feat: streaming file transfer RPCs (UploadFile / DownloadFile)

[zanetworker]

Problem

There is no programmatic way to move files in and out of sandboxes. Every file transfer today goes through either:

1. CLI --upload flag — works only at creation time, requires the CLI binary, single directory only (feat: support multiple --upload flags on sandbox create #1635 tracks multi-upload)
2. SSH/SCP — requires SSH access to the sandbox, not available in all deployment modes (Kubernetes, remote gateways)
3. exec with stdin piping — exec(["cat", ">", path], stdin=bytes) breaks on binary content, hits the 4MB gRPC message size limit, loses permissions, and requires shell tools in the image

None of these work for SDK consumers who need typed, streaming, resumable file transfer as part of their programmatic workflow.

Use cases

1. Platform-managed sandboxes (Mode 2)

A platform worker (Anthropic, OpenAI) creates sandboxes on behalf of agents. The worker needs to seed files before execution and retrieve artifacts after.

Anthropic Managed Agents: Worker polls a queue, claims a session, downloads skill files into /workspace/skills/, runs the agent, then uploads output artifacts back to the platform.

OpenAI Agents SDK: Developer writes Manifest(entries={"repo": LocalDir(src="./myproject")}). The SDK calls session.write() per file during materialization. Without an UploadFile RPC, session.write() has no clean implementation — the adapter either raises NotImplementedError or falls back to exec-based hacks.

2. Framework sandbox extensions (Mode 3)

The developer's process owns the agent loop. The SDK is embedded in the same process.

OpenClaw mirror mode: Before every command, the workspace is synced into the sandbox. After every command, changes are synced back. Currently shells out to the CLI 5+ times per cycle. A streaming file transfer RPC would replace all of this with direct gRPC calls.

CI/CD pipelines: Spin up a sandbox, seed the repo checkout and test fixtures, run tests, retrieve coverage reports and build artifacts. All from a script, no CLI binary packaging needed.

3. Read-only file injection at creation time

Operators need to inject tamper-proof configuration files (persona files, policy overrides, credential bundles) that the agent can read but never modify. The current --upload runs after Landlock enforcement as the sandbox user, so it cannot write to read-only paths. File injection through gRPC before Landlock enforcement solves this. (See #1268 for the full proposal.)

4. Multi-file uploads at creation time

Today --upload accepts only one path. Seeding a sandbox with multiple directories requires tarring them together or running sandbox upload separately after creation. (See #1635 for the CLI-side fix; this issue covers the underlying RPC that both CLI and SDK would use.)

5. Async SDK consumers

SDK consumers in async frameworks (Temporal, FastAPI) need non-blocking file transfer. The current Python SDK is entirely synchronous — every call requires asyncio.to_thread(). Streaming file transfer should be designed async-first so consumers don't need to wrap every call.

6. Large file and binary content handling

Agent workloads increasingly involve large artifacts: model weights, datasets, compiled binaries, container images. The 4MB default gRPC message size limit and exec-based workarounds cannot handle these. Streaming with 64KB chunks keeps memory constant regardless of file size.

Proposed design

From RFC 0006:

rpc UploadFile(stream UploadFileRequest) returns (UploadFileResponse);
rpc DownloadFile(DownloadFileRequest) returns (stream DownloadFileResponse);

• Client-streaming upload: First message carries metadata (sandbox ID, remote path, size, mode, archive flag). Subsequent messages carry 64KB chunks.
• Server-streaming download: First message carries a header (size, mode, archive flag). Subsequent messages carry chunks.
• Archive mode: is_archive flag enables directory transfers as tar streams without requiring tar in the sandbox image.
• Routing: Gateway routes file streams to the supervisor via the existing ConnectSupervisor/RelayStream infrastructure. No new transport layer needed.

Why streaming, not tar-over-exec-stdin

	Streaming RPC	tar-over-exec
Memory	Constant (64KB chunks)	Entire file in memory
Size limit	None (streaming)	4MB gRPC default
Progress	Count chunks	Not possible
Permissions	Metadata header	Inconsistent
Binary safety	Native bytes	Breaks on NUL bytes
Image deps	None	Requires tar

Implementation scope

1. Proto definitions for UploadFile/DownloadFile (messages above)
2. Gateway-side handler: route streams to supervisor via relay
3. Supervisor-side handler: file read/write alongside existing SSH and exec relay handlers
4. Python SDK: upload_path() / download_path() methods
5. TypeScript SDK: equivalent methods (after shared Rust core lands, per RFC 0005)
6. CLI: migrate --upload to use the new RPC internally

Open questions

1. Archive format: tar (uncompressed, gRPC compresses at transport layer), tar.gz, tar.zstd, or configurable?
2. Size limits: Should the gateway enforce per-file or per-request size limits? What defaults?
3. Resume on failure: Should the protocol support resumable uploads (offset-based chunking), or is retry-from-zero acceptable for v1?

Related issues

• feat: support multiple --upload flags on sandbox create #1635 — feat: support multiple --upload flags on sandbox create (CLI-side, would use this RPC)
• feat: inject read-only files into sandbox at creation time without custom images #1268 — feat: inject read-only files into sandbox at creation time (pre-Landlock injection, complementary)
• Python and TypeScript SDK Support #1044 — Python and TypeScript SDK Support (parent roadmap item)
• RFC 0006 — SDK Consumption Entrypoints and File Transfer
• RFC 0005 / PR feat(sdk): shared SDK core and TypeScript binding (RFC 0005) #1617 — Shared Rust SDK core and TypeScript binding