Skip to content

Add Sandbox SDK on a unified httpx client (0.2.0)#19

Merged
ats3v merged 6 commits into
mainfrom
sandbox-sdk
Jul 20, 2026
Merged

Add Sandbox SDK on a unified httpx client (0.2.0)#19
ats3v merged 6 commits into
mainfrom
sandbox-sdk

Conversation

@ats3v

@ats3v ats3v commented Jul 10, 2026

Copy link
Copy Markdown
Member

Adds the Sandboxes feature (deepinfra/backend#3148) and modernizes the package around one unified client.

What changed

  • Sandbox API (sandbox_api.py): create (blocks until running, "10m"-style timeouts), from_id, list(tags=...), exec/run_python (aggregate the NDJSON exec stream), fs.read/fs.write, stop/start/terminate, context managers, and async twins (acreate, aexec, ...) for every network method. stop()/start() block until the state transition completes (wait=False to fire-and-forget) — stop/terminate are asynchronous server-side (~2s), so sb.stop(); sb.start() used to 409. wait_until_stopped/await_until_stopped are public alongside wait_until_running.
  • One DeepInfraClient, rewritten in place onto httpx (was requests, single-URL, POST-only): sync + async, streaming, pooled connections, typed exceptions parsed from both {"error"} and FastAPI {"detail"} error shapes. Retries: connect-errors where marked, 502/503/504 only on GETs, never on non-idempotent POSTs (the old client retried 4xx ×5).
  • Typed exception hierarchy under deepinfra/deepinfra.exceptions: 429 maps to RateLimitError (TooManySandboxesError kept as an alias), MaxRetriesExceededError is preserved as a subclass of APIConnectionError, and SandboxTimeoutError/SandboxFailedError carry .sandbox_id so a failed Sandbox.create(wait=True) still leaves a handle for cleanup.
  • Legacy inference wrappers keep their public API (TextGeneration, Embeddings, AutomaticSpeechRecognition, TextToImage) but now run on the unified client; requests/requests-toolbelt dropped. Also fixes from deepinfra import TextGeneration, which was missing from the exports in 0.1.0.
  • Packaging/CI: hatchling pyproject.toml (0.2.0, py>=3.9, deps httpx+pydantic only), py.typed, CI on GitHub-hosted runners with a 3.9–3.13 matrix (ruff, mypy, pytest), PyPI trusted publishing on v* tags with a tag==version guard, README/CHANGELOG/examples.

Not in this PR (designed, lands next)

exec_stream, snapshot/from_snapshot, expose_port, fs.upload_dir — no stubs shipped; each lands behind its backend endpoint. The server side of exec/fs is stubbed in deepinfra/backend#3474; end-to-end verification follows once the implementation PR lands.

Pre-publish audit round (2026-07-20)

A full review pass (core client, sandbox API, legacy wrappers, tests, packaging, docs) before the repo goes public. Fixed, each with regression tests:

  • Constructing a legacy wrapper with no API key crashed with AttributeError instead of warning (refactor regression; the missing-key path had no test — now it does).
  • parse_duration(0.5) silently truncated to 0 ("use server default"); fractional seconds now round up.
  • 429 raised the sandbox-named TooManySandboxesError for inference rate limits too → RateLimitError introduced (alias kept).
  • Wait-failure exceptions gained .sandbox_id (create-leak cleanup path).
  • Coverage gaps closed: async retry loop (connect error / 502 / retries exhausted), ReadTimeout → APITimeoutError mapping, exec-stream heartbeats on the async path.
  • Docs: the process-wide async client's pool binds to the first event loop — README/docstring note recommends a per-loop DeepInfraClient when calling asyncio.run() repeatedly. deepinfra/utils/ got its missing __init__.py.

Verification

Prod e2e (2026-07-20, real sandboxes against api.deepinfra.com): every deployed surface passes, sync + async — create (wait/no-wait), wait_until_running, refresh, list + tag filtering, from_id, stop(wait=True)start() (previously 409ed), stop(wait=False) + wait_until_stopped, terminate, both context managers, 404 mapping on bogus ids, and all four inference wrappers on the new client (including multipart ASR). Verified server semantics now documented: stop/terminate take effect ~2s after the call returns; terminate is a soft delete (sandbox stays fetchable by id as "deleted" briefly, disappears from list immediately). exec/run_python/fs return 501 Not implemented yet from prod — SDK raises a clean typed error; e2e for those is gated on the backend deploy, and the 0.2.0 tag should wait for a green run of that suite.

Earlier prod e2e (2026-07-10): 24/24 lifecycle + break-attempt checks (path-traversal ids, empty id, junk/negative durations, unknown plan, non-string tags, instant wait-timeouts, double-stop), no sandboxes leaked. Break-testing found and fixed two bugs: sandbox ids are now URL-quoted in paths (from_id("../models") escaped to /v1/models; now a clean 404 + ValueError on empty id), and create(wait=False) populates fields with one GET instead of returning an empty-state handle. Legacy wrappers verified against live models (GLM-4.5-Air incl. full-URL endpoint form, gte-large, whisper-tiny.en file-path and raw-bytes multipart, sd3.5-medium); the 2024 response dataclasses crashed on fields the API has since added (request_id, words) — pre-existing 0.1.0 breakage, fixed with drift-tolerant parsing + regression test.

Unit suite: 107 passed (respx-mocked, exercising the real request path), on Python 3.9 (oldest supported); mypy strict + ruff clean; wheel build + twine check verified, py.typed and all subpackages ship.

ats3v added 5 commits July 10, 2026 12:15
Self-hosted runners are unsafe on a public repo (fork PRs execute
arbitrary code on the runner). Restore the GitHub-hosted matrix so the
full supported Python range (>=3.9) is actually tested, and advertise
it in the trove classifiers.
- Fix AttributeError when constructing an inference wrapper with no API
  key: the env-lookup refactor stopped assigning self.auth_token before
  _warn_about_missing_api_key read it; warn and continue as in 0.1.x
- Map 429 to a new RateLimitError; keep TooManySandboxesError as an
  alias so sandbox code reads naturally
- Add SandboxWaitError base carrying .sandbox_id to
  SandboxTimeoutError/SandboxFailedError, so a failed
  Sandbox.create(wait=True) leaves a handle for cleanup
- parse_duration: round fractional seconds up so timeout=0.5 no longer
  silently truncates to 0 ("use server default")
- Document that the async client's connection pool binds to the first
  event loop (client docstring + README)
- Add deepinfra/utils/__init__.py so every subpackage is a regular
  package under the py.typed root
- Tests: missing-key regression, async retry loop (connect error, 502,
  retries exhausted), ReadTimeout -> APITimeoutError mapping,
  sandbox_id assertions, duration rounding (99 -> 105 tests)
Stopping is asynchronous server-side (~2s): stop() used to return with
the sandbox still running, so the documented sb.stop(); sb.start()
pattern raced the server and failed with 409. stop()/astop() now accept
wait/wait_timeout like start() and block until the state is "stopped"
(wait=False to fire-and-forget). Adds public wait_until_stopped/
await_until_stopped; the running/stopped wait loops share one
_wait_for_state implementation, and wait-timeout messages now name the
awaited state.

Also dedupe iter_ndjson/aiter_ndjson into a shared _parse_event helper,
stream a heartbeat through the async exec test path, and document
terminate()'s soft-delete semantics (verified live: sandboxes stay
fetchable by id as "deleted" briefly, vanish from list immediately).
@ats3v
ats3v requested a review from ovuruska July 20, 2026 12:53
@ats3v
ats3v merged commit 432dd90 into main Jul 20, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant