Add Sandbox SDK on a unified httpx client (0.2.0)#19
Merged
Merged
Conversation
Self-hosted runners are unsafe on a public repo (fork PRs execute arbitrary code on the runner). Restore the GitHub-hosted matrix so the full supported Python range (>=3.9) is actually tested, and advertise it in the trove classifiers.
- Fix AttributeError when constructing an inference wrapper with no API
key: the env-lookup refactor stopped assigning self.auth_token before
_warn_about_missing_api_key read it; warn and continue as in 0.1.x
- Map 429 to a new RateLimitError; keep TooManySandboxesError as an
alias so sandbox code reads naturally
- Add SandboxWaitError base carrying .sandbox_id to
SandboxTimeoutError/SandboxFailedError, so a failed
Sandbox.create(wait=True) leaves a handle for cleanup
- parse_duration: round fractional seconds up so timeout=0.5 no longer
silently truncates to 0 ("use server default")
- Document that the async client's connection pool binds to the first
event loop (client docstring + README)
- Add deepinfra/utils/__init__.py so every subpackage is a regular
package under the py.typed root
- Tests: missing-key regression, async retry loop (connect error, 502,
retries exhausted), ReadTimeout -> APITimeoutError mapping,
sandbox_id assertions, duration rounding (99 -> 105 tests)
Stopping is asynchronous server-side (~2s): stop() used to return with the sandbox still running, so the documented sb.stop(); sb.start() pattern raced the server and failed with 409. stop()/astop() now accept wait/wait_timeout like start() and block until the state is "stopped" (wait=False to fire-and-forget). Adds public wait_until_stopped/ await_until_stopped; the running/stopped wait loops share one _wait_for_state implementation, and wait-timeout messages now name the awaited state. Also dedupe iter_ndjson/aiter_ndjson into a shared _parse_event helper, stream a heartbeat through the async exec test path, and document terminate()'s soft-delete semantics (verified live: sandboxes stay fetchable by id as "deleted" briefly, vanish from list immediately).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the Sandboxes feature (deepinfra/backend#3148) and modernizes the package around one unified client.
What changed
SandboxAPI (sandbox_api.py):create(blocks until running,"10m"-style timeouts),from_id,list(tags=...),exec/run_python(aggregate the NDJSON exec stream),fs.read/fs.write,stop/start/terminate, context managers, and async twins (acreate,aexec, ...) for every network method.stop()/start()block until the state transition completes (wait=Falseto fire-and-forget) — stop/terminate are asynchronous server-side (~2s), sosb.stop(); sb.start()used to 409.wait_until_stopped/await_until_stoppedare public alongsidewait_until_running.DeepInfraClient, rewritten in place onto httpx (wasrequests, single-URL, POST-only): sync + async, streaming, pooled connections, typed exceptions parsed from both{"error"}and FastAPI{"detail"}error shapes. Retries: connect-errors where marked, 502/503/504 only on GETs, never on non-idempotent POSTs (the old client retried 4xx ×5).deepinfra/deepinfra.exceptions: 429 maps toRateLimitError(TooManySandboxesErrorkept as an alias),MaxRetriesExceededErroris preserved as a subclass ofAPIConnectionError, andSandboxTimeoutError/SandboxFailedErrorcarry.sandbox_idso a failedSandbox.create(wait=True)still leaves a handle for cleanup.TextGeneration,Embeddings,AutomaticSpeechRecognition,TextToImage) but now run on the unified client;requests/requests-toolbeltdropped. Also fixesfrom deepinfra import TextGeneration, which was missing from the exports in 0.1.0.pyproject.toml(0.2.0, py>=3.9, deps httpx+pydantic only),py.typed, CI on GitHub-hosted runners with a 3.9–3.13 matrix (ruff, mypy, pytest), PyPI trusted publishing onv*tags with a tag==version guard, README/CHANGELOG/examples.Not in this PR (designed, lands next)
exec_stream,snapshot/from_snapshot,expose_port,fs.upload_dir— no stubs shipped; each lands behind its backend endpoint. The server side of exec/fs is stubbed in deepinfra/backend#3474; end-to-end verification follows once the implementation PR lands.Pre-publish audit round (2026-07-20)
A full review pass (core client, sandbox API, legacy wrappers, tests, packaging, docs) before the repo goes public. Fixed, each with regression tests:
AttributeErrorinstead of warning (refactor regression; the missing-key path had no test — now it does).parse_duration(0.5)silently truncated to0("use server default"); fractional seconds now round up.TooManySandboxesErrorfor inference rate limits too →RateLimitErrorintroduced (alias kept)..sandbox_id(create-leak cleanup path).ReadTimeout → APITimeoutErrormapping, exec-stream heartbeats on the async path.DeepInfraClientwhen callingasyncio.run()repeatedly.deepinfra/utils/got its missing__init__.py.Verification
Prod e2e (2026-07-20, real sandboxes against api.deepinfra.com): every deployed surface passes, sync + async — create (wait/no-wait), wait_until_running, refresh, list + tag filtering, from_id,
stop(wait=True)→start()(previously 409ed),stop(wait=False)+wait_until_stopped, terminate, both context managers, 404 mapping on bogus ids, and all four inference wrappers on the new client (including multipart ASR). Verified server semantics now documented: stop/terminate take effect ~2s after the call returns; terminate is a soft delete (sandbox stays fetchable by id as"deleted"briefly, disappears fromlistimmediately).exec/run_python/fsreturn501 Not implemented yetfrom prod — SDK raises a clean typed error; e2e for those is gated on the backend deploy, and the 0.2.0 tag should wait for a green run of that suite.Earlier prod e2e (2026-07-10): 24/24 lifecycle + break-attempt checks (path-traversal ids, empty id, junk/negative durations, unknown plan, non-string tags, instant wait-timeouts, double-stop), no sandboxes leaked. Break-testing found and fixed two bugs: sandbox ids are now URL-quoted in paths (
from_id("../models")escaped to/v1/models; now a clean 404 +ValueErroron empty id), andcreate(wait=False)populates fields with one GET instead of returning an empty-state handle. Legacy wrappers verified against live models (GLM-4.5-Air incl. full-URL endpoint form, gte-large, whisper-tiny.en file-path and raw-bytes multipart, sd3.5-medium); the 2024 response dataclasses crashed on fields the API has since added (request_id,words) — pre-existing 0.1.0 breakage, fixed with drift-tolerant parsing + regression test.Unit suite: 107 passed (respx-mocked, exercising the real request path), on Python 3.9 (oldest supported); mypy strict + ruff clean; wheel build +
twine checkverified,py.typedand all subpackages ship.