Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
E2B_API_KEY=
GUEST_TEMPLATE=
3 changes: 3 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Preserve attributed upstream server formatting; maintained files keep Git's whitespace checks.
template/files/server/main.py -whitespace
template/files/server/pyxcursor.py -whitespace
65 changes: 65 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
name: CI

on:
pull_request:
push:
branches: [main]

permissions:
contents: read

jobs:
verify:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
fetch-depth: 0
- uses: astral-sh/setup-uv@d0cc045d04ccac9d8b7881df0226f9e82c39688e # v6
with:
enable-cache: true
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
with:
python-version: "3.11"
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: "22"
cache: npm
- run: uv sync --locked --all-groups
- run: uv run pytest -q
- run: uv run ruff check .
- run: uv run ruff format --check .
- run: scripts/audit_python_dependencies.sh
- run: npm ci
- run: npm run typecheck
- run: npm test
- run: npm audit --omit=dev --audit-level=high
- name: Validate shell and JSON
run: |
bash -n runner/*.sh scripts/*.sh template/files/*.sh
python3 -m json.tool upstream.lock.json >/dev/null
python3 -m json.tool template/inputs.lock.json >/dev/null
for file in validation/*.json; do python3 -m json.tool "$file" >/dev/null; done
- run: uv run python scripts/check_public_tree.py

secrets:
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
fetch-depth: 0
- name: Scan complete Git history with Gitleaks
env:
GITLEAKS_VERSION: 8.30.1
GITLEAKS_SHA256: 551f6fc83ea457d62a0d98237cbad105af8d557003051f41f3e7ca7b3f2470eb
run: |
set -euo pipefail
archive="$RUNNER_TEMP/gitleaks_${GITLEAKS_VERSION}_linux_x64.tar.gz"
curl --fail --location --retry 3 --output "$archive" \
"https://github.com/gitleaks/gitleaks/releases/download/v${GITLEAKS_VERSION}/gitleaks_${GITLEAKS_VERSION}_linux_x64.tar.gz"
echo "${GITLEAKS_SHA256} ${archive}" | sha256sum --check --strict
tar -xzf "$archive" -C "$RUNNER_TEMP" gitleaks
"$RUNNER_TEMP/gitleaks" git --redact --verbose .
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,12 @@ dist/
*.egg-info/
results/
repro/
artifacts/
cache/
out/
runner/OSWorld/
*.log
*.pid
*.token
.coverage
htmlcov/
15 changes: 15 additions & 0 deletions NOTICE
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
OSWorld on E2B
Copyright 2026 E2B, Inc.

This product includes software derived from OSWorld at commit
7a17d3abc86d524420ea4ec96752f84d245fea74, Copyright 2024 XLANG NLP Lab,
licensed under the Apache License, Version 2.0.

The following files originate from OSWorld's desktop_env/server package:

- template/files/server/main.py: modified by E2B to tolerate transient AT-SPI objects with null
names, roles, action descriptions, or key bindings.
- template/files/server/pyxcursor.py: copied without functional changes.

The remaining E2B adapter, relay, Template, setup, and validation code is maintained in this
repository.
85 changes: 77 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,83 @@
# OSWorld on E2B

This repository hosts the E2B-native OSWorld environment port. The implementation is developed
through reviewed pull requests; the default branch remains a minimal public scaffold until the
initial release is approved.
This repository reconstructs OSWorld's Ubuntu desktop as an E2B Template and provides the provider,
relay, runner, and validation contracts needed to execute the pinned OSWorld benchmark. It is a
native port: it does not run OSWorld's qcow2 image or a nested hypervisor.

## Security
The implementation targets OSWorld commit `7a17d3abc86d524420ea4ec96752f84d245fea74`.
`upstream.lock.json` is the machine-readable source lock. The checkout is fetched into the ignored
`runner/OSWorld/` directory; no benchmark dataset or vendored repository is committed here.

Do not commit credentials, dotenv files, vendored OSWorld checkouts, generated results, or raw
trajectories. Report vulnerabilities according to [SECURITY.md](SECURITY.md).
## Architecture

## License
- `template/` builds an Ubuntu 22.04 GNOME desktop and installs OSWorld's guest-control surface.
- `realkit/` owns restricted-ingress E2B lifecycle, authenticated HTTP/WebSocket relay, fresh-task
replacement, and named runtime snapshots.
- `runner/` fetches and patches the exact OSWorld revision, runs OSWorld, and performs smoke or
environment-path validation.
- `validation/` selects a fixed cross-application task sample without embedding task contents.

Apache-2.0. See [LICENSE](LICENSE).
Every launch path requires an immutable `name:build_id` Template reference. A mutable Template name
is rejected before sandbox creation.

## Build identity

`template/inputs.lock.json` pins the OSWorld commit and the Ubuntu base image by platform-specific
digest. The guest Python environment is a complete transitive lock with artifact hashes, and npm
uses `package-lock.json`. The build command hashes those inputs, the Template sources, all copied
guest files, and the CPU and memory allocation into a deterministic recipe digest. It derives the
Template name `osworld-gnome-<digest-prefix>` from that digest; callers cannot supply a mutable
name.

The recipe is not byte-reproducible. Ubuntu packages, Google Chrome stable, and the VS Code Debian
download are resolved during the build as documented in `template/inputs.lock.json`. Each build
records the recipe digest, per-input digests, unresolved inputs, and E2B build ID in the ignored
`results/template-build.json` receipt. The guest also records the resolved Debian package inventory.
Only the receipt's `name:build_id` is an immutable runnable artifact identity; a matching recipe
digest alone does not identify identical output bytes.

## Local checks

Python 3.11+, Node.js 20+, `uv`, and npm are required.

```bash
uv sync --locked --all-groups
uv run pytest -q
uv run ruff check .
uv run ruff format --check .
scripts/audit_python_dependencies.sh
npm ci
npm run typecheck
npm test
python3 scripts/check_public_tree.py
```

The guest build and live validation require an E2B account and are deliberately separate from the
offline checks:

```bash
cp .env.example .env.local
export E2B_API_KEY='...'
npm run build
export GUEST_TEMPLATE='osworld-gnome-<recipe-digest-prefix>:<build-id-from-build-output>'

runner/setup.sh
uv run --with-requirements runner/OSWorld/requirements.txt \
--with-requirements runner/requirements-e2b.txt runner/validate.sh
```

Build receipts and validation output are written beneath ignored generated-output directories.
`PATH_PASS` means reset, setup, observation, action, and evaluator transport completed. It is not a
benchmark task pass and must not be reported as one.

See [Fidelity and validation](docs/FIDELITY.md) for the compatibility boundary and
[runner/README.md](runner/README.md) for operator commands.

## Security and licensing

The guest server exposes command execution, so the relay disables public traffic and authenticates
each proxied request with the sandbox traffic token. Never commit credentials, dotenv files,
vendored OSWorld checkouts, generated results, or raw trajectories. Report vulnerabilities through
the process in [SECURITY.md](SECURITY.md).

The repository is Apache-2.0. Portions derived from OSWorld are identified in [NOTICE](NOTICE).
57 changes: 57 additions & 0 deletions docs/FIDELITY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Fidelity and validation

## Compatibility boundary

This port reconstructs the OSWorld Linux desktop on Ubuntu 22.04 with GNOME 42, X11, a 1920×1080
framebuffer, the expected productivity applications, AT-SPI, Chrome CDP, VLC HTTP control, and the
OSWorld guest-control API. It is not a bit-for-bit copy of the reference qcow2 image.

The Template pins VS Code 1.91.1 and holds the Chrome package installed during each immutable build.
LibreOffice 7.3.7.2, GIMP 2.10.30, VLC 3.0.16, and VS Code 1.91.1 matched the pinned OSWorld image's
recorded application metadata in the source validation. The reference Chrome version, complete font
inventory, profiles, extensions, accounts, credentials, and proxy state have not been certified as
identical. Tasks that depend on those details can behave differently.

Tasks declaring `proxy=true` are excluded from the default environment-path manifest because this
port does not provision OSWorld's proxy service. `validation/proxy-required.json` retains one such
task as an explicit diagnostic boundary.

## Build identity boundary

The build recipe uses a platform-specific Ubuntu image digest, a pinned OSWorld commit, locked npm
dependencies, and a hash-locked transitive guest Python environment. Its content-derived name also
covers Template source, copied guest files, resource allocation, and those lock files.

The resulting filesystem is not claimed to be byte-reproducible. Ubuntu apt repositories and
Google's Chrome stable repository are rolling, and the versioned VS Code download is not verified
against a repository-owned digest. The exact Debian package versions resolved by a build are saved
inside that E2B artifact. The generated receipt keeps the recipe digest and E2B build ID separate:
the recipe digest identifies declared inputs, while `name:build_id` identifies the immutable
runnable artifact.

## Lifecycle contract

The host relay owns one sandbox at a time. Each OSWorld reset replaces it with a fresh sandbox from
the configured immutable Template. `save_state(name)` creates an E2B runtime snapshot;
`revert_to_snapshot(name)` creates a new sandbox from that snapshot. An unknown name such as
`init_state` creates a fresh sandbox from the immutable Template. Replacement kills the previous
sandbox after the new guest is ready, and shutdown is idempotent.

The guest control service can execute commands. Sandboxes therefore use restricted public ingress,
and the relay injects the per-sandbox traffic token into upstream HTTP and WebSocket requests while
redacting it from public relay state.

## Evidence contract

The fixed manifest covers 24 task IDs across Chrome, GIMP, LibreOffice, multiple applications, OS,
Thunderbird, VLC, and VS Code. For each task, the no-agent harness records reset, setup, screenshot,
accessibility, one scripted action, and evaluator execution. Generated evidence records the exact
OSWorld commit and immutable Template reference.

`PATH_PASS` proves only that the environment path completed. Evaluator score is recorded separately,
and only evaluator output from a real agent run can support a benchmark-success claim.

The source workspace recorded a successful live smoke and two 24/24 environment-path runs on
2026-07-16 against `osworld-gnome:62e8be41-4106-4850-96ea-afc822735b89`. Those generated artifacts
and sandbox identifiers are intentionally not published here. A candidate release must perform a
fresh Template build and live smoke; historical evidence does not satisfy that release gate.
19 changes: 19 additions & 0 deletions docs/SOURCE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Source and ownership

The E2B-native implementation was curated from its pre-publication working tree. That source tree
was not itself a Git repository. Only the following implementation families are part of this
repository:

- `template/build.ts`, `template/template.ts`, and the explicitly listed guest files;
- `realkit/*.py`;
- safe scripts and dependency pins under `runner/`;
- tests and task-ID-only validation manifests; and
- package locks and public operator documentation.

The nested `runner/OSWorld` checkout is not a source for repository history. It is fetched by
`runner/setup.sh` from `https://github.com/xlang-ai/OSWorld.git` at the commit recorded in
`upstream.lock.json`. Dotenv files, dependencies, build output, caches, local distributions,
generated evidence, raw trajectories, and experiment probes are outside the public artifact.

`template/files/server/main.py` and `template/files/server/pyxcursor.py` derive from the pinned
OSWorld server package. Their attribution and modifications are recorded in `NOTICE`.
Loading
Loading