diff --git a/.agents/skills/build-from-issue/SKILL.md b/.agents/skills/build-from-issue/SKILL.md index 4eb34036b7..89d9e04215 100644 --- a/.agents/skills/build-from-issue/SKILL.md +++ b/.agents/skills/build-from-issue/SKILL.md @@ -1,6 +1,8 @@ --- name: build-from-issue description: Given a GitHub issue number, plan and implement the work described in the issue. Supports direct user requests and unattended queue processing through the `agent:*` workflow labels. Includes tests, documentation updates, and PR creation. Trigger keywords - build from issue, implement issue, work on issue, build issue, start issue. +metadata: + internal: true --- # Build From Issue diff --git a/.agents/skills/build-openshell-mxc-windows/SKILL.md b/.agents/skills/build-openshell-mxc-windows/SKILL.md index 954fcedf2f..5f0d617b08 100644 --- a/.agents/skills/build-openshell-mxc-windows/SKILL.md +++ b/.agents/skills/build-openshell-mxc-windows/SKILL.md @@ -1,6 +1,8 @@ --- name: build-openshell-mxc-windows description: Maintain and validate OpenShell's build-only Windows MSVC lane for x64 and ARM64. Use when working on Windows compilation, `windows:*` mise tasks, unsupported Windows compute-driver contracts, or Windows build reports. This skill does not implement Docker, Kubernetes, Podman, VM, MXC driver, policy translation, MSI, service, or supervisor runtime support on Windows. +metadata: + internal: true --- # Build OpenShell-MXC for Windows diff --git a/.agents/skills/create-github-issue/SKILL.md b/.agents/skills/create-github-issue/SKILL.md index 1ab92a6ede..2510b79272 100644 --- a/.agents/skills/create-github-issue/SKILL.md +++ b/.agents/skills/create-github-issue/SKILL.md @@ -1,6 +1,8 @@ --- name: create-github-issue description: Create GitHub issues using the gh CLI. Use when the user wants to create a new issue, report a bug, request a feature, or create a task in GitHub. Trigger keywords - create issue, new issue, file bug, report bug, feature request, github issue. +metadata: + internal: true --- # Create GitHub Issue diff --git a/.agents/skills/create-github-pr/SKILL.md b/.agents/skills/create-github-pr/SKILL.md index d98aba37f2..8e14bc5f9a 100644 --- a/.agents/skills/create-github-pr/SKILL.md +++ b/.agents/skills/create-github-pr/SKILL.md @@ -1,6 +1,8 @@ --- name: create-github-pr description: Create GitHub pull requests using the gh CLI. Use when the user wants to create a new PR, submit code for review, or open a pull request. Trigger keywords - create PR, pull request, new PR, submit for review, code review. +metadata: + internal: true --- # Create GitHub Pull Request diff --git a/.agents/skills/create-rfc/SKILL.md b/.agents/skills/create-rfc/SKILL.md index f767e47587..6df87ef315 100644 --- a/.agents/skills/create-rfc/SKILL.md +++ b/.agents/skills/create-rfc/SKILL.md @@ -1,6 +1,8 @@ --- name: create-rfc description: Create OpenShell RFC proposals in rfc/ from a design request. Use when the user asks to write, draft, start, create, or update an RFC, Request for Comments, architecture proposal, API proposal, process proposal, or cross-cutting design proposal that should follow the OpenShell RFC process and template. +metadata: + internal: true --- # Create RFC diff --git a/.agents/skills/create-spike/SKILL.md b/.agents/skills/create-spike/SKILL.md index 01afee91e6..baafd173da 100644 --- a/.agents/skills/create-spike/SKILL.md +++ b/.agents/skills/create-spike/SKILL.md @@ -1,6 +1,8 @@ --- name: create-spike description: Investigate a plain-language problem description by deeply exploring the codebase, then create a structured GitHub issue with technical findings. Prequel to build-from-issue — maps vague ideas to concrete, buildable issues. Trigger keywords - spike, investigate, explore, research issue, technical investigation, create spike, new spike, feasibility, codebase exploration. +metadata: + internal: true --- # Create Spike diff --git a/.agents/skills/fix-security-issue/SKILL.md b/.agents/skills/fix-security-issue/SKILL.md index ea3a12bfbb..1452cbe66c 100644 --- a/.agents/skills/fix-security-issue/SKILL.md +++ b/.agents/skills/fix-security-issue/SKILL.md @@ -1,6 +1,8 @@ --- name: fix-security-issue description: Implement a fix for a reviewed security issue. Takes a directly requested issue number or scans for issues labeled `topic:security` and `agent:implementation-requested`. Reads the security review from the issue comments and implements the remediation plan. Trigger keywords - fix security issue, remediate security, implement security fix, patch vulnerability. +metadata: + internal: true --- # Fix Security Issue diff --git a/.agents/skills/helm-dev-environment/SKILL.md b/.agents/skills/helm-dev-environment/SKILL.md index c577cd577e..780cf8b9c6 100644 --- a/.agents/skills/helm-dev-environment/SKILL.md +++ b/.agents/skills/helm-dev-environment/SKILL.md @@ -1,6 +1,8 @@ --- name: helm-dev-environment description: Start up, tear down, and configure the local Kubernetes development environment for OpenShell. Uses k3d (Docker-backed k3s) + Skaffold + Helm. Covers cluster lifecycle, optional add-ons (Keycloak OIDC, Envoy Gateway), HA testing, and port mappings. Trigger keywords - local k8s, local cluster, k3d, skaffold, helm dev, start cluster, stop cluster, tear down cluster, delete cluster, create cluster, helm:k3s, helm:skaffold, local dev environment, dev cluster, k8s dev, envoy gateway local, keycloak local, high availability, HA. +metadata: + internal: true --- # Helm Dev Environment diff --git a/.agents/skills/launch-openshell-gator/SKILL.md b/.agents/skills/launch-openshell-gator/SKILL.md index c8e9fc9489..fb9ada09b0 100644 --- a/.agents/skills/launch-openshell-gator/SKILL.md +++ b/.agents/skills/launch-openshell-gator/SKILL.md @@ -1,6 +1,8 @@ --- name: launch-openshell-gator description: Launch and supervise OpenShell gator agents. Use when starting gator on issues or PRs, checking gator sandboxes, building the gator sandbox image, restarting stuck gators, inspecting gator logs, or experimenting with gator harness/model overrides. Trigger keywords - launch gator, start gator, run gator, gator sandbox, supervised gator, gator logs, restart gator. +metadata: + internal: true --- # Launch OpenShell Gator diff --git a/.agents/skills/openshell-cli/cli-reference.md b/.agents/skills/openshell-cli/cli-reference.md deleted file mode 100644 index a3dc91c958..0000000000 --- a/.agents/skills/openshell-cli/cli-reference.md +++ /dev/null @@ -1,665 +0,0 @@ -# OpenShell CLI Reference - -Quick-reference for the `openshell` command-line interface. For workflow guidance, see [SKILL.md](SKILL.md). - -> **Self-teaching**: If a command or flag is not listed here, use `openshell --help` to discover it. The CLI has comprehensive built-in help at every level. - -## Global Options - -| Flag | Description | -|------|-------------| -| `-v`, `--verbose` | Increase verbosity (`-v` = info, `-vv` = debug, `-vvv` = trace) | -| `-g`, `--gateway ` | Gateway to operate on. Also settable via `OPENSHELL_GATEWAY` env var. Falls back to active gateway in `~/.config/openshell/active_gateway`. | -| `--gateway-endpoint ` | Connect directly to a gateway endpoint without looking up stored metadata. Also settable via `OPENSHELL_GATEWAY_ENDPOINT`. | -| `--gateway-insecure` | Skip TLS certificate verification. Also settable via `OPENSHELL_GATEWAY_INSECURE`; use only for trusted development endpoints. | -| `--color ` | `auto` (default), `always`, or `never`. `auto` decides per stream, so a redirected stream is plain text while a stream still on the terminal stays styled. Covers tables, `-v` log lines, progress spinners, prompts, and error messages. Also settable via `OPENSHELL_COLOR`. | - -## Environment Variables - -| Variable | Description | -|----------|-------------| -| `OPENSHELL_GATEWAY` | Override active gateway name (same as `--gateway`) | -| `OPENSHELL_GATEWAY_ENDPOINT` | Connect directly to a gateway endpoint (same as `--gateway-endpoint`) | -| `OPENSHELL_GATEWAY_INSECURE` | Skip TLS verification when set (same as `--gateway-insecure`) | -| `OPENSHELL_COLOR` | When to colorize output: `auto`, `always`, `never` (same as `--color`) | -| `NO_COLOR` | Disable colorized output when set to any non-empty value ([no-color.org](https://no-color.org)) | -| `FORCE_COLOR` | Force colorized output when set to any non-empty value ([force-color.org](https://force-color.org)) | -| `OPENSHELL_SANDBOX_POLICY` | Path to default sandbox policy YAML (fallback when `--policy` is not provided) | -| `OPENSHELL_COMMUNITY_REGISTRY` | Override the community sandbox image registry prefix used by `sandbox create --from ` | -| `OPENSHELL_THEME` | TUI theme: `auto`, `dark`, or `light` | - ---- - -## Complete Command Tree - -``` -openshell -├── gateway -│ ├── add [opts] -│ ├── login [name] -│ ├── logout [name] -│ ├── remove [name] -│ ├── info [--name] -│ ├── list -│ └── select [name] -├── status -├── whoami [--output ] -├── inference -│ ├── set --provider --model -│ ├── update [--provider] [--model] -│ └── get -├── sandbox -│ ├── create [opts] [-- CMD...] -│ ├── get [name] -│ ├── list [opts] -│ ├── stop [name] -│ ├── start [name] -│ ├── delete [name]... [--all] -│ ├── exec [--name ] [opts] -- CMD... -│ ├── connect [name] [--editor ] -│ ├── upload [dest] -│ ├── download [dest] -│ ├── ssh-config [name] -│ └── provider -│ ├── list [name] [-o table|yaml|json] -│ ├── attach -│ └── detach -├── forward -│ ├── start [name] [-d] -│ ├── stop [name] -│ ├── list [-o table|yaml|json] -│ └── service [name] --target-port [opts] -├── service -│ ├── expose [service] -│ ├── list [sandbox] [-o table|yaml|json] -│ ├── get [service] -│ └── delete [service] -├── logs [name] [opts] -├── policy -│ ├── set [name] --policy [--global] [--wait] -│ ├── update [name] [opts] -│ ├── get [name] [--full|--base] [--global] -│ ├── list [name] [--global] [-o table|yaml|json] -│ ├── delete --global -│ └── prove --policy --credentials [opts] -├── settings -│ ├── get [name] [--global] -│ ├── set [name] --key --value [--global] -│ └── delete [name] --key [--global] -├── rule (advanced; hidden from top-level help) -│ ├── get [name] [--status ] -│ ├── approve [name] --chunk-id -│ ├── reject [name] --chunk-id [--reason ] -│ ├── approve-all [name] [--include-security-flagged] -│ ├── clear [name] -│ └── history [name] -├── provider -│ ├── create --name --type [opts] -│ ├── refresh -│ │ ├── status [opts] -│ │ ├── configure [opts] -│ │ ├── rotate --credential-key -│ │ └── delete --credential-key -│ ├── get -│ ├── list [opts] -│ ├── list-profiles [opts] -│ ├── profile -│ │ ├── export [opts] -│ │ ├── import (--file |--from ) -│ │ ├── update --file -│ │ ├── lint (--file |--from ) -│ │ └── delete -│ ├── update [opts] -│ └── delete ... -├── workspace -│ ├── create -│ ├── get -│ ├── list [opts] -│ ├── delete ... -│ └── member -│ ├── add --workspace --subject --role -│ ├── remove --workspace --subject -│ └── list --workspace [-o table|yaml|json] -├── doctor -│ └── check -├── term -├── completions -└── ssh-proxy [opts] -``` - ---- - -## Gateway Commands - -### `openshell gateway add ` - -Register an existing gateway endpoint. - -| Flag | Description | -|------|-------------| -| `--name ` | Gateway name | -| `--local` | Register a local mTLS gateway; with HTTP, store a local plaintext registration | -| `--remote ` | Register a remote mTLS gateway over SSH; with HTTP, store a remote plaintext registration | -| `--oidc-issuer ` | Register an OIDC-authenticated gateway | -| `--oidc-client-id ` | OIDC client ID (default: `openshell-cli`; requires `--oidc-issuer`) | -| `--oidc-audience ` | OIDC API audience (requires `--oidc-issuer`) | -| `--oidc-scopes ` | Space-separated OAuth2 scopes (requires `--oidc-issuer`) | - -Examples: - -- `openshell gateway add http://127.0.0.1:8080 --local --name local` -- `openshell gateway add https://gateway.example.com --name production` -- `openshell gateway add ssh://user@gateway.example.com:8080 --name remote` - -An `http://` endpoint is direct plaintext. A plain `https://` endpoint uses edge authentication. `--local` and `--remote` select mTLS registration modes when used with HTTPS; required certificates must already exist. An `ssh://` endpoint is shorthand for a remote gateway. - -### `openshell gateway remove [name]` - -Remove a local gateway registration. This removes CLI metadata and stored auth tokens only; package managers, systemd, Helm, Docker, and other platform tools still own the gateway process. - -### `openshell gateway login [name]` - -Refresh browser-based authentication for an edge-authenticated or OIDC gateway. - -### `openshell gateway logout [name]` - -Clear locally stored OIDC or edge credentials for a gateway. - -### `openshell gateway info` - -Show gateway details: endpoint, auth mode, and remote host metadata when present. - -| Flag | Description | -|------|-------------| -| `--name ` | Gateway name (defaults to active) | - -### `openshell gateway select [name]` - -Set the active gateway. Writes to `~/.config/openshell/active_gateway`. Without a name, opens an interactive chooser on a TTY or lists gateways in non-interactive mode. - -### `openshell gateway list` - -List registered gateways and mark the active one. `--output table|yaml|json` selects the format. - ---- - -## Doctor Commands - -### `openshell doctor check` - -Validate local Docker prerequisites for standalone gateway development. For -package-managed or Helm gateways, use `systemctl`, `journalctl`, `kubectl`, and -`helm` directly. - ---- - -## Status Command - -### `openshell status` - -Show server connectivity, authentication status, and version for the active -gateway. Connectivity uses the public health RPC; authentication is checked -with the protected gateway-info capability query and can fail while the gateway -remains connected. - -### `openshell whoami` - -Show the authenticated user identity: subject, display name, roles, scopes, and -identity provider. Requires an authenticated gateway connection. - -| Flag | Description | -|------|-------------| -| `--output ` | Output format: `table` (default), `json`, or `yaml` | - ---- - -## Sandbox Commands - -### `openshell sandbox create [OPTIONS] [-- COMMAND...]` - -Create a sandbox through the selected gateway and launch its canonical main -process. By default, the CLI attaches to that retained process after the -sandbox becomes ready. A trailing command defines the canonical main process; -without one, the default is `/bin/bash -l` with a PTY. Explicit commands remain -foreground in non-interactive automation: stdout and stderr stream to the -caller and the CLI returns the command's exact status. Exit 0 leaves -`Completed`; nonzero leaves `Error/MainProcessFailed`. -Starting either retained terminal result invalidates SSH sessions from the -previous runtime generation. - -| Flag | Description | -|------|-------------| -| `--name ` | Sandbox name (auto-generated if omitted) | -| `--from ` | Community name, Dockerfile path, directory, or image reference (BYOC) | -| `--no-keep` | Delete the sandbox after main output and the result drain | -| `--detach` | Start the canonical main process without attaching | -| `--editor vscode|cursor` | Launch a remote editor and keep the sandbox alive | -| `--gpu [COUNT]` | Request the driver's default GPU selection or a specific count | -| `--cpu ` | CPU limit (for example: `500m`, `1`, `2.5`) | -| `--memory ` | Memory limit (for example: `512Mi`, `4Gi`, `8G`) | - -`--detach` adds no attachment grace period: the sandbox reports the canonical -process result immediately when it exits. Foreground creation declares one -expected main-process SSH attachment; cleanup finalizes after that connection -drains and closes naturally. -| `--driver-config-json ` | Experimental driver-keyed configuration object | -| `--provider ` | Provider to attach (repeatable) | -| `--policy ` | Custom policy YAML; overrides the built-in default and `OPENSHELL_SANDBOX_POLICY` | -| `--forward <[BIND:]PORT>` | Start a local port forward and keep the sandbox alive | -| `--tty`, `--no-tty` | Force or disable pseudo-terminal allocation | -| `--auto-providers` | Auto-create missing providers from local credentials | -| `--no-auto-providers` | Never auto-create providers; error if a required provider is missing | -| `--label ` | Attach a label (repeatable) | -| `--env ` | Inject an environment variable (repeatable) | -| `--approval-mode manual|auto` | Handle agent-authored policy proposals; default: `manual` | -| `--upload [:]` | Upload local files to the working directory or an explicit destination (repeatable) | -| `--no-git-ignore` | Disable `.gitignore` filtering for `--upload` | -| `[-- COMMAND...]` | Canonical main command (defaults to `/bin/bash -l`) | - -`--upload` cannot be combined with a trailing main command because uploads -currently complete after the canonical process starts. Create the default -scratch sandbox, upload files, then use `sandbox exec`, or build the files into -the image. - -### `openshell sandbox get [name]` - -Show sandbox details and the active policy. Metadata identifies sandbox or global policy source and the corresponding revision. The name defaults to the last-used sandbox. - -| Flag | Description | -|------|-------------| -| `--policy-only` | Print only the active policy YAML to stdout | - -### `openshell sandbox list` - -| Flag | Default | Description | -|------|---------|-------------| -| `--limit ` | 100 | Maximum sandboxes | -| `--offset ` | 0 | Pagination offset | -| `--ids` | false | Print only sandbox IDs | -| `--names` | false | Print only sandbox names | -| `--selector ` | none | Filter by `key1=value1,key2=value2` | -| `--output table|yaml|json` | `table` | Output format | - -### `openshell sandbox delete [NAME]...` - -Delete one or more named sandboxes, or use `--all`. Deletion stops background port forwards. - -### `openshell sandbox stop [name]` - -Stop sandbox compute while retaining the sandbox and persistent workspace. The -name defaults to the last-used sandbox. The command stops background forwards -and waits for the `Stopped` phase. - -### `openshell sandbox start [name]` - -Start a stopped, failed, or completed sandbox and wait for `Ready`. This -launches a fresh canonical-main instance. The name defaults to the last-used -sandbox. - -### `openshell sandbox exec [OPTIONS] -- COMMAND...` - -Execute a command through the gRPC exec endpoint, stream its output, and exit with the remote command's exit code. - -| Flag | Default | Description | -|------|---------|-------------| -| `-n`, `--name ` | last-used | Sandbox name | -| `--workdir ` | none | Working directory in the sandbox | -| `--timeout ` | 0 | Command timeout; `0` disables it | -| `--tty`, `--no-tty` | auto | Force or disable a pseudo-terminal | -| `--env ` | none | Command environment variable (repeatable) | - -### `openshell sandbox connect [name]` - -Attach to the sandbox's retained canonical main process. Disconnecting leaves -the process running. Reconnecting targets the same process instance and -replays recent output. Use `sandbox exec --tty -- /bin/bash -l` when you need a -new shell. The name defaults to the last-used sandbox. - -`--editor vscode|cursor` launches a supported remote editor instead of -attaching to the canonical main process. - -### `openshell sandbox upload [dest]` - -Upload files using tar-over-SSH. The CLI discovers the canonical remote working directory when the destination is omitted. A named directory merges into an existing directory of the same name, overwriting matching entries without deleting unrelated entries. `.gitignore` filtering is enabled unless `--no-git-ignore` is passed. - -### `openshell sandbox download [dest]` - -Download files using tar-over-SSH. The sandbox source may be relative to the canonical remote working directory or an absolute path within it. The local destination defaults to `.`. - -### `openshell sandbox ssh-config [name]` - -Print an SSH config `Host` block. The name defaults to the last-used sandbox. - -### `openshell sandbox provider` - -Manage providers on an existing sandbox: - -- `openshell sandbox provider list [name] [--output table|yaml|json]` -- `openshell sandbox provider attach ` -- `openshell sandbox provider detach ` - -Structured list output contains `name`, `type`, and sorted `credential_keys` -and `config_keys` arrays. It excludes all credential, handle, and config values. - ---- - -## Port Forwarding Commands - -### `openshell forward start [name]` - -Start forwarding a local port to a sandbox. - -| Flag | Description | -|------|-------------| -| `` | `[bind_address:]port`; the port is used locally and remotely | -| `[name]` | Sandbox name (defaults to last-used) | -| `-d`, `--background` | Run in background | - -### `openshell forward stop [name]` - -Stop a background port forward. When the sandbox name is omitted, it is inferred from active forwards. - -### `openshell forward list [--output table|yaml|json]` - -List all tracked port forwards. Table output shows the sandbox, bind address, -port, PID, and status. JSON and YAML output expose `sandbox`, `bind_address`, -`port`, `pid`, and the boolean `alive`; an empty result is an empty collection. - -| Flag | Default | Description | -|------|---------|-------------| -| `-o`, `--output ` | `table` | Output format: `table`, `yaml`, or `json` | - -### `openshell forward service [name] --target-port ` - -Forward a local TCP port to a loopback service inside a sandbox over the gRPC relay. - -| Flag | Default | Description | -|------|---------|-------------| -| `--target-port ` | required | Service port inside the sandbox | -| `--target-host ` | `127.0.0.1` | Loopback service host | -| `--local <[BIND:]PORT>` | target port | Local bind; port `0` requests dynamic assignment | - ---- - -## Service Commands - -Gateway-managed HTTP service endpoints: - -- `openshell service expose [service]` -- `openshell service list [sandbox] [--limit N] [--offset N] [--output table|yaml|json]` -- `openshell service get [service]` -- `openshell service delete [service]` - -Structured list records contain `workspace`, `sandbox`, `service`, -`target_port`, and `url`. Empty lists serialize as empty collections. - ---- - -## Logs Command - -### `openshell logs [name]` - -View sandbox logs. Supports one-shot and streaming. - -| Flag | Default | Description | -|------|---------|-------------| -| `-n ` | 200 | Number of log lines | -| `--tail` | false | Stream live logs | -| `--since ` | none | Only show logs from this duration ago (e.g., `5m`, `1h`) | -| `--source ` | `all` | Filter: `gateway`, `sandbox`, or `all` (repeatable) | -| `--level ` | none | Minimum level: `error`, `warn`, `info`, `debug`, `trace` | - -The sandbox name defaults to the last-used sandbox. - ---- - -## Policy Commands - -### `openshell policy update [name]` - -Incrementally merge live network policy changes into the current sandbox policy when the selected compute driver supports live updates. Multiple flags in one invocation are applied as one atomic batch and create at most one new revision. Use `--wait` to verify that the active runtime loaded the new revision. - -| Flag | Default | Description | -|------|---------|-------------| -| `--add-endpoint ` | repeatable | `host:port[:access[:protocol[:enforcement[:options]]]]`. Adds or merges an endpoint. | -| `--remove-endpoint ` | repeatable | `host:port`. Removes the endpoint or just the requested port from a multi-port endpoint. | -| `--add-allow ` | repeatable | `host:port:METHOD:path_glob`. Adds REST or WebSocket allow rules. | -| `--add-deny ` | repeatable | `host:port:METHOD:path_glob`. Adds REST or WebSocket deny rules. | -| `--remove-rule ` | repeatable | Deletes a named network rule. | -| `--binary ` | repeatable | Adds binaries to each `--add-endpoint` rule. Valid only with `--add-endpoint`. | -| `--rule-name ` | none | Overrides the generated rule name. Valid only when exactly one `--add-endpoint` is provided. | -| `--dry-run` | false | Preview the merged policy locally without sending an update to the gateway. | -| `--wait` | false | Wait for the sandbox to confirm the new policy revision is loaded. | -| `--timeout ` | 60 | Timeout for `--wait`. | - -Notes: - -- The sandbox name defaults to the last-used sandbox. -- `--add-endpoint` options are comma-separated: `allowed-ip=`, `websocket-credential-rewrite`, `request-body-credential-rewrite`, and `allow-uninspected-credentials`. The last option is a security-sensitive exception for provider-credentialed L4-only, `tls: skip`, or otherwise uninspectable traffic. -- `protocol` accepts `tcp` for explicit L4-only host/port policy. It has the - same payload-handling behavior as omitting the protocol, but it requires a - valid DNS hostname and rejects hostless `allowed_ips` or literal-IP - selectors. It cannot be combined with `access`, `rules`, or L7 enforcement - options. -- `--add-allow` and `--add-deny` operate on REST and WebSocket endpoints. Use full YAML for JSON-RPC, MCP, SQL, or other policy structure. -- `--wait` cannot be combined with `--dry-run`. -- Use `policy set` when replacing the full policy or changing static sections. - -### `openshell policy set [name] --policy ` - -Replace the full policy on a live sandbox when the selected compute driver supports live updates. Only dynamic policy fields can be changed at runtime. Use `--wait` to verify that the active runtime loaded the new revision. - -| Flag | Default | Description | -|------|---------|-------------| -| `--policy ` | -- | Path to policy YAML (required) | -| `--global` | false | Apply as the gateway-global policy | -| `--yes` | false | Skip confirmation for a global update | -| `--wait` | false | Wait for sandbox to confirm policy is loaded | -| `--timeout ` | 60 | Timeout for `--wait` | - -Exit codes with `--wait`: 0 = loaded, 1 = failed, 124 = timeout. - -### `openshell policy get [name]` - -Show the current effective sandbox policy or stored global policy. - -| Flag | Default | Description | -|------|---------|-------------| -| `--rev ` | 0 | Show a stored revision; `0` shows the current effective policy | -| `--full` | false | Include the effective policy payload and provider-composed entries | -| `--base` | false | Include the base policy payload without provider-composed entries | -| `--output table|json` | `table` | Output format | -| `--global` | false | Show the global policy revision | - -### `openshell policy list [name]` - -List policy revision history (version, hash, status, created, error). - -| Flag | Default | Description | -|------|---------|-------------| -| `--limit ` | 20 | Max revisions to return | -| `--global` | false | List global policy revisions | -| `-o`, `--output ` | `table` | Output format: `table`, `yaml`, or `json` | - -Structured records use the policy metadata contract from `policy get`: scope, -sandbox when applicable, version, full hash, normalized status, and available -timestamps, load error, and provenance. - -### `openshell policy delete --global` - -Delete the global policy lock and restore sandbox-level policy control. `--yes` skips confirmation. - -### `openshell policy prove` - -Prove policy properties or find counterexamples. - -| Flag | Description | -|------|-------------| -| `--policy ` | Policy YAML (required) | -| `--credentials ` | Credential descriptor YAML (required) | -| `--registry ` | Capability registry directory (defaults to bundled) | -| `--accepted-risks ` | Accepted-risks YAML | -| `--compact` | One-line-per-finding output | - -### `openshell rule` (advanced) - -Review agent-authored network rule proposals. This command group is intentionally hidden from top-level help but is part of the policy-advisor workflow. - -- `openshell rule get [name] [--status pending|approved|rejected]` -- `openshell rule approve [name] --chunk-id ` -- `openshell rule reject [name] --chunk-id [--reason ]` -- `openshell rule approve-all [name] [--include-security-flagged]` -- `openshell rule clear [name]` -- `openshell rule history [name]` - -Sandbox names default to the last-used sandbox. The CLI fetches and submits each proposal's current review token; a changed live candidate remains pending until it is reviewed again. Bulk approval of security-flagged proposals requires explicit `--include-security-flagged`. - ---- - -## Workspace Member Commands - -### `openshell workspace member list --workspace ` - -List workspace members. Add `--output table|yaml|json` to select the format. -Structured records contain `subject` and a normalized `role` of `admin`, -`user`, or `unknown`; empty lists serialize as empty collections. - ---- - -## Settings Commands - -Settings support sandbox and gateway-global scopes: - -- `openshell settings get [name] [--global] [--json]` -- `openshell settings set [name] --key --value [--global] [--yes]` -- `openshell settings delete [name] --key [--global] [--yes]` - -Sandbox names default to the last-used sandbox. Global mutations prompt unless `--yes` is passed. - ---- - -## Provider Commands - -Provider types are defined by built-in and custom provider profiles. Use `openshell provider list-profiles` to discover the selected gateway's current inventory. - -### `openshell provider create --name --type ` - -Create a provider configuration. - -| Flag | Description | -|------|-------------| -| `--name ` | Provider name (required) | -| `--type ` | Provider type (required) | -| `--from-existing` | Load credentials and config from local state | -| `--credential KEY[=VALUE]` | Credential pair. Bare `KEY` reads from env var. Repeatable. | -| `--from-gcloud-adc` | Load a compatible credential from gcloud Application Default Credentials | -| `--runtime-credentials` | Resolve required credentials at runtime in the gateway or sandbox | -| `--config KEY=VALUE` | Config key/value pair. Repeatable. | - -Credential-source flags conflict with one another when supplied. A credential -source is optional when the selected profile declares no required static -credentials or can resolve its required credentials at runtime. - -### `openshell provider get ` - -Show provider details (id, name, type, credential keys, config keys). - -### `openshell provider list` - -List providers in a table. - -| Flag | Default | Description | -|------|---------|-------------| -| `--limit ` | 100 | Max providers | -| `--offset ` | 0 | Pagination offset | -| `--names` | false | Print only names | -| `--output table|yaml|json` | `table` | Output format | - -### `openshell provider update ` - -Update an existing provider without changing its type. - -| Flag | Description | -|------|-------------| -| `--from-existing` | Rediscover local credentials and config | -| `--credential KEY[=VALUE]` | Update a credential (repeatable) | -| `--config KEY=VALUE` | Update config (repeatable) | -| `--credential-expires-at KEY=TIMESTAMP` | Set or clear credential expiry; accepts epoch milliseconds or RFC3339, and `0` clears | - -### `openshell provider delete ...` - -Delete one or more providers by name. - -### Provider profiles - -- `openshell provider list-profiles [--output table|yaml|json]` -- `openshell provider profile export [--output table|yaml|json]` -- `openshell provider profile import (--file |--from )` -- `openshell provider profile update --file ` -- `openshell provider profile lint (--file |--from )` -- `openshell provider profile delete ...` - -### Provider credential refresh - -- `openshell provider refresh status [--credential-key ]` -- `openshell provider refresh rotate --credential-key ` -- `openshell provider refresh delete --credential-key ` - -`provider refresh configure ` accepts: - -| Flag | Description | -|------|-------------| -| `--credential-key ` | Injectable credential key (required) | -| `--strategy ` | `oauth2-refresh-token`, `oauth2-client-credentials`, or `google-service-account-jwt` | -| `--material KEY=VALUE` | Non-secret refresh material (repeatable) | -| `--secret-material-env KEY[=ENVVAR]` | Secret refresh material read from the CLI environment (repeatable) | -| `--secret-material-key KEY` | Mark a supplied material key secret (repeatable) | -| `--credential-expires-at TIMESTAMP` | Current credential expiry in epoch milliseconds or RFC3339 | - ---- - -## Inference Commands - -### `openshell inference set` - -Configure the gateway's user-facing `inference.local` route or the platform-only system route. Provider and model are required. - -| Flag | Default | Description | -|------|---------|-------------| -| `--provider ` | -- | Provider record name (required) | -| `--model ` | -- | Model identifier to use for generation requests (required) | -| `--system` | false | Configure the system inference route | -| `--no-verify` | false | Skip endpoint verification before saving | -| `--timeout ` | 0 | Request timeout; `0` uses the 60-second default | - -### `openshell inference update` - -Partially update the selected inference route. - -| Flag | Default | Description | -|------|---------|-------------| -| `--provider ` | unchanged | Provider record name | -| `--model ` | unchanged | Model identifier | -| `--system` | false | Target the system inference route | -| `--no-verify` | false | Skip endpoint verification before saving | -| `--timeout ` | unchanged | Request timeout; `0` uses the 60-second default | - -### `openshell inference get` - -Show both inference routes. `--system` shows only the system route. - ---- - -## Other Commands - -### `openshell term` - -Launch the OpenShell interactive TUI. `--theme auto|dark|light` overrides `OPENSHELL_THEME`. - -### `openshell completions ` - -Generate shell completion scripts. Supported shells: `bash`, `fish`, `zsh`, `powershell`. - -### `openshell ssh-proxy` - -SSH proxy used as a `ProxyCommand`. Not typically invoked directly. diff --git a/.agents/skills/review-github-pr/SKILL.md b/.agents/skills/review-github-pr/SKILL.md index 21cd131885..de56cb4770 100644 --- a/.agents/skills/review-github-pr/SKILL.md +++ b/.agents/skills/review-github-pr/SKILL.md @@ -1,6 +1,8 @@ --- name: review-github-pr description: Review a GitHub pull request by summarizing its diff and key design decisions. Use when the user wants to review a PR, understand changes in a branch, or get a code review summary. Trigger keywords - review PR, review pull request, summarize PR, summarize diff, code review, review branch, PR summary, diff summary. +metadata: + internal: true --- # Review GitHub Pull Request diff --git a/.agents/skills/review-security-issue/SKILL.md b/.agents/skills/review-security-issue/SKILL.md index 03dc2974be..efb054df80 100644 --- a/.agents/skills/review-security-issue/SKILL.md +++ b/.agents/skills/review-security-issue/SKILL.md @@ -1,6 +1,8 @@ --- name: review-security-issue description: Given a GitHub issue, review the issue for security implications. You'll make a determination if the claim in the issue is legitimate and should be addressed or will be a "won't fix." Trigger keywords - security issue, review security ticket, review security issue. +metadata: + internal: true --- # Review Security Issue diff --git a/.agents/skills/sbom/SKILL.md b/.agents/skills/sbom/SKILL.md index 6996ffcf66..e5c3e48c20 100644 --- a/.agents/skills/sbom/SKILL.md +++ b/.agents/skills/sbom/SKILL.md @@ -1,6 +1,8 @@ --- name: sbom description: Generate and manage Software Bill of Materials (SBOMs) for the OpenShell project. Covers SBOM generation with Syft, license resolution via public registries, and CSV export for compliance review. Trigger keywords - SBOM, sbom, bill of materials, license audit, license resolution, generate sbom, sbom csv, dependency license, supply chain, license scan. +metadata: + internal: true --- # SBOM Generation and License Resolution diff --git a/.agents/skills/sync-agent-infra/SKILL.md b/.agents/skills/sync-agent-infra/SKILL.md index 5dfbef12c8..184ce71d5b 100644 --- a/.agents/skills/sync-agent-infra/SKILL.md +++ b/.agents/skills/sync-agent-infra/SKILL.md @@ -1,6 +1,8 @@ --- name: sync-agent-infra description: Detect and fix drift across agent-first infrastructure files. Ensures skill inventories, workflow chains, architecture tables, issue/PR templates, and cross-references stay consistent when skills, crates, or workflows change. Run after adding, removing, or renaming skills or components. Trigger keywords - sync agent infra, sync skills, update agent docs, check agent consistency, agent infra drift, sync contributing, sync agents. +metadata: + internal: true --- # Sync Agent Infrastructure @@ -12,13 +14,13 @@ Detect and fix drift across the agent-first infrastructure files. These files re | `AGENTS.md` | Project identity, workflow chains, architecture overview, issue/PR conventions, skill maintenance pointer | | `CONTRIBUTING.md` | Skills table, workflow chains, "When to Open an Issue" guidance, skill references | | `CONTRIBUTING.md` issue lifecycle section | Human-facing issue states, roadmap decisions, acceptance signals, and direct-versus-queued agent ownership | -| `README.md` | "Built With Agents" section, "Explore with your agent" skill references | +| `README.md` | "Use OpenShell with Your Agent" and "Built With Agents" sections | | `.github/ISSUE_TEMPLATE/bug_report.yml` | Skill name references in diagnostic guidance | | `.github/ISSUE_TEMPLATE/feature_request.yml` | Skill name references in investigation guidance | | `.github/ISSUE_TEMPLATE/config.yml` | Contact link text referencing skills | | `.github/workflows/issue-triage.yml` | Comment text referencing skills | | `.agents/skills/triage-issue/SKILL.md` | Skill name references in gate check and diagnosis steps | -| `.agents/skills/openshell-cli/SKILL.md` | Companion skills table | +| `skills/*/SKILL.md` | Standalone user instructions and links to documentation, included files, and related skills | | `.agents/skills/create-github-pr/SKILL.md` | Pre-PR agent infrastructure check | | `.agents/skills/review-github-pr/SKILL.md` | Review-time agent infrastructure check | | `.agents/skills/build-from-issue/SKILL.md` | Label awareness and pre-commit agent infrastructure check | @@ -26,7 +28,7 @@ Detect and fix drift across the agent-first infrastructure files. These files re ## When to Run -- After adding, removing, or renaming a skill in `.agents/skills/` +- After adding, removing, renaming, or moving a skill in `skills/` or `.agents/skills/` - After adding, removing, or renaming a crate in `crates/` - After changing workflow chain relationships between skills - After changing which product or development areas a skill covers @@ -35,7 +37,7 @@ Detect and fix drift across the agent-first infrastructure files. These files re ## Skill Maintenance Map -Use this map when product behavior, commands, or development workflows change. It is a routing aid, not an exhaustive dependency list. Search `.agents/skills/` for the changed command, field, component, or workflow before concluding that no other skill needs an update. +Use this map when product behavior, commands, or development workflows change. It is a routing aid, not an exhaustive dependency list. Search both `skills/` and `.agents/skills/` for the changed command, field, component, or workflow before concluding that no other skill needs an update. | Change area | Skills to review | |---|---| @@ -53,7 +55,7 @@ Use this map when product behavior, commands, or development workflows change. I | PR template, review conventions, or vouch behavior | `create-github-pr`, `review-github-pr`, `build-from-issue` | | Security review or remediation workflow | `review-security-issue`, `fix-security-issue` | | RFC template, numbering, or lifecycle | `create-rfc` | -| Documentation structure, navigation, or doc-update workflow | `update-docs` | +| Documentation structure, navigation, or doc-update workflow | `update-docs-from-commits` | | Skills, crates, workflow chains, issue/PR templates, or agent cross-references | `sync-agent-infra` | ## Prerequisites @@ -66,13 +68,14 @@ Gather the source of truth for each category. ### Skills -List all skill directories: +List public and contributor skill directories separately: ```bash +ls -1 skills/ ls -1 .agents/skills/ ``` -This is the canonical skill list. Every other file must agree with it. +The directories are canonical by audience: `skills/` contains public, installable user/operator skills and `.agents/skills/` contains internal contributor workflows. Every other file must agree with both inventories. ### Crates @@ -96,16 +99,19 @@ For each file in the table above, check for the following inconsistencies: ### `CONTRIBUTING.md` -1. **Skills table** — Every skill in `.agents/skills/` must appear in the "Agent Skills for Contributors" table. No skill in the table should reference a directory that doesn't exist. -2. **Workflow chains** — Must match `AGENTS.md` workflow chains exactly. -3. **Skill references in prose** — Any skill mentioned by name in "Before You Open an Issue", "When to Open an Issue", or "When NOT to Open an Issue" must exist in `.agents/skills/`. +1. **Public skills table** — Every skill in `skills/` must appear in "Skills for Using OpenShell" and no contributor skill may appear there. +2. **Contributor skills table** — Every skill in `.agents/skills/` must appear in "Agent Skills for Contributors" and no public skill may appear there. +3. **Inventory paths** — No skill in either table should reference a directory that does not exist. +4. **Workflow chains** — Must match `AGENTS.md` workflow chains exactly. +5. **Skill references in prose** — Any named skill must exist in exactly one canonical skill directory. ### `AGENTS.md` 1. **Architecture overview** — Every crate in `crates/` must appear in the architecture table. The `python/`, `proto/`, `deploy/`, `.agents/` rows must also be present. -2. **Workflow chains** — Verify each skill named in a chain exists in `.agents/skills/`. -3. **Issue/PR conventions** — Verify referenced skills (`create-github-issue`, `create-github-pr`, `build-from-issue`) exist. -4. **Skill maintenance pointer** — Verify it still points to `sync-agent-infra` and does not duplicate the maintenance map from this skill. +2. **Skill layout** — The architecture table must contain separate `skills/` and `.agents/skills/` rows with accurate audience descriptions. +3. **Workflow chains** — Verify each skill named in a chain exists in exactly one of the two skill directories. +4. **Issue/PR conventions** — Verify referenced skills (`create-github-issue`, `create-github-pr`, `build-from-issue`) exist. +5. **Skill maintenance pointer** — Verify it still points to `sync-agent-infra` and does not duplicate the maintenance map from this skill. ### Issue Lifecycle Documentation @@ -115,8 +121,8 @@ For each file in the table above, check for the following inconsistencies: ### `README.md` -1. **"Explore with your agent"** — Skill names referenced must exist in `.agents/skills/`. -2. **"Built With Agents"** — Skill names referenced must exist. Workflow descriptions should be consistent with `AGENTS.md` chains. +1. **Public installation guidance** — The README must distinguish `skills/` from `.agents/skills/`, include `npx skills add NVIDIA/OpenShell`, and list only canonical public skills as installable. +2. **"Built With Agents"** — Contributor skill names must exist under `.agents/skills/`. Workflow descriptions should be consistent with `AGENTS.md` chains. ### Issue Templates @@ -131,12 +137,23 @@ For each file in the table above, check for the following inconsistencies: ### Skill Cross-References 1. **`triage-issue`** — Skills referenced in gate check and diagnosis steps must exist. -2. **`openshell-cli`** — Companion skills table entries must exist. +2. **`openshell-cli`** — Companion skills table entries must exist in one canonical location. 3. **`build-from-issue`** — Label names must match the project's label taxonomy. Lifecycle and request labels must gate unattended queue pickup, while direct requests warn on workflow discrepancies and continue. 4. **`create-spike`** — Reference to `build-from-issue` as next step must be accurate. 5. **`review-security-issue`** / **`fix-security-issue`** — Cross-references between the two must be accurate. 6. **PR creation and review checks** — The `create-github-pr`, `review-github-pr`, `build-from-issue`, and `principal-engineer-reviewer` references to `sync-agent-infra` must exist and use trigger conditions aligned with this skill. +### Skill Layout, Metadata, and Portability + +1. **Placement** — The four public skills (`openshell-cli`, `generate-sandbox-policy`, `debug-inference`, and `debug-openshell-cluster`) must live only in `skills/`. Every other repository skill must live only in `.agents/skills/`. +2. **Internal metadata** — Every `.agents/skills/*/SKILL.md` must set `metadata.internal: true`. Public skills must not set internal metadata. Treat this as a discovery filter, not an access-control boundary. +3. **Unique names** — Parse the `name` field from every `SKILL.md` under both roots. Every name must be globally unique and match the documented inventory. +4. **Local references** — Every relative Markdown link and referenced file in a skill must resolve within that installed skill directory unless the reference is an explicit published URL. +5. **Canonical paths** — Contributor skills that name the source location of a public skill must use `skills//...`, never `.agents/skills//...`. +6. **Public portability** — Public skills must not require repository-relative files under `docs/`, `architecture/`, `crates/`, `deploy/`, or `.agents/`; source builds; `mise`; or repository E2E workflows. Use installed `openshell --help` for command syntax and Markdown endpoints under `https://docs.nvidia.com/openshell/latest/` (URLs ending in `.md`) for product documentation. +7. **No canonical documentation copies** — Review public reference files and large command/schema blocks. Remove material that merely copies CLI help, policy schemas, architecture docs, or published operational documentation; retain only skill-specific reasoning and worked interactions. +8. **Discovery** — Run `npx -y skills add . --list` from a clean checkout or disposable copy. It must list exactly the four public skills. Remove any generated lock file or installed directory after the check. + ## Step 3: Report Drift If any inconsistencies are found, report them in a structured format: @@ -145,9 +162,12 @@ If any inconsistencies are found, report them in a structured format: ## Agent Infrastructure Drift Report ### Skills Inventory -- ADDED (exists in .agents/skills/ but missing from CONTRIBUTING.md): -- REMOVED (in CONTRIBUTING.md but missing from .agents/skills/): -- OK: skills consistent +- PUBLIC ADDED (exists in skills/ but missing from CONTRIBUTING.md): +- PUBLIC REMOVED (documented as public but missing from skills/): +- CONTRIBUTOR ADDED (exists in .agents/skills/ but missing from CONTRIBUTING.md): +- CONTRIBUTOR REMOVED (documented as contributor but missing from .agents/skills/): +- METADATA/PATH/NAME ERRORS: +- OK: public and contributor skills consistent ### Architecture Table - ADDED (exists in crates/ but missing from AGENTS.md): @@ -178,6 +198,7 @@ If drift is found, fix it by updating the affected files: 5. **Removed crate** — Remove the row from the AGENTS.md architecture table. 6. **Changed workflow chain** — Update chains in both `AGENTS.md` and `CONTRIBUTING.md`. Update the "Built With Agents" section in `README.md` if the change is user-visible. 7. **Changed skill coverage** — Update the skill maintenance map in this file and any affected cross-references or companion-skill tables. +8. **Audience or portability drift** — Move the skill to its canonical root, fix internal metadata, replace stale public-skill paths, repair local links, and replace copied product documentation with CLI self-discovery or published documentation links. After fixing, re-run Step 2 to verify consistency. diff --git a/.agents/skills/test-release-canary/SKILL.md b/.agents/skills/test-release-canary/SKILL.md index 5e8bbf394c..f17285b0f1 100644 --- a/.agents/skills/test-release-canary/SKILL.md +++ b/.agents/skills/test-release-canary/SKILL.md @@ -1,6 +1,8 @@ --- name: test-release-canary description: Manually dispatch and iterate on the Release Canary workflow that smoke-tests published OpenShell artifacts (install.sh on macOS/Ubuntu/Fedora, Helm chart on kind) after each Release Dev publish. Use when changing `.github/workflows/release-canary.yml`, validating a release before tagging, debugging a canary failure, or reproducing a canary job locally. Trigger keywords - release canary, release-canary, canary failed, canary dispatch, test release canary, post-release smoke, install.sh canary, helm chart canary, kind canary, dispatch canary. +metadata: + internal: true --- # Test Release Canary diff --git a/.agents/skills/triage-issue/SKILL.md b/.agents/skills/triage-issue/SKILL.md index 11594e13a6..76ab01bdbe 100644 --- a/.agents/skills/triage-issue/SKILL.md +++ b/.agents/skills/triage-issue/SKILL.md @@ -1,6 +1,8 @@ --- name: triage-issue description: Assess, validate, and route community-filed issues for human disposition and roadmap placement. Takes a specific issue number or processes a confirmed batch of issues labeled state:triage-needed. Investigates reported behavior, separates objective findings from product decisions, and prepares validated issues for a human yes/no decision. Trigger keywords - triage issue, triage, assess issue, review incoming issue, triage issues. +metadata: + internal: true --- # Triage Issue @@ -153,9 +155,9 @@ Based on the sub-agent's analysis, also attempt to validate the report directly: - For bug reports: check the relevant code paths, look for the described failure mode - For feature requests: assess feasibility against the existing architecture -- For gateway deployment or infrastructure issues: reference the `debug-openshell-cluster` skill's known failure patterns -- For inference and provider-topology issues: reference the `debug-inference` skill's known failure patterns -- For CLI/usage issues: reference the `openshell-cli` skill's command reference +- For gateway deployment or infrastructure issues: reference the known failure patterns in `skills/debug-openshell-cluster/SKILL.md` +- For inference and provider-topology issues: reference `skills/debug-inference/SKILL.md` +- For CLI/usage issues: reference the workflows in `skills/openshell-cli/SKILL.md` and confirm installed syntax with `openshell --help` Record impact signals for the human decision: affected users and scope, regression status, workaround availability, severity evidence, and evidence quality. Do not convert those facts into a roadmap or sequencing recommendation. diff --git a/.agents/skills/tui-development/SKILL.md b/.agents/skills/tui-development/SKILL.md index 5d61af6ecd..5abc534f77 100644 --- a/.agents/skills/tui-development/SKILL.md +++ b/.agents/skills/tui-development/SKILL.md @@ -1,6 +1,8 @@ --- name: tui-development description: Guide for developing the OpenShell TUI — a ratatui-based terminal UI for the OpenShell platform. Covers architecture, navigation, data fetching, theming, UX conventions, and development workflow. Trigger keywords - term, TUI, terminal UI, ratatui, openshell-tui, tui development, tui feature, tui bug. +metadata: + internal: true --- # OpenShell TUI Development Guide @@ -53,7 +55,7 @@ Gateway (discovered via openshell_bootstrap::list_gateways()) The **title bar** always reflects this hierarchy, reading left-to-right from general to specific: ``` - OpenShell │ Current Gateway: [source] () │ Workspace: │ + OpenShell v │ Current Gateway: [source] () │ Workspace: │ ``` ## 3. Navigation & Screen Architecture @@ -142,8 +144,8 @@ Every frame renders four vertical regions: ### Title bar examples -- Dashboard: ` >_ OpenShell ALPHA | Current Gateway: openshell [local] (Healthy) | Workspace: default | Dashboard` -- Sandbox detail: ` >_ OpenShell ALPHA | Current Gateway: openshell [local] (Healthy) | Workspace: team-a | Sandbox: my-sandbox` +- Dashboard: ` >_ OpenShell v | Current Gateway: openshell [local] (Healthy) | Workspace: default | Dashboard` +- Sandbox detail: ` >_ OpenShell v | Current Gateway: openshell [local] (Healthy) | Workspace: team-a | Sandbox: my-sandbox` ### Adding a new screen @@ -384,7 +386,7 @@ All actions are accessible via keyboard shortcuts displayed in the nav bar. The `[Tab] Switch Panel [Enter] Select [j/k] Navigate │ [:] Command [q] Quit` **Dashboard (Providers focus):** -`[Tab] Switch Panel [h/l] Switch Tab [j/k] Navigate [Enter] Detail [c] Create [u] Update [d] Delete │ [:] Command [q] Quit` +`[Tab] Switch Panel [h/l] Switch Tab [j/k] Navigate [Enter] Detail [c] Create [u] Update [d] Delete [w] Workspace │ [:] Command [q] Quit` **Dashboard (Global Settings focus):** `[Tab] Switch Panel [h/l] Switch Tab [j/k] Navigate [Enter] Edit [d] Delete │ [:] Command [q] Quit` @@ -535,7 +537,7 @@ On launch, before the event loop starts: ### Workspace switching lifecycle -1. User presses `[w]` on the sandboxes panel → `cycle_workspace()` advances through discovered workspace names, then "all" +1. User presses `[w]` on the providers or sandboxes panel → `cycle_workspace()` advances through discovered workspace names, then "all" 2. `pending_workspace_refresh = true` is set, cursor indices are reset 3. Event loop calls `refresh_providers()` and `refresh_sandboxes()` with the new workspace scope diff --git a/.agents/skills/update-docs/SKILL.md b/.agents/skills/update-docs/SKILL.md index f38d7f576e..aa2db8cd58 100644 --- a/.agents/skills/update-docs/SKILL.md +++ b/.agents/skills/update-docs/SKILL.md @@ -1,6 +1,8 @@ --- name: update-docs-from-commits description: Scan recent git commits for changes that affect user-facing behavior, then draft or update the corresponding documentation pages. Use when docs have fallen behind code changes, after a batch of features lands, or when preparing a release. Trigger keywords - update docs, draft docs, docs from commits, sync docs, catch up docs, doc debt, docs behind, docs drift. +metadata: + internal: true --- # Update Docs from Commits diff --git a/.agents/skills/watch-github-actions/SKILL.md b/.agents/skills/watch-github-actions/SKILL.md index a7e7ea46da..c9e2843311 100644 --- a/.agents/skills/watch-github-actions/SKILL.md +++ b/.agents/skills/watch-github-actions/SKILL.md @@ -1,6 +1,8 @@ --- name: watch-github-actions description: Watch and monitor GitHub Actions workflow runs using the gh CLI. Use when the user wants to check workflow status, watch a running workflow, view CI/CD jobs, or monitor build progress. Trigger keywords - watch pipeline, pipeline status, CI status, check build, monitor CI, view pipeline, pipeline progress, workflow status, actions status. +metadata: + internal: true --- # Watch GitHub Actions diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml index 28e6b201b5..0f6486fde8 100644 --- a/.github/ISSUE_TEMPLATE/config.yml +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -7,11 +7,10 @@ contact_links: Vouch Request discussion describing what you want to work on. A maintainer will approve you with /vouch. - name: Have a question? - url: https://github.com/NVIDIA/OpenShell/blob/main/CONTRIBUTING.md#agent-skills-for-contributors + url: https://github.com/NVIDIA/OpenShell/blob/main/CONTRIBUTING.md#skills-for-using-openshell about: > - Point your agent at the repo. It has skills for CLI usage, cluster and - inference debugging, policy generation, and more. See CONTRIBUTING.md - for the full skills table. + Install the public OpenShell skills for CLI usage, cluster and inference + debugging, and policy generation. See CONTRIBUTING.md for details. - name: Security vulnerability? url: https://github.com/NVIDIA/OpenShell/blob/main/SECURITY.md about: > diff --git a/.github/actions/release-helm-oci/action.yml b/.github/actions/release-helm-oci/action.yml index d20691ad0f..46649e0cb0 100644 --- a/.github/actions/release-helm-oci/action.yml +++ b/.github/actions/release-helm-oci/action.yml @@ -4,7 +4,7 @@ name: Release Helm OCI description: > Patch chart version/appVersion, refuse duplicate OCI versions on public - releases, package the chart, and push to GHCR OCI. + releases, package the gateway and workspace charts, and push them to GHCR OCI. inputs: chart-version: @@ -51,11 +51,16 @@ runs: shell: bash run: | set -euo pipefail - CHART_DIR="${RUNNER_TEMP}/chart-build" - cp -a deploy/helm/openshell/. "${CHART_DIR}" - sed -i "s/^version:.*/version: ${CHART_VERSION}/" "${CHART_DIR}/Chart.yaml" - sed -i "s/^appVersion:.*/appVersion: \"${APP_VERSION}\"/" "${CHART_DIR}/Chart.yaml" - echo "chart_dir=${CHART_DIR}" >> "$GITHUB_OUTPUT" + GATEWAY_CHART_DIR="${RUNNER_TEMP}/gateway-chart-build" + WORKSPACE_CHART_DIR="${RUNNER_TEMP}/workspace-chart-build" + cp -a deploy/helm/openshell/. "${GATEWAY_CHART_DIR}" + cp -a deploy/helm/openshell-workspace/. "${WORKSPACE_CHART_DIR}" + for chart_dir in "${GATEWAY_CHART_DIR}" "${WORKSPACE_CHART_DIR}"; do + sed -i "s/^version:.*/version: ${CHART_VERSION}/" "${chart_dir}/Chart.yaml" + sed -i "s/^appVersion:.*/appVersion: \"${APP_VERSION}\"/" "${chart_dir}/Chart.yaml" + done + echo "gateway_chart_dir=${GATEWAY_CHART_DIR}" >> "$GITHUB_OUTPUT" + echo "workspace_chart_dir=${WORKSPACE_CHART_DIR}" >> "$GITHUB_OUTPUT" echo "chart_version=${CHART_VERSION}" >> "$GITHUB_OUTPUT" - name: Refuse duplicate chart version @@ -65,38 +70,52 @@ runs: shell: bash run: | set -euo pipefail - OCI_CHART="oci://ghcr.io/nvidia/openshell/helm-chart" - if helm show chart "${OCI_CHART}" --version "${CHART_VERSION}" >/dev/null 2>&1; then - echo "::error::Chart ${CHART_VERSION} is already published. Use a new tag or delete the existing package first." - exit 1 - fi + for chart in helm-chart openshell-workspace; do + OCI_CHART="oci://ghcr.io/nvidia/openshell/${chart}" + if helm show chart "${OCI_CHART}" --version "${CHART_VERSION}" >/dev/null 2>&1; then + echo "::error::Chart ${chart}:${CHART_VERSION} is already published. Use a new tag or delete the existing package first." + exit 1 + fi + done - name: Package Helm chart env: - CHART_DIR: ${{ steps.prep.outputs.chart_dir }} + GATEWAY_CHART_DIR: ${{ steps.prep.outputs.gateway_chart_dir }} + WORKSPACE_CHART_DIR: ${{ steps.prep.outputs.workspace_chart_dir }} shell: bash run: | set -euo pipefail - helm package "${CHART_DIR}" --destination /tmp - ls /tmp/helm-chart-*.tgz + mkdir -p /tmp/helm-charts + helm package "${GATEWAY_CHART_DIR}" --destination /tmp/helm-charts + helm package "${WORKSPACE_CHART_DIR}" --destination /tmp/helm-charts + ls /tmp/helm-charts/*.tgz - name: Push Helm chart to GHCR OCI shell: bash run: | set -euo pipefail - helm push /tmp/helm-chart-*.tgz oci://ghcr.io/nvidia/openshell + for archive in /tmp/helm-charts/*.tgz; do + helm push "${archive}" oci://ghcr.io/nvidia/openshell + done - name: Push SHA-pinned chart if: inputs.pin-sha != '' env: PIN_SHA: ${{ inputs.pin-sha }} - CHART_DIR: ${{ steps.prep.outputs.chart_dir }} + GATEWAY_CHART_DIR: ${{ steps.prep.outputs.gateway_chart_dir }} + WORKSPACE_CHART_DIR: ${{ steps.prep.outputs.workspace_chart_dir }} shell: bash run: | set -euo pipefail - SHA_CHART_DIR="${RUNNER_TEMP}/chart-build-sha" - cp -a "${CHART_DIR}/." "${SHA_CHART_DIR}" - sed -i "s/^version:.*/version: 0.0.0-dev.${PIN_SHA}/" "${SHA_CHART_DIR}/Chart.yaml" - sed -i "s/^appVersion:.*/appVersion: \"${PIN_SHA}\"/" "${SHA_CHART_DIR}/Chart.yaml" - helm package "${SHA_CHART_DIR}" --destination /tmp/sha-pin - helm push /tmp/sha-pin/helm-chart-*.tgz oci://ghcr.io/nvidia/openshell + mkdir -p /tmp/sha-pin + for source_dir in "${GATEWAY_CHART_DIR}" "${WORKSPACE_CHART_DIR}"; do + chart_name="$(basename "${source_dir}")" + sha_chart_dir="${RUNNER_TEMP}/${chart_name}-sha" + cp -a "${source_dir}/." "${sha_chart_dir}" + sed -i "s/^version:.*/version: 0.0.0-dev.${PIN_SHA}/" "${sha_chart_dir}/Chart.yaml" + sed -i "s/^appVersion:.*/appVersion: \"${PIN_SHA}\"/" "${sha_chart_dir}/Chart.yaml" + helm package "${sha_chart_dir}" --destination /tmp/sha-pin + done + for archive in /tmp/sha-pin/*.tgz; do + helm push "${archive}" oci://ghcr.io/nvidia/openshell + done diff --git a/.github/workflows/build-rpm.yml b/.github/workflows/build-rpm.yml new file mode 100644 index 0000000000..71aa04dddf --- /dev/null +++ b/.github/workflows/build-rpm.yml @@ -0,0 +1,123 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +name: Build RPM + +on: + workflow_call: + inputs: + checkout-ref: + required: true + type: string + arch: + required: true + type: string + runner: + required: true + type: string + cli-target: + required: true + type: string + gateway-target: + required: true + type: string + rpm-version: + required: false + type: string + default: "" + rpm-release: + required: false + type: string + default: "" + cargo-version: + required: false + type: string + default: "" + +permissions: + contents: read + +defaults: + run: + shell: bash + +jobs: + build: + name: Build RPM Package (Linux ${{ inputs.arch }}) + runs-on: ${{ inputs.runner }} + timeout-minutes: 60 + container: + image: docker.io/library/fedora:44@sha256:43b29f65a41eb9c35e1cd5323e3bdf3b655c2357a9f4f1ff2f9c2798e5045d80 + steps: + - name: Install packaging dependencies + run: | + dnf install -y \ + packit rpm-build \ + cargo cargo-rpm-macros git-core \ + pandoc python3-devel systemd-rpm-macros + + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ inputs.checkout-ref }} + fetch-depth: 0 + + - name: Cache Cargo dependencies + uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2 + with: + shared-key: rpm-vendor-${{ inputs.arch }} + cache-targets: "false" + cache-bin: "false" + cache-on-failure: "true" + + - name: Download CLI artifact + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: openshell-${{ inputs.cli-target }} + path: package-binaries/ + + - name: Download gateway artifact + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: openshell-gateway-${{ inputs.gateway-target }} + path: package-binaries/ + + - name: Configure package inputs + run: | + set -euo pipefail + chmod +x package-binaries/openshell{,-gateway} + ls -lah package-binaries + + - name: Mark workspace safe for git + run: git config --global --add safe.directory "$GITHUB_WORKSPACE" + + - name: Fetch tags + run: git fetch --tags --force + + - name: Build RPMs via Packit + env: + OPENSHELL_RPM_VERSION: ${{ inputs.rpm-version }} + OPENSHELL_RPM_RELEASE: ${{ inputs.rpm-release }} + OPENSHELL_CARGO_VERSION: ${{ inputs.cargo-version }} + OPENSHELL_PREBUILT_BINARIES_DIR: ${{ github.workspace }}/package-binaries + run: packit build locally + + - name: Collect RPM artifacts + run: | + set -euo pipefail + mkdir -p artifacts + mapfile -t rpms < <(find "$GITHUB_WORKSPACE" -maxdepth 3 -type f -name '*.rpm' ! -name '*.src.rpm' | sort) + if [ "${#rpms[@]}" -eq 0 ]; then + echo "::error::No RPM artifacts found under $GITHUB_WORKSPACE" + find "$GITHUB_WORKSPACE" -maxdepth 3 -type f | sort + exit 1 + fi + cp "${rpms[@]}" artifacts/ + echo "=== Built RPMs ===" + ls -lah artifacts/ + + - name: Upload RPM artifacts + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7 + with: + name: rpm-linux-${{ inputs.arch }} + path: artifacts/*.rpm + retention-days: 5 diff --git a/.github/workflows/conformance.yml b/.github/workflows/conformance.yml new file mode 100644 index 0000000000..bae9e3d326 --- /dev/null +++ b/.github/workflows/conformance.yml @@ -0,0 +1,195 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +name: Conformance + +on: + workflow_dispatch: {} + +permissions: {} + +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +jobs: + pr_metadata: + name: Resolve PR metadata + runs-on: ubuntu-latest + permissions: + contents: read + pull-requests: read + outputs: + should_run: ${{ steps.gate.outputs.should_run }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - id: gate + uses: ./.github/actions/pr-gate + + version: + needs: pr_metadata + if: needs.pr_metadata.outputs.should_run == 'true' + runs-on: ubuntu-latest + timeout-minutes: 5 + permissions: + contents: read + outputs: + cargo: ${{ steps.version.outputs.cargo }} + rpm_version: ${{ steps.version.outputs.rpm_version }} + rpm_release: ${{ steps.version.outputs.rpm_release }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + fetch-depth: 0 + + - name: Compute versions + id: version + run: | + cargo="$(python3 tasks/scripts/release.py get-version --cargo)" + rpm_version="$(python3 tasks/scripts/release.py get-version --rpm-version)" + rpm_release="$(python3 tasks/scripts/release.py get-version --rpm-release)" + { + echo "cargo=$cargo" + echo "rpm_version=$rpm_version" + echo "rpm_release=$rpm_release" + } >> "$GITHUB_OUTPUT" + + build-cli: + needs: version + permissions: + contents: read + uses: ./.github/workflows/build-binaries.yml + with: + package: openshell-cli + binary: openshell + triple: x86_64-unknown-linux-musl + runner: linux-amd64-cpu8 + dev-shell: .#devShells.x86_64-linux.musl + cargo-version: ${{ needs.version.outputs.cargo }} + checkout-ref: ${{ github.sha }} + secrets: inherit + + build-conformance: + needs: version + permissions: + contents: read + uses: ./.github/workflows/build-binaries.yml + with: + package: openshell-conformance-cli + binary: openshell-conformance + triple: x86_64-unknown-linux-musl + runner: linux-amd64-cpu8 + dev-shell: .#devShells.x86_64-linux.musl + cargo-version: ${{ needs.version.outputs.cargo }} + checkout-ref: ${{ github.sha }} + secrets: inherit + + build-gateway: + needs: version + permissions: + contents: read + uses: ./.github/workflows/build-binaries.yml + with: + package: openshell-gateway + binary: openshell-gateway + triple: x86_64-unknown-linux-gnu + runner: linux-amd64-cpu8 + dev-shell: .#devShells.x86_64-linux.glibc-2-28 + cargo-version: ${{ needs.version.outputs.cargo }} + image-tag: dev + interpreter: /lib64/ld-linux-x86-64.so.2 + checkout-ref: ${{ github.sha }} + secrets: inherit + + build-rpm: + needs: [version, build-cli, build-gateway] + permissions: + contents: read + uses: ./.github/workflows/build-rpm.yml + with: + checkout-ref: ${{ github.sha }} + arch: x86_64 + runner: linux-amd64-cpu8 + cli-target: x86_64-unknown-linux-musl + gateway-target: x86_64-unknown-linux-gnu + cargo-version: ${{ needs.version.outputs.cargo }} + rpm-version: ${{ needs.version.outputs.rpm_version }} + rpm-release: ${{ needs.version.outputs.rpm_release }} + + fedora: + name: Fedora with Rootless Podman + needs: [build-conformance, build-rpm] + runs-on: ubuntu-24.04 + timeout-minutes: 45 + permissions: + actions: read + contents: read + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + + - name: Enable KVM access + run: | + set -euo pipefail + if [[ ! -c /dev/kvm ]]; then + echo "::error::The runner did not expose /dev/kvm" + exit 1 + fi + sudo chmod 0666 /dev/kvm + exec 3<>/dev/kvm + exec 3>&- + + - uses: ./.github/actions/setup-nix + + - name: Download RPM artifacts + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: rpm-linux-x86_64 + path: rpm-input + + - name: Download conformance CLI + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: openshell-conformance-x86_64-unknown-linux-musl + path: conformance-input + + - name: Install RPMs and check status + shell: bash + run: | + set -euo pipefail + chmod +x conformance-input/openshell-conformance + guest_script="$RUNNER_TEMP/conformance.sh" + cat > "$guest_script" <<'EOF' + set -euo pipefail + + on_exit() { + rc=$? + trap - EXIT + if [ "$rc" -ne 0 ]; then + systemctl --user status openshell-gateway --no-pager || true + journalctl --user -u openshell-gateway --no-pager -n 200 || true + podman info || true + podman ps --all || true + getenforce || true + fi + exit "$rc" + } + trap on_exit EXIT + + mkdir -p "$HOME/.config/openshell" + echo 'OPENSHELL_TELEMETRY_ENABLED=false' > "$HOME/.config/openshell/gateway.env" + systemctl --user enable --now openshell-gateway + openshell gateway add --local https://127.0.0.1:17670 --name openshell + /tmp/openshell-conformance run smoke + EOF + + OPENSHELL_TEST_GUEST_CACHE_DISABLE=1 nix run .#test-guest -- \ + --distro fedora \ + --with podman-rootless \ + --with selinux \ + --install rpm-input/openshell-[0-9]*.rpm \ + --install rpm-input/openshell-gateway-[0-9]*.rpm \ + --copy "$guest_script:/tmp/conformance.sh" \ + --copy conformance-input/openshell-conformance:/tmp/openshell-conformance \ + -- bash /tmp/conformance.sh diff --git a/.github/workflows/release-tag.yml b/.github/workflows/release-tag.yml index d47a6c7cb5..b3c45c97fb 100644 --- a/.github/workflows/release-tag.yml +++ b/.github/workflows/release-tag.yml @@ -732,7 +732,6 @@ jobs: release-helm: name: Release Helm Chart (OCI) needs: [compute-versions, release, tag-ghcr-release] - if: needs.compute-versions.outputs.is_prerelease != 'true' runs-on: ubuntu-latest timeout-minutes: 10 permissions: diff --git a/.github/workflows/rpm-package.yml b/.github/workflows/rpm-package.yml index 8701644585..5cda9a88c3 100644 --- a/.github/workflows/rpm-package.yml +++ b/.github/workflows/rpm-package.yml @@ -37,83 +37,20 @@ jobs: matrix: include: - arch: x86_64 - artifact_arch: amd64 runner: linux-amd64-cpu8 cli_target: x86_64-unknown-linux-musl - gnu_target: x86_64-unknown-linux-gnu + gateway_target: x86_64-unknown-linux-gnu - arch: aarch64 - artifact_arch: arm64 runner: linux-arm64-cpu8 cli_target: aarch64-unknown-linux-musl - gnu_target: aarch64-unknown-linux-gnu - runs-on: ${{ matrix.runner }} - timeout-minutes: 60 - container: - image: fedora:latest - steps: - - name: Install build dependencies - run: | - dnf install -y \ - packit rpm-build \ - rust cargo gcc gcc-c++ make cmake pkg-config \ - clang-devel z3-devel systemd-rpm-macros \ - pandoc python3-devel git-core \ - cargo-rpm-macros - - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - ref: ${{ inputs.checkout-ref }} - fetch-depth: 0 - - - name: Download CLI artifact - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 - with: - name: openshell-${{ matrix.cli_target }} - path: package-binaries/ - - - name: Download gateway artifact - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 - with: - name: openshell-gateway-${{ matrix.gnu_target }} - path: package-binaries/ - - - name: Configure package inputs - run: | - set -euo pipefail - chmod +x package-binaries/openshell{,-gateway} - ls -lah package-binaries - - - name: Mark workspace safe for git - run: git config --global --add safe.directory "$GITHUB_WORKSPACE" - - - name: Fetch tags - run: git fetch --tags --force - - - name: Build RPMs via Packit - env: - OPENSHELL_RPM_VERSION: ${{ inputs['rpm-version'] }} - OPENSHELL_RPM_RELEASE: ${{ inputs['rpm-release'] }} - OPENSHELL_CARGO_VERSION: ${{ inputs['cargo-version'] }} - OPENSHELL_PREBUILT_BINARIES_DIR: ${{ github.workspace }}/package-binaries - run: packit build locally - - - name: Collect RPM artifacts - run: | - set -euo pipefail - mkdir -p artifacts - mapfile -t rpms < <(find "$GITHUB_WORKSPACE" -maxdepth 3 -type f -name '*.rpm' ! -name '*.src.rpm' | sort) - if [ "${#rpms[@]}" -eq 0 ]; then - echo "::error::No RPM artifacts found under $GITHUB_WORKSPACE" - find "$GITHUB_WORKSPACE" -maxdepth 3 -type f | sort - exit 1 - fi - cp "${rpms[@]}" artifacts/ - echo "=== Built RPMs ===" - ls -lah artifacts/ - - - name: Upload RPM artifacts - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7 - with: - name: rpm-linux-${{ matrix.arch }} - path: artifacts/*.rpm - retention-days: 5 + gateway_target: aarch64-unknown-linux-gnu + uses: ./.github/workflows/build-rpm.yml + with: + checkout-ref: ${{ inputs.checkout-ref }} + arch: ${{ matrix.arch }} + runner: ${{ matrix.runner }} + cli-target: ${{ matrix.cli_target }} + gateway-target: ${{ matrix.gateway_target }} + rpm-version: ${{ inputs.rpm-version }} + rpm-release: ${{ inputs.rpm-release }} + cargo-version: ${{ inputs.cargo-version }} diff --git a/.packit.yaml b/.packit.yaml index 6379d8db83..d3b92eafae 100644 --- a/.packit.yaml +++ b/.packit.yaml @@ -11,7 +11,6 @@ specfile_path: openshell.spec # Packages needed in the SRPM build environment to create vendor tarball srpm_build_deps: - - rust - cargo - git-core @@ -45,10 +44,6 @@ actions: # dist-info stays at the RPM Version; dev build identity is carried by # Release so Fedora's Python RPM post-processing can normalize metadata. - 'bash -c "if [ -n \"${OPENSHELL_CARGO_VERSION:-}\" ]; then sed -i -r \"s/^%global openshell_cargo_version .*/%global openshell_cargo_version ${OPENSHELL_CARGO_VERSION}/\" openshell.spec; fi"' - # Override image_tag to 'latest' for tagged stable releases. - # For PR and commit-to-main builds the spec default ('dev') is kept, - # matching the :dev images pushed by release-dev.yml. - - 'bash -c "if git describe --exact-match --tags HEAD 2>/dev/null | grep -qE ''^v[0-9]+\.[0-9]+\.[0-9]+$''; then sed -i ''s/^%global image_tag.*/%global image_tag latest/'' openshell.spec; fi"' jobs: # Build on every pull request targeting main for CI validation diff --git a/AGENTS.md b/AGENTS.md index 33dad248d7..9a07ca0b7a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -10,7 +10,12 @@ OpenShell is built agent-first. We design systems and use agents to implement th ## Skills -Agent skills live in `.agents/skills/`. Your harness can discover and load them natively — do not rely on this file for a full inventory. The detailed skills table is in [CONTRIBUTING.md](CONTRIBUTING.md) (for humans). +OpenShell has two skill collections: + +- `skills/` contains public, installable skills for using and operating OpenShell. These skills must work outside a source checkout and use installed CLI help plus published documentation as their sources of truth. +- `.agents/skills/` contains internal contributor and maintainer workflows for developing OpenShell. Your repository-aware harness can discover and load them natively. + +Do not rely on this file for a full inventory. The detailed public and contributor skill tables are in [CONTRIBUTING.md](CONTRIBUTING.md) (for humans). ## Workflow Chains @@ -55,6 +60,7 @@ These pipelines connect skills into end-to-end workflows. Individual skill files | `crates/openshell-driver-mxc/` | MXC compute driver | Windows in-process `ComputeDriver` backend for MXC sandbox execution | | `crates/openshell-driver-podman/` | Podman compute driver | In-process `ComputeDriver` backend for local Podman sandbox containers | | `crates/openshell-driver-vm/` | VM compute driver | Standalone libkrun-backed `ComputeDriver` subprocess (embeds its own rootfs + runtime) | +| `crates/openshell-driver-mxc/` | Microsoft MXC compute driver | In-process Windows AppContainer and isolation-session compute backend | | `crates/openshell-prover/` | Policy prover | Policy verification and proof generation | | `crates/openshell-server-macros/` | Server macros | Compile-time helpers for gateway RPC authorization | | `crates/openshell-supervisor-middleware/` | Middleware runtime | Generic middleware registry, remote service integration, and chain execution | @@ -68,7 +74,8 @@ These pipelines connect skills into end-to-end workflows. Individual skill files | `deploy/` | Docker, Helm, K8s | Dockerfiles, Helm chart, manifests | | `docs/` | Published docs | MDX pages, navigation, and content assets | | `fern/` | Docs site config | Fern site config, components, and theme assets | -| `.agents/skills/` | Agent skills | Workflow automation for development | +| `skills/` | Public agent skills | Installable workflows for using and operating OpenShell | +| `.agents/skills/` | Contributor agent skills | Repository-aware workflows for developing OpenShell | | `.agents/agents/` | Agent personas | Sub-agent definitions (e.g., reviewer, doc writer) | | `architecture/` | Architecture docs | Design decisions and component documentation | @@ -239,7 +246,7 @@ ocsf_emit!(event); ## Cluster Infrastructure Changes -- If you change gateway deployment infrastructure (e.g., Helm values/templates, gateway image packaging, or deploy logic in `openshell-cli`), update the `debug-openshell-cluster` skill in `.agents/skills/debug-openshell-cluster/SKILL.md` to reflect those changes. +- If you change gateway deployment infrastructure (e.g., Helm values/templates, gateway image packaging, or deploy logic in `openshell-cli`), update the `debug-openshell-cluster` skill in `skills/debug-openshell-cluster/SKILL.md` to reflect those changes. ## Skill Maintenance @@ -253,7 +260,7 @@ When behavior, commands, or development workflows change, review the related age - `fern/` contains the Fern site config, components, preview workflow inputs, and publish settings. - Follow the docs style guide in [docs/CONTRIBUTING.mdx](docs/CONTRIBUTING.mdx): active voice, minimal formatting, no filler introductions, `shell` fences for copyable commands, and no duplicate body H1. - Fern PR previews run through `.github/workflows/branch-docs.yml`, and production publish runs through the `publish-fern-docs` job in `.github/workflows/release-tag.yml` for stable release tags. -- Use the `update-docs` skill to scan recent commits and draft doc updates. +- Use the `update-docs-from-commits` skill to scan recent commits and draft doc updates. ### Architecture Docs diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index b1a789496f..1ee7d15cef 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -48,7 +48,7 @@ Search open and closed issues for the same need. Bug reports and feature request Feature requests must also propose a user-facing workflow and describe alternatives considered. Define the externally observable behavior and leave internal implementation choices open. Bug reports instead include minimal reproduction steps, the OpenShell version and relevant environment, and a small, redacted log excerpt when it materially clarifies the behavior. -The project includes optional [agent skills](#agent-skills-for-contributors) for self-service troubleshooting and exploration. Use them when they help you, but summarize any useful result in your own words rather than pasting a diagnostic transcript. +The project includes optional [agent skills](#agent-skills) for using OpenShell and contributing to the repository. Use them when they help you, but summarize any useful result in your own words rather than pasting a diagnostic transcript. ### When to Open an Issue @@ -68,15 +68,29 @@ Do not start substantial issue-backed work until a maintainer has accepted the i Use agents and the repository skills as needed to understand the affected code, evaluate tradeoffs, implement the smallest coherent change, and verify it. The pull request should explain what changed and how it was tested; it should not substitute an agent transcript for the contributor's understanding. -## Agent Skills for Contributors +## Agent Skills -Skills live in `.agents/skills/`. Your agent's harness can discover and load them natively. Here is the full inventory: +OpenShell keeps skills for using the product separate from skills for developing the repository. + +### Skills for Using OpenShell + +Public skills live in `skills/` and work without an OpenShell source checkout. Install them with `npx skills add NVIDIA/OpenShell`. + +| Skill | Purpose | +| --- | --- | +| `openshell-cli` | CLI usage, sandbox lifecycle, provider management, and BYOC workflows | +| `debug-openshell-cluster` | Diagnose gateway deployment and health issues | +| `debug-inference` | Diagnose managed, system, local, and direct external inference issues | +| `generate-sandbox-policy` | Generate YAML sandbox policies from requirements or API documentation | + +Public skills use `openshell --help` for installed command syntax and published OpenShell documentation for product concepts and configuration. They must not depend on repository-relative source or documentation files. + +### Agent Skills for Contributors + +Contributor and maintainer skills live in `.agents/skills/`. They are marked internal so the Agent Skills CLI excludes them from ordinary public discovery, but repository-aware agent harnesses can discover and load them natively. Internal metadata is a discovery filter, not an access-control boundary. | Category | Skill | Purpose | | --------------- | ------------------------- | --------------------------------------------------------------------------------------------------- | -| Getting Started | `openshell-cli` | CLI usage, sandbox lifecycle, provider management, BYOC workflows | -| Getting Started | `debug-openshell-cluster` | Diagnose gateway deployment and health issues | -| Getting Started | `debug-inference` | Diagnose `inference.local`, host-backed local inference, and direct external inference setup issues | | Contributing | `create-spike` | Investigate a problem, produce a structured GitHub issue | | Contributing | `create-rfc` | Create RFC proposals from the repository template | | Contributing | `build-from-issue` | Plan and implement work from a GitHub issue (maintainer workflow) | @@ -89,11 +103,10 @@ Skills live in `.agents/skills/`. Your agent's harness can discover and load the | Reviewing | `launch-openshell-gator` | Launch and supervise OpenShell gator agents for issue and PR monitoring | | Reviewing | `test-release-canary` | Dispatch and iterate on the Release Canary workflow that smoke-tests published artifacts | | Triage | `triage-issue` | Assess, classify, and route community-filed issues | -| Platform | `generate-sandbox-policy` | Generate YAML sandbox policies from requirements or API docs | | Platform | `helm-dev-environment` | Start and manage the local Kubernetes development environment | | Platform | `tui-development` | Development guide for the ratatui-based terminal UI | | Platform | `build-openshell-mxc-windows` | Maintain and validate the build-only x64 and ARM64 Windows MSVC lane | -| Documentation | `update-docs` | Scan recent commits and draft doc updates for user-facing changes | +| Documentation | `update-docs-from-commits` | Scan recent commits and draft doc updates for user-facing changes | | Maintenance | `sync-agent-infra` | Detect and fix drift across agent-first infrastructure files | | Reference | `sbom` | Generate SBOMs and resolve dependency licenses | @@ -429,7 +442,8 @@ These are the primary `mise` tasks for day-to-day development: | `fern/` | Fern site config, components, and theme assets | | `architecture/` | Architecture docs and plans | | `rfc/` | Request for Comments proposals | -| `.agents/` | Agent skills and persona definitions | +| `skills/` | Public skills for using and operating OpenShell | +| `.agents/` | Contributor skills and persona definitions | ## RFCs @@ -439,7 +453,7 @@ New features always start as GitHub issues using the feature request template. F If your change affects user-facing behavior (new flags, changed defaults, new features, bug fixes that contradict existing docs), update the relevant pages under `docs/` in the same PR and adjust `docs/index.yml` if navigation changes. For explicit navigation entries, keep `page:` aligned with `sidebar-title` when present and put relative `slug:` values in `docs/index.yml`. Reserve frontmatter `slug` for folder-discovered pages or absolute URL overrides. -To ensure your doc changes follow NVIDIA documentation style, use the `update-docs` skill. +To ensure your doc changes follow NVIDIA documentation style, use the `update-docs-from-commits` skill. It scans commits, identifies doc pages that need updates, and drafts content that follows the style guide in `docs/CONTRIBUTING.mdx`. To preview Fern docs locally: diff --git a/Cargo.lock b/Cargo.lock index 90240d6b3a..dc24adc6db 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -3861,8 +3861,11 @@ dependencies = [ "prost", "prost-types", "protoc-bin-vendored", + "rcgen", "reqwest 0.12.28", "rustix 1.1.4", + "rustls", + "rustls-pemfile", "serde", "serde_json", "tar", diff --git a/README.md b/README.md index 751eba780d..a7ecf4179e 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,7 @@ OpenShell is the safe, private runtime for autonomous AI agents. It provides sandboxed execution environments that protect your data, credentials, and infrastructure — governed by declarative YAML policies that prevent unauthorized file access, data exfiltration, and uncontrolled network activity. -OpenShell is built agent-first. The project ships with agent skills for everything from gateway troubleshooting to policy generation, and we expect contributors to use them. +OpenShell is built agent-first. It ships public agent skills for using and operating OpenShell, plus separate repository-aware workflows for contributors and maintainers. ## Quickstart @@ -206,35 +206,33 @@ openshell sandbox create --from registry.io/img:v1 # container image See the [OpenShell Community](https://github.com/NVIDIA/OpenShell-Community) catalog and the [BYOC example](https://github.com/NVIDIA/OpenShell/tree/main/examples/bring-your-own-container) for details. -## Explore with Your Agent +## Use OpenShell with Your Agent -Clone the repo and point your coding agent at it. The project includes agent skills that can answer questions, walk you through workflows, and diagnose problems — no issue filing required. +OpenShell provides four portable skills for users and operators: CLI workflows (`openshell-cli`), gateway troubleshooting (`debug-openshell-cluster`), inference troubleshooting (`debug-inference`), and policy generation (`generate-sandbox-policy`). Install them with the Agent Skills CLI: ```bash -git clone https://github.com/NVIDIA/OpenShell.git # or git@github.com:NVIDIA/OpenShell.git -cd OpenShell -# Point your agent here — it will discover the skills in .agents/skills/ automatically +npx skills add NVIDIA/OpenShell ``` -Your agent can load skills for CLI usage (`openshell-cli`), gateway troubleshooting (`debug-openshell-cluster`), inference troubleshooting (`debug-inference`), policy generation (`generate-sandbox-policy`), and more. See [CONTRIBUTING.md](CONTRIBUTING.md) for the full skills table. +These public, installable skills live in [`skills/`](skills/) and use the installed CLI help and [published documentation](https://docs.nvidia.com/openshell/latest/index.html) as their sources of truth. They do not require an OpenShell source checkout. ## Built With Agents -OpenShell is developed using the same agent-driven workflows it enables. The `.agents/skills/` directory contains workflow automation that powers the project's development cycle: +OpenShell is developed using the same agent-driven workflows it enables. Contributor and maintainer skills live separately in [`.agents/skills/`](.agents/skills/); they automate work on the OpenShell repository and are not included when users install the public skills: - **Spike and build:** Investigate a problem with `create-spike`; a human accepts it with `state:accepted` or [roadmap](https://github.com/orgs/NVIDIA/projects/233) placement, or declines it. Accepted work can remain human-owned or enter the optional, human-gated `agent:*` planning and implementation workflow. - **Triage and route:** Community issues are assessed with `triage-issue`. Agents establish technical validity and impact; humans decide whether the project should act and where the work sits on the roadmap. - **Security review:** `review-security-issue` produces a severity assessment and remediation plan. `fix-security-issue` implements it. -- **Policy authoring:** `generate-sandbox-policy` creates YAML policies from plain-language requirements or API documentation. +- **Repository maintenance:** `sync-agent-infra`, `update-docs-from-commits`, and other internal workflows keep code, documentation, and agent infrastructure consistent. -All agent implementation work is human-gated: maintainers explicitly request a plan, agents propose it, maintainers approve it, and agents build. See [AGENTS.md](AGENTS.md) for the full workflow chain documentation. +Agent implementation is human-directed: a user may request a phase directly, or maintainers may use the optional `agent:*` workflow to queue and approve planning and implementation. See [AGENTS.md](AGENTS.md) for the full workflow chain documentation. ## Getting Help - **Questions and discussion:** [GitHub Discussions](https://github.com/NVIDIA/OpenShell/discussions) - **Bug reports:** [GitHub Issues](https://github.com/NVIDIA/OpenShell/issues) — use the bug report template - **Security vulnerabilities:** See [SECURITY.md](SECURITY.md) — do not use GitHub Issues -- **Agent-assisted help:** Clone the repo and use the agent skills in `.agents/skills/` for self-service diagnostics +- **Agent-assisted help:** Install the public OpenShell skills with `npx skills add NVIDIA/OpenShell` ## Learn More @@ -250,7 +248,7 @@ All agent implementation work is human-gated: maintainers explicitly request a p ## Contributing -OpenShell is built agent-first. Issues should include a user story, problem statement, impact, and acceptance criteria. The impact should explain the consequences of the current behavior and why existing workarounds are insufficient. Feature requests also require a workflow-level proposed design and alternatives; bug reports add reproduction steps, environment details, and relevant logs. Once maintainers accept work, contributors should use the skills in `.agents/skills/` to investigate the current code and behavior, implement the change, and verify it. If an issue contains earlier diagnostics, verify them rather than relying on them. See [CONTRIBUTING.md](CONTRIBUTING.md) for the full agent skills table, contribution workflow, and development setup. +OpenShell is built agent-first. Issues should include a user story, problem statement, impact, and acceptance criteria. The impact should explain the consequences of the current behavior and why existing workarounds are insufficient. Feature requests also require a workflow-level proposed design and alternatives; bug reports add reproduction steps, environment details, and relevant logs. Once work is authorized through the project workflow or a direct request, contributors should use the skills in `.agents/skills/` to investigate the current code and behavior, implement the change, and verify it. If an issue contains earlier diagnostics, verify them rather than relying on them. See [CONTRIBUTING.md](CONTRIBUTING.md) for the full agent skills table, contribution workflow, and development setup. ## Telemetry diff --git a/architecture/build.md b/architecture/build.md index 393eb9e468..18f1561f50 100644 --- a/architecture/build.md +++ b/architecture/build.md @@ -120,13 +120,12 @@ and the supervisor image from `deploy/docker/Dockerfile.supervisor`. Neither Dockerfile compiles Rust — both copy a staged binary out of `deploy/docker/.build/prebuilt-binaries//` into the final image. -Local binary staging is driven by `tasks/scripts/stage-prebuilt-binaries.sh`. Because -staging cross-compiles on the host, it sources `tasks/scripts/build-env.sh` and -raises the per-process open-file limit before invoking `cargo zigbuild` on -macOS — the static musl link opens hundreds of `.rlib` files at once and would -otherwise fail with `ProcessFdQuotaExceeded` under macOS's default soft limit of -256. The guard is a no-op on Linux and when `cargo-zigbuild` is absent. Gateway -binaries use `cargo zigbuild` with GNU targets pinned to glibc 2.28, including +Local binary staging is driven by `tasks/scripts/stage-prebuilt-binaries.sh`. +It and the local gateway tasks source `tasks/scripts/build-env.sh` before host +Rust builds on macOS. The helper raises the per-process open-file limit because +sccache and static musl linking can each open hundreds of files, exceeding the +default soft limit of 256. The guard is a no-op on Linux. Gateway binaries use +`cargo zigbuild` with GNU targets pinned to glibc 2.28, including native-architecture builds, so the gateway image, standalone tarballs, and Linux packages share the same host portability floor. The gateway build enables `bundled-z3`. Linux VM driver release artifacts use the same glibc floor so diff --git a/architecture/compute-runtimes.md b/architecture/compute-runtimes.md index aa97fbcb78..e1e731a0ce 100644 --- a/architecture/compute-runtimes.md +++ b/architecture/compute-runtimes.md @@ -258,6 +258,16 @@ template resource limits. Docker and Podman apply them as runtime limits. Kubernetes mirrors each limit into the matching request. VM accepts the fields but currently ignores them. +Reusable sandbox workload templates are resolved before the compute-driver +boundary. Drivers do not receive a separate template resource; the gateway +lowers the selected `SandboxWorkloadTemplate` into the existing sandbox spec +and validates that spec before calling `ValidateSandboxCreate` or +`CreateSandbox`. Template CPU and memory become the same typed resource limits +described above. Template GPU settings become `ResourceRequirements`, preserving +the driver's default GPU assignment when the count is omitted. Template +`driver_config` remains a driver-keyed envelope until the compute layer selects +the active driver block and forwards only that block to the driver. + Docker and Podman also accept per-sandbox driver-config mounts for existing runtime-managed named volumes and tmpfs mounts. Podman additionally accepts image mounts through its image-volume API. User-supplied bind and volume mounts @@ -283,6 +293,17 @@ through the driver configuration. The Helm chart defaults sandbox agents to `Unconfined` so runtime/default AppArmor profiles do not block supervisor network namespace setup on AppArmor-enabled nodes. +The Kubernetes deployment packaging has two ownership boundaries. The gateway +chart owns the gateway workload, configuration, Services, PKI, and +cluster-scoped gateway resources. It can retain the legacy combined behavior, +or omit workspace resources. The workspace chart is installed into a +pre-provisioned sandbox namespace and owns only the sandbox ServiceAccount, +namespaced RBAC, and sandbox ingress NetworkPolicy. Its RoleBinding names the +gateway ServiceAccount and namespace explicitly, so the two releases have +disjoint lifecycle ownership. A shared-mode gateway can target one external +namespace, while operator mode maps workspace names to multiple +platform-provisioned namespaces. + Resource requirements enter the driver layer through `SandboxSpec.resource_requirements`. This includes a set of GPU requirements, where a user can request a specific number of GPUs or the driver-specific default behaviour. For all in-tree drivers, this is equivalent to selecting a single GPU. diff --git a/architecture/gateway.md b/architecture/gateway.md index 0430d95159..f86411511b 100644 --- a/architecture/gateway.md +++ b/architecture/gateway.md @@ -344,23 +344,33 @@ default WAL journal mode), which mirror the same sensitive contents. Persisted state includes sandboxes, providers, provider credential refresh state, SSH sessions, policy revisions, settings, inference configuration, and -deployment records. Provider refresh state is stored as a separate object -scoped to the provider instance through `objects.scope`. Its non-secret -configuration remains inline, while refresh tokens, client secrets, private -keys, and other secret source material are stored through the active credential -driver and represented by opaque handles. The provider record keeps only the -current injectable credential handles and optional per-credential expiry -timestamps. A refresh normally mints one credential, but a strategy may -co-mint several (AWS STS mints the access key, secret key, and session token in -one call); the refresh state pins the resolved set of env keys it owns so -collision checks reserve all of them before the first mint. Provider records -keep inline credential values only for legacy records created before credential -driver storage. New provider and refresh-material writes keep driver-owned -credential handles. When no external credential driver is configured, gateways -use server-owned encrypted database credential storage for defense in depth. -Multi-replica deployments can use that default with a shared database and -shared key-encryption key, or opt into an external backend such as Vault or -Kubernetes Secrets. +deployment records, and reusable sandbox workload templates. Provider refresh +state is stored as a separate object scoped to the provider instance through +`objects.scope`. Its non-secret configuration remains inline, while refresh +tokens, client secrets, private keys, and other secret source material are +stored through the active credential driver and represented by opaque handles. +The provider record keeps only the current injectable credential handles and +optional per-credential expiry timestamps. A refresh normally mints one +credential, but a strategy may co-mint several (AWS STS mints the access key, +secret key, and session token in one call); the refresh state pins the resolved +set of env keys it owns so collision checks reserve all of them before the +first mint. Provider records keep inline credential values only for legacy +records created before credential driver storage. New provider and +refresh-material writes keep driver-owned credential handles. When no external +credential driver is configured, gateways use server-owned encrypted database +credential storage for defense in depth. Multi-replica deployments can use that +default with a shared database and shared key-encryption key, or opt into an +external backend such as Vault or Kubernetes Secrets. + +Sandbox workload templates are workspace-scoped gateway resources. Workspace +admins create and delete them; workspace users can read and list them. A +template owns reusable workload intent: image, environment, CPU and memory +limits, GPU request, driver-specific config, and service-level hints. A sandbox +created from a template resolves that resource once and persists an ordinary +`SandboxSpec` snapshot. The create request still owns per-sandbox governance: +name, labels, annotations, provider attachments, and policy. The sandbox stores +template provenance as the template name and resource version used for the +snapshot, so later template edits or deletes do not mutate existing sandboxes. OAuth refresh failures retain a gateway-owned recovery classification alongside the refresh state. The gateway reads only a bounded error response and maps diff --git a/architecture/sandbox-limits.md b/architecture/sandbox-limits.md index 49f1ab7ce9..9635bc1c30 100644 --- a/architecture/sandbox-limits.md +++ b/architecture/sandbox-limits.md @@ -40,6 +40,15 @@ New limits should follow these rules: query parameters, or external free-form diagnostics. - Test time bounds with simulated time and test shared budgets under saturation. +## Gateway Sandbox Resources + +Gateway-owned sandbox resources also carry admission limits before they can +produce supervisor work. Reusable workload templates are capped at 1000 per +workspace. Template payloads reuse sandbox spec validation for environment +entry count and size, image and resource field sizes, driver-config serialized +size, and GPU count. Template names use the same DNS-style resource-name rules +as other named gateway resources. + ## Middleware Middleware limits are process-wide per sandbox. Registry replacement preserves diff --git a/architecture/sandbox.md b/architecture/sandbox.md index 6365ed0631..dd9621a09d 100644 --- a/architecture/sandbox.md +++ b/architecture/sandbox.md @@ -259,6 +259,12 @@ own DNS view, e.g. DoH tunneled via CONNECT, is a possible future enhancement and out of scope.) The workload child's proxy variables are unaffected — they are always rewritten to point at the local policy proxy. +Template environment is treated like user-provided sandbox environment. It can +shape the workload child, but it cannot override driver-controlled identity, +gateway callback, TLS, relay socket, proxy, provider, or supervisor coordination +variables. Drivers and the supervisor rewrite those reserved values after image +and template environment are considered. + The configuration is fail-closed: a setting that is present but invalid — an empty value, an unsupported or malformed proxy URL, an unreadable auth file or CA bundle, a malformed credential, or an auth file, `NO_PROXY` list, or CA @@ -294,6 +300,35 @@ file and builds the `Proxy-Authorization: Basic` header; a credential that is empty, contains control characters, or is not in `user:pass` form is fatal on both sides. +The VM driver has no argv seam of its own: its guest init script runs as PID 1 +and execs a fixed supervisor command line, and the libkrun and QEMU launch +backends both reach the supervisor through that script. Driver-owned +supervisor arguments therefore travel in a per-sandbox file the driver writes +into the overlay upperdir at a fixed guest path, one argument per line, which +the guest reads verbatim (no word splitting or globbing) and appends to every +supervisor exec. The file is written on **every** launch, including an empty +file when there is nothing to pass: the upperdir copy always shadows the +read-only image layer, so a sandbox image can neither supply its own +supervisor arguments by baking a file at that path nor disable the operator's +by omitting one. This mirrors the driver-authored `init.d` manifest, which +solves the same trust problem for guest init drop-ins. + +A microVM has no bind mounts or container secrets, so the VM driver stages the +credential and the CA bundle into the per-sandbox overlay disk instead — the +credential root-only, the CA world-readable, both at fixed `/opt/openshell` +paths and both removed with the sandbox state directory. The consequence, +which differs from the Podman secret model, is that the credential is at rest +inside that overlay image on the gateway host; the per-sandbox gateway JWT +already travels the same path. Proxy reachability differs by VM backend. libkrun-backed +sandboxes egress through gvproxy, so a proxy on the gateway host's loopback is +reachable through the host alias `host.openshell.internal`, which gvproxy NATs +to the host's `127.0.0.1`. QEMU/TAP sandboxes (GPU) have no equivalent: that +alias resolves to the TAP host address, and the driver's nftables `input` +chain accepts only the gateway port from the guest, so no gateway-host proxy +is reachable. The driver rejects a gateway-host proxy URL on the QEMU path at +launch rather than producing CONNECT timeouts. The guest's gateway callback is +unaffected in both backends and never traverses the proxy. + For Kubernetes sandboxes, the operator configures a Secret name and key rather than a gateway-host file path. Kubernetes projects that Secret only into the container that runs network supervision. Proxy credential Secrets require the diff --git a/crates/openshell-cli/src/color.rs b/crates/openshell-cli/src/color.rs index e6a185bdad..9b78b4ca90 100644 --- a/crates/openshell-cli/src/color.rs +++ b/crates/openshell-cli/src/color.rs @@ -34,7 +34,15 @@ //! 1. `--color always|never` on the command line. //! 2. `NO_COLOR`, set and non-empty, disables color (). //! 3. `FORCE_COLOR`, set and non-empty, forces color on (). -//! 4. Otherwise the stream is styled only when that stream is a terminal. +//! 4. Otherwise the stream is styled only when that stream is a terminal *and* +//! that terminal renders ANSI. +//! +//! Attachment and capability are separate questions. `TERM=dumb` is a terminal +//! that does not interpret escapes, so `auto` must not style it — and neither +//! `console` nor `miette` can apply their own `TERM` checks any more, because +//! [`init`] overrides both. Capability is consulted only under `auto`, so +//! `--color always` and `FORCE_COLOR` still force styling on a `dumb` terminal +//! for anyone who wants it. //! //! Step 4 is resolved per stream. Redirecting one must not decide for the other: //! `openshell ... 2> build.log` from a terminal should keep a styled stdout and @@ -87,6 +95,7 @@ pub enum ColorChoice { pub fn init(choice: ColorChoice) { let no_color = std::env::var_os("NO_COLOR"); let force_color = std::env::var_os("FORCE_COLOR"); + let term = std::env::var_os("TERM"); // Under `auto` each stream answers for itself. Redirecting one must not // decide for the other: `openshell ... 2> build.log` from a terminal has a @@ -95,13 +104,13 @@ pub fn init(choice: ColorChoice) { choice, no_color.as_deref(), force_color.as_deref(), - std::io::stdout().is_terminal(), + terminal_supports_ansi(std::io::stdout().is_terminal(), term.as_deref()), ); let stderr_enabled = resolve( choice, no_color.as_deref(), force_color.as_deref(), - std::io::stderr().is_terminal(), + terminal_supports_ansi(std::io::stderr().is_terminal(), term.as_deref()), ); STDOUT_ENABLED.store(stdout_enabled, Ordering::Relaxed); STDERR_ENABLED.store(stderr_enabled, Ordering::Relaxed); @@ -159,7 +168,7 @@ fn resolve( choice: ColorChoice, no_color: Option<&OsStr>, force_color: Option<&OsStr>, - stream_is_terminal: bool, + stream_supports_ansi: bool, ) -> bool { match choice { ColorChoice::Always => return true, @@ -177,7 +186,39 @@ fn resolve( return true; } + // Only `auto` consults the terminal. An explicit request above has already + // returned, so `--color always` and `FORCE_COLOR` still win on a terminal + // that reports no ANSI support. + stream_supports_ansi +} + +/// Whether this output stream's terminal renders ANSI escapes. +/// +/// Being attached to a terminal is not the same as that terminal rendering +/// ANSI. `TERM` is the unix signal for it; Windows consoles enable virtual +/// terminal processing instead and do not set `TERM`, so the check does not +/// apply there. +fn terminal_supports_ansi(stream_is_terminal: bool, term: Option<&OsStr>) -> bool { stream_is_terminal + && if cfg!(unix) { + term_supports_ansi(term) + } else { + true + } +} + +/// Whether the terminal named by `TERM` renders ANSI escapes on Unix. +/// +/// Follows the rule `console` applies on unix, which this module overrides: +/// `dumb` means no, and an unset `TERM` means no because nothing identifies a +/// capable terminal. +/// +/// Empty is treated as unset, which is a deliberate divergence: `console` reads +/// `TERM=""` as `Ok("")`, and since that is not `"dumb"` it counts as capable. +/// An empty value names no terminal type, and every other variable here already +/// treats empty as unset, so it is handled the same way. +fn term_supports_ansi(term: Option<&OsStr>) -> bool { + is_set(term) && term != Some(OsStr::new("dumb")) } /// Whether an environment variable counts as set: present and not empty. @@ -329,7 +370,7 @@ mod tests { // and make every `.green()` below ambiguous. use super::{ ColorChoice, Colorize, Ordering, STDERR_ENABLED, STDOUT_ENABLED, Style, painted_enabled, - resolve, + resolve, term_supports_ansi, }; use std::ffi::OsStr; @@ -500,6 +541,44 @@ mod tests { assert!(!resolve(ColorChoice::Auto, None, Some(env("")), false)); } + #[test] + fn term_capability_follows_the_console_rule() { + assert!(term_supports_ansi(Some(OsStr::new("xterm-256color")))); + assert!(term_supports_ansi(Some(OsStr::new("screen")))); + assert!(!term_supports_ansi(Some(OsStr::new("dumb")))); + // Nothing to suggest a capable terminal, so assume none. + assert!(!term_supports_ansi(None)); + // Empty names no terminal type; treated as unset, unlike `console`. + assert!(!term_supports_ansi(Some(OsStr::new("")))); + // Only an exact match counts; `dumb-something` is a different terminal. + assert!(term_supports_ansi(Some(OsStr::new("dumb-but-color")))); + } + + #[test] + fn auto_does_not_style_an_incapable_terminal() { + // A `dumb` terminal is still a terminal, so `is_terminal()` alone would + // wrongly enable color. + assert!(!resolve(ColorChoice::Auto, None, None, false)); + assert!(resolve(ColorChoice::Auto, None, None, true)); + } + + #[test] + fn explicit_requests_outrank_terminal_capability() { + // `--color always` and FORCE_COLOR are for callers who know better than + // the detection, so an incapable terminal must not veto them. + assert!(resolve(ColorChoice::Always, None, None, false)); + assert!(resolve(ColorChoice::Auto, None, Some(env("1")), false)); + // The negative direction still wins over capability too. + assert!(!resolve(ColorChoice::Never, None, None, true)); + assert!(!resolve(ColorChoice::Auto, Some(env("1")), None, true)); + } + + #[test] + fn capability_does_not_rescue_a_redirected_stream() { + // Capability is an additional requirement, not an alternative one. + assert!(!resolve(ColorChoice::Auto, None, None, false)); + } + #[test] fn auto_resolves_each_stream_independently() { // `openshell ... 2> build.log` from a terminal: stdout is styled, the diff --git a/crates/openshell-cli/src/main.rs b/crates/openshell-cli/src/main.rs index 7c042296f4..befac54759 100644 --- a/crates/openshell-cli/src/main.rs +++ b/crates/openshell-cli/src/main.rs @@ -1351,6 +1351,10 @@ enum SandboxCommands { #[arg(long, add = ArgValueCompleter::new(completers::complete_sandbox_names))] name: Option, + /// Create the sandbox from a named sandbox template. + #[arg(long, conflicts_with_all = ["from", "gpu", "cpu", "memory", "driver_config_json", "envs"])] + template: Option, + /// Sandbox source: a community sandbox name (e.g., `ollama`), a path /// to a Dockerfile or directory containing one, or a full container /// image reference (e.g., `myregistry.com/img:tag`). @@ -1693,6 +1697,10 @@ enum SandboxCommands { /// Manage providers attached to a sandbox. #[command(subcommand)] Provider(SandboxProviderCommands), + + /// Manage reusable sandbox workload templates. + #[command(subcommand)] + Template(SandboxTemplateCommands), } #[derive(Subcommand, Debug)] @@ -1734,6 +1742,121 @@ enum SandboxProviderCommands { }, } +#[derive(Subcommand, Debug)] +// `Create` carries several optional strings and repeated key-value flags. This +// enum is only used for clap parsing, so boxing fields would add friction +// without a meaningful runtime win. +#[allow(clippy::large_enum_variant)] +enum SandboxTemplateCommands { + /// Create a reusable sandbox workload template. + #[command(help_template = LEAF_HELP_TEMPLATE, next_help_heading = "FLAGS")] + Create { + /// Template name. + name: String, + + /// Container image for sandboxes created from this template. + /// When omitted, the gateway's default sandbox image is used at create time. + #[arg(long)] + image: Option, + + /// CPU limit for sandboxes created from this template (for example: 500m, 1, 2.5). + #[arg(long)] + cpu: Option, + + /// Memory limit for sandboxes created from this template (for example: 512Mi, 4Gi, 8G). + #[arg(long)] + memory: Option, + + /// Request GPU resources for sandboxes created from this template. + /// + /// Omit COUNT for the driver's default GPU selection, or pass COUNT + /// to request a specific number of GPUs. + #[arg(long, num_args = 0..=1, value_name = "COUNT", default_missing_value = "", value_parser = parse_gpu_request)] + gpu: Option, + + /// Experimental driver-keyed JSON object for driver-specific sandbox settings. + #[arg(long, value_name = "JSON")] + driver_config_json: Option, + + /// Target startup readiness duration for this template (for example: 30s, 5m, 1h). + #[arg(long, value_name = "DURATION")] + ready_within: Option, + + /// Maximum startup burst associated with this template. + #[arg(long, value_name = "COUNT", value_parser = clap::value_parser!(u32).range(1..))] + max_burst: Option, + + /// Attach labels to the template (key=value format, repeatable). + #[arg(long = "label", value_name = "KEY=VALUE")] + labels: Vec, + + /// Attach annotations to the template (key=value format, repeatable). + #[arg(long = "annotation", value_name = "KEY=VALUE")] + annotations: Vec, + + /// Set a non-secret environment variable in sandboxes created from this template. + /// Do not use this option for API keys, tokens, or other secrets; create + /// a provider and attach it when creating sandboxes instead. Repeatable. + #[arg(long = "env", value_name = "KEY=VALUE")] + envs: Vec, + + /// Suppress warnings when --env values look like credentials. + #[arg(long = "no-credential-warnings")] + no_credential_warnings: bool, + + /// Output format. + #[arg(short = 'o', long = "output", value_enum, default_value_t = OutputFormat::Table)] + output: OutputFormat, + }, + + /// Fetch a sandbox workload template by name. + #[command(help_template = LEAF_HELP_TEMPLATE, next_help_heading = "FLAGS")] + Get { + /// Template name. + name: String, + + /// Output format. + #[arg(short = 'o', long = "output", value_enum, default_value_t = OutputFormat::Table)] + output: OutputFormat, + }, + + /// List sandbox workload templates. + #[command(help_template = LEAF_HELP_TEMPLATE, next_help_heading = "FLAGS")] + List { + /// Maximum number of templates to return. + #[arg(long, default_value_t = 100)] + limit: u32, + + /// Offset into the template list. + #[arg(long, default_value_t = 0)] + offset: u32, + + /// Filter templates by labels, e.g. env=prod,team=runtime. + #[arg(long)] + label_selector: Option, + + /// Print only template names (one per line). + #[arg(long, conflicts_with = "output")] + names: bool, + + /// Output format. + #[arg(short = 'o', long = "output", value_enum, default_value_t = OutputFormat::Table, conflicts_with = "names")] + output: OutputFormat, + + /// List templates across all workspaces (overrides --workspace). + #[arg(long)] + all_workspaces: bool, + }, + + /// Delete sandbox workload templates. + #[command(help_template = LEAF_HELP_TEMPLATE, next_help_heading = "FLAGS")] + Delete { + /// Template names. + #[arg(required = true, num_args = 1.., value_name = "NAME")] + names: Vec, + }, +} + #[derive(Subcommand, Debug)] enum DraftCommands { /// Show network rules for a sandbox. @@ -3064,6 +3187,7 @@ async fn run_async() -> Result<()> { match command { SandboxCommands::Create { name, + template, from, upload, no_git_ignore, @@ -3157,6 +3281,7 @@ async fn run_async() -> Result<()> { &ctx.name, run::SandboxCreateConfig { name: name.as_deref(), + template: template.as_deref(), from: from.as_deref(), uploads: &upload_specs, keep, @@ -3393,6 +3518,89 @@ async fn run_async() -> Result<()> { .await?; } }, + SandboxCommands::Template(command) => match command { + SandboxTemplateCommands::Create { + name, + image, + cpu, + memory, + gpu, + driver_config_json, + ready_within, + max_burst, + labels, + annotations, + envs, + no_credential_warnings, + output, + } => { + let labels = run::parse_key_value_pairs(&labels, "--label")?; + let annotations = + run::parse_key_value_pairs(&annotations, "--annotation")?; + let environment = run::parse_env_pairs(&envs)?; + run::warn_credential_env_vars(&environment, no_credential_warnings); + let gpu_requirements: Option = + gpu.map(Into::into); + run::sandbox_template_create( + endpoint, + &name, + image.as_deref(), + cpu.as_deref(), + memory.as_deref(), + gpu_requirements, + driver_config_json.as_deref(), + ready_within.as_deref(), + max_burst, + labels, + annotations, + environment, + output.as_str(), + &cli.workspace, + &tls, + ) + .await?; + } + SandboxTemplateCommands::Get { name, output } => { + run::sandbox_template_get( + endpoint, + &name, + output.as_str(), + &cli.workspace, + &tls, + ) + .await?; + } + SandboxTemplateCommands::List { + limit, + offset, + label_selector, + names, + output, + all_workspaces, + } => { + run::sandbox_template_list( + endpoint, + limit, + offset, + label_selector.as_deref(), + names, + output.as_str(), + &cli.workspace, + all_workspaces, + &tls, + ) + .await?; + } + SandboxTemplateCommands::Delete { names } => { + run::sandbox_template_delete( + endpoint, + &names, + &cli.workspace, + &tls, + ) + .await?; + } + }, } } } @@ -5611,6 +5819,226 @@ mod tests { } } + #[test] + fn sandbox_create_template_flag_parses() { + let cli = Cli::try_parse_from([ + "openshell", + "sandbox", + "create", + "--template", + "gpu-kata", + "--provider", + "github", + ]) + .expect("sandbox create template flag should parse"); + + match cli.command { + Some(Commands::Sandbox { + command: + Some(SandboxCommands::Create { + template, + providers, + .. + }), + .. + }) => { + assert_eq!(template.as_deref(), Some("gpu-kata")); + assert_eq!(providers, vec!["github".to_string()]); + } + other => panic!("expected SandboxCommands::Create, got: {other:?}"), + } + } + + #[test] + fn sandbox_create_template_conflicts_with_inline_workload_flags() { + for (label, extra_args) in [ + ("--from", &["--from", "python:3.12"][..]), + ("--gpu", &["--gpu"][..]), + ("--cpu", &["--cpu", "1"][..]), + ("--memory", &["--memory", "2Gi"][..]), + ("--env", &["--env", "FOO=bar"][..]), + ( + "--driver-config-json", + &["--driver-config-json", r#"{"kubernetes":{}}"#][..], + ), + ] { + let args = ["openshell", "sandbox", "create", "--template", "base"] + .into_iter() + .chain(extra_args.iter().copied()); + let result = Cli::try_parse_from(args); + assert!(result.is_err(), "--template should conflict with {label}"); + } + } + + #[test] + fn sandbox_template_create_parses_workload_flags() { + let json = r#"{"kubernetes":{"pod":{"node_selector":{"pool":"gpu"}}}}"#; + let cli = Cli::try_parse_from([ + "openshell", + "sandbox", + "template", + "create", + "gpu-kata", + "--image", + "registry.example.com/agent:latest", + "--cpu", + "2", + "--memory", + "4Gi", + "--gpu", + "1", + "--driver-config-json", + json, + "--ready-within", + "5m", + "--max-burst", + "3", + "--label", + "team=runtime", + "--annotation", + "owner=platform", + "--env", + "FEATURE_FLAG=on", + "--no-credential-warnings", + "--output", + "json", + ]) + .expect("sandbox template create should parse"); + + match cli.command { + Some(Commands::Sandbox { + command: + Some(SandboxCommands::Template(SandboxTemplateCommands::Create { + name, + image, + cpu, + memory, + gpu, + driver_config_json, + ready_within, + max_burst, + labels, + annotations, + envs, + no_credential_warnings, + output, + })), + .. + }) => { + assert_eq!(name, "gpu-kata"); + assert_eq!(image.as_deref(), Some("registry.example.com/agent:latest")); + assert_eq!(cpu.as_deref(), Some("2")); + assert_eq!(memory.as_deref(), Some("4Gi")); + assert_eq!(gpu, Some(GpuCliRequest::Count(1))); + assert_eq!(driver_config_json.as_deref(), Some(json)); + assert_eq!(ready_within.as_deref(), Some("5m")); + assert_eq!(max_burst, Some(3)); + assert_eq!(labels, vec!["team=runtime".to_string()]); + assert_eq!(annotations, vec!["owner=platform".to_string()]); + assert_eq!(envs, vec!["FEATURE_FLAG=on".to_string()]); + assert!(no_credential_warnings); + assert!(matches!(output, OutputFormat::Json)); + } + other => panic!("expected SandboxTemplateCommands::Create, got: {other:?}"), + } + } + + #[test] + fn sandbox_template_list_parses_names_and_all_workspaces() { + let cli = Cli::try_parse_from([ + "openshell", + "sandbox", + "template", + "list", + "--names", + "--all-workspaces", + "--label-selector", + "team=runtime", + "--limit", + "25", + "--offset", + "5", + ]) + .expect("sandbox template list should parse"); + + match cli.command { + Some(Commands::Sandbox { + command: + Some(SandboxCommands::Template(SandboxTemplateCommands::List { + limit, + offset, + label_selector, + names, + all_workspaces, + .. + })), + .. + }) => { + assert_eq!(limit, 25); + assert_eq!(offset, 5); + assert_eq!(label_selector.as_deref(), Some("team=runtime")); + assert!(names); + assert!(all_workspaces); + } + other => panic!("expected SandboxTemplateCommands::List, got: {other:?}"), + } + } + + #[test] + fn sandbox_template_list_names_conflicts_with_output() { + let result = Cli::try_parse_from([ + "openshell", + "sandbox", + "template", + "list", + "--names", + "--output", + "json", + ]); + assert!(result.is_err()); + } + + #[test] + fn sandbox_template_create_image_is_optional() { + let cli = Cli::try_parse_from(["openshell", "sandbox", "template", "create", "base"]) + .expect("sandbox template create without --image should parse"); + + match cli.command { + Some(Commands::Sandbox { + command: + Some(SandboxCommands::Template(SandboxTemplateCommands::Create { image, .. })), + .. + }) => { + assert_eq!(image, None); + } + other => panic!("expected SandboxTemplateCommands::Create, got: {other:?}"), + } + } + + #[test] + fn sandbox_template_create_gpu_parses_driver_default() { + let cli = Cli::try_parse_from([ + "openshell", + "sandbox", + "template", + "create", + "gpu-kata", + "--gpu", + ]) + .expect("sandbox template create --gpu should parse"); + + match cli.command { + Some(Commands::Sandbox { + command: + Some(SandboxCommands::Template(SandboxTemplateCommands::Create { gpu, .. })), + .. + }) => { + assert_eq!(gpu, Some(GpuCliRequest::DriverDefault)); + } + other => panic!("expected SandboxTemplateCommands::Create, got: {other:?}"), + } + } + #[test] fn sandbox_create_gpu_parses_driver_default() { let cli = Cli::try_parse_from(["openshell", "sandbox", "create", "--gpu"]) diff --git a/crates/openshell-cli/src/run.rs b/crates/openshell-cli/src/run.rs index 36cdaf01ba..4cd4e34a76 100644 --- a/crates/openshell-cli/src/run.rs +++ b/crates/openshell-cli/src/run.rs @@ -37,26 +37,28 @@ use openshell_core::proto::ProviderProfileCategory; use openshell_core::proto::{ ApproveAllDraftChunksRequest, ApproveDraftChunkRequest, AttachSandboxProviderRequest, ClearDraftChunksRequest, ConfigureProviderRefreshRequest, CreateProviderRequest, - CreateSandboxRequest, CreateSshSessionRequest, DeleteInferenceRouteRequest, - DeleteProviderProfileRequest, DeleteProviderRefreshRequest, DeleteProviderRequest, - DeleteSandboxRequest, DeleteServiceRequest, DetachSandboxProviderRequest, ExecSandboxRequest, - ExposeServiceRequest, GetCurrentUserRequest, GetDraftHistoryRequest, GetDraftPolicyRequest, - GetGatewayConfigRequest, GetInferenceRouteRequest, GetProviderProfileRequest, - GetProviderRefreshStatusRequest, GetProviderRequest, GetSandboxConfigRequest, - GetSandboxConfigResponse, GetSandboxLogsRequest, GetSandboxPolicyStatusRequest, - GetSandboxRequest, GetServiceRequest, GpuResourceRequirements, ImportProviderProfilesRequest, - LintProviderProfilesRequest, ListProviderProfilesRequest, ListProvidersRequest, - ListSandboxPoliciesRequest, ListSandboxProvidersRequest, ListSandboxesRequest, + CreateSandboxRequest, CreateSandboxTemplateRequest, CreateSshSessionRequest, + DeleteInferenceRouteRequest, DeleteProviderProfileRequest, DeleteProviderRefreshRequest, + DeleteProviderRequest, DeleteSandboxRequest, DeleteSandboxTemplateRequest, + DeleteServiceRequest, DetachSandboxProviderRequest, ExecSandboxRequest, ExposeServiceRequest, + GetCurrentUserRequest, GetDraftHistoryRequest, GetDraftPolicyRequest, GetGatewayConfigRequest, + GetInferenceRouteRequest, GetProviderProfileRequest, GetProviderRefreshStatusRequest, + GetProviderRequest, GetSandboxConfigRequest, GetSandboxConfigResponse, GetSandboxLogsRequest, + GetSandboxPolicyStatusRequest, GetSandboxRequest, GetSandboxTemplateRequest, GetServiceRequest, + GpuResourceRequirements, ImportProviderProfilesRequest, LintProviderProfilesRequest, + ListProviderProfilesRequest, ListProvidersRequest, ListSandboxPoliciesRequest, + ListSandboxProvidersRequest, ListSandboxTemplatesRequest, ListSandboxesRequest, ListServicesRequest, PolicySource, PolicyStatus, Provider, ProviderCredentialRefreshRecoveryAction, ProviderCredentialRefreshStatus, ProviderCredentialRefreshStrategy, ProviderCredentialTokenGrantType, ProviderProfile, ProviderProfileDiagnostic, ProviderProfileImportItem, RejectDraftChunkRequest, ResourceRequirements, RevokeSshSessionRequest, RotateProviderCredentialRequest, Sandbox, - SandboxPhase, SandboxPolicy, SandboxSpec, SandboxTemplate, ServiceEndpointResponse, - SetInferenceRouteRequest, SettingScope, StartSandboxRequest, StopSandboxRequest, - TcpForwardFrame, TcpForwardInit, TcpRelayTarget, UpdateConfigRequest, - UpdateProviderProfilesRequest, UpdateProviderRequest, WatchSandboxRequest, exec_sandbox_event, - tcp_forward_init, + SandboxPhase, SandboxPolicy, SandboxResources, SandboxServiceLevel, SandboxSpec, + SandboxStartup, SandboxTemplate, SandboxWorkloadConfig, SandboxWorkloadTemplate, + SandboxWorkloadTemplateSpec, ServiceEndpointResponse, SetInferenceRouteRequest, SettingScope, + StartSandboxRequest, StopSandboxRequest, TcpForwardFrame, TcpForwardInit, TcpRelayTarget, + UpdateConfigRequest, UpdateProviderProfilesRequest, UpdateProviderRequest, WatchSandboxRequest, + exec_sandbox_event, tcp_forward_init, }; use openshell_core::settings; use openshell_core::{ObjectId, ObjectName, ObjectWorkspace}; @@ -400,6 +402,7 @@ async fn finalize_sandbox_create_session( #[derive(Debug)] pub struct SandboxCreateConfig<'a> { pub name: Option<&'a str>, + pub template: Option<&'a str>, pub from: Option<&'a str>, pub uploads: &'a [(String, Option, bool)], pub keep: bool, @@ -425,6 +428,7 @@ impl Default for SandboxCreateConfig<'_> { fn default() -> Self { Self { name: None, + template: None, from: None, uploads: &[], keep: false, @@ -458,6 +462,7 @@ pub async fn sandbox_create( ) -> Result { let SandboxCreateConfig { name, + template, from, uploads, keep, @@ -511,23 +516,42 @@ pub async fn sandbox_create( let effective_server = server.to_string(); let effective_tls = tls.clone(); + if template.is_some() + && (from.is_some() + || gpu_requirements.is_some() + || cpu.is_some() + || memory.is_some() + || driver_config_json.is_some() + || !environment.is_empty()) + { + return Err(miette::miette!( + "--template cannot be combined with inline workload flags" + )); + } + // Resolve the --from flag into a container image reference, building from - // a Dockerfile first if necessary. - let image: Option = match from { - Some(val) => { - let resolved = resolve_from(val)?; - match resolved { - ResolvedSource::Image(img) => Some(img), - ResolvedSource::Dockerfile { - dockerfile, - context, - } => { - let tag = build_from_dockerfile(&dockerfile, &context, gateway_name).await?; - Some(tag) + // a Dockerfile first if necessary. Template creates resolve workload shape + // on the gateway and skip local image handling. + let image: Option = if template.is_some() { + None + } else { + match from { + Some(val) => { + let resolved = resolve_from(val)?; + match resolved { + ResolvedSource::Image(img) => Some(img), + ResolvedSource::Dockerfile { + dockerfile, + context, + } => { + let tag = + build_from_dockerfile(&dockerfile, &context, gateway_name).await?; + Some(tag) + } } } + None => None, } - None => None, }; let inferred_types: Vec = inferred_provider_type(command).into_iter().collect(); let configured_providers = ensure_required_providers( @@ -540,12 +564,21 @@ pub async fn sandbox_create( .await?; let policy = load_sandbox_policy(policy)?; - let resource_limits = build_sandbox_resource_limits(cpu, memory)?; - let driver_config = driver_config_json - .map(parse_driver_config_json) - .transpose()?; + let resource_limits = if template.is_none() { + build_sandbox_resource_limits(cpu, memory)? + } else { + None + }; + let driver_config = if template.is_none() { + driver_config_json + .map(parse_driver_config_json) + .transpose()? + } else { + None + }; - let template = if image.is_some() || resource_limits.is_some() || driver_config.is_some() { + let inline_template = if image.is_some() || resource_limits.is_some() || driver_config.is_some() + { Some(SandboxTemplate { image: image.unwrap_or_default(), resources: resource_limits, @@ -582,10 +615,14 @@ pub async fn sandbox_create( let request = CreateSandboxRequest { spec: Some(SandboxSpec { resource_requirements, - environment, + environment: if template.is_none() { + environment + } else { + HashMap::new() + }, policy, providers: configured_providers, - template, + template: inline_template, command: main_command, tty: main_terminal, ..SandboxSpec::default() @@ -595,6 +632,7 @@ pub async fn sandbox_create( annotations, workspace: workspace.to_string(), await_main_process_attachment, + workload_template_name: template.unwrap_or_default().to_string(), }; let response = match client.create_sandbox(request).await { @@ -1441,6 +1479,15 @@ pub async fn sandbox_get( } } + if let Some(provenance) = &sandbox.created_from_workload_template { + println!( + " {} {}@{}", + "Workload template:".dimmed(), + provenance.name, + provenance.resource_version + ); + } + let policy_from_global = config.policy_source == PolicySource::Global as i32; println!( " {} {}", @@ -2226,6 +2273,16 @@ fn sandbox_to_json(sandbox: &Sandbox) -> serde_json::Value { || serde_json::json!({}), |m| serde_json::json!(m.annotations), ); + let created_from_workload_template = + sandbox + .created_from_workload_template + .as_ref() + .map(|provenance| { + serde_json::json!({ + "name": provenance.name, + "resource_version": provenance.resource_version, + }) + }); serde_json::json!({ "id": sandbox.object_id(), "name": sandbox.object_name(), @@ -2237,6 +2294,7 @@ fn sandbox_to_json(sandbox: &Sandbox) -> serde_json::Value { "phase": phase_name(sandbox.phase()), "current_policy_version": sandbox.current_policy_version(), "exit_code": sandbox.status.as_ref().and_then(|status| status.exit_code), + "created_from_workload_template": created_from_workload_template, }) } @@ -2496,6 +2554,583 @@ fn format_provider_attachment_table(providers: &[Provider], color: bool) -> Stri output } +#[allow(clippy::too_many_arguments, clippy::implicit_hasher)] +pub async fn sandbox_template_create( + server: &str, + name: &str, + image: Option<&str>, + cpu: Option<&str>, + memory: Option<&str>, + gpu_requirements: Option, + driver_config_json: Option<&str>, + ready_within: Option<&str>, + max_burst: Option, + labels: HashMap, + annotations: HashMap, + environment: HashMap, + output: &str, + workspace: &str, + tls: &TlsOptions, +) -> Result<()> { + let resources = if cpu.is_some() || memory.is_some() || gpu_requirements.is_some() { + Some(SandboxResources { + cpu: cpu + .map(validate_cpu_quantity) + .transpose()? + .unwrap_or_default(), + memory: memory + .map(validate_memory_quantity) + .transpose()? + .unwrap_or_default(), + gpu: gpu_requirements, + }) + } else { + None + }; + let driver_config = driver_config_json + .map(parse_driver_config_json) + .transpose()?; + let desired_service_level = build_template_service_level(ready_within, max_burst)?; + + let mut client = grpc_client(server, tls).await?; + let response = client + .create_sandbox_template(CreateSandboxTemplateRequest { + template: Some(SandboxWorkloadTemplate { + metadata: Some(openshell_core::proto::datamodel::v1::ObjectMeta { + id: String::new(), + name: name.to_string(), + created_at_ms: 0, + labels, + resource_version: 0, + annotations, + workspace: String::new(), + deletion_timestamp_ms: 0, + }), + spec: Some(SandboxWorkloadTemplateSpec { + workload: Some(SandboxWorkloadConfig { + image: image.unwrap_or_default().to_string(), + environment, + resources, + }), + driver_config, + desired_service_level, + }), + }), + workspace: workspace.to_string(), + }) + .await + .into_diagnostic()?; + + let template = response + .into_inner() + .template + .ok_or_else(|| miette!("sandbox template missing from response"))?; + if crate::output::print_output_single(output, &template, sandbox_template_to_json)? { + return Ok(()); + } + println!( + "{} Created sandbox template {}", + "✓".green().bold(), + template.object_name().bold() + ); + Ok(()) +} + +fn build_template_service_level( + ready_within: Option<&str>, + max_burst: Option, +) -> Result> { + if ready_within.is_none() && max_burst.is_none() { + return Ok(None); + } + let ready_within = ready_within + .map(parse_duration_to_ms) + .transpose()? + .map(|ms| { + if ms <= 0 { + Err(miette!("--ready-within must be greater than zero")) + } else { + Ok(duration_ms_to_proto(ms)) + } + }) + .transpose()?; + Ok(Some(SandboxServiceLevel { + startup: Some(SandboxStartup { + ready_within, + max_burst: max_burst.unwrap_or_default(), + }), + })) +} + +fn duration_ms_to_proto(ms: i64) -> prost_types::Duration { + prost_types::Duration { + seconds: ms / 1_000, + nanos: i32::try_from((ms % 1_000) * 1_000_000) + .expect("duration millisecond remainder fits in protobuf nanos"), + } +} + +pub async fn sandbox_template_get( + server: &str, + name: &str, + output: &str, + workspace: &str, + tls: &TlsOptions, +) -> Result<()> { + let mut client = grpc_client(server, tls).await?; + let response = client + .get_sandbox_template(GetSandboxTemplateRequest { + name: name.to_string(), + workspace: workspace.to_string(), + }) + .await + .into_diagnostic()?; + let template = response + .into_inner() + .template + .ok_or_else(|| miette!("sandbox template missing from response"))?; + + if crate::output::print_output_single(output, &template, sandbox_template_to_json)? { + return Ok(()); + } + + print_sandbox_template_detail(&template); + Ok(()) +} + +#[allow(clippy::too_many_arguments)] +pub async fn sandbox_template_list( + server: &str, + limit: u32, + offset: u32, + label_selector: Option<&str>, + names_only: bool, + output: &str, + workspace: &str, + all_workspaces: bool, + tls: &TlsOptions, +) -> Result<()> { + let mut client = grpc_client(server, tls).await?; + let response = client + .list_sandbox_templates(ListSandboxTemplatesRequest { + limit, + offset, + workspace: if all_workspaces { + String::new() + } else { + workspace.to_string() + }, + all_workspaces, + label_selector: label_selector.unwrap_or_default().to_string(), + }) + .await + .into_diagnostic()?; + let templates = response.into_inner().templates; + + if crate::output::print_output_collection(output, &templates, sandbox_template_to_json)? { + return Ok(()); + } + + if templates.is_empty() { + if !names_only { + println!("No sandbox templates found."); + } + return Ok(()); + } + + if names_only { + for template in &templates { + if all_workspaces { + println!("{}/{}", template.object_workspace(), template.object_name()); + } else { + println!("{}", template.object_name()); + } + } + return Ok(()); + } + + print_sandbox_template_table(&templates, all_workspaces); + Ok(()) +} + +pub async fn sandbox_template_delete( + server: &str, + names: &[String], + workspace: &str, + tls: &TlsOptions, +) -> Result<()> { + let mut client = grpc_client(server, tls).await?; + for name in names { + let response = client + .delete_sandbox_template(DeleteSandboxTemplateRequest { + name: name.clone(), + workspace: workspace.to_string(), + }) + .await + .into_diagnostic()?; + if response.into_inner().deleted { + println!("{} Deleted sandbox template {name}", "✓".green().bold()); + } else { + println!("Sandbox template {name} not found."); + } + } + Ok(()) +} + +fn sandbox_template_to_json(template: &SandboxWorkloadTemplate) -> serde_json::Value { + let mut obj = serde_json::Map::new(); + obj.insert("id".to_string(), serde_json::json!(template.object_id())); + obj.insert( + "name".to_string(), + serde_json::json!(template.object_name()), + ); + obj.insert( + "workspace".to_string(), + serde_json::json!(template.object_workspace()), + ); + + if let Some(metadata) = &template.metadata { + if metadata.resource_version != 0 { + obj.insert( + "resource_version".to_string(), + serde_json::json!(metadata.resource_version), + ); + } + if metadata.created_at_ms != 0 { + obj.insert( + "created_at".to_string(), + serde_json::json!(format_epoch_ms(metadata.created_at_ms)), + ); + } + if !metadata.labels.is_empty() { + obj.insert("labels".to_string(), serde_json::json!(metadata.labels)); + } + if !metadata.annotations.is_empty() { + obj.insert( + "annotations".to_string(), + serde_json::json!(metadata.annotations), + ); + } + } + + if let Some(spec) = &template.spec { + if let Some(workload) = &spec.workload { + obj.insert("image".to_string(), serde_json::json!(workload.image)); + if !workload.environment.is_empty() { + obj.insert( + "environment".to_string(), + serde_json::json!(workload.environment), + ); + } + if let Some(resources) = &workload.resources { + let mut resources_json = serde_json::Map::new(); + if !resources.cpu.is_empty() { + resources_json.insert("cpu".to_string(), serde_json::json!(resources.cpu)); + } + if !resources.memory.is_empty() { + resources_json + .insert("memory".to_string(), serde_json::json!(resources.memory)); + } + if let Some(gpu) = &resources.gpu { + let value = gpu + .count + .map_or_else(|| serde_json::json!("default"), serde_json::Value::from); + resources_json.insert("gpu".to_string(), value); + } + if !resources_json.is_empty() { + obj.insert( + "resources".to_string(), + serde_json::Value::Object(resources_json), + ); + } + } + } + if let Some(driver_config) = &spec.driver_config { + obj.insert( + "driver_config".to_string(), + openshell_core::proto_struct::struct_to_json_value(driver_config), + ); + } + if let Some(service_level) = &spec.desired_service_level + && let Some(startup) = &service_level.startup + { + let mut startup_json = serde_json::Map::new(); + if let Some(ready_within) = &startup.ready_within { + startup_json.insert( + "ready_within_ms".to_string(), + serde_json::json!(duration_to_ms(ready_within)), + ); + } + if startup.max_burst != 0 { + startup_json.insert( + "max_burst".to_string(), + serde_json::json!(startup.max_burst), + ); + } + if !startup_json.is_empty() { + obj.insert( + "startup".to_string(), + serde_json::Value::Object(startup_json), + ); + } + } + } + + serde_json::Value::Object(obj) +} + +fn print_sandbox_template_detail(template: &SandboxWorkloadTemplate) { + println!("{}", "Sandbox template:".cyan().bold()); + println!(); + println!(" {} {}", "Name:".dimmed(), template.object_name()); + println!( + " {} {}", + "Workspace:".dimmed(), + template.object_workspace() + ); + if let Some(metadata) = &template.metadata { + println!(" {} {}", "Id:".dimmed(), metadata.id); + println!( + " {} {}", + "Resource version:".dimmed(), + metadata.resource_version + ); + if metadata.created_at_ms != 0 { + println!( + " {} {}", + "Created:".dimmed(), + format_epoch_ms(metadata.created_at_ms) + ); + } + let labels = labels_display(&metadata.labels); + println!( + " {} {}", + "Labels:".dimmed(), + non_empty_or(&labels, "") + ); + } + if let Some(spec) = &template.spec + && let Some(workload) = &spec.workload + { + println!( + " {} {}", + "Image:".dimmed(), + non_empty_or(&workload.image, "") + ); + println!( + " {} {}", + "Environment:".dimmed(), + workload.environment.len() + ); + if let Some(resources) = &workload.resources { + println!( + " {} {}", + "CPU:".dimmed(), + non_empty_or(&resources.cpu, "") + ); + println!( + " {} {}", + "Memory:".dimmed(), + non_empty_or(&resources.memory, "") + ); + println!( + " {} {}", + "GPU:".dimmed(), + template_resources_gpu_display(resources).unwrap_or_else(|| "".to_string()) + ); + } + } + if let Some(startup) = template_startup(template) { + println!( + " {} {}", + "Ready within:".dimmed(), + startup + .ready_within + .as_ref() + .map_or_else(|| "".to_string(), duration_display) + ); + println!( + " {} {}", + "Max burst:".dimmed(), + if startup.max_burst == 0 { + "".to_string() + } else { + startup.max_burst.to_string() + } + ); + } +} + +fn print_sandbox_template_table(templates: &[SandboxWorkloadTemplate], show_workspace: bool) { + let name_width = templates + .iter() + .map(|template| template.object_name().len()) + .max() + .unwrap_or(4) + .max(4); + let workspace_width = if show_workspace { + templates + .iter() + .map(|template| template.object_workspace().len()) + .max() + .unwrap_or(9) + .max(9) + } else { + 0 + }; + let image_width = templates + .iter() + .map(|template| template_image(template).len()) + .max() + .unwrap_or(5) + .clamp(5, 48); + + if show_workspace { + println!( + "{: String { + template + .spec + .as_ref() + .and_then(|spec| spec.workload.as_ref()) + .map_or_else( + || "".to_string(), + |workload| non_empty_or(&workload.image, "").to_string(), + ) +} + +fn template_resources(template: &SandboxWorkloadTemplate) -> Option<&SandboxResources> { + template + .spec + .as_ref() + .and_then(|spec| spec.workload.as_ref()) + .and_then(|workload| workload.resources.as_ref()) +} + +fn template_resources_gpu_display(resources: &SandboxResources) -> Option { + if let Some(gpu) = &resources.gpu { + return Some( + gpu.count + .map_or_else(|| "default".to_string(), |count| count.to_string()), + ); + } + None +} + +fn template_startup(template: &SandboxWorkloadTemplate) -> Option<&SandboxStartup> { + template + .spec + .as_ref() + .and_then(|spec| spec.desired_service_level.as_ref()) + .and_then(|service_level| service_level.startup.as_ref()) +} + +fn duration_to_ms(duration: &prost_types::Duration) -> i64 { + duration.seconds.saturating_mul(1_000) + i64::from(duration.nanos / 1_000_000) +} + +fn duration_display(duration: &prost_types::Duration) -> String { + let total_ms = duration_to_ms(duration); + if total_ms % 3_600_000 == 0 { + format!("{}h", total_ms / 3_600_000) + } else if total_ms % 60_000 == 0 { + format!("{}m", total_ms / 60_000) + } else if total_ms % 1_000 == 0 { + format!("{}s", total_ms / 1_000) + } else { + format!("{total_ms}ms") + } +} + +fn labels_display(labels: &HashMap) -> String { + let mut pairs = labels + .iter() + .map(|(key, value)| format!("{key}={value}")) + .collect::>(); + pairs.sort(); + pairs.join(", ") +} + /// Delete a sandbox by name, or all sandboxes when `all` is true. pub async fn sandbox_delete( server: &str, @@ -4994,17 +5629,39 @@ pub async fn provider_update(options: ProviderUpdateOptions<'_>) -> Result<()> { } let mut client = grpc_client(server, tls).await?; + + // Look up the stored provider so the update can carry its type and profile + // workspace. Policy interceptors evaluate the request before the gateway + // merges it with stored state, so an update that omits them cannot be + // authorized against the profile that owns the provider. + // + // The read is best-effort. A caller holding `provider:write` without + // `provider:read` must still be able to rotate credentials, so a denied + // read keeps the previous behavior of sending empty metadata rather than + // failing the update. `--from-existing` and `--from-oidc-token` need the + // stored type, so they surface the error instead. + let existing = match client + .get_provider(GetProviderRequest { + name: name.to_string(), + workspace: workspace.to_string(), + }) + .await + { + Ok(response) => response.into_inner().provider, + Err(status) + if status.code() == Code::PermissionDenied && !from_existing && !from_oidc_token => + { + None + } + Err(status) => return Err(status).into_diagnostic(), + }; + + if existing.is_none() && (from_existing || from_oidc_token) { + return Err(miette::miette!("provider '{name}' not found")); + } + let oidc_profile = if from_oidc_token { - let existing = client - .get_provider(GetProviderRequest { - name: name.to_string(), - workspace: workspace.to_string(), - }) - .await - .into_diagnostic()? - .into_inner() - .provider - .ok_or_else(|| miette::miette!("provider '{name}' not found"))?; + let existing = existing.as_ref().expect("checked above"); Some( fetch_provider_profile(&mut client, &existing.r#type, &existing.profile_workspace) .await?, @@ -5023,25 +5680,11 @@ pub async fn provider_update(options: ProviderUpdateOptions<'_>) -> Result<()> { credential_expires_at_ms.extend(oidc_credential_expires_at_ms); if from_existing { - // Fetch the existing provider to discover its type for credential lookup. - let existing = client - .get_provider(GetProviderRequest { - name: name.to_string(), - workspace: workspace.to_string(), - }) - .await - .into_diagnostic()? - .into_inner() - .provider - .ok_or_else(|| miette::miette!("provider '{name}' not found"))?; - - let provider_type = existing.r#type; - let discovered = discover_existing_provider_data( - &mut client, - &provider_type, - &existing.profile_workspace, - ) - .await?; + let stored = existing.as_ref().expect("checked above"); + let provider_type = stored.r#type.clone(); + let discovered = + discover_existing_provider_data(&mut client, &provider_type, &stored.profile_workspace) + .await?; let Some(discovered) = discovered else { return Err(miette::miette!( "no existing local credentials/config found for provider type '{provider_type}'" @@ -5069,11 +5712,17 @@ pub async fn provider_update(options: ProviderUpdateOptions<'_>) -> Result<()> { workspace: workspace.to_string(), deletion_timestamp_ms: 0, }), - r#type: String::new(), + r#type: existing + .as_ref() + .map(|provider| provider.r#type.clone()) + .unwrap_or_default(), credentials: credential_map, config: config_map, credential_expires_at_ms: HashMap::new(), - profile_workspace: String::new(), + profile_workspace: existing + .as_ref() + .map(|provider| provider.profile_workspace.clone()) + .unwrap_or_default(), credential_handles: HashMap::new(), }), credential_expires_at_ms, @@ -7675,8 +8324,10 @@ mod tests { ProviderCredentialRefreshStatus, ProviderCredentialRefreshStrategy, ProviderCredentialTokenGrant, ProviderProfile, ProviderProfileCredential, ResourceRequirements, Sandbox, SandboxCondition, SandboxPhase, SandboxPolicy, - SandboxPolicyRevision, SandboxStatus, ServiceEndpoint, ServiceEndpointResponse, - WorkspaceMember, WorkspaceRole, datamodel::v1::ObjectMeta, + SandboxPolicyRevision, SandboxResources, SandboxStatus, SandboxWorkloadConfig, + SandboxWorkloadTemplate, SandboxWorkloadTemplateProvenance, SandboxWorkloadTemplateSpec, + ServiceEndpoint, ServiceEndpointResponse, WorkspaceMember, WorkspaceRole, + datamodel::v1::ObjectMeta, }; #[test] @@ -9139,6 +9790,74 @@ mod tests { assert_eq!(json["labels"]["env"], "prod"); } + #[test] + fn sandbox_template_to_json_includes_metadata_labels_and_annotations() { + let template = SandboxWorkloadTemplate { + metadata: Some(ObjectMeta { + id: "template-123".to_string(), + name: "gpu-kata".to_string(), + labels: std::collections::HashMap::from([( + "team".to_string(), + "runtime".to_string(), + )]), + annotations: std::collections::HashMap::from([( + "owner".to_string(), + "platform".to_string(), + )]), + workspace: "default".to_string(), + ..Default::default() + }), + ..Default::default() + }; + + let json = super::sandbox_template_to_json(&template); + + assert_eq!(json["labels"]["team"], "runtime"); + assert_eq!(json["annotations"]["owner"], "platform"); + } + + #[test] + fn sandbox_template_to_json_formats_default_gpu_like_display_output() { + let template = SandboxWorkloadTemplate { + spec: Some(SandboxWorkloadTemplateSpec { + workload: Some(SandboxWorkloadConfig { + resources: Some(SandboxResources { + gpu: Some(GpuResourceRequirements { count: None }), + ..Default::default() + }), + ..Default::default() + }), + ..Default::default() + }), + ..Default::default() + }; + + let json = super::sandbox_template_to_json(&template); + + assert_eq!(json["resources"]["gpu"], "default"); + } + + #[test] + fn sandbox_template_to_json_preserves_explicit_gpu_count_as_number() { + let template = SandboxWorkloadTemplate { + spec: Some(SandboxWorkloadTemplateSpec { + workload: Some(SandboxWorkloadConfig { + resources: Some(SandboxResources { + gpu: Some(GpuResourceRequirements { count: Some(2) }), + ..Default::default() + }), + ..Default::default() + }), + ..Default::default() + }), + ..Default::default() + }; + + let json = super::sandbox_template_to_json(&template); + + assert_eq!(json["resources"]["gpu"], 2); + } + #[test] fn provider_to_json_omits_zero_metadata_fields() { let metadata = ObjectMeta { @@ -9237,6 +9956,10 @@ mod tests { created_at_ms: 1_609_459_200_000, ..Default::default() }), + created_from_workload_template: Some(SandboxWorkloadTemplateProvenance { + name: "gpu-kata".to_string(), + resource_version: "7".to_string(), + }), ..Default::default() }; sandbox.set_phase(SandboxPhase::Ready as i32); @@ -9256,6 +9979,11 @@ mod tests { assert_eq!(json["policy_source"], "global"); assert_eq!(json["revision"], 3); assert!(json["policy"].is_null()); + assert_eq!(json["created_from_workload_template"]["name"], "gpu-kata"); + assert_eq!( + json["created_from_workload_template"]["resource_version"], + "7" + ); } #[test] diff --git a/crates/openshell-cli/tests/cli_color_integration.rs b/crates/openshell-cli/tests/cli_color_integration.rs index 23109e9774..54f2dc2f0a 100644 --- a/crates/openshell-cli/tests/cli_color_integration.rs +++ b/crates/openshell-cli/tests/cli_color_integration.rs @@ -176,15 +176,10 @@ fn tracing_output_is_free_of_escape_sequences_when_piped() { ); } -/// Run a failing command with stdout attached to a pseudo-terminal and stderr -/// on a pipe, returning what each stream received. -/// -/// `Command::output` gives both streams pipes, so it cannot distinguish a -/// per-stream decision from a single one resolved off stdout. This asymmetric -/// setup is the only way to catch a stream being handed the other stream's -/// answer. +/// Run a command with stdout attached to a pseudo-terminal and stderr on a +/// pipe, returning what each stream received. #[cfg(target_os = "linux")] -fn split_streams_stdout_tty(args: &[&str]) -> (String, String) { +fn run_with_stdout_tty(mut command: Command) -> (String, String) { use std::io::Read; use std::os::fd::{AsRawFd, OwnedFd}; @@ -192,26 +187,14 @@ fn split_streams_stdout_tty(args: &[&str]) -> (String, String) { let controller: OwnedFd = pty.master; let follower: OwnedFd = pty.slave; - let tmpdir = tempfile::tempdir().expect("create tmpdir"); - let mut child = Command::new(env!("CARGO_BIN_EXE_openshell")) - .args([ - "sandbox", - "list", - "--gateway", - "test-gateway", - "--gateway-endpoint", - "http://127.0.0.1:1", - ]) - .args(args) - .env("XDG_CONFIG_HOME", tmpdir.path()) - .env("RUST_LOG", "debug") - .env_remove("NO_COLOR") - .env_remove("FORCE_COLOR") - .env_remove("OPENSHELL_COLOR") + let mut child = command .stdout(follower.try_clone().expect("dup pty follower")) .stderr(std::process::Stdio::piped()) .spawn() .expect("spawn openshell"); + // `Command` retains its configured stdio handles after spawning. Drop it so + // the controller sees EIO once the child exits. + drop(command); // Drop every follower handle in this process, or reading the controller // blocks forever instead of returning EIO once the child exits. @@ -242,6 +225,135 @@ fn split_streams_stdout_tty(args: &[&str]) -> (String, String) { ) } +/// Run a failing command with stdout attached to a pseudo-terminal and stderr +/// on a pipe, returning what each stream received. +/// +/// `Command::output` gives both streams pipes, so it cannot distinguish a +/// per-stream decision from a single one resolved off stdout. This asymmetric +/// setup is the only way to catch a stream being handed the other stream's +/// answer. +#[cfg(target_os = "linux")] +fn split_streams_stdout_tty(args: &[&str]) -> (String, String) { + let tmpdir = tempfile::tempdir().expect("create tmpdir"); + let mut command = Command::new(env!("CARGO_BIN_EXE_openshell")); + command + .args([ + "sandbox", + "list", + "--gateway", + "test-gateway", + "--gateway-endpoint", + "http://127.0.0.1:1", + ]) + .args(args) + .env("XDG_CONFIG_HOME", tmpdir.path()) + .env("RUST_LOG", "debug") + // Pin TERM: `auto` now requires a capable terminal, and CI runners + // often leave TERM unset, which would make this test's outcome depend + // on the ambient environment. + .env("TERM", "xterm-256color") + .env_remove("NO_COLOR") + .env_remove("FORCE_COLOR") + .env_remove("OPENSHELL_COLOR"); + + run_with_stdout_tty(command) +} + +/// Run `forward list` with stdout on a pseudo-terminal, under the given `TERM`, +/// and return everything stdout received. +/// +/// When `stderr_on_tty` is true, both streams share the terminal so the +/// `owo-colors` table is styled too. Otherwise, stderr is redirected to +/// `/dev/null`, which verifies the conservative table behavior. +#[cfg(target_os = "linux")] +fn forward_list_on_pty(term: &str, args: &[&str], stderr_on_tty: bool) -> String { + use std::os::fd::{AsRawFd, OwnedFd}; + + let pty = nix::pty::openpty(None, None).expect("openpty"); + let controller: OwnedFd = pty.master; + let follower: OwnedFd = pty.slave; + + let tmpdir = tempfile::tempdir().expect("create tmpdir"); + config_dir_with_forward(tmpdir.path()); + + let mut command = Command::new(env!("CARGO_BIN_EXE_openshell")); + command + .args(["forward", "list"]) + .args(args) + .env("XDG_CONFIG_HOME", tmpdir.path()) + .env("TERM", term) + .env_remove("NO_COLOR") + .env_remove("FORCE_COLOR") + .env_remove("OPENSHELL_COLOR") + .stdout(follower.try_clone().expect("dup pty follower")); + if stderr_on_tty { + command.stderr(follower.try_clone().expect("dup pty follower")); + } else { + command.stderr(std::process::Stdio::null()); + } + let mut child = command.spawn().expect("spawn openshell"); + // `Command` retains its configured stdio handles after spawning. Drop it so + // the controller sees EIO once the child exits. + drop(command); + + // Drop every follower handle here, or the controller read never sees EIO. + drop(follower); + + let mut buf = Vec::new(); + let mut chunk = [0u8; 4096]; + loop { + match nix::unistd::read(controller.as_raw_fd(), &mut chunk) { + Ok(0) | Err(_) => break, + Ok(n) => buf.extend_from_slice(&chunk[..n]), + } + } + child.wait().expect("wait for openshell"); + + let out = String::from_utf8_lossy(&buf).into_owned(); + assert!( + out.contains(SANDBOX), + "expected the seeded forward in the table, got: {out:?}" + ); + out +} + +/// A terminal that does not render ANSI must not be styled under `auto`. +/// +/// `TERM=dumb` is still a terminal, so an `is_terminal()` check alone reports it +/// as styleable. `console` and `miette` apply their own `TERM` checks, but the +/// color switch overrides both, so the check has to live here. +#[cfg(target_os = "linux")] +#[test] +fn dumb_terminal_is_not_styled_under_auto() { + let dumb = forward_list_on_pty("dumb", &[], true); + // Positive control: the same session on a capable terminal is styled, so a + // plain result below means capability was consulted, not that the pty setup + // silently produced nothing. + let capable = forward_list_on_pty("xterm-256color", &[], true); + + assert!( + capable.contains(ESC), + "expected styling on a capable terminal; got: {capable:?}" + ); + assert!( + !dumb.contains(ESC), + "TERM=dumb must not be styled, got: {dumb:?}" + ); +} + +/// An explicit request outranks the capability check, for callers who know +/// their terminal better than `TERM` does. +#[cfg(target_os = "linux")] +#[test] +fn color_always_overrides_a_dumb_terminal() { + let forced = forward_list_on_pty("dumb", &["--color", "always"], true); + + assert!( + forced.contains(ESC), + "--color always must style even a dumb terminal, got: {forced:?}" + ); +} + /// Regression test for a redirected stream inheriting the other stream's /// terminal check. /// @@ -264,6 +376,23 @@ fn redirected_stderr_stays_plain_while_stdout_is_a_terminal() { ); } +/// `Painted` cannot identify its destination stream, so table styling is +/// deliberately disabled when either stream is redirected. +#[cfg(target_os = "linux")] +#[test] +fn status_table_is_plain_when_stderr_is_redirected() { + let stdout = forward_list_on_pty("xterm-256color", &[], false); + + assert!( + stdout.contains(SANDBOX), + "expected the seeded forward in the table, got: {stdout:?}" + ); + assert!( + !stdout.contains(ESC), + "STATUS table must stay plain when stderr is redirected, got: {stdout:?}" + ); +} + #[test] fn error_output_follows_the_color_setting() { // miette renders errors to stderr through its own handler. It already diff --git a/crates/openshell-cli/tests/ensure_providers_integration.rs b/crates/openshell-cli/tests/ensure_providers_integration.rs index 743cfec42c..2a4801b143 100644 --- a/crates/openshell-cli/tests/ensure_providers_integration.rs +++ b/crates/openshell-cli/tests/ensure_providers_integration.rs @@ -154,6 +154,8 @@ impl OpenShell for TestOpenShell { Ok(Response::new(ListSandboxesResponse::default())) } + unimplemented_sandbox_template_rpcs!(); + async fn list_sandbox_providers( &self, _request: tonic::Request, diff --git a/crates/openshell-cli/tests/helpers/mod.rs b/crates/openshell-cli/tests/helpers/mod.rs index a58e750b91..c4e9b4b75a 100644 --- a/crates/openshell-cli/tests/helpers/mod.rs +++ b/crates/openshell-cli/tests/helpers/mod.rs @@ -8,6 +8,95 @@ //! mod helpers; //! ``` +#[macro_export] +macro_rules! unimplemented_sandbox_template_rpcs { + () => { + fn create_sandbox_template<'life0, 'async_trait>( + &'life0 self, + _request: tonic::Request, + ) -> std::pin::Pin< + Box< + dyn std::future::Future< + Output = Result< + tonic::Response, + tonic::Status, + >, + > + Send + + 'async_trait, + >, + > + where + 'life0: 'async_trait, + Self: 'async_trait, + { + Box::pin(async { Err(tonic::Status::unimplemented("unused")) }) + } + + fn get_sandbox_template<'life0, 'async_trait>( + &'life0 self, + _request: tonic::Request, + ) -> std::pin::Pin< + Box< + dyn std::future::Future< + Output = Result< + tonic::Response, + tonic::Status, + >, + > + Send + + 'async_trait, + >, + > + where + 'life0: 'async_trait, + Self: 'async_trait, + { + Box::pin(async { Err(tonic::Status::unimplemented("unused")) }) + } + + fn list_sandbox_templates<'life0, 'async_trait>( + &'life0 self, + _request: tonic::Request, + ) -> std::pin::Pin< + Box< + dyn std::future::Future< + Output = Result< + tonic::Response, + tonic::Status, + >, + > + Send + + 'async_trait, + >, + > + where + 'life0: 'async_trait, + Self: 'async_trait, + { + Box::pin(async { Err(tonic::Status::unimplemented("unused")) }) + } + + fn delete_sandbox_template<'life0, 'async_trait>( + &'life0 self, + _request: tonic::Request, + ) -> std::pin::Pin< + Box< + dyn std::future::Future< + Output = Result< + tonic::Response, + tonic::Status, + >, + > + Send + + 'async_trait, + >, + > + where + 'life0: 'async_trait, + Self: 'async_trait, + { + Box::pin(async { Err(tonic::Status::unimplemented("unused")) }) + } + }; +} + use rcgen::{ BasicConstraints, Certificate, CertificateParams, ExtendedKeyUsagePurpose, IsCa, KeyPair, }; diff --git a/crates/openshell-cli/tests/mtls_integration.rs b/crates/openshell-cli/tests/mtls_integration.rs index e4735ac4c1..12c838baf1 100644 --- a/crates/openshell-cli/tests/mtls_integration.rs +++ b/crates/openshell-cli/tests/mtls_integration.rs @@ -113,6 +113,8 @@ impl OpenShell for TestOpenShell { )) } + unimplemented_sandbox_template_rpcs!(); + async fn list_sandbox_providers( &self, _request: tonic::Request, diff --git a/crates/openshell-cli/tests/provider_commands_integration.rs b/crates/openshell-cli/tests/provider_commands_integration.rs index 46e3b903c0..e48ca84af0 100644 --- a/crates/openshell-cli/tests/provider_commands_integration.rs +++ b/crates/openshell-cli/tests/provider_commands_integration.rs @@ -185,6 +185,7 @@ impl OpenShell for TestOpenShell { }), spec: None, status: None, + ..Sandbox::default() }), })) } @@ -196,6 +197,8 @@ impl OpenShell for TestOpenShell { Ok(Response::new(ListSandboxesResponse::default())) } + unimplemented_sandbox_template_rpcs!(); + async fn list_sandbox_providers( &self, request: tonic::Request, @@ -1218,6 +1221,59 @@ async fn install_test_profile(ts: &TestServer, id: &str, credential_key: &str) { ); } +/// A readable provider must carry its stored type and profile workspace into +/// the update request. Policy interceptors evaluate the request before the +/// gateway merges it with stored state, so an update that omits them cannot be +/// authorized against the profile that owns the provider. +/// +/// The stored `profile_workspace` is forwarded verbatim rather than recomputed +/// from the request workspace. The gateway treats it as immutable, so deriving +/// it here would look like a change and be rejected. +#[tokio::test] +async fn provider_update_preserves_stored_type_and_profile_workspace_when_readable() { + let ts = run_server().await; + + run::provider_create( + &ts.endpoint, + "my-claude", + "claude", + false, + &["API_KEY=abc".to_string()], + false, + &[], + "default", + &ts.tls, + ) + .await + .expect("provider create"); + + run::provider_update(run::ProviderUpdateOptions { + server: &ts.endpoint, + name: "my-claude", + from_existing: false, + from_oidc_token: false, + credentials: &["API_KEY=rotated".to_string()], + config: &[], + credential_expires_at: &[], + workspace: "default", + tls: &ts.tls, + }) + .await + .expect("provider update"); + + let requests = ts.state.provider_update_requests.lock().await; + let request = requests.last().expect("provider update request"); + // `claude` normalizes to the canonical `claude-code` at creation, so the + // update carries the stored type rather than the alias the caller typed. + assert_eq!(request.r#type, "claude-code"); + // Forwarded verbatim rather than recomputed. The gateway treats + // profile_workspace as immutable, so any substitution here would look like + // a change and be rejected. + let stored = ts.state.providers.lock().await; + let stored = stored.get("my-claude").expect("stored provider"); + assert_eq!(request.profile_workspace, stored.profile_workspace); +} + #[tokio::test] async fn provider_delete_continues_after_entry_failure() { let ts = run_server().await; diff --git a/crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs b/crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs index 51e2159a5c..7be771c442 100644 --- a/crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs +++ b/crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs @@ -14,20 +14,22 @@ use openshell_cli::tls::TlsOptions; use openshell_core::proto::open_shell_server::{OpenShell, OpenShellServer}; use openshell_core::proto::{ AttachSandboxProviderRequest, AttachSandboxProviderResponse, CreateProviderRequest, - CreateSandboxRequest, CreateSshSessionRequest, CreateSshSessionResponse, DeleteProviderRequest, - DeleteProviderResponse, DeleteSandboxRequest, DeleteSandboxResponse, - DetachSandboxProviderRequest, DetachSandboxProviderResponse, - ExchangeProviderSubjectTokenRequest, ExchangeProviderSubjectTokenResponse, ExecSandboxEvent, - ExecSandboxInput, ExecSandboxRequest, GatewayMessage, GetGatewayConfigRequest, - GetGatewayConfigResponse, GetProviderRequest, GetSandboxConfigRequest, - GetSandboxConfigResponse, GetSandboxProviderEnvironmentRequest, - GetSandboxProviderEnvironmentResponse, GetSandboxRequest, GpuResourceRequirements, - HealthRequest, HealthResponse, ListProvidersRequest, ListProvidersResponse, - ListSandboxProvidersRequest, ListSandboxProvidersResponse, ListSandboxesRequest, - ListSandboxesResponse, PlatformEvent, ProviderResponse, RevokeSshSessionRequest, + CreateSandboxRequest, CreateSandboxTemplateRequest, CreateSshSessionRequest, + CreateSshSessionResponse, DeleteProviderRequest, DeleteProviderResponse, DeleteSandboxRequest, + DeleteSandboxResponse, DeleteSandboxTemplateRequest, DetachSandboxProviderRequest, + DetachSandboxProviderResponse, ExchangeProviderSubjectTokenRequest, + ExchangeProviderSubjectTokenResponse, ExecSandboxEvent, ExecSandboxInput, ExecSandboxRequest, + GatewayMessage, GetGatewayConfigRequest, GetGatewayConfigResponse, GetProviderRequest, + GetSandboxConfigRequest, GetSandboxConfigResponse, GetSandboxProviderEnvironmentRequest, + GetSandboxProviderEnvironmentResponse, GetSandboxRequest, GetSandboxTemplateRequest, + GpuResourceRequirements, HealthRequest, HealthResponse, ListProvidersRequest, + ListProvidersResponse, ListSandboxProvidersRequest, ListSandboxProvidersResponse, + ListSandboxTemplatesRequest, ListSandboxTemplatesResponse, ListSandboxesRequest, + ListSandboxesResponse, PlatformEvent, Provider, ProviderResponse, RevokeSshSessionRequest, RevokeSshSessionResponse, Sandbox, SandboxCondition, SandboxLogLine, SandboxPhase, - SandboxResponse, SandboxStatus, SandboxStreamEvent, ServiceStatus, SettingValue, - SupervisorMessage, UpdateProviderRequest, WatchSandboxRequest, sandbox_stream_event, + SandboxResponse, SandboxStatus, SandboxStreamEvent, SandboxTemplateResponse, + SandboxWorkloadTemplate, ServiceStatus, SettingValue, SupervisorMessage, UpdateProviderRequest, + WatchSandboxRequest, sandbox_stream_event, }; use std::collections::HashMap; use std::fs; @@ -56,6 +58,11 @@ struct SandboxState { ssh_session_requests: Arc, global_settings: Arc>>, gateway_config_requests: Arc, + providers: Arc>>, + template_create_requests: Arc>>, + template_get_requests: Arc>>, + template_list_requests: Arc>>, + template_delete_requests: Arc>>, } #[derive(Clone, Default)] @@ -180,6 +187,95 @@ impl OpenShell for TestOpenShell { Ok(Response::new(ListSandboxesResponse::default())) } + async fn create_sandbox_template( + &self, + request: tonic::Request, + ) -> Result, Status> { + let request = request.into_inner(); + let mut template = request.template.clone().unwrap_or_default(); + let name = template + .metadata + .as_ref() + .map_or_else(|| "template".to_string(), |metadata| metadata.name.clone()); + template.metadata = Some(openshell_core::proto::datamodel::v1::ObjectMeta { + id: format!("template-{name}"), + name, + created_at_ms: 0, + labels: template + .metadata + .as_ref() + .map(|metadata| metadata.labels.clone()) + .unwrap_or_default(), + resource_version: 1, + annotations: HashMap::new(), + workspace: request.workspace.clone(), + deletion_timestamp_ms: 0, + }); + self.state + .template_create_requests + .lock() + .await + .push(request); + Ok(Response::new(SandboxTemplateResponse { + template: Some(template), + })) + } + + async fn get_sandbox_template( + &self, + request: tonic::Request, + ) -> Result, Status> { + let request = request.into_inner(); + self.state + .template_get_requests + .lock() + .await + .push(request.clone()); + Ok(Response::new(SandboxTemplateResponse { + template: Some(SandboxWorkloadTemplate { + metadata: Some(openshell_core::proto::datamodel::v1::ObjectMeta { + id: format!("template-{}", request.name), + name: request.name, + created_at_ms: 0, + labels: HashMap::new(), + resource_version: 1, + annotations: HashMap::new(), + workspace: request.workspace, + deletion_timestamp_ms: 0, + }), + spec: None, + }), + })) + } + + async fn list_sandbox_templates( + &self, + request: tonic::Request, + ) -> Result, Status> { + self.state + .template_list_requests + .lock() + .await + .push(request.into_inner()); + Ok(Response::new(ListSandboxTemplatesResponse { + templates: Vec::new(), + })) + } + + async fn delete_sandbox_template( + &self, + request: tonic::Request, + ) -> Result, Status> { + self.state + .template_delete_requests + .lock() + .await + .push(request.into_inner()); + Ok(Response::new( + openshell_core::proto::DeleteSandboxTemplateResponse { deleted: true }, + )) + } + async fn list_sandbox_providers( &self, _request: tonic::Request, @@ -337,7 +433,9 @@ impl OpenShell for TestOpenShell { &self, _request: tonic::Request, ) -> Result, Status> { - Ok(Response::new(ListProvidersResponse::default())) + Ok(Response::new(ListProvidersResponse { + providers: self.state.providers.lock().await.clone(), + })) } async fn list_provider_profiles( @@ -1209,6 +1307,63 @@ async fn create_requests(server: &TestServer) -> Vec { server.openshell.state.create_requests.lock().await.clone() } +async fn template_create_requests(server: &TestServer) -> Vec { + server + .openshell + .state + .template_create_requests + .lock() + .await + .clone() +} + +async fn template_list_requests(server: &TestServer) -> Vec { + server + .openshell + .state + .template_list_requests + .lock() + .await + .clone() +} + +async fn template_delete_requests(server: &TestServer) -> Vec { + server + .openshell + .state + .template_delete_requests + .lock() + .await + .clone() +} + +async fn add_provider(server: &TestServer, name: &str, provider_type: &str) { + server + .openshell + .state + .providers + .lock() + .await + .push(Provider { + metadata: Some(openshell_core::proto::datamodel::v1::ObjectMeta { + id: format!("provider-{name}"), + name: name.to_string(), + created_at_ms: 0, + labels: HashMap::new(), + resource_version: 0, + annotations: HashMap::new(), + workspace: "default".to_string(), + deletion_timestamp_ms: 0, + }), + r#type: provider_type.to_string(), + credentials: HashMap::new(), + config: HashMap::new(), + credential_expires_at_ms: HashMap::new(), + profile_workspace: "default".to_string(), + credential_handles: HashMap::new(), + }); +} + fn test_tls(server: &TestServer) -> TlsOptions { server.tls.with_gateway_name("openshell") } @@ -1529,6 +1684,201 @@ async fn sandbox_create_sends_driver_config_json() { ); } +#[tokio::test] +async fn sandbox_create_with_template_sends_workload_template_name() { + let server = run_server().await; + add_provider(&server, "github", "github").await; + let fake_ssh_dir = tempfile::tempdir().unwrap(); + let xdg_dir = tempfile::tempdir().unwrap(); + let _env = test_env(&fake_ssh_dir, &xdg_dir); + let tls = test_tls(&server); + install_fake_ssh(&fake_ssh_dir); + + run::sandbox_create( + &server.endpoint, + "openshell", + run::SandboxCreateConfig { + name: Some("from-template"), + template: Some("gpu-kata"), + providers: &["github".to_string()], + command: &["echo".into(), "OK".into()], + ..test_config() + }, + "default", + &tls, + ) + .await + .expect("sandbox create should succeed"); + + let requests = create_requests(&server).await; + let request = requests.first().expect("create request should be recorded"); + assert_eq!(request.workload_template_name, "gpu-kata"); + let spec = request + .spec + .as_ref() + .expect("governance spec should be sent"); + assert_eq!(spec.providers, vec!["github".to_string()]); + assert!(spec.template.is_none()); + assert!(spec.environment.is_empty()); + assert!(spec.resource_requirements.is_none()); +} + +#[tokio::test] +async fn sandbox_template_create_sends_workload_template_resource() { + let server = run_server().await; + let fake_ssh_dir = tempfile::tempdir().unwrap(); + let xdg_dir = tempfile::tempdir().unwrap(); + let _env = test_env(&fake_ssh_dir, &xdg_dir); + let tls = test_tls(&server); + + run::sandbox_template_create( + &server.endpoint, + "gpu-kata", + Some("registry.example.com/agent:latest"), + Some("2"), + Some("4Gi"), + Some(GpuResourceRequirements { count: Some(1) }), + Some(r#"{"kubernetes":{"pod":{"node_selector":{"pool":"gpu"}}}}"#), + Some("5m"), + Some(3), + HashMap::from([("team".to_string(), "runtime".to_string())]), + HashMap::from([("owner".to_string(), "platform".to_string())]), + HashMap::from([("FEATURE_FLAG".to_string(), "on".to_string())]), + "table", + "default", + &tls, + ) + .await + .expect("template create should succeed"); + + let requests = template_create_requests(&server).await; + let request = requests + .first() + .expect("template create request should be recorded"); + assert_eq!(request.workspace, "default"); + let template = request.template.as_ref().expect("template should be sent"); + let metadata = template.metadata.as_ref().expect("metadata should be sent"); + assert_eq!(metadata.name, "gpu-kata"); + assert_eq!(metadata.labels.get("team"), Some(&"runtime".to_string())); + assert_eq!( + metadata.annotations.get("owner"), + Some(&"platform".to_string()) + ); + + let spec = template.spec.as_ref().expect("spec should be sent"); + let workload = spec.workload.as_ref().expect("workload should be sent"); + assert_eq!(workload.image, "registry.example.com/agent:latest"); + assert_eq!( + workload.environment.get("FEATURE_FLAG"), + Some(&"on".to_string()) + ); + let resources = workload + .resources + .as_ref() + .expect("resources should be sent"); + assert_eq!(resources.cpu, "2"); + assert_eq!(resources.memory, "4Gi"); + assert_eq!(resources.gpu.as_ref().and_then(|gpu| gpu.count), Some(1)); + assert!(spec.driver_config.is_some()); + let startup = spec + .desired_service_level + .as_ref() + .and_then(|service_level| service_level.startup.as_ref()) + .expect("startup service level should be sent"); + assert_eq!(startup.max_burst, 3); + assert_eq!( + startup + .ready_within + .as_ref() + .map(|duration| duration.seconds), + Some(300) + ); +} + +#[tokio::test] +async fn sandbox_template_list_and_delete_send_workspace_requests() { + let server = run_server().await; + let fake_ssh_dir = tempfile::tempdir().unwrap(); + let xdg_dir = tempfile::tempdir().unwrap(); + let _env = test_env(&fake_ssh_dir, &xdg_dir); + let tls = test_tls(&server); + + run::sandbox_template_list( + &server.endpoint, + 25, + 5, + Some("team=runtime"), + false, + "table", + "default", + false, + &tls, + ) + .await + .expect("template list should succeed"); + run::sandbox_template_delete(&server.endpoint, &["gpu-kata".to_string()], "default", &tls) + .await + .expect("template delete should succeed"); + + let list_requests = template_list_requests(&server).await; + let list_request = list_requests + .first() + .expect("template list request should be recorded"); + assert_eq!(list_request.limit, 25); + assert_eq!(list_request.offset, 5); + assert_eq!(list_request.label_selector, "team=runtime"); + assert_eq!(list_request.workspace, "default"); + assert!(!list_request.all_workspaces); + + let delete_requests = template_delete_requests(&server).await; + let delete_request = delete_requests + .first() + .expect("template delete request should be recorded"); + assert_eq!(delete_request.name, "gpu-kata"); + assert_eq!(delete_request.workspace, "default"); +} + +#[tokio::test] +async fn sandbox_template_create_allows_omitted_image() { + let server = run_server().await; + let fake_ssh_dir = tempfile::tempdir().unwrap(); + let xdg_dir = tempfile::tempdir().unwrap(); + let _env = test_env(&fake_ssh_dir, &xdg_dir); + let tls = test_tls(&server); + + run::sandbox_template_create( + &server.endpoint, + "base", + None, + None, + None, + None, + None, + None, + None, + HashMap::new(), + HashMap::new(), + HashMap::new(), + "table", + "default", + &tls, + ) + .await + .expect("template create without image should succeed"); + + let requests = template_create_requests(&server).await; + let request = requests + .first() + .expect("template create request should be recorded"); + let workload = request + .template + .as_ref() + .and_then(|template| template.spec.as_ref()) + .and_then(|spec| spec.workload.as_ref()) + .expect("workload should be sent"); + assert_eq!(workload.image, ""); +} + #[tokio::test] async fn sandbox_create_sends_gpu_default_request() { let server = run_server().await; @@ -2258,6 +2608,44 @@ async fn run_cli_sandbox_create( .unwrap() } +async fn run_cli_sandbox_template_create( + server: &TestServer, + name: &str, + extra_args: &[&str], +) -> std::process::Output { + let xdg_dir = tempfile::tempdir().unwrap(); + let tls_dir = xdg_dir.path().join("openshell/gateways/openshell/mtls"); + fs::create_dir_all(&tls_dir).unwrap(); + for filename in ["ca.crt", "tls.crt", "tls.key"] { + fs::copy(server.dir.path().join(filename), tls_dir.join(filename)).unwrap(); + } + + let mut cmd = tokio::process::Command::new(env!("CARGO_BIN_EXE_openshell")); + for (key, _) in std::env::vars().filter(|(k, _)| k.starts_with("OPENSHELL_")) { + cmd.env_remove(&key); + } + cmd.args([ + "--gateway", + "openshell", + "--gateway-endpoint", + &server.endpoint, + "sandbox", + "template", + "create", + name, + "--image", + "registry.example.com/agent:latest", + "--output=json", + ]) + .args(extra_args) + .env("XDG_CONFIG_HOME", xdg_dir.path()) + .env("HOME", xdg_dir.path()) + .env("OPENSHELL_PROVISION_TIMEOUT", "5") + .output() + .await + .unwrap() +} + #[tokio::test] async fn sandbox_create_json_stdout_is_parseable() { let server = run_server().await; @@ -2287,3 +2675,63 @@ async fn sandbox_create_yaml_stdout_is_parseable() { serde_yml::from_str::(&stdout) .unwrap_or_else(|err| panic!("stdout should contain only YAML: {err}\n{stdout}")); } + +#[tokio::test] +async fn sandbox_template_create_warns_for_credential_env_vars() { + let server = run_server().await; + + let result = run_cli_sandbox_template_create( + &server, + "credential-env", + &["--env", "OPENAI_API_KEY=plain-secret"], + ) + .await; + + assert!( + result.status.success(), + "sandbox template create failed:\n{}", + String::from_utf8_lossy(&result.stderr) + ); + let stderr = String::from_utf8_lossy(&result.stderr); + assert!( + stderr.contains("OPENAI_API_KEY looks like a credential"), + "template create should warn for credential-looking --env values: {stderr}" + ); + assert!( + stderr.contains("To hide it from the agent, use a provider instead"), + "warning should point users toward providers: {stderr}" + ); + + let requests = template_create_requests(&server).await; + assert_eq!(requests.len(), 1); +} + +#[tokio::test] +async fn sandbox_template_create_suppresses_credential_env_warnings() { + let server = run_server().await; + + let result = run_cli_sandbox_template_create( + &server, + "credential-env-suppressed", + &[ + "--env", + "OPENAI_API_KEY=plain-secret", + "--no-credential-warnings", + ], + ) + .await; + + assert!( + result.status.success(), + "sandbox template create failed:\n{}", + String::from_utf8_lossy(&result.stderr) + ); + let stderr = String::from_utf8_lossy(&result.stderr); + assert!( + !stderr.contains("OPENAI_API_KEY looks like a credential"), + "template create should suppress credential-looking --env warnings: {stderr}" + ); + + let requests = template_create_requests(&server).await; + assert_eq!(requests.len(), 1); +} diff --git a/crates/openshell-cli/tests/sandbox_name_fallback_integration.rs b/crates/openshell-cli/tests/sandbox_name_fallback_integration.rs index ee72728aa9..5b62c7c15c 100644 --- a/crates/openshell-cli/tests/sandbox_name_fallback_integration.rs +++ b/crates/openshell-cli/tests/sandbox_name_fallback_integration.rs @@ -138,6 +138,8 @@ impl OpenShell for TestOpenShell { Ok(Response::new(ListSandboxesResponse::default())) } + unimplemented_sandbox_template_rpcs!(); + async fn list_sandbox_providers( &self, _request: tonic::Request, diff --git a/crates/openshell-core/Cargo.toml b/crates/openshell-core/Cargo.toml index d8483bd2cb..c96d536f07 100644 --- a/crates/openshell-core/Cargo.toml +++ b/crates/openshell-core/Cargo.toml @@ -27,6 +27,8 @@ serde_json = { workspace = true } tracing = { workspace = true } url = { workspace = true } ipnet = "2" +rustls = { workspace = true } +rustls-pemfile = { workspace = true } base64 = { workspace = true } chrono = { version = "0.4", default-features = false, features = ["clock", "std"], optional = true } reqwest = { workspace = true, features = ["blocking", "rustls-tls-native-roots"], optional = true } @@ -53,6 +55,7 @@ protoc-bin-vendored = { workspace = true } [dev-dependencies] tempfile = "3" +rcgen = { workspace = true } [lints] workspace = true diff --git a/crates/openshell-core/build.rs b/crates/openshell-core/build.rs index 9caaf8eb17..38c961b1d4 100644 --- a/crates/openshell-core/build.rs +++ b/crates/openshell-core/build.rs @@ -4,16 +4,20 @@ use std::env; use std::path::{Path, PathBuf}; +mod build_version; + const PROTO_REL: &str = "../../proto"; fn main() -> Result<(), Box> { // --- Git-derived version --- - // Compute a version from `git describe` for local builds. In Docker/CI - // builds where .git is absent, this silently does nothing and the binary - // falls back to CARGO_PKG_VERSION (which is already sed-patched by the - // build pipeline). + // Compute a version from tags and commit metadata for local builds. In + // Docker/CI builds where .git is absent, this silently does nothing and + // the binary falls back to CARGO_PKG_VERSION (which is already sed-patched + // by the build pipeline). println!("cargo:rerun-if-changed=../../.git/HEAD"); + println!("cargo:rerun-if-changed=../../.git/logs/HEAD"); println!("cargo:rerun-if-changed=../../.git/refs/tags"); + println!("cargo:rerun-if-changed=../../.git/packed-refs"); if let Some(version) = git_version() { println!("cargo:rustc-env=OPENSHELL_GIT_VERSION={version}"); @@ -72,53 +76,43 @@ fn collect_proto_files(dir: &Path, out: &mut Vec) -> std::io::Result<() Ok(()) } -/// Derive a version string from `git describe --tags`. +/// Derive the release or development version from git metadata. /// /// Implements the "guess-next-dev" convention used by the release pipeline -/// (`setuptools-scm`): when there are commits past the last tag, the patch -/// version is bumped and `-dev.+g` is appended. +/// (`tasks/scripts/release.py`): exact stable and prerelease tags retain their +/// version. Otherwise, the latest merged stable release gets a patch bump and +/// `-dev.+g` is appended. /// /// Examples: -/// on tag v0.0.3 → "0.0.3" -/// 3 commits past v0.0.3 → "0.0.4-dev.3+g2bf9969" +/// on tag v0.1.0-pre.1 → "0.1.0-pre.1" +/// 3 commits past v0.0.3 → "0.0.4-dev.3+g2bf9969ab" /// -/// Returns `None` when git is unavailable or the repo has no matching tags. +/// Returns `None` when git metadata cannot be read. fn git_version() -> Option { - // Match numeric release tags only (e.g. `v0.0.29`). The bare glob `v*` - // also matches non-release tags like `vm-dev` or `vm-prod`; when one of - // those lands on the same commit as a release tag, `git describe` picks - // it and the resulting version string collapses to `m-dev` after the - // leading `v` is stripped below. Requiring a digit after `v` excludes - // those development tags without losing any release tag. - let output = std::process::Command::new("git") - .args(["describe", "--tags", "--long", "--match", "v[0-9]*"]) - .output() - .ok()?; - - if !output.status.success() { - return None; + let exact_tags = git_output(&["tag", "--points-at", "HEAD"])?; + if let Some(version) = build_version::exact_release_version(exact_tags.lines()) { + return Some(version); } - let desc = String::from_utf8(output.stdout).ok()?; - let desc = desc.trim(); - let desc = desc.strip_prefix('v').unwrap_or(desc); + let merged_tags = git_output(&["tag", "--merged", "HEAD", "--list", "v*.*.*"])?; + let latest_tag = build_version::latest_stable_tag(merged_tags.lines()); + let revision_range = latest_tag + .as_deref() + .map_or_else(|| "HEAD".to_string(), |tag| format!("{tag}..HEAD")); + let distance = git_output(&["rev-list", "--count", &revision_range])? + .parse() + .ok()?; + let sha = git_output(&["rev-parse", "--short=9", "HEAD"])?; - // `git describe --long` format: --g - // Split from the right to handle tags that contain hyphens. - let (rest, sha) = desc.rsplit_once('-')?; - let (tag, commits_str) = rest.rsplit_once('-')?; - let commits: u32 = commits_str.parse().ok()?; + build_version::next_dev_version(latest_tag.as_deref(), distance, &sha) +} - if commits == 0 { - // Exactly on a tag — use the tag version as-is. - return Some(tag.to_string()); +fn git_output(args: &[&str]) -> Option { + let output = std::process::Command::new("git").args(args).output().ok()?; + if !output.status.success() { + return None; } - - // Bump patch version (guess-next-dev scheme). - let mut parts = tag.splitn(3, '.'); - let major = parts.next()?; - let minor = parts.next()?; - let patch: u32 = parts.next()?.parse().ok()?; - - Some(format!("{major}.{minor}.{}-dev.{commits}+{sha}", patch + 1)) + String::from_utf8(output.stdout) + .ok() + .map(|output| output.trim().to_string()) } diff --git a/crates/openshell-core/build_version.rs b/crates/openshell-core/build_version.rs new file mode 100644 index 0000000000..944808be93 --- /dev/null +++ b/crates/openshell-core/build_version.rs @@ -0,0 +1,103 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +type StableVersion = (u32, u32, u32); +type PrereleaseVersion = (u32, u32, u32, u32); + +fn parse_stable_tag(tag: &str) -> Option { + let tag = tag.strip_prefix('v').unwrap_or(tag); + let mut parts = tag.split('.'); + let version = ( + parts.next()?.parse().ok()?, + parts.next()?.parse().ok()?, + parts.next()?.parse().ok()?, + ); + parts.next().is_none().then_some(version) +} + +fn parse_prerelease_tag(tag: &str) -> Option { + let tag = tag.strip_prefix('v').unwrap_or(tag); + let (base, sequence) = tag.rsplit_once("-pre.")?; + let (major, minor, patch) = parse_stable_tag(base)?; + let sequence = sequence.parse().ok()?; + (sequence > 0).then_some((major, minor, patch, sequence)) +} + +pub fn exact_release_version<'a>(tags: impl Iterator) -> Option { + let tags = tags.collect::>(); + + if let Some(((major, minor, patch), _)) = tags + .iter() + .filter_map(|tag| parse_stable_tag(tag).map(|version| (version, tag))) + .max_by_key(|(version, _)| *version) + { + return Some(format!("{major}.{minor}.{patch}")); + } + + tags.iter() + .filter_map(|tag| parse_prerelease_tag(tag)) + .max() + .map(|(major, minor, patch, sequence)| format!("{major}.{minor}.{patch}-pre.{sequence}")) +} + +pub fn latest_stable_tag<'a>(tags: impl Iterator) -> Option { + tags.filter_map(|tag| parse_stable_tag(tag).map(|version| (version, tag))) + .max_by_key(|(version, _)| *version) + .map(|(_, tag)| tag.to_string()) +} + +pub fn next_dev_version(tag: Option<&str>, distance: u32, sha: &str) -> Option { + let (major, minor, patch) = tag.map_or(Some((0, 0, 0)), parse_stable_tag)?; + Some(format!( + "{major}.{minor}.{}-dev.{distance}+g{sha}", + patch + 1 + )) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn exact_stable_release_wins_over_prerelease() { + let tags = ["v0.1.0-pre.2", "v0.1.0", "vm-dev"]; + assert_eq!( + exact_release_version(tags.into_iter()).as_deref(), + Some("0.1.0") + ); + } + + #[test] + fn exact_prerelease_uses_highest_sequence() { + let tags = ["v0.1.0-pre.1", "v0.1.0-pre.2"]; + assert_eq!( + exact_release_version(tags.into_iter()).as_deref(), + Some("0.1.0-pre.2") + ); + } + + #[test] + fn latest_stable_ignores_prerelease_and_non_release_tags() { + let tags = ["v0.0.116", "v0.1.0-pre.1", "vm-dev", "v0.0.99"]; + assert_eq!( + latest_stable_tag(tags.into_iter()).as_deref(), + Some("v0.0.116") + ); + } + + #[test] + fn next_dev_version_bumps_latest_stable_patch() { + assert_eq!( + next_dev_version(Some("v0.0.116"), 32, "5b925dd8a").as_deref(), + Some("0.0.117-dev.32+g5b925dd8a") + ); + } + + #[test] + fn next_dev_version_without_a_release_starts_at_first_patch() { + assert_eq!( + next_dev_version(None, 7, "abcdef123").as_deref(), + Some("0.0.1-dev.7+gabcdef123") + ); + } +} diff --git a/crates/openshell-core/src/container_paths.rs b/crates/openshell-core/src/container_paths.rs index c63e4bcdd8..e26ea53f7a 100644 --- a/crates/openshell-core/src/container_paths.rs +++ b/crates/openshell-core/src/container_paths.rs @@ -65,6 +65,32 @@ pub const VM_GUEST_TLS_KEY_PATH: &str = "/opt/openshell/tls/tls.key"; pub const VM_GUEST_SANDBOX_TOKEN_PATH: &str = "/opt/openshell/auth/sandbox.jwt"; pub const VM_GUEST_INIT_DROPIN_DIR: &str = "/opt/openshell/init.d"; pub const VM_GUEST_INIT_DROPIN_MANIFEST: &str = "/opt/openshell/init.d.manifest"; + +/// Guest path for the corporate upstream-proxy credential in VM sandboxes. +/// +/// The VM driver stages the `user:pass` credential here (mode `0600`, +/// root-only) inside the per-sandbox overlay upperdir, and passes only this +/// path on the supervisor's argv. A microVM has no bind mounts or container +/// secrets, so this is the same delivery the per-sandbox JWT already uses. +pub const VM_GUEST_UPSTREAM_PROXY_AUTH_PATH: &str = "/opt/openshell/auth/upstream-proxy"; + +/// Guest path for the corporate proxy CA bundle in VM sandboxes. +/// +/// A CA certificate is not secret, so unlike the credential this is staged +/// world-readable. The supervisor trusts it for the handshake with an +/// `https://` proxy and for server certificates re-signed by a +/// TLS-intercepting proxy. +pub const VM_GUEST_PROXY_CA_PATH: &str = "/opt/openshell/tls/proxy-ca.pem"; + +/// Guest path for the driver-authored supervisor argument list in VM sandboxes. +/// +/// Podman and Kubernetes build the supervisor's command line directly; the VM +/// guest init script execs a fixed argv, so driver-owned arguments travel +/// through this file instead. The driver writes it into the overlay upperdir +/// on every launch — empty when it has no arguments to pass — so a sandbox +/// image can neither forge entries nor shadow the driver's copy, and the +/// guest appends exactly what it finds there and nothing else. +pub const VM_GUEST_SUPERVISOR_ARGS_PATH: &str = "/opt/openshell/supervisor-args"; pub const VM_UMOCI_PATH: &str = "/opt/openshell/bin/umoci"; pub const VM_SANDBOX_OWNER_NORMALIZED_MARKER: &str = "/opt/openshell/.sandbox-owner-normalized"; @@ -103,6 +129,9 @@ mod tests { VM_GUEST_SANDBOX_TOKEN_PATH, VM_GUEST_INIT_DROPIN_DIR, VM_GUEST_INIT_DROPIN_MANIFEST, + VM_GUEST_UPSTREAM_PROXY_AUTH_PATH, + VM_GUEST_PROXY_CA_PATH, + VM_GUEST_SUPERVISOR_ARGS_PATH, VM_UMOCI_PATH, VM_SANDBOX_OWNER_NORMALIZED_MARKER, ]; diff --git a/crates/openshell-core/src/driver_utils.rs b/crates/openshell-core/src/driver_utils.rs index c8be114ebd..74871751da 100644 --- a/crates/openshell-core/src/driver_utils.rs +++ b/crates/openshell-core/src/driver_utils.rs @@ -375,6 +375,123 @@ pub const MAX_UPSTREAM_PROXY_CREDENTIAL_BYTES: u64 = 4096; /// cannot be opened or stat'd, is not a regular file, or exceeds the size /// bound. pub fn read_upstream_proxy_credential_file(path: &str) -> Result { + read_regular_file_bounded(path, MAX_UPSTREAM_PROXY_CREDENTIAL_BYTES).map_err(|err| match err { + BoundedReadError::Open(e) => format!("failed to open proxy auth file '{path}': {e}"), + BoundedReadError::Stat(e) => format!("failed to stat proxy auth file '{path}': {e}"), + BoundedReadError::NotRegular => format!("proxy auth file '{path}' is not a regular file"), + BoundedReadError::TooLarge => format!( + "proxy auth file '{path}' exceeds the {MAX_UPSTREAM_PROXY_CREDENTIAL_BYTES}-byte limit" + ), + BoundedReadError::Read(e) => format!("failed to read proxy auth file '{path}': {e}"), + }) +} + +/// Hard upper bound on the size of a corporate proxy CA bundle file. +/// +/// A CA bundle holding every corporate trust anchor is a few tens of +/// kilobytes; this cap only exists so a hostile or misconfigured path (a huge +/// file, or a special file such as `/dev/zero`) cannot exhaust gateway, +/// driver, or supervisor memory during a bounded read. +pub const MAX_UPSTREAM_PROXY_CA_BUNDLE_BYTES: u64 = 1024 * 1024; + +/// Read and validate an operator corporate proxy CA bundle PEM file. +/// +/// Rejects non-regular files (e.g. `/dev/zero`, directories, FIFOs) and files +/// larger than [`MAX_UPSTREAM_PROXY_CA_BUNDLE_BYTES`], then requires the +/// bundle to contribute at least one trust anchor rustls actually accepts — +/// see [`validate_upstream_proxy_ca_bundle_pem`]. Returns the PEM contents. +/// +/// Shared by the compute driver (at sandbox-create time, so the operator gets +/// an error naming the setting) and the in-container supervisor (at startup), +/// so a bundle accepted on the host is never rejected inside the sandbox and +/// vice versa. This is a blocking read; async callers should wrap it (e.g. +/// `tokio::task::spawn_blocking`). +/// +/// `label` names the operator-facing setting (`proxy_ca_bundle`, or the +/// supervisor's argument name) and prefixes every error. +/// +/// # Errors +/// +/// Returns a descriptive error (never containing file contents) when the path +/// cannot be read, is not a regular file, exceeds the size bound, or holds no +/// usable certificate. +pub fn read_upstream_proxy_ca_bundle_file(path: &str, label: &str) -> Result { + let pem = read_regular_file_bounded(path, MAX_UPSTREAM_PROXY_CA_BUNDLE_BYTES).map_err( + |err| match err { + BoundedReadError::Open(e) | BoundedReadError::Stat(e) | BoundedReadError::Read(e) => { + format!("{label} '{path}' could not be read: {e}") + } + BoundedReadError::NotRegular => { + format!("{label} '{path}' is not a regular file") + } + BoundedReadError::TooLarge => format!( + "{label} '{path}' exceeds the {MAX_UPSTREAM_PROXY_CA_BUNDLE_BYTES}-byte limit" + ), + }, + )?; + validate_upstream_proxy_ca_bundle_pem(&pem, path, label)?; + Ok(pem) +} + +/// Require a CA bundle PEM to contribute at least one usable trust anchor. +/// +/// Fail-closed to match the rest of the operator-owned proxy configuration: +/// the operator explicitly pointed at this file, so a bundle with no usable +/// certificate is an error rather than a silent fall-back to the built-in +/// roots that would quietly weaken the trust boundary. +/// +/// Validating that rustls accepts an anchor — rather than only that PEM +/// framing base64-decodes — is what makes the host-side check equivalent to +/// the guest-side one: a PEM block holding invalid DER passes +/// `rustls_pemfile::certs` but is silently dropped by +/// `RootCertStore::add_parsable_certificates`, so counting PEM blocks alone +/// would accept on the host a bundle that contributes zero anchors at runtime. +/// +/// # Errors +/// +/// Returns a descriptive error, prefixed with `label` and naming `path`, when +/// the PEM holds no certificate block or no block contains valid X.509 DER. +pub fn validate_upstream_proxy_ca_bundle_pem( + pem: &str, + path: &str, + label: &str, +) -> Result<(), String> { + let certs: Vec<_> = rustls_pemfile::certs(&mut pem.as_bytes()) + .flatten() + .collect(); + if certs.is_empty() { + return Err(format!( + "{label} '{path}' contains no PEM certificate blocks" + )); + } + let mut store = rustls::RootCertStore::empty(); + let (added, _ignored) = store.add_parsable_certificates(certs); + if added == 0 { + return Err(format!( + "{label} '{path}' contains no usable trust anchors \ + (PEM blocks were found but none contain valid X.509 DER)" + )); + } + Ok(()) +} + +/// Failure modes of [`read_regular_file_bounded`], so each caller can phrase +/// them in terms of the operator setting it is reading. +enum BoundedReadError { + Open(std::io::Error), + Stat(std::io::Error), + NotRegular, + TooLarge, + Read(std::io::Error), +} + +/// Read a regular file into a `String`, rejecting anything larger than +/// `max_bytes` and anything that is not a regular file. +/// +/// Backs the operator-supplied proxy file readers, which must never let a +/// hostile or misconfigured path (`/dev/zero`, a FIFO, a directory, a huge +/// file) exhaust memory or block the caller. +fn read_regular_file_bounded(path: &str, max_bytes: u64) -> Result { use std::io::Read as _; // Windows rejects opening a directory before a file handle is available, @@ -384,10 +501,9 @@ pub fn read_upstream_proxy_credential_file(path: &str) -> Result // window if the path is replaced between these operations. #[cfg(target_os = "windows")] { - let path_metadata = std::fs::metadata(path) - .map_err(|e| format!("failed to open proxy auth file '{path}': {e}"))?; + let path_metadata = std::fs::metadata(path).map_err(BoundedReadError::Open)?; if !path_metadata.is_file() { - return Err(format!("proxy auth file '{path}' is not a regular file")); + return Err(BoundedReadError::NotRegular); } } @@ -405,31 +521,145 @@ pub fn read_upstream_proxy_credential_file(path: &str) -> Result #[cfg(not(unix))] let open_result = std::fs::File::open(path); - let file = open_result.map_err(|e| format!("failed to open proxy auth file '{path}': {e}"))?; - let metadata = file - .metadata() - .map_err(|e| format!("failed to stat proxy auth file '{path}': {e}"))?; + let file = open_result.map_err(BoundedReadError::Open)?; + let metadata = file.metadata().map_err(BoundedReadError::Stat)?; if !metadata.is_file() { - return Err(format!("proxy auth file '{path}' is not a regular file")); + return Err(BoundedReadError::NotRegular); } - if metadata.len() > MAX_UPSTREAM_PROXY_CREDENTIAL_BYTES { - return Err(format!( - "proxy auth file '{path}' exceeds the {MAX_UPSTREAM_PROXY_CREDENTIAL_BYTES}-byte limit" - )); + if metadata.len() > max_bytes { + return Err(BoundedReadError::TooLarge); } // Bound the read even if the file grows between stat and read. let mut buf = String::new(); - file.take(MAX_UPSTREAM_PROXY_CREDENTIAL_BYTES + 1) + file.take(max_bytes + 1) .read_to_string(&mut buf) - .map_err(|e| format!("failed to read proxy auth file '{path}': {e}"))?; - if buf.len() as u64 > MAX_UPSTREAM_PROXY_CREDENTIAL_BYTES { - return Err(format!( - "proxy auth file '{path}' exceeds the {MAX_UPSTREAM_PROXY_CREDENTIAL_BYTES}-byte limit" - )); + .map_err(BoundedReadError::Read)?; + if buf.len() as u64 > max_bytes { + return Err(BoundedReadError::TooLarge); } Ok(buf) } +/// Operator-supplied corporate upstream-proxy settings, as a borrowed view. +/// +/// Compute drivers store these keys under their own +/// `[openshell.drivers.]` table; this type exists so the pairing rules +/// between them live in one place instead of being restated per driver. +/// Field names map 1:1 onto the documented TOML keys `https_proxy`, +/// `no_proxy`, `proxy_auth_file`, `proxy_auth_allow_insecure`, +/// `proxy_connect_by_hostname`, and `proxy_ca_bundle`. +#[derive(Debug, Clone, Copy, Default)] +pub struct UpstreamProxySettings<'a> { + /// `https_proxy`: the corporate forward proxy URL. + pub url: Option<&'a str>, + /// `no_proxy`: comma-separated bypass list. + pub no_proxy: Option<&'a str>, + /// `proxy_auth_file`: host path to a `user:pass` credential file. + pub auth_file: Option<&'a str>, + /// `proxy_auth_allow_insecure`: acknowledgement that Basic auth to an + /// `http://` proxy travels in cleartext. + pub auth_allow_insecure: Option, + /// `proxy_connect_by_hostname`: send hostnames rather than validated IPs + /// in CONNECT requests. + pub connect_by_hostname: Option, + /// `proxy_ca_bundle`: host path to a PEM CA bundle trusted for the proxy. + pub ca_bundle: Option<&'a str>, +} + +/// Validate operator-supplied corporate upstream-proxy settings, fail-closed. +/// +/// Shares URL semantics with the in-container supervisor through +/// [`parse_upstream_proxy_url`], so a value accepted here can never be +/// rejected by the supervisor at sandbox startup (or vice versa). Every +/// auxiliary setting is only meaningful relative to a proxy boundary the +/// operator believed was in effect, so a stray one is rejected rather than +/// silently accepted while all egress dials directly. +/// +/// A present-but-empty string is rejected everywhere: the supervisor treats +/// an empty driver-supplied argument as a fatal misconfiguration, so a driver +/// must never accept (and later pass) one. +/// +/// # Errors +/// +/// Returns a message naming the offending key. +pub fn validate_upstream_proxy_settings( + settings: &UpstreamProxySettings<'_>, +) -> Result<(), String> { + let proxy_secure = if let Some(url) = settings.url { + let addr = parse_upstream_proxy_url(url).map_err(|err| match err { + UpstreamProxyUrlError::Empty => "https_proxy must not be empty when set".to_string(), + UpstreamProxyUrlError::InlineCredentials => { + "https_proxy must not embed credentials in the URL; supply them via \ + proxy_auth_file so they are not stored in config or sandbox metadata" + .to_string() + } + err => format!("https_proxy {err}"), + })?; + addr.secure + } else { + false + }; + + if let Some(list) = settings.no_proxy { + if list.trim().is_empty() { + return Err("no_proxy must not be empty when set; omit it instead".to_string()); + } + if settings.url.is_none() { + return Err("no_proxy is set but no https_proxy is configured".to_string()); + } + } + + if let Some(path) = settings.auth_file { + if path.trim().is_empty() { + return Err("proxy_auth_file must not be empty when set".to_string()); + } + if settings.url.is_none() { + return Err("proxy_auth_file is set but no https_proxy is configured".to_string()); + } + // Basic auth over the plain-TCP proxy connection is readable by + // anyone on the network path; sending it requires an explicit + // operator acknowledgement rather than being an implicit side effect + // of configuring credentials. For an https:// proxy the credential is + // inside the verified TLS session, so the acknowledgement is + // unnecessary (but tolerated). + if settings.auth_allow_insecure != Some(true) && !proxy_secure { + return Err( + "proxy_auth_file sends the credential as cleartext Basic auth over the \ + plain-TCP connection to the http:// proxy; set proxy_auth_allow_insecure \ + = true to accept that exposure, or remove proxy_auth_file" + .to_string(), + ); + } + } else if settings.auth_allow_insecure.is_some() { + // The acknowledgement without credentials means the operator believed + // an auth file was configured; surface the mismatch. + return Err( + "proxy_auth_allow_insecure is set but no proxy_auth_file is configured".to_string(), + ); + } + + if settings.connect_by_hostname.is_some() && settings.url.is_none() { + return Err( + "proxy_connect_by_hostname is set but no https_proxy is configured".to_string(), + ); + } + + // A CA bundle only makes sense relative to a proxy boundary (an https:// + // proxy handshake, or a TLS-intercepting proxy's re-sign CA). The file's + // readability and certificate content are checked at sandbox-create time + // by the driver and fail closed again in the supervisor. + if let Some(path) = settings.ca_bundle { + if path.trim().is_empty() { + return Err("proxy_ca_bundle must not be empty when set".to_string()); + } + if settings.url.is_none() { + return Err("proxy_ca_bundle is set but no https_proxy is configured".to_string()); + } + } + + Ok(()) +} + /// Container-side directory where the provider SPIFFE Workload API socket is mounted. pub const PROVIDER_SPIFFE_WORKLOAD_API_SOCKET_MOUNT_DIR: &str = "/spiffe-workload-api"; @@ -888,4 +1118,273 @@ mod tests { "reading a FIFO must not block" ); } + + /// Build settings with only the fields a case cares about. + #[test] + fn ca_bundle_file_accepts_a_real_certificate() { + // The positive case that pins host acceptance to guest acceptance: + // what the driver stages is exactly what rustls will trust. + let cert = rcgen::generate_simple_self_signed(vec!["proxy.corp.example".to_string()]) + .expect("test CA"); + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("proxy-ca.pem"); + std::fs::write(&path, cert.cert.pem()).unwrap(); + + let pem = + read_upstream_proxy_ca_bundle_file(path.to_str().unwrap(), "proxy_ca_bundle").unwrap(); + assert!(pem.contains("BEGIN CERTIFICATE")); + } + + #[test] + fn ca_bundle_file_rejects_non_regular_and_oversized_paths() { + // /dev/zero is the case that matters: an unbounded read of it would + // exhaust gateway or driver memory on any authorized sandbox create. + let dir = tempfile::tempdir().unwrap(); + let err = + read_upstream_proxy_ca_bundle_file(dir.path().to_str().unwrap(), "proxy_ca_bundle") + .unwrap_err(); + assert!(err.contains("regular file"), "{err}"); + assert!(err.contains("proxy_ca_bundle"), "{err}"); + + if Path::new("/dev/zero").exists() { + let err = + read_upstream_proxy_ca_bundle_file("/dev/zero", "proxy_ca_bundle").unwrap_err(); + assert!(err.contains("regular file"), "{err}"); + } + + let oversized = dir.path().join("oversized.pem"); + std::fs::write( + &oversized, + vec![b'x'; usize::try_from(MAX_UPSTREAM_PROXY_CA_BUNDLE_BYTES).unwrap() + 1], + ) + .unwrap(); + let err = + read_upstream_proxy_ca_bundle_file(oversized.to_str().unwrap(), "proxy_ca_bundle") + .unwrap_err(); + assert!(err.contains("exceeds"), "{err}"); + } + + #[test] + fn ca_bundle_file_missing_path_is_an_error() { + let err = + read_upstream_proxy_ca_bundle_file("/nonexistent/proxy-ca.pem", "proxy_ca_bundle") + .unwrap_err(); + assert!(err.contains("could not be read"), "{err}"); + } + + #[test] + fn ca_bundle_rejects_a_file_without_certificate_blocks() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("proxy-ca.pem"); + std::fs::write(&path, "this is not a certificate\n").unwrap(); + let err = read_upstream_proxy_ca_bundle_file(path.to_str().unwrap(), "proxy_ca_bundle") + .unwrap_err(); + assert!(err.contains("no PEM certificate blocks"), "{err}"); + + std::fs::write(&path, "").unwrap(); + let err = read_upstream_proxy_ca_bundle_file(path.to_str().unwrap(), "proxy_ca_bundle") + .unwrap_err(); + assert!(err.contains("no PEM certificate blocks"), "{err}"); + } + + #[test] + fn ca_bundle_rejects_pem_blocks_holding_invalid_der() { + // Passes `rustls_pemfile::certs` but contributes no trust anchor, so + // accepting it on the host would break every guest after boot. + let err = validate_upstream_proxy_ca_bundle_pem( + "-----BEGIN CERTIFICATE-----\nAQID\n-----END CERTIFICATE-----\n", + "/etc/openshell/tls/proxy-ca.pem", + "proxy_ca_bundle", + ) + .unwrap_err(); + assert!(err.contains("no usable trust anchors"), "{err}"); + } + + fn proxy_settings(url: Option<&str>) -> UpstreamProxySettings<'_> { + UpstreamProxySettings { + url, + ..UpstreamProxySettings::default() + } + } + + #[test] + fn upstream_proxy_settings_accept_a_bare_proxy_url() { + validate_upstream_proxy_settings(&proxy_settings(Some("http://proxy.corp.com:3128"))) + .expect("a lone proxy URL is a complete configuration"); + } + + #[test] + fn upstream_proxy_settings_accept_an_empty_configuration() { + validate_upstream_proxy_settings(&UpstreamProxySettings::default()) + .expect("no proxy configured at all is valid"); + } + + #[test] + fn upstream_proxy_settings_reject_an_unsupported_scheme() { + let err = validate_upstream_proxy_settings(&proxy_settings(Some("socks5://proxy:1080"))) + .expect_err("only http:// and https:// proxies are supported"); + assert!(err.starts_with("https_proxy "), "{err}"); + assert!(err.contains("unsupported proxy scheme"), "{err}"); + } + + #[test] + fn upstream_proxy_settings_reject_inline_credentials_by_naming_the_auth_file() { + let err = validate_upstream_proxy_settings(&proxy_settings(Some("http://u:p@proxy:3128"))) + .expect_err("inline credentials would be stored in gateway config"); + assert!(err.contains("proxy_auth_file"), "{err}"); + } + + #[test] + fn upstream_proxy_settings_reject_an_empty_proxy_url() { + let err = validate_upstream_proxy_settings(&proxy_settings(Some(" "))) + .expect_err("present-but-empty is a misconfiguration, not 'unset'"); + assert_eq!(err, "https_proxy must not be empty when set"); + } + + #[test] + fn upstream_proxy_settings_reject_auxiliary_keys_without_a_proxy_url() { + // Each auxiliary key implies a proxy boundary the operator believed + // was in effect; accepting one while every dial goes direct would + // hide a fail-open state. + for (settings, key) in [ + ( + UpstreamProxySettings { + no_proxy: Some("10.0.0.0/8"), + ..UpstreamProxySettings::default() + }, + "no_proxy", + ), + ( + UpstreamProxySettings { + auth_file: Some("/etc/openshell/secrets/proxy-auth"), + ..UpstreamProxySettings::default() + }, + "proxy_auth_file", + ), + ( + UpstreamProxySettings { + connect_by_hostname: Some(true), + ..UpstreamProxySettings::default() + }, + "proxy_connect_by_hostname", + ), + ( + UpstreamProxySettings { + ca_bundle: Some("/etc/openshell/tls/proxy-ca.pem"), + ..UpstreamProxySettings::default() + }, + "proxy_ca_bundle", + ), + ] { + let err = validate_upstream_proxy_settings(&settings) + .expect_err("an auxiliary key without a proxy URL must fail closed"); + assert_eq!( + err, + format!("{key} is set but no https_proxy is configured") + ); + } + } + + #[test] + fn upstream_proxy_settings_reject_empty_auxiliary_values() { + for (settings, expected) in [ + ( + UpstreamProxySettings { + url: Some("http://proxy:3128"), + no_proxy: Some(" "), + ..UpstreamProxySettings::default() + }, + "no_proxy must not be empty when set; omit it instead", + ), + ( + UpstreamProxySettings { + url: Some("http://proxy:3128"), + auth_file: Some(""), + ..UpstreamProxySettings::default() + }, + "proxy_auth_file must not be empty when set", + ), + ( + UpstreamProxySettings { + url: Some("http://proxy:3128"), + ca_bundle: Some(""), + ..UpstreamProxySettings::default() + }, + "proxy_ca_bundle must not be empty when set", + ), + ] { + let err = validate_upstream_proxy_settings(&settings) + .expect_err("present-but-empty must never be treated as unset"); + assert_eq!(err, expected); + } + } + + #[test] + fn upstream_proxy_credentials_require_the_cleartext_acknowledgement() { + let err = validate_upstream_proxy_settings(&UpstreamProxySettings { + url: Some("http://proxy:3128"), + auth_file: Some("/etc/openshell/secrets/proxy-auth"), + ..UpstreamProxySettings::default() + }) + .expect_err("Basic auth to an http:// proxy is cleartext on the wire"); + assert!(err.contains("proxy_auth_allow_insecure"), "{err}"); + + validate_upstream_proxy_settings(&UpstreamProxySettings { + url: Some("http://proxy:3128"), + auth_file: Some("/etc/openshell/secrets/proxy-auth"), + auth_allow_insecure: Some(true), + ..UpstreamProxySettings::default() + }) + .expect("the explicit acknowledgement makes the exposure an operator decision"); + } + + #[test] + fn upstream_proxy_credentials_need_no_acknowledgement_for_an_https_proxy() { + // The credential travels inside the verified TLS session to the proxy. + validate_upstream_proxy_settings(&UpstreamProxySettings { + url: Some("https://proxy:3130"), + auth_file: Some("/etc/openshell/secrets/proxy-auth"), + ..UpstreamProxySettings::default() + }) + .expect("an https:// proxy does not expose the credential on the wire"); + + // ... but setting it anyway is tolerated rather than an error. + validate_upstream_proxy_settings(&UpstreamProxySettings { + url: Some("https://proxy:3130"), + auth_file: Some("/etc/openshell/secrets/proxy-auth"), + auth_allow_insecure: Some(true), + ..UpstreamProxySettings::default() + }) + .expect("a redundant acknowledgement is tolerated"); + } + + #[test] + fn upstream_proxy_acknowledgement_without_credentials_is_rejected() { + // Including `= false`: the operator believed an auth file was + // configured, so the mismatch is surfaced rather than ignored. + for ack in [Some(true), Some(false)] { + let err = validate_upstream_proxy_settings(&UpstreamProxySettings { + url: Some("http://proxy:3128"), + auth_allow_insecure: ack, + ..UpstreamProxySettings::default() + }) + .expect_err("the acknowledgement is meaningless without a credential"); + assert_eq!( + err, + "proxy_auth_allow_insecure is set but no proxy_auth_file is configured" + ); + } + } + + #[test] + fn upstream_proxy_ca_bundle_is_valid_with_a_plain_http_proxy() { + // A TLS-intercepting proxy can be reached over plain HTTP while still + // re-signing tunneled server certificates with its own CA. + validate_upstream_proxy_settings(&UpstreamProxySettings { + url: Some("http://proxy:3128"), + ca_bundle: Some("/etc/openshell/tls/proxy-ca.pem"), + ..UpstreamProxySettings::default() + }) + .expect("an intercepting proxy's CA is meaningful without an https:// proxy URL"); + } } diff --git a/crates/openshell-core/src/lib.rs b/crates/openshell-core/src/lib.rs index 9aecfcb1f1..03ae8a30fd 100644 --- a/crates/openshell-core/src/lib.rs +++ b/crates/openshell-core/src/lib.rs @@ -64,15 +64,19 @@ pub use metadata::{ /// Build version string derived from git metadata. /// -/// For local builds this is computed by `build.rs` via `git describe` using -/// the guess-next-dev scheme (e.g. `0.0.4-dev.6+g2bf9969`). In Docker/CI -/// builds where `.git` is absent, falls back to `CARGO_PKG_VERSION` which -/// is already set correctly by the build pipeline's sed patch. +/// For local builds this is computed by `build.rs` from the exact release tag +/// or the latest merged stable tag using the guess-next-dev scheme (e.g. +/// `0.0.4-dev.6+g2bf9969ab`). In Docker/CI builds where `.git` is absent, it +/// falls back to `CARGO_PKG_VERSION`, which the build pipeline already stamps. pub const VERSION: &str = match option_env!("OPENSHELL_GIT_VERSION") { Some(v) => v, None => env!("CARGO_PKG_VERSION"), }; +#[cfg(test)] +#[path = "../build_version.rs"] +mod build_version; + /// Encoded protobuf `FileDescriptorSet` for every proto in `proto/`. /// /// Emitted by `build.rs` via `tonic_build::configure().file_descriptor_set_path(...)`. diff --git a/crates/openshell-core/src/metadata.rs b/crates/openshell-core/src/metadata.rs index 8794c11d5d..f885812c4e 100644 --- a/crates/openshell-core/src/metadata.rs +++ b/crates/openshell-core/src/metadata.rs @@ -6,8 +6,9 @@ //! These traits provide uniform access to `ObjectMeta` fields across all resource types. use crate::proto::{ - InferenceRoute, ObjectForTest, Provider, Sandbox, SandboxStatus, ServiceEndpoint, SshSession, - StoredProviderCredentialRefreshState, StoredProviderProfile, Workspace, WorkspaceMember, + InferenceRoute, ObjectForTest, Provider, Sandbox, SandboxStatus, SandboxWorkloadTemplate, + ServiceEndpoint, SshSession, StoredProviderCredentialRefreshState, StoredProviderProfile, + Workspace, WorkspaceMember, }; use std::collections::HashMap; @@ -104,6 +105,49 @@ impl Sandbox { } } +// Implementations for SandboxWorkloadTemplate +impl ObjectId for SandboxWorkloadTemplate { + fn object_id(&self) -> &str { + self.metadata.as_ref().map_or("", |m| m.id.as_str()) + } +} + +impl ObjectName for SandboxWorkloadTemplate { + fn object_name(&self) -> &str { + self.metadata.as_ref().map_or("", |m| m.name.as_str()) + } +} + +impl ObjectLabels for SandboxWorkloadTemplate { + fn object_labels(&self) -> Option> { + self.metadata.as_ref().map(|m| m.labels.clone()) + } +} + +impl SetResourceVersion for SandboxWorkloadTemplate { + fn set_resource_version(&mut self, version: u64) { + if let Some(meta) = self.metadata.as_mut() { + meta.resource_version = version; + } + } +} + +impl GetResourceVersion for SandboxWorkloadTemplate { + fn get_resource_version(&self) -> u64 { + self.metadata.as_ref().map_or(0, |m| m.resource_version) + } +} + +impl ObjectWorkspace for SandboxWorkloadTemplate { + fn object_workspace(&self) -> &str { + self.metadata.as_ref().map_or("", |m| m.workspace.as_str()) + } + + fn requires_workspace() -> bool { + true + } +} + // Implementations for Workspace impl ObjectId for Workspace { fn object_id(&self) -> &str { diff --git a/crates/openshell-core/src/telemetry.rs b/crates/openshell-core/src/telemetry.rs index 780e5c7920..63aa5f9181 100644 --- a/crates/openshell-core/src/telemetry.rs +++ b/crates/openshell-core/src/telemetry.rs @@ -145,6 +145,7 @@ impl PolicyDecisionOperation { pub enum SandboxTemplateSource { Default, Image, + WorkloadTemplate, Undefined, } @@ -154,6 +155,7 @@ impl SandboxTemplateSource { match self { Self::Default => "default", Self::Image => "image", + Self::WorkloadTemplate => "workload_template", Self::Undefined => "undefined", } } diff --git a/crates/openshell-driver-vm/README.md b/crates/openshell-driver-vm/README.md index 4fc9ace415..5c61ae1823 100644 --- a/crates/openshell-driver-vm/README.md +++ b/crates/openshell-driver-vm/README.md @@ -154,6 +154,14 @@ Select the VM driver with `--drivers vm`, `OPENSHELL_DRIVERS=vm`, or `compute_dr | `guest_tls_ca` | unset | CA cert for the guest's mTLS client bundle. Required when `grpc_endpoint` uses `https://`. | | `guest_tls_cert` | unset | Guest client certificate. | | `guest_tls_key` | unset | Guest client private key. | +| `https_proxy` | unset | Corporate forward proxy (`http://host:port` or `https://host:port`) the in-guest supervisor chains policy-approved TLS CONNECT egress through. On the libkrun backend a proxy on the gateway host's loopback must be addressed as `http://host.openshell.internal:` — guest egress leaves through gvproxy, which NATs `192.168.127.254` to the host's `127.0.0.1`. The QEMU/TAP backend (GPU sandboxes) has no such NAT and its nftables rules expose only the gateway port to the guest, so a gateway-host proxy URL is rejected at launch there; use an address routable from the guest's masqueraded egress. | +| `no_proxy` | unset | Comma-separated bypass list for the corporate proxy only. OpenShell policy evaluation still applies. | +| `proxy_auth_file` | unset | Gateway-host path to a `user:pass` credential file. Staged root-only into the per-sandbox overlay and removed with the sandbox. | +| `proxy_auth_allow_insecure` | unset | Required with `proxy_auth_file` against an `http://` proxy: acknowledges that Basic auth is cleartext on the connection to the proxy. | +| `proxy_connect_by_hostname` | unset | Send hostnames rather than validated IPs in CONNECT. Last resort for proxies whose ACLs reject IP targets. | +| `proxy_ca_bundle` | unset | Gateway-host path to a PEM CA bundle trusted for an `https://` proxy and for certificates a TLS-intercepting proxy re-signs. | + +The proxy settings are operator-owned and deployment-level: they are not accepted through `template.driver_config.vm`, and they reach the supervisor on its command line through a per-sandbox argument file the driver writes into the overlay upperdir on every launch, so a sandbox image cannot forge or shadow them. Every present-but-invalid value is fatal at gateway or sandbox startup rather than degrading to a direct dial. See [`openshell-gateway --help`](../openshell-server/src/cli.rs) for the gateway process flag surface. diff --git a/crates/openshell-driver-vm/scripts/openshell-vm-sandbox-init.sh b/crates/openshell-driver-vm/scripts/openshell-vm-sandbox-init.sh index 14dbc0466b..32d6ed1dff 100644 --- a/crates/openshell-driver-vm/scripts/openshell-vm-sandbox-init.sh +++ b/crates/openshell-driver-vm/scripts/openshell-vm-sandbox-init.sh @@ -192,6 +192,67 @@ prepare_guest_image_rootfs() { rm -rf "$payload_dir" } +# Driver-owned arguments appended to the supervisor's command line. +# +# The VM driver cannot build the supervisor's argv the way the container +# drivers do, so it writes the arguments it chose into the overlay upperdir +# and this script appends them verbatim. Populated by +# read_supervisor_extra_args; empty until then. +SUPERVISOR_EXTRA_ARGS=() + +# Upper bound on driver-supplied supervisor arguments. +# +# The corporate proxy settings are the only producer today and top out at ten +# entries. The cap exists so a corrupt or oversized file cannot expand into an +# unbounded command line. +SUPERVISOR_EXTRA_ARGS_MAX=32 + +read_supervisor_extra_args() { + # Read the driver-authored supervisor argument list, one argument per + # line, verbatim -- no word splitting, globbing, or expansion, so values + # containing spaces (e.g. a NO_PROXY list) survive intact. + # + # Security: this is the operator-owned egress boundary. The driver writes + # this file into the overlay upperdir on every launch, including an empty + # file when it has no arguments to pass, so the upperdir copy always + # shadows the read-only image layer. A sandbox image can therefore neither + # supply its own supervisor arguments by baking a file at this path nor + # disable the operator's by omitting one. A missing file means the driver + # passed nothing; a file it cannot read means the overlay is broken, and + # we fail closed rather than start a supervisor with a silently truncated + # egress configuration. + local args_file + args_file="$(root_path /opt/openshell/supervisor-args)" + + SUPERVISOR_EXTRA_ARGS=() + if [ ! -f "$args_file" ]; then + return 0 + fi + if [ ! -r "$args_file" ]; then + ts "FATAL: supervisor argument list ${args_file} is not readable" + exit 1 + fi + + local arg + while IFS= read -r arg; do + # render_guest_supervisor_args never emits a blank line, so one means + # the file was truncated or tampered with after the driver wrote it. + if [ -z "$arg" ]; then + ts "FATAL: empty entry in supervisor argument list" + exit 1 + fi + if [ "${#SUPERVISOR_EXTRA_ARGS[@]}" -ge "$SUPERVISOR_EXTRA_ARGS_MAX" ]; then + ts "FATAL: supervisor argument list exceeds ${SUPERVISOR_EXTRA_ARGS_MAX} entries" + exit 1 + fi + SUPERVISOR_EXTRA_ARGS+=("$arg") + done < "$args_file" + + if [ "${#SUPERVISOR_EXTRA_ARGS[@]}" -gt 0 ]; then + ts "supervisor arguments from driver: ${#SUPERVISOR_EXTRA_ARGS[@]} entries" + fi +} + exec_supervisor_in_newroot() { local chroot_bin local bootstrap="/.openshell-bootstrap" @@ -214,14 +275,16 @@ exec_supervisor_in_newroot() { "${bootstrap}/lib64/ld-linux-aarch64.so.1"; do if [ -x "/newroot${loader}" ]; then lib_path="${bootstrap}/lib:${bootstrap}/lib64:${bootstrap}/usr/lib:${bootstrap}/usr/lib64:${bootstrap}/lib/aarch64-linux-gnu:${bootstrap}/lib/x86_64-linux-gnu:${bootstrap}/usr/lib/aarch64-linux-gnu:${bootstrap}/usr/lib/x86_64-linux-gnu" - exec "$chroot_bin" /newroot "$loader" --library-path "$lib_path" "$supervisor" --workdir /sandbox + exec "$chroot_bin" /newroot "$loader" --library-path "$lib_path" \ + "$supervisor" --workdir /sandbox "${SUPERVISOR_EXTRA_ARGS[@]+"${SUPERVISOR_EXTRA_ARGS[@]}"}" fi done - exec "$chroot_bin" /newroot "$supervisor" --workdir /sandbox + exec "$chroot_bin" /newroot "$supervisor" --workdir /sandbox "${SUPERVISOR_EXTRA_ARGS[@]+"${SUPERVISOR_EXTRA_ARGS[@]}"}" fi if [ -x /newroot/opt/openshell/bin/openshell-sandbox ]; then - exec "$chroot_bin" /newroot /opt/openshell/bin/openshell-sandbox --workdir /sandbox + exec "$chroot_bin" /newroot /opt/openshell/bin/openshell-sandbox \ + --workdir /sandbox "${SUPERVISOR_EXTRA_ARGS[@]+"${SUPERVISOR_EXTRA_ARGS[@]}"}" fi done @@ -833,11 +896,13 @@ if [ -n "${OPENSHELL_SANDBOX_ID:-}" ]; then ts "OPENSHELL_SANDBOX_ID=${OPENSHELL_SANDBOX_ID}" fi +read_supervisor_extra_args + ts "starting openshell-sandbox supervisor" if [ "${ROOT_PREFIX:-}" = "/newroot" ]; then exec_supervisor_in_newroot fi -exec /opt/openshell/bin/openshell-sandbox --workdir /sandbox +exec /opt/openshell/bin/openshell-sandbox --workdir /sandbox "${SUPERVISOR_EXTRA_ARGS[@]+"${SUPERVISOR_EXTRA_ARGS[@]}"}" } if [ "${1:-}" != "--post-overlay" ]; then diff --git a/crates/openshell-driver-vm/src/driver.rs b/crates/openshell-driver-vm/src/driver.rs index 8adcc79f92..e70a6a48d3 100644 --- a/crates/openshell-driver-vm/src/driver.rs +++ b/crates/openshell-driver-vm/src/driver.rs @@ -60,7 +60,7 @@ use sha2::{Digest, Sha256}; use std::collections::{HashMap, HashSet}; use std::fs; use std::io::Read; -use std::net::Ipv4Addr; +use std::net::{IpAddr, Ipv4Addr}; #[cfg(unix)] use std::os::unix::fs::PermissionsExt; use std::path::{Component, Path, PathBuf}; @@ -132,7 +132,7 @@ impl VmSandboxDriverConfig { /// Code paths route via `GVPROXY_HOST_LOOPBACK_ALIAS` (DNS / /etc/hosts) /// instead so logs stay readable; this constant is kept for documentation /// and parity with the guest init script. -#[allow(dead_code)] +#[allow(dead_code)] // Documentation/parity anchor; all routing goes via the alias. const GVPROXY_HOST_LOOPBACK_IP: &str = "192.168.127.254"; const OPENSHELL_HOST_GATEWAY_ALIAS: &str = "host.openshell.internal"; /// Hostname gvproxy resolves (via its embedded DNS) to the host-loopback IP. @@ -162,6 +162,19 @@ const GUEST_INIT_DROPIN_DIR: &str = openshell_core::container_paths::VM_GUEST_IN /// upperdir on every launch, so the image cannot forge or shadow it. const GUEST_INIT_DROPIN_MANIFEST: &str = openshell_core::container_paths::VM_GUEST_INIT_DROPIN_MANIFEST; +/// Guest path of the root-only corporate proxy credential staged by the driver. +const GUEST_UPSTREAM_PROXY_AUTH_PATH: &str = + openshell_core::container_paths::VM_GUEST_UPSTREAM_PROXY_AUTH_PATH; +/// Guest path of the corporate proxy CA bundle staged by the driver. +const GUEST_PROXY_CA_PATH: &str = openshell_core::container_paths::VM_GUEST_PROXY_CA_PATH; +/// Guest path of the driver-authored supervisor argument list. +/// +/// The counterpart of [`GUEST_INIT_DROPIN_MANIFEST`] for the supervisor's own +/// command line: written into the overlay upperdir on every launch (empty +/// when there is nothing to pass) so the guest appends exactly the arguments +/// the driver chose and a sandbox image cannot forge or shadow them. +const GUEST_SUPERVISOR_ARGS_PATH: &str = + openshell_core::container_paths::VM_GUEST_SUPERVISOR_ARGS_PATH; const IMAGE_CACHE_ROOT_DIR: &str = "images"; const IMAGE_CACHE_ROOTFS_IMAGE: &str = "rootfs.ext4"; const OVERLAY_TEMPLATE_CACHE_DIR: &str = "overlay-templates"; @@ -217,7 +230,7 @@ enum GuestImagePayloadSource { LocalDocker { rootfs_archive: PathBuf }, } -#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)] +#[derive(Clone, serde::Serialize, serde::Deserialize)] pub struct VmDriverConfig { pub openshell_endpoint: String, pub state_dir: PathBuf, @@ -243,6 +256,104 @@ pub struct VmDriverConfig { /// When empty, defaults to the resolved UID. #[serde(default, skip_serializing_if = "Option::is_none")] pub sandbox_gid: Option, + + /// Corporate forward proxy URL (`http://host:port` or `https://host:port`) + /// passed to the in-guest supervisor. + /// + /// The supervisor chains policy-approved TLS tunnels through this proxy + /// with HTTP CONNECT instead of dialing destinations directly. This is an + /// operator-owned egress boundary: it travels on the supervisor's argv, + /// which sandbox spec/template environment and image `ENV` cannot + /// influence. A proxy on the gateway host's loopback is reachable from the + /// guest only through the gvproxy host alias + /// (`host.openshell.internal`). + #[serde(default, skip_serializing_if = "Option::is_none")] + pub https_proxy: Option, + + /// Comma-separated `NO_PROXY` list passed alongside the proxy URL. + /// + /// Matching destinations are dialed directly instead of through the + /// corporate proxy. This bypasses only the corporate proxy, never + /// `OpenShell` policy evaluation. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub no_proxy: Option, + + /// Path (on the gateway host) to a file containing the corporate proxy + /// credential in `user:pass` form. + /// + /// The driver validates it at sandbox-create time and stages it into the + /// per-sandbox overlay at [`GUEST_UPSTREAM_PROXY_AUTH_PATH`], root-only. + /// Credentials are never embedded in the proxy URL and never reach the + /// guest environment. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub proxy_auth_file: Option, + + /// Explicit acknowledgement that proxy credentials are sent in cleartext. + /// + /// `Proxy-Authorization: Basic` over the plain-TCP connection to an + /// `http://` proxy is recoverable by anyone on the network path, so + /// [`Self::proxy_auth_file`] requires this acknowledgement. An `https://` + /// proxy carries the credential inside the verified TLS session and does + /// not need it. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub proxy_auth_allow_insecure: Option, + + /// Send the destination hostname in CONNECT requests instead of a + /// validated IP. + /// + /// The default binds the tunnel to an address that passed the sandbox's + /// SSRF and `allowed_ips` validation. Set this only when the proxy's ACLs + /// filter on hostnames and reject IP CONNECT targets: the proxy then + /// resolves the name itself and its own ACLs become the effective egress + /// control for proxied TLS. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub proxy_connect_by_hostname: Option, + + /// Path (on the gateway host) to a PEM CA bundle trusted for the + /// corporate proxy. + /// + /// The driver stages it into the per-sandbox overlay at + /// [`GUEST_PROXY_CA_PATH`] and passes that path via + /// `--upstream-proxy-ca-bundle`. It is trusted both for the handshake + /// with an `https://` proxy and for server certificates re-signed by a + /// TLS-intercepting proxy. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub proxy_ca_bundle: Option, +} + +/// Redacting `Debug` so a proxy URL or credential path never reaches a log. +/// +/// A validated proxy URL cannot embed credentials, but `Debug` can be emitted +/// before validation runs, so presence is logged rather than the value. +impl std::fmt::Debug for VmDriverConfig { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("VmDriverConfig") + .field("openshell_endpoint", &self.openshell_endpoint) + .field("state_dir", &self.state_dir) + .field("launcher_bin", &self.launcher_bin) + .field("default_image", &self.default_image) + .field("bootstrap_image", &self.bootstrap_image) + .field("log_level", &self.log_level) + .field("krun_log_level", &self.krun_log_level) + .field("vcpus", &self.vcpus) + .field("mem_mib", &self.mem_mib) + .field("overlay_disk_mib", &self.overlay_disk_mib) + .field("guest_tls_ca", &self.guest_tls_ca) + .field("guest_tls_cert", &self.guest_tls_cert) + .field("guest_tls_key", &self.guest_tls_key) + .field("gpu_enabled", &self.gpu_enabled) + .field("gpu_mem_mib", &self.gpu_mem_mib) + .field("gpu_vcpus", &self.gpu_vcpus) + .field("sandbox_uid", &self.sandbox_uid) + .field("sandbox_gid", &self.sandbox_gid) + .field("https_proxy", &self.https_proxy.is_some()) + .field("no_proxy", &self.no_proxy) + .field("proxy_auth_file", &self.proxy_auth_file.is_some()) + .field("proxy_auth_allow_insecure", &self.proxy_auth_allow_insecure) + .field("proxy_connect_by_hostname", &self.proxy_connect_by_hostname) + .field("proxy_ca_bundle", &self.proxy_ca_bundle) + .finish() + } } /// Default sandbox UID used by the VM driver when no config value is set. @@ -269,6 +380,12 @@ impl Default for VmDriverConfig { gpu_vcpus: 4, sandbox_uid: None, sandbox_gid: None, + https_proxy: None, + no_proxy: None, + proxy_auth_file: None, + proxy_auth_allow_insecure: None, + proxy_connect_by_hostname: None, + proxy_ca_bundle: None, } } } @@ -307,6 +424,29 @@ impl VmDriverConfig { Ok(()) } + /// Validate the operator's corporate upstream-proxy settings, fail-closed. + /// + /// Delegates to the validator shared with the Podman and Kubernetes + /// drivers and with the in-guest supervisor, so a value accepted here is + /// never rejected inside the guest — and no misconfiguration can silently + /// degrade to a direct dial. + /// + /// # Errors + /// + /// Returns a message naming the offending key. + pub fn validate_proxy_config(&self) -> Result<(), String> { + openshell_core::driver_utils::validate_upstream_proxy_settings( + &openshell_core::driver_utils::UpstreamProxySettings { + url: self.https_proxy.as_deref(), + no_proxy: self.no_proxy.as_deref(), + auth_file: self.proxy_auth_file.as_deref(), + auth_allow_insecure: self.proxy_auth_allow_insecure, + connect_by_hostname: self.proxy_connect_by_hostname, + ca_bundle: self.proxy_ca_bundle.as_deref(), + }, + ) + } + fn requires_tls_materials(&self) -> bool { self.openshell_endpoint.starts_with("https://") } @@ -447,6 +587,7 @@ impl VmDriver { .validate() .map_err(|err| err.message().to_string())?; config.validate_sandbox_identity()?; + config.validate_proxy_config()?; if config.openshell_endpoint.trim().is_empty() { return Err("openshell endpoint is required".to_string()); } @@ -908,6 +1049,16 @@ impl VmDriver { return Err(err); } + // Staged on every launch, including a restart onto a preserved + // overlay, so the driver's copy always shadows the image layer. + if let Err(err) = inject_guest_upstream_proxy(&overlay_disk, &self.config).await { + self.lifecycle_extensions + .after_launch_failed(&sandbox, &state_dir, LaunchAbortReason::GuestPrepareFailed) + .await; + self.release_gpu_and_subnet(&sandbox.id); + return Err(err); + } + let endpoint_override = if plan.backend == VmBackend::Qemu { plan.host_ip.as_deref().map(|host_ip| { guest_visible_openshell_endpoint_for_tap(&self.config.openshell_endpoint, host_ip) @@ -1694,26 +1845,44 @@ impl VmDriver { if plan.gpu_bdf.is_none() { plan.gpu_bdf = gpu_bdf; } - if has_complete_qemu_network(plan) { - return Ok(()); + if !has_complete_qemu_network(plan) { + let subnet = self + .subnet_allocator + .lock() + .map_err(|e| Status::internal(format!("subnet allocator lock poisoned: {e}")))? + .allocate(sandbox_id) + .map_err(Status::failed_precondition)?; + let mac = mac_from_sandbox_id(sandbox_id); + plan.tap_device = Some(tap_device_name(sandbox_id)); + plan.guest_ip = Some(subnet.guest_ip.to_string()); + plan.host_ip = Some(subnet.host_ip.to_string()); + plan.vsock_cid = Some(allocate_vsock_cid()); + plan.guest_mac = Some(format!( + "{:02x}:{:02x}:{:02x}:{:02x}:{:02x}:{:02x}", + mac[0], mac[1], mac[2], mac[3], mac[4], mac[5] + )); + plan.gateway_port = gateway_port_from_endpoint(&self.config.openshell_endpoint); + } + + // The corporate-proxy host-loopback recipe is a libkrun/gvproxy + // property and has no QEMU/TAP equivalent (see + // `proxy_url_targets_gateway_host`). Run it here, after the subnet + // allocation above has settled `plan.host_ip`, because the address to + // compare against is this sandbox's own TAP host address. Fail the + // create with the reason rather than boot a sandbox whose + // policy-approved CONNECTs all time out against an unreachable proxy. + if let Some(url) = self.config.https_proxy.as_deref() + && proxy_url_targets_gateway_host(url, plan.host_ip.as_deref()) + { + let tap_host = plan.host_ip.as_deref().unwrap_or("the TAP host address"); + return Err(Status::failed_precondition(format!( + "https_proxy '{url}' addresses the gateway host, which a QEMU/TAP sandbox \ + (GPU sandboxes) cannot reach: host.openshell.internal resolves to this \ + sandbox's TAP host address {tap_host} and the driver's nftables rules allow \ + only the gateway port from the guest. Configure a proxy address routable \ + from the guest's masqueraded egress, or run this sandbox without a GPU" + ))); } - - let subnet = self - .subnet_allocator - .lock() - .map_err(|e| Status::internal(format!("subnet allocator lock poisoned: {e}")))? - .allocate(sandbox_id) - .map_err(Status::failed_precondition)?; - let mac = mac_from_sandbox_id(sandbox_id); - plan.tap_device = Some(tap_device_name(sandbox_id)); - plan.guest_ip = Some(subnet.guest_ip.to_string()); - plan.host_ip = Some(subnet.host_ip.to_string()); - plan.vsock_cid = Some(allocate_vsock_cid()); - plan.guest_mac = Some(format!( - "{:02x}:{:02x}:{:02x}:{:02x}:{:02x}:{:02x}", - mac[0], mac[1], mac[2], mac[3], mac[4], mac[5] - )); - plan.gateway_port = gateway_port_from_endpoint(&self.config.openshell_endpoint); Ok(()) } @@ -4437,6 +4606,53 @@ fn guest_visible_openshell_endpoint(endpoint: &str) -> String { endpoint.to_string() } +/// Whether a corporate proxy URL points at the gateway host itself, as seen +/// from a QEMU/TAP guest whose TAP host address is `tap_host_ip`. +/// +/// On the libkrun backend gvproxy NATs the host-loopback alias +/// `host.openshell.internal` (and any loopback URL, which the driver rewrites +/// to that alias) to the gateway host's `127.0.0.1`, so a proxy bound to host +/// loopback is reachable from the guest. The QEMU/TAP backend used for GPU +/// sandboxes has no equivalent: `host.openshell.internal` resolves to the TAP +/// host address, and the driver's own nftables `input` chain accepts only the +/// gateway port from the guest and drops the rest, so no proxy on the gateway +/// host is reachable regardless of the address it binds. +/// +/// The gateway host is therefore reached from a QEMU guest under exactly three +/// spellings: the guest's own loopback (never the host's, but a configuration +/// that plainly means the host), the documented host aliases that +/// `write_host_gateway_aliases` seeds to the TAP host address, and that TAP +/// host address written literally. `tap_host_ip` is this sandbox's allocated +/// address, so the comparison must be made after the launch plan's subnet +/// allocation; `None` means the plan carries no TAP host and only the +/// address-independent spellings are classified. +/// +/// gvproxy's `GVPROXY_HOST_LOOPBACK_IP` is deliberately **not** matched here. +/// It is special only to libkrun; on QEMU/TAP it is an ordinary address that +/// may well be routable through the guest's masqueraded egress, and rejecting +/// it would refuse a working configuration. +/// +/// Used to reject an unreachable configuration up front on the QEMU path +/// instead of letting every policy-approved CONNECT time out. +fn proxy_url_targets_gateway_host(url: &str, tap_host_ip: Option<&str>) -> bool { + let Ok(parsed) = Url::parse(url) else { + // Unparseable URLs are rejected by shared validation before launch. + return false; + }; + let tap_host = tap_host_ip.and_then(|ip| ip.parse::().ok()); + match parsed.host() { + Some(Host::Ipv4(ip)) => ip.is_loopback() || tap_host == Some(IpAddr::V4(ip)), + Some(Host::Ipv6(ip)) => ip.is_loopback() || tap_host == Some(IpAddr::V6(ip)), + Some(Host::Domain(host)) => { + host.eq_ignore_ascii_case("localhost") + || host.eq_ignore_ascii_case(OPENSHELL_HOST_GATEWAY_ALIAS) + || host.eq_ignore_ascii_case("host.containers.internal") + || host.eq_ignore_ascii_case("host.docker.internal") + } + None => false, + } +} + fn gateway_port_from_endpoint(endpoint: &str) -> Option { Url::parse(endpoint).ok().and_then(|url| url.port()) } @@ -5130,6 +5346,182 @@ fn inject_guest_init_dropins( span_status.finish(Ok(())) } +/// Build the corporate upstream-proxy arguments passed to the guest supervisor. +/// +/// This operator-owned egress boundary travels on the supervisor's argv, +/// which sandbox spec/template environment and image `ENV` cannot influence. +/// Credentials are never on argv — only the root-only guest path is passed; +/// the supervisor reads the credential from that file. +fn upstream_proxy_cli_args(config: &VmDriverConfig) -> Vec { + let mut args = Vec::new(); + if let Some(url) = &config.https_proxy { + args.push("--upstream-proxy".to_string()); + args.push(url.clone()); + } + if let Some(list) = &config.no_proxy { + args.push("--upstream-no-proxy".to_string()); + args.push(list.clone()); + } + if config.proxy_auth_file.is_some() { + args.push("--upstream-proxy-auth-file".to_string()); + // The guest path, never the gateway-host path the operator configured. + args.push(GUEST_UPSTREAM_PROXY_AUTH_PATH.to_string()); + } + // Config validation guarantees the acknowledgement is `true` whenever an + // auth file is configured against an http:// proxy; the supervisor + // independently refuses credentials without it. + if config.proxy_auth_allow_insecure == Some(true) { + args.push("--upstream-proxy-auth-allow-insecure".to_string()); + } + // Absent means the default validated-IP CONNECT binding; only the + // explicit hostname opt-in is passed through. + if config.proxy_connect_by_hostname == Some(true) { + args.push("--upstream-proxy-connect-by-hostname".to_string()); + } + if config.proxy_ca_bundle.is_some() { + args.push("--upstream-proxy-ca-bundle".to_string()); + args.push(GUEST_PROXY_CA_PATH.to_string()); + } + args +} + +/// Render the supervisor argument list as newline-separated arguments. +/// +/// One argument per line, verbatim: the guest reads the lines into an array +/// without word splitting or globbing, so values containing spaces survive +/// intact. An empty list renders an empty file, which the guest reads as "no +/// extra arguments". +fn render_guest_supervisor_args(args: &[String]) -> Vec { + let mut body = args.join("\n"); + if !body.is_empty() { + body.push('\n'); + } + body.into_bytes() +} + +/// Reject argument values the newline-delimited guest file cannot represent. +/// +/// Every value here is operator-supplied config, so this is a guard against +/// misconfiguration rather than an attack: a stray newline would otherwise +/// split one value into two arguments in the guest. +fn validate_guest_supervisor_args(args: &[String]) -> Result<(), String> { + for arg in args { + if arg.contains('\n') || arg.contains('\r') || arg.contains('\0') { + return Err( + "corporate proxy settings must not contain newline or NUL characters".to_string(), + ); + } + } + Ok(()) +} + +/// Read and validate the corporate proxy credential from the gateway host. +/// +/// Uses the validators shared with the supervisor, so a credential accepted +/// here is never rejected inside the guest. The error never carries the file +/// contents. +async fn read_sandbox_proxy_credential(path: &str) -> Result { + let path_owned = path.to_string(); + let raw = tokio::task::spawn_blocking(move || { + openshell_core::driver_utils::read_upstream_proxy_credential_file(&path_owned) + }) + .await + .map_err(|err| Status::internal(format!("proxy_auth_file read task failed: {err}")))? + .map_err(Status::invalid_argument)?; + let credential = openshell_core::driver_utils::parse_upstream_proxy_credential(&raw) + .map_err(|err| Status::invalid_argument(format!("proxy_auth_file '{path}': {err}")))?; + Ok(credential.to_string()) +} + +/// Read and validate the corporate proxy CA bundle from the gateway host. +/// +/// Uses the reader shared with the supervisor, so the bundle is bounded and +/// non-regular files are rejected (an operator path such as `/dev/zero` can +/// otherwise exhaust driver memory), and a bundle accepted here contributes at +/// least one trust anchor rustls accepts rather than merely looking like PEM. +/// Checked here rather than only in the guest so the operator gets an error +/// attributable to `proxy_ca_bundle` instead of an opaque supervisor startup +/// failure inside every sandbox. The error never carries the file contents. +async fn read_sandbox_proxy_ca_bundle(path: &str) -> Result, Status> { + let path_owned = path.to_string(); + let pem = tokio::task::spawn_blocking(move || { + openshell_core::driver_utils::read_upstream_proxy_ca_bundle_file( + &path_owned, + "proxy_ca_bundle", + ) + }) + .await + .map_err(|err| Status::internal(format!("proxy_ca_bundle read task failed: {err}")))? + .map_err(Status::invalid_argument)?; + Ok(pem.into_bytes()) +} + +/// Stage the corporate upstream-proxy configuration into the guest overlay. +/// +/// Writes three files into the overlay upperdir the driver owns: +/// +/// * the credential at [`GUEST_UPSTREAM_PROXY_AUTH_PATH`], mode `0600`; +/// * the CA bundle at [`GUEST_PROXY_CA_PATH`], mode `0644` (a CA certificate +/// is not secret); +/// * the supervisor argument list at [`GUEST_SUPERVISOR_ARGS_PATH`], mode +/// `0644`. +/// +/// All three are written on every launch, empty when the corresponding +/// setting is absent. Writing rather than skipping is what makes the channel +/// unforgeable: the upperdir copy always shadows the read-only image layer, so +/// a sandbox image cannot supply its own arguments or credential by baking a +/// file at these paths, and cannot disable the operator's by omitting one. It +/// also clears material a previous launch staged into a preserved overlay +/// after the operator removed the setting. +/// +/// A microVM has no bind mounts or container secrets, so the credential lives +/// at rest inside the per-sandbox overlay disk on the host — the same +/// delivery the per-sandbox gateway JWT already uses. It is removed with the +/// sandbox when the state directory is deleted. +#[allow(clippy::result_large_err)] +async fn inject_guest_upstream_proxy( + overlay_disk: &Path, + config: &VmDriverConfig, +) -> Result<(), Status> { + // Written whether or not they are configured. Writing empty files when + // the operator removed a setting clears material a previous launch staged + // into a preserved overlay, and shadows anything an image baked at these + // paths, so a staged file is only ever the one this launch produced. + let credential = match config.proxy_auth_file.as_deref() { + Some(path) => format!("{}\n", read_sandbox_proxy_credential(path).await?).into_bytes(), + None => Vec::new(), + }; + let credential_path = overlay_upper_path(GUEST_UPSTREAM_PROXY_AUTH_PATH); + write_rootfs_image_file(overlay_disk, &credential_path, &credential) + .map_err(|err| Status::internal(format!("write VM guest proxy credential: {err}")))?; + set_rootfs_image_file_mode(overlay_disk, &credential_path, 0o600) + .map_err(|err| Status::internal(format!("set VM guest proxy credential mode: {err}")))?; + + let ca_bundle = match config.proxy_ca_bundle.as_deref() { + Some(path) => read_sandbox_proxy_ca_bundle(path).await?, + None => Vec::new(), + }; + let ca_path = overlay_upper_path(GUEST_PROXY_CA_PATH); + write_rootfs_image_file(overlay_disk, &ca_path, &ca_bundle) + .map_err(|err| Status::internal(format!("write VM guest proxy CA bundle: {err}")))?; + set_rootfs_image_file_mode(overlay_disk, &ca_path, 0o644) + .map_err(|err| Status::internal(format!("set VM guest proxy CA bundle mode: {err}")))?; + + let args = upstream_proxy_cli_args(config); + validate_guest_supervisor_args(&args).map_err(Status::failed_precondition)?; + let guest_path = overlay_upper_path(GUEST_SUPERVISOR_ARGS_PATH); + write_rootfs_image_file( + overlay_disk, + &guest_path, + &render_guest_supervisor_args(&args), + ) + .map_err(|err| Status::internal(format!("write VM guest supervisor arguments: {err}")))?; + set_rootfs_image_file_mode(overlay_disk, &guest_path, 0o644).map_err(|err| { + Status::internal(format!("set VM guest supervisor arguments mode: {err}")) + })?; + Ok(()) +} + /// Render the drop-in allow-list as newline-separated, ASCII-sorted, /// de-duplicated names. Names are already validated to be path-safe by /// [`validate_guest_init_dropins`]. @@ -8074,6 +8466,12 @@ mod tests { } } + fn test_driver_with_proxy(https_proxy: &str) -> VmDriver { + let mut driver = test_driver_with_extensions(LifecycleExtensionRegistry::new()); + driver.config.https_proxy = Some(https_proxy.to_string()); + driver + } + #[derive(Debug)] struct QemuRequiringExtension { name: String, @@ -8403,4 +8801,466 @@ mod tests { assert!(err.is_resource_exhausted()); assert_eq!(err.message(), "pool empty"); } + + /// A driver config carrying only corporate proxy settings. + fn proxy_config( + https_proxy: Option<&str>, + auth_file: Option<&str>, + ca_bundle: Option<&str>, + ) -> VmDriverConfig { + VmDriverConfig { + openshell_endpoint: "http://127.0.0.1:8080".to_string(), + https_proxy: https_proxy.map(ToString::to_string), + proxy_auth_file: auth_file.map(ToString::to_string), + proxy_auth_allow_insecure: auth_file.map(|_| true), + proxy_ca_bundle: ca_bundle.map(ToString::to_string), + ..Default::default() + } + } + + #[test] + fn driver_config_debug_redacts_the_proxy_url_and_credential_path() { + // `Debug` can be emitted before validation runs, and an unvalidated + // proxy URL may still carry inline `user:pass@` credentials. + let rendered = format!( + "{:?}", + proxy_config( + Some("http://user:secret@proxy.corp.test:3128"), + Some("/etc/openshell/secrets/proxy-auth"), + Some("/etc/openshell/tls/corp-ca.pem"), + ) + ); + assert!( + !rendered.contains("secret") && !rendered.contains("proxy.corp.test"), + "the proxy URL must be logged as presence only: {rendered}" + ); + assert!( + !rendered.contains("/etc/openshell/secrets/proxy-auth"), + "the credential path must be logged as presence only: {rendered}" + ); + assert!( + rendered.contains("https_proxy: true") && rendered.contains("proxy_auth_file: true"), + "presence of each must still be visible for debugging: {rendered}" + ); + // A CA path is not sensitive and stays readable. + assert!( + rendered.contains("corp-ca.pem"), + "the CA bundle path is not a secret and should stay legible: {rendered}" + ); + } + + #[test] + fn proxy_material_is_staged_inside_the_per_sandbox_overlay() { + // Everything the driver stages lands in the overlay upperdir, which + // lives in the sandbox's own state directory. That is what makes the + // credential removable with the sandbox (remove_sandbox_state_dir + // deletes the whole directory) and unforgeable by the guest image + // (the upperdir shadows the read-only image layer). + for guest_path in [ + GUEST_UPSTREAM_PROXY_AUTH_PATH, + GUEST_PROXY_CA_PATH, + GUEST_SUPERVISOR_ARGS_PATH, + ] { + assert!( + guest_path.starts_with("/opt/openshell/"), + "{guest_path} must be under the reserved guest control root" + ); + assert_eq!( + overlay_upper_path(guest_path), + format!("/upper{guest_path}"), + "{guest_path} must be staged into the overlay upperdir" + ); + } + } + + #[test] + fn upstream_proxy_args_are_empty_without_a_configured_proxy() { + assert!(upstream_proxy_cli_args(&VmDriverConfig::default()).is_empty()); + // The file is still written, empty, so the guest cannot fall back to + // an image-baked argument list. + assert!(render_guest_supervisor_args(&[]).is_empty()); + } + + #[test] + fn upstream_proxy_args_pass_guest_paths_not_host_paths() { + let config = proxy_config( + Some("http://proxy.corp.test:3128"), + Some("/etc/openshell/secrets/proxy-auth"), + Some("/etc/openshell/tls/corp-ca.pem"), + ); + let args = upstream_proxy_cli_args(&config); + + // The credential and CA live at fixed guest paths; the gateway-host + // paths the operator configured must never reach the guest argv. + let auth = args + .iter() + .position(|arg| arg == "--upstream-proxy-auth-file") + .map(|i| args[i + 1].as_str()); + assert_eq!(auth, Some(GUEST_UPSTREAM_PROXY_AUTH_PATH)); + let ca = args + .iter() + .position(|arg| arg == "--upstream-proxy-ca-bundle") + .map(|i| args[i + 1].as_str()); + assert_eq!(ca, Some(GUEST_PROXY_CA_PATH)); + assert!( + !args + .iter() + .any(|arg| arg.contains("/etc/openshell/secrets") || arg.contains("corp-ca.pem")), + "host paths leaked into the guest argv: {args:?}" + ); + } + + #[test] + fn upstream_proxy_args_pass_only_explicit_opt_ins() { + let mut config = proxy_config(Some("https://proxy.corp.test:3130"), None, None); + config.no_proxy = Some("10.0.0.0/8,.svc.cluster.local".to_string()); + let args = upstream_proxy_cli_args(&config); + assert_eq!( + args, + vec![ + "--upstream-proxy".to_string(), + "https://proxy.corp.test:3130".to_string(), + "--upstream-no-proxy".to_string(), + "10.0.0.0/8,.svc.cluster.local".to_string(), + ] + ); + + // `Some(false)` must not be passed as the presence flag it is on the + // supervisor side. + config.proxy_connect_by_hostname = Some(false); + assert!( + !upstream_proxy_cli_args(&config) + .iter() + .any(|arg| arg == "--upstream-proxy-connect-by-hostname") + ); + config.proxy_connect_by_hostname = Some(true); + assert!( + upstream_proxy_cli_args(&config) + .iter() + .any(|arg| arg == "--upstream-proxy-connect-by-hostname") + ); + } + + #[test] + fn guest_supervisor_args_render_one_argument_per_line() { + let args = vec![ + "--upstream-proxy".to_string(), + "http://proxy.corp.test:3128".to_string(), + "--upstream-no-proxy".to_string(), + "a.example, b.example".to_string(), + ]; + // A value containing a space stays one line, so the guest reads it + // back as a single argument rather than word-splitting it. + assert_eq!( + String::from_utf8(render_guest_supervisor_args(&args)).unwrap(), + "--upstream-proxy\nhttp://proxy.corp.test:3128\n--upstream-no-proxy\na.example, b.example\n" + ); + } + + #[test] + fn guest_supervisor_args_reject_line_breaking_values() { + // A newline would split one operator value into two guest arguments. + for bad in ["a\nb", "a\rb", "a\0b"] { + assert!( + validate_guest_supervisor_args(&[bad.to_string()]).is_err(), + "{bad:?} must be rejected" + ); + } + validate_guest_supervisor_args(&["--upstream-proxy".to_string()]) + .expect("ordinary arguments are accepted"); + } + + #[test] + fn proxy_config_validation_rejects_settings_without_a_proxy_url() { + let config = VmDriverConfig { + no_proxy: Some("10.0.0.0/8".to_string()), + ..Default::default() + }; + let err = config + .validate_proxy_config() + .expect_err("a bypass list without a proxy would hide a fail-open state"); + assert!(err.contains("no_proxy"), "{err}"); + + let config = proxy_config(Some("http://proxy.corp.test:3128"), None, None); + config + .validate_proxy_config() + .expect("a lone proxy URL is a complete configuration"); + } + + #[test] + fn proxy_config_validation_requires_the_cleartext_acknowledgement() { + let mut config = proxy_config( + Some("http://proxy.corp.test:3128"), + Some("/etc/openshell/secrets/proxy-auth"), + None, + ); + config.proxy_auth_allow_insecure = None; + let err = config + .validate_proxy_config() + .expect_err("Basic auth to an http:// proxy is cleartext on the wire"); + assert!(err.contains("proxy_auth_allow_insecure"), "{err}"); + } + + #[tokio::test] + async fn proxy_ca_bundle_without_a_certificate_fails_the_sandbox() { + let dir = std::env::temp_dir().join(format!("openshell-vm-ca-{}", std::process::id())); + std::fs::create_dir_all(&dir).unwrap(); + let path = dir.join("not-a-ca.pem"); + std::fs::write(&path, b"this is not a certificate\n").unwrap(); + + let err = read_sandbox_proxy_ca_bundle(path.to_str().unwrap()) + .await + .expect_err("a certificate-free bundle must fail closed"); + assert_eq!(err.code(), Code::InvalidArgument); + assert!(err.message().contains("no PEM certificate"), "{err}"); + + std::fs::write(&path, b"").unwrap(); + let err = read_sandbox_proxy_ca_bundle(path.to_str().unwrap()) + .await + .expect_err("an empty bundle must fail closed"); + assert!(err.message().contains("no PEM certificate"), "{err}"); + + // PEM framing that base64-decodes but is not X.509 DER: accepted by + // `rustls_pemfile` alone, contributes zero trust anchors at runtime, + // and so would make every guest supervisor fail after boot. + std::fs::write( + &path, + b"-----BEGIN CERTIFICATE-----\nAQID\n-----END CERTIFICATE-----\n", + ) + .unwrap(); + let err = read_sandbox_proxy_ca_bundle(path.to_str().unwrap()) + .await + .expect_err("a bundle with invalid DER must fail closed"); + assert_eq!(err.code(), Code::InvalidArgument); + assert!(err.message().contains("no usable trust anchors"), "{err}"); + + let err = read_sandbox_proxy_ca_bundle(dir.join("missing.pem").to_str().unwrap()) + .await + .expect_err("an unreadable bundle must fail closed"); + assert!(err.message().contains("could not be read"), "{err}"); + + // A special file must be rejected on its type, not read: an + // unbounded read of /dev/zero would exhaust driver memory. + #[cfg(unix)] + { + let err = read_sandbox_proxy_ca_bundle("/dev/zero") + .await + .expect_err("a non-regular bundle path must fail closed"); + assert_eq!(err.code(), Code::InvalidArgument); + assert!(err.message().contains("not a regular file"), "{err}"); + } + + // Oversized regular file: rejected on the stat'd length, again + // without reading it whole. + let oversized = dir.join("oversized.pem"); + let bound = openshell_core::driver_utils::MAX_UPSTREAM_PROXY_CA_BUNDLE_BYTES; + std::fs::write(&oversized, vec![b'x'; usize::try_from(bound).unwrap() + 1]).unwrap(); + let err = read_sandbox_proxy_ca_bundle(oversized.to_str().unwrap()) + .await + .expect_err("an oversized bundle must fail closed"); + assert_eq!(err.code(), Code::InvalidArgument); + assert!(err.message().contains("exceeds"), "{err}"); + + std::fs::remove_dir_all(&dir).unwrap(); + } + + #[test] + fn qemu_backend_rejects_a_gateway_host_proxy() { + // gvproxy's host-loopback NAT has no QEMU/TAP equivalent, so a proxy + // on the gateway host is unreachable from a GPU sandbox and must be + // rejected rather than time out on every CONNECT. The address that + // reaches the gateway host from a QEMU guest is this sandbox's own + // TAP host address, so the classifier is parameterized by it. + let tap_host = Some("10.0.128.1"); + for url in [ + "http://host.openshell.internal:8080", + "http://host.containers.internal:8080", + "http://host.docker.internal:8080", + "http://127.0.0.1:8080", + "http://localhost:8080", + "https://[::1]:8080", + // The address the aliases above resolve to inside the guest. + "http://10.0.128.1:8080", + ] { + assert!(proxy_url_targets_gateway_host(url, tap_host), "{url}"); + } + for url in [ + "http://proxy.corp.example:8080", + "https://10.1.2.3:3128", + // Special only to libkrun/gvproxy. On QEMU/TAP it is an ordinary + // address that may be routable through the guest's masqueraded + // egress, so rejecting it would refuse a working configuration. + "http://192.168.127.254:8080", + // Another sandbox's TAP host, not this one's. + "http://10.0.128.5:8080", + "not a url", + ] { + assert!(!proxy_url_targets_gateway_host(url, tap_host), "{url}"); + } + + // Without an allocated TAP host only the address-independent + // spellings classify; the loopback and alias guards still hold. + assert!(proxy_url_targets_gateway_host( + "http://127.0.0.1:8080", + None + )); + assert!(proxy_url_targets_gateway_host( + "http://host.openshell.internal:8080", + None + )); + assert!(!proxy_url_targets_gateway_host( + "http://10.0.128.1:8080", + None + )); + } + + #[test] + fn qemu_launch_plan_rejects_a_proxy_at_the_allocated_tap_host() { + // The preflight has to run against the address this sandbox actually + // got, which only exists once the launch plan's subnet is allocated. + // A proxy there is what `host.openshell.internal` resolves to in the + // guest, and the driver's own nftables input chain drops the port. + let probe = test_driver_with_extensions(LifecycleExtensionRegistry::new()); + let tap_host = probe + .build_vm_launch_plan("sandbox-proxy-tap", true, true, None) + .expect("gpu plan should build") + .host_ip + .expect("a QEMU plan carries a TAP host address"); + probe.release_subnet("sandbox-proxy-tap"); + + let driver = test_driver_with_proxy(&format!("http://{tap_host}:8080")); + let mut plan = driver + .build_vm_launch_plan("sandbox-proxy-tap", true, true, None) + .expect("gpu plan should build"); + assert_eq!(plan.host_ip.as_deref(), Some(tap_host.as_str())); + + let err = driver + .resolve_launch_plan_backend("sandbox-proxy-tap", true, None, &mut plan) + .expect_err("a proxy at the TAP host address is unreachable from the guest"); + assert_eq!(err.code(), Code::FailedPrecondition); + assert!(err.message().contains(&tap_host), "{err}"); + + driver.release_subnet("sandbox-proxy-tap"); + } + + #[test] + fn qemu_launch_plan_allows_a_proxy_at_the_gvproxy_host_loopback_address() { + // 192.168.127.254 carries no meaning on QEMU/TAP, so a launch must + // proceed rather than be refused for a libkrun-only reason. + let driver = test_driver_with_proxy(&format!("http://{GVPROXY_HOST_LOOPBACK_IP}:8080")); + let mut plan = driver + .build_vm_launch_plan("sandbox-proxy-gvproxy", true, true, None) + .expect("gpu plan should build"); + assert_ne!(plan.host_ip.as_deref(), Some(GVPROXY_HOST_LOOPBACK_IP)); + + driver + .resolve_launch_plan_backend("sandbox-proxy-gvproxy", true, None, &mut plan) + .expect("a routable proxy address must not block a GPU launch"); + assert_eq!(plan.backend, VmBackend::Qemu); + + driver.release_subnet("sandbox-proxy-gvproxy"); + } + + #[tokio::test] + async fn proxy_credential_is_validated_against_the_supervisor_rules() { + let dir = std::env::temp_dir().join(format!("openshell-vm-cred-{}", std::process::id())); + std::fs::create_dir_all(&dir).unwrap(); + let path = dir.join("proxy-auth"); + + std::fs::write(&path, "proxyuser:proxypass\n").unwrap(); + assert_eq!( + read_sandbox_proxy_credential(path.to_str().unwrap()) + .await + .expect("a well-formed credential is accepted"), + "proxyuser:proxypass" + ); + + // Rejected here rather than inside every sandbox's supervisor. + std::fs::write(&path, "no-separator\n").unwrap(); + let err = read_sandbox_proxy_credential(path.to_str().unwrap()) + .await + .expect_err("a malformed credential must fail closed"); + assert_eq!(err.code(), Code::InvalidArgument); + assert!( + !err.message().contains("no-separator"), + "the error must not echo credential file contents: {err}" + ); + + std::fs::remove_dir_all(&dir).unwrap(); + } + + #[test] + fn guest_environment_carries_no_corporate_proxy_settings() { + // The egress boundary is argv-only: `build_guest_environment` merges + // user-supplied environment, so anything it emitted here would be + // attacker-influenced. + let config = proxy_config( + Some("http://proxy.corp.test:3128"), + Some("/etc/openshell/secrets/proxy-auth"), + Some("/etc/openshell/tls/corp-ca.pem"), + ); + let sandbox = Sandbox { + id: "sb-proxy".to_string(), + name: "proxy".to_string(), + spec: Some(SandboxSpec { + environment: [ + ( + "HTTPS_PROXY".to_string(), + "http://attacker:3128".to_string(), + ), + ("NO_PROXY".to_string(), "*".to_string()), + ] + .into_iter() + .collect(), + ..Default::default() + }), + ..Default::default() + }; + + let env = build_guest_environment(&sandbox, &config, None); + assert!( + !env.iter().any(|entry| entry.starts_with("--upstream")), + "driver environment must never carry supervisor arguments: {env:?}" + ); + // A sandbox may still set the conventional variables for its own + // workload, but the supervisor ignores them on this path -- what + // matters is that the driver never derives the boundary from them. + assert!( + !env.iter() + .any(|entry| entry.contains("proxy.corp.test") || entry.contains("proxy-auth")), + "operator proxy settings must not reach the guest environment: {env:?}" + ); + } + + #[test] + fn sandbox_driver_config_cannot_carry_proxy_settings() { + // The upstream proxy is host network topology, not a per-sandbox + // setting: the caller-supplied envelope must reject it outright + // rather than silently ignoring it. + for key in [ + "https_proxy", + "no_proxy", + "proxy_auth_file", + "proxy_auth_allow_insecure", + "proxy_connect_by_hostname", + "proxy_ca_bundle", + ] { + let template = SandboxTemplate { + driver_config: Some(Struct { + fields: std::iter::once(( + key.to_string(), + Value { + kind: Some(Kind::StringValue("http://attacker:3128".to_string())), + }, + )) + .collect(), + }), + ..Default::default() + }; + assert!( + VmSandboxDriverConfig::from_template(&template).is_err(), + "template.driver_config.vm must reject '{key}'" + ); + } + } } diff --git a/crates/openshell-driver-vm/src/main.rs b/crates/openshell-driver-vm/src/main.rs index 95ebf0f8b2..2546cb2606 100644 --- a/crates/openshell-driver-vm/src/main.rs +++ b/crates/openshell-driver-vm/src/main.rs @@ -146,6 +146,30 @@ struct Args { #[arg(long, env = "OPENSHELL_VM_SANDBOX_GID")] sandbox_gid: Option, + // Corporate forward proxy for sandbox egress. Operator-owned: these reach + // the guest supervisor on its argv, which the sandbox image and the + // user-supplied environment cannot influence. + #[arg(long, env = "OPENSHELL_VM_HTTPS_PROXY")] + https_proxy: Option, + + #[arg(long, env = "OPENSHELL_VM_NO_PROXY")] + no_proxy: Option, + + #[arg(long, env = "OPENSHELL_VM_PROXY_AUTH_FILE")] + proxy_auth_file: Option, + + // Value-taking rather than a presence flag so an explicit `false` in + // `[openshell.drivers.vm]` survives the gateway -> driver hop and still + // trips the "acknowledgement without a credential" check. + #[arg(long, env = "OPENSHELL_VM_PROXY_AUTH_ALLOW_INSECURE")] + proxy_auth_allow_insecure: Option, + + #[arg(long, env = "OPENSHELL_VM_PROXY_CONNECT_BY_HOSTNAME")] + proxy_connect_by_hostname: Option, + + #[arg(long, env = "OPENSHELL_VM_PROXY_CA_BUNDLE")] + proxy_ca_bundle: Option, + #[arg(long, hide = true)] vm_backend: Option, @@ -243,6 +267,12 @@ async fn main() -> Result<()> { gpu_vcpus: args.gpu_vcpus, sandbox_uid: args.sandbox_uid, sandbox_gid: args.sandbox_gid, + https_proxy: args.https_proxy.clone(), + no_proxy: args.no_proxy.clone(), + proxy_auth_file: args.proxy_auth_file.clone(), + proxy_auth_allow_insecure: args.proxy_auth_allow_insecure, + proxy_connect_by_hostname: args.proxy_connect_by_hostname, + proxy_ca_bundle: args.proxy_ca_bundle.clone(), }) .await .map_err(|err| miette::miette!("{err}"))?; @@ -620,6 +650,63 @@ mod tests { use clap::Parser; use std::path::PathBuf; + #[test] + fn corporate_proxy_flags_parse_into_driver_settings() { + let args = Args::parse_from([ + "openshell-driver-vm", + "--openshell-endpoint", + "https://host.openshell.internal:17670", + "--https-proxy", + "http://proxy.corp.com:8080", + "--no-proxy", + "10.0.0.0/8,.svc.cluster.local", + "--proxy-auth-file", + "/etc/openshell/secrets/proxy-auth", + "--proxy-auth-allow-insecure", + "true", + "--proxy-connect-by-hostname", + "false", + "--proxy-ca-bundle", + "/etc/openshell/tls/proxy-ca.pem", + ]); + + assert_eq!( + args.https_proxy.as_deref(), + Some("http://proxy.corp.com:8080") + ); + assert_eq!( + args.no_proxy.as_deref(), + Some("10.0.0.0/8,.svc.cluster.local") + ); + assert_eq!( + args.proxy_auth_file.as_deref(), + Some("/etc/openshell/secrets/proxy-auth") + ); + assert_eq!(args.proxy_auth_allow_insecure, Some(true)); + // Value-taking rather than a presence flag, so the gateway can + // forward an explicit `false` from `[openshell.drivers.vm]`. + assert_eq!(args.proxy_connect_by_hostname, Some(false)); + assert_eq!( + args.proxy_ca_bundle.as_deref(), + Some("/etc/openshell/tls/proxy-ca.pem") + ); + } + + #[test] + fn corporate_proxy_settings_default_to_unset() { + let args = Args::parse_from([ + "openshell-driver-vm", + "--openshell-endpoint", + "https://host.openshell.internal:17670", + ]); + assert!(args.https_proxy.is_none()); + assert!(args.no_proxy.is_none()); + assert!(args.proxy_auth_file.is_none()); + assert!(args.proxy_auth_allow_insecure.is_none()); + assert!(args.proxy_connect_by_hostname.is_none()); + assert!(args.proxy_ca_bundle.is_none()); + } + #[test] fn peer_authorization_accepts_matching_uid_and_pid() { authorize_peer_credentials( diff --git a/crates/openshell-gateway-interceptors/src/proto_json.rs b/crates/openshell-gateway-interceptors/src/proto_json.rs index b0d6b0f53d..d7ea5a241c 100644 --- a/crates/openshell-gateway-interceptors/src/proto_json.rs +++ b/crates/openshell-gateway-interceptors/src/proto_json.rs @@ -317,6 +317,7 @@ mod tests { annotations: HashMap::new(), workspace: String::new(), await_main_process_attachment: false, + workload_template_name: String::new(), }; let bytes = request.encode_to_vec(); let json = codec diff --git a/crates/openshell-gateway-interceptors/src/runtime.rs b/crates/openshell-gateway-interceptors/src/runtime.rs index 02afddab75..cd0f59ea3c 100644 --- a/crates/openshell-gateway-interceptors/src/runtime.rs +++ b/crates/openshell-gateway-interceptors/src/runtime.rs @@ -1076,6 +1076,7 @@ mod tests { annotations: HashMap::new(), workspace: String::new(), await_main_process_attachment: false, + workload_template_name: String::new(), }; let bytes = request.encode_to_vec(); diff --git a/crates/openshell-gateway/src/vm.rs b/crates/openshell-gateway/src/vm.rs index e86de28c12..f52bd9ddfa 100644 --- a/crates/openshell-gateway/src/vm.rs +++ b/crates/openshell-gateway/src/vm.rs @@ -99,6 +99,34 @@ pub struct VmComputeConfig { /// Host-side private key for the guest's mTLS client bundle. pub guest_tls_key: Option, + + /// Corporate forward proxy URL (`http://host:port` or `https://host:port`) + /// for policy-approved TLS egress from VM sandboxes. + /// + /// Deployment-level configuration, not a per-sandbox setting: it is passed + /// to the driver, which puts it on the guest supervisor's argv. A proxy on + /// this host's loopback is reachable from a guest only through the gvproxy + /// host alias `host.openshell.internal`. + pub https_proxy: Option, + + /// Comma-separated `NO_PROXY` list. Bypasses only the corporate proxy, + /// never `OpenShell` policy evaluation. + pub no_proxy: Option, + + /// Path on this host to a `user:pass` corporate proxy credential file. + pub proxy_auth_file: Option, + + /// Acknowledgement that Basic auth to an `http://` proxy is cleartext. + /// Required alongside `proxy_auth_file` unless the proxy is `https://`. + pub proxy_auth_allow_insecure: Option, + + /// Send hostnames rather than validated IPs in CONNECT requests. Last + /// resort for proxies whose ACLs reject IP CONNECT targets. + pub proxy_connect_by_hostname: Option, + + /// Path on this host to a PEM CA bundle trusted for the corporate proxy + /// and for server certificates a TLS-intercepting proxy re-signs. + pub proxy_ca_bundle: Option, } impl VmComputeConfig { @@ -135,6 +163,29 @@ impl VmComputeConfig { 4096 } + /// Validate the corporate upstream-proxy settings, fail-closed. + /// + /// Runs in the gateway as well as in the driver so an invalid + /// `[openshell.drivers.vm]` table reports the offending key instead of + /// surfacing as an opaque driver-startup timeout. + /// + /// # Errors + /// + /// Returns a [`Error::config`] naming the offending key. + pub fn validate_proxy_config(&self) -> Result<()> { + openshell_core::driver_utils::validate_upstream_proxy_settings( + &openshell_core::driver_utils::UpstreamProxySettings { + url: self.https_proxy.as_deref(), + no_proxy: self.no_proxy.as_deref(), + auth_file: self.proxy_auth_file.as_deref(), + auth_allow_insecure: self.proxy_auth_allow_insecure, + connect_by_hostname: self.proxy_connect_by_hostname, + ca_bundle: self.proxy_ca_bundle.as_deref(), + }, + ) + .map_err(Error::config) + } + #[must_use] fn default_driver_search_dirs(home: Option) -> Vec { let mut dirs = Vec::new(); @@ -163,6 +214,12 @@ impl Default for VmComputeConfig { guest_tls_ca: None, guest_tls_cert: None, guest_tls_key: None, + https_proxy: None, + no_proxy: None, + proxy_auth_file: None, + proxy_auth_allow_insecure: None, + proxy_connect_by_hostname: None, + proxy_ca_bundle: None, } } } @@ -460,6 +517,8 @@ pub async fn spawn( )); } + vm_config.validate_proxy_config()?; + let driver_bin = resolve_compute_driver_bin(vm_config)?; let socket_path = compute_driver_socket_path(vm_config); let guest_tls_paths = compute_driver_guest_tls_paths(vm_config)?; @@ -501,6 +560,7 @@ pub async fn spawn( command.arg("--guest-tls-cert").arg(tls.cert); command.arg("--guest-tls-key").arg(tls.key); } + append_upstream_proxy_args(&mut command, vm_config); let mut child = command.spawn().map_err(|e| { Error::execution(format!( @@ -515,6 +575,38 @@ pub async fn spawn( )) } +/// Forward the operator's corporate proxy settings to the driver subprocess. +/// +/// Only keys the operator actually set are passed, so the driver keeps the +/// same "omitted means no proxy" contract the supervisor enforces. The +/// booleans travel as explicit values rather than presence flags so an +/// explicit `false` still trips the driver's pairing checks. +#[cfg(unix)] +fn append_upstream_proxy_args(command: &mut Command, vm_config: &VmComputeConfig) { + if let Some(url) = &vm_config.https_proxy { + command.arg("--https-proxy").arg(url); + } + if let Some(list) = &vm_config.no_proxy { + command.arg("--no-proxy").arg(list); + } + if let Some(path) = &vm_config.proxy_auth_file { + command.arg("--proxy-auth-file").arg(path); + } + if let Some(allow) = vm_config.proxy_auth_allow_insecure { + command + .arg("--proxy-auth-allow-insecure") + .arg(allow.to_string()); + } + if let Some(by_hostname) = vm_config.proxy_connect_by_hostname { + command + .arg("--proxy-connect-by-hostname") + .arg(by_hostname.to_string()); + } + if let Some(path) = &vm_config.proxy_ca_bundle { + command.arg("--proxy-ca-bundle").arg(path); + } +} + #[cfg(unix)] fn append_otlp_args(command: &mut Command, otlp_config: Option<&OtlpConfig>, gateway_name: &str) { if let Some(config) = otlp_config { @@ -606,9 +698,10 @@ async fn connect_compute_driver(socket_path: &Path) -> Result { #[cfg(all(test, unix))] mod tests { use super::{ - VmComputeConfig, append_otlp_args, compute_driver_guest_tls_paths, - compute_driver_socket_path, current_euid, prepare_compute_driver_socket_path, - prepare_vm_state_dir, resolve_compute_driver_bin, resolve_driver_search_dirs, + VmComputeConfig, append_otlp_args, append_upstream_proxy_args, + compute_driver_guest_tls_paths, compute_driver_socket_path, current_euid, + prepare_compute_driver_socket_path, prepare_vm_state_dir, resolve_compute_driver_bin, + resolve_driver_search_dirs, }; use openshell_server::config_file::OtlpConfig; use std::os::unix::fs::PermissionsExt; @@ -644,6 +737,83 @@ mod tests { ); } + #[test] + fn vm_driver_command_forwards_corporate_proxy_settings() { + let mut command = tokio::process::Command::new("openshell-driver-vm"); + append_upstream_proxy_args( + &mut command, + &VmComputeConfig { + https_proxy: Some("http://proxy.corp.com:8080".to_string()), + no_proxy: Some("10.0.0.0/8".to_string()), + proxy_auth_file: Some("/etc/openshell/secrets/proxy-auth".to_string()), + proxy_auth_allow_insecure: Some(true), + proxy_connect_by_hostname: Some(false), + proxy_ca_bundle: Some("/etc/openshell/tls/proxy-ca.pem".to_string()), + ..VmComputeConfig::default() + }, + ); + + let args = command + .as_std() + .get_args() + .map(|arg| arg.to_string_lossy().into_owned()) + .collect::>(); + assert_eq!( + args, + [ + "--https-proxy", + "http://proxy.corp.com:8080", + "--no-proxy", + "10.0.0.0/8", + "--proxy-auth-file", + "/etc/openshell/secrets/proxy-auth", + "--proxy-auth-allow-insecure", + "true", + // Passed as an explicit value, not a presence flag, so the + // driver still sees the operator's `false`. + "--proxy-connect-by-hostname", + "false", + "--proxy-ca-bundle", + "/etc/openshell/tls/proxy-ca.pem", + ] + ); + } + + #[test] + fn vm_driver_command_omits_unset_corporate_proxy_settings() { + let mut command = tokio::process::Command::new("openshell-driver-vm"); + append_upstream_proxy_args(&mut command, &VmComputeConfig::default()); + assert_eq!(command.as_std().get_args().count(), 0); + } + + #[test] + fn invalid_corporate_proxy_config_is_rejected_before_the_driver_starts() { + // Without this the operator would see an opaque driver-readiness + // timeout instead of an error naming the offending key. + let err = VmComputeConfig { + https_proxy: Some("socks5://proxy.corp.com:1080".to_string()), + ..VmComputeConfig::default() + } + .validate_proxy_config() + .expect_err("only http:// and https:// proxies are supported"); + assert!(err.to_string().contains("https_proxy"), "{err}"); + + let err = VmComputeConfig { + proxy_ca_bundle: Some("/etc/openshell/tls/proxy-ca.pem".to_string()), + ..VmComputeConfig::default() + } + .validate_proxy_config() + .expect_err("a CA bundle without a proxy URL would hide a fail-open state"); + assert!(err.to_string().contains("proxy_ca_bundle"), "{err}"); + + VmComputeConfig { + https_proxy: Some("http://proxy.corp.com:8080".to_string()), + ..VmComputeConfig::default() + } + .validate_proxy_config() + .expect("a lone proxy URL is a complete configuration"); + } + #[test] fn resolve_driver_bin_uses_driver_dir_when_binary_present() { let dir = tempdir().unwrap(); diff --git a/crates/openshell-sdk/README.md b/crates/openshell-sdk/README.md index 93afc5359e..cb42e12dc1 100644 --- a/crates/openshell-sdk/README.md +++ b/crates/openshell-sdk/README.md @@ -9,7 +9,8 @@ gateway-name resolution. ## Two layers - `OpenShellClient` — the curated, sandbox-focused surface: health, sandbox - CRUD, readiness/deletion waits, and non-streaming exec. + CRUD, reusable sandbox template CRUD, readiness/deletion waits, and + non-streaming exec. - `raw` — direct access to the generated tonic clients for RPCs the curated surface doesn't yet cover (inference, providers, policy, logs, settings, SSH, forwarding). @@ -44,10 +45,53 @@ mTLS (client certificates) is not supported. `OpenShellClient::connect(ClientConfig)` returns a connected client exposing `health`, `create_sandbox`, `get_sandbox`, `list_sandboxes`, `delete_sandbox`, +`create_sandbox_from_template`, `create_sandbox_template`, +`get_sandbox_template`, `list_sandbox_templates`, `delete_sandbox_template`, `wait_ready`, `wait_deleted`, and `exec`. Curated types (`SandboxSpec`, -`SandboxRef`, `Health`, `ListOptions`, `ExecOptions`, `SandboxPhase`) use -SDK-shaped enums rather than raw proto integers. Failures map to a typed -`SdkError` with a discriminable kind. +`SandboxRef`, `Health`, `ListOptions`, `SandboxTemplateListOptions`, +`ExecOptions`, `SandboxPhase`) use SDK-shaped enums rather than raw proto +integers where practical. Reusable template resources are exposed as +`SandboxWorkloadTemplate` proto aliases so callers can populate the full +portable workload shape and driver config. Failures map to a typed `SdkError` +with a discriminable kind. + +```rust +use openshell_sdk::{ + ClientConfig, OpenShellClient, SandboxTemplateCreateSpec, + SandboxWorkloadConfig, SandboxWorkloadTemplate, SandboxWorkloadTemplateSpec, +}; + +# async fn run() -> Result<(), openshell_sdk::SdkError> { +let client = OpenShellClient::connect(ClientConfig::new("http://127.0.0.1:8080")).await?; +client + .create_sandbox_template(SandboxWorkloadTemplate { + metadata: Some(openshell_sdk::raw::proto::datamodel::v1::ObjectMeta { + name: "python".to_string(), + ..Default::default() + }), + spec: Some(SandboxWorkloadTemplateSpec { + workload: Some(SandboxWorkloadConfig { + image: "ghcr.io/nvidia/openshell-community/sandboxes/python:latest".to_string(), + ..Default::default() + }), + ..Default::default() + }), + }) + .await?; + +let _sandbox = client + .create_sandbox_from_template(SandboxTemplateCreateSpec { + template_name: "python".to_string(), + policy: Some(openshell_sdk::raw::proto::SandboxPolicy { + version: 1, + ..Default::default() + }), + ..Default::default() + }) + .await?; +# Ok(()) +# } +``` ## Modules diff --git a/crates/openshell-sdk/src/client.rs b/crates/openshell-sdk/src/client.rs index a02b5735ae..b5486812f7 100644 --- a/crates/openshell-sdk/src/client.rs +++ b/crates/openshell-sdk/src/client.rs @@ -16,7 +16,7 @@ use crate::refresh::{RefreshedToken, TokenSource}; use crate::transport; use crate::types::{ ExecOptions, ExecResult, Health, ListOptions, SandboxPhase, SandboxRef, SandboxSpec, - WorkspaceRef, + SandboxTemplateCreateSpec, SandboxTemplateListOptions, SandboxWorkloadTemplate, WorkspaceRef, }; use futures::StreamExt; use openshell_core::proto; @@ -163,6 +163,86 @@ impl OpenShellClient { sandbox_from_response(response.sandbox) } + /// Create a new sandbox from a workspace-scoped workload template name. + pub async fn create_sandbox_from_template( + &self, + spec: SandboxTemplateCreateSpec, + ) -> Result { + let request = create_sandbox_from_template_request(spec); + let response = self + .unary(|mut grpc| { + let request = request.clone(); + async move { grpc.create_sandbox(request).await } + }) + .await?; + sandbox_from_response(response.sandbox) + } + + /// Create a reusable sandbox template in the default workspace. + pub async fn create_sandbox_template( + &self, + template: SandboxWorkloadTemplate, + ) -> Result { + let response = self + .unary(|mut grpc| { + let request = proto::CreateSandboxTemplateRequest { + template: Some(template.clone()), + workspace: String::new(), + }; + async move { grpc.create_sandbox_template(request).await } + }) + .await?; + sandbox_template_from_response(response.template) + } + + /// Fetch a reusable sandbox template by name from the default workspace. + pub async fn get_sandbox_template(&self, name: &str) -> Result { + let response = self + .unary(|mut grpc| { + let request = proto::GetSandboxTemplateRequest { + name: name.to_string(), + workspace: String::new(), + }; + async move { grpc.get_sandbox_template(request).await } + }) + .await?; + sandbox_template_from_response(response.template) + } + + /// List reusable sandbox templates in the default workspace or across all workspaces. + pub async fn list_sandbox_templates( + &self, + opts: SandboxTemplateListOptions, + ) -> Result> { + let response = self + .unary(|mut grpc| { + let request = proto::ListSandboxTemplatesRequest { + limit: opts.limit, + offset: opts.offset, + workspace: String::new(), + all_workspaces: opts.all_workspaces, + label_selector: opts.label_selector.clone(), + }; + async move { grpc.list_sandbox_templates(request).await } + }) + .await?; + Ok(response.templates) + } + + /// Delete a reusable sandbox template by name from the default workspace. + pub async fn delete_sandbox_template(&self, name: &str) -> Result { + let response = self + .unary(|mut grpc| { + let request = proto::DeleteSandboxTemplateRequest { + name: name.to_string(), + workspace: String::new(), + }; + async move { grpc.delete_sandbox_template(request).await } + }) + .await?; + Ok(response.deleted) + } + /// Fetch a sandbox by name. pub async fn get_sandbox(&self, name: &str) -> Result { let response = self @@ -567,6 +647,96 @@ impl WorkspaceScopedClient { sandbox_from_response(response.sandbox) } + /// Create a new sandbox from a template in this workspace. + pub async fn create_sandbox_from_template( + &self, + spec: SandboxTemplateCreateSpec, + ) -> Result { + let mut request = create_sandbox_from_template_request(spec); + request.workspace = self.workspace.clone(); + let response = self + .client + .unary(|mut grpc| { + let request = request.clone(); + async move { grpc.create_sandbox(request).await } + }) + .await?; + sandbox_from_response(response.sandbox) + } + + /// Create a reusable sandbox template in this workspace. + pub async fn create_sandbox_template( + &self, + template: SandboxWorkloadTemplate, + ) -> Result { + let response = self + .client + .unary(|mut grpc| { + let request = proto::CreateSandboxTemplateRequest { + template: Some(template.clone()), + workspace: self.workspace.clone(), + }; + async move { grpc.create_sandbox_template(request).await } + }) + .await?; + sandbox_template_from_response(response.template) + } + + /// Fetch a reusable sandbox template by name in this workspace. + pub async fn get_sandbox_template(&self, name: &str) -> Result { + let response = self + .client + .unary(|mut grpc| { + let request = proto::GetSandboxTemplateRequest { + name: name.to_string(), + workspace: self.workspace.clone(), + }; + async move { grpc.get_sandbox_template(request).await } + }) + .await?; + sandbox_template_from_response(response.template) + } + + /// List reusable sandbox templates in this workspace, or across all workspaces. + pub async fn list_sandbox_templates( + &self, + opts: SandboxTemplateListOptions, + ) -> Result> { + let response = self + .client + .unary(|mut grpc| { + let request = proto::ListSandboxTemplatesRequest { + limit: opts.limit, + offset: opts.offset, + workspace: if opts.all_workspaces { + String::new() + } else { + self.workspace.clone() + }, + all_workspaces: opts.all_workspaces, + label_selector: opts.label_selector.clone(), + }; + async move { grpc.list_sandbox_templates(request).await } + }) + .await?; + Ok(response.templates) + } + + /// Delete a reusable sandbox template by name in this workspace. + pub async fn delete_sandbox_template(&self, name: &str) -> Result { + let response = self + .client + .unary(|mut grpc| { + let request = proto::DeleteSandboxTemplateRequest { + name: name.to_string(), + workspace: self.workspace.clone(), + }; + async move { grpc.delete_sandbox_template(request).await } + }) + .await?; + Ok(response.deleted) + } + /// Fetch a sandbox by name in this workspace. pub async fn get_sandbox(&self, name: &str) -> Result { let response = self @@ -837,6 +1007,36 @@ fn create_sandbox_request(spec: SandboxSpec) -> proto::CreateSandboxRequest { annotations: HashMap::new(), workspace: String::new(), await_main_process_attachment: false, + workload_template_name: String::new(), + } +} + +fn create_sandbox_from_template_request( + spec: SandboxTemplateCreateSpec, +) -> proto::CreateSandboxRequest { + let SandboxTemplateCreateSpec { + name, + template_name, + labels, + providers, + command, + tty, + policy, + } = spec; + proto::CreateSandboxRequest { + spec: Some(proto::SandboxSpec { + providers, + command, + tty, + policy, + ..proto::SandboxSpec::default() + }), + name: name.unwrap_or_default(), + labels, + annotations: HashMap::new(), + workspace: String::new(), + workload_template_name: template_name, + await_main_process_attachment: false, } } @@ -846,6 +1046,13 @@ fn sandbox_from_response(sandbox: Option) -> Result .ok_or_else(|| SdkError::invalid_config("sandbox missing from gateway response")) } +fn sandbox_template_from_response( + template: Option, +) -> Result { + template + .ok_or_else(|| SdkError::invalid_config("sandbox template missing from gateway response")) +} + fn map_status(status: tonic::Status) -> SdkError { let message = status.message().to_string(); match status.code() { diff --git a/crates/openshell-sdk/src/lib.rs b/crates/openshell-sdk/src/lib.rs index dbf2524a2a..985c7ecc05 100644 --- a/crates/openshell-sdk/src/lib.rs +++ b/crates/openshell-sdk/src/lib.rs @@ -6,7 +6,8 @@ //! Two layers: //! //! - [`OpenShellClient`] — the high-level sandbox-focused MVP surface: -//! health, sandbox CRUD, readiness/deletion waits, non-streaming exec. +//! health, sandbox CRUD, reusable sandbox templates, readiness/deletion +//! waits, and non-streaming exec. //! - [`raw`] — direct access to the generated tonic clients for RPCs the //! curated surface doesn't yet cover (inference, providers, policy, logs, //! settings, SSH, forwarding). @@ -46,6 +47,8 @@ pub use config::{AuthConfig, ClientConfig}; pub use error::SdkError; pub use refresh::{Refresh, RefreshError, RefreshedToken, TokenSource}; pub use types::{ - ExecOptions, ExecResult, Health, ListOptions, SandboxPhase, SandboxRef, SandboxSpec, - ServiceStatus, WorkspaceRef, + ExecOptions, ExecResult, Health, ListOptions, SandboxPhase, SandboxRef, SandboxResources, + SandboxServiceLevel, SandboxSpec, SandboxStartup, SandboxTemplateCreateSpec, + SandboxTemplateListOptions, SandboxWorkloadConfig, SandboxWorkloadTemplate, + SandboxWorkloadTemplateProvenance, SandboxWorkloadTemplateSpec, ServiceStatus, WorkspaceRef, }; diff --git a/crates/openshell-sdk/src/raw.rs b/crates/openshell-sdk/src/raw.rs index e974b19259..35d91f3325 100644 --- a/crates/openshell-sdk/src/raw.rs +++ b/crates/openshell-sdk/src/raw.rs @@ -22,11 +22,15 @@ pub use openshell_core::proto; pub use openshell_core::proto::inference_client::InferenceClient; pub use openshell_core::proto::open_shell_client::OpenShellClient as GrpcClient; pub use openshell_core::proto::{ - CreateSandboxRequest, CreateWorkspaceRequest, DeleteSandboxRequest, DeleteWorkspaceRequest, - ExecSandboxRequest, GetSandboxRequest, GetWorkspaceRequest, HealthRequest, - ListProvidersRequest, ListSandboxesRequest, ListWorkspacesRequest, Sandbox, - SandboxPhase as ProtoSandboxPhase, SandboxSpec as ProtoSandboxSpec, SandboxTemplate, - ServiceStatus as ProtoServiceStatus, StartSandboxRequest, StopSandboxRequest, Workspace, + CreateSandboxRequest, CreateSandboxTemplateRequest, CreateWorkspaceRequest, + DeleteSandboxRequest, DeleteSandboxTemplateRequest, DeleteWorkspaceRequest, ExecSandboxRequest, + GetSandboxRequest, GetSandboxTemplateRequest, GetWorkspaceRequest, HealthRequest, + ListProvidersRequest, ListSandboxTemplatesRequest, ListSandboxesRequest, ListWorkspacesRequest, + Sandbox, SandboxPhase as ProtoSandboxPhase, SandboxResources, SandboxServiceLevel, + SandboxSpec as ProtoSandboxSpec, SandboxStartup, SandboxTemplate, SandboxTemplateResponse, + SandboxWorkloadConfig, SandboxWorkloadTemplate, SandboxWorkloadTemplateProvenance, + SandboxWorkloadTemplateSpec, ServiceStatus as ProtoServiceStatus, StartSandboxRequest, + StopSandboxRequest, Workspace, }; /// Type alias for the gRPC client wrapped in the SDK's auth interceptor. diff --git a/crates/openshell-sdk/src/types.rs b/crates/openshell-sdk/src/types.rs index 9cb7171fd4..db2944474b 100644 --- a/crates/openshell-sdk/src/types.rs +++ b/crates/openshell-sdk/src/types.rs @@ -117,6 +117,60 @@ pub struct SandboxSpec { pub tty: bool, } +/// Caller intent for creating a sandbox from a named workload template. +#[derive(Clone, Debug, Default)] +pub struct SandboxTemplateCreateSpec { + /// Optional user-supplied sandbox name. When empty the server generates one. + pub name: Option, + /// Workspace-scoped template name to resolve at creation time. + pub template_name: String, + /// Labels attached to the sandbox. + pub labels: HashMap, + /// Provider names to attach. + pub providers: Vec, + /// Exact canonical command. Empty selects the gateway's scratch login shell. + pub command: Vec, + /// Allocate a retained pseudo-terminal for the canonical command. + pub tty: bool, + /// Create-time sandbox policy. The named workload template supplies runtime + /// workload fields; policy remains part of the sandbox's governance spec. + pub policy: Option, +} + +/// Reusable sandbox workload template resource. +/// +/// This is a raw proto alias because template specs intentionally expose the +/// full portable workload shape plus driver-owned config. +pub type SandboxWorkloadTemplate = proto::SandboxWorkloadTemplate; + +/// Desired reusable workload shape for a [`SandboxWorkloadTemplate`]. +pub type SandboxWorkloadTemplateSpec = proto::SandboxWorkloadTemplateSpec; + +/// Portable sandbox workload configuration for template-backed sandboxes. +pub type SandboxWorkloadConfig = proto::SandboxWorkloadConfig; + +/// Portable resource requirements for template-backed sandboxes. +pub type SandboxResources = proto::SandboxResources; + +/// Desired service level for sandboxes created from a template. +pub type SandboxServiceLevel = proto::SandboxServiceLevel; + +/// Startup service-level settings for template-backed sandboxes. +pub type SandboxStartup = proto::SandboxStartup; + +/// Options for listing reusable sandbox templates. +#[derive(Clone, Debug, Default)] +pub struct SandboxTemplateListOptions { + /// Maximum templates to return. `0` defers to the server default. + pub limit: u32, + /// Offset into the result list. + pub offset: u32, + /// Optional label selector in `key=value,key2=value2` form. + pub label_selector: String, + /// List templates across all workspaces. + pub all_workspaces: bool, +} + /// Reference to a sandbox owned by the gateway. #[derive(Clone, Debug)] #[non_exhaustive] @@ -128,12 +182,28 @@ pub struct SandboxRef { pub labels: HashMap, pub resource_version: u64, pub exit_code: Option, + pub created_from_workload_template: Option, +} + +/// Reusable workload template revision used to create a sandbox. +#[derive(Clone, Debug, Eq, PartialEq)] +#[non_exhaustive] +pub struct SandboxWorkloadTemplateProvenance { + pub name: String, + pub resource_version: String, } impl SandboxRef { pub(crate) fn from_proto(sandbox: proto::Sandbox) -> Self { let phase = sandbox.phase().into(); let exit_code = sandbox.status.as_ref().and_then(|status| status.exit_code); + let created_from_workload_template = + sandbox + .created_from_workload_template + .map(|p| SandboxWorkloadTemplateProvenance { + name: p.name, + resource_version: p.resource_version, + }); let meta = sandbox.metadata.unwrap_or_default(); Self { id: meta.id, @@ -143,6 +213,7 @@ impl SandboxRef { labels: meta.labels, resource_version: meta.resource_version, exit_code, + created_from_workload_template, } } } diff --git a/crates/openshell-sdk/tests/client_mock.rs b/crates/openshell-sdk/tests/client_mock.rs index 1cdac7da41..58633ceb17 100644 --- a/crates/openshell-sdk/tests/client_mock.rs +++ b/crates/openshell-sdk/tests/client_mock.rs @@ -12,7 +12,8 @@ use openshell_core::proto; use openshell_core::proto::open_shell_server::{OpenShell, OpenShellServer}; use openshell_sdk::{ AuthConfig, ClientConfig, ExecOptions, ListOptions, OpenShellClient, Refresh, RefreshError, - RefreshedToken, SandboxPhase, SandboxSpec, ServiceStatus as SdkServiceStatus, + RefreshedToken, SandboxPhase, SandboxSpec, SandboxTemplateCreateSpec, + SandboxTemplateListOptions, ServiceStatus as SdkServiceStatus, }; use std::collections::HashMap; use std::sync::Arc; @@ -29,6 +30,10 @@ struct MockState { last_get_name: Mutex>, last_get_workspace: Mutex>, last_create: Mutex>, + last_template_create: Mutex>, + last_template_get: Mutex>, + last_template_list: Mutex>, + last_template_delete: Mutex>, last_delete_name: Mutex>, last_delete_workspace: Mutex>, last_stop: Mutex>, @@ -61,6 +66,11 @@ fn sandbox_with_phase_ws( phase: proto::SandboxPhase, workspace: &str, ) -> proto::Sandbox { + let created_from_workload_template = + (name == "from-template").then(|| proto::SandboxWorkloadTemplateProvenance { + name: "python".to_string(), + resource_version: "7".to_string(), + }); proto::Sandbox { metadata: Some(proto::datamodel::v1::ObjectMeta { id: format!("id-{name}"), @@ -77,6 +87,7 @@ fn sandbox_with_phase_ws( phase: phase.into(), ..Default::default() }), + created_from_workload_template, } } @@ -98,6 +109,34 @@ fn workspace_proto(name: &str, phase: proto::datamodel::v1::WorkspacePhase) -> p } } +fn workload_template_proto(name: &str, workspace: &str) -> proto::SandboxWorkloadTemplate { + proto::SandboxWorkloadTemplate { + metadata: Some(proto::datamodel::v1::ObjectMeta { + id: format!("template-{workspace}-{name}"), + name: name.to_string(), + created_at_ms: 1_000_000, + labels: HashMap::new(), + annotations: HashMap::new(), + resource_version: 1, + deletion_timestamp_ms: 0, + workspace: workspace.to_string(), + }), + spec: Some(proto::SandboxWorkloadTemplateSpec { + workload: Some(proto::SandboxWorkloadConfig { + image: format!("ghcr.io/test/{name}:latest"), + environment: HashMap::new(), + resources: Some(proto::SandboxResources { + cpu: "1".to_string(), + memory: "512Mi".to_string(), + ..proto::SandboxResources::default() + }), + }), + driver_config: None, + desired_service_level: None, + }), + } +} + #[tonic::async_trait] impl OpenShell for TestOpenShell { async fn report_main_process_exit( @@ -177,6 +216,62 @@ impl OpenShell for TestOpenShell { })) } + async fn create_sandbox_template( + &self, + request: tonic::Request, + ) -> Result, Status> { + let request = request.into_inner(); + let template = request + .template + .clone() + .ok_or_else(|| Status::invalid_argument("missing template"))?; + *self.state.last_template_create.lock().await = Some(request); + Ok(Response::new(proto::SandboxTemplateResponse { + template: Some(template), + })) + } + + async fn get_sandbox_template( + &self, + request: tonic::Request, + ) -> Result, Status> { + let request = request.into_inner(); + let workspace = if request.workspace.is_empty() { + "default" + } else { + &request.workspace + }; + let template = workload_template_proto(&request.name, workspace); + *self.state.last_template_get.lock().await = Some(request); + Ok(Response::new(proto::SandboxTemplateResponse { + template: Some(template), + })) + } + + async fn list_sandbox_templates( + &self, + request: tonic::Request, + ) -> Result, Status> { + let request = request.into_inner(); + *self.state.last_template_list.lock().await = Some(request); + Ok(Response::new(proto::ListSandboxTemplatesResponse { + templates: vec![ + workload_template_proto("python", "default"), + workload_template_proto("cuda", "gpu"), + ], + })) + } + + async fn delete_sandbox_template( + &self, + request: tonic::Request, + ) -> Result, Status> { + *self.state.last_template_delete.lock().await = Some(request.into_inner()); + Ok(Response::new(proto::DeleteSandboxTemplateResponse { + deleted: true, + })) + } + async fn stop_sandbox( &self, request: tonic::Request, @@ -817,6 +912,92 @@ async fn create_sandbox_passes_spec_through() { ); } +#[tokio::test] +async fn create_sandbox_from_template_passes_template_name() { + let state = Arc::new(MockState::default()); + let endpoint = start_mock(state.clone()).await; + let client = connect(&endpoint).await; + + let sandbox = client + .create_sandbox_from_template(SandboxTemplateCreateSpec { + name: Some("from-template".to_string()), + template_name: "python".to_string(), + providers: vec!["openai".to_string()], + command: vec!["python".to_string(), "-m".to_string(), "agent".to_string()], + tty: false, + policy: Some(proto::SandboxPolicy { + version: 1, + ..Default::default() + }), + ..Default::default() + }) + .await + .unwrap(); + assert_eq!(sandbox.name, "from-template"); + + let observed = state.last_create.lock().await.clone().unwrap(); + assert_eq!(observed.name, "from-template"); + assert_eq!(observed.workload_template_name, "python"); + let observed_spec = observed.spec.unwrap(); + assert_eq!(observed_spec.providers, vec!["openai".to_string()]); + assert_eq!(observed_spec.command, vec!["python", "-m", "agent"]); + assert!(!observed_spec.tty); + assert_eq!(observed_spec.policy.unwrap().version, 1); +} + +#[tokio::test] +async fn sandbox_template_crud_uses_default_workspace() { + let state = Arc::new(MockState::default()); + let endpoint = start_mock(state.clone()).await; + let client = connect(&endpoint).await; + + let created = client + .create_sandbox_template(workload_template_proto("python", "")) + .await + .unwrap(); + assert_eq!(created.metadata.as_ref().unwrap().name, "python"); + + let observed_create = state.last_template_create.lock().await.clone().unwrap(); + assert!(observed_create.workspace.is_empty()); + assert_eq!( + observed_create + .template + .as_ref() + .and_then(|template| template.metadata.as_ref()) + .unwrap() + .name, + "python" + ); + + let fetched = client.get_sandbox_template("python").await.unwrap(); + assert_eq!(fetched.metadata.as_ref().unwrap().name, "python"); + let observed_get = state.last_template_get.lock().await.clone().unwrap(); + assert_eq!(observed_get.name, "python"); + assert!(observed_get.workspace.is_empty()); + + let listed = client + .list_sandbox_templates(SandboxTemplateListOptions { + limit: 10, + offset: 2, + label_selector: String::new(), + all_workspaces: true, + }) + .await + .unwrap(); + assert_eq!(listed.len(), 2); + let observed_list = state.last_template_list.lock().await.clone().unwrap(); + assert_eq!(observed_list.limit, 10); + assert_eq!(observed_list.offset, 2); + assert!(observed_list.workspace.is_empty()); + assert!(observed_list.all_workspaces); + + let deleted = client.delete_sandbox_template("python").await.unwrap(); + assert!(deleted); + let observed_delete = state.last_template_delete.lock().await.clone().unwrap(); + assert_eq!(observed_delete.name, "python"); + assert!(observed_delete.workspace.is_empty()); +} + #[tokio::test] async fn get_sandbox_sends_name_and_maps_phase() { let state = Arc::new(MockState { @@ -835,6 +1016,21 @@ async fn get_sandbox_sends_name_and_maps_phase() { assert_eq!(observed.as_deref(), Some("my-box")); } +#[tokio::test] +async fn get_sandbox_preserves_workload_template_provenance() { + let state = Arc::new(MockState::default()); + let endpoint = start_mock(state.clone()).await; + let client = connect(&endpoint).await; + + let sandbox = client.get_sandbox("from-template").await.unwrap(); + + let provenance = sandbox + .created_from_workload_template + .expect("template provenance"); + assert_eq!(provenance.name, "python"); + assert_eq!(provenance.resource_version, "7"); +} + #[tokio::test] async fn list_sandboxes_propagates_filters() { let state = Arc::new(MockState::default()); @@ -1137,6 +1333,33 @@ async fn workspace_scoped_create_passes_workspace() { assert_eq!(observed.workspace, "staging"); } +#[tokio::test] +async fn workspace_scoped_create_from_template_passes_workspace() { + let state = Arc::new(MockState::default()); + let endpoint = start_mock(state.clone()).await; + let client = connect(&endpoint).await; + + let sandbox = client + .workspace("staging") + .create_sandbox_from_template(SandboxTemplateCreateSpec { + name: Some("from-template".to_string()), + template_name: "python".to_string(), + policy: Some(proto::SandboxPolicy { + version: 2, + ..Default::default() + }), + ..Default::default() + }) + .await + .unwrap(); + assert_eq!(sandbox.name, "from-template"); + + let observed = state.last_create.lock().await.clone().unwrap(); + assert_eq!(observed.workspace, "staging"); + assert_eq!(observed.workload_template_name, "python"); + assert_eq!(observed.spec.unwrap().policy.unwrap().version, 2); +} + #[tokio::test] async fn workspace_scoped_get_passes_workspace() { let state = Arc::new(MockState { @@ -1169,6 +1392,50 @@ async fn workspace_scoped_list_passes_workspace() { assert!(!observed.all_workspaces); } +#[tokio::test] +async fn workspace_scoped_sandbox_template_crud_passes_workspace() { + let state = Arc::new(MockState::default()); + let endpoint = start_mock(state.clone()).await; + let client = connect(&endpoint).await; + let ws = client.workspace("staging"); + + ws.create_sandbox_template(workload_template_proto("python", "staging")) + .await + .unwrap(); + let observed_create = state.last_template_create.lock().await.clone().unwrap(); + assert_eq!(observed_create.workspace, "staging"); + + ws.get_sandbox_template("python").await.unwrap(); + let observed_get = state.last_template_get.lock().await.clone().unwrap(); + assert_eq!(observed_get.name, "python"); + assert_eq!(observed_get.workspace, "staging"); + + let listed = ws + .list_sandbox_templates(SandboxTemplateListOptions::default()) + .await + .unwrap(); + assert_eq!(listed.len(), 2); + let observed_list = state.last_template_list.lock().await.clone().unwrap(); + assert_eq!(observed_list.workspace, "staging"); + assert!(!observed_list.all_workspaces); + + ws.list_sandbox_templates(SandboxTemplateListOptions { + all_workspaces: true, + ..Default::default() + }) + .await + .unwrap(); + let observed_all = state.last_template_list.lock().await.clone().unwrap(); + assert!(observed_all.workspace.is_empty()); + assert!(observed_all.all_workspaces); + + let deleted = ws.delete_sandbox_template("python").await.unwrap(); + assert!(deleted); + let observed_delete = state.last_template_delete.lock().await.clone().unwrap(); + assert_eq!(observed_delete.name, "python"); + assert_eq!(observed_delete.workspace, "staging"); +} + #[tokio::test] async fn workspace_scoped_delete_passes_workspace() { let state = Arc::new(MockState::default()); diff --git a/crates/openshell-server/src/compute/mod.rs b/crates/openshell-server/src/compute/mod.rs index 053b08de9c..4fdae8c38e 100644 --- a/crates/openshell-server/src/compute/mod.rs +++ b/crates/openshell-server/src/compute/mod.rs @@ -35,7 +35,7 @@ use openshell_core::proto::compute::v1::{ }; use openshell_core::proto::{ PlatformEvent, Sandbox, SandboxCondition, SandboxPhase, SandboxSpec, SandboxStatus, - SandboxTemplate, ServiceEndpoint, SshSession, + SandboxTemplate, SandboxWorkloadTemplate, ServiceEndpoint, SshSession, }; use openshell_core::telemetry::TelemetryComputeDriver; use openshell_core::{ObjectLabels, ObjectWorkspace}; @@ -3974,6 +3974,12 @@ impl ObjectType for Sandbox { } } +impl ObjectType for SandboxWorkloadTemplate { + fn object_type() -> &'static str { + "sandbox_workload_template" + } +} + fn compute_error_from_status(status: Status) -> ComputeError { match status.code() { Code::AlreadyExists => ComputeError::AlreadyExists, diff --git a/crates/openshell-server/src/grpc/mod.rs b/crates/openshell-server/src/grpc/mod.rs index 957a77cbac..8143e6058e 100644 --- a/crates/openshell-server/src/grpc/mod.rs +++ b/crates/openshell-server/src/grpc/mod.rs @@ -17,29 +17,31 @@ use openshell_core::proto::{ AttachSandboxProviderRequest, AttachSandboxProviderResponse, ClearDraftChunksRequest, ClearDraftChunksResponse, ComputeDriverCapabilities, ComputeDriverInfo, ConfigureProviderRefreshRequest, ConfigureProviderRefreshResponse, CreateProviderRequest, - CreateSandboxRequest, CreateSshSessionRequest, CreateSshSessionResponse, - CreateWorkspaceRequest, CreateWorkspaceResponse, DeleteProviderProfileRequest, - DeleteProviderProfileResponse, DeleteProviderRefreshRequest, DeleteProviderRefreshResponse, - DeleteProviderRequest, DeleteProviderResponse, DeleteSandboxRequest, DeleteSandboxResponse, - DeleteServiceRequest, DeleteServiceResponse, DeleteWorkspaceRequest, DeleteWorkspaceResponse, - DetachSandboxProviderRequest, DetachSandboxProviderResponse, EditDraftChunkRequest, - EditDraftChunkResponse, ExchangeProviderSubjectTokenRequest, - ExchangeProviderSubjectTokenResponse, ExecSandboxEvent, ExecSandboxInput, ExecSandboxRequest, - ExposeServiceRequest, FinalizeMainProcessExitRequest, FinalizeMainProcessExitResponse, - GatewayMessage, GetCurrentUserRequest, GetCurrentUserResponse, GetDraftHistoryRequest, - GetDraftHistoryResponse, GetDraftPolicyRequest, GetDraftPolicyResponse, + CreateSandboxRequest, CreateSandboxTemplateRequest, CreateSshSessionRequest, + CreateSshSessionResponse, CreateWorkspaceRequest, CreateWorkspaceResponse, + DeleteProviderProfileRequest, DeleteProviderProfileResponse, DeleteProviderRefreshRequest, + DeleteProviderRefreshResponse, DeleteProviderRequest, DeleteProviderResponse, + DeleteSandboxRequest, DeleteSandboxResponse, DeleteSandboxTemplateRequest, + DeleteSandboxTemplateResponse, DeleteServiceRequest, DeleteServiceResponse, + DeleteWorkspaceRequest, DeleteWorkspaceResponse, DetachSandboxProviderRequest, + DetachSandboxProviderResponse, EditDraftChunkRequest, EditDraftChunkResponse, + ExchangeProviderSubjectTokenRequest, ExchangeProviderSubjectTokenResponse, ExecSandboxEvent, + ExecSandboxInput, ExecSandboxRequest, ExposeServiceRequest, FinalizeMainProcessExitRequest, + FinalizeMainProcessExitResponse, GatewayMessage, GetCurrentUserRequest, GetCurrentUserResponse, + GetDraftHistoryRequest, GetDraftHistoryResponse, GetDraftPolicyRequest, GetDraftPolicyResponse, GetGatewayConfigRequest, GetGatewayConfigResponse, GetGatewayInfoRequest, GetGatewayInfoResponse, GetProviderProfileRequest, GetProviderRefreshStatusRequest, GetProviderRefreshStatusResponse, GetProviderRequest, GetSandboxConfigRequest, GetSandboxConfigResponse, GetSandboxLogsRequest, GetSandboxLogsResponse, GetSandboxPolicyStatusRequest, GetSandboxPolicyStatusResponse, GetSandboxProviderEnvironmentRequest, GetSandboxProviderEnvironmentResponse, GetSandboxRequest, - GetServiceRequest, GetWorkspaceRequest, GetWorkspaceResponse, HealthRequest, HealthResponse, - ImportProviderProfilesRequest, ImportProviderProfilesResponse, IssueSandboxTokenRequest, - IssueSandboxTokenResponse, LintProviderProfilesRequest, LintProviderProfilesResponse, - ListProviderProfilesRequest, ListProviderProfilesResponse, ListProvidersRequest, - ListProvidersResponse, ListSandboxPoliciesRequest, ListSandboxPoliciesResponse, - ListSandboxProvidersRequest, ListSandboxProvidersResponse, ListSandboxesRequest, + GetSandboxTemplateRequest, GetServiceRequest, GetWorkspaceRequest, GetWorkspaceResponse, + HealthRequest, HealthResponse, ImportProviderProfilesRequest, ImportProviderProfilesResponse, + IssueSandboxTokenRequest, IssueSandboxTokenResponse, LintProviderProfilesRequest, + LintProviderProfilesResponse, ListProviderProfilesRequest, ListProviderProfilesResponse, + ListProvidersRequest, ListProvidersResponse, ListSandboxPoliciesRequest, + ListSandboxPoliciesResponse, ListSandboxProvidersRequest, ListSandboxProvidersResponse, + ListSandboxTemplatesRequest, ListSandboxTemplatesResponse, ListSandboxesRequest, ListSandboxesResponse, ListServicesRequest, ListServicesResponse, ListWorkspaceMembersRequest, ListWorkspaceMembersResponse, ListWorkspacesRequest, ListWorkspacesResponse, ProviderProfileResponse, ProviderResponse, PushSandboxLogsRequest, PushSandboxLogsResponse, @@ -48,12 +50,12 @@ use openshell_core::proto::{ RemoveWorkspaceMemberResponse, ReportMainProcessExitRequest, ReportMainProcessExitResponse, ReportPolicyStatusRequest, ReportPolicyStatusResponse, RevokeSshSessionRequest, RevokeSshSessionResponse, RotateProviderCredentialRequest, RotateProviderCredentialResponse, - SandboxResponse, ServiceEndpointResponse, ServiceStatus, StartSandboxRequest, - StopSandboxRequest, SubmitPolicyAnalysisRequest, SubmitPolicyAnalysisResponse, - SupervisorMessage, TcpForwardFrame, UndoDraftChunkRequest, UndoDraftChunkResponse, - UpdateConfigRequest, UpdateConfigResponse, UpdateProviderProfilesRequest, - UpdateProviderProfilesResponse, UpdateProviderRequest, WatchSandboxRequest, - open_shell_server::OpenShell, + SandboxResponse, SandboxTemplateResponse, ServiceEndpointResponse, ServiceStatus, + StartSandboxRequest, StopSandboxRequest, SubmitPolicyAnalysisRequest, + SubmitPolicyAnalysisResponse, SupervisorMessage, TcpForwardFrame, UndoDraftChunkRequest, + UndoDraftChunkResponse, UpdateConfigRequest, UpdateConfigResponse, + UpdateProviderProfilesRequest, UpdateProviderProfilesResponse, UpdateProviderRequest, + WatchSandboxRequest, open_shell_server::OpenShell, }; use serde::{Deserialize, Serialize}; use std::collections::BTreeMap; @@ -299,6 +301,34 @@ impl OpenShell for OpenShellService { sandbox::handle_list_sandboxes(&self.state, request).await } + async fn create_sandbox_template( + &self, + request: Request, + ) -> Result, Status> { + sandbox::handle_create_sandbox_template(&self.state, request).await + } + + async fn get_sandbox_template( + &self, + request: Request, + ) -> Result, Status> { + sandbox::handle_get_sandbox_template(&self.state, request).await + } + + async fn list_sandbox_templates( + &self, + request: Request, + ) -> Result, Status> { + sandbox::handle_list_sandbox_templates(&self.state, request).await + } + + async fn delete_sandbox_template( + &self, + request: Request, + ) -> Result, Status> { + sandbox::handle_delete_sandbox_template(&self.state, request).await + } + async fn list_sandbox_providers( &self, request: Request, diff --git a/crates/openshell-server/src/grpc/provider.rs b/crates/openshell-server/src/grpc/provider.rs index 35765781c4..764c0bbfde 100644 --- a/crates/openshell-server/src/grpc/provider.rs +++ b/crates/openshell-server/src/grpc/provider.rs @@ -11365,6 +11365,7 @@ mod tests { ..SandboxSpec::default() }), status: None, + ..Sandbox::default() }; sandbox.set_phase(SandboxPhase::Ready as i32); store.put_message(&sandbox).await.unwrap(); @@ -11401,6 +11402,7 @@ mod tests { }), spec: Some(SandboxSpec::default()), status: None, + ..Sandbox::default() }; sandbox.set_phase(SandboxPhase::Ready as i32); store.put_message(&sandbox).await.unwrap(); diff --git a/crates/openshell-server/src/grpc/sandbox.rs b/crates/openshell-server/src/grpc/sandbox.rs index 5364e0541e..d4d08da251 100644 --- a/crates/openshell-server/src/grpc/sandbox.rs +++ b/crates/openshell-server/src/grpc/sandbox.rs @@ -16,14 +16,19 @@ use crate::auth::workspace_authz::{ use crate::persistence::{ObjectLabels, ObjectType, WriteCondition, generate_name}; use futures::future; use openshell_core::net::set_tcp_nodelay_best_effort; +use openshell_core::proto::datamodel::v1::ObjectMeta; use openshell_core::proto::{ AttachSandboxProviderRequest, AttachSandboxProviderResponse, CreateSandboxRequest, - CreateSshSessionRequest, CreateSshSessionResponse, DeleteSandboxRequest, DeleteSandboxResponse, - DetachSandboxProviderRequest, DetachSandboxProviderResponse, ExecSandboxEvent, ExecSandboxExit, - ExecSandboxInput, ExecSandboxRequest, ExecSandboxStderr, ExecSandboxStdout, GetSandboxRequest, - ListSandboxProvidersRequest, ListSandboxProvidersResponse, ListSandboxesRequest, - ListSandboxesResponse, Provider, RevokeSshSessionRequest, RevokeSshSessionResponse, - SandboxResponse, SandboxStreamEvent, SshRelayTarget, StartSandboxRequest, StopSandboxRequest, + CreateSandboxTemplateRequest, CreateSshSessionRequest, CreateSshSessionResponse, + DeleteSandboxRequest, DeleteSandboxResponse, DeleteSandboxTemplateRequest, + DeleteSandboxTemplateResponse, DetachSandboxProviderRequest, DetachSandboxProviderResponse, + ExecSandboxEvent, ExecSandboxExit, ExecSandboxInput, ExecSandboxRequest, ExecSandboxStderr, + ExecSandboxStdout, GetSandboxRequest, GetSandboxTemplateRequest, ListSandboxProvidersRequest, + ListSandboxProvidersResponse, ListSandboxTemplatesRequest, ListSandboxTemplatesResponse, + ListSandboxesRequest, ListSandboxesResponse, Provider, ResourceRequirements, + RevokeSshSessionRequest, RevokeSshSessionResponse, SandboxResources, SandboxResponse, + SandboxSpec, SandboxStreamEvent, SandboxTemplateResponse, SandboxWorkloadTemplate, + SandboxWorkloadTemplateProvenance, SshRelayTarget, StartSandboxRequest, StopSandboxRequest, TcpForwardFrame, TcpForwardInit, TcpRelayTarget, WatchSandboxRequest, relay_open, tcp_forward_init, }; @@ -31,8 +36,9 @@ use openshell_core::proto::{Sandbox, SandboxPhase, SandboxTemplate, SshSession}; use openshell_core::telemetry::{ LifecycleOperation, LifecycleResource, SandboxTemplateSource, TelemetryOutcome, }; -use openshell_core::{ObjectId, ObjectName, ObjectWorkspace}; +use openshell_core::{GetResourceVersion, ObjectId, ObjectName, ObjectWorkspace}; use prost::Message; +use prost_types::{Struct, Value, value::Kind}; use std::collections::HashMap; use std::net::IpAddr; use std::pin::Pin; @@ -52,14 +58,16 @@ use super::provider::{ get_provider_record, is_valid_env_key, validate_provider_environment_keys_unique_with_catalog, }; use super::validation::{ - level_matches, source_matches, validate_exec_request_fields, - validate_no_reserved_provider_policy_keys, validate_policy_safety, validate_sandbox_spec, + level_matches, source_matches, validate_dns1123_label, validate_exec_request_fields, + validate_no_reserved_provider_policy_keys, validate_policy_safety, + validate_sandbox_governance_spec, validate_sandbox_spec, }; use super::{MAX_PAGE_SIZE, MAX_PROVIDERS, MAX_ROUTABLE_NAME_LEN, clamp_limit}; use crate::persistence::current_time_ms; const TCP_FORWARD_CHUNK_SIZE: usize = 64 * 1024; const NO_LOGIN_SHELL_ENV: (&str, &str) = ("OPENSHELL_NO_LOGIN_SHELL", "1"); +const MAX_TEMPLATES_PER_WORKSPACE: u32 = 1000; #[derive(Debug)] pub struct WatchSandboxStream { @@ -158,30 +166,69 @@ pub(super) async fn handle_create_sandbox( ) -> Result, Status> { let create_request = request.get_ref().clone(); let result = handle_create_sandbox_inner(state, request).await; + let created_sandbox = result + .as_ref() + .ok() + .and_then(|response| response.get_ref().sandbox.as_ref()); emit_sandbox_create_telemetry( state, &create_request, + created_sandbox, TelemetryOutcome::from_success(result.is_ok()), ); result } +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +struct SandboxCreateTelemetryAttrs { + requested_gpu: bool, + provider_count: u64, + has_custom_policy: bool, + template_source: SandboxTemplateSource, +} + fn emit_sandbox_create_telemetry( state: &Arc, request: &CreateSandboxRequest, + created_sandbox: Option<&Sandbox>, outcome: TelemetryOutcome, ) { let compute_driver = state.compute.telemetry_compute_driver(); + let attrs = sandbox_create_telemetry_attrs(request, created_sandbox); + openshell_core::telemetry::emit_sandbox_create( + outcome, + attrs.requested_gpu, + attrs.provider_count, + attrs.has_custom_policy, + attrs.template_source, + compute_driver, + ); +} + +fn sandbox_create_telemetry_attrs( + request: &CreateSandboxRequest, + created_sandbox: Option<&Sandbox>, +) -> SandboxCreateTelemetryAttrs { + if !request.workload_template_name.trim().is_empty() { + let spec = created_sandbox + .and_then(|sandbox| sandbox.spec.as_ref()) + .or(request.spec.as_ref()); + return SandboxCreateTelemetryAttrs { + requested_gpu: spec.is_some_and(|spec| { + openshell_core::gpu::sandbox_gpu_requested(spec.resource_requirements.as_ref()) + }), + provider_count: spec.map_or(0, |spec| spec.providers.len() as u64), + has_custom_policy: spec.is_some_and(|spec| spec.policy.is_some()), + template_source: SandboxTemplateSource::WorkloadTemplate, + }; + } let Some(spec) = request.spec.as_ref() else { - openshell_core::telemetry::emit_sandbox_create( - outcome, - false, - 0, - false, - SandboxTemplateSource::Undefined, - compute_driver, - ); - return; + return SandboxCreateTelemetryAttrs { + requested_gpu: false, + provider_count: 0, + has_custom_policy: false, + template_source: SandboxTemplateSource::Undefined, + }; }; let template_source = if spec .template @@ -194,14 +241,12 @@ fn emit_sandbox_create_telemetry( }; let gpu_requested = openshell_core::gpu::sandbox_gpu_requested(spec.resource_requirements.as_ref()); - openshell_core::telemetry::emit_sandbox_create( - outcome, - gpu_requested, - spec.providers.len() as u64, - spec.policy.is_some(), + SandboxCreateTelemetryAttrs { + requested_gpu: gpu_requested, + provider_count: spec.providers.len() as u64, + has_custom_policy: spec.policy.is_some(), template_source, - compute_driver, - ); + } } async fn handle_create_sandbox_inner( @@ -211,27 +256,9 @@ async fn handle_create_sandbox_inner( let principal = super::extract_principal(&request)?; let request = request.into_inner(); let await_main_process_attachment = request.await_main_process_attachment; - let mut spec = request - .spec - .ok_or_else(|| Status::invalid_argument("spec is required"))?; - - // Every newly persisted sandbox has one explicit canonical process. This - // portable default also preserves compatibility with callers compiled - // before the main-process field was introduced. - if spec.command.is_empty() { - spec.command = vec!["/bin/bash".to_string(), "-l".to_string()]; - spec.tty = true; - } - - // Validate field sizes before any I/O (fail fast on oversized payloads). - validate_sandbox_spec(&request.name, &spec)?; + let workload_template_name = request.workload_template_name.trim().to_string(); - // Validate labels (keys and values must meet Kubernetes requirements). - for (key, value) in &request.labels { - crate::grpc::validation::validate_label_key(key)?; - crate::grpc::validation::validate_label_value(value)?; - } - crate::grpc::validation::validate_annotations(&request.annotations, "annotations")?; + validate_create_sandbox_request_pre_io(&request, &workload_template_name)?; let authz = authorize_workspace( &state.store, @@ -245,6 +272,42 @@ async fn handle_create_sandbox_inner( .await? .ensure_active()?; + let (mut spec, created_from_workload_template) = if workload_template_name.is_empty() { + let spec = request + .spec + .ok_or_else(|| Status::invalid_argument("spec is required"))?; + (spec, None) + } else { + let governance_spec = request.spec.unwrap_or_default(); + let template = state + .store + .get_message_by_name::(&workspace, &workload_template_name) + .await + .map_err(|e| Status::internal(format!("fetch sandbox template failed: {e}")))? + .ok_or_else(|| Status::not_found("sandbox template not found"))?; + let provenance = SandboxWorkloadTemplateProvenance { + name: template.object_name().to_string(), + resource_version: template.get_resource_version().to_string(), + }; + let mut resolved = sandbox_spec_from_stored_workload_template(&template)?; + resolved.policy = governance_spec.policy; + resolved.providers = governance_spec.providers; + resolved.command = governance_spec.command; + resolved.tty = governance_spec.tty; + (resolved, Some(provenance)) + }; + + // Every newly persisted sandbox has one explicit canonical process. This + // portable default also preserves compatibility with callers compiled + // before the main-process field was introduced. + if spec.command.is_empty() { + spec.command = vec!["/bin/bash".to_string(), "-l".to_string()]; + spec.tty = true; + } + + // Validate field sizes before any create-side effects. + validate_sandbox_spec(&request.name, &spec)?; + let _sandbox_sync_guard = if spec.providers.is_empty() { None } else { @@ -274,7 +337,7 @@ async fn handle_create_sandbox_inner( // Ensure the template always carries the resolved image. let template = spec.template.get_or_insert_with(SandboxTemplate::default); - if template.image.is_empty() { + if template.image.trim().is_empty() { template.image = state.compute.default_image().to_string(); } @@ -302,7 +365,7 @@ async fn handle_create_sandbox_inner( let now_ms = current_time_ms(); let mut sandbox = Sandbox { - metadata: Some(openshell_core::proto::datamodel::v1::ObjectMeta { + metadata: Some(ObjectMeta { id: id.clone(), name: name.clone(), created_at_ms: now_ms, @@ -314,6 +377,7 @@ async fn handle_create_sandbox_inner( }), spec: Some(spec), status: None, + created_from_workload_template, }; sandbox.set_phase(SandboxPhase::Provisioning as i32); @@ -372,6 +436,135 @@ async fn handle_create_sandbox_inner( })) } +fn validate_create_sandbox_request_pre_io( + request: &CreateSandboxRequest, + workload_template_name: &str, +) -> Result<(), Status> { + // Validate labels (keys and values must meet Kubernetes requirements). + for (key, value) in &request.labels { + crate::grpc::validation::validate_label_key(key)?; + crate::grpc::validation::validate_label_value(value)?; + } + crate::grpc::validation::validate_annotations(&request.annotations, "annotations")?; + + if workload_template_name.is_empty() { + let spec = request + .spec + .as_ref() + .ok_or_else(|| Status::invalid_argument("spec is required"))?; + return validate_sandbox_spec(&request.name, spec); + } + + validate_dns1123_label(workload_template_name, "workload_template_name")?; + if let Some(spec) = request.spec.as_ref() { + validate_template_create_governance_spec(spec)?; + validate_sandbox_governance_spec(&request.name, spec)?; + } else { + validate_sandbox_governance_spec(&request.name, &SandboxSpec::default())?; + } + Ok(()) +} + +fn validate_template_create_governance_spec(spec: &SandboxSpec) -> Result<(), Status> { + if !spec.log_level.is_empty() { + return Err(Status::invalid_argument( + "spec.log_level cannot be set when workload_template_name is set", + )); + } + if !spec.environment.is_empty() { + return Err(Status::invalid_argument( + "spec.environment cannot be set when workload_template_name is set", + )); + } + if spec.template.is_some() { + return Err(Status::invalid_argument( + "spec.template cannot be set when workload_template_name is set", + )); + } + if spec.resource_requirements.is_some() { + return Err(Status::invalid_argument( + "spec.resource_requirements cannot be set when workload_template_name is set", + )); + } + Ok(()) +} + +fn sandbox_spec_from_stored_workload_template( + template: &SandboxWorkloadTemplate, +) -> Result { + sandbox_spec_from_workload_template(template, tonic::Code::Internal) +} + +fn sandbox_spec_from_user_workload_template( + template: &SandboxWorkloadTemplate, +) -> Result { + sandbox_spec_from_workload_template(template, tonic::Code::InvalidArgument) +} + +fn sandbox_spec_from_workload_template( + template: &SandboxWorkloadTemplate, + missing_field_code: tonic::Code, +) -> Result { + let spec = template + .spec + .as_ref() + .ok_or_else(|| Status::new(missing_field_code, "sandbox template spec is required"))?; + let workload = spec + .workload + .as_ref() + .ok_or_else(|| Status::new(missing_field_code, "sandbox template workload is required"))?; + let resources = workload.resources.as_ref(); + Ok(SandboxSpec { + environment: workload.environment.clone(), + template: Some(SandboxTemplate { + image: workload.image.clone(), + resources: resources.and_then(template_resource_struct), + driver_config: spec.driver_config.clone(), + ..SandboxTemplate::default() + }), + resource_requirements: resources.and_then(template_gpu_requirements), + ..SandboxSpec::default() + }) +} + +fn template_gpu_requirements(resources: &SandboxResources) -> Option { + Some(ResourceRequirements { + gpu: Some(resources.gpu?), + }) +} + +fn template_resource_struct(resources: &SandboxResources) -> Option { + let mut limits = std::collections::BTreeMap::new(); + if !resources.cpu.is_empty() { + limits.insert( + "cpu".to_string(), + Value { + kind: Some(Kind::StringValue(resources.cpu.clone())), + }, + ); + } + if !resources.memory.is_empty() { + limits.insert( + "memory".to_string(), + Value { + kind: Some(Kind::StringValue(resources.memory.clone())), + }, + ); + } + if limits.is_empty() { + None + } else { + let mut fields = std::collections::BTreeMap::new(); + fields.insert( + "limits".to_string(), + Value { + kind: Some(Kind::StructValue(Struct { fields: limits })), + }, + ); + Some(Struct { fields }) + } +} + pub(super) async fn handle_get_sandbox( state: &Arc, request: Request, @@ -472,12 +665,106 @@ pub(super) async fn handle_list_sandboxes( Ok(Response::new(ListSandboxesResponse { sandboxes })) } -pub(super) async fn handle_list_sandbox_providers( +pub(super) async fn handle_create_sandbox_template( state: &Arc, - request: Request, -) -> Result, Status> { + request: Request, +) -> Result, Status> { + let principal = super::extract_principal(&request)?; + let req = request.into_inner(); + let template = req + .template + .ok_or_else(|| Status::invalid_argument("template is required"))?; + let metadata = template.metadata.clone().unwrap_or_default(); + let authz = authorize_workspace( + &state.store, + &state.admin_role, + &principal, + &req.workspace, + MinWorkspaceRole::Admin, + ) + .await?; + let workspace = super::workspace::resolve_workspace(state.store.as_ref(), &authz.workspace) + .await? + .ensure_active()?; + if !metadata.workspace.is_empty() && metadata.workspace != workspace { + return Err(Status::invalid_argument( + "template.metadata.workspace must match request workspace", + )); + } + if metadata.name.is_empty() { + return Err(Status::invalid_argument( + "template.metadata.name is required", + )); + } + + let mut resolved = template; + resolved.metadata = Some(ObjectMeta { + id: uuid::Uuid::new_v4().to_string(), + name: metadata.name, + created_at_ms: current_time_ms(), + labels: metadata.labels, + resource_version: 0, + annotations: metadata.annotations, + workspace: workspace.clone(), + deletion_timestamp_ms: 0, + }); + validate_sandbox_workload_template(&resolved)?; + + let labels_map = resolved.object_labels(); + let labels_json = if labels_map.as_ref().is_none_or(HashMap::is_empty) { + None + } else { + Some( + serde_json::to_string(&labels_map) + .map_err(|e| Status::internal(format!("failed to serialize labels: {e}")))?, + ) + }; + let write = state + .store + .create_if_workspace_count_below( + SandboxWorkloadTemplate::object_type(), + resolved.object_id(), + resolved.object_name(), + &workspace, + &resolved.encode_to_vec(), + labels_json.as_deref(), + u64::from(MAX_TEMPLATES_PER_WORKSPACE), + ) + .await; + let write = match write { + Ok(Some(write)) => write, + Ok(None) => { + return Err(Status::resource_exhausted(format!( + "workspace has reached the maximum of {MAX_TEMPLATES_PER_WORKSPACE} sandbox templates" + ))); + } + Err(crate::persistence::PersistenceError::UniqueViolation { .. }) => { + return Err(Status::already_exists("sandbox template already exists")); + } + Err(err) => { + return Err(Status::internal(format!( + "persist sandbox template failed: {err}" + ))); + } + }; + if let Some(metadata) = resolved.metadata.as_mut() { + metadata.resource_version = write.resource_version; + } + + Ok(Response::new(SandboxTemplateResponse { + template: Some(resolved), + })) +} + +pub(super) async fn handle_get_sandbox_template( + state: &Arc, + request: Request, +) -> Result, Status> { let principal = super::extract_principal(&request)?; let req = request.into_inner(); + if req.name.is_empty() { + return Err(Status::invalid_argument("name is required")); + } let authz = authorize_workspace( &state.store, &state.admin_role, @@ -489,35 +776,221 @@ pub(super) async fn handle_list_sandbox_providers( let workspace = super::workspace::resolve_workspace(state.store.as_ref(), &authz.workspace) .await? .name; - let sandbox = sandbox_by_name(state, &workspace, &req.sandbox_name).await?; - let providers = providers_for_sandbox(state, &sandbox, &workspace).await?; - Ok(Response::new(ListSandboxProvidersResponse { providers })) + let template = state + .store + .get_message_by_name::(&workspace, &req.name) + .await + .map_err(|e| Status::internal(format!("fetch sandbox template failed: {e}")))? + .ok_or_else(|| Status::not_found("sandbox template not found"))?; + Ok(Response::new(SandboxTemplateResponse { + template: Some(template), + })) } -pub(super) async fn handle_attach_sandbox_provider( +pub(super) async fn handle_list_sandbox_templates( state: &Arc, - request: Request, -) -> Result, Status> { + request: Request, +) -> Result, Status> { let principal = super::extract_principal(&request)?; let request = request.into_inner(); + if request.all_workspaces && !request.workspace.is_empty() { + return Err(Status::invalid_argument( + "all_workspaces and workspace are mutually exclusive", + )); + } + let limit = clamp_limit(request.limit, 100, MAX_PAGE_SIZE); + let templates = if request.all_workspaces { + require_platform_admin(&state.admin_role, &principal)?; + if request.label_selector.is_empty() { + state + .store + .list_all_messages::(limit, request.offset) + .await + .map_err(|e| Status::internal(format!("list sandbox templates failed: {e}")))? + } else { + crate::grpc::validation::validate_label_selector(&request.label_selector)?; + state + .store + .list_all_messages_with_selector::( + &request.label_selector, + limit, + request.offset, + ) + .await + .map_err(|e| Status::internal(format!("list sandbox templates failed: {e}")))? + } + } else { + let authz = authorize_workspace( + &state.store, + &state.admin_role, + &principal, + &request.workspace, + MinWorkspaceRole::User, + ) + .await?; + let workspace = super::workspace::resolve_workspace(state.store.as_ref(), &authz.workspace) + .await? + .name; + if request.label_selector.is_empty() { + state + .store + .list_messages::(&workspace, limit, request.offset) + .await + .map_err(|e| Status::internal(format!("list sandbox templates failed: {e}")))? + } else { + crate::grpc::validation::validate_label_selector(&request.label_selector)?; + state + .store + .list_messages_with_selector::( + &workspace, + &request.label_selector, + limit, + request.offset, + ) + .await + .map_err(|e| { + Status::internal(format!("list sandbox templates with selector failed: {e}")) + })? + } + }; + Ok(Response::new(ListSandboxTemplatesResponse { templates })) +} + +pub(super) async fn handle_delete_sandbox_template( + state: &Arc, + request: Request, +) -> Result, Status> { + let principal = super::extract_principal(&request)?; + let req = request.into_inner(); + if req.name.is_empty() { + return Err(Status::invalid_argument("name is required")); + } let authz = authorize_workspace( &state.store, &state.admin_role, &principal, - &request.workspace, - MinWorkspaceRole::User, + &req.workspace, + MinWorkspaceRole::Admin, ) .await?; let workspace = super::workspace::resolve_workspace(state.store.as_ref(), &authz.workspace) .await? - .ensure_active()?; - if request.provider_name.is_empty() { - return Err(Status::invalid_argument("provider_name is required")); + .name; + let deleted = state + .store + .delete_by_name( + SandboxWorkloadTemplate::object_type(), + &workspace, + &req.name, + ) + .await + .map_err(|e| Status::internal(format!("delete sandbox template failed: {e}")))?; + Ok(Response::new(DeleteSandboxTemplateResponse { deleted })) +} + +fn validate_sandbox_workload_template(template: &SandboxWorkloadTemplate) -> Result<(), Status> { + super::validation::validate_object_metadata(template.metadata.as_ref(), "sandbox_template")?; + let name = template.object_name().to_string(); + validate_dns1123_label(&name, "template.metadata.name")?; + validate_sandbox_workload_template_service_level(template)?; + let spec = sandbox_spec_from_user_workload_template(template)?; + validate_sandbox_spec(&name, &spec)?; + Ok(()) +} + +fn validate_sandbox_workload_template_service_level( + template: &SandboxWorkloadTemplate, +) -> Result<(), Status> { + let Some(startup) = template + .spec + .as_ref() + .and_then(|spec| spec.desired_service_level.as_ref()) + .and_then(|service_level| service_level.startup.as_ref()) + else { + return Ok(()); + }; + if let Some(ready_within) = &startup.ready_within { + validate_positive_normalized_duration( + ready_within, + "template.spec.desired_service_level.startup.ready_within", + )?; } + Ok(()) +} - // Validate provider name would not violate sandbox spec constraints if added - // (pre-validation ensures CAS mutations preserve invariants) - if request.provider_name.len() > super::MAX_NAME_LEN { +fn validate_positive_normalized_duration( + duration: &prost_types::Duration, + field: &str, +) -> Result<(), Status> { + const MAX_DURATION_SECONDS: u64 = 315_576_000_000; + if duration.seconds.unsigned_abs() > MAX_DURATION_SECONDS + || duration.nanos.unsigned_abs() >= 1_000_000_000 + { + return Err(Status::invalid_argument(format!( + "{field} must be a valid protobuf Duration" + ))); + } + if (duration.seconds > 0 && duration.nanos < 0) || (duration.seconds < 0 && duration.nanos > 0) + { + return Err(Status::invalid_argument(format!( + "{field} must be a normalized protobuf Duration" + ))); + } + if duration.seconds < 0 || duration.nanos < 0 || (duration.seconds == 0 && duration.nanos == 0) + { + return Err(Status::invalid_argument(format!( + "{field} must be greater than zero" + ))); + } + Ok(()) +} + +pub(super) async fn handle_list_sandbox_providers( + state: &Arc, + request: Request, +) -> Result, Status> { + let principal = super::extract_principal(&request)?; + let req = request.into_inner(); + let authz = authorize_workspace( + &state.store, + &state.admin_role, + &principal, + &req.workspace, + MinWorkspaceRole::User, + ) + .await?; + let workspace = super::workspace::resolve_workspace(state.store.as_ref(), &authz.workspace) + .await? + .name; + let sandbox = sandbox_by_name(state, &workspace, &req.sandbox_name).await?; + let providers = providers_for_sandbox(state, &sandbox, &workspace).await?; + Ok(Response::new(ListSandboxProvidersResponse { providers })) +} + +pub(super) async fn handle_attach_sandbox_provider( + state: &Arc, + request: Request, +) -> Result, Status> { + let principal = super::extract_principal(&request)?; + let request = request.into_inner(); + let authz = authorize_workspace( + &state.store, + &state.admin_role, + &principal, + &request.workspace, + MinWorkspaceRole::User, + ) + .await?; + let workspace = super::workspace::resolve_workspace(state.store.as_ref(), &authz.workspace) + .await? + .ensure_active()?; + if request.provider_name.is_empty() { + return Err(Status::invalid_argument("provider_name is required")); + } + + // Validate provider name would not violate sandbox spec constraints if added + // (pre-validation ensures CAS mutations preserve invariants) + if request.provider_name.len() > super::MAX_NAME_LEN { return Err(Status::invalid_argument(format!( "provider_name exceeds maximum length ({} > {})", request.provider_name.len(), @@ -1753,7 +2226,7 @@ pub(super) async fn handle_create_ssh_session( 0 }; let session = SshSession { - metadata: Some(openshell_core::proto::datamodel::v1::ObjectMeta { + metadata: Some(ObjectMeta { id: token.clone(), name: generate_name(), created_at_ms: now_ms, @@ -2532,6 +3005,7 @@ mod tests { }; use crate::provider_profile_sources::ProviderProfileSources; use openshell_core::GatewayProviderProfileSourceConfig; + use openshell_core::proto::GpuResourceRequirements; use openshell_core::proto::datamodel::v1::ObjectMeta; async fn test_server_state_with_user_only_github_profile() -> Arc { @@ -2590,6 +3064,66 @@ mod tests { } } + #[test] + fn sandbox_create_telemetry_uses_resolved_template_gpu_request() { + let request = CreateSandboxRequest { + spec: Some(SandboxSpec { + providers: vec!["github".to_string()], + policy: Some(openshell_core::proto::SandboxPolicy::default()), + ..SandboxSpec::default() + }), + workload_template_name: "gpu-kata".to_string(), + ..CreateSandboxRequest::default() + }; + let created = Sandbox { + spec: Some(SandboxSpec { + providers: vec!["github".to_string()], + policy: Some(openshell_core::proto::SandboxPolicy::default()), + resource_requirements: Some(ResourceRequirements { + gpu: Some(GpuResourceRequirements { count: Some(1) }), + }), + ..SandboxSpec::default() + }), + created_from_workload_template: Some(SandboxWorkloadTemplateProvenance { + name: "gpu-kata".to_string(), + resource_version: "7".to_string(), + }), + ..Sandbox::default() + }; + + assert_eq!( + sandbox_create_telemetry_attrs(&request, Some(&created)), + SandboxCreateTelemetryAttrs { + requested_gpu: true, + provider_count: 1, + has_custom_policy: true, + template_source: SandboxTemplateSource::WorkloadTemplate, + } + ); + } + + #[test] + fn sandbox_create_telemetry_falls_back_to_request_for_unresolved_template() { + let request = CreateSandboxRequest { + spec: Some(SandboxSpec { + providers: vec!["github".to_string()], + ..SandboxSpec::default() + }), + workload_template_name: "missing-template".to_string(), + ..CreateSandboxRequest::default() + }; + + assert_eq!( + sandbox_create_telemetry_attrs(&request, None), + SandboxCreateTelemetryAttrs { + requested_gpu: false, + provider_count: 1, + has_custom_policy: false, + template_source: SandboxTemplateSource::WorkloadTemplate, + } + ); + } + #[test] fn shell_escape_safe_chars_pass_through() { assert_eq!(shell_escape("ls").unwrap(), "ls"); @@ -2899,7 +3433,7 @@ mod tests { workspace: "default".to_string(), deletion_timestamp_ms: 0, }), - spec: Some(openshell_core::proto::SandboxSpec { + spec: Some(SandboxSpec { log_level: "debug".to_string(), policy: Some(openshell_core::proto::SandboxPolicy::default()), providers, @@ -2912,6 +3446,41 @@ mod tests { sandbox } + fn test_workload_template(name: &str) -> SandboxWorkloadTemplate { + SandboxWorkloadTemplate { + metadata: Some(ObjectMeta { + id: String::new(), + name: name.to_string(), + created_at_ms: 0, + labels: HashMap::from([("team".to_string(), "runtime".to_string())]), + resource_version: 0, + annotations: HashMap::new(), + workspace: String::new(), + deletion_timestamp_ms: 0, + }), + spec: Some(openshell_core::proto::SandboxWorkloadTemplateSpec { + workload: Some(openshell_core::proto::SandboxWorkloadConfig { + image: "registry.example.com/agent:latest".to_string(), + environment: HashMap::from([("FEATURE_FLAG".to_string(), "on".to_string())]), + resources: Some(SandboxResources { + cpu: "2".to_string(), + memory: "4Gi".to_string(), + gpu: Some(GpuResourceRequirements { count: Some(1) }), + }), + }), + driver_config: None, + desired_service_level: None, + }), + } + } + + fn proto_string_value(value: &Value) -> Option<&str> { + match value.kind.as_ref() { + Some(Kind::StringValue(value)) => Some(value.as_str()), + _ => None, + } + } + #[tokio::test] #[ignore = "flaky under concurrent test execution"] async fn watch_producer_releases_request_span_when_client_disconnects() { @@ -3471,7 +4040,7 @@ mod tests { &state, authed_request(CreateSandboxRequest { name: "collision".to_string(), - spec: Some(openshell_core::proto::SandboxSpec { + spec: Some(SandboxSpec { providers: vec!["provider-a".to_string(), "provider-b".to_string()], ..Default::default() }), @@ -3479,6 +4048,7 @@ mod tests { annotations: HashMap::new(), workspace: String::new(), await_main_process_attachment: false, + workload_template_name: String::new(), }), ) .await @@ -3498,11 +4068,12 @@ mod tests { &state, authed_request(CreateSandboxRequest { name: "user-catalog".to_string(), - spec: Some(openshell_core::proto::SandboxSpec::default()), + spec: Some(SandboxSpec::default()), labels: HashMap::new(), annotations: HashMap::new(), workspace: String::new(), await_main_process_attachment: false, + workload_template_name: String::new(), }), ) .await @@ -3531,7 +4102,7 @@ mod tests { &state, authed_request(CreateSandboxRequest { name: "reserved-policy-key".to_string(), - spec: Some(openshell_core::proto::SandboxSpec { + spec: Some(SandboxSpec { policy: Some(policy), ..Default::default() }), @@ -3539,6 +4110,7 @@ mod tests { annotations: HashMap::new(), workspace: String::new(), await_main_process_attachment: false, + workload_template_name: String::new(), }), ) .await @@ -3559,11 +4131,12 @@ mod tests { &state, authed_request(CreateSandboxRequest { name: "annotated".to_string(), - spec: Some(openshell_core::proto::SandboxSpec::default()), + spec: Some(SandboxSpec::default()), labels: HashMap::new(), annotations: HashMap::from([(annotation_key.clone(), annotation_value.clone())]), workspace: String::new(), await_main_process_attachment: false, + workload_template_name: String::new(), }), ) .await @@ -3616,7 +4189,7 @@ mod tests { &state, authed_request(CreateSandboxRequest { name: "partial-id".to_string(), - spec: Some(openshell_core::proto::SandboxSpec { + spec: Some(SandboxSpec { policy: Some(policy), ..Default::default() }), @@ -3624,6 +4197,7 @@ mod tests { annotations: HashMap::new(), workspace: String::new(), await_main_process_attachment: false, + workload_template_name: String::new(), }), ) .await @@ -3680,7 +4254,7 @@ mod tests { &state, authed_request(CreateSandboxRequest { name: "kube-partial-id".to_string(), - spec: Some(openshell_core::proto::SandboxSpec { + spec: Some(SandboxSpec { policy: Some(policy), ..Default::default() }), @@ -3688,6 +4262,7 @@ mod tests { annotations: HashMap::new(), workspace: String::new(), await_main_process_attachment: false, + workload_template_name: String::new(), }), ) .await @@ -3714,11 +4289,12 @@ mod tests { &state, authed_request(CreateSandboxRequest { name: "bad-label".to_string(), - spec: Some(openshell_core::proto::SandboxSpec::default()), + spec: Some(SandboxSpec::default()), labels: HashMap::from([("team".to_string(), "x".repeat(512))]), annotations: HashMap::new(), workspace: String::new(), await_main_process_attachment: false, + workload_template_name: String::new(), }), ) .await @@ -3744,7 +4320,7 @@ mod tests { &task_state, authed_request(CreateSandboxRequest { name: "guarded-create".to_string(), - spec: Some(openshell_core::proto::SandboxSpec { + spec: Some(SandboxSpec { providers: vec!["work-github".to_string()], ..Default::default() }), @@ -3752,6 +4328,7 @@ mod tests { annotations: HashMap::new(), workspace: String::new(), await_main_process_attachment: false, + workload_template_name: String::new(), }), ) .await @@ -3776,6 +4353,815 @@ mod tests { ); } + #[tokio::test] + async fn sandbox_template_handlers_create_get_list_and_delete_workspace_resource() { + let state = test_server_state().await; + + let created = handle_create_sandbox_template( + &state, + authed_request(CreateSandboxTemplateRequest { + template: Some(test_workload_template("gpu-kata")), + workspace: "default".to_string(), + }), + ) + .await + .expect("template create should succeed") + .into_inner() + .template + .expect("template response"); + + let metadata = created.metadata.as_ref().expect("metadata"); + assert_eq!(metadata.name, "gpu-kata"); + assert_eq!(metadata.workspace, "default"); + assert!(!metadata.id.is_empty()); + assert_ne!(metadata.resource_version, 0); + + let fetched = handle_get_sandbox_template( + &state, + authed_request(GetSandboxTemplateRequest { + name: "gpu-kata".to_string(), + workspace: "default".to_string(), + }), + ) + .await + .expect("template get should succeed") + .into_inner() + .template + .expect("fetched template"); + assert_eq!(fetched.object_name(), "gpu-kata"); + assert_eq!(fetched.object_workspace(), "default"); + + let listed = handle_list_sandbox_templates( + &state, + authed_request(ListSandboxTemplatesRequest { + limit: 100, + offset: 0, + workspace: "default".to_string(), + all_workspaces: false, + label_selector: String::new(), + }), + ) + .await + .expect("template list should succeed") + .into_inner() + .templates; + assert_eq!(listed.len(), 1); + assert_eq!(listed[0].object_name(), "gpu-kata"); + + let deleted = handle_delete_sandbox_template( + &state, + authed_request(DeleteSandboxTemplateRequest { + name: "gpu-kata".to_string(), + workspace: "default".to_string(), + }), + ) + .await + .expect("template delete should succeed") + .into_inner(); + assert!(deleted.deleted); + + let missing = handle_get_sandbox_template( + &state, + authed_request(GetSandboxTemplateRequest { + name: "gpu-kata".to_string(), + workspace: "default".to_string(), + }), + ) + .await + .expect_err("deleted template should not be fetchable"); + assert_eq!(missing.code(), tonic::Code::NotFound); + } + + #[tokio::test] + async fn sandbox_template_list_filters_by_label_selector() { + let state = test_server_state().await; + + let mut gpu = test_workload_template("gpu-kata"); + gpu.metadata + .as_mut() + .expect("metadata") + .labels + .insert("team".to_string(), "runtime".to_string()); + handle_create_sandbox_template( + &state, + authed_request(CreateSandboxTemplateRequest { + template: Some(gpu), + workspace: "default".to_string(), + }), + ) + .await + .expect("gpu template create should succeed"); + + let mut cpu = test_workload_template("cpu-base"); + cpu.metadata + .as_mut() + .expect("metadata") + .labels + .insert("team".to_string(), "batch".to_string()); + handle_create_sandbox_template( + &state, + authed_request(CreateSandboxTemplateRequest { + template: Some(cpu), + workspace: "default".to_string(), + }), + ) + .await + .expect("cpu template create should succeed"); + + let listed = handle_list_sandbox_templates( + &state, + authed_request(ListSandboxTemplatesRequest { + limit: 100, + offset: 0, + workspace: "default".to_string(), + all_workspaces: false, + label_selector: "team=runtime".to_string(), + }), + ) + .await + .expect("template list with label selector should succeed") + .into_inner() + .templates; + + assert_eq!(listed.len(), 1); + assert_eq!(listed[0].object_name(), "gpu-kata"); + } + + #[tokio::test] + async fn sandbox_template_create_rejects_whitespace_name() { + let state = test_server_state().await; + + let err = handle_create_sandbox_template( + &state, + authed_request(CreateSandboxTemplateRequest { + template: Some(test_workload_template(" gpu-kata ")), + workspace: "default".to_string(), + }), + ) + .await + .expect_err("template names must be canonical DNS-1123 labels"); + + assert_eq!(err.code(), tonic::Code::InvalidArgument); + assert!(err.message().contains("template.metadata.name")); + + let listed = handle_list_sandbox_templates( + &state, + authed_request(ListSandboxTemplatesRequest { + limit: 100, + offset: 0, + workspace: "default".to_string(), + all_workspaces: false, + label_selector: String::new(), + }), + ) + .await + .expect("template list should succeed") + .into_inner() + .templates; + assert!(listed.is_empty()); + } + + #[tokio::test] + async fn sandbox_template_create_empty_workspace_ignores_metadata_workspace() { + use openshell_core::proto::CreateWorkspaceRequest; + + let state = test_server_state().await; + crate::grpc::workspace::handle_create_workspace( + &state, + Request::new(CreateWorkspaceRequest { + name: "beta".to_string(), + labels: HashMap::new(), + }), + ) + .await + .expect("beta workspace should be created"); + + let mut template = test_workload_template("copied-template"); + template.metadata.as_mut().unwrap().workspace = "beta".to_string(); + + let err = handle_create_sandbox_template( + &state, + authed_request(CreateSandboxTemplateRequest { + template: Some(template), + workspace: String::new(), + }), + ) + .await + .expect_err("empty request workspace must default to default, not metadata workspace"); + + assert_eq!(err.code(), tonic::Code::InvalidArgument); + assert!(err.message().contains("template.metadata.workspace")); + + let listed = handle_list_sandbox_templates( + &state, + authed_request(ListSandboxTemplatesRequest { + limit: 100, + offset: 0, + workspace: "beta".to_string(), + all_workspaces: false, + label_selector: String::new(), + }), + ) + .await + .expect("template list should succeed") + .into_inner() + .templates; + assert!(listed.is_empty()); + } + + #[tokio::test] + async fn sandbox_template_create_rejects_missing_spec_as_invalid_argument() { + let state = test_server_state().await; + let mut template = test_workload_template("missing-spec"); + template.spec = None; + + let err = handle_create_sandbox_template( + &state, + authed_request(CreateSandboxTemplateRequest { + template: Some(template), + workspace: "default".to_string(), + }), + ) + .await + .expect_err("template create should reject missing spec"); + + assert_eq!(err.code(), tonic::Code::InvalidArgument); + assert!(err.message().contains("sandbox template spec")); + } + + #[test] + fn sandbox_template_validation_allows_positive_ready_within() { + let mut template = test_workload_template("gpu-kata"); + template.metadata.as_mut().unwrap().id = "template-gpu-kata".to_string(); + template.spec.as_mut().unwrap().desired_service_level = + Some(openshell_core::proto::SandboxServiceLevel { + startup: Some(openshell_core::proto::SandboxStartup { + ready_within: Some(prost_types::Duration { + seconds: 1, + nanos: 0, + }), + max_burst: 1, + }), + }); + + validate_sandbox_workload_template(&template).expect("positive ready_within should pass"); + } + + #[test] + fn sandbox_template_validation_rejects_non_positive_ready_within() { + for (duration, expected) in [ + ( + prost_types::Duration { + seconds: 0, + nanos: 0, + }, + "greater than zero", + ), + ( + prost_types::Duration { + seconds: -1, + nanos: 0, + }, + "greater than zero", + ), + ( + prost_types::Duration { + seconds: 0, + nanos: -1, + }, + "greater than zero", + ), + ] { + let mut template = test_workload_template("gpu-kata"); + template.metadata.as_mut().unwrap().id = "template-gpu-kata".to_string(); + template.spec.as_mut().unwrap().desired_service_level = + Some(openshell_core::proto::SandboxServiceLevel { + startup: Some(openshell_core::proto::SandboxStartup { + ready_within: Some(duration), + max_burst: 1, + }), + }); + + let err = validate_sandbox_workload_template(&template) + .expect_err("non-positive ready_within should be rejected"); + assert_eq!(err.code(), tonic::Code::InvalidArgument); + assert!(err.message().contains(expected), "{err:?}"); + assert!( + err.message() + .contains("template.spec.desired_service_level.startup.ready_within"), + "{err:?}" + ); + } + } + + #[test] + fn sandbox_template_validation_rejects_malformed_ready_within() { + for (duration, expected) in [ + ( + prost_types::Duration { + seconds: 1, + nanos: -1, + }, + "normalized", + ), + ( + prost_types::Duration { + seconds: 0, + nanos: 1_000_000_000, + }, + "valid protobuf Duration", + ), + ( + prost_types::Duration { + seconds: 315_576_000_001, + nanos: 0, + }, + "valid protobuf Duration", + ), + ] { + let mut template = test_workload_template("gpu-kata"); + template.metadata.as_mut().unwrap().id = "template-gpu-kata".to_string(); + template.spec.as_mut().unwrap().desired_service_level = + Some(openshell_core::proto::SandboxServiceLevel { + startup: Some(openshell_core::proto::SandboxStartup { + ready_within: Some(duration), + max_burst: 1, + }), + }); + + let err = validate_sandbox_workload_template(&template) + .expect_err("malformed ready_within should be rejected"); + assert_eq!(err.code(), tonic::Code::InvalidArgument); + assert!(err.message().contains(expected), "{err:?}"); + assert!( + err.message() + .contains("template.spec.desired_service_level.startup.ready_within"), + "{err:?}" + ); + } + } + + #[tokio::test] + async fn sandbox_template_create_rejects_workspace_quota() { + let state = test_server_state().await; + for index in 0..MAX_TEMPLATES_PER_WORKSPACE { + let mut template = test_workload_template(&format!("tmpl-{index}")); + let metadata = template.metadata.as_mut().expect("metadata"); + metadata.id = format!("template-{index}"); + metadata.workspace = "default".to_string(); + state.store.put_message(&template).await.unwrap(); + } + + let err = handle_create_sandbox_template( + &state, + authed_request(CreateSandboxTemplateRequest { + template: Some(test_workload_template("overflow")), + workspace: "default".to_string(), + }), + ) + .await + .expect_err("template create must reject a full workspace"); + + assert_eq!(err.code(), tonic::Code::ResourceExhausted); + assert!(err.message().contains("1000 sandbox templates")); + } + + #[tokio::test] + async fn sandbox_template_create_enforces_workspace_quota_concurrently() { + let state = test_server_state().await; + for index in 0..(MAX_TEMPLATES_PER_WORKSPACE - 1) { + let mut template = test_workload_template(&format!("tmpl-{index}")); + let metadata = template.metadata.as_mut().expect("metadata"); + metadata.id = format!("template-{index}"); + metadata.workspace = "default".to_string(); + state.store.put_message(&template).await.unwrap(); + } + + let mut handles = vec![]; + for index in 0..8 { + let state = Arc::clone(&state); + let handle = tokio::spawn(async move { + handle_create_sandbox_template( + &state, + authed_request(CreateSandboxTemplateRequest { + template: Some(test_workload_template(&format!("overflow-{index}"))), + workspace: "default".to_string(), + }), + ) + .await + }); + handles.push(handle); + } + + let results: Vec<_> = future::join_all(handles) + .await + .into_iter() + .map(|r| r.unwrap()) + .collect(); + + let successes = results.iter().filter(|r| r.is_ok()).count(); + let exhausted = results + .iter() + .filter(|r| { + r.as_ref() + .err() + .is_some_and(|e| e.code() == tonic::Code::ResourceExhausted) + }) + .count(); + + assert_eq!(successes, 1); + assert_eq!(exhausted, 7); + let count = state + .store + .count_in_workspace(SandboxWorkloadTemplate::object_type(), "default") + .await + .unwrap(); + assert_eq!(count, u64::from(MAX_TEMPLATES_PER_WORKSPACE)); + } + + #[test] + fn template_create_sandbox_spec_field_policy_is_exhaustive() { + assert_proto_fields_classified( + "openshell.v1.SandboxSpec", + &["policy", "providers", "command", "tty"], + &[ + "log_level", + "environment", + "template", + "resource_requirements", + ], + ); + } + + fn assert_proto_fields_classified( + message_name: &str, + copied_from_create_request: &[&str], + rejected_template_workload_overrides: &[&str], + ) { + let pool = prost_reflect::DescriptorPool::decode(openshell_core::FILE_DESCRIPTOR_SET) + .expect("decode descriptor set"); + let message = pool + .get_message_by_name(message_name) + .expect("message descriptor"); + let classified: std::collections::HashSet<&str> = copied_from_create_request + .iter() + .chain(rejected_template_workload_overrides.iter()) + .copied() + .collect(); + let actual: std::collections::HashSet = message + .fields() + .map(|field| field.name().to_string()) + .collect(); + + for field in &actual { + assert!( + classified.contains(field.as_str()), + "{message_name}.{field} is not classified for template-backed sandbox creates. \ + Add it to copied_from_create_request when callers own the create-time value, \ + or to rejected_template_workload_overrides when the workload template owns it." + ); + } + + for field in classified { + assert!( + actual.contains(field), + "{message_name}.{field} is classified for template-backed sandbox creates, \ + but the proto field no longer exists" + ); + } + } + + #[tokio::test] + async fn create_sandbox_from_workload_template_resolves_workload_and_preserves_governance() { + let state = test_server_state().await; + state + .store + .put_message(&test_provider("work-github", "github")) + .await + .unwrap(); + handle_create_sandbox_template( + &state, + authed_request(CreateSandboxTemplateRequest { + template: Some(test_workload_template("gpu-kata")), + workspace: "default".to_string(), + }), + ) + .await + .expect("template create should succeed"); + + let mut policy = openshell_core::proto::SandboxPolicy { + version: 1, + ..Default::default() + }; + policy.network_policies.insert( + "example".to_string(), + openshell_core::proto::NetworkPolicyRule { + name: "example".to_string(), + ..Default::default() + }, + ); + + let created = handle_create_sandbox( + &state, + authed_request(CreateSandboxRequest { + name: "from-template".to_string(), + spec: Some(SandboxSpec { + providers: vec!["work-github".to_string()], + policy: Some(policy), + command: vec!["echo".to_string(), "template-create".to_string()], + tty: false, + ..Default::default() + }), + labels: HashMap::new(), + annotations: HashMap::new(), + workspace: "default".to_string(), + workload_template_name: "gpu-kata".to_string(), + await_main_process_attachment: false, + }), + ) + .await + .expect("sandbox create from template should succeed") + .into_inner() + .sandbox + .expect("created sandbox"); + + let provenance = created + .created_from_workload_template + .expect("template provenance"); + assert_eq!(provenance.name, "gpu-kata"); + assert!(!provenance.resource_version.is_empty()); + + let spec = created.spec.expect("resolved sandbox spec"); + assert_eq!(spec.providers, vec!["work-github".to_string()]); + assert!(spec.policy.is_some()); + assert_eq!( + spec.command, + vec!["echo".to_string(), "template-create".to_string()] + ); + assert!(!spec.tty); + assert_eq!( + spec.environment.get("FEATURE_FLAG"), + Some(&"on".to_string()) + ); + + let template = spec.template.expect("resolved inline template"); + assert_eq!(template.image, "registry.example.com/agent:latest"); + let limits = template + .resources + .as_ref() + .and_then(|resources| resources.fields.get("limits")) + .and_then(|limits| limits.kind.as_ref()) + .and_then(|kind| match kind { + Kind::StructValue(value) => Some(&value.fields), + _ => None, + }) + .expect("resource limits"); + assert_eq!(limits.get("cpu").and_then(proto_string_value), Some("2")); + assert_eq!( + limits.get("memory").and_then(proto_string_value), + Some("4Gi") + ); + assert_eq!( + spec.resource_requirements + .and_then(|requirements| requirements.gpu) + .and_then(|gpu| gpu.count), + Some(1) + ); + } + + #[tokio::test] + async fn create_sandbox_from_workload_template_defaults_whitespace_image() { + let state = test_server_state().await; + let mut template = test_workload_template("default-image"); + template + .spec + .as_mut() + .and_then(|spec| spec.workload.as_mut()) + .expect("test template workload") + .image = " ".to_string(); + handle_create_sandbox_template( + &state, + authed_request(CreateSandboxTemplateRequest { + template: Some(template), + workspace: "default".to_string(), + }), + ) + .await + .expect("template create should succeed"); + + let created = handle_create_sandbox( + &state, + authed_request(CreateSandboxRequest { + name: "from-template".to_string(), + spec: Some(SandboxSpec::default()), + labels: HashMap::new(), + annotations: HashMap::new(), + workspace: "default".to_string(), + workload_template_name: "default-image".to_string(), + await_main_process_attachment: false, + }), + ) + .await + .expect("sandbox create from template should succeed") + .into_inner() + .sandbox + .expect("created sandbox"); + + let image = created + .spec + .and_then(|spec| spec.template) + .map(|template| template.image) + .expect("resolved template image"); + assert_eq!(image, state.compute.default_image()); + } + + #[tokio::test] + async fn create_sandbox_from_workload_template_preserves_default_gpu_request() { + let state = test_server_state().await; + let mut template = test_workload_template("default-gpu"); + template + .spec + .as_mut() + .and_then(|spec| spec.workload.as_mut()) + .and_then(|workload| workload.resources.as_mut()) + .expect("test template resources") + .gpu = Some(GpuResourceRequirements { count: None }); + handle_create_sandbox_template( + &state, + authed_request(CreateSandboxTemplateRequest { + template: Some(template), + workspace: "default".to_string(), + }), + ) + .await + .expect("template create should succeed"); + + let created = handle_create_sandbox( + &state, + authed_request(CreateSandboxRequest { + name: "from-template".to_string(), + spec: Some(SandboxSpec::default()), + labels: HashMap::new(), + annotations: HashMap::new(), + workspace: "default".to_string(), + workload_template_name: "default-gpu".to_string(), + await_main_process_attachment: false, + }), + ) + .await + .expect("sandbox create from template should succeed") + .into_inner() + .sandbox + .expect("created sandbox"); + + let gpu = created + .spec + .as_ref() + .and_then(|spec| spec.resource_requirements.as_ref()) + .and_then(|requirements| requirements.gpu.as_ref()) + .expect("default GPU request should be preserved"); + assert_eq!(gpu.count, None); + } + + #[tokio::test] + async fn create_sandbox_from_corrupted_workload_template_returns_internal() { + let state = test_server_state().await; + let mut template = test_workload_template("corrupt-template"); + let metadata = template.metadata.as_mut().expect("metadata"); + metadata.id = "template-corrupt-template".to_string(); + metadata.workspace = "default".to_string(); + template.spec = None; + state.store.put_message(&template).await.unwrap(); + + let err = handle_create_sandbox( + &state, + authed_request(CreateSandboxRequest { + name: "from-corrupt".to_string(), + spec: Some(SandboxSpec::default()), + labels: HashMap::new(), + annotations: HashMap::new(), + workspace: "default".to_string(), + workload_template_name: "corrupt-template".to_string(), + await_main_process_attachment: false, + }), + ) + .await + .expect_err("corrupted stored template should fail as server data corruption"); + + assert_eq!(err.code(), tonic::Code::Internal, "{}", err.message()); + assert!(err.message().contains("sandbox template spec")); + } + + #[tokio::test] + async fn create_sandbox_from_workload_template_rejects_inline_workload_overrides() { + let state = test_server_state().await; + handle_create_sandbox_template( + &state, + authed_request(CreateSandboxTemplateRequest { + template: Some(test_workload_template("gpu-kata")), + workspace: "default".to_string(), + }), + ) + .await + .expect("template create should succeed"); + + let err = handle_create_sandbox( + &state, + authed_request(CreateSandboxRequest { + name: "bad-template-create".to_string(), + spec: Some(SandboxSpec { + environment: HashMap::from([("INLINE".to_string(), "blocked".to_string())]), + ..Default::default() + }), + labels: HashMap::new(), + annotations: HashMap::new(), + workspace: "default".to_string(), + workload_template_name: "gpu-kata".to_string(), + await_main_process_attachment: false, + }), + ) + .await + .expect_err("inline workload overrides should be rejected"); + + assert_eq!(err.code(), tonic::Code::InvalidArgument); + assert!(err.message().contains("spec.environment")); + } + + #[tokio::test] + async fn create_sandbox_from_workload_template_rejects_malformed_template_name() { + let state = test_server_state().await; + + let err = handle_create_sandbox( + &state, + authed_request(CreateSandboxRequest { + name: "bad-template-create".to_string(), + spec: Some(SandboxSpec::default()), + labels: HashMap::new(), + annotations: HashMap::new(), + workspace: "default".to_string(), + workload_template_name: "Invalid_Template_Name".to_string(), + await_main_process_attachment: false, + }), + ) + .await + .expect_err("malformed template name should be rejected before lookup"); + + assert_eq!(err.code(), tonic::Code::InvalidArgument); + assert!(err.message().contains("workload_template_name")); + } + + #[tokio::test] + async fn create_sandbox_from_workload_template_rejects_oversized_governance_before_lookup() { + let state = test_server_state().await; + + let err = handle_create_sandbox( + &state, + authed_request(CreateSandboxRequest { + name: "bad-template-create".to_string(), + spec: Some(SandboxSpec { + providers: (0..=MAX_PROVIDERS).map(|i| format!("p-{i}")).collect(), + ..Default::default() + }), + labels: HashMap::new(), + annotations: HashMap::new(), + workspace: "default".to_string(), + workload_template_name: "missing-template".to_string(), + await_main_process_attachment: false, + }), + ) + .await + .expect_err("oversized governance spec should be rejected before template lookup"); + + assert_eq!(err.code(), tonic::Code::InvalidArgument); + assert!(err.message().contains("providers")); + } + + #[tokio::test] + async fn create_sandbox_rejects_oversized_direct_spec_before_workspace_lookup() { + let state = test_server_state().await; + + let err = handle_create_sandbox( + &state, + authed_request(CreateSandboxRequest { + name: "bad-direct-create".to_string(), + spec: Some(SandboxSpec { + providers: (0..=MAX_PROVIDERS).map(|i| format!("p-{i}")).collect(), + ..Default::default() + }), + labels: HashMap::new(), + annotations: HashMap::new(), + workspace: "missing-workspace".to_string(), + workload_template_name: String::new(), + await_main_process_attachment: false, + }), + ) + .await + .expect_err("oversized direct spec should be rejected before workspace lookup"); + + assert_eq!(err.code(), tonic::Code::InvalidArgument); + assert!(err.message().contains("providers")); + } + #[tokio::test] async fn attach_sandbox_provider_rejects_credential_key_collisions() { let state = test_server_state().await; @@ -4678,7 +6064,7 @@ mod tests { &state, non_member_request(CreateSandboxRequest { workspace: "no-such-ws".into(), - spec: Some(openshell_core::proto::SandboxSpec::default()), + spec: Some(SandboxSpec::default()), ..Default::default() }), ) diff --git a/crates/openshell-server/src/grpc/validation.rs b/crates/openshell-server/src/grpc/validation.rs index eb53c80f29..c20dad7780 100644 --- a/crates/openshell-server/src/grpc/validation.rs +++ b/crates/openshell-server/src/grpc/validation.rs @@ -10,7 +10,7 @@ use openshell_core::proto::{ CredentialHandle, ExecSandboxRequest, Provider, SandboxPolicy as ProtoSandboxPolicy, - SandboxTemplate, + SandboxSpec, SandboxTemplate, }; use prost::Message; use tonic::Status; @@ -150,26 +150,12 @@ pub(super) fn validate_dns1123_label(name: &str, field: &str) -> Result<(), Stat /// Validate field sizes on a `CreateSandboxRequest` before persisting. /// /// Returns `INVALID_ARGUMENT` on the first field that exceeds its limit. -pub(super) fn validate_sandbox_spec( - name: &str, - spec: &openshell_core::proto::SandboxSpec, -) -> Result<(), Status> { +pub(super) fn validate_sandbox_spec(name: &str, spec: &SandboxSpec) -> Result<(), Status> { // --- request.name --- - if !name.is_empty() && name.len() > MAX_ROUTABLE_NAME_LEN { - return Err(Status::invalid_argument(format!( - "name exceeds maximum length ({} > {MAX_ROUTABLE_NAME_LEN})", - name.len() - ))); - } - validate_dns1123_label(name, "name")?; + validate_sandbox_name(name)?; // --- spec.providers --- - if spec.providers.len() > MAX_PROVIDERS { - return Err(Status::invalid_argument(format!( - "providers list exceeds maximum ({} > {MAX_PROVIDERS})", - spec.providers.len() - ))); - } + validate_sandbox_provider_count(spec)?; // --- spec.log_level --- if spec.log_level.len() > MAX_LOG_LEVEL_LEN { @@ -203,6 +189,45 @@ pub(super) fn validate_sandbox_spec( } // --- spec.policy serialized size --- + validate_sandbox_policy_size(spec)?; + + Ok(()) +} + +pub(super) fn validate_sandbox_governance_spec( + name: &str, + spec: &SandboxSpec, +) -> Result<(), Status> { + validate_sandbox_name(name)?; + validate_sandbox_provider_count(spec)?; + if !spec.command.is_empty() { + validate_main_process_command(&spec.command)?; + } + validate_sandbox_policy_size(spec)?; + Ok(()) +} + +fn validate_sandbox_name(name: &str) -> Result<(), Status> { + if !name.is_empty() && name.len() > MAX_ROUTABLE_NAME_LEN { + return Err(Status::invalid_argument(format!( + "name exceeds maximum length ({} > {MAX_ROUTABLE_NAME_LEN})", + name.len() + ))); + } + validate_dns1123_label(name, "name") +} + +fn validate_sandbox_provider_count(spec: &SandboxSpec) -> Result<(), Status> { + if spec.providers.len() > MAX_PROVIDERS { + return Err(Status::invalid_argument(format!( + "providers list exceeds maximum ({} > {MAX_PROVIDERS})", + spec.providers.len() + ))); + } + Ok(()) +} + +fn validate_sandbox_policy_size(spec: &SandboxSpec) -> Result<(), Status> { if let Some(ref policy) = spec.policy { let size = policy.encoded_len(); if size > MAX_POLICY_SIZE { @@ -244,7 +269,7 @@ fn validate_main_process_command(command: &[String]) -> Result<(), Status> { Ok(()) } -fn validate_gpu_request_fields(spec: &openshell_core::proto::SandboxSpec) -> Result<(), Status> { +fn validate_gpu_request_fields(spec: &SandboxSpec) -> Result<(), Status> { if openshell_core::gpu::sandbox_gpu_count(spec.resource_requirements.as_ref()) == Some(0) { return Err(Status::invalid_argument("gpu count must be greater than 0")); } diff --git a/crates/openshell-server/src/grpc/workspace.rs b/crates/openshell-server/src/grpc/workspace.rs index d83ffab0e8..7446938156 100644 --- a/crates/openshell-server/src/grpc/workspace.rs +++ b/crates/openshell-server/src/grpc/workspace.rs @@ -14,9 +14,9 @@ use openshell_core::proto::{ CreateWorkspaceResponse, DeleteWorkspaceRequest, DeleteWorkspaceResponse, GetWorkspaceRequest, GetWorkspaceResponse, InferenceRoute, ListWorkspaceMembersRequest, ListWorkspaceMembersResponse, ListWorkspacesRequest, ListWorkspacesResponse, Provider, - RemoveWorkspaceMemberRequest, RemoveWorkspaceMemberResponse, Sandbox, ServiceEndpoint, - SshSession, StoredProviderCredentialRefreshState, StoredProviderProfile, Workspace, - WorkspaceMember, WorkspaceRole, + RemoveWorkspaceMemberRequest, RemoveWorkspaceMemberResponse, Sandbox, SandboxWorkloadTemplate, + ServiceEndpoint, SshSession, StoredProviderCredentialRefreshState, StoredProviderProfile, + Workspace, WorkspaceMember, WorkspaceRole, }; use prost::Message; use tonic::{Request, Response, Status}; @@ -375,6 +375,7 @@ pub(super) async fn handle_delete_workspace( let mut blocking = Vec::new(); for (object_type, label) in [ (Sandbox::object_type(), "sandbox"), + (SandboxWorkloadTemplate::object_type(), "sandbox template"), (Provider::object_type(), "provider"), (StoredProviderProfile::object_type(), "provider profile"), (ServiceEndpoint::object_type(), "service"), @@ -819,6 +820,72 @@ mod tests { assert!(resp.deleted); } + #[tokio::test] + async fn delete_workspace_blocked_by_sandbox_template() { + let state = test_server_state().await; + + handle_create_workspace( + &state, + Request::new(CreateWorkspaceRequest { + name: "templated".to_string(), + labels: HashMap::new(), + }), + ) + .await + .unwrap(); + + let template = SandboxWorkloadTemplate { + metadata: Some(ObjectMeta { + id: "template-1".to_string(), + name: "gpu-kata".to_string(), + created_at_ms: 1_000_000, + labels: HashMap::new(), + annotations: HashMap::new(), + resource_version: 0, + workspace: "templated".to_string(), + deletion_timestamp_ms: 0, + }), + spec: None, + }; + state.store.put_message(&template).await.unwrap(); + + let err = handle_delete_workspace( + &state, + Request::new(DeleteWorkspaceRequest { + name: "templated".to_string(), + }), + ) + .await + .unwrap_err(); + assert_eq!(err.code(), Code::FailedPrecondition); + assert!( + err.message().contains("sandbox template"), + "error should name sandbox templates as blocking resources: {}", + err.message() + ); + + state + .store + .delete_by_name( + SandboxWorkloadTemplate::object_type(), + "templated", + "gpu-kata", + ) + .await + .unwrap(); + + let resp = handle_delete_workspace( + &state, + Request::new(DeleteWorkspaceRequest { + name: "templated".to_string(), + }), + ) + .await + .unwrap() + .into_inner(); + assert!(resp.deleted); + } + #[tokio::test] async fn delete_workspace_blocked_by_ssh_session() { let state = test_server_state().await; diff --git a/crates/openshell-server/src/persistence/mod.rs b/crates/openshell-server/src/persistence/mod.rs index 6934630110..716f26dad9 100644 --- a/crates/openshell-server/src/persistence/mod.rs +++ b/crates/openshell-server/src/persistence/mod.rs @@ -401,6 +401,39 @@ impl Store { )) } + /// Atomically insert a named object only if its workspace has fewer than + /// `max_count` objects of the same type. + /// + /// Returns `Ok(None)` when the quota is already full. A duplicate id or + /// `(object_type, workspace, name)` still returns + /// [`PersistenceError::UniqueViolation`]. + #[allow(clippy::too_many_arguments)] + #[tracing::instrument( + name = "store", + skip_all, + fields(otel.name = "store.create_if_workspace_count_below", otel.status_code = tracing::field::Empty, object_type = %object_type, object.id = %id, object.name = %name, workspace = %workspace, max_count = max_count) + )] + pub async fn create_if_workspace_count_below( + &self, + object_type: &str, + id: &str, + name: &str, + workspace: &str, + payload: &[u8], + labels: Option<&str>, + max_count: u64, + ) -> PersistenceResult> { + store_dispatch_traced!(self.create_if_workspace_count_below( + object_type, + id, + name, + workspace, + payload, + labels, + max_count + )) + } + /// Fetch an object by id. #[tracing::instrument( name = "store", diff --git a/crates/openshell-server/src/persistence/postgres.rs b/crates/openshell-server/src/persistence/postgres.rs index 89fe442695..19c50c6187 100644 --- a/crates/openshell-server/src/persistence/postgres.rs +++ b/crates/openshell-server/src/persistence/postgres.rs @@ -337,6 +337,73 @@ RETURNING resource_version, created_at_ms, updated_at_ms }) } + #[allow(clippy::too_many_arguments)] + pub async fn create_if_workspace_count_below( + &self, + object_type: &str, + id: &str, + name: &str, + workspace: &str, + payload: &[u8], + labels: Option<&str>, + max_count: u64, + ) -> PersistenceResult> { + let now_ms = current_time_ms(); + let labels_jsonb: Option = labels + .map(serde_json::from_str) + .transpose() + .map_err(|e| PersistenceError::Encode(format!("invalid labels JSON: {e}")))?; + let mut tx = self.pool.begin().await.map_err(|e| map_db_error(&e))?; + + sqlx::query("SELECT pg_advisory_xact_lock(hashtext($1), hashtext($2))") + .bind(object_type) + .bind(workspace) + .execute(&mut *tx) + .await + .map_err(|e| map_db_error(&e))?; + + let row: (i64,) = sqlx::query_as( + "SELECT COUNT(*) FROM objects WHERE object_type = $1 AND workspace = $2", + ) + .bind(object_type) + .bind(workspace) + .fetch_one(&mut *tx) + .await + .map_err(|e| map_db_error(&e))?; + let count = u64::try_from(row.0).unwrap_or(0); + if count >= max_count { + tx.commit().await.map_err(|e| map_db_error(&e))?; + return Ok(None); + } + + let row = sqlx::query( + r" +INSERT INTO objects (object_type, id, name, workspace, payload, created_at_ms, updated_at_ms, labels, resource_version) +VALUES ($1, $2, $3, $4, $5, $6, $6, COALESCE($7, '{}'::jsonb), 1) +RETURNING resource_version, created_at_ms, updated_at_ms +", + ) + .bind(object_type) + .bind(id) + .bind(name) + .bind(workspace) + .bind(payload) + .bind(now_ms) + .bind(labels_jsonb) + .fetch_one(&mut *tx) + .await + .map_err(|e| map_db_error(&e))?; + + tx.commit().await.map_err(|e| map_db_error(&e))?; + + let resource_version_i64: i64 = row.try_get("resource_version").unwrap_or(1); + Ok(Some(WriteResult { + resource_version: resource_version_i64.max(1).cast_unsigned(), + created_at_ms: row.get("created_at_ms"), + updated_at_ms: row.get("updated_at_ms"), + })) + } + pub async fn get( &self, object_type: &str, diff --git a/crates/openshell-server/src/persistence/sqlite.rs b/crates/openshell-server/src/persistence/sqlite.rs index c28318d261..c945c417f9 100644 --- a/crates/openshell-server/src/persistence/sqlite.rs +++ b/crates/openshell-server/src/persistence/sqlite.rs @@ -357,6 +357,67 @@ VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?7, ?8, 1) }) } + #[allow(clippy::too_many_arguments)] + pub async fn create_if_workspace_count_below( + &self, + object_type: &str, + id: &str, + name: &str, + workspace: &str, + payload: &[u8], + labels: Option<&str>, + max_count: u64, + ) -> PersistenceResult> { + let now_ms = current_time_ms(); + let mut tx = self + .pool + .begin_with("BEGIN IMMEDIATE") + .await + .map_err(|e| map_db_error(&e))?; + + let row: (i64,) = sqlx::query_as( + r#" +SELECT COUNT(*) FROM "objects" +WHERE "object_type" = ?1 AND "workspace" = ?2 +"#, + ) + .bind(object_type) + .bind(workspace) + .fetch_one(&mut *tx) + .await + .map_err(|e| map_db_error(&e))?; + let count = u64::try_from(row.0).unwrap_or(0); + if count >= max_count { + tx.commit().await.map_err(|e| map_db_error(&e))?; + return Ok(None); + } + + sqlx::query( + r#" +INSERT INTO "objects" ("object_type", "id", "name", "workspace", "payload", "created_at_ms", "updated_at_ms", "labels", "resource_version") +VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?6, ?7, 1) +"#, + ) + .bind(object_type) + .bind(id) + .bind(name) + .bind(workspace) + .bind(payload) + .bind(now_ms) + .bind(labels.unwrap_or("{}")) + .execute(&mut *tx) + .await + .map_err(|e| map_db_error(&e))?; + + tx.commit().await.map_err(|e| map_db_error(&e))?; + + Ok(Some(WriteResult { + resource_version: 1, + created_at_ms: now_ms, + updated_at_ms: now_ms, + })) + } + pub async fn get( &self, object_type: &str, diff --git a/crates/openshell-server/src/persistence/tests.rs b/crates/openshell-server/src/persistence/tests.rs index 479f46e493..7882c9246c 100644 --- a/crates/openshell-server/src/persistence/tests.rs +++ b/crates/openshell-server/src/persistence/tests.rs @@ -1819,6 +1819,7 @@ async fn cas_update_message_cas_succeeds() { }), spec: None, status: None, + ..Sandbox::default() }; store.put_message(&sandbox).await.unwrap(); @@ -1861,6 +1862,7 @@ async fn cas_update_message_cas_conflicts_on_concurrent_updates() { }), spec: None, status: None, + ..Sandbox::default() }; store.put_message(&sandbox).await.unwrap(); @@ -1931,6 +1933,7 @@ async fn cas_update_message_cas_rejects_workspace_change() { }), spec: None, status: None, + ..Sandbox::default() }; store.put_message(&sandbox).await.unwrap(); @@ -1973,6 +1976,7 @@ async fn cas_update_message_cas_rejects_name_change() { }), spec: None, status: None, + ..Sandbox::default() }; store.put_message(&sandbox).await.unwrap(); diff --git a/crates/openshell-server/tests/common/mod.rs b/crates/openshell-server/tests/common/mod.rs index 9e3957abb6..a60fc8696b 100644 --- a/crates/openshell-server/tests/common/mod.rs +++ b/crates/openshell-server/tests/common/mod.rs @@ -98,6 +98,34 @@ impl OpenShell for TestOpenShell { Ok(Response::new(SandboxResponse::default())) } + async fn create_sandbox_template( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("unused")) + } + + async fn get_sandbox_template( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("unused")) + } + + async fn list_sandbox_templates( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("unused")) + } + + async fn delete_sandbox_template( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("unused")) + } + async fn stop_sandbox( &self, _request: tonic::Request, diff --git a/crates/openshell-server/tests/supervisor_relay_integration.rs b/crates/openshell-server/tests/supervisor_relay_integration.rs index 6cfd3b009b..91ac50dcbd 100644 --- a/crates/openshell-server/tests/supervisor_relay_integration.rs +++ b/crates/openshell-server/tests/supervisor_relay_integration.rs @@ -82,6 +82,34 @@ impl OpenShell for RelayGateway { // ------ unused stubs ------ + async fn create_sandbox_template( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("unused")) + } + + async fn get_sandbox_template( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("unused")) + } + + async fn list_sandbox_templates( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("unused")) + } + + async fn delete_sandbox_template( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("unused")) + } + type ConnectSupervisorStream = ReceiverStream>; async fn connect_supervisor( &self, diff --git a/crates/openshell-supervisor-network/src/upstream_proxy.rs b/crates/openshell-supervisor-network/src/upstream_proxy.rs index 253fa52263..f95c5a3e81 100644 --- a/crates/openshell-supervisor-network/src/upstream_proxy.rs +++ b/crates/openshell-supervisor-network/src/upstream_proxy.rs @@ -612,31 +612,12 @@ fn parse_proxy_url(raw: &str, var_name: &str) -> Result<(ProxyEndpoint, bool), S /// fall-back to the built-in roots that would quietly weaken the trust /// boundary. The error names `var_name` so the operator can locate the setting. pub(crate) fn read_proxy_ca_bundle(path: &str, var_name: &str) -> Result { - let pem = std::fs::read_to_string(path) - .map_err(|err| format!("{var_name} '{path}' could not be read: {err}"))?; - // Validate that the bundle contributes at least one trust anchor that - // rustls actually accepts, not just that PEM framing base64-decodes. - // A PEM block with invalid DER passes `rustls_pemfile::certs` but is - // silently rejected by `RootCertStore::add_parsable_certificates`; - // counting only PEM blocks would let such a bundle satisfy the check - // while contributing zero usable anchors at runtime. - let certs: Vec<_> = rustls_pemfile::certs(&mut pem.as_bytes()) - .flatten() - .collect(); - if certs.is_empty() { - return Err(format!( - "{var_name} '{path}' contains no PEM certificate blocks" - )); - } - let mut store = rustls::RootCertStore::empty(); - let (added, _ignored) = store.add_parsable_certificates(certs); - if added == 0 { - return Err(format!( - "{var_name} '{path}' contains no usable trust anchors \ - (PEM blocks were found but none contain valid X.509 DER)" - )); - } - Ok(pem) + // Shared with the compute driver, which validates the same file on the + // gateway host before staging it, so host acceptance and guest acceptance + // cannot diverge. It also bounds the read: the file arrives from the + // driver, but a bundle the size of the sandbox disk should fail rather + // than be loaded whole. + openshell_core::driver_utils::read_upstream_proxy_ca_bundle_file(path, var_name) } /// Build the TLS client config used to connect to an `https://` corporate diff --git a/crates/openshell-tui/src/app.rs b/crates/openshell-tui/src/app.rs index 13a00aaca3..cf7464fcd8 100644 --- a/crates/openshell-tui/src/app.rs +++ b/crates/openshell-tui/src/app.rs @@ -1407,6 +1407,9 @@ impl App { self.input_mode = InputMode::Command; self.command_input.clear(); } + KeyCode::Char('w') => { + self.cycle_workspace(); + } KeyCode::Char('j') | KeyCode::Down => { if self.provider_count > 0 && self.provider_selected < self.provider_count - 1 { self.provider_selected += 1; @@ -3825,6 +3828,32 @@ mod tests { assert_eq!(gateway.source_label(), "unknown"); } + #[tokio::test] + async fn providers_workspace_shortcut_cycles_scope_and_resets_selections() { + let mut app = test_app(); + app.screen = Screen::Dashboard; + app.focus = Focus::Providers; + app.workspace_names = vec!["default".to_string(), "team-b".to_string()]; + app.provider_selected = 3; + app.sandbox_selected = 4; + + app.handle_key(key(KeyCode::Char('w'))); + + assert_eq!(app.current_workspace, "team-b"); + assert!(!app.all_workspaces); + assert_eq!(app.provider_selected, 0); + assert_eq!(app.sandbox_selected, 0); + assert!(app.pending_workspace_refresh); + + app.handle_key(key(KeyCode::Char('w'))); + assert!(app.all_workspaces); + assert_eq!(app.workspace_display(), "all"); + + app.handle_key(key(KeyCode::Char('w'))); + assert!(!app.all_workspaces); + assert_eq!(app.current_workspace, "default"); + } + // -- selected_sandbox_workspace ---------------------------------------- #[test] diff --git a/crates/openshell-tui/src/lib.rs b/crates/openshell-tui/src/lib.rs index f08f44a3ac..174f9910d0 100644 --- a/crates/openshell-tui/src/lib.rs +++ b/crates/openshell-tui/src/lib.rs @@ -1405,6 +1405,7 @@ fn spawn_create_sandbox(app: &mut App, tx: mpsc::UnboundedSender) { annotations: HashMap::new(), workspace: workspace.clone(), await_main_process_attachment: false, + workload_template_name: String::new(), }; let sandbox_name = diff --git a/crates/openshell-tui/src/ui/mod.rs b/crates/openshell-tui/src/ui/mod.rs index 7ad1589186..2521184320 100644 --- a/crates/openshell-tui/src/ui/mod.rs +++ b/crates/openshell-tui/src/ui/mod.rs @@ -143,10 +143,8 @@ fn draw_title_bar(frame: &mut Frame<'_>, app: &App, area: Rect) { .find(|gateway| gateway.name == app.gateway_name) .map_or("unknown", app::GatewayEntry::source_label); - let mut parts: Vec> = vec![ - Span::styled(" >_ OpenShell ", t.accent_bold), - Span::styled(" ALPHA ", t.badge), - Span::styled(" | ", t.muted), + let mut parts: Vec> = title_bar_brand_spans(t); + parts.extend([ Span::styled("Current Gateway: ", t.text), Span::styled(&app.gateway_name, t.heading), Span::styled(" [", t.muted), @@ -155,7 +153,7 @@ fn draw_title_bar(frame: &mut Frame<'_>, app: &App, area: Rect) { status_span, Span::styled(")", t.muted), Span::styled(" | ", t.muted), - ]; + ]); parts.push(Span::styled("Workspace: ", t.text)); parts.push(Span::styled(app.workspace_display(), t.heading)); @@ -180,6 +178,14 @@ fn draw_title_bar(frame: &mut Frame<'_>, app: &App, area: Rect) { frame.render_widget(Paragraph::new(title).style(t.title_bar), area); } +fn title_bar_brand_spans(theme: &Theme) -> Vec> { + vec![ + Span::styled(" >_ OpenShell ", theme.accent_bold), + Span::styled(format!("v{}", openshell_core::VERSION), theme.muted), + Span::styled(" | ", theme.muted), + ] +} + fn draw_nav_bar(frame: &mut Frame<'_>, app: &App, area: Rect) { let t = &app.theme; let spans = match app.screen { @@ -223,6 +229,9 @@ fn draw_nav_bar(frame: &mut Frame<'_>, app: &App, area: Rect) { Span::styled(" ", t.text), Span::styled("[c/u/d]", t.key_hint), Span::styled(" Create/Update/Delete", t.text), + Span::styled(" ", t.text), + Span::styled("[w]", t.key_hint), + Span::styled(" Workspace", t.text), Span::styled(" | ", t.border), Span::styled("[:]", t.muted), Span::styled(" Command ", t.muted), @@ -662,3 +671,70 @@ pub fn centered_popup(percent_x: u16, height: u16, area: Rect) -> Rect { ]) .split(vert[1])[1] } + +#[cfg(test)] +mod tests { + use super::*; + use openshell_core::auth::EdgeAuthInterceptor; + use openshell_core::proto::open_shell_client::OpenShellClient; + use ratatui::Terminal; + use ratatui::backend::TestBackend; + + fn test_app() -> App { + let channel = tonic::transport::Endpoint::from_static("http://127.0.0.1:1").connect_lazy(); + let client = OpenShellClient::with_interceptor(channel, EdgeAuthInterceptor::noop()); + let mut app = App::new( + client, + "test".to_string(), + "http://127.0.0.1:1".to_string(), + "default".to_string(), + Theme::dark(), + ); + app.screen = Screen::Dashboard; + app.focus = Focus::Providers; + app + } + + #[tokio::test] + async fn providers_navigation_advertises_workspace_shortcut() { + let app = test_app(); + let mut terminal = Terminal::new(TestBackend::new(180, 1)).unwrap(); + + terminal + .draw(|frame| draw_nav_bar(frame, &app, frame.size())) + .unwrap(); + + let text: String = terminal + .backend() + .buffer() + .content() + .iter() + .map(ratatui::buffer::Cell::symbol) + .collect(); + assert!(text.contains("[w] Workspace"), "nav bar was: {text:?}"); + } + + #[test] + fn title_bar_brand_renders_resolved_version_without_alpha_badge() { + let expected = format!(" >_ OpenShell v{} | ", openshell_core::VERSION); + let width = u16::try_from(expected.len()).unwrap(); + let backend = TestBackend::new(width, 1); + let mut terminal = Terminal::new(backend).unwrap(); + + terminal + .draw(|frame| { + frame.render_widget( + Paragraph::new(Line::from(title_bar_brand_spans(&Theme::dark()))), + frame.size(), + ); + }) + .unwrap(); + + let buffer = terminal.backend().buffer(); + let rendered = (0..width) + .map(|x| buffer.get(x, 0).symbol()) + .collect::(); + assert_eq!(rendered, expected); + assert!(!rendered.contains("ALPHA")); + } +} diff --git a/crates/openshell-tui/src/ui/splash.rs b/crates/openshell-tui/src/ui/splash.rs index 7f889f9951..a46667fb32 100644 --- a/crates/openshell-tui/src/ui/splash.rs +++ b/crates/openshell-tui/src/ui/splash.rs @@ -104,16 +104,11 @@ pub fn draw(frame: &mut Frame<'_>, area: Rect, theme: &crate::theme::Theme) { frame.render_widget(Paragraph::new(content_lines), chunks[0]); - // -- Footer: version + ALPHA badge on line 1, prompt on line 2 -- + // -- Footer: version on line 1, prompt on line 2 -- let version = format!("v{}", openshell_core::VERSION); - let alpha_badge = "ALPHA"; let footer = Paragraph::new(vec![ - Line::from(vec![ - Span::styled(version, t.accent), - Span::styled(" ", t.muted), - Span::styled(alpha_badge, t.title_bar), - ]), + Line::from(Span::styled(version, t.accent)), Line::from(Span::styled("press any key ░", t.muted)), ]); diff --git a/deploy/helm/openshell-workspace/Chart.yaml b/deploy/helm/openshell-workspace/Chart.yaml new file mode 100644 index 0000000000..03a3919d46 --- /dev/null +++ b/deploy/helm/openshell-workspace/Chart.yaml @@ -0,0 +1,9 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +apiVersion: v2 +name: openshell-workspace +description: Namespace-scoped prerequisites for OpenShell Kubernetes sandboxes +type: application +version: 0.0.0 +appVersion: "0.0.0" diff --git a/deploy/helm/openshell-workspace/README.md b/deploy/helm/openshell-workspace/README.md new file mode 100644 index 0000000000..96ae4ca6e4 --- /dev/null +++ b/deploy/helm/openshell-workspace/README.md @@ -0,0 +1,47 @@ +# OpenShell Workspace Helm Chart + + + +> **Experimental** - the shared-gateway, multi-namespace deployment path is +> under active design. + +This chart installs the namespace-scoped ServiceAccount, RBAC, and NetworkPolicy +needed for OpenShell Kubernetes sandboxes. Install it once in every +platform-managed workspace namespace. It does not create a namespace or deploy +an OpenShell gateway. + +Install the gateway chart with `workspaceResources.enabled=false`, then install +this chart with the gateway ServiceAccount identity. Configure the gateway's +Kubernetes driver in `operator` workspace mode when it serves more than one +pre-provisioned workspace namespace: + +```shell +helm install openshell-workspace ./deploy/helm/openshell-workspace \ + --namespace app-a \ + --set gateway.serviceAccount.name=openshell \ + --set gateway.serviceAccount.namespace=openshell +``` + +Keep `sandboxServiceAccount.name` aligned with the gateway chart's +`sandboxServiceAccount.name`. The defaults for both charts are +`openshell-sandbox`. + +## Values + +| Key | Type | Default | Description | +|-----|------|---------|-------------| +| fullnameOverride | string | `""` | Override the full generated resource name. | +| gateway.networkPolicy.podSelector | object | `{"app.kubernetes.io/instance":"openshell","app.kubernetes.io/name":"openshell"}` | Labels selecting gateway pods allowed to reach sandbox SSH. | +| gateway.serviceAccount.name | string | `"openshell"` | Name of the shared gateway ServiceAccount. | +| gateway.serviceAccount.namespace | string | `"openshell"` | Namespace containing the shared gateway ServiceAccount. | +| nameOverride | string | `""` | Override the chart name used in generated resource names. | +| networkPolicy.enabled | bool | `true` | Restrict sandbox SSH ingress to the shared gateway pods. | +| sandboxServiceAccount.annotations | object | `{}` | Annotations added to the generated sandbox ServiceAccount. | +| sandboxServiceAccount.create | bool | `true` | Create the ServiceAccount assigned to sandbox pods. | +| sandboxServiceAccount.name | string | `"openshell-sandbox"` | Sandbox ServiceAccount name. | + +---------------------------------------------- +Autogenerated from chart metadata using [helm-docs v1.14.2](https://github.com/norwoodj/helm-docs/releases/v1.14.2) diff --git a/deploy/helm/openshell-workspace/README.md.gotmpl b/deploy/helm/openshell-workspace/README.md.gotmpl new file mode 100644 index 0000000000..a233eef165 --- /dev/null +++ b/deploy/helm/openshell-workspace/README.md.gotmpl @@ -0,0 +1,33 @@ +# OpenShell Workspace Helm Chart + + + +> **Experimental** - the shared-gateway, multi-namespace deployment path is +> under active design. + +This chart installs the namespace-scoped ServiceAccount, RBAC, and NetworkPolicy +needed for OpenShell Kubernetes sandboxes. Install it once in every +platform-managed workspace namespace. It does not create a namespace or deploy +an OpenShell gateway. + +Install the gateway chart with `workspaceResources.enabled=false`, then install +this chart with the gateway ServiceAccount identity. Configure the gateway's +Kubernetes driver in `operator` workspace mode when it serves more than one +pre-provisioned workspace namespace: + +```shell +helm install openshell-workspace ./deploy/helm/openshell-workspace \ + --namespace app-a \ + --set gateway.serviceAccount.name=openshell \ + --set gateway.serviceAccount.namespace=openshell +``` + +Keep `sandboxServiceAccount.name` aligned with the gateway chart's +`sandboxServiceAccount.name`. The defaults for both charts are +`openshell-sandbox`. + +{{ template "chart.valuesSection" . }} +{{ template "helm-docs.versionFooter" . }} diff --git a/deploy/helm/openshell-workspace/templates/_helpers.tpl b/deploy/helm/openshell-workspace/templates/_helpers.tpl new file mode 100644 index 0000000000..eb948b9907 --- /dev/null +++ b/deploy/helm/openshell-workspace/templates/_helpers.tpl @@ -0,0 +1,44 @@ +{{/* +Expand the name of the chart. +*/}} +{{- define "openshell-workspace.name" -}} +{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* +Create a default fully qualified app name. +*/}} +{{- define "openshell-workspace.fullname" -}} +{{- if .Values.fullnameOverride }} +{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }} +{{- else }} +{{- $name := default .Chart.Name .Values.nameOverride }} +{{- if contains $name .Release.Name }} +{{- .Release.Name | trunc 63 | trimSuffix "-" }} +{{- else }} +{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }} +{{- end }} +{{- end }} +{{- end }} + +{{/* +Common labels. +*/}} +{{- define "openshell-workspace.labels" -}} +helm.sh/chart: {{ printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} +app.kubernetes.io/name: {{ include "openshell-workspace.name" . }} +app.kubernetes.io/instance: {{ .Release.Name }} +{{- if .Chart.AppVersion }} +app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} +{{- end }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +{{- end }} + +{{/* +Validate required cross-namespace gateway identity values. +*/}} +{{- define "openshell-workspace.validateValues" -}} +{{- $gatewayServiceAccountName := required "gateway.serviceAccount.name is required" .Values.gateway.serviceAccount.name -}} +{{- $gatewayServiceAccountNamespace := required "gateway.serviceAccount.namespace is required" .Values.gateway.serviceAccount.namespace -}} +{{- $sandboxServiceAccountName := required "sandboxServiceAccount.name is required" .Values.sandboxServiceAccount.name -}} +{{- end }} diff --git a/deploy/helm/openshell-workspace/templates/networkpolicy.yaml b/deploy/helm/openshell-workspace/templates/networkpolicy.yaml new file mode 100644 index 0000000000..f88cf84d04 --- /dev/null +++ b/deploy/helm/openshell-workspace/templates/networkpolicy.yaml @@ -0,0 +1,30 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +{{- include "openshell-workspace.validateValues" . }} +{{- if .Values.networkPolicy.enabled }} +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: {{ include "openshell-workspace.fullname" . }}-sandbox-ssh + namespace: {{ .Release.Namespace }} + labels: + {{- include "openshell-workspace.labels" . | nindent 4 }} +spec: + podSelector: + matchLabels: + openshell.ai/managed-by: openshell + policyTypes: + - Ingress + ingress: + - from: + - namespaceSelector: + matchLabels: + kubernetes.io/metadata.name: {{ .Values.gateway.serviceAccount.namespace }} + podSelector: + matchLabels: + {{- toYaml .Values.gateway.networkPolicy.podSelector | nindent 14 }} + ports: + - protocol: TCP + port: 2222 +{{- end }} diff --git a/deploy/helm/openshell-workspace/templates/role.yaml b/deploy/helm/openshell-workspace/templates/role.yaml new file mode 100644 index 0000000000..45b3beb831 --- /dev/null +++ b/deploy/helm/openshell-workspace/templates/role.yaml @@ -0,0 +1,39 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +{{- include "openshell-workspace.validateValues" . }} +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: {{ include "openshell-workspace.fullname" . }}-sandbox + namespace: {{ .Release.Namespace }} + labels: + {{- include "openshell-workspace.labels" . | nindent 4 }} +rules: + - apiGroups: + - agents.x-k8s.io + resources: + - sandboxes + - sandboxes/status + verbs: + - create + - delete + - get + - list + - patch + - update + - watch + - apiGroups: + - "" + resources: + - events + verbs: + - get + - list + - watch + - apiGroups: + - "" + resources: + - pods + verbs: + - get diff --git a/deploy/helm/openshell-workspace/templates/rolebinding.yaml b/deploy/helm/openshell-workspace/templates/rolebinding.yaml new file mode 100644 index 0000000000..669ec47628 --- /dev/null +++ b/deploy/helm/openshell-workspace/templates/rolebinding.yaml @@ -0,0 +1,19 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +{{- include "openshell-workspace.validateValues" . }} +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: {{ include "openshell-workspace.fullname" . }}-sandbox + namespace: {{ .Release.Namespace }} + labels: + {{- include "openshell-workspace.labels" . | nindent 4 }} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: {{ include "openshell-workspace.fullname" . }}-sandbox +subjects: + - kind: ServiceAccount + name: {{ .Values.gateway.serviceAccount.name }} + namespace: {{ .Values.gateway.serviceAccount.namespace }} diff --git a/deploy/helm/openshell-workspace/templates/serviceaccount.yaml b/deploy/helm/openshell-workspace/templates/serviceaccount.yaml new file mode 100644 index 0000000000..20bb263b54 --- /dev/null +++ b/deploy/helm/openshell-workspace/templates/serviceaccount.yaml @@ -0,0 +1,17 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +{{- include "openshell-workspace.validateValues" . }} +{{- if .Values.sandboxServiceAccount.create }} +apiVersion: v1 +kind: ServiceAccount +metadata: + name: {{ .Values.sandboxServiceAccount.name }} + namespace: {{ .Release.Namespace }} + labels: + {{- include "openshell-workspace.labels" . | nindent 4 }} + {{- with .Values.sandboxServiceAccount.annotations }} + annotations: + {{- toYaml . | nindent 4 }} + {{- end }} +{{- end }} diff --git a/deploy/helm/openshell-workspace/tests/workspace_test.yaml b/deploy/helm/openshell-workspace/tests/workspace_test.yaml new file mode 100644 index 0000000000..2f71920eaf --- /dev/null +++ b/deploy/helm/openshell-workspace/tests/workspace_test.yaml @@ -0,0 +1,61 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +suite: workspace namespace resources +templates: + - templates/serviceaccount.yaml + - templates/role.yaml + - templates/rolebinding.yaml + - templates/networkpolicy.yaml +release: + name: tenant-a + namespace: app-a + +tests: + - it: creates the workspace role in the release namespace + template: templates/role.yaml + asserts: + - hasDocuments: + count: 1 + - equal: + path: metadata.namespace + value: app-a + + - it: binds the shared gateway service account + template: templates/rolebinding.yaml + set: + gateway.serviceAccount.name: shared-gateway + gateway.serviceAccount.namespace: openshell-system + asserts: + - equal: + path: metadata.namespace + value: app-a + - equal: + path: subjects[0].name + value: shared-gateway + - equal: + path: subjects[0].namespace + value: openshell-system + + - it: selects gateway pods in the gateway namespace + template: templates/networkpolicy.yaml + set: + gateway.serviceAccount.namespace: openshell-system + gateway.networkPolicy.podSelector: + app.kubernetes.io/name: openshell + app.kubernetes.io/instance: central + asserts: + - equal: + path: spec.ingress[0].from[0].namespaceSelector.matchLabels["kubernetes.io/metadata.name"] + value: openshell-system + - equal: + path: spec.ingress[0].from[0].podSelector.matchLabels["app.kubernetes.io/instance"] + value: central + + - it: supports a pre-existing sandbox service account + template: templates/serviceaccount.yaml + set: + sandboxServiceAccount.create: false + asserts: + - hasDocuments: + count: 0 diff --git a/deploy/helm/openshell-workspace/values.yaml b/deploy/helm/openshell-workspace/values.yaml new file mode 100644 index 0000000000..2c52b0460d --- /dev/null +++ b/deploy/helm/openshell-workspace/values.yaml @@ -0,0 +1,31 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# -- Override the chart name used in generated resource names. +nameOverride: "" +# -- Override the full generated resource name. +fullnameOverride: "" + +gateway: + serviceAccount: + # -- Name of the shared gateway ServiceAccount. + name: openshell + # -- Namespace containing the shared gateway ServiceAccount. + namespace: openshell + networkPolicy: + # -- Labels selecting gateway pods allowed to reach sandbox SSH. + podSelector: + app.kubernetes.io/name: openshell + app.kubernetes.io/instance: openshell + +sandboxServiceAccount: + # -- Create the ServiceAccount assigned to sandbox pods. + create: true + # -- Sandbox ServiceAccount name. + name: openshell-sandbox + # -- Annotations added to the generated sandbox ServiceAccount. + annotations: {} + +networkPolicy: + # -- Restrict sandbox SSH ingress to the shared gateway pods. + enabled: true diff --git a/deploy/helm/openshell/README.md b/deploy/helm/openshell/README.md index 9d1b9cd7dd..7dbd7964d0 100644 --- a/deploy/helm/openshell/README.md +++ b/deploy/helm/openshell/README.md @@ -9,6 +9,14 @@ Edit README.md.gotmpl and values.yaml, then run `mise run helm:docs`. This chart deploys the OpenShell gateway into a Kubernetes cluster. It is published as an OCI artifact to GHCR at `oci://ghcr.io/nvidia/openshell/helm-chart`. +By default, this chart also creates the namespace-scoped resources needed by +sandboxes. For a shared-gateway deployment, install it with +`workspaceResources.enabled=false`, then install the +`deploy/helm/openshell-workspace` chart in every pre-provisioned workspace +namespace. The gateway and workspace releases can then be upgraded and removed +independently. Use Kubernetes `operator` workspace mode when one gateway serves +multiple pre-provisioned workspace namespaces. + ## Prerequisites The Kubernetes Agent Sandbox CRDs and controller must be installed on the cluster before deploying OpenShell. Install them with: @@ -306,6 +314,7 @@ discovery endpoint or its TLS CA. | upstreamProxy.url | string | `""` | HTTP proxy URL in http://host:port form. HTTPS-to-proxy is not supported. | | workload.allowMultiReplicaStatefulSet | bool | `false` | Allow replicaCount > 1 while rendering a StatefulSet. Prefer workload.kind=deployment for external database-backed multi-replica gateways; this override exists for operators who explicitly require StatefulSet identity or storage semantics. | | workload.kind | string | `"statefulset"` | Gateway workload controller kind. Use `statefulset` for the default SQLite database, or `deployment` when server.externalDbSecret points at an external database. | +| workspaceResources.enabled | bool | `true` | Create the sandbox ServiceAccount, Role, RoleBinding, and NetworkPolicy from this chart. Disable for a gateway-only release. | ---------------------------------------------- Autogenerated from chart metadata using [helm-docs v1.14.2](https://github.com/norwoodj/helm-docs/releases/v1.14.2) diff --git a/deploy/helm/openshell/README.md.gotmpl b/deploy/helm/openshell/README.md.gotmpl index 73ebb39c88..cf8677741e 100644 --- a/deploy/helm/openshell/README.md.gotmpl +++ b/deploy/helm/openshell/README.md.gotmpl @@ -9,6 +9,14 @@ Edit README.md.gotmpl and values.yaml, then run `mise run helm:docs`. This chart deploys the OpenShell gateway into a Kubernetes cluster. It is published as an OCI artifact to GHCR at `oci://ghcr.io/nvidia/openshell/helm-chart`. +By default, this chart also creates the namespace-scoped resources needed by +sandboxes. For a shared-gateway deployment, install it with +`workspaceResources.enabled=false`, then install the +`deploy/helm/openshell-workspace` chart in every pre-provisioned workspace +namespace. The gateway and workspace releases can then be upgraded and removed +independently. Use Kubernetes `operator` workspace mode when one gateway serves +multiple pre-provisioned workspace namespaces. + ## Prerequisites The Kubernetes Agent Sandbox CRDs and controller must be installed on the cluster before deploying OpenShell. Install them with: diff --git a/deploy/helm/openshell/templates/_helpers.tpl b/deploy/helm/openshell/templates/_helpers.tpl index 5b67c018c4..3d9f2f3e0b 100644 --- a/deploy/helm/openshell/templates/_helpers.tpl +++ b/deploy/helm/openshell/templates/_helpers.tpl @@ -70,6 +70,19 @@ Create the name of the service account assigned to sandbox pods {{- end }} {{- end }} +{{/* +Whether this chart owns workspace-scoped resources. Missing legacy values +default to enabled so upgrades with --reuse-values preserve the old topology. +*/}} +{{- define "openshell.workspaceResourcesEnabled" -}} +{{- $workspaceResources := .Values.workspaceResources | default dict -}} +{{- $enabled := true -}} +{{- if hasKey $workspaceResources "enabled" -}} +{{- $enabled = get $workspaceResources "enabled" -}} +{{- end -}} +{{- if $enabled -}}true{{- end -}} +{{- end }} + {{/* Gateway image reference. Uses image.tag when set; falls back to .Chart.AppVersion so a released chart automatically pulls the matching image without extra overrides. diff --git a/deploy/helm/openshell/templates/networkpolicy.yaml b/deploy/helm/openshell/templates/networkpolicy.yaml index e85571e5f5..c9e4a760e3 100644 --- a/deploy/helm/openshell/templates/networkpolicy.yaml +++ b/deploy/helm/openshell/templates/networkpolicy.yaml @@ -1,7 +1,7 @@ # SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 -{{- if .Values.networkPolicy.enabled }} +{{- if and (include "openshell.workspaceResourcesEnabled" .) .Values.networkPolicy.enabled }} # NetworkPolicy restricting SSH ingress on sandbox pods to the gateway pod. # Sandbox pods are dynamically created by the server and labelled with # openshell.ai/managed-by=openshell. This policy ensures only the gateway diff --git a/deploy/helm/openshell/templates/role.yaml b/deploy/helm/openshell/templates/role.yaml index 8def13f310..dfd6423615 100644 --- a/deploy/helm/openshell/templates/role.yaml +++ b/deploy/helm/openshell/templates/role.yaml @@ -1,5 +1,5 @@ {{- $workspaceMode := .Values.server.drivers.kubernetes.workspaceMode | default "shared" -}} -{{- if eq $workspaceMode "shared" }} +{{- if and (eq $workspaceMode "shared") (include "openshell.workspaceResourcesEnabled" .) }} # SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 apiVersion: rbac.authorization.k8s.io/v1 diff --git a/deploy/helm/openshell/templates/rolebinding.yaml b/deploy/helm/openshell/templates/rolebinding.yaml index 381473a58b..32f11644bf 100644 --- a/deploy/helm/openshell/templates/rolebinding.yaml +++ b/deploy/helm/openshell/templates/rolebinding.yaml @@ -1,5 +1,5 @@ {{- $workspaceMode := .Values.server.drivers.kubernetes.workspaceMode | default "shared" -}} -{{- if eq $workspaceMode "shared" }} +{{- if and (eq $workspaceMode "shared") (include "openshell.workspaceResourcesEnabled" .) }} # SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 apiVersion: rbac.authorization.k8s.io/v1 diff --git a/deploy/helm/openshell/templates/serviceaccount.yaml b/deploy/helm/openshell/templates/serviceaccount.yaml index a98ad5363e..1a9245d4dc 100644 --- a/deploy/helm/openshell/templates/serviceaccount.yaml +++ b/deploy/helm/openshell/templates/serviceaccount.yaml @@ -13,10 +13,10 @@ metadata: {{- toYaml . | nindent 4 }} {{- end }} {{- end }} -{{- if and .Values.serviceAccount.create .Values.sandboxServiceAccount.create }} +{{- if and .Values.serviceAccount.create (include "openshell.workspaceResourcesEnabled" .) .Values.sandboxServiceAccount.create }} --- {{- end }} -{{- if .Values.sandboxServiceAccount.create }} +{{- if and (include "openshell.workspaceResourcesEnabled" .) .Values.sandboxServiceAccount.create }} apiVersion: v1 kind: ServiceAccount metadata: diff --git a/deploy/helm/openshell/tests/sandbox_namespace_test.yaml b/deploy/helm/openshell/tests/sandbox_namespace_test.yaml index ee89fce53d..864e3a8512 100644 --- a/deploy/helm/openshell/tests/sandbox_namespace_test.yaml +++ b/deploy/helm/openshell/tests/sandbox_namespace_test.yaml @@ -75,3 +75,21 @@ tests: path: metadata.namespace value: other-ns documentIndex: 1 + + - it: omits workspace resources in gateway-only mode + set: + workspaceResources.enabled: false + networkPolicy.enabled: true + asserts: + - hasDocuments: + count: 1 + template: templates/gateway-config.yaml + - hasDocuments: + count: 0 + template: templates/networkpolicy.yaml + - hasDocuments: + count: 0 + template: templates/role.yaml + - hasDocuments: + count: 0 + template: templates/rolebinding.yaml diff --git a/deploy/helm/openshell/tests/sandbox_service_account_test.yaml b/deploy/helm/openshell/tests/sandbox_service_account_test.yaml index c426415823..c9f10868fe 100644 --- a/deploy/helm/openshell/tests/sandbox_service_account_test.yaml +++ b/deploy/helm/openshell/tests/sandbox_service_account_test.yaml @@ -29,3 +29,13 @@ tests: asserts: - hasDocuments: count: 1 + + - it: renders only the gateway service account in gateway-only mode + set: + workspaceResources.enabled: false + asserts: + - hasDocuments: + count: 1 + - equal: + path: metadata.name + value: openshell diff --git a/deploy/helm/openshell/values.yaml b/deploy/helm/openshell/values.yaml index 25236eb474..8f5b6fa51a 100644 --- a/deploy/helm/openshell/values.yaml +++ b/deploy/helm/openshell/values.yaml @@ -102,6 +102,14 @@ sandboxServiceAccount: # -- Existing service account name for sandbox pods when sandboxServiceAccount.create is false. name: "" +# Namespace-scoped resources needed to run sandboxes. Disable this when the +# gateway and workspace prerequisites are managed as separate Helm releases +# using the openshell-workspace chart. +workspaceResources: + # -- Create the sandbox ServiceAccount, Role, RoleBinding, and NetworkPolicy + # from this chart. Disable for a gateway-only release. + enabled: true + # -- Extra annotations to add to the gateway pod. podAnnotations: {} # -- Extra labels to add to the gateway pod. diff --git a/deploy/helm/test-split-ownership.sh b/deploy/helm/test-split-ownership.sh new file mode 100755 index 0000000000..30fd7360d4 --- /dev/null +++ b/deploy/helm/test-split-ownership.sh @@ -0,0 +1,66 @@ +#!/usr/bin/env bash +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +set -euo pipefail + +repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd -P)" +work_dir="$(mktemp -d)" +trap 'rm -rf "${work_dir}"' EXIT + +helm template openshell "${repo_root}/deploy/helm/openshell" \ + --namespace openshell \ + --set agentSandbox.preflight.enabled=false \ + --set workspaceResources.enabled=false \ + >"${work_dir}/gateway.yaml" + +if yq ea -e \ + 'select(.kind == "Role" and .metadata.name == "openshell-sandbox")' \ + "${work_dir}/gateway.yaml" >/dev/null 2>&1; then + echo "gateway chart rendered workspace resources despite workspaceResources.enabled=false" >&2 + exit 1 +fi + +helm template openshell-workspace "${repo_root}/deploy/helm/openshell-workspace" \ + --namespace app-a \ + --set gateway.serviceAccount.name=openshell \ + --set gateway.serviceAccount.namespace=openshell \ + >"${work_dir}/workspace.yaml" + +invalid_workspace_docs="$( + yq ea -N -r \ + 'select(. != null and (.apiVersion == null or .kind == null)) | document_index' \ + "${work_dir}/workspace.yaml" +)" +if [[ -n "${invalid_workspace_docs}" ]]; then + echo "workspace chart rendered documents without apiVersion or kind: ${invalid_workspace_docs}" >&2 + exit 1 +fi + +helm template openshell "${repo_root}/deploy/helm/openshell" \ + --namespace openshell \ + --set agentSandbox.preflight.enabled=false \ + --set-json workspaceResources=null \ + >"${work_dir}/legacy-reuse-values.yaml" + +yq ea -e \ + 'select(.kind == "Role" and .metadata.name == "openshell-sandbox") | + .apiVersion == "rbac.authorization.k8s.io/v1"' \ + "${work_dir}/legacy-reuse-values.yaml" >/dev/null + +yq ea -N -r \ + 'select(.kind != null) | [.apiVersion, .kind, (.metadata.namespace // "openshell"), .metadata.name] | @tsv' \ + "${work_dir}/gateway.yaml" | sort -u >"${work_dir}/gateway.objects" +yq ea -N -r \ + 'select(.kind != null) | [.apiVersion, .kind, (.metadata.namespace // "app-a"), .metadata.name] | @tsv' \ + "${work_dir}/workspace.yaml" | sort -u >"${work_dir}/workspace.objects" + +comm -12 "${work_dir}/gateway.objects" "${work_dir}/workspace.objects" \ + >"${work_dir}/overlap.objects" +if [[ -s "${work_dir}/overlap.objects" ]]; then + echo "gateway and workspace charts claim the same Kubernetes objects:" >&2 + cat "${work_dir}/overlap.objects" >&2 + exit 1 +fi + +echo "gateway and workspace chart object ownership is disjoint" diff --git a/docs/CONTRIBUTING.mdx b/docs/CONTRIBUTING.mdx index 9a69c4b445..b7a47ce986 100644 --- a/docs/CONTRIBUTING.mdx +++ b/docs/CONTRIBUTING.mdx @@ -14,10 +14,10 @@ If you use an AI coding agent (Cursor, Claude Code, Codex, etc.), the repo inclu | Skill | What it does | When to use | |---|---|---| -| `update-docs` | Scans recent commits for user-facing changes and drafts doc updates. | After landing features, before a release, or to find doc gaps. | +| `update-docs-from-commits` | Scans recent commits for user-facing changes and drafts doc updates. | After landing features, before a release, or to find doc gaps. | | `build-from-issue` | Plans and implements work from a GitHub issue, including doc updates. | When working from an issue that has doc impact. | -The skills live in `.agents/skills/` and follow the style guide below automatically. To use one, ask your agent to run it (e.g., "catch up the docs for everything merged since v0.2.0"). +These contributor workflows live in `.agents/skills/` and follow the style guide below automatically. They are separate from the public skills in `skills/`, which help users operate OpenShell and can be installed without cloning the repository. To use a contributor skill, ask your repository-aware agent to run it (e.g., "catch up the docs for everything merged since v0.2.0"). ## When to Update Docs diff --git a/docs/get-started/quickstart.mdx b/docs/get-started/quickstart.mdx index 8a8b82046e..4d7a3e2deb 100644 --- a/docs/get-started/quickstart.mdx +++ b/docs/get-started/quickstart.mdx @@ -33,7 +33,7 @@ The install script uses Homebrew, RPM, or a Debian package based on your machine After installing the CLI, run `openshell --help` in your terminal to view the full CLI reference. -You can also clone the [NVIDIA OpenShell GitHub repository](https://github.com/NVIDIA/OpenShell) and use the `/openshell-cli` skill to load the CLI reference into your agent. +Install the public OpenShell agent skills with `npx skills add NVIDIA/OpenShell`. The `openshell-cli` skill guides your agent through common workflows and uses the installed CLI help as the command reference; no OpenShell source checkout is required. ## Create Your First OpenShell Sandbox diff --git a/docs/kubernetes/setup.mdx b/docs/kubernetes/setup.mdx index 221f935eb6..fb7881af2d 100644 --- a/docs/kubernetes/setup.mdx +++ b/docs/kubernetes/setup.mdx @@ -99,6 +99,36 @@ helm upgrade --install openshell \ The chart automatically generates PKI secrets on first install using pre-install Helm hooks. No manual secret creation is required. +### Split gateway and workspace releases + +For a platform-managed namespace, install the gateway without namespace-scoped +sandbox resources, then install the workspace chart in the sandbox namespace: + +```shell +helm upgrade --install openshell \ + oci://ghcr.io/nvidia/openshell/helm-chart \ + --version \ + --namespace openshell \ + --set workspaceResources.enabled=false \ + --set server.sandboxNamespace=app-a + +helm upgrade --install openshell-workspace \ + oci://ghcr.io/nvidia/openshell/openshell-workspace \ + --version \ + --namespace app-a \ + --set gateway.serviceAccount.name=openshell \ + --set gateway.serviceAccount.namespace=openshell +``` + +The workspace chart does not create the namespace or deploy a gateway. It owns +only the sandbox ServiceAccount, Role, RoleBinding, and NetworkPolicy in its +release namespace. For one pre-provisioned namespace, keep +`server.drivers.kubernetes.workspaceMode=shared` and set +`server.sandboxNamespace=app-a`. To map multiple workspaces to separately +provisioned namespaces, use `workspaceMode=operator`, configure exactly one of +`operatorNamespaceLabel` or `operatorNamespaceFile`, and install the workspace +chart in every allowlisted namespace. + ## Wait for the gateway to be ready ```shell @@ -163,6 +193,7 @@ The most commonly changed values are: | `workload.kind` | Gateway workload controller. Use `statefulset` for SQLite or `deployment` with `server.externalDbSecret`. | | `workload.allowMultiReplicaStatefulSet` | Allow `replicaCount > 1` with `workload.kind=statefulset`. Prefer Deployment for external database-backed multi-replica gateways. | | `server.sandboxNamespace` | Namespace where sandbox pods are created. Defaults to the Helm release namespace when left empty. | +| `workspaceResources.enabled` | Create namespace-scoped sandbox prerequisites from the gateway chart. Disable when installing the workspace chart separately. | | `server.externalDbSecret` | Secret containing a PostgreSQL connection URI in the `uri` key. Use when the database is managed outside the chart. | | `server.telemetryEnabled` | Enable anonymous OpenShell telemetry from the gateway and its sandbox supervisors. Set to `false` to opt out. | | `server.sandboxImage` | Default sandbox image used when a sandbox does not specify one. | diff --git a/docs/reference/gateway-config.mdx b/docs/reference/gateway-config.mdx index c04c0040d0..7553e7deb4 100644 --- a/docs/reference/gateway-config.mdx +++ b/docs/reference/gateway-config.mdx @@ -796,6 +796,50 @@ guest_tls_key = "/var/lib/openshell/guest-tls/client-key.pem" # Defaults to 10001 when unset; matching GID is used if sandbox_gid is empty. # Any non-root Linux UID/GID is valid. # sandbox_uid = 20001 +# Corporate forward proxy for sandbox egress. The keys, their semantics, and +# the fail-closed contract are identical to the Podman driver above: only TLS +# (CONNECT) egress is chained, plain-HTTP destination requests always dial +# directly, credentials must come from proxy_auth_file rather than the URL, +# an http:// proxy with credentials requires proxy_auth_allow_insecure, and +# any present-but-invalid value is rejected at gateway startup rather than +# degrading to a direct dial. proxy_auth_file and proxy_ca_bundle are paths on +# the gateway host. +# +# The sandbox cannot select or override these settings. They reach the guest +# supervisor on its command line through a per-sandbox file the driver writes +# into the overlay upperdir on every launch, so a sandbox image cannot supply +# its own values or disable the operator's by baking a file at that path. +# +# Reachability: a proxy on the corporate network needs no special address and +# works on every VM sandbox. The guest's callback to the gateway is unaffected +# and never traverses the proxy. +# +# A proxy on the gateway host itself is reachable only from libkrun-backed +# (non-GPU) sandboxes: their egress leaves through gvproxy, which NATs +# 192.168.127.254 to the host's 127.0.0.1, so address it as +# http://host.openshell.internal: rather than http://127.0.0.1:. +# GPU sandboxes run on the QEMU/TAP backend, which has no such NAT — +# host.openshell.internal resolves to the TAP host address, and the driver's +# nftables rules let the guest reach only the gateway port on the host. A +# gateway-host proxy URL is therefore rejected when the sandbox launches on +# QEMU, rather than timing out on every CONNECT; give GPU sandboxes a proxy +# address routable from the guest's masqueraded egress. +# +# Because a microVM has no bind mounts or container secrets, the driver stages +# the credential and the CA into the per-sandbox overlay disk: the credential +# root-only inside the guest, and both removed with the sandbox. The +# credential is therefore at rest in that overlay image on the gateway host — +# the same delivery the per-sandbox gateway token already uses, and a +# difference from the Podman secret model worth noting when choosing where to +# keep proxy credentials. +# https_proxy = "http://host.openshell.internal:8080" +# no_proxy = "10.0.0.0/8,.internal.example" +# proxy_auth_file = "/etc/openshell/secrets/proxy-auth" +# proxy_auth_allow_insecure = true +# Last resort for hostname-filtering proxy ACLs; see the Podman section above. +# proxy_connect_by_hostname = true +# Corporate CA trusted for an https:// proxy and TLS-intercepting proxies. +# proxy_ca_bundle = "/etc/openshell/tls/proxy-ca.pem" ``` ### Extension Driver diff --git a/docs/reference/sandbox-compute-drivers.mdx b/docs/reference/sandbox-compute-drivers.mdx index f00c49afd6..987e66b0d9 100644 --- a/docs/reference/sandbox-compute-drivers.mdx +++ b/docs/reference/sandbox-compute-drivers.mdx @@ -360,6 +360,18 @@ The VM driver creates nftables rules on the host for each sandbox VM's TAP netwo On hosts with restrictive firewalls (e.g. firewalld), the host firewall may additionally block VM traffic that the driver's rules accept. If VM sandboxes cannot reach the network, verify that the host firewall allows forwarding and input for `vmtap-*` interfaces. See the [VM driver README](https://github.com/NVIDIA/OpenShell/blob/main/crates/openshell-driver-vm/README.md#host-side-nftables-rules) for details. +### Corporate Proxy Egress + +For proxy-required networks, the VM driver accepts the same corporate egress proxy keys as the Podman driver: `https_proxy`, `no_proxy`, `proxy_auth_file`, `proxy_auth_allow_insecure`, `proxy_connect_by_hostname`, and `proxy_ca_bundle`. The in-guest supervisor chains policy-approved TLS tunnels through the proxy with HTTP CONNECT instead of dialing destinations directly. + +The settings reach the guest supervisor on its command line through a per-sandbox argument file the driver writes into the overlay upperdir on every launch, so a sandbox cannot select, alter, or disable the proxy from inside the guest — including through image `ENV`, the sandbox environment, or files baked into the image at the paths the driver uses. + +A proxy on the corporate network needs no special address and works on every VM sandbox. The guest's callback to the gateway never traverses the proxy. + +A proxy on the gateway host itself works only for libkrun-backed (non-GPU) sandboxes, whose egress leaves through gvproxy: configure `https_proxy = "http://host.openshell.internal:"` rather than a `127.0.0.1` URL, because gvproxy NATs that alias to the host's `127.0.0.1`. GPU sandboxes use the QEMU/TAP backend, where `host.openshell.internal` resolves to the TAP host address and the driver's [host firewall rules](#host-firewall) allow the guest to reach only the gateway port on the host. The driver rejects a gateway-host proxy URL when a sandbox launches on QEMU instead of letting every CONNECT time out, so give GPU sandboxes a proxy address routable from the guest's masqueraded egress. + +Because a microVM has no bind mounts or container secrets, the driver stages the credential (root-only) and the CA bundle into the per-sandbox overlay disk and removes them with the sandbox. See the [Gateway Configuration File](./gateway-config) reference for the full contract, including the cleartext-credential acknowledgement and the validated-IP CONNECT behavior. + ## Kubernetes Driver Kubernetes-backed sandboxes run as pods in the configured sandbox namespace. Use Kubernetes for shared clusters, remote compute, GPU scheduling, and operator-managed environments. diff --git a/docs/sandboxes/manage-sandboxes.mdx b/docs/sandboxes/manage-sandboxes.mdx index a28d653ead..96538eeeaa 100644 --- a/docs/sandboxes/manage-sandboxes.mdx +++ b/docs/sandboxes/manage-sandboxes.mdx @@ -157,6 +157,59 @@ Local directories and Dockerfiles require a local gateway because the CLI builds through the local Docker daemon. Use a registry image reference for remote gateways. +## Reuse Workload Templates + +Sandbox workload templates let workspace admins define reusable runtime shapes for a workspace. A template stores the image, environment, resource requests, and driver-specific configuration that sandboxes should inherit. When you create a sandbox from a template, the create request can still attach providers, labels, and policy, but the workload comes from the named template. + +Create a template: + +```shell +openshell sandbox template create gpu-kata \ + --image registry.example.com/agent:latest \ + --cpu 2 \ + --memory 4Gi \ + --gpu 1 \ + --label team=runtime \ + --env FEATURE_FLAG=on +``` + +Use `--gpu` without a count when the template should request the active +driver's default GPU assignment. Use `--gpu COUNT` when the template needs a +specific number of GPUs. + +Add driver-specific settings when the active compute driver needs them: + +```shell +openshell sandbox template create gpu-kata \ + --image registry.example.com/agent:latest \ + --driver-config-json '{"kubernetes":{"pod":{"runtime_class_name":"kata-containers","node_selector":{"pool":"gpu"}}}}' +``` + +If you omit `--image`, the gateway applies its default sandbox image when a sandbox is created from the template. Use this when the template should only define resource, environment, or driver settings. + +Create a sandbox from a template: + +```shell +openshell sandbox create --template gpu-kata --provider github -- claude +``` + +The `--template` flag cannot be combined with inline workload flags such as `--from`, `--cpu`, `--memory`, `--gpu`, `--env`, or `--driver-config-json`. Put those values on the template instead. Create-time policy and provider attachments remain part of the sandbox request, so each sandbox can keep its own access boundary. + +Inspect and manage templates: + +```shell +openshell sandbox template list +openshell sandbox template list --label-selector team=runtime +openshell sandbox template get gpu-kata +openshell sandbox template delete gpu-kata +``` + +Use `--all-workspaces` with `sandbox template list` when you need an admin view across workspaces: + +```shell +openshell sandbox template list --all-workspaces +``` + ## Base Sandbox Container The `base` sandbox container is the default runtime image for standard OpenShell sandboxes unless the gateway overrides its default sandbox image. It is published as `ghcr.io/nvidia/openshell-community/sandboxes/base:latest` and maintained in the [OpenShell Community](https://github.com/NVIDIA/OpenShell-Community/tree/main/sandboxes/base) repository. @@ -295,6 +348,24 @@ with SandboxClient.from_active_cluster() as client: assert sandbox.id in {s.id for s in matches} ``` +Create reusable sandbox templates through the Python SDK when several runs +should share the same workload shape: + +```python +from openshell import SandboxClient + +with SandboxClient.from_active_cluster() as client: + templates = client.sandbox_templates() + + templates.create( + workspace="default", + name="python", + image="ghcr.io/nvidia/openshell-community/sandboxes/python:latest", + ) + + sandbox = client.create_from_template(workspace="default", template_name="python") +``` + For non-interactive automation, pass a renewable client-credentials provider. Omitted issuer, client ID, audience, and scopes are read from the active gateway's metadata. The client requires TLS for non-loopback gateways: @@ -456,7 +527,7 @@ Structured output includes `sandbox`, `bind_address`, `port`, `pid`, and expected OpenShell SSH forward; it does not probe the forwarded socket. When no forwards are tracked, structured output returns an empty collection. -The default table colorizes the `STATUS` column, but only when the stream it is written to is a terminal, so piping or redirecting gives plain text. Each stream is decided on its own, so redirecting one leaves the other styled. Prefer `--output json` for automation rather than matching on the table. Set `NO_COLOR` to any non-empty value or pass `--color never` to suppress color, and `--color always` to keep it when piping into a pager. `--color` applies to every `openshell` command and covers all styled output: tables, log lines from `-v`, progress spinners, prompts, and error messages. +The default table colorizes the `STATUS` column only when both standard output and standard error are capable ANSI terminals; piping or redirecting either stream, or running under `TERM=dumb`, gives a plain-text table. Other styled output—including `-v` log lines, progress spinners, prompts, and error messages—is decided per stream, so redirecting one stream leaves the other styled. Prefer `--output json` for automation rather than matching on the table. Set `NO_COLOR` to any non-empty value or pass `--color never` to suppress ANSI formatting, and `--color always` to force it when piping into a pager. `--color` applies to every `openshell` command. You can also forward a port at creation time with `--forward`: diff --git a/docs/sandboxes/manage-workspaces.mdx b/docs/sandboxes/manage-workspaces.mdx index 05378af5f5..86da25fbb9 100644 --- a/docs/sandboxes/manage-workspaces.mdx +++ b/docs/sandboxes/manage-workspaces.mdx @@ -9,8 +9,9 @@ position: 3 --- An OpenShell workspace is an access and resource isolation boundary. Sandboxes, -providers, services, policies, settings, and inference routes belong to a -workspace and are not visible to members of other workspaces. +sandbox workload templates, providers, services, policies, settings, and +inference routes belong to a workspace and are not visible to members of other +workspaces. The CLI targets the `default` workspace unless you set `--workspace` or `OPENSHELL_WORKSPACE`. The logical OpenShell workspace described here is @@ -49,6 +50,8 @@ The following table summarizes common operations. | Add Workspace Users or remove members | Any workspace | Assigned workspace | No | | Assign the Workspace Admin role | Any workspace | No | No | | Create, use, or delete sandboxes and services | Any workspace | Assigned workspace | Assigned workspace | +| Create or delete sandbox workload templates | Any workspace | Assigned workspace | No | +| Read or list sandbox workload templates | Any workspace | Assigned workspace | Assigned workspace | | Create, update, or delete providers | Any workspace | Assigned workspace | No | | Change workspace policy or settings | Any workspace | Assigned workspace | No | | Manage platform profiles or global configuration | Yes | No | No | @@ -187,11 +190,11 @@ be deleted. openshell workspace delete team-ml ``` -A custom workspace must not contain sandboxes, providers, provider profiles, -services, SSH sessions, settings, policies, draft policy chunks, or credential -refresh state. Remove those resources before retrying deletion. OpenShell -removes membership records and inference routes as part of successful -workspace deletion. +A custom workspace must not contain sandboxes, sandbox workload templates, +providers, provider profiles, services, SSH sessions, settings, policies, draft +policy chunks, or credential refresh state. Remove those resources before +retrying deletion. OpenShell removes membership records and inference routes as +part of successful workspace deletion. ## Next Steps diff --git a/e2e/rust/Cargo.toml b/e2e/rust/Cargo.toml index 6dca586879..18556d0f7b 100644 --- a/e2e/rust/Cargo.toml +++ b/e2e/rust/Cargo.toml @@ -108,6 +108,11 @@ name = "vm_gateway_start" path = "tests/vm_gateway_start.rs" required-features = ["e2e-vm"] +[[test]] +name = "vm_corporate_proxy" +path = "tests/vm_corporate_proxy.rs" +required-features = ["e2e-vm"] + [[test]] name = "provider_token_exchange" path = "tests/provider_token_exchange.rs" @@ -168,6 +173,11 @@ name = "workspace_lifecycle" path = "tests/workspace_lifecycle.rs" required-features = ["e2e"] +[[test]] +name = "sandbox_templates" +path = "tests/sandbox_templates.rs" +required-features = ["e2e"] + [[test]] name = "proxy_egress_pipeline" path = "tests/proxy_egress_pipeline.rs" diff --git a/e2e/rust/e2e-vm.sh b/e2e/rust/e2e-vm.sh index 9acc633d65..96da2a879f 100755 --- a/e2e/rust/e2e-vm.sh +++ b/e2e/rust/e2e-vm.sh @@ -409,4 +409,5 @@ else run_e2e_test host_gateway_alias run_e2e_test vm_overlay run_e2e_test vm_gateway_start + run_e2e_test vm_corporate_proxy fi diff --git a/e2e/rust/src/harness/host_process.rs b/e2e/rust/src/harness/host_process.rs new file mode 100644 index 0000000000..f9fefd75a6 --- /dev/null +++ b/e2e/rust/src/harness/host_process.rs @@ -0,0 +1,103 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +//! Host-process TCP fixtures for e2e tests. +//! +//! [`HostSupportContainer`](super::container::HostSupportContainer) publishes +//! the same shape of fixture through a container engine. VM sandboxes reach +//! the host through gvproxy's `host.openshell.internal` alias and the VM e2e +//! lane has no container runtime of its own, so this variant runs the fixture +//! as a plain host process instead — keeping the lane free of a container +//! dependency it does not otherwise need. + +use std::io::Read as _; +use std::path::PathBuf; +use std::process::{Child, Command, Stdio}; +use std::time::Duration; + +use super::port::wait_for_port; + +/// A `python3` fixture listening on a host TCP port. +/// +/// Output is captured to a temp file rather than a pipe: these fixtures log +/// every request they serve, and a full pipe buffer would block the process +/// mid-test. [`logs`](Self::logs) reads the file, which is where a test finds +/// its evidence (the CONNECT targets a proxy saw, for example). +pub struct HostPythonFixture { + /// Host port the fixture listens on. + pub port: u16, + child: Child, + log_path: PathBuf, +} + +impl HostPythonFixture { + /// Start `python3 -c