diff --git a/README.md b/README.md index bc241e805c..1cf5b93e0f 100644 --- a/README.md +++ b/README.md @@ -157,7 +157,6 @@ Browse pre-built agents, tools, filesystems, and software packages at the [agent | `@agentos-software/git` | git | git version control | rust | - | - | | `@agentos-software/grep` | grep | GNU grep pattern matching (grep, egrep, fgrep) | rust | - | - | | `@agentos-software/gzip` | gzip | GNU gzip compression (gzip, gunzip, zcat) | rust | - | - | -| `@agentos-software/http-get` | http-get | Minimal HTTP GET fetch helper | c | - | - | | `@agentos-software/jq` | jq | jq JSON processor | rust | - | - | | `@agentos-software/ripgrep` | ripgrep | ripgrep fast recursive search | rust | - | - | | `@agentos-software/sed` | sed | GNU sed stream editor | rust | - | - | diff --git a/docs-internal/registry-flatten-colocation-spec.md b/docs-internal/registry-flatten-colocation-spec.md index 3cd23d7c06..f4298f4cc9 100644 --- a/docs-internal/registry-flatten-colocation-spec.md +++ b/docs-internal/registry-flatten-colocation-spec.md @@ -59,7 +59,7 @@ repo-root/ │ │ ├── sh/ cat/ ls/ cp/ mv/ sort/ … # ~80 command crates │ │ └── (du/ expr/ column/ rev/ strings/ — 1:1 libs that belong to coreutils) │ │ -│ ├── sqlite3/ duckdb/ wget/ zip/ unzip/ http-get/ # C-based, native/c/.c +│ ├── sqlite3/ duckdb/ wget/ zip/ unzip/ # C-based, native/c/.c │ ├── grep/ sed/ gawk/ jq/ yq/ fd/ ripgrep/ tree/ file/ tar/ gzip/ diffutils/ findutils/ # Rust, native/crates/ │ │ │ ├── claude/ codex/ opencode/ pi/ pi-cli/ # agent adapters — @agentos-software/*, JS only, no native/ diff --git a/docs-internal/registry-parity-worklist.md b/docs-internal/registry-parity-worklist.md index 2b9883875c..5742b46e40 100644 --- a/docs-internal/registry-parity-worklist.md +++ b/docs-internal/registry-parity-worklist.md @@ -102,7 +102,7 @@ actual backing: |---|---|---|---| | **curl** | DONE | our custom driver over a libcurl fork | real `curl` CLI (upstream `src/tool_*.c`) | | **wget** | DONE | our 174-line `wget.c` (dropped) | real GNU Wget vs our sysroot — stub `getrlimit`/`getgroups`, then build | -| **http-get** | TODO | our 95-line `http_get.c` | drop, or a real tool | +| **http-get** | DONE | our 95-line `http_get.c` | dropped; real curl covers HTTP fetches | | **git** | TODO | our hand-rolled git from `sha1`+`flate2` | **real git** (upstream C), patched for WASI — **NOT gitoxide** | | **fd** | TODO | our `secureexec-fd` on raw `regex` (not sharkdp/fd) | real **fd** (sharkdp) | | **findutils** (`find`,`xargs`) | TODO | our hand-rolled on `regex`/shims | real GNU findutils, or `uutils/findutils` | @@ -184,10 +184,26 @@ works (`wasi-spawn` broker), so `xargs` is not a blocker. aggregate C `programs` builds 57 commands in `2026-07-08T05-09-53-0700-sqlite3-make-programs-final.log`. Rev: `typytnkk` — `fix(sqlite3): build official SQLite shell`. -- **http-get — drop, don't port.** `http_get.c` is a 95-line raw-socket **loopback - test client**, not an HTTP fetcher; real curl covers GET. But it's imported by - `packages/shell` and is the client in cross-runtime network tests — migrate those - dependents first, then drop the command. +- **http-get — DONE.** Dropped the 95-line raw-socket loopback client instead of + porting it: real curl now covers HTTP fetch behavior in DuckDB remote CSV tests, + runtime cross-network loopback tests, and the C conformance loopback row. + Removed the `@agentos-software/http-get` package, fallback command, shell/core + dependencies, registry listing, C command install entries, and lockfile package + edges. The low-level `http_get_test` fixture remains because it is a socket + diagnostic test program, not a shipped registry command. Proof: pnpm lockfile + refresh succeeds in + `2026-07-08T05-39-44-0700-http-get-pnpm-lock-refresh.log`; core and shell + type checks pass in + `2026-07-08T05-40-57-0700-http-get-core-check-types-after-install.log` and + `2026-07-08T05-41-34-0700-http-get-shell-check-types-final.log`; DuckDB test + file typecheck passes in + `2026-07-08T05-40-57-0700-http-get-duckdb-test-typecheck-after-install.log`; + aggregate C `programs` builds 57 commands, with `http_get` absent after stale + generated cleanup/install, in + `2026-07-08T05-41-34-0700-http-get-make-c-programs-without-command.log` and + `2026-07-08T05-43-24-0700-http-get-clear-stale-generated-and-install.log`; + cross-runtime network tests pass 11/11 with the WASM curl rows in + `2026-07-08T05-47-43-0700-http-get-runtime-cross-network-test-pass.log`. - **tree — DONE.** Replaced the custom Rust `secureexec-tree`/`cmd-tree` crates with upstream Steve Baker `tree` 2.3.2 from `OldManProgrammer/unix-tree`. It builds as a C toolchain command from pinned source, stages into @@ -421,7 +437,7 @@ real e2e tests that prove Linux-parity behavior — not smoke tests. ### 12. No tests at all — 12 software + 5 agents - **Broken:** zero e2e coverage: `gawk, sed, grep, tar, gzip, jq, ripgrep, yq, - diffutils, file, http-get, vim`; agents `claude, codex, opencode, pi, pi-cli`. + diffutils, file, vim`; agents `claude, codex, opencode, pi, pi-cli`. - **Objective:** write real e2e tests proving each behaves like its Linux counterpart (jq processes real JSON, sed edits streams, tar round-trips archives, grep/rg search real trees, gzip round-trips, etc.); agents exercise the real ACP diff --git a/packages/core/AGENTS.md b/packages/core/AGENTS.md index 25e194d226..8540ca7f41 100644 --- a/packages/core/AGENTS.md +++ b/packages/core/AGENTS.md @@ -118,7 +118,7 @@ An agent's launch config lives entirely in its `/opt/agentos` package `agentos-p - Sandbox toolkit quickstarts/tests that depend on external Docker should use an explicit `SKIP_DOCKER=1` gate instead of `skipIf`, and the truthful host-tool path is to read `AGENTOS_TOOLS_PORT` inside the VM and `POST` `{ toolkit, tool, input }` to `http://127.0.0.1:$AGENTOS_TOOLS_PORT/call` from a guest Node script. - Shared Vitest helpers under `src/test/` should register optional capability coverage conditionally in code instead of with `describe.skipIf` / `test.skipIf`; `US-088` treats those markers as product-debt skips even when they only guard backend capability differences. - Pi bash-tool E2E coverage depends on registry WASM commands being built locally. Gate those tests with `tests/helpers/registry-commands.ts` `hasRegistryCommands` and include the `@agentos-software/common` software package only when the command artifacts exist. -- Software package tests for C-built commands such as `duckdb` and `http_get` should go through `tests/helpers/registry-commands.ts`: prefer copied `../secure-exec/software/*/wasm` artifacts, fall back to `../secure-exec/toolchain/c/build` when available, and let the helper build missing C-source artifacts on demand before declaring the command unavailable. When bootstrapping from secure-exec `toolchain/c`, build `make sysroot` first and then run a second `make` for the concrete `build/...` targets so `SYSROOT` resolves to the patched tree instead of the vanilla SDK sysroot chosen at parse time; in that second pass, treat `sysroot/lib/wasm32-wasi/libc.a` as already built so `make` does not loop back through the patch pipeline because of preserved sysroot timestamps. +- Software package tests for C-built commands such as `duckdb` and `curl` should go through `tests/helpers/registry-commands.ts`: prefer copied `../secure-exec/software/*/wasm` artifacts, fall back to `../secure-exec/toolchain/c/build` when available, and let the helper build missing C-source artifacts on demand before declaring the command unavailable. When bootstrapping from secure-exec `toolchain/c`, build `make sysroot` first and then run a second `make` for the concrete `build/...` targets so `SYSROOT` resolves to the patched tree instead of the vanilla SDK sysroot chosen at parse time; in that second pass, treat `sysroot/lib/wasm32-wasi/libc.a` as already built so `make` does not loop back through the patch pipeline because of preserved sysroot timestamps. - `tests/claude-session.test.ts` is the Claude SDK truth suite. It runs the real `@anthropic-ai/claude-agent-sdk` session path through llmock and covers PATH-backed `xu`, text-only replies, nested `node` `execSync` and `spawn`, metadata, lifecycle, and mode updates. Run it with `pnpm --dir packages/core exec vitest run tests/claude-session.test.ts --reporter=verbose` when verifying Claude regressions. - **Kernel permissions are declarative pass-through config.** `AgentOsOptions.permissions` should stay JSON-serializable and be forwarded to the native sidecar without host-side probing or callback evaluation; Rust owns glob matching and policy decisions. - ACP session events are live-only over `onSessionEvent()`. Do not reintroduce sequence numbers, local replay buffers, or event cursor recovery. diff --git a/packages/core/CLAUDE.md b/packages/core/CLAUDE.md index 25e194d226..8540ca7f41 100644 --- a/packages/core/CLAUDE.md +++ b/packages/core/CLAUDE.md @@ -118,7 +118,7 @@ An agent's launch config lives entirely in its `/opt/agentos` package `agentos-p - Sandbox toolkit quickstarts/tests that depend on external Docker should use an explicit `SKIP_DOCKER=1` gate instead of `skipIf`, and the truthful host-tool path is to read `AGENTOS_TOOLS_PORT` inside the VM and `POST` `{ toolkit, tool, input }` to `http://127.0.0.1:$AGENTOS_TOOLS_PORT/call` from a guest Node script. - Shared Vitest helpers under `src/test/` should register optional capability coverage conditionally in code instead of with `describe.skipIf` / `test.skipIf`; `US-088` treats those markers as product-debt skips even when they only guard backend capability differences. - Pi bash-tool E2E coverage depends on registry WASM commands being built locally. Gate those tests with `tests/helpers/registry-commands.ts` `hasRegistryCommands` and include the `@agentos-software/common` software package only when the command artifacts exist. -- Software package tests for C-built commands such as `duckdb` and `http_get` should go through `tests/helpers/registry-commands.ts`: prefer copied `../secure-exec/software/*/wasm` artifacts, fall back to `../secure-exec/toolchain/c/build` when available, and let the helper build missing C-source artifacts on demand before declaring the command unavailable. When bootstrapping from secure-exec `toolchain/c`, build `make sysroot` first and then run a second `make` for the concrete `build/...` targets so `SYSROOT` resolves to the patched tree instead of the vanilla SDK sysroot chosen at parse time; in that second pass, treat `sysroot/lib/wasm32-wasi/libc.a` as already built so `make` does not loop back through the patch pipeline because of preserved sysroot timestamps. +- Software package tests for C-built commands such as `duckdb` and `curl` should go through `tests/helpers/registry-commands.ts`: prefer copied `../secure-exec/software/*/wasm` artifacts, fall back to `../secure-exec/toolchain/c/build` when available, and let the helper build missing C-source artifacts on demand before declaring the command unavailable. When bootstrapping from secure-exec `toolchain/c`, build `make sysroot` first and then run a second `make` for the concrete `build/...` targets so `SYSROOT` resolves to the patched tree instead of the vanilla SDK sysroot chosen at parse time; in that second pass, treat `sysroot/lib/wasm32-wasi/libc.a` as already built so `make` does not loop back through the patch pipeline because of preserved sysroot timestamps. - `tests/claude-session.test.ts` is the Claude SDK truth suite. It runs the real `@anthropic-ai/claude-agent-sdk` session path through llmock and covers PATH-backed `xu`, text-only replies, nested `node` `execSync` and `spawn`, metadata, lifecycle, and mode updates. Run it with `pnpm --dir packages/core exec vitest run tests/claude-session.test.ts --reporter=verbose` when verifying Claude regressions. - **Kernel permissions are declarative pass-through config.** `AgentOsOptions.permissions` should stay JSON-serializable and be forwarded to the native sidecar without host-side probing or callback evaluation; Rust owns glob matching and policy decisions. - ACP session events are live-only over `onSessionEvent()`. Do not reintroduce sequence numbers, local replay buffers, or event cursor recovery. diff --git a/packages/core/package.json b/packages/core/package.json index 9b71bf2af0..e09daf4caf 100644 --- a/packages/core/package.json +++ b/packages/core/package.json @@ -88,7 +88,6 @@ "@agentos-software/git": "workspace:*", "@agentos-software/grep": "workspace:*", "@agentos-software/gzip": "workspace:*", - "@agentos-software/http-get": "workspace:*", "@agentos-software/jq": "workspace:*", "@agentos-software/opencode": "workspace:*", "@agentos-software/pi": "workspace:*", diff --git a/packages/core/tests/duckdb-package.test.ts b/packages/core/tests/duckdb-package.test.ts index 9b6c0913e2..2c02753dd4 100644 --- a/packages/core/tests/duckdb-package.test.ts +++ b/packages/core/tests/duckdb-package.test.ts @@ -5,19 +5,19 @@ import { type ServerResponse, } from "node:http"; import coreutils from "@agentos-software/coreutils"; +import curl from "@agentos-software/curl"; import { afterEach, beforeEach, describe, expect, test } from "vitest"; import duckdb from "@agentos-software/duckdb"; -import httpGet from "@agentos-software/http-get"; import { AgentOs } from "../dist/index.js"; import { cSysrootPackageSkipReason } from "./helpers/registry-commands.js"; const DUCKDB_PACKAGE = duckdb; -const HTTP_GET_PACKAGE = httpGet; +const CURL_PACKAGE = curl; // C-sysroot packages are the ONE sanctioned skip: they need the patched wasi C // sysroot most checkouts don't build (see helpers/registry-commands.ts). const duckdbPackageSkipReason = cSysrootPackageSkipReason( { pkg: DUCKDB_PACKAGE, name: "duckdb" }, - { pkg: HTTP_GET_PACKAGE, name: "http-get" }, + { pkg: CURL_PACKAGE, name: "curl" }, ); function closeServer(server: Server) { @@ -47,7 +47,7 @@ describe("duckdb registry package", () => { await vm.dispose(); } vm = await AgentOs.create({ - software: options?.software ?? [coreutils, HTTP_GET_PACKAGE, DUCKDB_PACKAGE], + software: options?.software ?? [coreutils, CURL_PACKAGE, DUCKDB_PACKAGE], ...(options?.loopbackExemptPorts ? { loopbackExemptPorts: options.loopbackExemptPorts } : {}), @@ -106,7 +106,7 @@ describe("duckdb registry package", () => { await recreateVm({ loopbackExemptPorts: [address.port] }); let result = await vm.exec( - `http_get ${address.port} /remote.csv /tmp/remote.csv`, + `curl -fsS -o /tmp/remote.csv http://127.0.0.1:${address.port}/remote.csv`, ); expect(result.exitCode, result.stderr || result.stdout).toBe(0); @@ -166,11 +166,11 @@ describe("duckdb registry package", () => { async () => { await vm.dispose(); - const httpGetReadOnly = { - ...HTTP_GET_PACKAGE, - commands: [{ name: "http_get", permissionTier: "read-only" as const }], + const curlReadOnly = { + ...CURL_PACKAGE, + commands: [{ name: "curl", permissionTier: "read-only" as const }], }; - vm = await AgentOs.create({ software: [coreutils, httpGetReadOnly] }); + vm = await AgentOs.create({ software: [coreutils, curlReadOnly] }); const server = createServer((req: IncomingMessage, res: ServerResponse) => { res.writeHead(200, { "Content-Type": "text/plain" }); @@ -187,11 +187,13 @@ describe("duckdb registry package", () => { throw new Error("failed to bind test HTTP server"); } await recreateVm({ - software: [coreutils, httpGetReadOnly], + software: [coreutils, curlReadOnly], loopbackExemptPorts: [address.port], }); - const result = await vm.exec(`http_get ${address.port} /blocked`); + const result = await vm.exec( + `curl -fsS http://127.0.0.1:${address.port}/blocked`, + ); expect(result.exitCode).not.toBe(0); } finally { await closeServer(server); diff --git a/packages/core/tests/helpers/registry-commands.ts b/packages/core/tests/helpers/registry-commands.ts index 65cdcb9ed6..2a0b781c5e 100644 --- a/packages/core/tests/helpers/registry-commands.ts +++ b/packages/core/tests/helpers/registry-commands.ts @@ -12,8 +12,8 @@ * when it exists (local registry builds produce both), so a stale or empty * command set still fails loudly. * - * The only sanctioned exception is the C-sysroot package set (duckdb, - * http-get, sqlite3, zip, unzip): those need the patched wasi C sysroot + * The only sanctioned exception is the C-sysroot package set (curl, duckdb, + * sqlite3, zip, unzip): those need the patched wasi C sysroot * that most checkouts don't have, so `cSysrootPackageSkipReason` reports a * skip reason instead of throwing. Everything else is load-or-throw. */ @@ -160,7 +160,7 @@ export function requireBuilt( } /** - * Skip reason for the C-sysroot package set ONLY (duckdb, http-get, sqlite3, + * Skip reason for the C-sysroot package set ONLY (curl, duckdb, sqlite3, * zip, unzip). These need the patched wasi C sysroot * (`make -C toolchain/c`), which most checkouts don't build — a missing * artifact is an environment limitation, not a forgotten build, so suites may diff --git a/packages/runtime-core/commands/http_get b/packages/runtime-core/commands/http_get deleted file mode 100644 index 52a1fb8342..0000000000 Binary files a/packages/runtime-core/commands/http_get and /dev/null differ diff --git a/packages/runtime-core/tests/integration/cross-runtime-network.test.ts b/packages/runtime-core/tests/integration/cross-runtime-network.test.ts index 5167ea3bad..9336b2320c 100644 --- a/packages/runtime-core/tests/integration/cross-runtime-network.test.ts +++ b/packages/runtime-core/tests/integration/cross-runtime-network.test.ts @@ -19,7 +19,7 @@ import { } from '@rivet-dev/agentos-vm-test-harness'; import type { IntegrationKernelResult, Kernel } from '@rivet-dev/agentos-vm-test-harness'; -const WASM_HTTP_GET = resolve(C_BUILD_DIR, 'http_get'); +const WASM_CURL = resolve(C_BUILD_DIR, 'curl'); const WASM_HTTP_SERVER = resolve(C_BUILD_DIR, 'http_server'); const WASM_TCP_ECHO = resolve(C_BUILD_DIR, 'tcp_echo'); const WASM_TCP_SERVER = resolve(C_BUILD_DIR, 'tcp_server'); @@ -28,7 +28,7 @@ function skipReasonWasmNetwork(): string | false { const wasmSkipReason = skipUnlessWasmBuilt(); if (wasmSkipReason) return wasmSkipReason; for (const [name, path] of [ - ['http_get', WASM_HTTP_GET], + ['curl', WASM_CURL], ['http_server', WASM_HTTP_SERVER], ['tcp_echo', WASM_TCP_ECHO], ['tcp_server', WASM_TCP_SERVER], @@ -231,7 +231,7 @@ describe('cross-runtime network integration', { timeout: 90_000 }, () => { expect(client.stdout.trim()).toBe('js-pong:ping'); }); - itIf(!wasmNetworkSkipReason, 'W1 WASM http_get -> JS node:http server over VM loopback', async () => { + itIf(!wasmNetworkSkipReason, 'W1 WASM curl -> JS node:http server over VM loopback', async () => { ctx = await createIntegrationKernel({ runtimes: ['wasmvm', 'node'], commandDirs: [C_BUILD_DIR, COMMANDS_DIR], @@ -239,13 +239,13 @@ describe('cross-runtime network integration', { timeout: 90_000 }, () => { const server = spawnGuestNodeProgram(ctx.kernel, guestJsHttpServer(3102)); await waitForOutput(server, 'js http listening 3102', 'JS HTTP server'); - const wasm = await ctx.kernel.exec('http_get 3102 /from-wasm'); + const wasm = await ctx.kernel.exec('curl -fsS http://127.0.0.1:3102/from-wasm'); server.process.kill(15); await server.process.wait().catch(() => {}); expect(wasm.exitCode).toBe(0); - expect(wasm.stderr).not.toContain('socket error'); - expect(wasm.stdout).toContain('body: js:GET:/from-wasm'); + expect(wasm.stderr).toBe(''); + expect(wasm.stdout.trim()).toBe('js:GET:/from-wasm'); }); itIf(!wasmNetworkSkipReason, 'J3 JS fetch -> WASM HTTP server over VM loopback', async () => { @@ -341,7 +341,7 @@ describe('cross-runtime network integration', { timeout: 90_000 }, () => { expect(wasm.stdout).toContain('received: js-pong:hello'); }); - itIf(!wasmNetworkSkipReason, 'W3 WASM http_get -> WASM HTTP server over VM loopback', async () => { + itIf(!wasmNetworkSkipReason, 'W3 WASM curl -> WASM HTTP server over VM loopback', async () => { ctx = await createIntegrationKernel({ runtimes: ['wasmvm', 'node'], commandDirs: [C_BUILD_DIR, COMMANDS_DIR], @@ -349,12 +349,12 @@ describe('cross-runtime network integration', { timeout: 90_000 }, () => { const server = spawnGuestProgram(ctx.kernel, 'http_server', ['3108']); await waitForListener(ctx.kernel, 3108, 'WASM HTTP server'); - const wasm = await ctx.kernel.exec('http_get 3108 /from-wasm'); + const wasm = await ctx.kernel.exec('curl -fsS http://127.0.0.1:3108/from-wasm'); const serverExit = await server.process.wait(); expect(wasm.exitCode).toBe(0); - expect(wasm.stderr).not.toContain('socket error'); - expect(wasm.stdout).toContain('body: wasm:GET:/from-wasm'); + expect(wasm.stderr).toBe(''); + expect(wasm.stdout.trim()).toBe('wasm:GET:/from-wasm'); expect(serverExit).toBe(0); }); @@ -426,7 +426,7 @@ describe('cross-runtime network integration', { timeout: 90_000 }, () => { } }); - itIf(!wasmNetworkSkipReason, 'O2 WASM http_get -> host loopback requires loopback exemption', async () => { + itIf(!wasmNetworkSkipReason, 'O2 WASM curl -> host loopback requires loopback exemption', async () => { const seenRequests: string[] = []; const hostServer = createHttpServer((req, res) => { seenRequests.push(req.url ?? ''); @@ -443,9 +443,9 @@ describe('cross-runtime network integration', { timeout: 90_000 }, () => { runtimes: ['wasmvm', 'node'], commandDirs: [C_BUILD_DIR, COMMANDS_DIR], }); - const noExemption = await ctx.kernel.exec(`http_get ${port} /blocked`); + const noExemption = await ctx.kernel.exec(`curl -fsS http://127.0.0.1:${port}/blocked`); expect(noExemption.exitCode).not.toBe(0); - expect(noExemption.stderr).toMatch(/EACCES|Bad address|Connection refused|connect/); + expect(noExemption.stderr).toMatch(/EACCES|Bad address|Connection refused|connect|Failed to connect|Invalid argument/); expect(seenRequests).toEqual([]); await ctx.dispose(); @@ -454,10 +454,10 @@ describe('cross-runtime network integration', { timeout: 90_000 }, () => { commandDirs: [C_BUILD_DIR, COMMANDS_DIR], loopbackExemptPorts: [port], }); - const allowed = await ctx.kernel.exec(`http_get ${port} /allowed`); + const allowed = await ctx.kernel.exec(`curl -fsS http://127.0.0.1:${port}/allowed`); expect(allowed.exitCode).toBe(0); - expect(allowed.stderr).not.toContain('socket error'); - expect(allowed.stdout).toContain('body: host:/allowed'); + expect(allowed.stderr).toBe(''); + expect(allowed.stdout.trim()).toBe('host:/allowed'); expect(seenRequests).toEqual(['/allowed']); } finally { await new Promise((resolveClose) => hostServer.close(() => resolveClose())); diff --git a/packages/shell/package.json b/packages/shell/package.json index 254004e6b4..38631b38c9 100644 --- a/packages/shell/package.json +++ b/packages/shell/package.json @@ -26,7 +26,6 @@ "@agentos-software/git": "workspace:*", "@agentos-software/grep": "workspace:*", "@agentos-software/gzip": "workspace:*", - "@agentos-software/http-get": "workspace:*", "@agentos-software/jq": "workspace:*", "@agentos-software/ripgrep": "workspace:*", "@agentos-software/sed": "workspace:*", diff --git a/packages/shell/src/main.ts b/packages/shell/src/main.ts index f3e6f6996b..d7abe702f4 100644 --- a/packages/shell/src/main.ts +++ b/packages/shell/src/main.ts @@ -35,7 +35,6 @@ import gawk from "@agentos-software/gawk"; import git from "@agentos-software/git"; import grep from "@agentos-software/grep"; import gzip from "@agentos-software/gzip"; -import httpGet from "@agentos-software/http-get"; import jq from "@agentos-software/jq"; import ripgrep from "@agentos-software/ripgrep"; import sed from "@agentos-software/sed"; @@ -153,7 +152,6 @@ const software: SoftwareInput[] = [ yq, codex, git, - httpGet, sqlite3, wget, ] diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 6ab0cccb90..86a6acee1d 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -2544,9 +2544,6 @@ importers: '@agentos-software/gzip': specifier: workspace:* version: link:../../software/gzip - '@agentos-software/http-get': - specifier: workspace:* - version: link:../../software/http-get '@agentos-software/jq': specifier: workspace:* version: link:../../software/jq @@ -2853,9 +2850,6 @@ importers: '@agentos-software/gzip': specifier: workspace:* version: link:../../software/gzip - '@agentos-software/http-get': - specifier: workspace:* - version: link:../../software/http-get '@agentos-software/jq': specifier: workspace:* version: link:../../software/jq @@ -3472,27 +3466,6 @@ importers: specifier: ^2.1.9 version: 2.1.9(@types/node@22.19.15) - software/http-get: - devDependencies: - '@agentos-software/manifest': - specifier: workspace:* - version: link:../../packages/manifest - '@agentos/test-harness': - specifier: workspace:* - version: link:../../test-harness - '@rivet-dev/agentos-toolchain': - specifier: workspace:* - version: link:../../packages/agentos-toolchain - '@types/node': - specifier: ^22.10.2 - version: 22.19.15 - typescript: - specifier: ^5.9.2 - version: 5.9.3 - vitest: - specifier: ^2.1.9 - version: 2.1.9(@types/node@22.19.15) - software/jq: devDependencies: '@agentos-software/manifest': diff --git a/software/duckdb/test/duckdb.test.ts b/software/duckdb/test/duckdb.test.ts index 178039b040..06349c73fb 100644 --- a/software/duckdb/test/duckdb.test.ts +++ b/software/duckdb/test/duckdb.test.ts @@ -33,7 +33,6 @@ import { setTimeout as sleep } from 'node:timers/promises'; const hasWasmDuckDB = existsSync(resolve(C_BUILD_DIR, 'duckdb')); const hasWasmCurl = (existsSync(resolve(COMMANDS_DIR, 'curl')) || existsSync(resolve(C_BUILD_DIR, 'curl'))); -const hasWasmHttpGet = existsSync(resolve(C_BUILD_DIR, 'http_get')); async function mountKernel( filesystem: ReturnType, @@ -52,7 +51,7 @@ async function mountKernel( commandDirs, permissions: { duckdb: 'read-write', - http_get: 'full', + curl: 'full', }, }) ); @@ -215,7 +214,7 @@ describeIf(hasWasmDuckDB, 'duckdb command', { timeout: 120_000 }, () => { }); itIf( - hasWasmCurl || hasWasmHttpGet, + hasWasmCurl, 'queries data fetched over the network through the shared VFS', async () => { const filesystem = createInMemoryFileSystem(); @@ -243,18 +242,10 @@ describeIf(hasWasmDuckDB, 'duckdb command', { timeout: 120_000 }, () => { loopbackExemptPorts: [address.port], }); - let result; - if (hasWasmHttpGet) { - result = await kernel.exec( - `http_get ${address.port} /remote.csv /tmp/remote.csv` - ); - expect(result.exitCode).toBe(0); - } else { - result = await kernel.exec( - `curl -fsS -o /tmp/remote.csv http://127.0.0.1:${address.port}/remote.csv` - ); - expect(result.exitCode).toBe(0); - } + let result = await kernel.exec( + `curl -fsS -o /tmp/remote.csv http://127.0.0.1:${address.port}/remote.csv` + ); + expect(result.exitCode).toBe(0); expect(await filesystem.readTextFile('/tmp/remote.csv')).toContain('city,value'); diff --git a/software/http-get/agentos-package.json b/software/http-get/agentos-package.json deleted file mode 100644 index 4f8fb816f9..0000000000 --- a/software/http-get/agentos-package.json +++ /dev/null @@ -1,10 +0,0 @@ -{ - "commands": [ - "http_get" - ], - "registry": { - "title": "http-get", - "description": "Minimal HTTP GET fetch helper.", - "category": "networking" - } -} diff --git a/software/http-get/native/c/http_get.c b/software/http-get/native/c/http_get.c deleted file mode 100644 index 0e7dc466ea..0000000000 --- a/software/http-get/native/c/http_get.c +++ /dev/null @@ -1,95 +0,0 @@ -/* http_get.c — connect to an HTTP server, send GET request, print response body */ -#include -#include -#include -#include -#include -#include -#include - -int main(int argc, char *argv[]) { - if (argc < 2) { - fprintf(stderr, "usage: http_get [path] [output_file]\n"); - return 1; - } - - int port = atoi(argv[1]); - const char *path = argc >= 3 ? argv[2] : "/"; - const char *output_file = argc >= 4 ? argv[3] : NULL; - - int fd = socket(AF_INET, SOCK_STREAM, 0); - if (fd < 0) { - perror("socket"); - return 1; - } - - struct sockaddr_in addr; - memset(&addr, 0, sizeof(addr)); - addr.sin_family = AF_INET; - addr.sin_port = htons((uint16_t)port); - inet_pton(AF_INET, "127.0.0.1", &addr.sin_addr); - - if (connect(fd, (struct sockaddr *)&addr, sizeof(addr)) < 0) { - perror("connect"); - close(fd); - return 1; - } - - char request[1024]; - int request_len = snprintf( - request, - sizeof(request), - "GET %s HTTP/1.0\r\nHost: localhost\r\n\r\n", - path - ); - if (request_len < 0 || (size_t)request_len >= sizeof(request)) { - fprintf(stderr, "request path too long\n"); - close(fd); - return 1; - } - - ssize_t sent = send(fd, request, (size_t)request_len, 0); - if (sent < 0) { - perror("send"); - close(fd); - return 1; - } - - /* Read full response */ - char response[4096]; - size_t total = 0; - ssize_t n; - while ((n = recv(fd, response + total, sizeof(response) - total - 1, 0)) > 0) { - total += (size_t)n; - } - response[total] = '\0'; - - close(fd); - - /* Find body after \r\n\r\n */ - const char *body = strstr(response, "\r\n\r\n"); - if (body) { - body += 4; - if (output_file) { - FILE *out = fopen(output_file, "wb"); - if (!out) { - perror("fopen"); - return 1; - } - size_t body_len = total - (size_t)(body - response); - if (fwrite(body, 1, body_len, out) != body_len) { - perror("fwrite"); - fclose(out); - return 1; - } - fclose(out); - } else { - printf("body: %s\n", body); - } - } else { - printf("body: (no separator found)\n"); - return 1; - } - - return 0; -} diff --git a/software/http-get/package.json b/software/http-get/package.json deleted file mode 100644 index ef23733846..0000000000 --- a/software/http-get/package.json +++ /dev/null @@ -1,33 +0,0 @@ -{ - "name": "@agentos-software/http-get", - "version": "0.3.3", - "type": "module", - "license": "Apache-2.0", - "description": "Minimal HTTP GET fetch helper for secure-exec VMs", - "main": "./dist/index.js", - "types": "./dist/index.d.ts", - "files": [ - "dist", - "!dist/package", - "!dist/package.tar" - ], - "exports": { - ".": { - "types": "./dist/index.d.ts", - "import": "./dist/index.js" - } - }, - "scripts": { - "build": "agentos-toolchain stage --commands-dir ../../toolchain/target/wasm32-wasip1/release/commands --if-missing skip && tsc && agentos-toolchain build", - "check-types": "tsc --noEmit", - "test": "vitest run test/ --passWithNoTests" - }, - "devDependencies": { - "@agentos-software/manifest": "workspace:*", - "@rivet-dev/agentos-toolchain": "workspace:*", - "@types/node": "^22.10.2", - "typescript": "^5.9.2", - "@agentos/test-harness": "workspace:*", - "vitest": "^2.1.9" - } -} diff --git a/software/http-get/src/index.ts b/software/http-get/src/index.ts deleted file mode 100644 index dba100e2e1..0000000000 --- a/software/http-get/src/index.ts +++ /dev/null @@ -1,5 +0,0 @@ -import type { SoftwarePackageRef } from "@agentos-software/manifest"; - -const packagePath = new URL("./package.aospkg", import.meta.url).pathname; - -export default { packagePath } satisfies SoftwarePackageRef; diff --git a/software/http-get/test/manifest.test.ts b/software/http-get/test/manifest.test.ts deleted file mode 100644 index 89139d33b8..0000000000 --- a/software/http-get/test/manifest.test.ts +++ /dev/null @@ -1,19 +0,0 @@ -import { readFileSync } from "node:fs"; -import { join } from "node:path"; -import { describe, expect, it } from "vitest"; - -const packageDir = new URL("..", import.meta.url).pathname; - -describe("package manifest", () => { - it("declares command binaries", () => { - const manifest = JSON.parse( - readFileSync(join(packageDir, "agentos-package.json"), "utf8"), - ); - - expect(manifest.commands?.length ?? 0).toBeGreaterThan(0); - for (const command of manifest.commands) { - expect(typeof command).toBe("string"); - expect(command.length).toBeGreaterThan(0); - } - }); -}); diff --git a/software/http-get/tsconfig.json b/software/http-get/tsconfig.json deleted file mode 100644 index 03ce790ab7..0000000000 --- a/software/http-get/tsconfig.json +++ /dev/null @@ -1,8 +0,0 @@ -{ - "extends": "../tsconfig.base.json", - "compilerOptions": { - "outDir": "./dist", - "rootDir": "./src" - }, - "include": ["src/**/*"] -} diff --git a/toolchain/Makefile b/toolchain/Makefile index 1761fd6dd2..06b41a7c08 100644 --- a/toolchain/Makefile +++ b/toolchain/Makefile @@ -213,7 +213,7 @@ wasm: vendor patch-vendor patch-std wasm-opt-check # codex the codex fork build (codex, codex-exec) # `just toolchain-cmd ` calls this. `make wasm` builds the fast # Rust set; `make -C c programs install` builds/installs the fast C set. -C_COMMANDS := zip unzip envsubst sqlite3 curl wget duckdb vim http_get +C_COMMANDS := zip unzip envsubst sqlite3 curl wget duckdb vim .PHONY: cmd/% cmd/%: diff --git a/toolchain/c/Makefile b/toolchain/c/Makefile index 1d23d461e2..d11cc666b8 100644 --- a/toolchain/c/Makefile +++ b/toolchain/c/Makefile @@ -68,10 +68,10 @@ COMMANDS_DIR ?= ../target/wasm32-wasip1/release/commands # Fast real commands installed by the default software gate. Slow/heavy commands # (duckdb, vim) remain available through explicit parent `make cmd/`. -COMMANDS := zip unzip envsubst sqlite3 curl wget tree http_get +COMMANDS := zip unzip envsubst sqlite3 curl wget tree # Programs requiring patched sysroot (Tier 2+ custom host imports) -PATCHED_PROGRAMS := http_get_test isatty_test getpid_test getppid_test getppid_verify userinfo pipe_test dup_test spawn_child spawn_exit_code pipeline kill_child waitpid_return waitpid_edge syscall_coverage getpwuid_test signal_tests sigaction_self sigaction_behavior delayed_tcp_echo delayed_kill pipe_edge tcp_accept_spawn tcp_echo tcp_server http_server udp_echo unix_socket signal_handler http_get dns_lookup sqlite3 curl wget tree fs_probe +PATCHED_PROGRAMS := http_get_test isatty_test getpid_test getppid_test getppid_verify userinfo pipe_test dup_test spawn_child spawn_exit_code pipeline kill_child waitpid_return waitpid_edge syscall_coverage getpwuid_test signal_tests sigaction_self sigaction_behavior delayed_tcp_echo delayed_kill pipe_edge tcp_accept_spawn tcp_echo tcp_server http_server udp_echo unix_socket signal_handler dns_lookup sqlite3 curl wget tree fs_probe # Discover all package command and test-program C source files. ALL_SOURCES := $(foreach dir,$(C_SOURCE_DIRS),$(wildcard $(dir)/*.c)) diff --git a/toolchain/conformance/c-parity.test.ts b/toolchain/conformance/c-parity.test.ts index a106b22530..17e3271bb3 100644 --- a/toolchain/conformance/c-parity.test.ts +++ b/toolchain/conformance/c-parity.test.ts @@ -942,7 +942,7 @@ describeIf(!skipReason(), 'C parity: native vs WASM', { timeout: 30_000 }, () => } }); - itIf(!netSkip, 'http_get: connect to HTTP server, receive response body', async () => { + itIf(!netSkip, 'curl: connect to HTTP server, receive response body', async () => { // Start a local HTTP server const server = createHttpServer((_req, res) => { res.writeHead(200, { 'Content-Type': 'text/plain' }); @@ -953,14 +953,11 @@ describeIf(!skipReason(), 'C parity: native vs WASM', { timeout: 30_000 }, () => try { await recreateKernel({ loopbackExemptPorts: [port] }); - const native = await runNative('http_get', [String(port)]); - const wasm = await kernel.exec(`http_get ${port}`); + const wasm = await kernel.exec(`curl -fsS http://127.0.0.1:${port}/`); - expect(wasm.exitCode).toBe(native.exitCode); expect(wasm.exitCode).toBe(0); - expect(wasm.stdout).toBe(native.stdout); - expect(normalizeStderr(wasm.stderr)).toBe(normalizeStderr(native.stderr)); - expect(wasm.stdout).toContain('body: hello from http'); + expect(wasm.stderr).toBe(''); + expect(wasm.stdout.trim()).toBe('hello from http'); } finally { server.close(); } diff --git a/website/src/generated/registry.json b/website/src/generated/registry.json index d65f37e211..c3c75c37fb 100644 --- a/website/src/generated/registry.json +++ b/website/src/generated/registry.json @@ -403,18 +403,6 @@ "package": "@agentos-software/codex-cli", "status": "available", "image": "/images/registry/codex.svg" - }, - { - "slug": "http-get", - "title": "http-get", - "description": "Minimal HTTP GET fetch helper.", - "types": [ - "software" - ], - "category": "networking", - "priority": 0, - "package": "@agentos-software/http-get", - "status": "available" } ] }