Skip to content

Add executeTool() #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
MiguelsPizza merged 1 commit into from
Sep 17, 2026
Merged

Add executeTool() #2

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Working on the polyfill

This package implements document-local WebMCP registration and discovery. Use
the official `webmcp-types` dependency without duplicating or augmenting its
declarations. Execution is deferred to a separate change alongside types PR #3.
This package implements the document-local imperative WebMCP draft. Use the
official `webmcp-types` dependency; do not duplicate its declarations. Keep the
temporary `executeTool` augmentation only until the upstream package ships it.
No MCP server, transport, extension product, navigator aliases, or legacy API
compatibility belongs here.

Expand Down
7 changes: 4 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,8 @@ await context.registerTool(
{ signal: registration.signal },
);

console.log(await context.getTools());
const [tool] = await context.getTools();
console.log(await context.executeTool(tool, {})); // {"title":"WebMCP demo"}

// Remove the tool when it is no longer needed.
registration.abort();
Expand All @@ -49,9 +50,9 @@ To install explicitly, import and call `installWebMCP` from `webmcp-polyfill`. I

Tools stay in the current document. Cross-document tools, declarative forms, lifecycle window events, and browser agent integration aren't implemented. Nonempty `exposedTo` and `fromOrigins` options reject.

This version supports registration, discovery, and `toolchange`. It does not invoke registered callbacks; `executeTool()` follows alongside [the types update](https://github.com/webmachinelearning/webmcp-types/pull/3).
`executeTool()` accepts an object and returns a JSON-serialized result. Callbacks must validate their inputs; schema inference provides TypeScript checks only.

The implementation follows [draft source `cc45efc`](https://github.com/webmachinelearning/webmcp/blob/cc45efcaf0/index.bs) and passes all 27 selected assertions of the [upstream WPT](https://github.com/web-platform-tests/wpt/tree/1a21db90adf8a264370ad806ed761f39e1d435a0/webmcp) at the pin, with no expected failures. [TESTING.md](https://github.com/webmachinelearning/webmcp-polyfill/blob/main/TESTING.md) records the selection and what it excludes.
The implementation follows [draft source `cc45efc`](https://github.com/webmachinelearning/webmcp/blob/cc45efcaf0/index.bs). Of the 56 selected assertions of the [upstream WPT](https://github.com/web-platform-tests/wpt/tree/1a21db90adf8a264370ad806ed761f39e1d435a0/webmcp) at the pin, 51 pass and five fail where the pinned tests disagree with the draft's `executeTool()` input and result rules. [TESTING.md](https://github.com/webmachinelearning/webmcp-polyfill/blob/main/TESTING.md) records each one and what the selection excludes.

The API tracks the draft. Breaking changes ship with notes: in minor releases while the version is 0.x, in majors after 1.0.

Expand Down
99 changes: 71 additions & 28 deletions TESTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,15 @@ pnpm test
pnpm test:package
```

| Check | What it runs |
| ------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| `index.test-d.ts` | TypeScript against the built package declarations, including upstream schema inference |
| `index.test.ts` | Built bundle served over HTTP in Chromium (native WebMCP disabled), Firefox, and WebKit; coercion, metadata, events, registration abort, errors, detached documents |
| `app.test.ts` | A served application, real button interactions, discovery, unregistration, and reload |
| `native.test.ts` | Real native Chromium registration, then polyfill loading; context and getter identities must survive |
| `pnpm test:package` | Packed tarball installed into a fresh consumer, public type imports, SSR-safe entry points, and package contents |
| `pnpm test:wpt` | Unmodified upstream registration/discovery WPT in real Chrome Canary, with native WebMCP disabled |
| Check | What it runs |
| ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `index.test-d.ts` | TypeScript against the built package declarations, including upstream schema inference |
| `index.test.ts` | Built bundle served over HTTP in Chromium (native WebMCP disabled), Firefox, and WebKit; coercion, metadata, events, errors, detached documents |
| `execute.test.ts` | The same three engines: object input, JSON results, cancellation, concurrent calls, and dispatch-time failures |
| `app.test.ts` | A served application, real button interactions, callback side effects, invalid input, unregistration, and reload |
| `native.test.ts` | Real native Chromium registration, then polyfill loading; context and getter identities must survive |
| `pnpm test:package` | Packed tarball installed into a fresh consumer, public type imports, SSR-safe entry points, and package contents |
| `pnpm test:wpt` | Unmodified upstream WPT and IDL in real Chrome Canary, with native WebMCP disabled |

The fixture server binds 127.0.0.1:8793 and sets the required `Origin-Agent-Cluster`
header; Playwright never reuses an existing server, so free that port first.
Expand Down Expand Up @@ -46,39 +47,74 @@ WPT_ROOT=../wpt CHROME_BIN=/path/to/chrome-canary pnpm test:wpt
```

`WPT_PYTHON` and `WPT_VENV` optionally select the interpreter and environment.
The runner checks the revision, rejects tracked source changes, requires all 13 selected files
The runner checks the revision, rejects tracked source changes, requires all 18 selected files
to run exactly once, checks assertion counts, and retains `wpt-results/report.json`
with browser and upstream revisions. Runner errors, unexpected failures, and
unexpected passes fail the command, and nothing is retried.

At this pin, **all 27 selected assertions pass**, with no expected failures.
At this pin, 56 assertions run: **51 pass and five have explicit expected FAIL
metadata**. All 22 IDL assertions pass. API shape coverage does not prove runtime
defaults or complete conformance.

### Known draft disagreements

The [published draft](https://webmachinelearning.github.io/webmcp/#dom-modelcontext-executetool)
accepts optional `any inputObject`, rejects non-objects, and has no `{}` default.
It also JSON-serializes callback results, including strings. The pinned WPT
expects a default object and raw string results in the following cases:

| Upstream file | Expected failures | Local coverage |
| --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------- |
| `executeTool-invalid-dictionary.https.html` | Missing tool invoked without input expects `UnknownError`, rather than the draft's earlier input `TypeError` | Descriptor validation and input errors |
| `executeTool-error-window-onerror.https.html` | Both cases omit input and expect execution to start | Explicit object input, callback/serialization failures, and absence of page errors |
| `executeTool-unregister-resolution-race.https.html` | Invocation omits input | Unregistration during an invocation and its successful result |
| `object-arguments.https.html` | Expects an unquoted string result; also expects omitted input to become `{}` | Array/object inputs, rejected primitive inputs, and JSON result serialization |

These are five assertions in four files, recorded individually in `wpt-metadata`.
The pin, [1a21db9](https://github.com/web-platform-tests/wpt/commit/1a21db90adf8a264370ad806ed761f39e1d435a0),
is the WPT export of the Chromium change for
[spec PR #246](https://github.com/webmachinelearning/webmcp/pull/246) and
[#251](https://github.com/webmachinelearning/webmcp/pull/251), and it rewrote all four of these
files. The draft's input and result rules have not changed since #251 merged, so these are not
stale tests awaiting an update: four of the assertions invoke `executeTool()` with no input,
which the draft rejects with a `TypeError` before it looks the tool up, and `object-arguments`
asserts an unquoted `"Success"` from a callback whose result the draft JSON-serializes. Re-read the live
draft before changing any expectation. An expected failure can stop at its first assertion, so
the local tests cover the behavior after that point.

`webmcp-types` PR #3 and the augmentation in `index.ts` both declare `inputObject?: object`,
which is narrower than the IDL's `any`: omitting the argument type-checks and then rejects at
runtime, as the draft requires.

### Excluded coverage

Execution tests and the full IDL harness are deferred with `executeTool()`.
The IDL includes that method, so running the full shape suite against this
registration/discovery subset would intentionally fail. Keep that exclusion
explicit rather than adding expected failures for an API not yet included.

Cross-document discovery, frame-tree routing, declarative forms, browser
permissions integration, CSS states, navigation/BFCache, and lifecycle window
events are also outside this initial implementation.
Cross-document discovery/execution, frame-tree routing, navigation cancellation,
declarative forms, browser permissions integration, CSS states, and lifecycle
window events are outside this initial implementation. The pinned
`executeTool-abort.https.html` is excluded rather than carried as expected
failures: its second subtest waits forever on a `toolactivated` event that the
draft still leaves [unspecified](https://github.com/webmachinelearning/webmcp/issues/146),
so the file times out and its last three subtests never run at all. Expected-failure
metadata cannot express that, and cancellation is exercised directly in all three
engines instead.
`exposedTo-invalid-origins.https.html` is excluded for the `exposedTo` divergence
recorded below, not for cross-document routing.

WPT's `--inject-script` modifies testharness pages; it does not install the bundle
in `/common/blank.html` helper frames. Detached-frame WPT therefore cannot run
unchanged in this lane. The local test serves an instrumented iframe, removes
it, and checks registration, discovery, and exception realms.
it, and checks all three operations and exception realms.

## Where to look when upstream changes

| Source | Use |
| --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ |
| [Draft source and history](https://github.com/webmachinelearning/webmcp/commits/main/index.bs) | Normative algorithms, Web IDL, open issues; compare with the revision below |
| [WPT webmcp](https://github.com/web-platform-tests/wpt/tree/master/webmcp) and [results](https://wpt.fyi/results/webmcp) | Executable assertions and native cross-browser results; these results are not polyfill results |
| [WPT IDL](https://github.com/web-platform-tests/wpt/blob/master/interfaces/webmcp.idl) | Generated interface snapshot; it can lag the published draft |
| [Official types](https://github.com/webmachinelearning/webmcp-types) | Public declarations, schema inference, and pending API updates |
| [Blink script_tools](https://chromium.googlesource.com/chromium/src/+/main/third_party/blink/renderer/core/script_tools/) | Chromium IDL, implementation, tests, and commit-linked bugs |
| [Gecko source search](https://searchfox.org/mozilla-central/search?q=ModelContext) and [Mozilla position](https://github.com/mozilla/standards-positions/issues/1412) | Locate Firefox implementation work and discussion; a position is not evidence of shipped support |
| Source | Use |
| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ |
| [Draft source and history](https://github.com/webmachinelearning/webmcp/commits/main/index.bs) | Normative algorithms, Web IDL, open issues; compare with the revision below |
| [WPT webmcp](https://github.com/web-platform-tests/wpt/tree/master/webmcp) and [results](https://wpt.fyi/results/webmcp) | Executable assertions and native cross-browser results; these results are not polyfill results |
| [WPT IDL](https://github.com/web-platform-tests/wpt/blob/master/interfaces/webmcp.idl) | Generated interface snapshot; it can lag the published draft |
| [Official types](https://github.com/webmachinelearning/webmcp-types) | Public declarations, schema inference, and pending API updates |
| [Blink script_tools](https://chromium.googlesource.com/chromium/src/+/main/third_party/blink/renderer/core/script_tools/) | Chromium IDL, implementation, tests, and commit-linked bugs |
| [Gecko source search](https://searchfox.org/mozilla-central/search?q=ModelContext) and [Mozilla position](https://github.com/mozilla/standards-positions/issues/1412) | Locate Firefox implementation work and discussion; a position is not evidence of shipped support |

Reproduce a disagreement before changing code or expectations, and record what
changed in the draft, types, WPT, and browser implementation separately.
Expand All @@ -93,7 +129,10 @@ explicit `any`. Warnings fail the command, and `pnpm test` runs it before compil
The implementation was compared with [draft source `cc45efc`](https://github.com/webmachinelearning/webmcp/blob/cc45efcaf0/index.bs).
It uses timers to queue tasks; JavaScript cannot reproduce the browser's WebMCP
task source, nor the draft's abort *algorithms*, which run before an abort event
rather than as a listener.
rather than as a listener. One consequence is observable: a signal aborted before
the dispatch timer fires rejects the caller and never runs the callback, where the
draft dispatches to the target document and then cancels through the callback's own
signal.

The runtime checks the `tools` Permissions Policy when the browser exposes it.
No engine lists `tools` in `permissionsPolicy.features()` today, so that branch
Expand All @@ -110,6 +149,10 @@ implementation has nowhere to expose a tool to; rejecting keeps the polyfill fro
implying cross-document support it does not have. Origins are validated first, so
an untrustworthy one still fails with the `SecurityError` the draft requires.

`webmcp-types@0.1.7` does not declare `executeTool()`. The method augmentation
in `index.ts` is temporary; remove it when [types PR #3](https://github.com/webmachinelearning/webmcp-types/pull/3)
is included in a release.

To test types changes, keep `webmcp-types` beside this checkout and run
`pnpm link ../webmcp-types`, then `pnpm typecheck`. Restore the published
dependency with `pnpm install --force` before package checks. Do not commit
Expand Down
14 changes: 8 additions & 6 deletions app.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { test, expect } from "@playwright/test";

test("a served application registers, discovers, unregisters, and resets on reload", async ({
test("a served application executes, rejects invalid input, unregisters, and resets on reload", async ({
page,
browser,
}, testInfo) => {
Expand All @@ -12,11 +12,13 @@ test("a served application registers, discovers, unregisters, and resets on relo
page.on("pageerror", (error) => errors.push(error));
await page.goto("/app");
await expect(page.locator("#status")).toHaveText("registered");
expect(
await page.evaluate(async () =>
(await document.modelContext!.getTools()).map((tool) => tool.name),
),
).toEqual(["increment"]);
await page.getByRole("button", { name: "Execute on page" }).click();
await expect(page.locator("#result")).toHaveText('{"count":2}');
await expect(page.locator("#count")).toHaveText("2");
await page.getByLabel("Amount").fill("-1");
await page.getByRole("button", { name: "Execute on page" }).click();
await expect(page.locator("#result")).toHaveText("UnknownError");
await expect(page.locator("#count")).toHaveText("2");
await page.getByRole("button", { name: "Unregister", exact: true }).click();
await expect(page.locator("#status")).toHaveText("unregistered");
expect(await page.evaluate(() => document.modelContext!.getTools())).toEqual([]);
Expand Down
Loading