Repository navigation
Pin the Harness at 4afed45f and contract 0.5 - #24
Merged
Merged
Conversation
The Harness at `4afed45fe5a4198ee9c9617396377d83b88f5e16` performs each file operation of a tool as a `Vfs` effect and records it, and contract 0.5 lets a library Plugin change files. The seven pinned lines of `Cargo.toml` move to that commit, to `52a4905aca0a9c945a93dc4ee7ea815db85612e4` of `runvil-plugin-api`, to `a806bf63e3b0a23c85f3599a1b03fea578c77a2d` of `runvil-plugin-std`, which implements Write, StrReplace, and Delete, and to `f348f7a67b9087bcbc55ea7cb61c8de3fc409c82` of `runvil-plugin-bashkit`, whose shell changes files. Workshop changes one line of production code for the new API, and its tests pin what the tools do now. - `parse` in `conversation-run.rs` returns `Arc<Prompt>`, the type that `host.parse` now gives and the Harness takes. No other production code changes: the UI shows events and no effect, and the run log stores the payload of each record as the Harness gives it. - `std_real_directory.rs` lends its `ToolContext` the files `AnsweredAtOnce`, which answers each request with `perform` on the `Access` of the test, because a `ToolContext` holds no access now. Its new test makes, edits, and deletes a file of the real directory. - `chat_gate/files.rs` is new. In GATE 15 the `Write`, `StrReplace`, and `Delete` calls of the model change a file of the run of the chat, and the run log holds the write as a `Vfs` effect that names the `Write` call, with its answer. In GATE 16 the `Shell` call of the model makes a file that `Read` reads. GATE 13 also expects `Write`, `StrReplace`, and `Delete` in the tool list of each round. - `spawn_over_gateway` and `closed_run_log` in `chat_gate.rs` hold code that `compaction.rs` and `lifecycle.rs` had, so the new gates share it. - Two tests in `agents/tests.rs` change because the tools changed. The std test expected the text of the Write stub. It now expects `Wrote 1 byte.`, `Made 1 replacement.`, and `Deleted the file.`, and it reads each change back. The bashkit test expected exit code 1 and `the shell cannot change files in this build`. It now expects exit code 0, a file under a new directory, and an edit by `sed -i`. - The definition of the Engine in `AGENTS.md` says "a file operation" where it said "a store operation", because a file request of a tool is a `Vfs` effect too. The comment in `Cargo.toml` says version 0.5. - A model, the `tools.call` of a prompt, and the shell can now create, change, and delete files of a run, and no approval is asked. The filesystem of a run has the default policy, `AllowAll`, which the module doc of `agents.rs` now states. That filesystem is an in-memory root that ends with the run, so no file on disk changes. The two mounts of real directories stay read-only with both switches off, and two new tests in `language_guide/tests.rs` show that the three tools and the shell change nothing under such a mount. - An agent can name agents of the agents folder under `prompts:`, by the name that discovery lists, and run them with `execute`. `an_agent_executes_an_agent_of_the_folder_that_its_prompts_list_names` pins it. The contract route parses a prompt that has a `prompts:` list from the posted text alone, which the doc of `contract` and one new test state. - `Cargo.lock` changes the 19 `source` lines of the crates of the four repositories, and the `version` of `runvil-plugin-sdk` and of `runvil-plugin-sys` from 0.4.0 to 0.5.0. It adds no crate and removes none. - The pins move across 38 commits of `runvil-harness`, 4 of `runvil-plugin-api`, 15 of `runvil-plugin-std`, and 5 of `runvil-plugin-bashkit`. - The built-in chat prompt, the README, the UI, the protocol, the run log, and the build tooling do not change. Glob is still a stub in the pinned std Plugin. - No test covers a write of more than a few bytes in `runs.db`, a Pack of another vendor, which every run now holds, or a run with one of the two switches on. No gate covers a file tool call of the model that fails. Such a call still reaches the operator as "Tool call failed". Plan: runvil-harness/plan/2026-11-16-1-tool-file-effects.plan.md
The advisory `RUSTSEC-2026-0542` marks `humantime` as unmaintained, and cargo-deny fails on it, which stops the pre-push hook and the supply-chain job. The `ignore` list of `deny.toml` now holds the advisory with a `reason`, so the check passes. The advisory reports no vulnerability and no safe upgrade. - The entry is a stopgap. Its comment says that `workshop-workspace` uses one function of the crate, `format_rfc3339_seconds`, to stamp a time, and that the entry goes when the crate is replaced. - The diff changes no code and no manifest. `humantime` stays a dependency and stays in the lock file, and nothing replaces it yet. - The diff adds no ignore list for cargo-audit. This repository has none, and cargo-audit reports the advisory as a warning and exits with 0.
There was a problem hiding this comment.
🔵 Needs a closer look
The ABI-breaking Plugin contract upgrade depends on unavailable upstream implementations, and rejection of contract-0.4 libraries remains untested, requiring human validation of the dynamic-loading boundary.
No open issues 🎉
What changed in this PR
Updates Workshop to Harness 4afed45f and Plugin contract 0.5, enabling file tools and the shell to change run-local files through operations the Harness performs and records.
Changes:
- Aligns dependency pins and adapts prompt parsing to return
Arc<Prompt>. - Adds coverage for file changes, read-only mounts, recorded file operations, and nested agent execution; shares chat test helpers.
- Documents filesystem behavior and temporarily accepts the
humantimeunmaintained advisory.
| File | Description |
|---|---|
deny.toml |
Adds a documented exception for the humantime advisory. |
crates/workshop/server/tests/it/std_real_directory.rs |
Adapts to ToolFiles and tests real-directory file changes. |
crates/workshop/server/tests/it/chat_gate/lifecycle.rs |
Reuses the shared closed-run-log helper. |
crates/workshop/server/tests/it/chat_gate/files.rs |
Adds chat gates for file changes, shell interoperability, and recording. |
crates/workshop/server/tests/it/chat_gate/compaction.rs |
Reuses shared server setup. |
crates/workshop/server/tests/it/chat_gate/canonical_sequence.rs |
Checks that chat advertises file-changing tools. |
crates/workshop/server/tests/it/chat_gate.rs |
Extracts shared helpers and registers file gates. |
crates/workshop/server/src/routes/prompts.rs |
Documents contract parsing with referenced prompts. |
crates/workshop/server/src/routes/prompts-tests.rs |
Tests contract responses for prompts listing other prompts. |
crates/workshop/server/src/agents/tests.rs |
Tests nested execution and file-changing Plugin tools. |
crates/workshop/server/src/agents/language_guide/tests.rs |
Tests read-only mount protection and adjacent writable storage. |
crates/workshop/server/src/agents.rs |
Documents default filesystem policy and run-local file lifetime. |
crates/workshop/agents/src/conversation-run.rs |
Updates the parse return type to Arc<Prompt>. |
Cargo.toml |
Aligns Harness and Plugin dependency pins with contract 0.5. |
Cargo.lock |
Locks the updated commits and contract versions. |
AGENTS.md |
Updates the Engine definition to name file operations. |
Review effort: Balanced
Member
Author
|
@claude review |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Workshop moves its four pins to the commits where a tool's file operations are
Vfseffects that the Harness performs and records. The Plugin contract goes from 0.4 to 0.5, thestdPack's Write, StrReplace, and Delete do their work, and the shell of thebashPack changes files. All seven pinned lines move in one commit, because one build holds one version of the contract.This pull request has two commits. The first is the pin, with everything that must change with it. The second adds one entry to
deny.tomlthat accepts an advisory onhumantime, which failscargo deny checkonmastertoday. It is a commit of its own so that the owner can drop it or replace it with a real fix. The sections below describe the first commit, except the section on the advisory.What the pins move to
Cargo.tomlcppalliance/runvil-harnessrunvil,harness-gateway-client,plugin-web,plugin-mcp1664eaf34afed45fe5a4198ee9c9617396377d83b88f5e16cppalliance/runvil-plugin-apirunvil-plugin-sdk, version 0.4.0 to 0.5.003a9fa3752a4905aca0a9c945a93dc4ee7ea815db85612e4, the tagapi-v0.5.0cppalliance/runvil-plugin-stdstd-plugin782ad23ea806bf63e3b0a23c85f3599a1b03fea578c77a2dcppalliance/runvil-plugin-bashkitbashkit-plugin0864f1fff348f7a67b9087bcbc55ea7cb61c8de3fc409c82Each target is the head of
masterin its repository. The Harness at4afed45fpinsrunvil-plugin-sysat52a4905a, andstdandbashkitpin the SDK at the same commit.Cargo.lockchanges the 19sourcelines of the crates of these four repositories, and theversionofrunvil-plugin-sdkand ofrunvil-plugin-sysfrom 0.4.0 to 0.5.0. It adds no crate, removes none, and changes no other version.What changed in Workshop, and why
parseincrates/workshop/agents/src/conversation-run.rsreturnsArc<Prompt>, becauseHost::parsenow returns the prompt behind anArcandHarness::newtakes it that way.Vfseffect.AGENTS.md. The definition of the Engine says "a file operation" where it said "a store operation", as the Harness's ownAGENTS.mddoes, because a tool's file request is aVfseffect too.server/src/agents.rsnow says that a run's filesystem has the default policy,AllowAll. The doc of the contract route says what it does with aprompts:list. The comment inCargo.tomlsays version 0.5 of the contract.stdtest expected the fixed text of the Write stub. It now expectsWrote 1 byte.,Made 1 replacement., andDeleted the file., and it reads each change back from the Host's side. Thebashtest expected exit code 1 andthe shell cannot change files in this build. It now expects exit code 0, a file under a new directory, and an edit bysed -i.std_real_directory.rsbuilt aToolContextover anAccess, which aToolContextno longer holds. It now lends the call a smallToolFilesthat answers each request withAccess::perform.chat_gate/files.rsrun the built-in chat against a scripted model. The model'sWrite,StrReplace, andDeletecalls change a file of the run, itsShellcall makes a file thatReadreads, and the run log holds the write as aVfseffect that names theWritecall, with its answer.std_real_directory.rswrites, edits, and deletes a file of a real directory through the Plugin's library.language_guide/tests.rsshow that Write, StrReplace, Delete, and the shell change nothing under a read-only mount, and that the shell still writes beside it.prompts:and runs it withexecute, through Workshop's own finder.prompts:list to/prompts/contractand gets its contract.Write,StrReplace, andDeletein the tool list of each chat round.spawn_over_gatewayandclosed_run_loginchat_gate.rshold code thatcompaction.rsandlifecycle.rshad, so the new gates use it too.What an operator will notice
Write,StrReplace,Delete, andShell. Workshop sets no policy on a run's filesystem, so the default,AllowAll, admits every change, and nothing asks the operator to approve one. This change adds no policy and no approval step.C:drive mounts are behind switches that are committed as off. With a switch on, a tool's change under the mount is refused, and two new tests hold that.runs.db, the effect and its answer, and the effect of a write holds the bytes that were written.cargo workshopagain after this change. The Harness change states that a Plugin library built for contract 0.4 does not load, and a plain Cargo build does not refresh the copies inplugins. That refusal was not tested here.prompts:, by the name the agent list shows, and run them withexecute.stdPlugin.What was tested here
All of this ran on Windows with the pinned toolchain 1.99.1, on the tree of the first commit. The second commit changes
deny.tomlalone, andcargo deny check,cargo audit,cargo fmt --all --check, theclippyjob's command, and the wording scan ran again on the final tree.cargo fmt --all --check: pass.clippyjob's command, withCARGO_BUILD_WARNINGS=deny: pass.testjob'scargo nextest run: 703 tests passed, 1 skipped. The tree stayed clean.docsjob'scargo doc, withRUSTDOCFLAGS=-D warnings: pass.check-workshopjob, with its stub Gateway staged before and removed after: clippy passed,cargo nextest runpassed 508 tests and skipped 4, and thecargo docofworkshop-serverwith private items passed. The tree stayed clean.uijob, on Node 24.19 where CI uses Node 22:npm ci, the typecheck, and the build passed. The tests passed: 194 of 195 inuiwith 1 skipped, 12 inlook, and 13 inplatform.cargo deny check, with the entry of the second commit: pass. Advisories, bans, licenses, and sources are all ok.cargo deny --log-level error check, the form the pre-push hook runs, passes too.cargo audit: exit code 0, with 14 allowed warnings, the same 14 as on623eb7aa.ringis not in the Workshop binary's closure on Windows.cargo dylint --all -- --locked --workspace --all-targets --all-features: pass, on Windows.cargo test -p build-xtask: 86 tests passed.node --test crates/workshop/ui/test/docs-claims.mjs: 2 tests passed.The advisory on
humantimehumantimeas unmaintained. It reports no vulnerability, and it says that no safe upgrade is available.deny.tomldenies an unmaintained crate that the workspace depends on directly, andhumantime2.4.0 is a direct dependency ofworkshop-workspace. Socargo deny checkfails onmasterat623eb7aa, and it stops thesupply-chainjob and the pre-push hook. The pin does not touchhumantime.humantimeis used. In one place:now_rfc3339incrates/workshop/workspace/src/workspace_file/actor.rscallshumantime::format_rfc3339_secondsto stamp a time.ignorelist indeny.toml, with this reason: "humantime is unmaintained with no vulnerability reported and no safe upgrade; remove this entry when the crate is replaced". A comment above the entry names the one use.humantimestays inCargo.lock. Whether to accept the entry or to replace it with a real fix is the owner's decision.cargo audithas no ignore list in this repository, so it still reports the advisory as a warning and exits with 0, as it did before.What was not run
check-workshop-linux, which builds the desktop app on Linux, and the Linux runs offmt,clippy,test,docs,ui,supply-chain, anddylint. No job runs on macOS.What no test covers
runs.db.Known gaps in the
stdPluginLinks
stdPlugin change: Move to contract 0.5 and implement Write, StrReplace, and Delete runvil-plugin-std#2Summary by CodeRabbit