Skip to content

Pin the Harness at 4afed45f and contract 0.5 - #24

Merged
vinniefalco merged 2 commits into
cppalliance:masterfrom
vinniefalco:master
Nov 18, 2026
Merged

vinniefalco merged 2 commits into
cppalliance:masterfrom
vinniefalco:master

Conversation

@vinniefalco

@vinniefalco vinniefalco commented Nov 18, 2026 •

Copy link
Copy Markdown
Member

Workshop moves its four pins to the commits where a tool's file operations are Vfs effects that the Harness performs and records. The Plugin contract goes from 0.4 to 0.5, the std Pack's Write, StrReplace, and Delete do their work, and the shell of the bash Pack changes files. All seven pinned lines move in one commit, because one build holds one version of the contract.

This pull request has two commits. The first is the pin, with everything that must change with it. The second adds one entry to deny.toml that accepts an advisory on humantime, which fails cargo deny check on master today. It is a commit of its own so that the owner can drop it or replace it with a real fix. The sections below describe the first commit, except the section on the advisory.

What the pins move to

Repository Lines in Cargo.toml From To Commits
cppalliance/runvil-harness runvil, harness-gateway-client, plugin-web, plugin-mcp 1664eaf3 4afed45fe5a4198ee9c9617396377d83b88f5e16 38
cppalliance/runvil-plugin-api runvil-plugin-sdk, version 0.4.0 to 0.5.0 03a9fa37 52a4905aca0a9c945a93dc4ee7ea815db85612e4, the tag api-v0.5.0 4
cppalliance/runvil-plugin-std std-plugin 782ad23e a806bf63e3b0a23c85f3599a1b03fea578c77a2d 15
cppalliance/runvil-plugin-bashkit bashkit-plugin 0864f1ff f348f7a67b9087bcbc55ea7cb61c8de3fc409c82 5

Each target is the head of master in its repository. The Harness at 4afed45f pins runvil-plugin-sys at 52a4905a, and std and bashkit pin the SDK at the same commit.

Cargo.lock changes the 19 source lines of the crates of these four repositories, and the version of runvil-plugin-sdk and of runvil-plugin-sys from 0.4.0 to 0.5.0. It adds no crate, removes none, and changes no other version.

What changed in Workshop, and why

  • One line of production code. parse in crates/workshop/agents/src/conversation-run.rs returns Arc<Prompt>, because Host::parse now returns the prompt behind an Arc and Harness::new takes it that way.
  • No change to the UI, the protocol, the run log, the built-in chat prompt, the README, or the build tooling. The UI shows a run's events and never an effect, and the run log stores the payload of each record as the Harness gives it, so neither names a Vfs effect.
  • AGENTS.md. The definition of the Engine says "a file operation" where it said "a store operation", as the Harness's own AGENTS.md does, because a tool's file request is a Vfs effect too.
  • Two doc comments. The module doc of server/src/agents.rs now says that a run's filesystem has the default policy, AllowAll. The doc of the contract route says what it does with a prompts: list. The comment in Cargo.toml says version 0.5 of the contract.
  • Two tests changed because the tools changed. The std test expected the fixed text of the Write stub. It now expects Wrote 1 byte., Made 1 replacement., and Deleted the file., and it reads each change back from the Host's side. The bash test expected exit code 1 and the shell cannot change files in this build. It now expects exit code 0, a file under a new directory, and an edit by sed -i.
  • One test moved to the new API. std_real_directory.rs built a ToolContext over an Access, which a ToolContext no longer holds. It now lends the call a small ToolFiles that answers each request with Access::perform.
  • Seven new tests.
    • GATE 15 and GATE 16 in the new chat_gate/files.rs run the built-in chat against a scripted model. The model's Write, StrReplace, and Delete calls change a file of the run, its Shell call makes a file that Read reads, and the run log holds the write as a Vfs effect that names the Write call, with its answer.
    • A second test in std_real_directory.rs writes, edits, and deletes a file of a real directory through the Plugin's library.
    • Two tests in language_guide/tests.rs show that Write, StrReplace, Delete, and the shell change nothing under a read-only mount, and that the shell still writes beside it.
    • One test runs an agent that names another agent under prompts: and runs it with execute, through Workshop's own finder.
    • One test posts a prompt with a prompts: list to /prompts/contract and gets its contract.
  • GATE 13 also expects Write, StrReplace, and Delete in the tool list of each chat round.
  • Shared test support. spawn_over_gateway and closed_run_log in chat_gate.rs hold code that compaction.rs and lifecycle.rs had, so the new gates use it too.

What an operator will notice

  • A model and the shell can now change the files of a run. The built-in chat offers Write, StrReplace, Delete, and Shell. Workshop sets no policy on a run's filesystem, so the default, AllowAll, admits every change, and nothing asks the operator to approve one. This change adds no policy and no approval step.
  • Those files are in memory. Workshop gives each run an empty in-memory filesystem. A file that the model or the shell writes exists until the run ends, which for a chat is the end of the conversation. No file on disk changes, and the folders of the workspace are not in a run's filesystem. A real folder for a run is still planned work.
  • The two demo mounts stay read-only and off. The language guide and the C: drive mounts are behind switches that are committed as off. With a switch on, a tool's change under the mount is refused, and two new tests hold that.
  • The run log holds more. Each file operation of a tool adds two records to runs.db, the effect and its answer, and the effect of a write holds the bytes that were written.
  • Run cargo workshop again after this change. The Harness change states that a Plugin library built for contract 0.4 does not load, and a plain Cargo build does not refresh the copies in plugins. That refusal was not tested here.
  • Every run holds every installed Pack. A Pack of any vendor that a library in the Plugin directory inserts is in every run, whether or not the prompt declares it.
  • An agent can run another agent. An agent in the agents folder can name other agents under prompts:, by the name the agent list shows, and run them with execute.
  • Glob is still a stub in the pinned std Plugin.

What was tested here

All of this ran on Windows with the pinned toolchain 1.99.1, on the tree of the first commit. The second commit changes deny.toml alone, and cargo deny check, cargo audit, cargo fmt --all --check, the clippy job's command, and the wording scan ran again on the final tree.

  • cargo fmt --all --check: pass.
  • The clippy job's command, with CARGO_BUILD_WARNINGS=deny: pass.
  • The test job's cargo nextest run: 703 tests passed, 1 skipped. The tree stayed clean.
  • The docs job's cargo doc, with RUSTDOCFLAGS=-D warnings: pass.
  • The check-workshop job, with its stub Gateway staged before and removed after: clippy passed, cargo nextest run passed 508 tests and skipped 4, and the cargo doc of workshop-server with private items passed. The tree stayed clean.
  • The ui job, on Node 24.19 where CI uses Node 22: npm ci, the typecheck, and the build passed. The tests passed: 194 of 195 in ui with 1 skipped, 12 in look, and 13 in platform.
  • cargo deny check, with the entry of the second commit: pass. Advisories, bans, licenses, and sources are all ok. cargo deny --log-level error check, the form the pre-push hook runs, passes too.
  • cargo audit: exit code 0, with 14 allowed warnings, the same 14 as on 623eb7aa.
  • The check for one rustls backend: ring is not in the Workshop binary's closure on Windows.
  • cargo dylint --all -- --locked --workspace --all-targets --all-features: pass, on Windows.
  • cargo test -p build-xtask: 86 tests passed.
  • node --test crates/workshop/ui/test/docs-claims.mjs: 2 tests passed.

The advisory on humantime

  • What it is. The advisory RUSTSEC-2026-0542 marks humantime as unmaintained. It reports no vulnerability, and it says that no safe upgrade is available.
  • It fails without this pull request. deny.toml denies an unmaintained crate that the workspace depends on directly, and humantime 2.4.0 is a direct dependency of workshop-workspace. So cargo deny check fails on master at 623eb7aa, and it stops the supply-chain job and the pre-push hook. The pin does not touch humantime.
  • Where humantime is used. In one place: now_rfc3339 in crates/workshop/workspace/src/workspace_file/actor.rs calls humantime::format_rfc3339_seconds to stamp a time.
  • What the entry says. The second commit adds the advisory to the ignore list in deny.toml, with this reason: "humantime is unmaintained with no vulnerability reported and no safe upgrade; remove this entry when the crate is replaced". A comment above the entry names the one use.
  • It is a stopgap. The entry changes no code, and humantime stays in Cargo.lock. Whether to accept the entry or to replace it with a real fix is the owner's decision. cargo audit has no ignore list in this repository, so it still reports the advisory as a warning and exits with 0, as it did before.

What was not run

  • check-workshop-linux, which builds the desktop app on Linux, and the Linux runs of fmt, clippy, test, docs, ui, supply-chain, and dylint. No job runs on macOS.

What no test covers

  • A write of more than a few bytes in runs.db.
  • A Pack of another vendor in a run.
  • A file tool call of a model that fails. It still reaches the operator as "Tool call failed".
  • A run with one of the two demo switches on.

Known gaps in the std Plugin

Links

Summary by CodeRabbit

  • New Features
    • Chats now offer file tools to read, list, search, create, edit, and delete files. Shell commands can also work with files in a run’s temporary workspace.
    • Runs can optionally access mapped directories in read-only mode; changes made in the temporary workspace do not persist after the run.
  • Bug Fixes
    • Prompts with references to unavailable prompts can still be parsed and returned.

The Harness at `4afed45fe5a4198ee9c9617396377d83b88f5e16` performs each file operation of a tool as a `Vfs` effect and records it, and contract 0.5 lets a library Plugin change files. The seven pinned lines of `Cargo.toml` move to that commit, to `52a4905aca0a9c945a93dc4ee7ea815db85612e4` of `runvil-plugin-api`, to `a806bf63e3b0a23c85f3599a1b03fea578c77a2d` of `runvil-plugin-std`, which implements Write, StrReplace, and Delete, and to `f348f7a67b9087bcbc55ea7cb61c8de3fc409c82` of `runvil-plugin-bashkit`, whose shell changes files. Workshop changes one line of production code for the new API, and its tests pin what the tools do now.

- `parse` in `conversation-run.rs` returns `Arc<Prompt>`, the type that `host.parse` now gives and the Harness takes. No other production code changes: the UI shows events and no effect, and the run log stores the payload of each record as the Harness gives it.
- `std_real_directory.rs` lends its `ToolContext` the files `AnsweredAtOnce`, which answers each request with `perform` on the `Access` of the test, because a `ToolContext` holds no access now. Its new test makes, edits, and deletes a file of the real directory.
- `chat_gate/files.rs` is new. In GATE 15 the `Write`, `StrReplace`, and `Delete` calls of the model change a file of the run of the chat, and the run log holds the write as a `Vfs` effect that names the `Write` call, with its answer. In GATE 16 the `Shell` call of the model makes a file that `Read` reads. GATE 13 also expects `Write`, `StrReplace`, and `Delete` in the tool list of each round.
- `spawn_over_gateway` and `closed_run_log` in `chat_gate.rs` hold code that `compaction.rs` and `lifecycle.rs` had, so the new gates share it.
- Two tests in `agents/tests.rs` change because the tools changed. The std test expected the text of the Write stub. It now expects `Wrote 1 byte.`, `Made 1 replacement.`, and `Deleted the file.`, and it reads each change back. The bashkit test expected exit code 1 and `the shell cannot change files in this build`. It now expects exit code 0, a file under a new directory, and an edit by `sed -i`.
- The definition of the Engine in `AGENTS.md` says "a file operation" where it said "a store operation", because a file request of a tool is a `Vfs` effect too. The comment in `Cargo.toml` says version 0.5.
- A model, the `tools.call` of a prompt, and the shell can now create, change, and delete files of a run, and no approval is asked. The filesystem of a run has the default policy, `AllowAll`, which the module doc of `agents.rs` now states. That filesystem is an in-memory root that ends with the run, so no file on disk changes. The two mounts of real directories stay read-only with both switches off, and two new tests in `language_guide/tests.rs` show that the three tools and the shell change nothing under such a mount.
- An agent can name agents of the agents folder under `prompts:`, by the name that discovery lists, and run them with `execute`. `an_agent_executes_an_agent_of_the_folder_that_its_prompts_list_names` pins it. The contract route parses a prompt that has a `prompts:` list from the posted text alone, which the doc of `contract` and one new test state.
- `Cargo.lock` changes the 19 `source` lines of the crates of the four repositories, and the `version` of `runvil-plugin-sdk` and of `runvil-plugin-sys` from 0.4.0 to 0.5.0. It adds no crate and removes none.
- The pins move across 38 commits of `runvil-harness`, 4 of `runvil-plugin-api`, 15 of `runvil-plugin-std`, and 5 of `runvil-plugin-bashkit`.
- The built-in chat prompt, the README, the UI, the protocol, the run log, and the build tooling do not change. Glob is still a stub in the pinned std Plugin.
- No test covers a write of more than a few bytes in `runs.db`, a Pack of another vendor, which every run now holds, or a run with one of the two switches on. No gate covers a file tool call of the model that fails. Such a call still reaches the operator as "Tool call failed".

Plan: runvil-harness/plan/2026-11-16-1-tool-file-effects.plan.md
The advisory `RUSTSEC-2026-0542` marks `humantime` as unmaintained, and cargo-deny fails on it, which stops the pre-push hook and the supply-chain job. The `ignore` list of `deny.toml` now holds the advisory with a `reason`, so the check passes. The advisory reports no vulnerability and no safe upgrade.

- The entry is a stopgap. Its comment says that `workshop-workspace` uses one function of the crate, `format_rfc3339_seconds`, to stamp a time, and that the entry goes when the crate is replaced.
- The diff changes no code and no manifest. `humantime` stays a dependency and stays in the lock file, and nothing replaces it yet.
- The diff adds no ignore list for cargo-audit. This repository has none, and cargo-audit reports the advisory as a warning and exits with 0.
Copilot AI balanced review requested due to automatic review settings November 18, 2026 19:26

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@coderabbitai

coderabbitai Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The changes update dependency pins, expand file-operation coverage across agent and chat-gate tests, add prompt parsing and named-agent execution tests, and add a RustSec advisory ignore.

Changes: Run file operations

Layer / File(s) Summary
Filesystem setup and mapped-directory behavior
Cargo.toml, AGENTS.md, crates/workshop/server/src/agents.rs, crates/workshop/server/src/agents/language_guide/tests.rs
Dependency pins and filesystem documentation are updated. Tests check that write and shell operations under read-only mappings fail, while a write in the in-memory root succeeds.
Agent file tools and real-directory operations
crates/workshop/server/src/agents/tests.rs, crates/workshop/server/tests/it/std_real_directory.rs
Agent tests cover std file operations and bash writes. A real-directory test covers write, replace, read, and delete operations through ToolFiles.
Chat-gate file-tool integration
crates/workshop/server/tests/it/chat_gate.rs, crates/workshop/server/tests/it/chat_gate/*
Chat-gate tests check file-tool results, VFS effects, and shell-created files. Shared server setup and run-log polling support these tests.

Prompt parsing and execution

Layer / File(s) Summary
Listed prompts and named-agent execution
crates/workshop/agents/src/conversation-run.rs, crates/workshop/server/src/agents/tests.rs, crates/workshop/server/src/routes/prompts.rs, crates/workshop/server/src/routes/prompts-tests.rs
Prompt parsing returns a shared Prompt value. Tests cover parsing when listed prompts are unavailable and executing a named agent in the same run.

Advisory policy

Layer / File(s) Summary
RustSec advisory ignore
deny.toml
The ignore list adds RUSTSEC-2026-0542 with a rationale and removal condition.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ChatGateTest
  participant ChatServer
  participant ScriptedCompletionServer
  participant VFS
  ChatGateTest->>ChatServer: Start a chat turn
  ChatServer->>ScriptedCompletionServer: Send completion request
  ScriptedCompletionServer-->>ChatServer: Return a file-tool call
  ChatServer->>VFS: Perform the requested file operation
  ChatServer-->>ScriptedCompletionServer: Send the tool result
  ChatGateTest->>ChatServer: Read the closed run log
Loading
Merge Risk: Minimal · up to 956e7

This change updates dependency pins and expands tests for file operations and prompt execution. It also adds a documented ignore for a maintenance-only advisory. No concrete defects were identified, so it appears ready to merge after normal CI checks.

Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the two main changes: the Harness pin moves to commit 4afed45f, and contract 0.5 is adopted. The raw summary confirms the Cargo.toml pin change to harness revision 4afed45f and the plu…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the two main changes: the Harness pin moves to commit 4afed45f, and contract 0.5 is adopted. The raw summary confirms the Cargo.toml pin change to harness revision 4afed45f and the plugin SDK move from 0.4.0 to 0.5.0. The PR objectives describe the same work. The title leaves out the parse change, the new file-effect tests, and the deny.toml advisory exception, but it does not need to cover every detail.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Essentials
  • Run ID: eafccaa5-86bf-45ed-a40a-d3070b27ea1f
Commits

Reviewing files that changed from the base of the PR and between 623eb7a and 956e780.

Files ignored due to path filters: 1

  • Cargo.lock is excluded by !**/*.lock
Files selected for processing: 15

  • AGENTS.md
  • Cargo.toml
  • crates/workshop/agents/src/conversation-run.rs
  • crates/workshop/server/src/agents.rs
  • crates/workshop/server/src/agents/language_guide/tests.rs
  • crates/workshop/server/src/agents/tests.rs
  • crates/workshop/server/src/routes/prompts-tests.rs
  • crates/workshop/server/src/routes/prompts.rs
  • crates/workshop/server/tests/it/chat_gate.rs
  • crates/workshop/server/tests/it/chat_gate/canonical_sequence.rs
  • crates/workshop/server/tests/it/chat_gate/compaction.rs
  • crates/workshop/server/tests/it/chat_gate/files.rs
  • crates/workshop/server/tests/it/chat_gate/lifecycle.rs
  • crates/workshop/server/tests/it/std_real_directory.rs
  • deny.toml

Included review availability: This review used your included allowance. 4 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 5 reviews per hour.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The ABI-breaking Plugin contract upgrade depends on unavailable upstream implementations, and rejection of contract-0.4 libraries remains untested, requiring human validation of the dynamic-loading boundary.

No open issues 🎉

What changed in this PR

Updates Workshop to Harness 4afed45f and Plugin contract 0.5, enabling file tools and the shell to change run-local files through operations the Harness performs and records.

Changes:

  • Aligns dependency pins and adapts prompt parsing to return Arc<Prompt>.
  • Adds coverage for file changes, read-only mounts, recorded file operations, and nested agent execution; shares chat test helpers.
  • Documents filesystem behavior and temporarily accepts the humantime unmaintained advisory.
File Description
deny.toml Adds a documented exception for the humantime advisory.
crates/​workshop/​server/​tests/​it/​std_real_directory.rs Adapts to ToolFiles and tests real-directory file changes.
crates/​workshop/​server/​tests/​it/​chat_gate/​lifecycle.rs Reuses the shared closed-run-log helper.
crates/​workshop/​server/​tests/​it/​chat_gate/​files.rs Adds chat gates for file changes, shell interoperability, and recording.
crates/​workshop/​server/​tests/​it/​chat_gate/​compaction.rs Reuses shared server setup.
crates/​workshop/​server/​tests/​it/​chat_gate/​canonical_sequence.rs Checks that chat advertises file-changing tools.
crates/​workshop/​server/​tests/​it/​chat_gate.rs Extracts shared helpers and registers file gates.
crates/​workshop/​server/​src/​routes/​prompts.rs Documents contract parsing with referenced prompts.
crates/​workshop/​server/​src/​routes/​prompts-tests.rs Tests contract responses for prompts listing other prompts.
crates/​workshop/​server/​src/​agents/​tests.rs Tests nested execution and file-changing Plugin tools.
crates/​workshop/​server/​src/​agents/​language_guide/​tests.rs Tests read-only mount protection and adjacent writable storage.
crates/​workshop/​server/​src/​agents.rs Documents default filesystem policy and run-local file lifetime.
crates/​workshop/​agents/​src/​conversation-run.rs Updates the parse return type to Arc<Prompt>.
Cargo.toml Aligns Harness and Plugin dependency pins with contract 0.5.
Cargo.lock Locks the updated commits and contract versions.
AGENTS.md Updates the Engine definition to name file operations.

Review effort: Balanced

@vinniefalco

Copy link
Copy Markdown
Member Author

@claude review

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

@vinniefalco
vinniefalco merged commit 956e780 into cppalliance:master Nov 18, 2026
26 of 28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants