Skip to content

feat(jev-decisions): add calibrated judgment tools plugin via jev-mcp - #45

Closed
kittimzhe wants to merge 5 commits into
zai-org:mainfrom
kittimzhe:feat/jev-decisions
Closed

kittimzhe wants to merge 5 commits into
zai-org:mainfrom
kittimzhe:feat/jev-decisions

Conversation

@kittimzhe

Copy link
Copy Markdown
Contributor

User problem: judgment-class questions — is this claim supported by the evidence? is this fetched text safe to bring into context? is this diff actually done? — are exactly where agents answer from their own reading and get it wrong (the off-by-one that looked fine, the injection that read like instructions, the "tests pass" that did not). ZCode has no bundled way to route these decisions through a calibrated judgment layer.

Visible behavior: new plugin jev-decisions:

  • .mcp.json declares the community @jkudish/jev-mcp@0.9.0 server (eleven judgment tools: verify, screen, noul, find, rerank, classify, decide, compare, extract, review, gate). The server ships disabled; enabling requires a TypeSafe API key entered in the plugin's typesafeApiKey user config.
  • skills/jev-judgment/SKILL.md teaches the agent when to call each tool vs ripgrep/plain reading, and the call-even-when-obvious rule for verify/screen/review/gate. Adapted from the canonical skill shipped in @jkudish/jev-mcp (MIT), with attribution.
  • Bilingual README.md / README_CN.md; registered in marketplace.json.

Testing: python3 scripts/validate.py (27 plugins OK), python3 scripts/build_dist.py (jev-decisions 0.1.0 plugin.zip builds), git diff --check — all pass from the repo root. Real ZCode runtime verification (install → set key → enable server → screen a sample) is outstanding in this environment and recorded explicitly.

Version and registration: new plugin at 0.1.0; manifest and marketplace entry agree on name, version, and localized descriptions; category developer-tools.

Dependencies, network, side effects: enabling the server runs npx -y @jkudish/jev-mcp@0.9.0 (npm download on first run, version pinned); each tool call sends input text to the TypeSafe Jev API (billed per input token; usage reported per result); the plugin writes no files, registers no hooks, and runs no other commands. All documented in both READMEs.

Licensing/provenance: plugin MIT; @jkudish/jev-mcp MIT; skill adapted from upstream skills/jev with link; TypeSafe Jev is a commercial API. No bundled binaries or third-party assets.

@panda920

Copy link
Copy Markdown
Collaborator

✅ Internal validation passed. A ZCode maintainer will review the change next; once merged it is published to the marketplace and this PR is closed automatically.

@panda920

Copy link
Copy Markdown
Collaborator

✅ Internal validation passed. A ZCode maintainer will review the change next; once merged it is published to the marketplace and this PR is closed automatically.

@panda920

Copy link
Copy Markdown
Collaborator

Thanks for the contribution! This PR no longer applies cleanly on top of the latest main (conflicts in: marketplace.json). Please rebase onto main and push again; it will be picked up automatically.

🤖 Copy this prompt into Claude Code / Codex to fix it
You are updating my pull request #45 to github.com/zai-org/zcode-plugins (the ZCode plugin marketplace).
The maintainers' automated intake reported: the PR conflicts with the latest main

Conflicting files: marketplace.json

Fix it in this repository:
- New external plugins belong under external_plugins/<my-plugin>/; keep existing plugins in their current directory;
  never modify .github/, scripts/, tests/, .gitignore, .gitattributes or AGENTS.md.
- Follow CONTRIBUTING.md and AGENTS.md: the manifest lives at <plugin-directory>/.zcode-plugin/plugin.json,
  its name/version must match marketplace.json, names are kebab-case, keep the required i18n fields.
- Fetch the latest main of zai-org/zcode-plugins and rebase onto it; if marketplace.json conflicts,
  keep every existing entry and re-add mine.
- Run `python scripts/validate.py` until it passes, then force-push my PR branch.

@panda920

Copy link
Copy Markdown
Collaborator

✅ Internal validation passed. A ZCode maintainer will review the change next; after acceptance, release checks run separately and this PR is closed automatically.

@jacksonliuyn

Copy link
Copy Markdown
Collaborator

Could you move the new jev-decisions plugin directory from plugins/jev-decisions/ to external_plugins/jev-decisions/ and update its marketplace.json path accordingly? New external contributions belong under external_plugins/; the existing plugin directories should remain unchanged. Thanks!

kittimzhe and others added 3 commits October 23, 2026 12:51
Bundles the community jev-mcp server (disabled by default, TypeSafe API
key via userConfig) and a jev-judgment skill adapted from the canonical
skill shipped in @jkudish/jev-mcp, so ZCode agents get typed judgments:
verify claims against evidence, screen untrusted text, classify, decide,
review patches, and gate completion.
0.10.0 ships the jev_classify fallback-id collision and null-prototype
tally fixes (zai-org#36), plus the jev_verify none-id fix (zai-org#38). Pinning 0.9.0
would ship the classify bugs this plugin's skill teaches agents to rely
on.
New external contributions belong under external_plugins/; update the
marketplace.json source path to match.
@kittimzhe

Copy link
Copy Markdown
Contributor Author

Done — moved the plugin directory to external_plugins/jev-decisions/ and updated the marketplace.json source path to ./external_plugins/jev-decisions. Also rebased onto the latest main. Existing plugin directories untouched. Thanks for the review!

@panda920

Copy link
Copy Markdown
Collaborator

✅ Internal validation passed. A ZCode maintainer will review the change next; after acceptance, release checks run separately and this PR is closed automatically.

@jacksonliuyn

Copy link
Copy Markdown
Collaborator

Could you enable the MCP server by default? ZCode does not currently provide a separate manual toggle for MCP servers bundled with plugins; they are enabled along with the plugin. Please remove "enabled": false from .mcp.json and update the setup instructions accordingly.

@kittimzhe

Copy link
Copy Markdown
Contributor Author

Done — enabled: false is removed and every mention of the manual-enable step is aligned with the bundled-always-on behavior: the setup steps and dependency notes in both READMEs, the typesafeApiKey userConfig description, and the skill's requirements line. build_dist.py regenerated the zip (sha 994ff586…) and validate.py passes 27/27.

ZCode bundles no separate toggle for plugin-carried MCP servers — they
enable along with the plugin — so the shipped "enabled": false just
left the judgment tools dead until a toggle that does not exist was
found. Drop it, and align every mention of the old manual-enable step:
the setup steps, both READMEs' dependency notes, the typesafeApiKey
userConfig description, and the skill's requirements line.
@panda920

Copy link
Copy Markdown
Collaborator

✅ Internal validation passed. A ZCode maintainer will review the change next; after acceptance, release checks run separately and this PR is closed automatically.

Three spots still described the old opt-in behavior after the
enable-with-plugin change: the What-it-does bullets in both READMEs and
the SKILL.md Setup paragraph. Align them with the bundled-always-on
behavior.
@panda920

Copy link
Copy Markdown
Collaborator

✅ Internal validation passed. A ZCode maintainer will review the change next; after acceptance, release checks run separately and this PR is closed automatically.

@panda920

Copy link
Copy Markdown
Collaborator

Thanks for your contribution! The change has been accepted by the maintainers. Publication to the marketplace is tracked separately; this contribution is now closed.

@panda920 panda920 closed this Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants