Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
bump: none
---

The playgrounds eval cases list an eval-only `playground` stand-in, so the plugin loads in `claude plugin eval` and its suite measures something.
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
schema_version: "1.1"
name: use-absent-install-scope
plugins: ["../..", "playground-stub"]
graders:
- name: install-command
type: regex
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"name": "playground",
Comment thread
kyle-sexton marked this conversation as resolved.
Resolved
Comment thread
kyle-sexton marked this conversation as resolved.
Resolved
"description": "Eval-only stand-in for playground@claude-plugins-official with no skill, so the playgrounds plugin loads while its upstream skill stays absent.",
"version": "0.0.0"
Comment thread
kyle-sexton marked this conversation as resolved.
Resolved
Comment thread
kyle-sexton marked this conversation as resolved.
Resolved
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
schema_version: "1.1"
name: use-boundary-visual-form
plugins: ["../..", "playground-stub"]
graders:
- name: no-playground-skill
type: tool_used
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"name": "playground",
"description": "Eval-only stand-in for playground@claude-plugins-official with no skill, so the playgrounds plugin loads while its upstream skill stays absent.",
"version": "0.0.0"
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
schema_version: "1.1"
name: use-install-policy-blocked
plugins: ["../..", "playground-stub"]
graders:
- name: names-policy-block
type: regex
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"name": "playground",
"description": "Eval-only stand-in for playground@claude-plugins-official with no skill, so the playgrounds plugin loads while its upstream skill stays absent.",
"version": "0.0.0"
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
schema_version: "1.1"
name: use-reads-consumer-notes
plugins: ["../..", "playground-stub"]
graders:
- name: reads-consumer-notes
type: tool_used
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"name": "playground",
"description": "Eval-only stand-in for playground@claude-plugins-official with no skill, so the playgrounds plugin loads while its upstream skill stays absent.",
"version": "0.0.0"
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
schema_version: "1.1"
name: use-routes-natural-explorer
plugins: ["../..", "playground-stub"]
graders:
- name: routes-use
type: tool_used
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"name": "playground",
"description": "Eval-only stand-in for playground@claude-plugins-official with no skill, so the playgrounds plugin loads while its upstream skill stays absent.",
"version": "0.0.0"
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
description: "Routing from plain words (SKILL.md line 2: 'interactive explorer for X', 'tune a parameter space visually') in an eval run, where the upstream playground plugin is never installed (the run copy drops the manifest dependencies key). The skill must reach Step 3 (lines 65-75) and print the marketplace-qualified install command, and must not build the explorer itself (lines 18-19 and 147-148: this skill never builds a playground). Write is granted so the no-write grader can fail: a baseline is expected to hand-build the HTML page. The marketplace-add line is not graded, since lines 72-74 and 78-79 disagree on when it belongs. Needs --allow-tools Write."
description: "Routing from plain words (SKILL.md line 2: 'interactive explorer for X', 'tune a parameter space visually') in an eval run, where the upstream playground plugin is never installed (a stand-in with no skill meets the manifest dependency). This is an absent-path case: allowed_tools has no Bash, so Step 1 cannot run `claude plugin list` and falls back to the skill listing (lines 28-32), where the stand-in lists no `playground` skill. The skill must reach Step 3 (lines 65-75) and print the marketplace-qualified install command, and must not build the explorer itself (lines 18-19 and 147-148: this skill never builds a playground). Write is granted so the no-write grader can fail: a baseline is expected to hand-build the HTML page. The marketplace-add line is not graded, since lines 72-74 and 78-79 disagree on when it belongs. Needs --allow-tools Write."
tags: [playgrounds, use, outcome, routing]
max_turns: 15
timeout_seconds: 600
Expand Down
Loading