Repository navigation
feat: evals for the packaged install's two ways in; /upgrade stamps the release's commit - #30
Merged
Merged
Conversation
…he release's commit Two session evals for paths that had never run: the adopt suite's `packaged` case (/aplyca-adf:adopt on a new project, choosing the packaged install) and a new upgrade suite whose `switch-to-packaged` case has /aplyca-adf:upgrade move a committed adoption at v1.0.0 — built from that tag — to the newest release and switch it to packaged. Both check the end state with evals/dynamic/check-packaged.sh; the runner passes this checkout's path to a suite's project.sh. The upgrade run stamped `v1.0.6 · 130753a`, an annotated tag's own ID from `git rev-parse v1.0.6`, not the release's commit ab56cb6. /upgrade and /adopt now name `git rev-parse --short '<tag>^{commit}'`, and the checker checks the stamp's commit (it marks that run ✘). The packaged smoke test in docs/SETUP.md sets CLAUDE_PLUGIN_ROOT, which the plugin's hooks need since v1.0.2. Sonnet: packaged 8/8 ($0.83); switch-to-packaged 10/10 ($0.81), and after the fix 10/10 with the commit stamped ($1.20). Report in evals/dynamic/reports/2026-10-04-packaged-paths.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mauricios
marked this pull request as ready for review
October 5, 2026 03:35
mauricios
added a commit
that referenced
this pull request
Oct 5, 2026
…al sessions (#32) A minor release: the packaged install becomes the default for new projects (decision 0018), with evals for its paths. Unreleased becomes v1.1.0, covering #29, #30, #31, and one more fix: /aplyca-adf:adopt took the skeleton from wherever the framework source was while pinning the newest release tag, so a project's committed files could be newer than its pinned plugin. It now finds the newest release tag, takes the framework at that tag (a shallow clone of it, or a worktree of a local checkout), and stamps that tag's commit; SETUP.md's manual copy says the same. The packaged adopt eval reran on Sonnet: 8/8, stamped ab56cb6 from a scratch worktree at v1.0.6. plugin.json 1.1.0; the README names the release. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed and why
Two paths into the packaged install had never run in a real session. This PR adds session evals for both, and fixes the defect they found.
The evals:
adoptsuite, newpackagedcase./aplyca-adf:adopton a new project, where the follow-up chooses the packaged install.New
upgradesuite,switch-to-packagedcase.project.shbuilds a committed adoption at v1.0.0 from that tag: the skeleton without the Antigravity and Cursor layers, the plugin pinned tov1.0.0, PDR-0001, and the stamp./aplyca-adf:upgradethen moves it to the newest release, and the follow-up accepts the switch.evals/dynamic/check-packaged.shchecks the packaged layout for both cases, marking each check ✓ or ✘:install: packaged;hooksblock, pin the newest release, and turn onaplyca-adf;CLAUDE.md, andDEV-SETUP.mduses the full command names;Each suite's
inspect.shadds its own checks: for the switch, the PDR, PDR-0001 marked amended, andmainuntouched.Runner: passes this checkout's path to a suite's
project.sh; theupgradesuite runs likeadopt.What the runs found, and the fixes:
/upgradestamped an annotated tag's ID instead of the release's commit. The stamp readv1.0.6 · 130753a, fromgit rev-parse v1.0.6, and the branch and commit message used the same ID. The real commit isab56cb6./upgradeand/adoptnow namegit rev-parse --short '<tag>^{commit}'.check-packaged.shnow checks the stamp's commit, and it marks the first run ✘. The rerun stampedab56cb6.docs/SETUP.mdlackedCLAUDE_PLUGIN_ROOT, which the plugin's hooks need since v1.0.2. Both sessions noticed and set it themselves; the doc now does./adoptcopies the skeleton from the framework source as it is, which can be past the release it pins. In this run the two were identical. Adopting from the release tag would close it, and that's a separate change.Upgrade impact
The evals are framework-internal. The fixes change
/upgrade's and/adopt's instructions and one line ofdocs/SETUP.md. They reach projects with the next release; the plugin's version moves then.How to verify
Verified / not verified
packaged: 8 of 8, $0.83.switch-to-packaged: 10 of 10, $0.81, though its stamp named the tag's ID, which the checker didn't yet check.switch-to-packagedafter the fix: 10 of 10, stampedab56cb6, $1.20..expected.md(report:evals/dynamic/reports/2026-10-04-packaged-paths.md).Merge danger
🤖 Generated with Claude Code