[jongio]

Summary

Consolidated repo-maintain pass across governance, workflow caching, vetted dependency updates (with >= 7-day release cooldown and MIT/permissive license verification), code/configuration checks, test coverage gates, and strict Vally 0.21.0 skill/eval lint checks.

Phase Breakdown

Phase	Commit	Summary
1. Governance & Runtime Parity	94d4c07	Add lockfile-keyed pnpm store caching (actions/cache@caa296126883cff596d87d8935842f9db880ef25) in .github/workflows/ci.yml while preserving node24 / .node-version (24.21.0) parity and actions/checkout@v7.
2. Dependencies & Companion Syncs	c17d061, 577d3a5	Bump @microsoft/vally-cli (0.19.0 -> 0.21.0) with mechanical companion syncs (tests/native.test.ts dynamic manifest version lookup, .github/workflows/ci.yml smoke pins, README.md), oxlint (1.87.0 -> 1.90.0), @types/node (24.19.0 -> 24.19.2, keeping Node 24 alignment), fast-uri (3.1.8 -> 3.1.9), and fast-xml-parser (5.11.2 -> 5.12.0), and rebuild dist/.
3. Hardening & Config Audit	—	Clean pass; zero secret alerts, zero CodeQL findings, and repository settings verified.
4. Idiomatic Refactoring	—	Clean pass; no Level 1–2 behavior-preserving cleanups needed beyond dynamic test fixture version derivation.
5. Coverage & Runtime Quality	—	96.82% line / 91.08% branch coverage (180/180 Vitest tests + 6/6 packaged tests passing; diff coverage 100%).
6. Docs, Skills & Evals	—	vally 0.21.0 strict skill + eval lint passing across examples/copilot-smoke and tests/fixtures; 0 broken local links and 100% action.yml input/output parity.

Repo Health Score: 89/100 (B) -> 97/100 (A+) (+8)

Dimension	Before	After	Max	Evidence
CI & Workflow Hygiene	17	20	20	Default branch green; SHA-pinned actions/setup-node & actions/cache; lockfile-keyed pnpm store cache added
Test & Branch Coverage	20	20	20	96.82% lines, 91.08% branches, 100% diff coverage, 0 untested critical entrypoints
Dependency & License Health	12	15	15	Companion sync resolved for @microsoft/vally-cli@0.21.0; >= 7-day cooldown and permissive SPDX checks passed
Hardening & Config Posture	15	15	15	0 secret/CodeQL findings; repository protection settings enabled
Documentation Accuracy	10	10	10	0 broken links; README.md inputs/outputs match action.yml
Skill & Eval Quality	10	10	10	vally lint --strict clean on examples/copilot-smoke (skill + eval) and tests/fixtures
Backlog SLA	5	7	10	Open items triaged; #1 remains open >24h without comments
Total	89 (B)	97 (A+)	100	+8 points

Open Dependabot PR Disposition (Left Open Until Merge)

• Superseded on merge:
  • Bump @microsoft/vally-cli from 0.19.0 to 0.20.0 #16 — @microsoft/vally-cli 0.19.0 -> 0.20.0 (superseded by 0.21.0, including mechanical companion version syncs)
  • Bump oxlint from 1.87.0 to 1.90.0 #22 — oxlint 1.87.0 -> 1.90.0 (exact package / / directory / version match)
  • Bump fast-uri from 3.1.8 to 3.1.9 in the npm_and_yarn group across 1 directory #19 — fast-uri 3.1.8 -> 3.1.9 (exact package / / directory / version match)
• Not superseded (requires maintainer review / closure):
  • Bump @types/node from 24.19.0 to 26.6.6 #23 — @types/node 24.19.0 -> 26.6.6 (semver-major Node 26 definitions skipped to preserve node24 Action runtime alignment; updated @types/node to 24.19.2 instead)

Verification

• pnpm install --frozen-lockfile --ignore-scripts (zero private registry URLs, tarball: keys, or sha1- hashes in pnpm-lock.yaml)
• pnpm check (tsc --noEmit, oxlint --deny-warnings src tests, vitest run --coverage, node tooling/build.mjs --check, node --test tooling/packaged.test.mjs)
• pnpm exec vally lint --strict examples/copilot-smoke --eval-spec=examples/copilot-smoke
• pnpm exec vally lint --strict --eval-spec=tests/fixtures

[github-actions]

Evals: passed

Suite: copilot-smoke

Lint: validated (no model execution) - Evidence

Run: passed - 1 trial - Evidence

Eval	Trials	Result
copilot-skill-smoke	1	passed

Quality & token summary

Eval	Verdict	Score	Duration	Tokens
copilot-skill-smoke	passed	1.00 (threshold 1.00)	2221ms	33055 tokens (32982 in / 73 out)

Exact revision | Workflow run

[Copilot]

🔵 Needs a closer look

The Vally 0.20.0 upgrade affects coupled adapter/CLI contracts for suite selection, grading thresholds, and JSONL evidence parsing, requiring human review of model-free native acceptance.

No open issues 🎉

1 file not reviewed

• pnpm-lock.yaml: Generated file

What changed in this PR

This maintenance pass updates dependencies and CI caching for the thin Vally GitHub Action adapter, while keeping its development tooling aligned with Node 24.

Changes:

• Add lockfile-keyed pnpm store caching to CI.
• Upgrade the Vally test fixture to 0.20.0, refresh tooling and transitive dependencies, and synchronize smoke-test pins and documentation.
• Derive the native test fixture version from its installed manifest and refresh bundled dependency notices.

File	Description
tests/​native.test.ts	Read the fixture’s Vally version from its installed package manifest.
README.md	Update the Vally pin example and development fixture documentation.
pnpm-lock.yaml	Lock updated direct and transitive dependencies.
package.json	Bump Vally CLI, Node 24 types, and oxlint.
dist/​THIRD-PARTY-NOTICES.txt	Refresh bundled dependency version notices.
dist/​bundled-dependencies.json	Update bundled dependency inventory versions.
.github/​workflows/​ci.yml	Add pnpm caching and update native smoke tests to Vally 0.20.0.

_{Review effort: Balanced (auto)}

[wbreza]

Approved at c17d061. No actionable findings or outstanding fixes were identified in github/evals-action#27.

The existing model-free tests cover suite selection, weighted scoring, native result parsing, and failed or skipped runs. CI for this head passed those tests, the packaged smoke checks, and the bundle rebuild comparison. I did not run PR scripts locally; generated bundle consistency was verified through that CI comparison.

Generated by 🌳Breza Bot

[wbreza]

Approval remains at 577d3a5 for github/evals-action#27. No substantiated code findings remain from this review.

I assessed the new cache-output concern in its existing thread. The proposed attack requires control over code this job already executes, so I could not establish an additional security risk. That thread remains open for maintainer review; I did not resolve or dismiss it.

CI for this head passed the model-free native contract tests, packaged smoke checks and bundle verification. I inspected the results without running PR scripts locally.

Generated by 🌳Breza Bot

[wbreza]

Approved at c6e32ac for github/evals-action#27. No outstanding actionable findings remain.

The cache-output formatting request is addressed in the code: the path is now written using a random delimiter and quoted printf arguments. CI for this head confirms the cache receives the expected pnpm directory and restores it successfully; all listed checks pass. I inspected the code and saved CI evidence without running PR scripts locally.

Generated by 🌳Breza Bot

[Copilot]

🔵 Needs a closer look

The pass combines a Vally CLI/runtime upgrade, transitive and generated bundle changes, and new hosted-runner caching, whose Linux compatibility and cross-workflow cache behavior require validation on actual GitHub runners.

No open issues 🎉

1 file not reviewed

• pnpm-lock.yaml: Generated file

_{Review effort: Lite (auto)}