Skip to content

test(skills): cover frontmatter parse diagnostics during discovery - #84

Open
easonLiangWorldedtech wants to merge 1 commit into
units/skills-05-update-path-resolutionfrom
units/skills-06-frontmatter-diagnostics
Open

easonLiangWorldedtech wants to merge 1 commit into
units/skills-05-update-path-resolutionfrom
units/skills-06-frontmatter-diagnostics

Conversation

@easonLiangWorldedtech

Copy link
Copy Markdown
Owner

Unit 6 of the 8-way split of the SkillsManager test harness described on Zoo-Code-Org#1321.

Merge order for this fork chain: units 2, 3, 4, 5, 6, 7, 8, which is #76, #80, #81, #82, this pull request, and the two unit branches still to open. The org-repo pull requests for these units are opened after Zoo-Code-Org#2417 merges, in the same order.

One file, src/services/skills/__tests__/SkillsManager.spec.ts, 134 added lines and 0 removed. No production change and no new mock surface in this unit.

  • head: units/skills-06-frontmatter-diagnostics at d9cf6ae995c0ee0263f833436f1d577f7cc5ff2a
  • base: units/skills-05-update-path-resolution at 2809ebf7ed5c3a110870878120af87c7fd6c6452. The base class is the immediately preceding unit branch in this fork, a fork branch: not the org repo and not main.
  • labels: this PR sets none, and I read none.

Size and mutation price, both measured

  • Size axis: git diff --numstat 2809ebf7ed5c d9cf6ae995c0 printed 134\t0\tsrc/services/skills/__tests__/SkillsManager.spec.ts, so 134 added and 0 removed, 134 added-plus-deleted against the 400-line budget.
  • Line axis: selectFromGit on the worktree at this head, base 2809ebf7ed5c3a110870878120af87c7fd6c6452, head d9cf6ae995c0ee0263f833436f1d577f7cc5ff2a, printed "packages": [] and "advisories": [] with mergeBase equal to the base, which is 0 counted changed executable lines against the 500-line cap. The unit touches only a file under __tests__/, which the selector excludes, so this unit generates no mutants and its mutation digest is empty by construction.
  • a green local preflight over Valid 0, Killed 0, Survived 0 and No coverage 0 with blocking 0 and advisories 2 is not evidence in either direction on this machine.
  • the mutation leg is unmeasured, a PASS over an empty digest is not evidence in either direction, and CI at the first pushed unit is the only proof of the gate.

Rows and cases, with the counting basis named

3 it headers, all of them scalar rows, so the header count and the case count are the same number for this unit. The suite moves from 116 cases at the base to 119 cases at this head, and 116 + 3 = 119. Row counts in this body are it headers in the spec file at the named head; case counts are what vitest reports at the named head. The two bases are named beside both numbers, because a header count and a case count in one paragraph are the kind of pair a reader conflates.

The three rows assert that a SKILL.md with invalid YAML frontmatter, with other malformed frontmatter, and with unterminated frontmatter plus a dangling quote is skipped during discovery and that the reason is recorded.

Built width versus published width, as a named comparison

Built width 134, published width 134, difference 0, so nothing is published that the build did not measure. The build instrument is a two-pointer subsequence scan between the unit 5 spec and this unit's spec: it introduced 134 lines, removed 0 lines, and matched 3,811 of 3,811 lines of the unit 5 spec, which is 100 percent, and git diff --numstat on the same pair printed 134 added and 0 removed. The spec blob is ca85df011a0b7fd333216eac131724e9ac0eda08 in the worktree, in the commit and in the file I assembled.

The added ranges carry 153 lines of which 19 are blank. The blank-adjacency rule this split uses keeps a blank only when neither the line after the insertion anchor nor the line after that is blank in the parent file; all 19 fall under the rule, so 153 - 19 = 134 and the diff is what a reviewer sees in the merged file. None of this unit's blanks survive. The 21 blanks that survive across the whole chain belong to unit 8: 13 inside its row spans and 8 separators between spans. Blank accounting comes from the assembly manifest plus the rule, not from a diff between two consecutive unit children, because such a diff cannot tell which copy of a repeated line it is reading and its blank counts are position-shifted by construction.

Verification, interpreter named for each leg

  • vitest: Tests 119 passed (119) in 1.90 s (vitest 4.1.12, node 22.23.4, run from src).
  • prettier --check .: exit 0 with no failing files and All matched files use Prettier code style! after 15 s. compile is a required context and its log runs prettier --check ., so the formatting leg is checked rather than assumed.
  • tsc -p tsconfig.json --noEmit: exit 0 with empty stdout after 4 s (node 22.23.4).
  • eslint --prune-suppressions --max-warnings=0 on this spec: exit 0 after 7 s. The suppressions map after the run parses to 265 keys and is deep-equal to the map at HEAD: 0 value-differing keys and 0 keys present only on disk. eslint rewrites that file's indentation when it prunes, so it is restored with git checkout -- and this unit carries no churn in it.
  • Cold pair before the push, both legs turbo run check-types --force with a fresh --cache-dir per leg so that they are cold by construction rather than by a cache directory I guessed at: base leg at 2809ebf7ed5c exit 0 with 12 tasks successful and 0 cached in 12.605 s; head leg at d9cf6ae995c0 exit 0 with 12 tasks successful and 0 cached in 13.371 s; 0 error TS lines on either leg. Both legs ran on node v25.9.0 with pnpm 10.8.1 and turbo 2.11.10 while package.json asks for node 22.23.4, so these are local readings on a different interpreter from CI's and are labelled as local readings.

Negative control, decomposed

Against the pre-delta production blobs at b66149f6c09e2e49a677049c9b47a336c6a3de (src/services/skills/SkillsManager.ts 40,393 B, blob 92ff4e0e094b4d54dff72876e548cac83914a953; src/shared/skills.ts 898 B, blob f5151181f6d3577bec864890a19bb36ac0dabd14), with this unit's spec in place and production otherwise untouched: 27 failed of 119 cases. Machine-summed:

    3  unit 6 rows, 3 of 3 discriminating
  + 6  unit 5 rows
  + 7  unit 4: 3 scalar rows plus 4 discriminating cases of its it.each table row
  + 7  unit 3 rows
  + 2  moveSkill rows that fail only at this chain position, which is order coupling
  + 2  rows that already fail at the parent head
  = 27

The 2 rows in the last bucket are not killers of anything in this split: the parent spec alone against the same production prints 2 failed of 94 cases, and they are the same two rows, should create a new global skill and should allow creating a .roo skill when the duplicate lives in the lower-priority .agents root. The 2 order-coupled rows are moveSkill rows that pass at the parent head and fail only once the chain is this long, so they are a property of the chain position and not of this unit's rows.

Attribution rule used here: every failing row is assigned to the first unit child whose spec file contains its title, and the parent-failing set is measured by running the parent spec against the same production rather than assumed. Matching a title as a substring of a line range is not used, because it steals rows between units and it misses the table row whose header spans several lines. That table row is why unit 4 reads as 3 scalar rows plus 4 cases: its header is on multiple lines.

Two instrument failures in my own lane were fixed before this number was taken, and both are stated rather than tidied. An unverified production restore wrote a 0-byte SkillsManager.ts, because git show had failed and I had not checked it, and the empty file has no export, so the run printed TypeError: SkillsManager is not a constructor and 119 failed for 119, which was an artifact and not a property of the chain; the restore step now checks the exit status and compares the byte length against cat-file -s before it writes. And the base sha I had been passing was a 38-character string rather than the 40-character commit: it resolved while a different commit with that prefix was reachable, and git gc --auto has since pruned that commit, so the control base for units 3 to 5 was held by a dangling object. The base is now read from refs/remotes/origin/prep/skills-test-harness and verified with cat-file before each run, with gc.auto=0 on those runs.

What these rows are killers for, and what is not claimed

These three rows are killers for the frontmatter-diagnostics areas of unit 2's production delta: loadSkillMetadata 43 lines, recordDiagnostic 18, getFrontmatterLine 28, isHiddenEntry 15 and findEscapingRelativeSymlink 6, which is 43 + 18 + 28 + 15 + 6 = 110 of the 405 changed lines in that delta. Naming is a proxy for reachability, not proof that a row kills a mutant: nothing in that map came from running a mutant. The killers for those areas land in the merged state in the declared order (this unit, then units 7 and 8), not on any intermediate head, and unit 2's own rows contribute 0 discriminating killers for its own delta.

Unit 6 of the 8-way split of the SkillsManager test harness described on
Zoo-Code-Org#1321. Chain order: units 2, 3, 4, 5, 6, 7, 8; this
unit merges after unit 5 and before units 7 and 8.

Adds the three discovery rows that assert a SKILL.md with invalid,
malformed or unterminated YAML frontmatter is skipped and the reason is
recorded, against the existing harness only - no production change and no
new mock surface in this unit.

One file, 134 added lines, 0 removed, on top of unit 5 at
2809ebf. The row text is the feature
branch text. The added ranges carry 153 lines of which 19 are blank; all
19 are dropped by the blank-adjacency rule this split uses - a blank is
kept only when neither the line after the insertion anchor nor the line
after that is blank in the parent file - so 153 - 19 = 134 and the diff is
what a reviewer sees in the merged file.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant