Skip to content

test(skills): cover skill diagnostics read and clear paths - #85

Open
easonLiangWorldedtech wants to merge 1 commit into
units/skills-06-frontmatter-diagnosticsfrom
units/skills-07-skill-diagnostics
Open

easonLiangWorldedtech wants to merge 1 commit into
units/skills-06-frontmatter-diagnosticsfrom
units/skills-07-skill-diagnostics

Conversation

@easonLiangWorldedtech

Copy link
Copy Markdown
Owner

Unit 7 of the 8-way split of the SkillsManager test harness described on Zoo-Code-Org#1321.

Merge order for this fork chain: units 2, 3, 4, 5, 6, 7, 8, which is #76, #80, #81, #82, #84, this pull request, and unit 8 still to open. The org-repo pull requests for these units are opened after Zoo-Code-Org#2417 merges, in the same order.

One file, src/services/skills/__tests__/SkillsManager.spec.ts, 206 added lines and 0 removed. No production change and no new mock surface in this unit.

  • head: units/skills-07-skill-diagnostics at 1a29b1cd613d258919f3ddd6f4e00b0d69b516ab
  • base: units/skills-06-frontmatter-diagnostics at d9cf6ae995c0ee0263f833436f1d577f7cc5ff2a. The base class is the immediately preceding unit branch in this fork, a fork branch: not the org repo and not main.
  • labels: this PR sets none, and I read none.
  • spec blob at this head, named by instrument: git blob id 58c5f25fbecc73ca925fbe6c4c38b527ec9437d7; sha256(16) of the same file db3f4b98dfe62d0e. A digest without its instrument named is how a reader comparing git hash-object against a lane report finds a mismatch that does not exist.

Size and mutation price, both measured

  • Size axis: git diff --numstat d9cf6ae995c0 1a29b1cd613d printed 206\t0\tsrc/services/skills/__tests__/SkillsManager.spec.ts, so 206 added and 0 removed, 206 added-plus-deleted against the 400-line budget.
  • Line axis: selectFromGit on the worktree at this head, base d9cf6ae995c0ee0263f833436f1d577f7cc5ff2a, head 1a29b1cd613d258919f3ddd6f4e00b0d69b516ab, printed "packages": [] and "advisories": [] with mergeBase equal to the base, which is 0 counted changed executable lines against the 500-line cap. The unit touches only a file under __tests__/, which the selector excludes, so this unit generates no mutants and its mutation digest is empty by construction.
  • a green local preflight over Valid 0, Killed 0, Survived 0 and No coverage 0 with blocking 0 and advisories 2 is not evidence in either direction on this machine.
  • the mutation leg is unmeasured, a PASS over an empty digest is not evidence in either direction, and CI at the first pushed unit is the only proof of the gate.

Rows and cases, with the counting basis named

6 rows, all scalar it headers, inside the skill diagnostics describe block. The suite moves from 125 cases at this head where 116 are it headers in this file: the 9-case gap is the multi-line it.each headers carried since unit 3, which a header-line count misses and vitest counts as cases. The base head reports 119 cases and this head 125, and 119 + 6 = 125. Row counts here are it headers in the spec file at the named head; case counts are what vitest reports at the named head.

The six rows assert that malformed YAML frontmatter is reported with its location while discovery still returns the healthy skills, that overlapping scans are serialized so an older scan cannot overwrite a newer snapshot, that a reader never sees a half-built snapshot, that the parser does not hint at unescaped quotes when the parse failure is elsewhere, that the same malformed skill keeps reporting on repeated scans, and that a non-Error parse failure still records a diagnostic.

Built width versus published width, as a named comparison

Built width 206, published width 206, difference 0. The added ranges carry 206 lines of which 0 are blank: the file has 555 blank lines at this head and 555 at the base head, so blank growth is 0 and the blank-adjacency rule this split uses drops nothing here, 206 - 0 = 206. Composition: 204 lines of row content plus the 2 carried container lines of the skill diagnostics describe block, its opening line and its closing brace, which arrive with this unit because its rows are the first rows inside that block; 204 + 2 = 206. Every blank that survives across the whole chain belongs to unit 8, 13 inside its row spans and 8 separators between its spans.

Verification, interpreter named for each leg

  • vitest: Tests 125 passed (125) in 2.35 s (vitest 4.1.12, node 22.23.4, run from src).
  • prettier --check .: exit 0, 0 failing files, All matched files use Prettier code style!. compile is a required context and its log runs prettier --check ., so a prettier-forced trailing newline is not a fidelity failure.
  • tsc -p tsconfig.json --noEmit: exit 0, empty stdout (node 22.23.4).
  • eslint --prune-suppressions --max-warnings=0 on this spec: exit 0. The suppressions map after the run parses to 265 keys and is deep-equal to the map at HEAD: 0 value-differing keys, 0 keys present only on disk. eslint rewrites that file's indentation when it prunes, so it is restored with git checkout -- and this unit carries no churn in it.
  • Cold pair before the push, both legs turbo run check-types --force with a fresh --cache-dir per leg so they are cold by construction: base leg at d9cf6ae995c0 12 tasks successful and 0 cached in 14.169 s, exit 0; head leg at 1a29b1cd613d 12 tasks successful and 0 cached in 13.470 s, exit 0; 0 error TS lines on either leg. Both legs ran on node v25.9.0 with pnpm 10.8.1 and turbo 2.11.10 while package.json asks for node 22.23.4, one engine-mismatch line per leg, so these are local readings on a different interpreter from CI's and are labelled as local readings.

Negative control, decomposed

With this unit's spec in place and production otherwise untouched at b66149f6c09e2e4649a677049c9b47a336c6a3de (src/services/skills/SkillsManager.ts 40,393 B, blob 92ff4e0e094b4d54dff72876e548cac83914a953; src/shared/skills.ts 898 B, blob f5151181f6d3577bec864890a19bb36ac0dabd14; the 40-character sha printed by refs/remotes/origin/prep/skills-test-harness and proved with cat-file -e): 33 failed of 125 cases. Machine-summed:

    6  unit 7 rows, 6 of 6 discriminating
  + 7  unit 3 rows
  + 7  unit 4 tests from 4 rows (3 scalar plus 4 discriminating cases of its multi-line it.each table row)
  + 6  unit 5 rows
  + 3  unit 6 rows
  + 2  moveSkill rows that fail only at this chain position, which is order coupling
  + 2  rows that already fail at the parent head
  = 33

The set difference against unit 6's control run on the same base is exact: unit 6 prints 27 distinct failing rows, unit 7 prints 33, and the 6 that appear here and not there are unit 7's own six rows, with 0 rows lost, so 27 + 6 = 33 is a measured delta and not an inferred one. The 2 parent-failing rows are should create a new global skill and should allow creating a .roo skill when the duplicate lives in the lower-priority .agents root, measured by running the parent spec against the same production: 2 failed of 94. The 2 order-coupled rows are should move a skill from generic to mode-specific directory and should move a skill from one mode to another, both already present in the parent spec and passing there, failing only once the chain is this long.

Attribution rule: parent-first. A failing row belongs to a unit only if its title is absent from the parent spec, and then to the first unit child whose spec contains it; the parent-failing set is measured rather than assumed. A title present in the parent that fails only at chain length is attributed to the chain position, not to the unit whose text it resembles. Matching a title as a substring of a line range is not used, because it steals rows from short spans when a longer span contains similar text.

Two restores were proved before this number was taken, because a control run against a broken baseline does not fail loudly, it fails plausibly: each write is gated on git show exiting 0 and the written byte length equalling git cat-file -s, which here printed 40,393 of 40,393 B and 898 of 898 B with blob ids matching, and production was restored to the head blobs afterwards on the same gate, with the tree equal to the head tree before the push.

What these rows are killers for, and what is not claimed

The six rows name one area of unit 2's production delta on the call-site basis: getSkillDiagnostics, 3 changed lines. The basis is named because the two available bases disagree here and the weaker one reads larger. Every one of the six row spans calls discoverSkills, getSkillsMetadata and getSkillDiagnostics on the skillsManager receiver; the first two are not among the map's areas, because the map's areas are the declarations unit 2's delta actually changed, and neither name appears on an added line of git diff --unified=0 b66149f6c09e 6471e7fbd8a9 -- src/services/skills/SkillsManager.ts in the reading taken here (1 added lines mention discoverSkills and 0 mention getSkillsMetadata), so they are exercised but not newly introduced. On the weaker basis, the identifier recordDiagnostic appears inside one row span, and it is stated as identifier presence rather than folded into the call-site tally. Taken with unit 6 on the call-site basis, units 6 and 7 name loadSkillMetadata 43, recordDiagnostic 18, getFrontmatterLine 28, isHiddenEntry 15, findEscapingRelativeSymlink 6 and getSkillDiagnostics 3, which as a union is 110 + 3 = 113 of the 405 changed lines.

Two defects in the derivation instrument for this paragraph, recorded rather than tidied. It first looked for the receiver name manager, while the spec's receiver is skillsManager, so it printed no calls at all for six rows that call three methods each. And it looked up two area line counts by short key while the map stores them under SkillsManager.getFrontmatterLine (module function) and SkillsManager.isHiddenEntry (module function), so the first receipt printed NaN where the map says 28 and 15. Both are lookup errors in my own instrument, not properties of the rows or of the map, and the numbers in this paragraph come from the map's own stored counts read through their stored keys.

Naming is a proxy for reachability, not proof that a row kills a mutant: nothing in that map came from running a mutant. The killers for these areas land in the merged state in the declared order - this unit is the second of the three diagnostics units, with unit 8 carrying the container rows - not on any intermediate head, and unit 2's own rows contribute 0 discriminating killers for its own delta.

Not claimed: any CI mutation reading for this head or for units 3 to 6, none having been read; and nothing about unit 8, whose rows complete the diagnostics container and whose built width is 197 + 13 + 8 = 218 and will be reported against its published width as its own named comparison.

Adds the skill-diagnostics rows to src/services/skills/__tests__/SkillsManager.spec.ts:
206 added lines, 0 removed, one file. The added ranges carry 206 lines of which 0
are blank, so nothing is dropped by the blank-adjacency rule this split uses - a
blank is kept only when neither the line after the insertion anchor nor the line
after that is blank in the parent file - and 206 - 0 = 206, which is what a
reviewer sees in the merged file.

Two of the 206 lines are the carried container lines of the skill diagnostics
describe block, its opening line and its closing brace. They arrive with unit 7
because unit 7's rows are the first rows inside that block; every blank that
survives across the whole chain belongs to unit 8, 13 inside its row spans and 8
separators between spans.

This is unit 7 of the 8-way split of the SkillsManager test harness described on
Zoo-Code-Org#1321. Merge order for the fork chain is units 2, 3, 4, 5, 6,
7, 8; the org-repo pull requests for the same units are opened after Zoo-Code-Org#2417 merges,
in the same order.
@easonLiangWorldedtech

Copy link
Copy Markdown
Owner Author

Superseded sentence in the body of fork #85, named here rather than edited out of that body.

The paragraph on what unit 7's rows are killers for says that neither discoverSkills nor getSkillsMetadata "appears on an added line" of unit 2's production delta, and prints the counts in the same sentence as 1 and 0. The counts are the measurement; the sentence contradicting them is wrong, and the reader who noticed had the right evidence in front of them.

Measured again for this comment, on git diff --unified=0 b66149f6c09e2e4649a677049c9b47a336c6a3de 6471e7fbd8a94ac732197f2ee488e8fbc47902ce -- src/services/skills/SkillsManager.ts counting lines that begin with + and are not the file header:

  • discoverSkills: 1 added line, await this.discoverSkills(), inside hunk @@ -81,3 +129,71 @@ export class SkillsManager {.
  • getSkillsMetadata: 0 added lines.

So the corrected statement is: getSkillsMetadata appears on no added line of that delta, and discoverSkills appears on one. discoverSkills still contributes 0 to the union, for a different reason than the one #85 gave: the map's areas are the declarations the delta actually changed, and no area key contains discoverSkills or getSkillsMetadata (measured: 0 area keys match discoverSkills, 0 match getSkillsMetadata; the nearest entry is SkillsManager.runDiscovery, 1 changed line). The added line above is attributed by the map to whichever declaration encloses it, and my map artifact stores counts by enclosing declaration without storing the line ranges, so I am not stating here which declaration owns that line - that is a limit of the artifact, not a claim that none does.

What does not change: unit 7's six rows name getSkillDiagnostics, 3 changed lines, on the call-site basis; units 6 and 7 together name 110 + 3 = 113 of the 405 changed lines; the control reading stands at 33 failed of 125 cases decomposed 6 + 7 + 7 + 6 + 3 + 2 + 2; the diff stands at 206 added and 0 removed in one file. No number in #85 is superseded by this comment; one sentence about a count is.

posted by u5a-2359, unit 7 lane.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant