Skip to content

fix(skills): Python test runs, ship to prod, one Stop ask per fix, whole preferences, Codex trust line - #675

Merged
shhdwi merged 77 commits into
mainfrom
fix/skills-e2e
Oct 23, 2026
Merged

shhdwi merged 77 commits into
mainfrom
fix/skills-e2e

Conversation

@shhdwi

@shhdwi shhdwi commented Oct 23, 2026

Copy link
Copy Markdown
Collaborator

Fixes from a real end-to-end run of skills with the other features: real Claude Code sessions on pytest-dev/pluggy, with trail init, learnings, skills, guardrails and the map commands together.

What was wrong, and the fix

  1. Python test runs never counted as a check, so most Python repos never got a tests skill. .venv/bin/pytest, uv run pytest, tox, nox, poetry run and python -m unittest didn't match. A leading <dir>/bin/ is now stripped before matching, those runners are known, and source <venv>/bin/activate && (a plain path in the repo) carries into the steps. Other setup is still refused.
  2. A skill saved from "ship it to prod" didn't come back on "ship it to prod". git push prod main took the branch as the target, so the skill was ship-main. A push to a remote named prod, production, staging or live now targets that remote, and the asks the skill was saved from count when matching.
  3. Stop's save-a-learning ask fired again on a signal already answered. It was keyed on the newest passing run, so every rerun looked new. It's now keyed on the first pass after the last failure: one fail-then-pass asks once.
  4. A kept preference was cut at 120 characters mid-command, and session start injected the cut text. The body keeps the whole line, only the title is cut, at a word, and session start points to the whole line when the title was cut.
  5. Codex users were never told to trust Trail's hooks in /hooks. The line only printed in verbose mode. It's now a ⚠ line, so it always prints.
  6. The on-file-open line no longer prints "· null" when a learning has no credit.

Each fix has a test that failed first. Not in here: the offer's "worked out N times" counts every day the job ran, not only passing runs. That count also decides when the offer comes, so it needs its own change.

🤖 Generated with Claude Code

Shrish Dwivedi added 30 commits October 22, 2026 09:16
- anchors and the timeout guard split PATH on ';' and look for git.exe,
  npm.cmd and the like (PATHEXT); Windows has no exec bit to check
- a scratchpad folder is found with either slash, and Windows paths are
  compared without case (scratch folders, tamper's protected places)
- tamper's cd from an unknown folder stays unknown on Windows, where
  resolve() puts a drive before the '/\0' it starts from
- POSIX file modes are asserted only off Windows; the FIFO and /dev/zero
  cases run only where those exist
- expected paths are built with resolve()/join(), and the fold fixture
  compares checkouts resolved
- capped children import the source by file URL, and say why they failed
- os.homedir() follows USERPROFILE on Windows, so the cd test sets it too
A Windows stack frame joins its folders with '\', so the hook-errors log
kept the whole path, user name included, with the slashes stripped.
…backtracking pattern still misses by seconds
A small matcher with no model reads tool and command preferences from a
learning ("use pnpm, not npm", "never run X", "always run Y before Z",
"run X instead of editing F") and matches them against each step's program
and arguments. It is conservative: when unsure it says nothing.

- At a save or offer, steps follow the person's live learnings: a step one
  goes against is rewritten when that is safe, else left out, and the offer
  names it in one plain line.
- At recall, a learning that only repeats a step is said once, in the run
  block, and left out of the prompt's learnings lines. One that goes against
  a step wins; the block marks the step, and the update offer proposes the
  change with its usual typed yes.
- At trail init, a preference that only repeats a step of a skill kept on
  the same screen goes into that skill as a lesson.

Retired, replaced and held learnings are ignored.
command -v split PATH on the platform's delimiter but looked only for the
bare name, so on Windows every program read as not installed. It now uses
pathDirs and programFiles from guard-fast, as anchors and the timeout guard do.
- POSIX file modes are asserted only off Windows
- the FIFO cases and the socket, /tmp and ':' PATH probe test run only off Windows
- a non-executable timeout on PATH counts on Windows, which has no exec bit
- children import the source by file URL
On Windows HOME is often unset. The update gate then showed a path in
full instead of as ~/..., and a saved skill's repairs never turned the
home folder into $HOME. Both now fall back to homedir().
- the FIFO half of the value-line link test runs only off Windows
- apply_patch's relative file and tamper's hit are compared resolved
- the wall fixes' program check splits PATH on ';' and looks for git.exe,
  npm.cmd and the like (PATHEXT); Windows has no exec bit to check
- look-back finds a scratchpad folder with either slash and ignores case
  on Windows, so it no longer mines scratch sessions there
- the FIFO cases and the file name with a newline run only off Windows
- capped children import the source by file URL
- the naming calls' shebang stand-ins run only off Windows, as in
  init-claude-call
# Conflicts:
#	test/hosts-codex-trust.test.ts
… only USERPROFILE is set

As in the save gate: on Windows HOME is often unset, so fall back to homedir().
The file-open text said "it's a note from your team". It now says what the
prompt hook says: a learning from past sessions, this repo's or the user's own.
- the FIFO cases, the unwritable Trail home and the cache's file mode are
  tried only off Windows; the corrupt-file cases run everywhere
- PATHs are joined with the platform's delimiter
- the stand-down test sets USERPROFILE with HOME, which os.homedir() reads
- the naming call's shebang stand-in runs only off Windows
Shrish Dwivedi and others added 25 commits October 22, 2026 20:48
…p, and run again when one finds the lock busy

On a loaded Windows runner the two processes took longer than the fixed 1.5 s
to start, and the first held the ledger lock past the 200 ms the second waits,
so the second gave up and wrote nothing (by design). The lock itself works
there: the real-processes test in the same file passed with no lock file left.
Each hook now warms up, says it is ready, and starts at an instant sent after
both are up. A round where one found the lock busy proves nothing and runs
again, up to 5 times. Reading the ledger outside the lock still fails it 5/5.
…s part 4 adds and the ones that failed on Windows

timeBar(ms, floor) in machine-speed.ts: the bar where it was set, times
slowness(), never under a floor of 3 to 5 times it; exactly the bar under
TRAIL_PERF=1. The 2 MB walls probe times CPU, not the wall clock. The busy
value.json lock wait is measured over the same call with the lock free.
The 20,000-line clustering bar and the time caps' slack (the 1.5 s replay
cap, the prompt's 400 ms deadline) use timeBar. The caps themselves stay as
they are. The stand-in timeouts that are posix-only and already bounded by
what a broken cap would take are left alone.
Absolute bars use timeBar (3 to 5 times the bar as the floor). Budget tests
keep the budget and scale only the slack around it, held under whatever a
broken budget would take (the 4 s default, the stub's 3 s parse). The
pre-bash host runs kill the child 3.5 s past the scaled bar, as before.
TRAIL_PERF-only tests keep their bars.
feat(skills): learnings and skills never repeat or contradict each other
feat(skills): wire skills into the hooks and the CLI, on by default
…check

A Python test run from a venv (.venv/bin/python -m pytest, .venv/bin/pytest),
through uv, poetry or pdm, or as tox, nox or python -m unittest never matched
the tests family, so init recorded the job as no-check and offered no tests
skill. The runner's bin folder now goes before matching, the new runners match,
and unittest's Ran/OK summary is read.

A venv's own activate chained with && (source .venv/bin/activate && STEP) is
known-safe setup: it is carried into the step word for word instead of
refusing the run as needing its shell set up. Any other setup still refuses.
git push prod HEAD:main made compile take the branch main as the target, so
a skill saved from "ship it to prod" was named ship-main and the ask naming
prod never matched it. A push to a remote named for a deploy target (prod,
production, staging, live) now goes to that remote; the branch it pushed and
the places the person named when the skill was saved go into its asks, so
"ship it to prod", "ship it" and "push to main" all bring it back.
The fixed sign was keyed on the newest passing run, so each rerun of the same
passing check made a new key and the Stop hook asked to save a learning again
after "Nothing worth keeping", until the cap of three. It is now keyed on the
first pass after the last failure: one episode asks at most once, and a new
failure and fix is still a new sign.
A kept preference was cut at 120 characters mid-word, so pluggy's line ended
PYTHONPATH=sr and the agent was told to follow a wrong command. The body now
keeps the person's whole line; only the title is cut, after the last whole
word, with an ellipsis. The standing learnings at session start say the whole
line when the title was cut.
The line went through say(), and init's short output keeps only lines that
start with ⚠, so without --verbose a Codex user never heard that Codex runs
Trail's hooks only after they trust them in /hooks. It is now a ⚠ line and
prints either way.
A learning with no measured cost has no saving to claim, and the example
credit line printed "· null" after its name. The suffix is now left out.
…chors

- a shell edit's file keeps its drive and backslashes, so the Stop ask names it
- trail check queues a learning's file name, not its whole Windows path
- a plain-file read refuses a link on Windows, where open has no O_NOFOLLOW
- a bin target and a node <file> word compare as dist/cli.js on every platform
- the cp line for a hand-edited skill uses forward slashes, which Git Bash and PowerShell both take
- the journey box aliases CWD's own key (local/d-home-dev-shop on Windows), not a hand-written one
- the fail-open hook commands run in Git Bash on Windows, the shell Claude Code runs hooks in
- the J16 deny tests skip where Trail still finds a timeout: Windows ships System32\timeout.exe
- the Codex sandbox tests skip where chmod refuses nothing (Windows, root)
- CRLF in TELEMETRY.md, native separators in a skills path, a Claude projects folder name
- the prompt-activity ask child from source gets the installed 60 s, not the hook's 4 s
…fuse a nested repeat

A grep takes a regex, so the pattern's syntax is kept: every literal character
goes through the metacharacter escape and only fixed syntax is written as
itself, so RegExp gets the typed pattern rebuilt from known parts (CodeQL
js/regex-injection, alert 23). A repeat of a group that repeats inside, like
(a+)+, is refused as an invalid pattern is: on one long line it backtracks for
minutes and holds the MCP server. Every other pattern compiles exactly as before.
…/skills-e2e

# Conflicts:
#	test/trail-note-ask.test.ts
@trailhq-graft

trailhq-graft Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

🌱 graft blast radius

5 areas changed → 3 areas can be affected. 15 dependent symbols, depth 2.
Tests: no test reaches Preference Persistence, Test Validation, CLI Wiring; 2 areas updated their tests.
Tag: @anirudhkumar-nanonets — 3 of 8 areas

flowchart TB
  A0(("Touch Learning Events<br/>7 symbols"))
  A1(("Skill Offers<br/>6 symbols"))
  A2(("Initialization Lookback<br/>2 symbols"))
  classDef reached fill:#D9EDF3,stroke:#3AA7C9,stroke-width:1.5px,color:#0E313C;
  class A0,A1,A2 reached;
Loading
Can be affected Symbols Nearest hop Reached from
Touch Learning Events 7 src/claude/hooks.ts:L166-L190 learningsOnTouch — calls, depth 1 Note Formatting
Skill Offers 6 src/skills/compile.ts:L302-L442 build — calls, depth 1 stableBranches
Initialization Lookback 2 src/init/run.ts:L150-L165 itemsOf — calls, depth 1 Preference Persistence
Who knows this code — 1 person across 8 areas
Area Who knows it
Note Formatting · changed @anirudhkumar-nanonets — 12 commits, last yesterday
CLI Wiring · changed @anirudhkumar-nanonets — 61 commits, last yesterday
Touch Learning Events · affected @anirudhkumar-nanonets — 16 commits, last yesterday
stableBranches · changed only you — nobody else has touched these files
Preference Persistence · changed only you — nobody else has touched these files
Test Validation · changed only you — nobody else has touched these files
Skill Offers · affected only you — nobody else has touched these files
Initialization Lookback · affected only you — nobody else has touched these files

Ownership is git history over each area's own files, weighted towards recent work (120-day half-life). Merge commits and bots are dropped, and you are dropped from your own PR. A name with no @ has no GitHub handle in its commit email — tag them by hand, or add a .mailmap entry. A suggestion from history, not a CODEOWNERS rule.

All 15 dependent symbols, grouped by area

Touch Learning Events — 7 symbols in 2 files

  • src/claude/hooks.ts:L166-L190 — learningsOnTouch (calls, depth 1)
    183: const text = notesOnTouch(notes, touch, s);
  • src/claude/hooks.ts:L413-L426 — learningsAtStart (calls, depth 1)
    414: // 1b (P21, J16): every-repo personal learnings load in every repo, a plain one too, skills on or off; never a
  • src/claude/stop.ts:L141-L155 — askForNote (calls, depth 1)
    147: const sign = noteSignals(readFileSync(transcript, 'utf8'), dir).find((x) => !asked.includes(x.key));
  • src/claude/hooks.ts:L90-L103 — handlePostEdit (calls, depth 2)
    101: // One hook, one answer: two JSON objects on stdout would be read as neither.
  • src/claude/hooks.ts:L197-L202 — withTouchLearnings (calls, depth 2)
  • src/claude/hooks.ts:L479-L558 — main (calls, depth 2)
    486: // Registered in the repo and in ~/.claude, Claude Code runs both: one answers.
  • src/claude/stop.ts:L182-L201 — handleStopEvent (calls, depth 2)
    191: // A turn always ends; an offer can wait for the next one.

Skill Offers — 6 symbols in 4 files

  • src/skills/compile.ts:L302-L442 — build (calls, depth 1)
    302: function build(sheet: RunSheet, opts: CompileOpts): CompiledSkill {
  • src/skills/offer-moment.ts:L248-L295 — repeatedOfferDue (calls, depth 1)
    260: if (!hit || !hit.passed || !families.includes(hit.family)) return { due: false, why: 'family' };
  • src/skills/offer-repair.ts:L143-L202 — repairOfferDue (calls, depth 1)
    151: const sheet = i.sheet;
  • src/skills/compile.ts:L478-L488 — compileRun (calls, depth 2)
    481: const sheet = joinCarriedSheet(input, opts.readSheet);
  • src/skills/offer-moment.ts:L209-L227 — offerDue (calls, depth 2)
  • src/skills/update-plan.ts:L121-L165 — gather (calls, depth 2)
    127: const sheet = readRunSheet(runId, env);

Initialization Lookback — 2 symbols in 1 file

  • src/init/run.ts:L150-L165 — itemsOf (calls, depth 1)
    159: const line = prefLine(c, env);
  • src/init/run.ts:L175-L360 — finishLookBack (calls, depth 2)
    198: const modelWhy = agent ? 'agent' : reads.plain ? 'plain' : !person.tty ? 'no-tty' : lookBackOff(env) ? 'off' : !person.ok ? 'not-person' : !skillsToName.length ? 'no-skills' : null;
Test signal per changed area — 2 ✓ · 3 ✗

Reached = a node under a test path has a resolved edge into the changed symbol. It undercounts anything called indirectly — through a CLI, a spawned process or a dynamic import — so read a low ratio as “look here”, never as a coverage gate.

  • ✓ stableBranches — 1 of 8 reached · 1 test file changed here: test/skills-ship-remote-target.test.ts
    • not reached: venvActivate, walk, emit, deployRemote, pushedStable, same, jobWords
  • ✓ Note Formatting — 3 of 3 reached · 3 test files changed here: test/init-keep.test.ts, test/trail-learning-on-touch.test.ts, test/trail-note-ask.test.ts
  • ✗ Preference Persistence — 0 of 3 reached · no test file reaches it
    • not reached: prefLine, prefTitle, keepPreference
  • ✗ Test Validation — 0 of 2 reached · no test file reaches it
    • not reached: simpleFamily, testsPassed
  • ✗ CLI Wiring — 0 of 1 reached · no test file reaches it
    • not reached: wireTarget
7 test suites also reference this code

7 symbols, kept out of the diagram and the table so they cannot crowd out the areas a reviewer has to look at.

  • test/init-b6-review.test.ts
  • test/init-fold-lessons.test.ts
  • test/learnings-link-1b.test.ts
  • test/skills-compile.test.ts
  • test/skills-offer-moment-2.test.ts
  • test/trail-text-safety.test.ts
  • test/trail-words.test.ts

graft blast · origin/main...HEAD · depth 2 · 13 changed files

Open the interactive graph → — click an area to see its dependent symbols at file:line.

github-actions Bot added a commit that referenced this pull request Oct 23, 2026
@shhdwi
shhdwi merged commit 49c0218 into main Oct 23, 2026
8 checks passed
github-actions Bot added a commit that referenced this pull request Oct 23, 2026
@shhdwi shhdwi mentioned this pull request Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant