Skip to content

docs(gemini-task): mutation tables for test briefs, review rounds as PR comments - #12474

Merged
MarkusNeusinger merged 3 commits into
mainfrom
docs/gemini-task-test-lessons
Oct 30, 2026
Merged

MarkusNeusinger merged 3 commits into
mainfrom
docs/gemini-task-test-lessons

Conversation

@MarkusNeusinger

Copy link
Copy Markdown
Owner

Summary

Lessons from the first parallel Gemini task (#12470, brief section-tests) folded into the gemini-task skill:

  • Test briefs name where the behavior lives and carry a mutation table. The section-tests briefs allowed new files only under app/src/sections/, while the state those sections show lives in app/src/pages/*.tsx. Gemini copied the page logic into test harnesses (StatefulPaletteGrid, StatefulSpecMatrix, and others), every gate passed, and deliberate breaks in the real pages went unnoticed. A test brief now lets the package add tests next to the real code, and its Acceptance lists, per behavior, one break that must turn a test red and one harmless refactor that must stay green. Gemini reverts each break with git restore (its agy rules deny git checkout).
  • The review subagent runs the mutation table and adds breaks of its own (section 3, step 3). That is how the review of test(app): add render tests for palette, debug, and map sections #12470 found the copied logic in two of three packages.
  • Review rounds go back as one PR comment with a section per package (section 3, step 4), and Gemini works them like the first round: packages branch from origin/gemini/<slug> as gemini/<slug>-r<n>-<k>, merge back with --no-ff, path checks diff against the task branch, and Gemini comments with the merge commits and the mutation results (section 2, step 1). The PR stays a draft until the last round, because marking it ready starts its one Copilot review. Round 1 of test(app): add render tests for palette, debug, and map sections #12470 ran this way: three subagents in parallel, 13 of 14 mutations caught, and Claude fixed the last gap.
  • Cleanup asks before deleting package branches. CLAUDE.md lets only the PR's own worktree and local branch go without asking, and the skill said to delete package branches without that check. It now checks that each branch is part of the merged head, and asks with the branch names in the question. Today a general OK to the cleanup plan did not get past the permission classifier.

Test plan

  • uv run python -m tools.changelog check --base origin/main: the fragment is well-formed and the diff passes.
  • git commands quoted in the skill exist in the agy rules: git restore is allowed and git checkout is denied (agy-permissions.json), and the push regex covers gemini/<slug>-r<n>-<k>.
  • No automated check covers skill prose; the next Gemini test task is the real test.

🤖 Generated with Claude Code

…as PR comments

Test briefs name where the behavior lives and carry a mutation table in
Acceptance; the review subagent runs it. Larger review findings go back as
one PR comment with a section per package, which Gemini works in parallel on
gemini/<slug>-r<n>-<k> branches from the task branch. Cleanup asks before
deleting package branches, as CLAUDE.md requires.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings October 30, 2026 11:44
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Approval is blocked by two critical workflow-safety issues involving destructive restoration and stale ancestry checks, plus a minor documentation grammar issue.

3 open issues
What changed in this PR

This pull request updates the gemini-task workflow with mutation-oriented test briefs, structured review rounds, and safer package-branch cleanup guidance. It also documents the lessons in a changelog fragment.

Changes:

  • Requires test briefs to identify production behavior and define mutation tables.
  • Documents review-round branching, mutation validation, and PR-comment reporting.
  • Adds ancestry checks and confirmation requirements before deleting package branches.
  • Adds the corresponding changelog entry.
File Summary Review findings
.claude/​skills/​gemini-task/​SKILL.md Defines mutation testing, review-round handling, and package-branch cleanup procedures. Mutation breaks must be reverted, including ad-hoc breaks. The current git restore guidance may discard implementation changes, so mutations should use a clean checkpoint or reversible patch. Fetch package branches immediately before ancestry checks.
changelog.d/​gemini-task-test-lessons.md Records the lessons motivating the workflow changes. Fix the incomplete sentence describing test briefs and qualify the parallel-review wording for coupled packages.

This pull request requires changes.

Review effort: Lite

Comment thread .claude/skills/gemini-task/SKILL.md Resolved
Comment thread .claude/skills/gemini-task/SKILL.md Resolved
Comment thread changelog.d/gemini-task-test-lessons.md Resolved
…e the ancestry check

Copilot review on #12474: git restore would also drop uncommitted work in the
mutated file, and a stale remote-tracking ref could make an old tip look merged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The change relies on interactive Gemini/agy branching, mutation validation, recovery, and remote cleanup behavior that repository checks cannot exercise, so owner verification of an end-to-end review round is needed.

No open issues 🎉

3 resolved since last review

Review effort: Lite

@MarkusNeusinger
MarkusNeusinger merged commit 7c256b6 into main Oct 30, 2026
21 checks passed
@MarkusNeusinger
MarkusNeusinger deleted the docs/gemini-task-test-lessons branch October 30, 2026 11:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants