Repository navigation
Say what a non-interactive agent should do at the scope check (for discussion) - #54
Open
anakanemison wants to merge 1 commit into
Conversation
…scussion) The scope check tells the agent to surface custom commands, codegen and non-system find_package uses "and confirm before proceeding". In a non-interactive run there is nobody to confirm with. The skill does not say what to do instead, so the agent proceeds and the findings the gate exists to surface are not recorded anywhere. Measured in any2bazel-lab: the gate was passed in 13 of 13 applicable non-interactive runs on small projects, and in every larger- project run but one (which stopped with a correct scope report). The outcomes were mostly good; what was lost is the audit trail. Adds the fallback the proposal suggests: if no one can answer, record the findings and the decision taken in any2bazel.json under "scope_findings" (record-only; config.load ignores unknown keys; listed with the other fields) and proceed, unless the not-yet-supported list applies, where the stop holds even unattended. The fallback replaces the confirmation, not the stop, so for an item on that list no answer still means stop and report. In the lab's later runs with this change, every migrating agent recorded its scope findings, and the YugabyteDB agent stopped with a scope report. This is a proposal for discussion: whether an unanswerable gate should become "record and proceed", or stay a stop the agent is expected to honor, is the maintainers' design call. Lab-Proposal: P4 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part 19 of 23 of a stacked series (tracking issue: #35). Based on
D1-project-agent-instructions, the previous part: review and merge that first.Kind: discussion. Lab proposal: P4.
What the migration runs showed
Unattended agents never honored the step-1 confirmation. With this, every migrating run recorded
scope_findings.The change
The scope check tells the agent to surface custom commands, codegen and
non-system find_package uses "and confirm before proceeding". In a
non-interactive run there is nobody to confirm with. The skill does not say what to do instead,
so the agent proceeds and the findings the gate exists to surface are not
recorded anywhere.
Measured in any2bazel-lab: the gate was passed in 13 of 13
applicable non-interactive runs on small projects, and in every larger-
project run but one (which stopped with a correct scope report). The
outcomes were mostly good; what was lost is the audit trail.
Adds the fallback the proposal suggests: if no one can answer, record the
findings and the decision taken in any2bazel.json under "scope_findings"
(record-only; config.load ignores unknown keys; listed with the other
fields) and proceed, unless the not-yet-supported list applies, where the
stop holds even unattended. The fallback replaces the confirmation, not the
stop, so for an item on that list no answer still means stop and report.
In the lab's later runs with this change, every migrating agent recorded its
scope findings, and the YugabyteDB agent stopped with a scope report.
This is a proposal for discussion: whether an unanswerable gate should
become "record and proceed", or stay a stop the agent is expected to
honor, is the maintainers' design call.
Testing
pytest tests/at this commit: 196 passed (130 onmain).🤖 Generated with Claude Code