feat(competition): engine-lane auto-merge on dethrone, quarantined - #578
Open
plind-junior wants to merge 1 commit into
Open
feat(competition): engine-lane auto-merge on dethrone, quarantined#578plind-junior wants to merge 1 commit into
plind-junior wants to merge 1 commit into
Conversation
a winning strategy submission now auto-merges on the ladder branch - the benchmark decides what enters contrib/, with no human in the loop. the arm job is deliberately separate from the scoring job: scoring executes the untrusted challenger and keeps a read-only token; arming holds the write scopes, checks out nothing, and runs no challenger code. auto-merge fires only when the pr base is KOTH_LADDER_BASE, exactly like the kit lane. what keeps this sane is the quarantine, now stated as the contract in the workflow header and the strategy module docstring: merged contrib strategies execute exclusively through the sandbox child (both gates and the local bench wrap them in SandboxProxy), the ledger and ratchet scripts treat the winner as text and import nothing, and promotion into src/vouch as trusted importable default code remains a human-reviewed pr. accepted residual, documented: a sandbox escape during scoring can forge its own verdict and land its pr - inside the quarantine it already runs in, with no write token, secrets, or import path reachable. docs and the scorecard comment updated to stop claiming engine code never auto-merges.
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. 🗂️ Base branches to auto review (2)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
removes the human from the engine-lane merge loop: a strategy submission that clears the dethrone band against the ladder branch now arms native auto-merge, same as the kit lane. the benchmark decides what enters
contrib/.the write path is deliberately split. the scoring job is unchanged - it executes the untrusted challenger under the sandbox and holds a read-only token plus the comment scope. a new
armjob holds the write scopes, checks out nothing, runs no challenger code, and only turns the gate's verdict intogh pr merge --auto --squash, gated on the pr base beingKOTH_LADDER_BASE. prs against main/test remain scored-and-commented only.the containment contract is now stated explicitly in the workflow header and the
vouch.strategydocstring, and each leg was verified in code: merged contrib strategies execute exclusively through the sandbox child (score_strategy.pywraps both arms inSandboxProxy, and so doesvouch bench run --strategy/--against); the ledger and ratchet scripts are stdlib-only file rewriters that import nothing from the merged tree (promote_champion.pycopies bytes); and promotion intosrc/vouchas trusted importable default code remains a human-reviewed pr - the benchmark is never the sole gate to code users install.the accepted residual is documented rather than papered over: a sandbox escape during scoring can forge its own verdict and auto-merge its pr. what it lands in is the quarantine it already runs in - no write token, no secrets, no in-process import path. the sealed monthly run still settles payouts, so a gamed daily verdict buys a provisional throne, not money.
docs (
koth-strategy-lane.mdtable + honest-boundary section) and the scorecard comment stop claiming engine code never auto-merges. verification: zizmor offline clean, ruff + mypy clean, strategy/bench tests green.