[mithunyc]

Why

Buildloop told agents to stay inside allowed_files, write honest receipts, and not call work done without evidence. Almost none of that was checked by code. Two outside reviews (Codex gpt-6.1-sol and Gemini 4 Argon) independently rated it a NO_GO for enterprise use for that reason. This PR turns the Definition of Done into a script and adds a way to measure how agents actually perform.

What changed

Definition of Done as code. buildloop verify --contract <c> --receipt <r> --rerun --base <ref> returns DONE or NOT_DONE. It fails a slice when:

• a changed file is outside the contract, including files hidden by .git/info/exclude, skip-worktree, or a git mv
• .buildloop.yml, package.json, AGENTS/CLAUDE/GEMINI.md, workflows, CODEOWNERS, or an existing contract changed without a named human approver
• the gates failed, no test gate ran, the gate commands modified the files they tested, or code changed after the gates ran (per-file hashes)
• the receipt disagrees with the evidence, or leaves acceptance-to-proof or rollback empty

With --rerun, verify runs the gates itself and does not trust stored JSON. It also runs no gate commands until the scope and rule checks pass.

Learning loop. buildloop learn reads the verdict ledger and reports first-pass, eventual, and current DoD rates, plus checks that keep failing. It recommends; a human decides whether a lesson becomes a test, a gate, or a rule (templates/lessons.md). Agents can't edit their own governance through it.

Founder path. buildloop init writes .buildloop.yml from your package scripts, so there are no placeholders to fill in. buildloop receipt generates the receipt from the gate evidence.

Installers now ship the runtime (scripts, schemas, templates, references) inside the enterprise-ai-dev skill. Before this, installed agents couldn't run any of the scripts the skill told them to use.

Sandbox. Rejects option-like image names (--privileged) and host-path volume names (/var/run/docker.sock). Drops capabilities, sets no-new-privileges, mounts .git read-only (hooks run on the host), shadows .env* files at any depth, and removes the container if the client times out. "Allowlist" networking is now labeled as unenforced and needs the same approval as full network.

Docs and CI. SECURITY.md no longer says protected paths are blocked before an edit; it now has an "Enforced vs. Advisory" table. CI actions are pinned to commit SHAs, and the DoD suite runs in CI. New templates: release plan with rehearsed rollback, lessons ledger, a PR workflow that runs verify --rerun, and thin CLAUDE.md/GEMINI.md adapters.

How it was tested

• 43 DoD tests, 30 sandbox, 17 CLI, 13 capability, and 10 install tests, all passing locally and in CI. The repo now has its own .buildloop.yml, and its gate runs every suite.
• Real installs into a temp CODEX_HOME with both install.sh and install.ps1, then ran the bundled scripts from the install location.
• Dry run with a real agent: Codex (gpt-6.1-sol) implemented a slice in a fresh repo using only the installed runtime. In the first run, the repo's test script was broken and fixing it needed a file outside the contract. The agent stopped and reported NOT_DONE instead of working around the gate. With the fixture fixed, it reached DONE on the first attempt.
• Adversarial runs: Codex gpt-6-astra found four ways to get a false DONE: edited gate results, code imported from an ignored folder, a test that rewrites its own source, and a skipped test gate. Gemini's re-score found that --rerun executed gates before checking the rule files. Each one is fixed and has a regression test.
• Dogfood: running verify --rerun --base main on this branch in a clean worktree passes scope, gates, and freshness. It fails approval and governance, because this PR changes CI and .buildloop.yml and nobody has approved it yet. That's the intended result: the approval is this review.

Review follow-ups

A full review of this PR found ten issues, all fixed in 34e1e48 with regression tests. The ones that mattered most:

• .env shadowing used a tmpfs mount over a file, which Docker rejects, so the sandbox wouldn't start in any repo with a .env. It now binds /dev/null read-only.
• The gate mutation check counted ignored tool caches, so eslint --cache or incremental tsc would have failed every run. It now covers only non-ignored work. Ignored files are hashed after the gates for the freshness check.
• Any file passed as --receipt was exempt from scope; now only markdown files are.
• Nested package.json files now count as rule files.

The README was also reworked (19a6e6b): it says plainly which parts are enforced and which are guidance, and it adds a quick start that was run end to end on a fresh repo.

What this does not do

• It can't tell you the software is correct. It checks that the rules above held.
• approved_by is a string in the contract. Actual enforcement of human approval needs CODEOWNERS and required reviews on the rule files.
• Without --rerun, local evidence can still be edited by an agent with shell access. CI should always use --rerun.
• Files under dependency caches (node_modules, dist, ...) are not hashed. A fresh install in CI is the defense there.
• The one-line installers still fetch from main. Pinning them to a release tag is the next supply-chain step.
• A second round of OpenAI testing was blocked by its cybersecurity filter, so the final code has had one Gemini review and my own tests, not a second Codex round.

Before merging

• Review the CI change (.github/workflows/ci.yml: SHA pins plus the new test step)
• Decide on a version bump; README and manifest still say 2.1.0
• Consider CODEOWNERS for .buildloop.yml, package.json, and .github/workflows/

🤖 Generated with Claude Code