feat(gov): prove every rule reaches the runtime that obeys it - #25
Merged
Conversation
M06_001 opens on four delivery gaps a cross-runtime instruction audit found: pack files that drift in the checkout that reads them, a DOC READ recorder that belongs to no pack, a Rust error standard no machine decides, and a verification cadence the prose calls "always" where it means "before the boundary". Test Baseline: unit=148 integration=0 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
install skips writing a managed file into the checkout that owns its source, so the copy living beside the source is maintained by hand and no install ever corrects it. Both halves of that had already happened: dispatch/write_rust.md carried 1,776 bytes against a 9,971-byte source — the entire Rust error discipline missing from the copy this repository's agents, its dispatch-coverage audit, and its rule ledger read — and dispatch/write_go.md 1,204 against 1,751. orly verify gains packs.sources.current, which asks managedContent (now exported, so one definition decides what bytes belong at a target) what each pack would ship and compares it to what is there. A target the checkout does not carry stays silent: a consumer-only path is not drift. Both drifted copies are regenerated from their sources. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The operating model tells every agent to record a triggered read with `bash audits/doc-read.sh log <path>`, and pre-commit compares that record against the staged diff. The rule shipped through universal.authoring; the script belonged to no pack at all. In a consumer the command the rule names was simply absent — a runtime read hook invoked nothing, the check had nothing to compare, and the proof-line stayed a self-report about itself. universal.authoring now carries the recorder and the library it sources. The test installs into a fresh repository and RUNS the pair there, since a missing library fails at run time and would pass any copied-file check. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Rust had no deterministic façade at all, so RULE ERR-RS — written, cited by the doc-read map, called mandatory by the operating model — was enforced by review alone. dispatch/write_rust.sh runs audits/rust-error.sh over staged *.rs and blocks the two clauses a machine can decide from the text: a map_err whose closure stringifies its OWN binding (to_string()/format! on the cause destroys the source() chain, while the same call on a path is correct — the binding is what makes it decidable), and a crate declaring a public Error type with no `pub type Result<...>` beside it. The other two clauses stay judgment on purpose. Whether source() returns its own kind needs the Display body read beside it, and whether a context addition is warranted is a call-site question; a gate that guesses at those is one people learn to ignore. One scanner emits both signals so the #[cfg(test)] carve-out is implemented once and the halves cannot disagree about what a test module is. Carve-outs match audits/logging.sh exactly, so one Rust scope answers to both leaves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The cadence prose said the declared verify.* set runs "always", which reads as every done-claim. The same suites then ran again in the hooks and a third time inside a bare `orly gate` — the comprehensive tier's wall-clock spent answering a question the scoped lane already answered. VERIFY now states two cadences: a Section lane (CONFORM plus the lane covering the surface the Section touched, reported as a Section claim) and the milestone boundary, where the full declared set runs once. CHORE(close) names `orly gate pr` — the bare chain pays the fast tier twice on the way to the same verdict. Three claims that were false are corrected: dispatch/verify.md and docs/VERIFY_TIERS.md still said no lane needs live Postgres or Redis while the config declares an integration command; the skill chain called the integration skill "never skipped" while dispatch/verify.md called it conditional; and SOUL.md credited `orly gate verify` with enforcing the banned-word list, which docs.language reports without failing by design. Each claim is now pinned by a render test, since none of it is visible to a determinism check. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
persona.indy is opt-in and five engineering clauses were hiding inside it, so a repository dropping the voice silently dropped symlink-edit routing, dotfiles backup, vault naming, sibling-repo precedent, and the read-the-reference mandate along with it — 9,575 rendered bytes charged to every session, of which only some was voice. The clauses move to the always-on blocks. The vault line folds into the credentials rule it half-duplicated. Only the machine-local checkout paths stay behind, which is what a pack named for a person is for. A render test asserts the split so it cannot quietly reverse. Four packaged skill descriptions shrink under a 320-character bound. Every host renders skill metadata into a fixed budget and truncates the set when it overflows — Codex says so out loud — and a truncated description is a skill the model can no longer route to. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every Dimension DONE, rubric graded against real output, spec moved to done/. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
The hook orly generates for pre-commit runs `orly gate work`, which judged branch shape and tree cleanliness. A commit hook's tree is dirty by construction — that is what is being committed — and the operating model itself says a new spec is committed on the default branch. So the hook failed twice over on a normal commit, a fresh `orly init` could not commit its own installation, and the only way past was `--no-verify`, which Hard Safety forbids outright. Each gate now asks only what it can answer honestly at its own cadence: work the declared conform command, no git state (pre-commit) verify spec dimensions, docs language, fast verify.* (pre-push) pr branch, tree, pushed, spec criteria, slow (CHORE(close)) conform becomes its own tier rather than riding in verify, so the chain runs it once. `orly gate pr` skips the fast tier because its own git.pushed criterion proves HEAD is the commit pre-push already graded. Two consequences followed. CONFORM now runs at every commit, so this repository's own declared conform — `make audit`, which contains the `bun test src` its verify.unit also declares — splits into `make conform` (four deterministic rule gates, about two seconds) with `make audit` keeping the full chain for pre-push and CI. And the dispatch stops judging files the engine wrote: staging a fresh install put orly's own 444-line audits/logging.sh in front of orly's own 350-line cap, a finding no consumer can act on because the next update overwrites the file. dispatch/lifecycle.md gains a stage-by-stage table of what runs where. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Close the delivery gaps between a written rule and the agent bound by it, and stop charging every session for context that enforces nothing. Four gaps came out of a read-only audit of the instruction and skill surface across Claude Code, Codex, and OpenCode in a consuming repository; each one let a written rule govern nothing.
Spec:
docs/v1/done/M06_001_P1_GOV_CLI_HARNESS_ROBUSTNESS.mdWhat was broken
dispatch/write_rust.mdcarried 1,776 bytes against a 9,971-byte pack source — the entire Rust error discipline missing from the copy this repository's agents,evals/dispatch/coverage.sh, and the rule ledger read.dispatch/write_go.md: 1,204 against 1,751.audits/doc-read.shappeared in nomanaged_filesrow, so in a consumer the command the operating model names was absent: a runtime read hook invoked nothing and pre-commit had nothing to compare.dispatch/write_rust.sh, no leaf.verify.*set runs "always"; the same suites then ran in the hooks and again inside a bareorly gate.Three prose claims were also simply false:
dispatch/verify.mdanddocs/VERIFY_TIERS.mdsaid no lane needs live Postgres or Redis while the config declares an integration command; the skill chain called the integration skill "never skipped" whiledispatch/verify.mdcalled it conditional; andSOUL.mdcreditedorly gate verifywith enforcing the banned-word list, whichdocs.languagereports without failing by design (src/criteria.ts:150-160).What changed
§1 —
orly verifygainspacks.sources.current.planFilesdeliberately skips writing a managed file into the checkout that owns its source, so that copy is maintained by hand and no install corrects it. The new check asksmanagedContent(now exported, so one definition decides what bytes belong at a target) what each pack would ship and compares. A target the checkout does not carry stays silent — a consumer-only path is not drift. Both drifted copies are regenerated.§2 —
universal.authoringships the recorder and the library it sources. The test installs into a fresh repository and runs the pair there, because a missing library fails at run time and would pass any copied-file check.§3 — Rust gets a deterministic façade.
dispatch/write_rust.shrunsaudits/rust-error.shover staged*.rsand blocks the two clauses a machine can decide from the text: amap_errwhose closure stringifies its own binding, and a crate declaring a publicErrorwith nopub type Result<…>beside it. The binding is what makes the first decidable —to_string()on the cause destroys thesource()chain, the same call on a path is correct.source()shape and whether a context addition is warranted stay judgment; a gate that guesses at those is one people learn to ignore.§4 — Two cadences, one boundary. A Section lane (CONFORM plus the lane covering the surface touched, reported as a Section claim) and the milestone boundary, where the full declared set runs once. CHORE(close) names
orly gate pr; the bare chain pays the fast tier twice for the same verdict.§5 — persona costs voice, not enforcement. Five engineering clauses were hiding inside the opt-in persona pack, so dropping the voice silently dropped symlink-edit routing, dotfiles backup, vault naming, sibling-repo precedent, and the read-the-reference mandate. They move to the always-on blocks; only the machine-local checkout paths stay behind. Four packaged skill descriptions shrink under a 320-character bound — every host renders skill metadata into a fixed budget and truncates when it overflows.
Effect on a consuming repository
Measured by rendering agentsfleet's pack selection at this commit:
Dropping
persona.indyis now a lossless 8,955-byte cut. It was not before this branch.Verification
make audit(declaredconform)bun test src(declaredverify.unit)make install-evals(packed-tarball distribution)bash evals/dispatch/run.shbash evals/dispatch/coverage.shgitleaks detectorly gate prSession notes
dispatch/write_rust.mdafter last reading it, so the recorded read no longer matched the content and pre-commit refused. Working as designed, and the first evidence that §2's recorder is load-bearing rather than decorative./orly-write-integration-test: N/A — no module boundary with real input/output changed. The distribution lane (make install-evals) is this repository's integration tier and ran green.🤖 Generated with Claude Code
Added after review of the lifecycle wiring (§6–§8)
Indy asked for the fixes to land in the engine rather than per-project, since orly is what he builds every project with. Three more defects surfaced there.
§6 — The generated hooks refused the commits they exist to protect.
orly gate workjudged branch shape and tree cleanliness, and the generated pre-commit hook runs exactly that. A commit hook's tree is dirty by construction — that is what is being committed — and the operating model itself prescribes committing a new spec on the default branch. A freshorly initcould not commit its own installation; the only way past was--no-verify, which Hard Safety forbids outright. Never seen inagentsfleetbecause it hand-wrote its hooks with--accept-dirty.Each gate now asks only what it can answer honestly at its own cadence:
conformis its own tier rather than riding inverify, so the chain runs it once.orly gate prskips the fast tier because its owngit.pushedcriterion proves HEAD is the commit pre-push already graded.§7 — CONFORM has to cost seconds now that it runs at every commit. This repository's own declared
conformwasmake audit, which contains thebun test srcitsverify.unitalso declares — soorly gateran the unit suite twice. Split intomake conform(four deterministic rule gates, ~2s), withmake auditunchanged for pre-push and CI.§6.4 — The dispatch stopped judging files the engine wrote. Staging a fresh install put orly's own 444-line
audits/logging.shin front of orly's own 350-line cap — a finding no consumer can act on, because the nextorly updateoverwrites the file.dispatch_managed_pathsfilters the install record out of the staged set.§8 —
dispatch/lifecycle.mdgains a stage-by-stage table naming the command each stage runs and the gate that fires it.Verification (re-run)
make conform(declaredconform)make audit(full invariance chain)bun test src(declaredverify.unit)make install-evalsorly gate prSession notes (continued)
conformstays the repository's call.SOUL.mdsays it should: adding a rule means making room. Rationale tails were cut, triggers kept./orly-write-integration-test: N/A — no module boundary with real input/output changed.make install-evalsis this repository's integration tier and ran green.