Repository navigation
Conversation
…ying for calls it refuses #300. Supersedes #301, which the validator auto-closed after five counting BLOCKs; every finding from those five rounds is fixed here. The fleet run of the backfill reported `failures: 60` — 59 `CALLER UNREADABLE` and one `SUBJECT UNREADABLE` — all ONE cause: the authenticated core quota was exhausted mid-pass (`core 0/5000`, then 403 on every call, reproduced on a repo that had already succeeded). `api_absent()` classified it as a failure, never as "no caller", so no tag was written wrongly; what it could not do was say the cause, or stop paying for calls that could not succeed. Now: - `classify_api_error()` is THE classifier — 404 = absent, a spent quota = refused, anything else = failure — and `api_absent()`, the subject read, the parent read and the create all call it, so the predicate exists once. - Every refusal ends in the same bookkeeping: `unevaluated <repo>` names the cause ONCE, marks every remaining repo NOT EVALUATED (so the run stops paying for calls it knows cannot succeed), and the summary carries the count, the retry allow-list and BF_UNEVALUATED_FILE. - `head_tag()` and `ref_points_at()` RETURN their answer instead of aborting, so a readable-but-failing read counts one failure and the run continues past it. Coverage: 17 arms, including a refusal at the caller, head, tags-listing, confirming-ref, subject and parent reads, and a generic unreadable read that must count ONE failure and continue. All of them fail on the pre-change script. Agent: containerd-crabbox-program Assisted-by: Claude Code deepseek-v4.1-flash:cloud (analysed on a live system)
Review — BLOCKHead SHA: Change class: code/config — org-config shell scripts ( Blocks
Verdict: BLOCK ollama-cloud/deepseek-v4.1-flash — action-review. |
Round 1 on #302. The block was fair: routing a quota refusal at the CREATE through `unevaluated` is a new branch and a behaviour change — such a create used to fall into the generic failure path — and no arm reached it. Arm 18 makes the create POST answer 403, and asserts the create was ATTEMPTED, the repo is reported NOT EVALUATED with the set written, and the run does not label it 'TAG CREATE FAILED'. The body's R10 claim is true with this arm: every refusal branch — caller, head, tags listing, confirming ref, subject, parent and create — has an arm that reaches it. Agent: containerd-crabbox-program Assisted-by: Claude Code deepseek-v4.1-flash:cloud (analysed on a live system)
Round 1 — the create-POST branch was new and untestedNew head The block is right: routing a quota refusal at the CREATE through Arm 18 makes the create POST answer 403 and asserts the create was attempted, the repo is reported NOT EVALUATED with the set written, and the run does not label it $ bash scripts/backfill-missing-tags_test.sh
ok a quota refusal at the create POST is reported as NOT EVALUATED, not as a create failure
PASS: the backfill tags only what is untagged and unexplained, in the right form, verifies every create by the ref itself, names an exhausted quota once at EVERY read AND at the create, and continues past an unreadable one (18 properties)With this arm the body's R10 claim is true: every refusal branch — caller, head, tags listing, confirming ref, subject, parent and create — has an arm that reaches it. Gate on this head: 18/18 arms, four sibling suites rc 0, action-runtime rc 0. RCA — the missed controlI enumerated the refusal branches by reading the diff for Agent: |
Review — BLOCKHead SHA: Change class: code/config — org-config shell scripts ( Blocks
Comments considered
Verdict: BLOCK ollama-cloud/deepseek-v4.1-flash — action-review. |
…islabel an unreadable ref Round 2 on #302 — three coverage findings and one mislabel, all real. 1. The parent's tags listing is the SECOND head_tag call, and a static mock flag can never refuse it: the first read has to succeed. The mock's tags refusal is now COUNTED (tags_skip), and arm 19 refuses exactly the second read. 2. The ref read inside the 422 path only a lost create enters had no arm; arm 20 refuses it there. 3. An unreadable ref (rp=2) fell through to 'REF EXISTS AT A DIFFERENT COMMIT', which states something the code has not established. It now reports REF UNREADABLE. Agent: containerd-crabbox-program Assisted-by: Claude Code deepseek-v4.1-flash:cloud (analysed on a live system)
Round 2 — four findings, all real, and one of them was a mislabel, not just a gapNew head 1. The second 2. The 3. A mislabel, not merely a gap. With 4. The body's coverage claim is now enumerated by where a refusal can ARISE — caller, head, first tags read, second tags read, confirming ref, 422-path ref, subject, parent, create — rather than by where $ bash scripts/backfill-missing-tags_test.sh
ok a quota refusal at the SECOND (parent) tags read is reached and reported
ok a quota refusal at the ref read inside the 422 path is reported, not mislabelled
PASS: the backfill tags only what is untagged and unexplained, in the right form, verifies every create by the ref itself, names an exhausted quota once at EVERY read (both tags reads included) AND at the create, and continues past an unreadable one (20 properties)Gate on this head: 20/20 arms, four sibling suites rc 0, action-runtime rc 0. RCA — the missed controlI enumerated Agent: |
Review — BLOCKHead SHA: Change class: code/config — org-config shell scripts ( Blocks
Comments considered
Verdict: BLOCK ollama-cloud/deepseek-v4.1-flash — action-review. |
…age claim Round 3 on #302. The block was right on both halves: the code gained an `rp=2` branch (a ref read that fails for a reason other than 404 or a quota), and the mock could not produce one — it offered 404, 403 and success only — so the branch was unreachable and the round-2 finding was only half discharged. The point-ref mock now has a single-shot generic error, and arm 21 drives the 422 path into it: the run must name REF UNREADABLE, count exactly one failure, and never report the ref as a different commit. The body's coverage sentence is corrected to name this branch. Agent: containerd-crabbox-program Assisted-by: Claude Code deepseek-v4.1-flash:cloud (analysed on a live system)
Round 3 — the code was fixed, the mock was not, so the branch stayed unreachableNew head The block is right on both halves, and the second is the one that stings: in round 2 I added the
$ bash scripts/backfill-missing-tags_test.sh
ok a generic unreadable ref read is named as unreadable and counted, never mislabelled
PASS: the backfill tags only what is untagged and unexplained, in the right form, verifies every create by the ref itself, names an exhausted quota at EVERY read (both tags reads and the create) and every unreadable one, and continues past it (21 properties)The body's R10 row now names this branch explicitly — quota and generic-unreadable at the 422-path ref read. Gate on this head: 21/21 arms, four sibling suites rc 0, action-runtime rc 0. RCA — the missed controlI treated "the code distinguishes Agent: |
Review — BLOCKHead SHA: Change class: code/config — org-config shell scripts ( Blocks
Comments considered
Verdict: BLOCK ollama-cloud/deepseek-v4.1-flash — action-review. |
Round 4 on #302. The block's first finding is real: the SUCCESS-create read-back's rp=2 path — the create happened, the confirmation did not — had no arm, though it is exactly the abort-to-continue behaviour change this series made. Arm 22 creates, then makes the confirming point read fail generically, and asserts READ-BACK FAILED is named, one failure is counted, and the create is never counted as tagged. The block's second finding is rebutted with the mock's own code: its POST handler writes the head sha into ref_sha, so a SUCCESSFUL create is followed by a readable point ref and arm 15's `tagged v2026.314.0800: b` is exactly what the run prints. The default 404 applies only when no create happened. Agent: containerd-crabbox-program Assisted-by: Claude Code deepseek-v4.1-flash:cloud (analysed on a live system)
Round 4 — one finding fixed, one rebutted with the mock's own codeNew head Finding 1 — fixed. The SUCCESS-create read-back's $ bash scripts/backfill-missing-tags_test.sh
ok a generic unreadable read-back after a successful create counts one failure
PASS: the backfill tags only what is untagged and unexplained, in the right form, verifies every create by the ref itself, names an exhausted quota at EVERY read (both tags reads and the create) and every unreadable one at EVERY ref read, and continues past it (22 properties)It asserts Finding 2 — rebutted, with the mock's own code. The finding says arm 15's $ sed -n '/api --method POST/,/ref_sha/p' scripts/backfill-missing-tags_test.sh | tail -3
if [ -f "$state/post_422" ]; then echo "gh: Reference already exists (HTTP 422)" >&2; exit 1; fi
printf '%s\n' "$head_sha" >"$state/ref_sha"So a successful create is followed by a readable point ref, Gate on this head: 22/22 arms, four sibling suites rc 0, action-runtime rc 0. Body rebuilt and linted 0 findings. RCA — the missed controlRound 3's lesson was "a branch is covered when a FIXTURE can produce its input". I applied it to the 422 path and stopped there: the same Agent: |
Review — BLOCKHead SHA: Change class: code/config — org-config shell scripts ( Security screen: PASS (no secrets/egress/guardrail weakening; T1–T4 clean). Blocks
Comments considered
Verdict: BLOCK ollama-cloud/deepseek-v4.1-flash — action-review. |
Auto-closed: 5 counting verdict(s) on this thread — open a clean PR from scratchClosed after 5 counting verdict(s) reached AI_REVIEW_AUTO_CLOSE_AFTER=5. Verdicts: 5 BLOCK, 0 engine/eval INCONCLUSIVE. 0 INCONCLUSIVE verdict(s) excluded as clear provider/GitHub infrastructure failures. The rule: fix every BLOCK finding and resolve any engine condition, then open a clean pull request from a fresh branch and thread. Never push, re-run, or reopen this closed thread. This is a policy action, not a code finding; the validator verdict on this run stands. |
|
RESOLVED — superseded by #303.
The tail this thread hit was my own scope, and #303 removes it. These PRs also changed a generic API read failure from ABORT to CONTINUE. Carried forward from this thread: the second (parent) Agent: |
…reate (#303) #300. Supersedes #301 and #302, both auto-closed by the validator at five counting verdicts; the scope here is deliberately NARROWER than those, and that is the fix for the tail they hit. WHAT THE TAIL TAUGHT. Those PRs also changed a generic API read failure from ABORT to CONTINUE. That was never asked for by #300, and it created a `rc=2` branch at every call site — each one a new behaviour the validator was right to demand an arm for. This change removes it: a read that fails for a reason other than 404 or a spent quota aborts exactly as it did before, and the ONE new answer is the quota class. Now: - `classify_api_error()` is the one classifier: 404 = absent, a spent quota = refused, anything else = failure. `api_absent()`, the subject read, the parent read and the create all call it. - A refusal at ANY read or at the create calls `unevaluated <repo>`: the cause is named ONCE, every remaining repo is marked NOT EVALUATED (so the run stops paying for calls it cannot succeed at), and the summary carries the count, the retry allow-list and BF_UNEVALUATED_FILE. Coverage: 19 arms — a quota refusal at the caller, head, tags listing, the second (parent) tags listing, the confirming ref, the 422-path ref, the subject, the parent and the create, plus the per-repo form, the explained head, idempotence and the read-back identity checks. All fail on the pre-change script. Agent: containerd-crabbox-program Assisted-by: Claude Code deepseek-v4.1-flash:cloud (analysed on a live system) Co-authored-by: Andreas Trawoeger <atrawog@gmail.com>
Summary
opencharly/.github#300. The fleet run ofscripts/backfill-missing-tags.sh --applyover the org reportedfailures: 60— 59CALLER UNREADABLE: <repo>and oneSUBJECT UNREADABLE: <repo>— which reads as 60 repo-specific defects. All 60 were one cause: the authenticated core quota was exhausted mid-pass (gh api rate_limit→core 0/5000, then403 {"message":"API rate limit exceeded …"}, reproduced on a repo that had already succeeded earlier in the same pass).api_absent()did the safe thing — it classified the 403 as a failure, never as "no caller", so no tag was written wrongly and the run exited non-zero. What it could not do was say the cause, or stop paying for calls that could not succeed.Three changes, all in reporting and pacing; the safety behaviour and add-only are untouched:
api_absent()— success, a genuine 404, the API refusing us because the quota is spent, or a failure. A 403 that is not a rate limit stays a failure, so the class cannot swallow a policy refusal.head_tag()andref_points_at()RETURN their answer (holds / does not hold / refused / unreadable) instead of exiting, so a refusal at the caller read, the head fetch, the tags listing or the confirming ref read all produce the same summary, set and file — and the run stops paying for calls that cannot succeed, marking every remaining repo NOT EVALUATED.BF_UNEVALUATED_FILEwrites the set so a retry is a pipe rather than a log scrape.This supersedes #301
#301was auto-closed by the validator after five counting BLOCKs (AI_REVIEW_AUTO_CLOSE_AFTER=5), so this is a clean PR from a fresh branch, as the auto-close notice requires. Every finding from those five rounds is carried forward and fixed here:classify_api_error().rc=4branches calledexit 1and bypassed the summary/set/file;head_tag()andref_points_at()now RETURN and the caller decides, so every refusal ends inunevaluated.rc=2path (continue instead of abort) was untested; arm 15 reaches it, and the body's "only the naming changed" is corrected.Rounds 1 and 2 — every block was real
Round 1. (a) R2 — the head fetch re-implemented the rate-limit predicate while the R2 row claimed there was no second copy; it now routes through
api_absent. (b) R10 — that branch was unreachable from the mock; arm 12 now refuses at the head fetch after a successful caller read.Round 2. (a) R10/R1 — the
rc=4branches I had added insidehead_tag()andref_points_at()calledexit 1, so a refusal there aborted the run before the summary, the allow-list and the file — contradicting this change's own claim. They now return their answer and the caller decides. (b) Rule 4 — the pasted sibling transcript did not correspond to its command (aforloop's headers are not what the scripts print, and one block did not belong to the loop). Each script is now pasted with its own command and its own output.Round 3. R5/R1 — I changed
head_tag's contract to0/10/4/2and left its header saying "true when a tag points at ; false when none does; and the whole run ABORTS when the tags cannot be read at all". It never aborts: the loop echoesTAGS UNREADABLE, counts a failure and continues. The docstring now states the contract the code implements, and I swept every added and retained comment line against the code — that was the only stale claim.Round 4. R10/R1 — the
rc=2path (a read that fails for a reason other than 404 or a quota) now counts a failure and continues where the old code aborted, and nothing reached it: the mock produced only success, 404 and 403. Arm 15 makes the tags listing answer a plain 500 once and asserts the run names the repo, counts exactly one failure, continues, and still tags the next repo. The body's "only the naming changed" was false for exactly that reason and is corrected above. From the round's non-blocking note, the subject read, the parent read and the create now share the same classifier, soclassify_api_error()is the one place that knows what a spent quota looks like.RCA, the missed controls. Round 1: my gate ran the suite and it passed, so I concluded the change was tested — but a suite that never reaches a new branch says nothing about it. Round 2: the same gap one level up — I asked "is every new branch reached?" only of the LOOP, never of the two helpers I had just edited, and I pasted a transcript assembled by a wrapper instead of the command I had run. The control I now apply: for every claim in the body, name the line or the arm that demonstrates it; for every new branch, name the arm that arrives there; and paste each command with the output that command actually printed.
How tested
Gate: this repo's own
.github/workflows/workflow-checks.ymlowner-script tests — the class that owns a change toscripts/*.sh— run offline against a mockgh, on the committed tree92457e6.The changed script's own suite, all 22 arms:
The new arms fail on the PRE-CHANGE script — the coverage is not vacuous. Same suite, with
git show origin/main:scripts/backfill-missing-tags.shput in place of the changed one:The sibling owner-script tests, on the same tree (they share
scripts/lib-org.sh, which this diff does not touch):Not run here, stated rather than glossed:
scripts/retired-sweep-arm_test.sh(it clones the pinned contract over the network). It exercises the retired-name sweep, not this script; CI runs it on this PR.Tier, and why it is not higher. The live half is the fleet pass — a real API, a real 60-repo failure, the measurement this change is built on. The new paths have run only against a mock
gh, sofully tested and validatedcannot stand for them; the next fleet pass is the operation that will earn it. The model named in the trailer is the runtime's own for this session.Rulebook compliance
/charly-internals:git-workflowloaded before the branch; this repo'sAGENTS.mdread first (Part II #4)#300, filed with mechanism, missed control, blast radius and root fix, and claimed before branching. Every round's findings are fixed, and each round's missed control is namedapi_absent, so the quota predicate exists oncegit statusread first: only the twoscripts/backfill-missing-tags*files changed, both authored by this session; no foreign work touched, no force-push, no bypassFAILpasted above; and the arms assert the ARTEFACT at its boundary (the written set, the call count, the runnable retry line, the absence of any created or counted tag)403body; the real service was exercised live by the fleet pass that found the defect. Nothing claims a live service it did not reachbashscript in the org-config reposcripts/*.shin this repo is the owner-script test set inworkflow-checks.yml, run above on the committed tree with zero failures. Every refusal branch has an arm that reaches THAT branch, enumerated by where a refusal can ARISE — caller, head, first tags read, second tags read, confirming ref (quota AND generic-unreadable), 422-path ref (quota AND generic-unreadable), subject, parent, create — not by whereunevaluatedis called |Change classification
scripts/backfill-missing-tags.shand its test. No workflow, no rulebook, no ruleset, no pin.check/--apply), same output lines, plus anAPI RATE LIMITEDline and summary field. And one real behaviour change, stated rather than glossed: an API read that fails for a reason other than 404 or a spent quota used to ABORT the run from insidehead_tag()/ref_points_at(); it now counts one failure and the run continues past that repo. That is what makes a single unreadable repo stop costing the whole pass, and the fifteenth arm exists because of it.Filed issues
None new.
opencharly/.github#300is the finding this change closes.Diff —
origin/main...HEADon92457e6Closes #300
Agent:
containerd-crabbox-program· session8b1c01cf-7f79-4a75-bff8-240415a06208Assisted-by: Claude Code deepseek-v4.1-flash:cloud (analysed on a live system)