provenance sweep sub-topic: repo-wide sweep, upstream-drift convention engagement, and the cleaned-state gate

[kyle-sexton]

The provenance build plan's Phase 8 promotes the repo-wide sweep, the combined upstream-drift
convention engagement, and the narrow CI regression gate for the cleaned state into their own
sub-topic. This issue is that promotion artifact, and it is the Brief the sub-topic runs from.

Why this is an issue rather than docs/topics/provenance-sweep/PLAN.md

The plan named a contract slice as the promotion target. That target is not merge-able:
docs/topics/<slug>/ is Contract tier, branch-only, and scripts/check-contract-slice-prune.sh
hard-fails any new path there on the way to main — the parent topic's own slice was pruned for
exactly this reason (#3456). The work-item tracker is the graduation route the topic-docs
lifecycle names for actionable follow-ups, so the Brief lands here. Nothing about the plan's
content changes; only its vessel.

Precondition, and it is a hard one

The sweep cannot apply dispositions yet, and this is the gate's own arithmetic rather than a
blocker to route around. The golden set ships at 10 cases across four groupings, so no class
reaches min_n_per_class 10 and every class is report-only at v1. fix and sweep apply
dispositions to fix-eligible findings only, and at v1 there are none.

So the sub-topic has two distinct stages, and the first does not wait on the second:

1. A report-only sweep is runnable now and is worth running: it produces the corpus-scale
   telemetry (fetch counts, cache hits, carve-out declines, not-found rates) that the convention
   engagement needs as evidence, and it harvests adjudications into golden cases.
2. A disposition-applying sweep unlocks only once the growth loop carries verbatim and
   near-verbatim past n=10 at or above the 0.95 precision bar. That is the named exit condition
   of the first growth round in provenance plugin: phases 6 to 8 (golden set and measurement, crosswalk registration, sweep) #3458.

Running stage 1 and feeding its adjudications back is what makes stage 2 reachable. Reversing
them is not an option.

Measured baseline entering the sweep

From the Phase 6 measurement (recorded in plugins/provenance/CHANGELOG.md): over 1,347 tracked
markdown files after carve-outs, 525 stamp candidates, 482 parsed, 43 declined, 0 expired at the
180-day default. Golden set: 10 cases, overall 8 tp / 0 fp / 0 fn / 2 tn, every class report-only.

The adversarial rotation probe (design thread T15) returned a real limit worth carrying into the
sweep: at roughly one substitution every four words, containment goes to 0.0 with no matched
spans against a source that was fetched and identity-checked. Word-shingling is evadable by an
author who intends to evade it, and no value of min_containment above zero recovers a passage
with zero matching shingles. The sweep should expect a floor on what the deterministic half can
confirm, and should not read a clean fingerprint as proof of originality.

Execution contract (inherited from the parent Brief)

• One tracked file at a time. Apply the verdict, verify (semantic diff plus pointer liveness),
  close.
• A file is closed when every finding in it carries a disposition or an explicit neutral
  outcome — never when the interesting ones are done.
• Record each closure in the sweep ledger so an interrupted sweep resumes without re-deciding
  closed files, and so the closure count is a fact rather than a memory.
• Carve-outs stay categorical. If the sweep starts accumulating per-instance suppressions, that is
  evidence a carve-out is drawn wrongly, and the fix is to redraw it in rubric.md — not to keep
  a suppression list. This matters beyond tidiness: see the convention engagement below.

The two additions the parent plan requires this Brief to carry

1. The live no-breadcrumb (not-found) probe, as a first-run validation item. That disposition
   has never been exercised live — every spike source was supplied by hand. The sweep's first
   genuine no-breadcrumb candidate validates that the searched-surfaces listing names every
   surface actually checked, and that budget exhaustion reads as a first-class neutral outcome
   rather than as an acquittal.
2. Sweep resume semantics. The fetch ceiling and the response cache are scoped per SWEEP, not
   per invocation, and both are recorded in the closure ledger. A resumed sweep therefore neither
   resets its spend nor silently reuses a stale cache. The ledger is checkout-local; state that
   explicitly, because a sweep resumed in a different checkout is a new sweep.

The convention engagement (fires at sweep completion only)

Drafted in full at docs/specs/provenance-convention-engagement.md. It executes as one entry
in docs/conventions/upstream-drift/CHANGELOG.md, carrying three parts:

1. Recorded-decision re-derivation. chore(conventions): plugins restating upstream harness facts are not adopters of upstream-drift, so stale claims never surface #2273 deferred an adoption gate with its own stated
   trigger: "a detector is demonstrated that separates an upstream restatement from an in-repo one
   without a suppression list." Sweep completion either is that demonstration or it is not, and
   the engagement says which from evidence — the closure ledger, the measured precision/recall at
   the stated minimum n, and the carve-out record showing categorical exclusions only. If the
   sweep accumulated per-instance suppressions, the bar is not cleared, the trigger has not fired,
   and the engagement records exactly that with the count.
   The honest expected outcome, stated now so the engagement cannot overclaim later: the detector
   is LLM-led with deterministic verification, so the enforceability row moves at most from
   reasoning-only to detect-then-judge, and a deterministic suppression-free CI gate in the
   *-gate pattern remains unavailable.
2. Enforceability table update, version bump conditional. Major bump ONLY if an enforceability
   verdict changes; otherwise a minor entry with the as-of date refreshed and the outcome stated,
   drift or no drift.
3. The trigger-less-stamp table-row flip. Partially pre-consumed: the check's code shipped in
   Phase 3 and its crosswalk row in Phase 7 (feat(provenance): documentation-provenance audit plugin (phases 1 to 5) #3456 and the Phase 7 work). What remains here is
   exactly the convention's own row moving from "named but not built" to "built, off-by-default,
   repo-override enabled", plus the build-trigger bookkeeping. The registry row's early presence
   is scheduled, not drifted.

Also recorded at the same firing: the hash-store designed-issue trigger evaluated against the
sweep's fetch telemetry; the dead-pointer round-trip wiring (the weekly link-check lane routing
to the demotion path in reference/dispositions.md); and the adopters table, where rows are
added only for surfaces that actually conform, with #2297 getting its closure evidence if the
repository ends the sweep conforming.

What the engagement never does: declare the migration playbook's plugin-eval deferral trigger
fired; rewrite prior changelog entries or dated records; or convert a judgment verdict into a
deterministic claim — report-only tiers stay report-only whatever the sweep measured.

Checklist

• Grow the golden set past min_n_per_class for verbatim and near-verbatim (provenance plugin: phases 6 to 8 (golden set and measurement, crosswalk registration, sweep) #3458)
• Run the report-only sweep; record closure ledger, telemetry, and adjudications
• Validate the live no-breadcrumb not-found probe on its first genuine case
• Confirm sweep resume semantics (per-sweep ceiling and cache, checkout-local ledger)
• Disposition-applying sweep, once a class is fix-eligible
• Execute the convention engagement as one upstream-drift/CHANGELOG.md entry
• Evaluate the hash-store designed-issue trigger against the sweep's fetch telemetry
• Wire the dead-pointer round-trip to the weekly link-check lane
• Narrow CI regression gate for the cleaned state

Related

• provenance plugin: phases 6 to 8 (golden set and measurement, crosswalk registration, sweep) #3458 — phases 6 and 7, and the golden-set growth this sub-topic depends on
• feat(provenance): documentation-provenance audit plugin (phases 1 to 5) #3456 — phases 1 to 5, where the plugin, its scripts, and the trigger-less check shipped
• Refs chore(conventions): plugins restating upstream harness facts are not adopters of upstream-drift, so stale claims never surface #2273 — the upstream-drift recorded decision this engagement re-derives
• Refs chore(plugins): 12 unstamped upstream harness claims across 11 plugins — upstream-drift adoption backlog #2297 — the fleet's open-carriers issue, which gets its closure evidence here
• docs/specs/provenance-convention-engagement.md — the engagement, drafted in full
• docs/specs/provenance-design-threads.md — T15, the adversarial probe whose limit is recorded above

[kyle-sexton]

Stage 1 report-only sweep: first run, recorded

Ran the report-only sweep across the whole corpus at d7e391da, split into two judgment legs (docs/, plugins/) plus a mechanical telemetry leg, each with an independent fresh-context verifier. No dispositions were applied and no tracked file was edited by any sweep leg: at v1 no class reaches min_n_per_class, so applying one would be a correctness violation rather than a shortcut.

Checklist movement

• Run the report-only sweep; record closure ledger, telemetry, and adjudications — below.
• Validate the live no-breadcrumb not-found probe on its first genuine case — validated, with a gap found.
• Confirm sweep resume semantics — measured, and the answer is that they are not implemented.
• Grow the golden set past min_n_per_class — candidates harvested below; not yet converted.
• Disposition-applying sweep — still blocked, and now further away (see rubric v3).
• Convention engagement, hash-store trigger, dead-pointer wiring, cleaned-state gate.

Telemetry

metric	value at d7e391da, --as-of 2026-08-28
tracked markdown after carve-outs	1,352 (1,395 considered, 43 declined at path level)
stamp candidates / parsed / declined	535 / 491 / 44 (20 month-name, 24 bare-year)
expired at 180-day default	0
oldest parsed stamp	2026-04-08, plugins/work-items/skills/track/actions/add.md:104
findings at a 60-day window	9
fetches attempted / succeeded / failed	9 / 7 / 2 (both HTTP 404)
rate-limit failures (429)	0
fingerprint comparisons	784
categorical carve-outs in effect	3
per-instance suppressions accumulated	0

Two of these figures are date-relative and expire, which is why the as-of date is pinned: the 180-day zero holds only until 2026-10-05 on the current oldest stamp, and the 60-day count moves daily.

The adoption-bar evidence this issue asks for

This issue states that if the sweep accumulates per-instance suppressions, the #2273 adoption bar is not cleared. It accumulated none. Three categorical carve-outs, no per-instance suppression list anywhere, and two skill surfaces forbid them by design. That is the first evidence on record pointing at the trigger, and it is recorded here rather than in the convention, because the engagement fires at sweep completion and this is one run.

The excluded_paths placement holds as designed: the fixtures exclusion lives in the config layer, not unconditionally in list-corpus.sh, so the eval harness lifts it while a normal run declines it and says so. The #3041 precedent is live.

Resume semantics are not implemented

All three claims in this Brief measured as NOT IMPLEMENTED:

claim	verdict	evidence
fetch ceiling scoped per sweep	not implemented	reference/source-fetch.md:110 scopes corpus_fetch_ceiling to "the run"
response cache scoped per sweep	not implemented	source-fetch.md:113-115, cache "lives in the run's memory slice"
both recorded in the closure ledger	not implemented	reference/dispositions.md:152-154, SKILL.md:163-165 record closures and guard outcomes only

Checkout-local holds by design (docs/specs/provenance-plugin-topology.md:60 places the ledger under .work/, which .gitignore:30 ignores) but no script enforces it — .work/ is referenced by no script at all. The Brief describes intended semantics; whether they were built is what this checklist item measured, and they were not. Recording rather than building, since this run was scoped to measure.

Findings

docs/ (148 files) does not lexically copy the upstream it cites. Exhaustive 296-comparison sweep, corpus-wide max containment 0.011. The five docs/upstream/*.md records tested against a real clone of mattpocock/skills scored 0.001–0.009 with zero firings, and their self-declared near-verbatim adoption cites upstream to the correct line.

plugins/ produced six findings that stand, four of them in one file:

location	class	tier
plugins/context7/skills/lookup/context/lookup.md:40-41	verbatim	fingerprint-confirmed
same file, three further spans	near-verbatim	see caveat below
plugins/firecrawl/skills/firecrawl/context/commands.md:36	paraphrase	llm-suspected
plugins/playwright/skills/playwright/reference/test-generation.md:80	near-verbatim	source-fetched-similar

lookup.md reproduces a five-step procedure from the live upstream in order, containment 0.217 with a 26-word span, and carries zero attribution and zero external URLs across 77 non-empty lines. The plugin does attribute upstash — in a sibling file — and C3 grades what is on the page.

Tier caveat, found by the verifier: two of those four were tiered on the file-level separation flag, but their per-finding longest spans are 14 words each, below the 15 floor, and file containment 0.217 is below 0.3. Neither arm fires for them individually. They are genuine short copies below the deliberate floor — the floor working as designed — and must not be recorded as fingerprint-confirmed. Related process deviation: the cited ranges match no module span, so nomination ranges were carried through unrefined against SKILL.md step 7.

A materialized drift, outside the tier table's domain: docs/NATIVE-SURFACES.md:98 records a native description absent from both shipped claude binaries in this container. The row's own recheck trigger has already fired. This is not drift risk; it is drift. check-stamps.sh structurally cannot see it.

T15 reproduced live, twice

The rotation limit is no longer only a synthetic probe. firecrawl/.../commands.md:36 restates (default 5, max 100) from the Firecrawl CLI docs at containment 0.000 with zero matched spans, against a source that was fetched and identity-checked. The verifier control-tested the module rather than trusting the zero (local-vs-itself returns 0.927), so this is a genuine detector miss on a table-row-to-prose restatement, not an operational failure. llm-suspected is the forced tier: no lexical evidence is possible.

Neither leg proposed moving the 0.3/15 constants, and neither read a clean fingerprint as proof of originality.

The no-breadcrumb probe validated, and found a gap

Resolved at rung 4 (local binaries) after a zero-URL file. Budget exhaustion was exercised on a second candidate (404 then 403) and held as a neutral outcome, not an acquittal — correctly, because a further rung then resolved it, and stopping early would have lost the materialized drift above as a false negative.

The gap: the searched-surfaces listing has no mechanical floor. emit-findings.sh has no field for it, no budget block, no schema. Three prose requirements, zero enforcement — a listing omitting a rung is indistinguishable downstream from a complete one.

Also unplanned and worth a decision before stage 2: the stale string's authoritative home is docs/native-surfaces/records.json:108, outside the markdown corpus and generated, and its rendering says never to hand-edit it. A fix pass would edit a generated file.

Detector defects found by running it on itself

• False negative, being fixed: docs/upstream/aihero-course.md:127 declines as "bare year" while ending in a conforming ISO date — a real stamp silently exempted from expiry.
• False positive, found independently by both legs: the ordinary English word "may" read as a month name (docs/conventions/standards/README.md:134, plugins/claude-config/skills/audit/SKILL.md:171). Also SC2034 and "cache_read_input_tokens": 2000 read as bare years.
• The declined block's text field reports the keyword line rather than the date line, so whoever adjudicates the 44 declines sees misleading context.

Adjudication candidates harvested

Rejected findings are the high-value output, because a case converted from a rejected finding is one nobody authored to pass. Harvested, not yet converted: a byte-identical 16-word list of eight product-defined identifiers that fires the separation rule but fails C2 (no other way to write it); a conforming record whose trigger has fired, a shape no golden case has; carve-out 5's reach over RESEARCH-*.md; and a 0.589-containment/142-word case that turned entirely on grading scope.

That last one produced a rubric change: C3 and C4 are graded at different scopes and v2 never said so. Two readers agreed on the verdict and disagreed on which scope produced it. Rubric v3 states it — C3 outward across the file, C4 on the passage — and takes the invalidation: the v2 golden-set measurement does not carry forward, so the set must be re-scored before any precision figure is cited. That pushes fix-eligibility further out, not closer.

Honest limits on this run

• Files were enumerated, not model-read. Model-led nomination covered roughly 35 files in plugins/; the rest got path- and stamp-level treatment only.
• nomination_passes ran once, not twice — nominations were not unioned.
• judge_samples was not met: judgments are single-context, not three blind panels. No verdict here carries panel unanimity. Both judgment legs reported this themselves rather than being caught at it.
• Recall is therefore lower than designed. Moot for eligibility, since no class reaches minimum n either way, but it means the absence of findings in a tree is weaker evidence than the presence of one.
• One leg's verifier caught a wrong-artifact identity error (two claude binaries in this container, 2.1.246 and 2.1.250; the finding survived at both, but the pin was wrong). Another leg could not obtain a verifier at all and said so rather than claiming one.

Recommended before stage 2

1. Decide the generated-file question — a fix pass must not edit records.json through its rendering.
2. Give the searched-surfaces listing a mechanical floor, or stop claiming it as a validation.
3. Re-score the golden set against rubric v3.
4. Add a tier row for a vendored-snapshot source: strong provenance, weak currency, and every current row gates on "fetched" or "no source".

---

Generated by Claude Code