Skip to content

SameWrite 1.3.1 — fail closed on partial history evidence - #7

Open
ipeterpetrus wants to merge 5 commits into
mainfrom
fix/1.3.1-partial-evidence-fail-closed
Open

ipeterpetrus wants to merge 5 commits into
mainfrom
fix/1.3.1-partial-evidence-fail-closed

Conversation

@ipeterpetrus

@ipeterpetrus ipeterpetrus commented Sep 15, 2026

Copy link
Copy Markdown
Owner

Patch release candidate for SW-1303-PARTIAL-EVIDENCE. Not 1.4. v1.3.0 is not moved and its
released runtime is unaffected.

This PR has been through two full independent adversarial reviews. The first one blocked it.
That history is kept below rather than tidied away: the point of a release gate is that it sometimes
catches things.

1. The original defect, before and after

                                        status              candidate files   exit
before  PARTIAL history, no accept      CANDIDATE                 1             0
before  PARTIAL history + COMPLETE live CANDIDATE                 2             0
after   PARTIAL history, no accept      PARTIAL_EVIDENCE          0             0
after   PARTIAL history + COMPLETE live PARTIAL_EVIDENCE          0             0

tools/optimize.py read evidence quality from the live sweep only, and the trend finding had no
quality gate at all. A history built from bounded sweeps — carry.py --max-files N, the normal way
to keep a large profile affordable — was indistinguishable from one swept in full.

The rule now lives in one place: a finding may be promoted only when the worst quality among the
evidence actually eligible to support it is COMPLETE, or is PARTIAL and the caller passed
--accept-partial. Worst, not majority. Eligible, not everything.

2. The first review blocked this branch

Head bd7ada5 was reviewed on the complete diff and returned 19 hypotheses: 5 HIGH, 3 MEDIUM.
Every one was reproduced locally before being accepted — none taken on the reviewer's assertion.

what it was origin
H1 a record claimed COMPLETE while its own counters recorded 5 unreadable files, 2 oversize lines and a file cap, and was promoted; so was one with sessions=turns=carry_bytes=0 pre-existing
H2 a torn JSON line was skipped silently, so a sweep that lost evidence still reported COMPLETE — no history needed to defeat the new gate pre-existing
H3 a candidate from accepted-PARTIAL evidence was written with effective_evidence_quality: COMPLETE introduced here
H4 HOST_BEHAVIOR_SHIFT reported status 30 and strict exit 40 while writing a candidate file invariant introduced here, broken on an uncovered path
H5 the pre-1.3.1 artifact check was a substring search: prose satisfied it, an unreadable file counted as current, invalid UTF-8 crashed the run introduced here
M1 candidates_existing turned from ids into prose while the schema version stayed 1 introduced here
M2 the newest INVALID record became the comparability anchor, then vanished, stranding nine complete records pre-existing
M3 bounded carry took newest-by-mtime, bounded listing took discovery order pre-existing

3. What closed them

  • H1derive_quality() reconstructs the producer's rule from carry.accumulate(), the one
    place that knows how a sweep went, and quality_of() returns the worst of what a record claims
    and what it can show. Negative, non-numeric or boolean counters make it INVALID; sessions==0
    is INVALID or EMPTY on the producer's own terms; a schema-2 record showing no counters cannot
    claim completeness and is UNKNOWN.
  • H2scan_full() counts the lines json.loads rejects, accumulate() aggregates them and
    degrades the sweep to PARTIAL, and the count reaches the history record as malformed_lines.
  • H3 — provenance is attached where a finding is created, through sampled() and
    from_ledger(). The positional sampled_count stamping is gone; no finding infers its source
    from where it landed in a list.
  • H4 — a population the analyser has declared is not one world joins the single promotion gate.
    A status that refuses promotion now leaves nothing on disk, on every path.
  • H5artifact_metadata() parses the header block structurally, bounded to 64 KiB and 200
    lines, returning PROVEN_CURRENT, PRE_1_3_1_REVIEW_REQUIRED or UNREADABLE_OR_UNVERIFIABLE.
    OSError and UnicodeDecodeError both fail closed. Nothing is rewritten or deleted.
  • M1candidates_existing is bare ids again; the review states moved to
    candidates_existing_review_required and candidates_existing_unverifiable.
  • M2comparable() decides eligibility first and anchors only on eligible records, using the
    derived quality rather than the claimed one.
  • M3carry.bounded_paths() is the single definition of a bounded sample, used by both the
    carry sweep and the listing scan.

4. Coverage

Two frozen matrices, both written with their expected outcomes before implementation: the
original A–O partial-evidence matrix, and a second one for the five blockers. 29 mutants, each
RED when the fix it guards is reversed and GREEN on the real implementation. A harness self-check
that plants three corruptions in the regression harness and requires the harness itself to fail.

645 assertions across fifteen suites          29/29 mutants
81/81 mandatory adversarial attacks           harness self-check 8/8
privacy: 5 canary classes x 6 sinks, 0 leaks  AI-VOS readiness 27/27
original SW-1303 arms: PARTIAL_EVIDENCE, 0 candidate files
AI-VOS path: 6 bounded sweeps -> PARTIAL_EVIDENCE, 0 files; with acceptance -> analysable

Positive controls are part of the gate, because a fail-closed patch can look excellent by refusing
everything: a clean COMPLETE population still promotes, an accepted PARTIAL population still
promotes, and unrelated invalid evidence still does not poison a valid run.

5. Test debts this created, fixed rather than papered over

Four fixtures claimed COMPLETE without the acquisition counters a real sweep always writes. Each
now states them, with an added assertion locking the new rule so the fixture edit hides nothing.
Three mutation patterns moved with the code and were re-pointed. Two mutants were masked by a fix
firing earlier than the branch they guard, and their oracles now isolate that branch.
emit_candidates() gained two result classes, so its callers were updated — including one in
experiments/ that CI caught and my local sweep had missed, which is why that sweep now runs every
step CI runs, by exit code.

6. Scope

CANONICAL_SKILL_DELTA=0, DEFAULT_MODEL_CONTEXT_DELTA=0 — no file under skills/ or hooks/ is
touched. carry.history()'s append protocol is unchanged and the concurrency suites still pass.
DEFECT-ENV-EMPTY-ANSWER is out of scope: it was attributed to the machine's global agent
instructions, not to SameWrite. The deferred LOW findings from the earlier acceptance review are
untouched except where legacy quality handling was inseparable from correct partial-evidence
semantics.

TOTAL_SAVINGS=NOT_PROVEN and WORLD_BEST_CLAIM=NOT_TESTED are unchanged.

🤖 Generated with Claude Code

https://claude.ai/code/session_013pnJ85a3DYFU2ZW7CqQKCv

ipeterpetrus and others added 5 commits September 15, 2026 10:30
…ccepted

SW-1303-PARTIAL-EVIDENCE. optimize.py read evidence quality from the live sweep only, so the
quality each history record carries never entered the decision. A history built from bounded
sweeps — carry.py --max-files N, the normal way to keep a large profile affordable — produced a
candidate exactly as if the corpus had been swept in full, and a complete live scan masked it.

Measured on the released code, and again after the fix:

  before   PARTIAL history, no --accept-partial     CANDIDATE, 1 candidate file, exit 0
  before   PARTIAL history + COMPLETE live scan     CANDIDATE, 2 candidate files, exit 0
  after    both                                     PARTIAL_EVIDENCE, 0 candidate files

The module docstring already stated the opposite rule. The guard existed and worked — on one of
the two paths evidence can arrive by.

The rule, now in one place (effective_quality/worst_quality/emittable/quality_of): a finding may
be promoted only when the WORST quality among the evidence ACTUALLY ELIGIBLE to support it is
COMPLETE, or is PARTIAL and the caller accepted it. Worst, not majority — one partial observation
among nine complete ones still means part of the corpus was never swept. Eligible, not everything —
comparable() already excludes other scopes, other workload classes and non-comparable corpora, and
a partial record there must not block a run it was never part of. A gate that blocks on evidence a
finding never used is not correct, it is merely stuck.

--accept-partial authorises PARTIAL and nothing below it. Nobody can say "this bounded population
is the one I meant" about evidence that is corrupt, empty, or of unestablished provenance.

Legacy records (schema 0/1) have no evidence_quality field, so absence proves nothing about how
that sweep was taken: they now read as UNKNOWN rather than COMPLETE. They stay readable and stay
visible in reports; they simply cannot carry a promotion.

Contract: --json gains effective_evidence_quality and partial_evidence_accepted. No existing field
changed meaning, so output_schema_version stays 1. Candidates written from accepted partial
evidence record both fields in HYPOTHESIS.md. Exit codes unchanged — normal mode still exits 0,
--strict-exit still returns 40. The fix is that no candidate is produced.

Held by: a 15-case matrix (A-O) frozen before implementation, five mutants that each go RED when
the fix is reversed, a self-check that plants three corruptions in the regression harness and
requires the harness itself to fail, and the real integration path asserted end to end.

Two pre-existing fixtures in test_optimize.py were legacy-shaped and started failing on the new
rule. They now declare their provenance, and two assertions were ADDED to lock the stricter
default rather than let the fixture edit hide it.

CANONICAL_SKILL_DELTA=0, DEFAULT_MODEL_CONTEXT_DELTA=0. carry.history() untouched.
TOTAL_SAVINGS=NOT_PROVEN and WORLD_BEST_CLAIM=NOT_TESTED unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013pnJ85a3DYFU2ZW7CqQKCv
…ce gate

The hotfix diff was handed to an independent reviewer with instructions to make the patched
optimizer promote evidence that should fail closed. It found four ways. Each was reproduced
directly before being accepted, and each now has a regression case and, where it is a rule rather
than a report, a mutant that goes RED when reversed.

  schema laundering     A schema-1 record carrying evidence_quality: COMPLETE was trusted. Schema
                        0/1 predates the field, so that claim cannot have come from the sweep — it
                        comes from a hand-edited file or a migration default. Any record declaring
                        a schema older than 2 is UNKNOWN whatever it asserts.

  retry laundering      One run_id saying COMPLETE once and PARTIAL once resolved in favour of the
                        better claim, because the duplicate was dropped whole. The retry is still
                        dropped as an observation; its worse quality now survives on the record
                        that remains.

  status contradiction  A run could report PARTIAL_EVIDENCE with strict-exit 40 while a ledger
                        candidate sat on disk. Ledger findings count writes a hook actually denied,
                        which no sweep bound can make partial, so they stay promotable — but then
                        the status has to say so. Status now describes what the run produced; the
                        refused sampled evidence travels in effective_evidence_quality.

  stale artifacts       A candidate written before 1.3.1 carries no provenance and was reported as
                        ordinary "existing". It is still never rewritten or deleted, but it is now
                        named as pre-1.3.1: no evidence provenance, re-review.

One finding was NOT taken: that a partial history should not demote a concentration finding
measured purely from a complete live sweep. True of the measurement, and the gate here is
deliberately broader. For a patch closing a fail-open the conservative direction is the safe one,
--accept-partial is the documented way through, and the cost is written down rather than hidden.

Two test debts this created, fixed rather than papered over: the missing-quality mutant was masked
by the new schema gate firing first, so its oracle now uses a schema-2 record with no quality field
— the only shape that reaches that default; and the pre-existing no-proposal-spam mutant targeted
an emitter block this change rewrote, so it was re-pointed.

560 assertions across fourteen suites, 20 mutants, all green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013pnJ85a3DYFU2ZW7CqQKCv
The complete diff was reviewed independently and blocked. Five HIGH findings, each reproduced here
before being accepted — nothing taken on assertion.

H1  A record could claim COMPLETE while its own counters recorded 5 unreadable files, 2 oversize
    lines and a file cap, and still be promoted; so could one with sessions=turns=carry_bytes=0.
    derive_quality() now mirrors the producer's rule in carry.accumulate() and a record's quality is
    the worst of what it claims and what it can show. Counters that are negative, non-numeric or
    boolean make it INVALID. A schema-2 record showing no counters cannot claim completeness.

H2  A torn JSON line was skipped silently, so a sweep that lost evidence still reported COMPLETE —
    no history needed to walk through the gate this release adds. scan_full() now counts what it
    could not parse, accumulate() degrades the sweep to PARTIAL, and the count travels into the
    history record as malformed_lines.

H3  A candidate built from accepted-PARTIAL evidence was written with COMPLETE provenance, because
    provenance was stamped by a finding's POSITION in a list and listing-prune is appended after
    that boundary. Provenance is now attached where a finding is created, by sampled() and
    from_ledger(); no finding infers its source from where it landed.

H4  HOST_BEHAVIOR_SHIFT reported status 30 and strict exit 40 while writing a candidate file. A
    population declared not to be one world is not evidence you may promote from, so it joins the
    single promotion gate. A status that refuses promotion now leaves nothing on disk, on every path.

H5  The pre-1.3.1 artifact check was a substring search over the whole file: a sentence mentioning
    the field satisfied it, an unreadable file counted as current, and invalid UTF-8 escaped the
    handler and killed the run. artifact_metadata() parses the header block structurally and returns
    PROVEN_CURRENT, PRE_1_3_1_REVIEW_REQUIRED or UNREADABLE_OR_UNVERIFIABLE. Unreadable is never
    trusted; nothing is rewritten or deleted.

Three MEDIUM closed with them. candidates_existing keeps its original meaning as bare ids, with the
review states in additive fields, so a consumer treating an id as an id no longer breaks on upgrade.
comparable() picks its anchor from eligible records, so an INVALID arrival can no longer define the
workload and corpus every other record is measured against and then vanish. carry.bounded_paths() is
the single definition of a bounded sample, so the carry sweep and the listing scan stop describing
different corpora under --max-files.

Test debts this created, fixed rather than papered over: four fixtures claimed COMPLETE without the
counters a real sweep always writes, and each now states them with an added assertion locking the new
rule so the fixture edit hides nothing; three mutation patterns moved with the code and were
re-pointed; two mutants were masked by a fix firing earlier than the branch they guard, and their
oracles now isolate that branch; and emit_candidates gained two result classes, so its callers were
updated.

645 assertions across fifteen suites, 29 mutants all RED-on-break, harness self-check 8/8, the
original SW-1303 arms still PARTIAL_EVIDENCE with 0 candidate files, 81/81 mandatory attacks, privacy
5 canary classes x 6 sinks with 0 leaks, AI-VOS readiness 27/27.

CANONICAL_SKILL_DELTA=0, DEFAULT_MODEL_CONTEXT_DELTA=0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013pnJ85a3DYFU2ZW7CqQKCv
…er it broke

experiments/aivos/longrun.py unpacked emit_candidates into three values; it now reports two more
classes. My local sweep only ran tests/, so CI found it and I did not — the sweep now runs every
step CI runs, by exit code.

experiments/aivos/readiness.py builds the POSITIVE control: a clean, threshold-crossing trend that
must still produce a candidate. Its records claimed COMPLETE without the acquisition counters a real
sweep always writes, so the new rule correctly refused to promote them and the control went dark.
That is the failure mode a fail-closed patch is supposed to have a control for, and it worked —
the fixture now states its counters.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013pnJ85a3DYFU2ZW7CqQKCv
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant