Skip to content

Wave Loop 577 — gate adequacy: 13 of 13 gates bite #2000

Description

@gHashTag

☣️ [VERDICT]

Wave 576 established that every claim has a check, and said explicitly that it did not establish any check was sufficient. This is the missing half — the vacuity oracle of Prop. 12a redirected at the gate map. A gate that cannot fail is not a gate, exactly as a property whose guard is unreachable is not a property.

Method

For each gate, apply one mutation that should violate the claim it guards, then run that gate alone. It must go red. Mutations hit the generated RTL rather than the emitters — one yosys run instead of a rebuild — and each mutation string is asserted to occur exactly once before use.

13 of 13

gate mutation verdict
Prop. 7 interrupt_controller revert clear-then-set → set-then-clear red
Prop. 8 axi_lite_slave ready stays high while a response is pending red
Prop. 9 dma_controller advance the burst without a handshake red
Prop. 13 layer_sequencer drop the zero-neuron guard red
Prop. 25 integration double buffer stops alternating red
Prop. 24 liveness tie inference start off, stalling the engine red
Prop. 26 DMA / multilayer re-drop both zero-sized requests red, red
Prop. 11 assumptions drop -set-assumes red
Prop. 27 doc gate ×3: remove a gate line, cite bare t27c, empty bash block red ×3
Prop. 1 seals edit a spec, leave its seal stale red

The liveness mutation is the one worth separating out. Stalling the engine leaves every safety property true — an engine that does nothing violates nothing — so the liveness witnesses are the only reason it goes red. That gate exists for a mutation no safety property can see, and it caught it.

Why the first two phases matter as much as the third

phase requirement
baseline unmutated build, every gate passes — else "went red" isn't evidence the mutation did it
control dead wire added everywhere, every gate still passes — catches a gate that fires on any edit and would score 13/13 detecting nothing
mutation each gate goes red for its own mutation

A clean sweep is a reason to check the harness, not to celebrate: 8/8 on the first batch is exactly where the last three waves found harness defects. Both control phases came back clean, and the workflow runs all three in order every time.

Still not established: each gate detects the mutation chosen for it — one point per claim, not adequacy over all violations. Mutation testing bounds from below, never from above.

New formal-mutation.yml (weekly + dispatch). Verified by extracting the harness from the workflow YAML and running it, so what was tested is what CI runs. 28 propositions · 28 gates · 1208 passed, 0 failed · seals 496/496.


Three variants for the next wave

A — Sweep the maximum-sized request. Twice deferred, and the last untested end of every range: num_chunks is 8 bits with a num_chunks-1 terminator; beats_owed = (bytes_remaining + 7) >> 3 can overflow 32 bits. Zero found two real defects; the other end has had no attention at all.

B — Close Prop. 25 with a per-buffer written-flag. The one open RTL defect. Now the cheapest it has ever been: baseline gate, zero-size gate and mutation harness all catch a regression on the first run rather than the fourth.

C — Raise mutation coverage from one point per claim to several. Prop. 28e is the honest limit of this wave. Generate mutations systematically — negate each guard, off-by-one each comparison, drop each handshake term — and measure what fraction the gates catch. Turns "the gates bite" into a coverage number that can go up.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions