From 43ab3bfd76febf2e48f5f7f373cc651a523fcf96 Mon Sep 17 00:00:00 2001 From: gHashTag Date: Sun, 5 Jul 2026 15:36:15 +0000 Subject: [PATCH 1/2] fix(meshd): key neighbor map on SocketAddr, not IpAddr Loopback smoke on 127.0.0.1:5011/5012/5013 exposed a HashMap collision: all three simulated nodes shared 127.0.0.1, so the last insert won and node-11 lost identity. On real hardware every P203 Mini has a unique IP, so the defect is sandbox-only, but it makes CI smoke unreliable. Change: - HashMap -> HashMap (rename ip_to_id -> addr_to_id) - Central RX dispatches on full src: SocketAddr from recv_from - drop unused IpAddr import Regression gate: - smoke/m2_loopback_smoke.sh now ends with a strict "triangle convergence gate" that fails with exit 2 if any node does not see both peers at steady ETX 1.0x. - smoke/M2_LOOPBACK_FIX_RESULTS.md documents defect, fix, and reproduction with actual sandbox output. Verification (all -sim, this sandbox, 2026-07-05): - cargo build --bin trios_meshd --release: clean - cargo test --workspace --release: 137 passed, 0 failed - DURATION=10 ./smoke/m2_loopback_smoke.sh: all 3 nodes PASS, exit 0 Behaviour on real hardware unchanged (unique IPs collapse to unique SocketAddrs); this is a strict superset. Ref PR #48 (parent, introduces smoke). phi^2 + phi^-2 = 3 --- smoke/M2_LOOPBACK_FIX_RESULTS.md | 87 ++++++++++++++++++++++++++++++++ smoke/m2_loopback_smoke.sh | 40 +++++++++++++++ src/bin/trios_meshd.rs | 17 ++++--- 3 files changed, 138 insertions(+), 6 deletions(-) create mode 100644 smoke/M2_LOOPBACK_FIX_RESULTS.md diff --git a/smoke/M2_LOOPBACK_FIX_RESULTS.md b/smoke/M2_LOOPBACK_FIX_RESULTS.md new file mode 100644 index 00000000..b1924756 --- /dev/null +++ b/smoke/M2_LOOPBACK_FIX_RESULTS.md @@ -0,0 +1,87 @@ +# M2 Loopback Smoke — Fix Verification (2026-07-05) + +Anchor: `phi^2 + phi^-2 = 3` + +## Context + +On 2026-07-05 the initial run of `smoke/m2_loopback_smoke.sh` (introduced +in PR #48) surfaced a sandbox-testability defect in `src/bin/trios_meshd.rs`: +the neighbor identity map was keyed on `IpAddr`, so all three loopback +processes on `127.0.0.1` collided into a single entry. Node-12 ↔ node-13 +would still converge on ETX ~1.00 because the last `insert` wins, but +node-11 dropped out of the graph and stayed isolated. + +Real hardware (three P203 Mini boards on distinct interfaces) is unaffected +— every board has a unique IP. The defect only manifests when the test +harness forces multiple daemons onto one loopback address, which is exactly +what a CI smoke rig has to do. + +## Fix + +`src/bin/trios_meshd.rs`: + +- `HashMap` → `HashMap` (rename + `ip_to_id` → `addr_to_id` to keep the code honest about the key). +- Central RX now dispatches on the full `src: SocketAddr` returned by + `recv_from`, not just `src.ip()`. +- `use std::net::IpAddr` dropped (no longer needed). + +Behaviour is a strict superset of the previous code on real hardware +(unique IPs collapse trivially into unique `SocketAddr`s once the map +is keyed that way), and correct on any loopback scenario. + +## Regression gate + +`smoke/m2_loopback_smoke.sh` now ends with a "triangle convergence gate": +every node's last `neighbors` log line must list **both** peers at steady +ETX in the range `1.00–1.09`. Any missing peer or non-steady ETX fails +the script with exit code 2. This is the automated tripwire for anyone +who accidentally regresses the fix. + +## Reproduction + +```bash +cd /home/user/workspace/tri-net +cargo build --bin trios_meshd --release +DURATION=10 ./smoke/m2_loopback_smoke.sh ; echo "exit=$?" +``` + +## Result (2026-07-05, this session, `-sim`) + +``` +=== triangle convergence gate === +node 11: PASS — both peers at steady ETX ([meshd] node 11 neighbors { 12=1.00, 13=1.00 }) +node 12: PASS — both peers at steady ETX ([meshd] node 12 neighbors { 11=1.00, 13=1.00 }) +node 13: PASS — both peers at steady ETX ([meshd] node 13 neighbors { 11=1.00, 12=1.00 }) + +smoke duration: 10s +exit=0 +``` + +## What this smoke is NOT + +- It is not a hardware M2 datapoint. The triangle is over loopback UDP, + no radios, no PHY, no interference. All measurements are `-sim`. +- It does not exercise TUN/IP forwarding — that is the next M2 sub-step + once the image-bake milestone (`docs/IMAGE_BAKE_MILESTONE.md`) unblocks + three-board deployment. +- It does not prove the fix under IPv6, dual-stack, or non-localhost + aliases; but for the pre-hardware regression tripwire it is sufficient. + +## Discipline hooks + +- no-fabricated-metrics: every number above came from a real run on this + sandbox at 2026-07-05 22:2x +07 and is labelled `-sim`. +- SHA-advance rule: any approval of this fix binds to the commit SHA that + the reviewer explicitly cites. Advancing the branch requires + `Re-reviewed at : delta `. +- Skill update: `tri-net-m2-m4-workflow` v1.1 removes the "known defect" + section and adds a "closed defect + regression gate" entry so future + loops don't re-discover this hole. + +## Full test suite + +`cargo test --workspace --release`: **137 tests passed, 0 failed**. +The fix does not touch any pure-logic surface. + +phi^2 + phi^-2 = 3 diff --git a/smoke/m2_loopback_smoke.sh b/smoke/m2_loopback_smoke.sh index ac971357..de53b0f6 100755 --- a/smoke/m2_loopback_smoke.sh +++ b/smoke/m2_loopback_smoke.sh @@ -71,8 +71,48 @@ for ID in 11 12 13; do fi done +echo "" +# --------------------------------------------------------------------------- +# REGRESSION GATE: all three nodes MUST see both peers at steady-state ETX. +# Historic defect (pre-2026-07-05): keying on IpAddr collided on loopback, +# leaving node-11 isolated. Fix moved the map to SocketAddr. This block +# fails loudly if that regresses. +# phi^2 + phi^-2 = 3 +# --------------------------------------------------------------------------- +echo "" +echo "=== triangle convergence gate ===" +GATE_FAIL=0 +for ID in 11 12 13; do + # Take the last neighbors line; require both peers listed, each at ETX 1.00-1.09. + LAST=$(grep "neighbors" "$LOGDIR/node${ID}.log" | tail -1 || true) + if [[ -z "$LAST" ]]; then + echo "node $ID: FAIL — no neighbors line in log" + GATE_FAIL=1 + continue + fi + OK_COUNT=0 + for PEER in 11 12 13; do + [[ "$PEER" == "$ID" ]] && continue + if echo "$LAST" | grep -qE "${PEER}=1\.0[0-9]"; then + OK_COUNT=$((OK_COUNT + 1)) + fi + done + if [[ "$OK_COUNT" -eq 2 ]]; then + echo "node $ID: PASS — both peers at steady ETX ($LAST)" + else + echo "node $ID: FAIL — expected 2 peers at ETX 1.0x, got $OK_COUNT ($LAST)" + GATE_FAIL=1 + fi +done + echo "" echo "logs preserved at: $LOGDIR" echo "smoke duration: ${DURATION}s" echo "" echo "phi^2 + phi^-2 = 3" + +if [[ "$GATE_FAIL" -ne 0 ]]; then + echo "" + echo "SMOKE GATE FAILED — three-node triangle did not converge." >&2 + exit 2 +fi diff --git a/src/bin/trios_meshd.rs b/src/bin/trios_meshd.rs index 44582bb3..879dc58b 100644 --- a/src/bin/trios_meshd.rs +++ b/src/bin/trios_meshd.rs @@ -18,7 +18,7 @@ use sha2::{Digest, Sha256}; use std::collections::{HashMap, HashSet}; use std::io; -use std::net::{IpAddr, SocketAddr, UdpSocket}; +use std::net::{SocketAddr, UdpSocket}; use std::sync::{Arc, Mutex}; use std::thread; use std::time::{Duration, Instant}; @@ -122,7 +122,12 @@ fn main() { let sock = Arc::new(UdpSocket::bind(cfg.listen).expect("bind")); let mut router = MeshRouter::new(me, ETX_WINDOW); let mut peer_ids: Vec = Vec::new(); - let mut ip_to_id: HashMap = HashMap::new(); + // Key by full SocketAddr (IP + port) so that loopback smokes with + // three nodes on 127.0.0.1:5011/5012/5013 don't collide on a shared IP. + // On real hardware every board has a unique IP, so this is a strict + // superset of the previous behaviour and never wrong. + // phi^2 + phi^-2 = 3 + let mut addr_to_id: HashMap = HashMap::new(); for (pid, addr) in &cfg.peers { let peer_pub = StaticKey::from_seed(seed_for(*pid)).public(); let session = my_key.session_with(&peer_pub, me < *pid); @@ -135,7 +140,7 @@ fn main() { }), ); peer_ids.push(*pid); - ip_to_id.insert(addr.ip(), *pid); + addr_to_id.insert(*addr, *pid); } let router = Arc::new(Mutex::new(router)); let rx = Arc::new(Mutex::new(RxShared::default())); @@ -151,11 +156,11 @@ fn main() { // Central RX: dispatch every datagram through the router. { - let (sock, router, rx, ip_to_id, dropped) = ( + let (sock, router, rx, addr_to_id, dropped) = ( sock.clone(), router.clone(), rx.clone(), - ip_to_id.clone(), + addr_to_id.clone(), dropped.clone(), ); thread::spawn(move || { @@ -165,7 +170,7 @@ fn main() { Ok(v) => v, Err(_) => continue, }; - let from = match ip_to_id.get(&src.ip()) { + let from = match addr_to_id.get(&src) { Some(f) => *f, None => continue, }; From 293978a67552c1ba86d71b376ceecefaef54cd45 Mon Sep 17 00:00:00 2001 From: gHashTag Date: Sun, 5 Jul 2026 16:05:56 +0000 Subject: [PATCH 2/2] =?UTF-8?q?fix(smoke):=20gate=20v2=20=E2=80=94=20finit?= =?UTF-8?q?e-ETX=20visibility=20across=20N=3D5=20samples?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The v1 gate ("last neighbors line == 1.0x for both peers") was non-deterministic under WMEWMA. Reviewer ran the smoke twice back-to-back on same host, same binary (SHA 43ab3bf), only pkill between runs: run #1: node-11 { 13=2.03 } -> gate FAIL (coin-flip red) run #2: node-11 { 12=1.00, 13=1.00 } -> gate PASS (coin-flip green) Both are correct measurements. The gate itself was the problem: WMEWMA with alpha=0.5, HELLO_MS=300, ETX_WINDOW=3 bounces ETX for 3-4 ticks after any single dropped HELLO on any real channel. The v1 gate tested the algorithm's asymptote (steady 1.0x) instead of the SocketAddr fix's actual invariant (both peers visible, ETX finite). Category error. v2 gate design: - Grep last N=5 neighbors lines per node. - Require both peers with a finite decimal ETX in every sample. - Reject missing peers (pre-fix regression signal). - Reject inf/nan (persistent link death). - Accept transient spikes like 2.03 (correct WMEWMA response). Sandbox verification: - Positive: 5/5 runs PASS with new gate. - Negative: synthetic pre-fix log (node-11 with empty neighbors) correctly triggers FAIL for node-11. Also added smoke/m2_loopback_smoke_n_runs.sh: convenience wrapper that runs the smoke N times (default 5) and requires every run to exit 0. Encodes the "results-without-repro-check" discipline: a regression tripwire must be verified deterministic before trusted. Reference: WMEWMA rationale (Woo, Tong & Culler, SenSys 2003; Rosati et al., arXiv:1307.6350). Ref PR #48 (parent), PR #49 (this branch). phi^2 + phi^-2 = 3 --- smoke/M2_LOOPBACK_FIX_RESULTS.md | 11 +++-- smoke/m2_loopback_smoke.sh | 71 +++++++++++++++++++++++-------- smoke/m2_loopback_smoke_n_runs.sh | 52 ++++++++++++++++++++++ 3 files changed, 113 insertions(+), 21 deletions(-) create mode 100755 smoke/m2_loopback_smoke_n_runs.sh diff --git a/smoke/M2_LOOPBACK_FIX_RESULTS.md b/smoke/M2_LOOPBACK_FIX_RESULTS.md index b1924756..58101424 100644 --- a/smoke/M2_LOOPBACK_FIX_RESULTS.md +++ b/smoke/M2_LOOPBACK_FIX_RESULTS.md @@ -75,9 +75,14 @@ exit=0 - SHA-advance rule: any approval of this fix binds to the commit SHA that the reviewer explicitly cites. Advancing the branch requires `Re-reviewed at : delta `. -- Skill update: `tri-net-m2-m4-workflow` v1.1 removes the "known defect" - section and adds a "closed defect + regression gate" entry so future - loops don't re-discover this hole. +- Results-without-repro-check: the reviewer's insistence on running the + smoke a second time surfaced non-determinism that no single run could + have exposed. New rule captured in `tri-net-m2-m4-workflow` v1.3: + smoke-gate outcomes require N-run confirmation (default N=5) before they + are trusted as regression tripwires. +- Skill update: `tri-net-m2-m4-workflow` v1.3 records both the sandbox- + testability defect closure and the gate-design lesson (WMEWMA + non-determinism, aspiration-vs-property confusion). ## Full test suite diff --git a/smoke/m2_loopback_smoke.sh b/smoke/m2_loopback_smoke.sh index de53b0f6..75dd4773 100755 --- a/smoke/m2_loopback_smoke.sh +++ b/smoke/m2_loopback_smoke.sh @@ -73,34 +73,69 @@ done echo "" # --------------------------------------------------------------------------- -# REGRESSION GATE: all three nodes MUST see both peers at steady-state ETX. +# REGRESSION GATE (v2): visibility with finite ETX in the last N samples. +# +# What the SocketAddr fix guarantees deterministically: +# 1. Every node learns BOTH peers (no IpAddr collision suppressing one). +# 2. Both link-ETX values are FINITE (WMEWMA est > DEAD_EPS = 0.15 both +# directions). +# +# What the fix does NOT guarantee (and the ETX algorithm does not promise): +# - Steady ETX = 1.00. WMEWMA with alpha=0.5, HELLO period 300 ms, +# ETX_WINDOW = 3 will bounce for 3-4 ticks after ANY single dropped +# HELLO on any real channel (loopback, radio, tunnel). A single-sample +# "tail -1 == 1.0x" gate is non-deterministic by construction. +# +# Reference: WMEWMA rationale (Woo, Tong & Culler, SenSys 2003; +# Rosati et al., arXiv:1307.6350). Confirmed empirically: reviewer saw +# node-11 report {13=2.03} on run #1 and {13=1.00} on run #2, same binary, +# same host, back-to-back. +# +# This gate accepts the last N=5 neighbors-samples per node and requires +# that in EVERY sample both peers are present with a finite numeric ETX. +# Any missing peer, ANY infinite ETX ("inf"), or fewer than N samples => +# gate fails with exit 2. +# # Historic defect (pre-2026-07-05): keying on IpAddr collided on loopback, -# leaving node-11 isolated. Fix moved the map to SocketAddr. This block -# fails loudly if that regresses. +# leaving node-11 isolated. Fix moved the map to SocketAddr. This gate is +# the tripwire against that regression. +# # phi^2 + phi^-2 = 3 # --------------------------------------------------------------------------- echo "" -echo "=== triangle convergence gate ===" +echo "=== triangle visibility gate (v2: last N=5 samples, finite ETX) ===" +SAMPLES_REQUIRED=5 GATE_FAIL=0 for ID in 11 12 13; do - # Take the last neighbors line; require both peers listed, each at ETX 1.00-1.09. - LAST=$(grep "neighbors" "$LOGDIR/node${ID}.log" | tail -1 || true) - if [[ -z "$LAST" ]]; then - echo "node $ID: FAIL — no neighbors line in log" + # All neighbors lines for this node, take the last N. + LAST_N=$(grep "neighbors" "$LOGDIR/node${ID}.log" | tail -${SAMPLES_REQUIRED}) + NLINES=$(echo -n "$LAST_N" | grep -c "neighbors" || true) + if [[ "$NLINES" -lt "$SAMPLES_REQUIRED" ]]; then + echo "node $ID: FAIL — only $NLINES neighbors samples (need $SAMPLES_REQUIRED)" GATE_FAIL=1 continue fi - OK_COUNT=0 - for PEER in 11 12 13; do - [[ "$PEER" == "$ID" ]] && continue - if echo "$LAST" | grep -qE "${PEER}=1\.0[0-9]"; then - OK_COUNT=$((OK_COUNT + 1)) - fi - done - if [[ "$OK_COUNT" -eq 2 ]]; then - echo "node $ID: PASS — both peers at steady ETX ($LAST)" + NODE_FAIL=0 + SAMPLE_IDX=0 + while IFS= read -r LINE; do + SAMPLE_IDX=$((SAMPLE_IDX + 1)) + # For each peer, require: PEER=. Reject "inf" and missing. + for PEER in 11 12 13; do + [[ "$PEER" == "$ID" ]] && continue + # Match PEER=. (finite decimal). Reject inf/INF/NaN. + if ! echo "$LINE" | grep -qE "${PEER}=[0-9]+\.[0-9]+"; then + echo "node $ID: FAIL sample #${SAMPLE_IDX} — peer $PEER missing or non-finite ($LINE)" + NODE_FAIL=1 + elif echo "$LINE" | grep -qiE "${PEER}=(inf|nan)"; then + echo "node $ID: FAIL sample #${SAMPLE_IDX} — peer $PEER has non-finite ETX ($LINE)" + NODE_FAIL=1 + fi + done + done <<< "$LAST_N" + if [[ "$NODE_FAIL" -eq 0 ]]; then + LAST_ONE=$(echo "$LAST_N" | tail -1) + echo "node $ID: PASS — both peers visible with finite ETX across last $SAMPLES_REQUIRED samples (last: $LAST_ONE)" else - echo "node $ID: FAIL — expected 2 peers at ETX 1.0x, got $OK_COUNT ($LAST)" GATE_FAIL=1 fi done diff --git a/smoke/m2_loopback_smoke_n_runs.sh b/smoke/m2_loopback_smoke_n_runs.sh new file mode 100755 index 00000000..15fbee29 --- /dev/null +++ b/smoke/m2_loopback_smoke_n_runs.sh @@ -0,0 +1,52 @@ +#!/usr/bin/env bash +# Wrapper: run m2_loopback_smoke.sh N times and require every run to +# exit 0. Any non-zero exit fails the wrapper. This is the deterministic- +# regression-tripwire discipline recorded in tri-net-m2-m4-workflow v1.3 +# ("results-without-repro-check"). +# +# Usage: N=5 DURATION=10 ./smoke/m2_loopback_smoke_n_runs.sh +# +# phi^2 + phi^-2 = 3 + +set -uo pipefail + +N="${N:-5}" +DURATION="${DURATION:-10}" +BIN="${BIN:-./target/release/trios_meshd}" + +if [[ ! -x "$BIN" ]]; then + echo "FATAL: $BIN not found. Run: cargo build --bin trios_meshd --release" >&2 + exit 1 +fi + +FAIL_RUNS=() +for i in $(seq 1 "$N"); do + echo "" + echo "======================================" + echo " N-run smoke: run $i / $N" + echo "======================================" + if DURATION="$DURATION" BIN="$BIN" ./smoke/m2_loopback_smoke.sh; then + echo "run $i: OK (exit 0)" + else + echo "run $i: FAIL (exit non-zero)" >&2 + FAIL_RUNS+=("$i") + fi + sleep 1 +done + +echo "" +echo "======================================" +echo " N-run summary" +echo "======================================" +echo "Total runs: $N" +echo "Failed runs: ${#FAIL_RUNS[@]}${FAIL_RUNS:+ (indices: ${FAIL_RUNS[*]})}" + +if [[ ${#FAIL_RUNS[@]} -eq 0 ]]; then + echo "GATE PASS: $N/$N runs succeeded. Regression tripwire is deterministic on this host." + echo "phi^2 + phi^-2 = 3" + exit 0 +else + echo "GATE FAIL: ${#FAIL_RUNS[@]} of $N runs failed. Gate is non-deterministic on this host \u2014 do NOT trust as regression tripwire." >&2 + echo "phi^2 + phi^-2 = 3" + exit 2 +fi