Skip to content

test_lab: fail when a zeta>=1 physics-sweep row oscillates or its flags contradict its reading - #65

Merged
InauguralPhysicist merged 1 commit into
mainfrom
claude/ci58-dyn-16
Nov 1, 2026
Merged

InauguralPhysicist merged 1 commit into
mainfrom
claude/ci58-dyn-16

Conversation

@InauguralPhysicist

Copy link
Copy Markdown
Member

What does this PR do?

tests/test_lab.sh claimed to check that the critically damped and overdamped rows of the physics.eigs zeta-sweep (zeta=1, zeta=2) do not oscillate. What it actually did was count lines containing settles. Every underdamped row reads x OSCILLATES, energy settles, so the count was always at least 1, and the gate stayed green even when the zeta>=1 rows oscillated.

This PR replaces that count with a row-by-row check:

  • tests/check_physics_sweep.py (new) parses the sweep table and makes five checks:
    1. The zeta rows are exactly -0.05, 0, 0.1, 0.4, 1, 2, in that order.
    2. Each row has its required reading. zeta<0 must read DIVERGES (energy grew), 0<=zeta<1 must read x OSCILLATES, energy settles, and zeta>=1 must read exactly settles.
    3. The energy-grew flag is Y exactly when the reading is DIVERGES.
    4. When energy did not grow, the x:osc flag is Y exactly when the reading is x OSCILLATES.
    5. The zeta=0 E:conv flag is -. This is the lab's stated claim: at zeta=0 energy is conserved, so it never converges.
  • tests/test_lab.sh: the grep -c settles lines are replaced by a call to the checker. The DONE, DIVERGES and x OSCILLATES greps are kept, so no existing assertion is removed.
  • tests/test_physics_sweep_plants.sh (new) is a CI plant harness:
    • The unmodified physics.eigs must pass first.
    • Then six planted faults are each applied to a copy of physics.eigs, at least one per check.
    • Each plant must change exactly 1 line, so a plant that stops applying fails the harness.
    • The planted program must still exit 0 and print DONE, so a crash does not count as a rejection.
    • The checker must reject each plant with the exact message of the check that plant targets. A plant rejected for any other reason fails the harness.
  • .github/workflows/test.yml: one new row in runCmd runs the plant harness. This is unavoidable, because runCmd is the only place tests are enrolled. python3 is already used by test_lab.sh (check_solver_output.py on main), so the devcontainer image needs nothing new.

There is no changes/ fragment. This PR changes only tests and CI, and dynamics CI has no fragment check (#55 landed without one).

Checklist

  • Every changed/added specimen parses and runs (eigenscript <file>.eigs). No specimen changed. physics.eigs runs unmodified and under every plant.
  • bash tests/test_smoke.sh and bash tests/test_lab.sh pass locally (pinned v0.48.0)
  • Observer usage is load-bearing. Not applicable: no specimen changed.
  • Any runtime gap or surprising predicate behavior is logged in FINDINGS.md. None was found; the defect is in the test.
  • eigs.json version bumped if the importable surface changed. Not applicable: the surface is unchanged.

Finding

  • DYN-LAB-1. The physics zeta-sweep overdamped check passes even when the zeta>=1 rows oscillate.
  • Location: tests/test_lab.sh:16-18 (origin/main). The sweep rows are printed at physics.eigs:116-126. The E:/x: flag columns were never checked.
  • Tracking: https://github.com/InauguralSystems/hq/issues/58 ("treat CI as a product", Wave 2).

Plant: red before, green after

All runs below use the pinned runtime, EigenScript v0.48.0 (546848c0, the .devcontainer/Dockerfile EIGS_REF). "Old gate" is git show origin/main:tests/test_lab.sh.

Plant A is the Wave-1 reproduction. It is a 1-line change to physics.eigs:122:

122c122
<     elif p[3] == 1:
---
>     elif (p[3] == 1) or (z >= 1.0):

Old gate, rc=0. It lies: both zeta>=1 rows read oscillating and the gate still prints PASS.

  1	| E:- Y - -     | x:- Y -      | x OSCILLATES, energy settles
  2	| E:- Y - -     | x:- - -      | x OSCILLATES, energy settles
PASS: physics zeta-sweep produces the expected regime spectrum

New gate, EIGENSCRIPT=$E bash tests/test_lab.sh, rc=1:

FAIL: physics sweep: zeta=1 reads 'x OSCILLATES, energy settles', expected 'settles'

All six plants, old gate compared with new gate. Each plant changes 1 line.

plant old gate new gate
overdamped-reads-oscillating (A) rc=0 PASS rc=1 zeta=1 reads 'x OSCILLATES, energy settles', expected 'settles'
x-osc-false-positive (or (zeta >= 1.0) on the x observer) rc=0 PASS rc=1 same message
overdamped-row-dropped (2.0 removed from the sweep) rc=0 PASS rc=1 expected zeta rows ['-0.05', '0', '0.1', '0.4', '1', '2'], got ['-0.05', '0', '0.1', '0.4', '1']
x-flag-column-desync (x:osc printed from p[4]) rc=0 PASS rc=1 zeta=0 x:osc flag '-' contradicts reading 'x OSCILLATES, energy settles'
grew-column-desync (grew printed from p[5]) rc=0 PASS rc=1 zeta=-0.05 energy-grew flag '-' contradicts reading 'DIVERGES (energy grew)'
zeta0-energy-converged (E:conv printed from p[0]) rc=0 PASS rc=1 zeta=0 energy reported converged; the undamped row must conserve energy

Green after the fix. On the unmodified tree, EIGENSCRIPT=$E bash tests/test_lab.sh gives rc=0:

PASS: zeta=-0.05 reads 'DIVERGES (energy grew)' (grew=Y, x:osc=Y)
PASS: zeta=0 reads 'x OSCILLATES, energy settles' (grew=-, x:osc=Y)
PASS: zeta=0.1 reads 'x OSCILLATES, energy settles' (grew=-, x:osc=Y)
PASS: zeta=0.4 reads 'x OSCILLATES, energy settles' (grew=-, x:osc=Y)
PASS: zeta=1 reads 'settles' (grew=-, x:osc=-)
PASS: zeta=2 reads 'settles' (grew=-, x:osc=-)
PASS: zeta=0 energy never converges while x oscillates
PASS: physics zeta-sweep produces the expected regime spectrum

EIGENSCRIPT=$E bash tests/test_physics_sweep_plants.sh gives rc=0:

PASS: unmodified physics.eigs accepted
PASS: plant overdamped-reads-oscillating rejected: FAIL: physics sweep: zeta=1 reads 'x OSCILLATES, energy settles', expected 'settles'
PASS: plant x-osc-false-positive rejected: FAIL: physics sweep: zeta=1 reads 'x OSCILLATES, energy settles', expected 'settles'
PASS: plant overdamped-row-dropped rejected: FAIL: physics sweep: expected zeta rows ['-0.05', '0', '0.1', '0.4', '1', '2'], got ['-0.05', '0', '0.1', '0.4', '1']
PASS: plant x-flag-column-desync rejected: FAIL: physics sweep: zeta=0 x:osc flag '-' contradicts reading 'x OSCILLATES, energy settles'
PASS: plant grew-column-desync rejected: FAIL: physics sweep: zeta=-0.05 energy-grew flag '-' contradicts reading 'DIVERGES (energy grew)'
PASS: plant zeta0-energy-converged rejected: FAIL: physics sweep: zeta=0 energy reported converged; the undamped row must conserve energy
PASS: every physics-sweep plant is rejected

Every check is load-bearing. In scratch copies, each of the checker's five checks was disabled in turn (its if replaced by if False:), and the checker was also gutted with sys.exit(0). The plant harness then gave:

check removed harness rc first failure
reading 1 plant overdamped-reads-oscillating rejected for the wrong reason
grew consistency 1 plant grew-column-desync rejected for the wrong reason
x:osc consistency 1 plant x-flag-column-desync was accepted
row list 1 plant overdamped-row-dropped was accepted
zeta=0 E:conv 1 plant zeta0-energy-converged was accepted
whole checker gutted 1 plant overdamped-reads-oscillating was accepted

Checks run

All locally, on pinned v0.48.0, covering the full CI runCmd:

  • test_lint rc=0
  • test_smoke rc=0
  • test_lab rc=0
  • test_physics_sweep_plants rc=0
  • test_orbit_hist rc=0
  • test_bifurcation rc=0
  • test_orbit_oracle rc=0 (Xvfb)
  • test_orbit_mouse rc=0
  • test_bif_mem rc=0
  • bash -n and py_compile pass on the new and changed files.

The added CI time is about 0.3s (the plant harness reruns physics.eigs 7 times). For context, the "suite (in devcontainer)" job takes 139s p50 and 155s p90 (n=40, 2026-10-04..2026-11-01). This PR is not a speedup.

Coverage goes up and nothing is removed. One weak count assertion becomes 6 row checks plus 1 claim check, and 6 plants are added.

Done when

  • On this PR's CI, the suite (in devcontainer) log shows PASS: every physics-sweep plant is rejected and PASS: zeta=2 reads 'settles' (grew=-, x:osc=-)
  • Applying the plant-A line to physics.eigs makes EIGENSCRIPT=eigenscript bash tests/test_lab.sh exit 1 with FAIL: physics sweep: zeta=1 reads 'x OSCILLATES, energy settles', expected 'settles'
  • Replacing the rows[1][2] != "-" condition in tests/check_physics_sweep.py with False makes bash tests/test_physics_sweep_plants.sh exit 1

Overlaps

There are no open PRs in dynamics (GET /repos/InauguralSystems/dynamics/pulls?state=open returned nothing).

Not done in this PR:

  • dynamics has no required status check (DYN-REPO-1). Until one is required, this gate cannot block a merge. Requiring a check is a settings change, so it is out of scope for this PR.
  • There is no dynamics issue for DYN-LAB-1, so this PR references hq#58.
  • The flag columns that the lab makes no claim about are deliberately not pinned. These are E:osc/E:div, and x:conv/x:div outside the checks above. They follow observer-predicate semantics, and pinning them would force a re-baseline at every pin bump.

Generated by Claude Code

The overdamped assertion counted lines containing "settles", and every
underdamped row reads "x OSCILLATES, energy settles", so it passed with the
zeta=1 and zeta=2 rows oscillating (hq#58, DYN-LAB-1). tests/check_physics_sweep.py
parses the table: each zeta row must carry its required reading, the reading
must agree with its energy-grew and x:osc flags, and the zeta=0 energy must
never read converged. tests/test_physics_sweep_plants.sh runs six planted
faults through it in CI, one or more per check; each must apply to exactly
one line and be rejected with the message of the check it targets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0149GnPvac4QFD5y1CrHCcb1

Copy link
Copy Markdown
Member Author

@codex review


Generated by Claude Code

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-01T21:46:11.963266Z 6ea7dd8 Draft marked ready
🔒 Security Review ✅ Completed 2026-11-01T21:46:52.678280Z 6ea7dd8 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: 6ea7dd8b80

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@InauguralPhysicist
InauguralPhysicist marked this pull request as ready for review November 1, 2026 21:45
@InauguralPhysicist
InauguralPhysicist merged commit a7903e4 into main Nov 1, 2026
1 check passed
@InauguralPhysicist
InauguralPhysicist deleted the claude/ci58-dyn-16 branch November 1, 2026 21:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants