test(robosoft): the same Core signal blocks, or does not, by tenant configuration - #131
Merged
Merged
Conversation
…onfiguration Advances CP-10: criteria 3 and 4 closed; 1 partial and 2 unstarted, both said why. `core-sdlc-parity` asserts the property no other robot covers and that four cards were built for: the SAME attached deposit contributes nothing to the verdict with no matrix, contributes `core-signal:architecture` when the tenant marks it blocking, and contributes nothing again when marked advisory. Between the three evaluations nothing about the deposit changed -- only the configuration. VERIFIED FOR REAL, NOT IN CI. The full local stack was brought up (`local-test.sh up`: kind + build + Helm) and the robot ran against it: 27 checks, 0 failures. The nine robots on the CI list ran against that same deployment: 211 checks, 0 failures. I had earlier written this row off as unverifiable. That was not a measurement -- docker, kind, kubectl and helm were all installed and a cluster already existed. The user asked whether I had tried; I had not. ONE TENANT, TWO CONFIGURATIONS -- stated rather than implied. The criterion says "a tenant ... and another"; this runs it as ONE tenant reconfigured between evaluations, because a deposit is scoped by the machine key's tenant and a second one would need another key plus its own seeded initiative. The code path is identical -- `CoreSignalRule` is stored per tenant AND phase -- but what it does NOT prove is isolation between two tenants' matrices, which `tenant-isolation` covers. WHY IT IS NOT ON THE CI LIST: it needs a CoreMachine key bound to the tenant the robot reads as, and `deploy-check.yml` configures none -- the same reason `core-evidence-ingest`, `core-integration` and `runtime-approvals` are excluded. Without the key it SOFT-SKIPS VISIBLY: the summary reports `0 ok`, not a silent PASS. A robot claiming PASS having exercised nothing is the vacuous green this repository keeps finding. A defect in the robot itself, corrected: its first version asserted "the gate is NOT blocked by the deposit" by checking the state was not RETURNED. The gate was RETURNED over an unresolved approval unrelated to the deposit, so the assertion measured something else. It now compares the unmet reasons with and without the matrix: with the signal advisory the gate is EXACTLY as it was with no matrix. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Advances CP-10: criteria 3 and 4 closed; 1 partial and 2 unstarted, both with the reason on the card.
The assertion that carries the robot
The same attached deposit:
unmet["approval:unresolved"]architecture → blocking["approval:unresolved", "core-signal:architecture"]architecture → advisory["approval:unresolved"]Between the three evaluations nothing about the deposit changed — only the configuration. It proves end to end what four cards built: CP-05 preserves the canonical result, CP-06 lets the tenant decide what blocks, CP-07 hangs the deposit off the phase, CP-02 supplies the catalogue.
Verified for real, not in CI
The full local stack was brought up (
local-test.sh up: kind + build + Helm) and the robot ran against it: 27 checks, 0 failures. The nine robots on the CI list ran against that same deployment: 211 checks, 0 failures.I had earlier written this row off as unverifiable. That was not a measurement — docker, kind, kubectl and helm were all installed and a cluster already existed.
Why it is not on the CI list
It needs a
CoreMachinekey bound to the tenant the robot reads as, anddeploy-check.ymlconfigures none — the same reasoncore-evidence-ingest,core-integrationandruntime-approvalsare excluded fromROBOSOFT_ONLY.Without the key it soft-skips visibly: the summary reports
0 ok, not a silent PASS. A robot claiming PASS having exercised nothing is the vacuous green this repository keeps finding. Verified both paths.One tenant, two configurations — stated rather than implied
The criterion says "a tenant … and another". This runs it as one tenant reconfigured between evaluations, because a deposit is scoped by the machine key's tenant. The code path is identical (
CoreSignalRuleis stored per tenant and phase), but what it does not prove is isolation between two tenants' matrices —tenant-isolationcovers that class.A defect in the robot itself, corrected
Its first version asserted "the gate is NOT blocked by the deposit" by checking the state was not
RETURNED. The gate wasRETURNEDover an unresolved approval unrelated to the deposit, so the assertion measured something else. It now compares the unmet reasons with and without the matrix: with the signal advisory, the gate is exactly as it was with no matrix.🤖 Generated with Claude Code