docs(testing): master test plan — 17 chapters, 1,186 rows, a three-tier release gate - #60
Open
wshallwshall wants to merge 1 commit into
Open
docs(testing): master test plan — 17 chapters, 1,186 rows, a three-tier release gate#60wshallwshall wants to merge 1 commit into
wshallwshall wants to merge 1 commit into
Conversation
…three-tier gate The umbrella test plan the two existing artifacts plug into. FEATURE-COVERAGE-PLAN.md owns the per-subsystem coverage-gap audit and WIN2025-TEST-PLAN.md owns what only a real Server 2025 host can prove; neither covers the product as a whole, and neither covers the IDE, the tray, the Steps editor or the publish path at all. This states what the product is, what has to be true before it ships, and who proves it — delegating rather than restating, with foreign IDs prefixed FCP:/W25: because both predecessors use IDs that collide with this plan's own (HA-20 and API-13 exist in both spaces). Three things are load-bearing: Row classes. Only class-T rows gate a release. A criterion of "Pass = report received" or "Pass = a published number" cannot fail, so it cannot be a gate — those are class C (characterisation, tracked) and class A (external assurance, gates production exposure only). The draft had a penetration-test procurement event sitting at P0. The gate is now 196 blocking T-rows + 9 campaign-gated + 3 C + 3 A = 211 P0-marked, reconciled row-for-row against an independent parse. The publishing boundary. docs/BACKLOG.md is a published baseline ending at #231 and docs/security/ is gitignored post-cutover, so a citation above the baseline or into a withheld tree is valid evidence, not a dangling reference. An earlier pass had this backwards and was filing withheld material as coverage defects, including one that reached a release-gate criterion. Two owner-named items were structurally unprovable. The tray had zero P0 rows — all eight sat on NSSM/service/install — so the icon-state mapping, action routing, Win32 message pump and the lying-tray case are now blocking. HA had no P0 that proved a failover works; six now do. Also folds in surfaces nothing owned: last_resort.py (the PHI-egress backstop), netaddr.py (the shared allow-list decision the co-location exists to keep consistent), pki.py, the shipped PySide6 harness GUI, disk-full/ENOSPC, licence/copyleft compliance, destructive operator errors and non-ASCII patient data end to end. Ships as an index plus 20 chapter files: the assembled document is 1.8 MB, past GitHub's markdown render limit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The umbrella test plan the two existing artifacts plug into.
FEATURE-COVERAGE-PLAN.mdowns the per-subsystemcoverage-gap audit;
WIN2025-TEST-PLAN.mdowns what only a realServer 2025 host under the NSSM service identity can prove. Neither covers the product as a whole,
and neither covers the IDE, the tray, the Steps editor or the publish path at all.
Ships as
docs/testing/MASTER-TEST-PLAN.md(index) plus 20chapter files under
docs/testing/master-test-plan/— the assembled document is 1.8 MB, pastGitHub's markdown render limit. Every chapter carries scope, what is already covered and must not
be re-tested, a risk analysis, the test matrix, detailed scenarios, an automation disposition,
prerequisites, exit criteria and open questions.
Foreign IDs are prefixed
FCP:/W25:throughout, because both predecessors use IDs that collidewith this plan's own —
HA-20andAPI-13exist in both spaces.What is load-bearing
Row classes. Only class-T rows gate a release. A criterion of "Pass = report received" or
"Pass = a published number" cannot fail, so it cannot be a gate — those are class C
(characterisation, tracked) and class A (external assurance, gates production exposure only).
The draft had a penetration-test procurement event sitting at P0. The gate is now 196 blocking
T-rows + 9 campaign-gated + 3 C + 3 A = 211 P0-marked, reconciled row-for-row against an
independent parse of all 17 matrices.
The publishing boundary.
docs/BACKLOG.mdis a published baseline ending at## 231.anddocs/security/is gitignored post-cutover, so a citation above the baseline or into a withheld treeis valid evidence, not a dangling reference. An earlier pass had this backwards and was filing
withheld material as coverage defects — including one that had reached a release-gate criterion.
Two owner-named items were structurally unprovable. The tray application held zero P0 rows
(all eight sat on NSSM/install) even though ADR 0113 names the untested Win32 layer as the design's
main risk; the state→icon map, action routing, message-pump dispatch and the "lying tray" are now
blocking. HA had no P0 proving a failover works — six now do, all on
container-CIwith aservice container, none needing a second host.
Surfaces nothing previously owned
last_resort.py(the process-level PHI-egress backstop),netaddr.py(the shared allow-listdecision its co-location exists to keep consistent),
pki.py, the shipped PySide6 harness GUI —which is both a distribution and the plan's only route to hostile-input injection — plus disk-full /
ENOSPC, licence and copyleft compliance, destructive operator errors, and non-ASCII patient data
end to end.
Review notes
whether
load_settingsshould refuse unknown keys, which transports are deliberately notauthorable from
connections.toml).messagefoundry checkis invoked by no workflow, and onemeta-test plus one
ci.ymlregex fix turns 36 already-written tests from dormant intoexecuting. That is items 1–2 of §24's prioritised ten.
🤖 Generated with Claude Code