Skip to content

docs(testing): master test plan — 17 chapters, 1,186 rows, a three-tier release gate - #60

Open
wshallwshall wants to merge 1 commit into
mainfrom
master-test-plan
Open

docs(testing): master test plan — 17 chapters, 1,186 rows, a three-tier release gate#60
wshallwshall wants to merge 1 commit into
mainfrom
master-test-plan

Conversation

@wshallwshall

Copy link
Copy Markdown
Collaborator

The umbrella test plan the two existing artifacts plug into.
FEATURE-COVERAGE-PLAN.md owns the per-subsystem
coverage-gap audit; WIN2025-TEST-PLAN.md owns what only a real
Server 2025 host under the NSSM service identity can prove. Neither covers the product as a whole,
and neither covers the IDE, the tray, the Steps editor or the publish path at all.

Ships as docs/testing/MASTER-TEST-PLAN.md (index) plus 20
chapter files under docs/testing/master-test-plan/ — the assembled document is 1.8 MB, past
GitHub's markdown render limit. Every chapter carries scope, what is already covered and must not
be re-tested
, a risk analysis, the test matrix, detailed scenarios, an automation disposition,
prerequisites, exit criteria and open questions.

Foreign IDs are prefixed FCP: / W25: throughout, because both predecessors use IDs that collide
with this plan's own — HA-20 and API-13 exist in both spaces.

What is load-bearing

Row classes. Only class-T rows gate a release. A criterion of "Pass = report received" or
"Pass = a published number" cannot fail, so it cannot be a gate — those are class C
(characterisation, tracked) and class A (external assurance, gates production exposure only).
The draft had a penetration-test procurement event sitting at P0. The gate is now 196 blocking
T-rows + 9 campaign-gated + 3 C + 3 A = 211 P0-marked, reconciled row-for-row against an
independent parse of all 17 matrices.

The publishing boundary. docs/BACKLOG.md is a published baseline ending at ## 231. and
docs/security/ is gitignored post-cutover, so a citation above the baseline or into a withheld tree
is valid evidence, not a dangling reference. An earlier pass had this backwards and was filing
withheld material as coverage defects — including one that had reached a release-gate criterion.

Two owner-named items were structurally unprovable. The tray application held zero P0 rows
(all eight sat on NSSM/install) even though ADR 0113 names the untested Win32 layer as the design's
main risk; the state→icon map, action routing, message-pump dispatch and the "lying tray" are now
blocking. HA had no P0 proving a failover works — six now do, all on container-CI with a
service container, none needing a second host.

Surfaces nothing previously owned

last_resort.py (the process-level PHI-egress backstop), netaddr.py (the shared allow-list
decision its co-location exists to keep consistent), pki.py, the shipped PySide6 harness GUI —
which is both a distribution and the plan's only route to hostile-input injection — plus disk-full /
ENOSPC, licence and copyleft compliance, destructive operator errors, and non-ASCII patient data
end to end.

Review notes

  • 225 open questions are addressed to the owner; several are blocking decisions (the WCAG target,
    whether load_settings should refuse unknown keys, which transports are deliberately not
    authorable from connections.toml).
  • The plan's own headline finding: messagefoundry check is invoked by no workflow, and one
    meta-test plus one ci.yml regex fix turns 36 already-written tests from dormant into
    executing. That is items 1–2 of §24's prioritised ten.
  • All traffic in the plan is synthetic and PHI-free; reports are metrics-and-metadata only.
  • Docs only — no code, no test, no CI change in this PR.

🤖 Generated with Claude Code

…three-tier gate

The umbrella test plan the two existing artifacts plug into. FEATURE-COVERAGE-PLAN.md
owns the per-subsystem coverage-gap audit and WIN2025-TEST-PLAN.md owns what only a real
Server 2025 host can prove; neither covers the product as a whole, and neither covers the
IDE, the tray, the Steps editor or the publish path at all. This states what the product
is, what has to be true before it ships, and who proves it — delegating rather than
restating, with foreign IDs prefixed FCP:/W25: because both predecessors use IDs that
collide with this plan's own (HA-20 and API-13 exist in both spaces).

Three things are load-bearing:

Row classes. Only class-T rows gate a release. A criterion of "Pass = report received" or
"Pass = a published number" cannot fail, so it cannot be a gate — those are class C
(characterisation, tracked) and class A (external assurance, gates production exposure
only). The draft had a penetration-test procurement event sitting at P0. The gate is now
196 blocking T-rows + 9 campaign-gated + 3 C + 3 A = 211 P0-marked, reconciled row-for-row
against an independent parse.

The publishing boundary. docs/BACKLOG.md is a published baseline ending at #231 and
docs/security/ is gitignored post-cutover, so a citation above the baseline or into a
withheld tree is valid evidence, not a dangling reference. An earlier pass had this
backwards and was filing withheld material as coverage defects, including one that reached
a release-gate criterion.

Two owner-named items were structurally unprovable. The tray had zero P0 rows — all eight
sat on NSSM/service/install — so the icon-state mapping, action routing, Win32 message
pump and the lying-tray case are now blocking. HA had no P0 that proved a failover works;
six now do.

Also folds in surfaces nothing owned: last_resort.py (the PHI-egress backstop),
netaddr.py (the shared allow-list decision the co-location exists to keep consistent),
pki.py, the shipped PySide6 harness GUI, disk-full/ENOSPC, licence/copyleft compliance,
destructive operator errors and non-ASCII patient data end to end.

Ships as an index plus 20 chapter files: the assembled document is 1.8 MB, past GitHub's
markdown render limit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant