[pranems]

Summary

Describe every chronological window in complete retained qualification
rounds, without changing their assessment or creating admission.

Stacked on held #251 after reviewed #263. Actual current-image Linux run
41615647439 failed four of twelve first-control comparisons. All ordinary
exact-tip CI, 66 functional checks, 200 privacy/logging checks and exact cleanup
passed independently. No later block ran. This PR does not rerun qualification,
change warmups/quotas/samples/bounds, attribute a unique cause or admit deployment.

The collector retains all 16,000 samples in 64 chronological windows, shares
the original nearest-rank estimator and receipt validation, records image
roles/input hashes, and refuses output overwrite. Smaller windows are
descriptive, not independent replications or new acceptance criteria.

Linked Defect / Plan Reference

• DELIVERY_PLAN.md defect ID: N/A; calibration evidence prerequisite for ci: qualify membership performance on isolated Linux runners #251.
• Audit doc reference: docs/ENDPOINT_SNAPSHOT_REVISION.md, Descriptive calibration prerequisite.
• ADR (if applicable): N/A; small extension of existing assessor, no new architecture.

TDD Process

• RED test written first; observed it fail for the expected reason
• GREEN: minimal implementation that turns the test green
• Refactor (if any) keeps the test green
• Permanent regression guard added (where applicable - see existing patterns in api/src/security/)

RED: AssertionError, Missing descriptive stability collector. Initial GREEN:
64/64. Independent review identified two genuine integrity gaps; both were
reproduced before correction. Current GREEN: 66/66, including eight new
numerical, actual-CLI and expected-error cases. The new tests reconstruct
unsorted observations and mutate both input files after their first read.
Negative cases cannot pass merely because an unrelated TypeError throws.

Feature / Bug-Fix Commit Checklist (standing rule)

Every behavioral change must include all that apply. Tick or strike through with N/A + reason.

• Unit Tests - service-level and controller-level coverage of the new behavior. N/A for serving services/controllers; changed Node tooling has eight new permanent contracts.
• E2E Tests - end-to-end spec under api/test/e2e/ exercising the feature through HTTP. N/A; actual CLI and retained real Linux receipt integration cover this offline tool.
• Live Integration Tests - new section in scripts/live-test.ps1 runnable against local + Docker + Azure. N/A; no HTTP or deployment behavior changes.
• Feature Documentation - dedicated doc in docs/ with architecture, RFC references, Mermaid diagrams, test coverage. Existing snapshot and engineering-pattern documents updated; no new protocol or diagram.
• INDEX.md Update - add the new doc reference to docs/INDEX.md. N/A; no new document.
• CHANGELOG.md Update - [Unreleased] entry with full test counts and feature summary
• Session & Context Updates - Session_starter.md and docs/CONTEXT_INSTRUCTIONS.md
• Version Management - bump api/package.json (and any other version refs) when behavior changed. N/A; remains a tooling prerequisite for unreleased parent 0.55.67, no new product version or lockfile rewrite.
• Response Contract Tests - API responses contain ONLY documented fields (key allowlist assertion at unit + E2E + live). N/A; serving source and response contracts unchanged.

Validation Output

node --test scripts/test/membership-performance.test.cjs scripts/test/membership-stability.test.cjs
PASS: 66 tests, zero skips or failures, after observed semantic RED.

node scripts/performance/assess-membership.cjs --describe-stability <actual-protocol> aa-baab <actual-rounds> <new-output>
PASS: descriptive execution on retained Linux run41615647439, not admission.
Independent deep equality reproduces all12 original checks, including4 failures.
Actual output-shape assertion:64 windows,16000 retained samples.
Original protocol, observations and assessment remain unchanged.

Initial-tip protected push with PREPUSH_MODE=Validate (0c9e07b2, historical):
$env:PREPUSH_MODE='Validate'
$env:DATABASE_URL='postgresql://localhost:5432/scim_codegen?schema=public'
git push --set-upstream origin fix/qualification-calibration-20261103
PASS: all 30 gates, exit 0, 1,468.2 seconds. Normal hooks enabled.
Complete membership tooling: 233 PASS, 8 explicit opt-in/retained-fixture skips,
0 failures, 241 total.
API unit: 6,254 PASS, 216 suites.
API inmemory E2E: 2,514 PASS, 42 explicit skips; 118 suites PASS, 2 skipped.
Web Vitest: 1,661 PASS, 125 files.
API/web builds and types, lint, actual-browser Mermaid render, source/ownership,
codegen/migration lint, supply chain, documentation and governance all PASS.
Post-validation worktree clean; frozen protocol and manifests/locks unchanged.

Review-fix tip d374a3d00736d65143b623dddbc5876f98b90ecd:
$env:PREPUSH_MODE='Validate'
$env:DATABASE_URL='postgresql://localhost:5432/scim_codegen?schema=public'
git push origin fix/qualification-calibration-20261103
PASS: all 30 normal protected gates, exit 0, 1154.8 seconds.
Complete tooling: 235 PASS / 8 explicit skips / 0 failures, 243 total.
API unit: 6254 PASS / 216 suites.
API inmemory E2E: 2514 PASS / 42 explicit skips, 118 passing/2 skipped suites.
Web Vitest: 1661 PASS / 125 files.
The corrected real-data report reconstructs all16000 original observations
in order and reproduces all12 original checks/four failures exactly.
Separate corrected output preserves the earlier immutable report.

The report explicitly does not estimate uncertainty, establish warmup
convergence or confer deployment authority. A prospective stabilization
study and any eventual admission-policy change remain separate reviewed
boundaries. Operator unavailable is not paid-host or gate-waiver approval.

Both independent-review findings are addressed in d374a3d: windows include
the raw unsorted samples and hashes use the exact once-read parsed buffers.
Exact-tip hosted run41625825632 passes validate and build-and-push.
Refreshed independent Copilot review at d374a3d reports no active issues;
both original threads are resolved. Its state is COMMENTED, not formal
APPROVED. This prerequisite merged normally into held #251 at
ff0b3a4. #251 remains draft, not merged
into master; the original failed qualification and deployment hold remain.

Standing Rules Acknowledgment

• No em-dash (Unicode U+2014) anywhere in the diff. Verified via Select-String -Pattern ([char]0x2014).
• No git commit --amend on history that has been pushed.
• No new credentials, tokens, or secrets committed to source. Forbidden patterns regression spec at api/src/security/forbidden-source-patterns.spec.ts will catch known anti-patterns.
• Migrations are additive-only (expand-contract for column changes). Migration linter at api/src/scripts/lint-migrations.ts enforces this. N/A; no migration change.

Destructive Migration Override (skip if not applicable)

N/A; no database or migration change.

• This PR contains a NEW destructive Prisma migration (DROP TABLE/COLUMN, ALTER COLUMN ... TYPE, RENAME, INSERT ... SELECT FROM)
• I have set ALLOW_DESTRUCTIVE_MIGRATION=1 in the CI environment for this PR
• Justification (required if box above is ticked):

  <!-- Why this destructive change is necessary, what the rollback path is, and which release window it ships in -->
  

Reviewer Hint

Diff areas worth special attention from the reviewer:

• Shared percentile extraction must preserve the original 2000-sample
  assessor exactly; actual failed-result deep equality verifies this.
• Chronological window boundaries, sample accounting, role/visit identity,
  explicit non-admission flags, immutable inputs and exclusive output.
• No causal, convergence or confidence claim from dependent request windows.
• Design disposition: accepted. Small cohesive assessor extension; no
  production dependency, framework, new lifecycle or speculative strategy.
• Self-improvement applied: numerical outcomes and expected error classes,
  not report presence or an unrelated exception.

[Copilot]

🟡 Changes recommended

The report does not retain the observations it claims to retain, and its hashes may identify different input bytes from those analyzed.

2 open issues

• Report omits the original window samples · New
• Hash the same input bytes used for analysis · New

What changed in this PR

Extends the membership performance assessor with descriptive analysis of retained qualification rounds, supporting investigation of #251’s failed same-image controls without changing acceptance criteria or authorizing deployment.

Changes:

• Adds chronological window summaries, shared nearest-rank percentiles, input hashes, and overwrite protection.
• Adds six numerical, receipt-validation, and CLI regression tests.
• Documents the failed current-image qualification and the separate need for prospective stabilization work.

File	Description
Session_starter.md	Records calibration status and deployment restrictions.
scripts/​test/​membership-stability.test.cjs	Tests window statistics, validation, and CLI output.
scripts/​performance/​assess-membership.cjs	Adds descriptive stability reporting and shares percentile calculation.
docs/​strategy/​ENGINEERING_LESSONS_AND_PATTERNS.md	Documents limits of warmup and window-based evidence.
docs/​ENDPOINT_SNAPSHOT_REVISION.md	Details calibration results, constraints, and next steps.
docs/​CONTEXT_INSTRUCTIONS.md	Updates delivery context with the qualification hold.
CHANGELOG.md	Records the tooling extension and focused test coverage.

_{Review effort: Balanced}

[Copilot]

🟢 Approval recommended

No active review issues remain for the offline stability-reporting change.

No open issues 🎉

2 resolved since last review

• Hash the same input bytes used for analysis
• Report omits the original window samples

_{Review effort: Balanced}