From bce49cbe7c3376434d3e3ce9209c6b3b32021718 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 01:21:31 +0000 Subject: [PATCH 01/45] Queue in the inbox: ContentParams expressions fail CE1613 on mxcli v0.24 learned-mdl-preflight row 12's toString(Attr) fix no longer builds; check refuses function calls but passes if/then/else. Five-form probe attached. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Pm41sykqDGcvioGzsY5H2N --- CHANGELOG.md | 1 + ...26-09-26-contentparams-expressions-v024.md | 44 +++++++++++++++++++ 2 files changed, 45 insertions(+) create mode 100644 contrib/inbox/2026-09-26-contentparams-expressions-v024.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 30ebae9..77833ec 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,7 @@ three commits past it), and a bug report can name a release instead of a sha nob Sections dated before 2026-09-19 predate the cycle and stay as they are. ## Unreleased +- process(contrib): **`learned-mdl-preflight.md` row 12 is outdated on mxcli v0.24 — queued in `contrib/inbox/`.** A dynamictext's ContentParams builds only with a bare attribute or a quoted literal: `toString(Attr)` (the row's own fix) and `if … then … else` both fail CE1613 at mxbuild, and `check --references` refuses the first but passes the second. Five-form probe on mx check 11.13.0. From a card-disbursement requirements-driven build (gallery row 2.14a). - learn(skills/learned-file-upload-widget.md): **a file upload mxcli can author, with proof that it uploads.** The Mendix File Uploader 2.5.0 bound to a `System.FileDocument` specialisation: the MDL shape (entities, grants, create/delete microflows, advanced formats), which upload widgets mxcli cannot author (classic FileManager, PDS uploader), the two traps with workarounds (a simple-mode `allowedfileformat` passes exec and fails `mx check` with CE0463; an uploader DESCRIBE will not re-exec, `exposes 2 datasources`), and the six-step upload instrument. Field run on stock v0.24.0: the section-4 MDL taken verbatim from the skill gave `mx check` 0 errors, 2/2 files stored, 2/2 downloads sha256-equal, `.csv` rejected with 0 rows; both traps reproduce unchanged on v0.24.0 — a Mendix app-rebuild project - new(skills/mendix-best-practices-index.md): **one row per Mendix best-practice area: the Mendix docs page, the bundled `assess-quality` section, the toolkit skill that applies it before the write, and the `mxcli lint` rule that catches it after exec.** An index, not a copy — the practice text stays on the Mendix pages (17 URLs verified HTTP 200 on 2026-09-25) and in the mxcli-bundled skill; Mendix's own Best Practice Recommender rules (MXP001–016) anchor the performance rows, and four rows say out loud that no lint rule exists and the preflight checklist is the only check. Routed `all` agents, stages 3/5/6, group reference — Maurits Visser - learn(skills/microflow-preflight.md, agents/mdl-agent.md): **a microflow now gets a tier before any MDL — Simple, Guided or Split-first — and Split-first means a posted split plan (thin orchestrator + one `SUB_` per responsibility, with signatures) that the user confirms first.** Prompted by a colleague's session refusing a long microflow as "too difficult" while the same task went through on a stronger model: the piece was too big, not the task. mdl-agent gains two rules — never hand back "too difficult", hand back the split plan; escalate one failing `SUB_` by name after one retry, never the whole script. Also records the mxcli team's answer on positioning: a standalone `mxcli layout` command — which on v0.24.0 and upstream main (2026-09-25) arranges domain models only (`--dry-run` on a scratch copy of a PoC model: 10 entity moves, no microflow), so the no-`@position` rule and the if-branch workaround stand until a microflow mode ships; noted in the bug ledger and `learned-microflow-patterns.md` — Maurits Visser diff --git a/contrib/inbox/2026-09-26-contentparams-expressions-v024.md b/contrib/inbox/2026-09-26-contentparams-expressions-v024.md new file mode 100644 index 0000000..d1cb585 --- /dev/null +++ b/contrib/inbox/2026-09-26-contentparams-expressions-v024.md @@ -0,0 +1,44 @@ +# ContentParams on mxcli v0.24: only a bare attribute or a quoted literal builds — preflight row 12 is outdated + +**From:** card-disbursement requirements-driven build (gallery row 2.14a) +**Date:** 2026-09-26 +**Kind:** bug +**Field evidence:** two execs of a list-view snippet failed CE1613 at mxbuild with `mxcli check --references` green; then a five-form probe on a scratch copy of the model, `mx check` 11.13.0, mxcli v0.24.0 (f18c307) — output below. +**Proposed target:** `skills/learned-mdl-preflight.md` row 12; `skills/learned-detection-gaps.md` line 47 (the "customContent column" scope is too narrow — it is any dynamictext in an object context); `bug-logs/mxcli-bugs.md` as a BUG-23 variant (BUG-23's v0.21 retest covered the `$currentObject/` prefix only) + +--- + +Finding (reproduced, not a hypothesis): + +| ContentParams value | `mxcli check --references` | mxbuild (`mx check`) | +|---|---|---| +| `CaseRef` (bare attribute) | clean | clean | +| `'literal'` | clean | clean | +| `toString(CreditLimit)` | **error**: "looks like an expression, and MDL cannot author an expression-typed template parameter yet — an unquoted value is stored as an attribute" | CE1613 | +| `toString($currentObject/CreditLimit)` | **error** (same) | CE1613 | +| `if IsUrgent then 'x' else 'y'` | **clean — check is blind** | CE1613 | + +Verbatim mxbuild: + +``` +[error] [CE1613] "The selected attribute 'StyleGallery.SampleCase.toString(CreditLimit)' no longer exists." at Text 'txtA' +[error] [CE1613] "The selected attribute 'StyleGallery.SampleCase.toString($currentObject/CreditLimit)' no longer exists." at Text 'txtB' +[error] [CE1613] "The selected attribute 'StyleGallery.SampleCase.ifIsUrgentthen'x'else'y'' no longer exists." at Text 'txtE' +``` + +`describe snippet` prints the if-form with its whitespace stripped (`{1} = ifIsUrgentthen'x'else'y'`) — +the one visible tell before mxbuild. + +Source reading (v0.24.0): `mdl/visitor/visitor_page_v3.go:1232` takes `expr.GetText()` (ANTLR joins the +tokens without whitespace); `mdl/executor/cmd_pages_builder_v3_widgets.go:836-846` sends every +unquoted value to `resolveTemplateAttributePathFull`. So there is no expression-typed parameter at all; +check's refusal is a heuristic that keys on the parenthesis. + +What is now wrong in the toolkit: +- `learned-mdl-preflight.md` row 12 — "Function calls themselves are fine; the prefix is the trigger" and + the fix `ContentParams: [{1} = toString(Attr)]`. On v0.24 check refuses that fix and mxbuild fails it. +- `learned-detection-gaps.md` line 47 — scoped to "inside a customContent column"; the gap is any + dynamictext in an object context, and check now catches the parenthesised forms but not `if`. + +Workaround used: make the per-row label data (a seeded/derived String attribute bound as a bare +attribute); tone variants go in `DynamicClasses`, which does take an expression. From e2ecb980769450bac78e13c661863bb804e5de5e Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 07:26:52 +0000 Subject: [PATCH 02/45] fix(coverage-preflight): LEVEL 1 measures per-BRD ledger directories find_ledger() only tested for files, so a module that splits its ledger into coverage-ledger/.md with an index file beside it had every BRD measured against the index: all BRDs UNCLAIMED while coverage-check-all.sh reported the same files clean. The directory form now wins over the index file. Each BRD is measured against its own file, the denominator (N of M BRDs measured) is stated, BRDs owned by another module are named rather than measured, and an orphan ledger file or a directory matching no BRD is a FAULT (rc 2), never a clean 0. An empty directory falls through to the file form. Fixture: tests/wave2/test-coverage-preflight-ledger-dir.sh (10 assertions, 7 red against the previous script). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Pm41sykqDGcvioGzsY5H2N --- CHANGELOG.md | 1 + project-bin/coverage-preflight.sh | 78 ++++++++++-- .../test-coverage-preflight-ledger-dir.sh | 119 ++++++++++++++++++ 3 files changed, 191 insertions(+), 7 deletions(-) create mode 100755 tests/wave2/test-coverage-preflight-ledger-dir.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index 77833ec..086f824 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,7 @@ three commits past it), and a bug report can name a release instead of a sha nob Sections dated before 2026-09-19 predate the cycle and stay as they are. ## Unreleased +- fix(project-bin/coverage-preflight.sh): **LEVEL 1 now finds the per-BRD ledger directory `architecture/modules//coverage-ledger/.md` and measures each BRD against its own file.** Before, discovery tested for files only: a module that splits its ledger per BRD (the `coverage-check-all.sh` convention) keeps `coverage-ledger.md` beside the directory as an index, so the preflight found the index and ran every BRD in the project against it — `verify-module.sh`'s coverage rung read 9 of 9 BRDs UNCLAIMED (2,300+ leaves) for a module that owns one, while `coverage-check-all.sh` reported all 9 clean from the same files. The directory now wins over the file beside it; the run states its denominator ("1 of 9 BRD(s) measured", the rest named as another module's), a ledger file named after no BRD faults, a directory matching no BRD faults instead of reading clean, and an empty directory falls through to the file shapes. Field run on all 4 modules of that project: 9 of 9 BRDs measured once each, all clean. Fixture `tests/wave2/test-coverage-preflight-ledger-dir.sh`, 10 assertions, 7 of them red against the previous script — a card-disbursement requirements-driven build (build-plan row 3.9) - process(contrib): **`learned-mdl-preflight.md` row 12 is outdated on mxcli v0.24 — queued in `contrib/inbox/`.** A dynamictext's ContentParams builds only with a bare attribute or a quoted literal: `toString(Attr)` (the row's own fix) and `if … then … else` both fail CE1613 at mxbuild, and `check --references` refuses the first but passes the second. Five-form probe on mx check 11.13.0. From a card-disbursement requirements-driven build (gallery row 2.14a). - learn(skills/learned-file-upload-widget.md): **a file upload mxcli can author, with proof that it uploads.** The Mendix File Uploader 2.5.0 bound to a `System.FileDocument` specialisation: the MDL shape (entities, grants, create/delete microflows, advanced formats), which upload widgets mxcli cannot author (classic FileManager, PDS uploader), the two traps with workarounds (a simple-mode `allowedfileformat` passes exec and fails `mx check` with CE0463; an uploader DESCRIBE will not re-exec, `exposes 2 datasources`), and the six-step upload instrument. Field run on stock v0.24.0: the section-4 MDL taken verbatim from the skill gave `mx check` 0 errors, 2/2 files stored, 2/2 downloads sha256-equal, `.csv` rejected with 0 rows; both traps reproduce unchanged on v0.24.0 — a Mendix app-rebuild project - new(skills/mendix-best-practices-index.md): **one row per Mendix best-practice area: the Mendix docs page, the bundled `assess-quality` section, the toolkit skill that applies it before the write, and the `mxcli lint` rule that catches it after exec.** An index, not a copy — the practice text stays on the Mendix pages (17 URLs verified HTTP 200 on 2026-09-25) and in the mxcli-bundled skill; Mendix's own Best Practice Recommender rules (MXP001–016) anchor the performance rows, and four rows say out loud that no lint rule exists and the preflight checklist is the only check. Routed `all` agents, stages 3/5/6, group reference — Maurits Visser diff --git a/project-bin/coverage-preflight.sh b/project-bin/coverage-preflight.sh index bee29c5..329ba08 100755 --- a/project-bin/coverage-preflight.sh +++ b/project-bin/coverage-preflight.sh @@ -37,12 +37,23 @@ # engine's own verdict through unchanged; every other level is strictly *less* severe than # today's FAULT. Every level states what it knows, what it does not, and what would upgrade it. # -# Ledger path shapes — all three are read, the first is canonical: +# Ledger path shapes — all four are read, in this order: +# architecture/modules//coverage-ledger/ one ledger per BRD, .md each +# (project-bin/coverage-check-all.sh's form) # architecture/modules//coverage-ledger.md canonical (per-module, scales) # architecture/modules/-coverage-ledger.md flat module layout (PROJECT-A's shape) # architecture/coverage-ledger.md single-BRD project (sanctioned by # skills/coverage-ledger.md's header) -# When none exists the message names all three, so an operator can act without reading source. +# When none exists the message names all four, so an operator can act without reading source. +# +# The directory wins over the .md beside it: a module that splits its ledger per BRD keeps +# coverage-ledger.md as an index, and measuring BRDs against the index is a false red. Real +# case (card-disbursement requirements-driven build, 2026-09-26): 9 BRDs, 4 module ledger +# directories holding 9 per-BRD files, every one clean under coverage-check-all.sh — and this +# script found each module's index file, ran all 9 BRDs against it, and printed 9 of 9 +# UNCLAIMED (2,300+ leaves) for a module that owns one BRD. In the directory form each BRD is +# measured against its own .md; a BRD with no file there belongs to another module and +# is counted, named, and not measured. # # Usage: # project-bin/coverage-preflight.sh [--summary] [--module ] [ []] @@ -130,17 +141,30 @@ find_engine() { # --------------------------------------------------------------------------- # Discovery # --------------------------------------------------------------------------- +LEDGER_DIR="" LEDGER_CANON="" LEDGER_FLAT="" LEDGER_PROJECT="architecture/coverage-ledger.md" if [ -n "$MODULE" ]; then + LEDGER_DIR="architecture/modules/$MODULE/coverage-ledger" LEDGER_CANON="architecture/modules/$MODULE/coverage-ledger.md" LEDGER_FLAT="architecture/modules/$MODULE-coverage-ledger.md" fi +# A ledger directory counts only when it holds at least one per-BRD file — an empty directory +# is not a ledger, and falling through to the file shapes is the honest reading of it. +ledger_dir_ok() { + [ -n "$1" ] && [ -d "$1" ] || return 1 + ls "$1"/*.md >/dev/null 2>&1 +} + find_ledger() { local c - for c in "$ARG_LEDGER" "${LEDGER_FILE:-}" "$LEDGER_CANON" "$LEDGER_FLAT" "$LEDGER_PROJECT"; do + for c in "$ARG_LEDGER" "${LEDGER_FILE:-}"; do + [ -n "$c" ] && { [ -f "$c" ] || ledger_dir_ok "$c"; } && { printf '%s\n' "$c"; return 0; } + done + ledger_dir_ok "$LEDGER_DIR" && { printf '%s\n' "$LEDGER_DIR"; return 0; } + for c in "$LEDGER_CANON" "$LEDGER_FLAT" "$LEDGER_PROJECT"; do [ -n "$c" ] && [ -f "$c" ] && { printf '%s\n' "$c"; return 0; } done return 1 @@ -228,6 +252,7 @@ WHERE="${MODULE:-this project}" # The paths tried, printed verbatim so "not found" is actionable rather than a riddle. print_paths_tried() { echo " looked for a ledger at:" + [ -n "$LEDGER_DIR" ] && printf ' %-50s (one .md per BRD)\n' "$LEDGER_DIR/" [ -n "$LEDGER_CANON" ] && printf ' %-50s (canonical, per-module)\n' "$LEDGER_CANON" [ -n "$LEDGER_FLAT" ] && printf ' %-50s (flat module layout)\n' "$LEDGER_FLAT" printf ' %-50s (single-BRD project)\n' "$LEDGER_PROJECT" @@ -392,10 +417,49 @@ if [ "$NBRD" -eq 0 ]; then fi RC=0 -for b in $BRDS; do - # shellcheck disable=SC2086 - "$ENGINE" $ENGINE_ARGS "$b" "$LEDGER" || RC=$? -done +if [ -d "$LEDGER" ]; then + # Directory form: each BRD against its own .md. The denominator is stated both ways — + # BRDs measured here, and BRDs that have no ledger here (another module's) — so "1 measured" + # can never be read as "1 exists". + NMEAS=0; OTHERS="" + for b in $BRDS; do + bid="$(basename "$b" .brd.json)" + if [ -f "$LEDGER/$bid.md" ]; then + NMEAS=$((NMEAS + 1)) + # shellcheck disable=SC2086 + "$ENGINE" $ENGINE_ARGS "$b" "$LEDGER/$bid.md" || RC=$? + else + OTHERS="$OTHERS $bid" + fi + done + NOTHERS=0; [ -n "$OTHERS" ] && NOTHERS=$(printf '%s\n' $OTHERS | grep -c .) + # A ledger file named after no BRD is only knowable when every BRD was discovered — with a + # single BRD passed in, the directory's other files are simply not this run's business. + ORPHANS="" + if [ -z "$ARG_BRD" ] && [ -z "${BRD_FILE:-}" ]; then + for l in "$LEDGER"/*.md; do + lid="$(basename "$l" .md)"; hit=0 + for b in $BRDS; do [ "$(basename "$b" .brd.json)" = "$lid" ] && { hit=1; break; }; done + [ "$hit" -eq 1 ] || ORPHANS="$ORPHANS $lid" + done + fi + echo "" + echo " per-BRD ledgers: $NMEAS of $NBRD BRD(s) measured against $LEDGER/.md;" + echo " $NOTHERS BRD(s) have no ledger here (another module's, not measured):${OTHERS:- none}" + if [ -n "$ORPHANS" ]; then + echo " FAULT: ledger file(s) with no matching BRD:$ORPHANS — named after a BRD id that does not exist." >&2 + [ "$RC" -lt 2 ] && RC=2 + fi + if [ "$NMEAS" -eq 0 ]; then + echo "FAULT: $LEDGER holds no ledger matching any of the $NBRD BRD(s) — cannot evaluate, which is not a pass." >&2 + RC=2 + fi +else + for b in $BRDS; do + # shellcheck disable=SC2086 + "$ENGINE" $ENGINE_ARGS "$b" "$LEDGER" || RC=$? + done +fi if [ "$LEVEL" -eq 2 ]; then echo "" diff --git a/tests/wave2/test-coverage-preflight-ledger-dir.sh b/tests/wave2/test-coverage-preflight-ledger-dir.sh new file mode 100755 index 0000000..ab215b3 --- /dev/null +++ b/tests/wave2/test-coverage-preflight-ledger-dir.sh @@ -0,0 +1,119 @@ +#!/usr/bin/env bash +# Usage: bash test-coverage-preflight-ledger-dir.sh [path-to-coverage-preflight.sh] +# +# Fixture for project-bin/coverage-preflight.sh's ledger discovery — the per-BRD directory form +# `architecture/modules//coverage-ledger/.md` (project-bin/coverage-check-all.sh's +# convention) at LEVEL 1. +# +# The defect (card-disbursement requirements-driven build, 2026-09-26): a module that splits its +# ledger per BRD keeps coverage-ledger.md beside the directory as an index. find_ledger() only +# tested for files, found the index, and ran EVERY BRD against it — 9 of 9 BRDs UNCLAIMED for a +# module that owns one, while coverage-check-all.sh reported all 9 clean from the same files. +# Case 1 below is that project's shape in miniature, including the index file. +# +# The BRDs and ledgers here are SYNTHETIC and hand-written. That is deliberate and allowed: the +# field-proof rule's "golden input is captured" governs parsers of tool output, and nothing below +# parses tool output — it lays out files and asserts which ledger the script measures against. +# The field run that motivated it is cited in CHANGELOG.md. + +set -uo pipefail + +SUT="${1:-}" +[ -z "$SUT" ] && SUT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)/project-bin/coverage-preflight.sh" +if [ ! -f "$SUT" ]; then + echo "SKIP: subject not found at $SUT" + echo "SCORE: 0/0 — nothing to test" + exit 0 +fi +command -v jq >/dev/null 2>&1 || { echo "SKIP: jq not installed (the engine needs it)"; exit 0; } +export MXTK_ROOT="${MXTK_ROOT:-$(cd "$(dirname "$SUT")/.." && pwd)}" + +PASS=0; FAIL=0 +ok() { PASS=$((PASS+1)); echo " ok — $1"; } +bad() { FAIL=$((FAIL+1)); echo " FAIL — $1"; [ -n "${2:-}" ] && echo " got: $2"; } + +WORK="$(mktemp -d "${TMPDIR:-/tmp}/cov-ledger-dir.XXXXXX")" +trap 'rm -rf "$WORK"' EXIT + +# Two BRDs, 4 leaves each; Orders owns FA01, Billing owns FB02. +mk_project() { + local r="$1" + mkdir -p "$r/analysis/src/knowledge-base/brd" "$r/architecture/modules/Orders" "$r/architecture/modules/Billing" + printf '{"id":"FA01","title":"Alpha","useCases":[{"id":"UC1","name":"Place order"}]}\n' \ + > "$r/analysis/src/knowledge-base/brd/FA01-alpha.brd.json" + printf '{"id":"FB02","title":"Beta","useCases":[{"id":"UC1","name":"Send invoice"}]}\n' \ + > "$r/analysis/src/knowledge-base/brd/FB02-beta.brd.json" +} +ledger() { # ledger [omit-usecase] + { + echo "## BUILDABLE"; echo "" + echo "| pointer | type | title | slice | writeMode | acceptance | status |" + echo "|---|---|---|---|---|---|---|" + [ "${2:-}" = "omit-usecase" ] || echo "| \`/useCases/0/*\` (2) | usecase | UC1 | 1.1 | test | e2e | not-built |" + echo ""; echo "## NON-BUILDABLE"; echo "" + echo "| pointer | category | reason |" + echo "|---|---|---|" + echo "| \`/id\` | provenance | metadata |" + echo "| \`/title\` | provenance | metadata |" + } > "$1" +} +run() { PROJECT_ROOT="$1" bash "$SUT" --summary --module "$2" >"$WORK/out" 2>&1; echo $?; } + +# --- 1. directory form beside an index file: each BRD against its own ledger ------------------ +P="$WORK/p1"; mk_project "$P" +mkdir -p "$P/architecture/modules/Orders/coverage-ledger" +ledger "$P/architecture/modules/Orders/coverage-ledger/FA01-alpha.md" +printf '# Coverage ledger — Orders (index)\n\n- FA01 → coverage-ledger/FA01-alpha.md\n' \ + > "$P/architecture/modules/Orders/coverage-ledger.md" +rc=$(run "$P" Orders) +[ "$rc" -eq 0 ] && ok "dir form beside an index file is clean (rc 0)" || bad "dir form not clean" "rc=$rc $(tr '\n' ' ' < "$WORK/out" | cut -c1-300)" +grep -q 'ledger: architecture/modules/Orders/coverage-ledger$' "$WORK/out" \ + && ok "the directory is chosen over the index file" || bad "index file measured instead of the directory" +grep -q '1 of 2 BRD(s) measured' "$WORK/out" \ + && ok "denominator stated: 1 of 2 BRDs measured" || bad "no measured-of denominator" +grep -q 'no ledger here.*FB02-beta' "$WORK/out" \ + && ok "the other module's BRD is named, not measured" || bad "other module's BRD not named" +if grep -qE 'UNCLAIMED: +[1-9]' "$WORK/out"; then + bad "a BRD was measured against the wrong ledger (the 2026-09-26 false red)" +else + ok "no BRD measured against the index" +fi + +# --- 2. a finding in a per-BRD ledger passes through as rc 1 --------------------------------- +ledger "$P/architecture/modules/Orders/coverage-ledger/FA01-alpha.md" omit-usecase +rc=$(run "$P" Orders) +[ "$rc" -eq 1 ] && ok "an UNCLAIMED leaf in a per-BRD ledger reports rc 1" || bad "finding did not pass through" "rc=$rc" + +# --- 3. a ledger file named after no BRD is a fault ------------------------------------------ +ledger "$P/architecture/modules/Orders/coverage-ledger/FA01-alpha.md" +ledger "$P/architecture/modules/Orders/coverage-ledger/FZ99-ghost.md" +rc=$(run "$P" Orders) +[ "$rc" -eq 2 ] && grep -q 'FZ99-ghost' "$WORK/out" \ + && ok "orphan ledger file faults (rc 2) and is named" || bad "orphan ledger not faulted" "rc=$rc" +rm -f "$P/architecture/modules/Orders/coverage-ledger/FZ99-ghost.md" + +# --- 4. a directory matching no BRD cannot evaluate ------------------------------------------ +P4="$WORK/p4"; mk_project "$P4" +mkdir -p "$P4/architecture/modules/Billing/coverage-ledger" +ledger "$P4/architecture/modules/Billing/coverage-ledger/FA01-alpha.md" +mv "$P4/analysis/src/knowledge-base/brd/FA01-alpha.brd.json" "$P4/analysis/src/knowledge-base/brd/FC03-gamma.brd.json" +rc=$(run "$P4" Billing) +[ "$rc" -eq 2 ] && ok "a directory with no ledger for any BRD faults (rc 2), never a clean 0" || bad "unmatched directory read as clean" "rc=$rc" + +# --- 5. the single-file form still works (regression) ---------------------------------------- +P5="$WORK/p5"; mk_project "$P5" +ledger "$P5/architecture/modules/Orders/coverage-ledger.md" +rm "$P5/analysis/src/knowledge-base/brd/FB02-beta.brd.json" +rc=$(run "$P5" Orders) +[ "$rc" -eq 0 ] && grep -q 'ledger: architecture/modules/Orders/coverage-ledger.md' "$WORK/out" \ + && ok "single-file ledger still measured (rc 0)" || bad "single-file form regressed" "rc=$rc" + +# --- 6. an empty directory is not a ledger: falls through to the file ------------------------ +mkdir -p "$P5/architecture/modules/Orders/coverage-ledger" +rc=$(run "$P5" Orders) +[ "$rc" -eq 0 ] && grep -q 'ledger: architecture/modules/Orders/coverage-ledger.md' "$WORK/out" \ + && ok "empty directory falls through to the file" || bad "empty directory shadowed the file" "rc=$rc" + +echo "" +echo "PASS=$PASS FAIL=$FAIL" +[ "$FAIL" -eq 0 ] From d0535501d221532bd20d1a72ebf5f8964ce975bd Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 07:45:48 +0000 Subject: [PATCH 03/45] fix(design-audit): defined-class corpus reads rules nested in at-rules classesInCss harvested selectors at brace depth 0 only, so a class Atlas defines only inside @media (spacing-outer-bottom-large, the class design-spacing.md prescribes) was reported invented/unmatched on every page using it. Comments and strings are now stripped and the prelude before every non-at-rule '{' is harvested. Field run on a real mxbuild theme.compiled.css: 2768 -> 3349 classes. New fixture over a verbatim capture: 7/7 green, 4 red on the old code. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Pm41sykqDGcvioGzsY5H2N --- CHANGELOG.md | 1 + project-tests/e2e/design-audit.js | 30 ++++--- .../design-audit-css/theme-excerpt.css | 45 +++++++++++ tests/wave2/test-design-audit-css-corpus.sh | 81 +++++++++++++++++++ 4 files changed, 146 insertions(+), 11 deletions(-) create mode 100644 tests/wave2/fixtures/design-audit-css/theme-excerpt.css create mode 100755 tests/wave2/test-design-audit-css-corpus.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index 086f824..9504cc7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,7 @@ three commits past it), and a bug report can name a release instead of a sha nob Sections dated before 2026-09-19 predate the cycle and stay as they are. ## Unreleased +- fix(project-tests/e2e/design-audit.js): **the defined-class corpus now reads rules nested in `@media` / `@supports` / `@layer`, not just depth 0.** Before, `classesInCss` harvested selectors only at brace depth 0. Atlas emits `spacing-outer-bottom-large` only inside three breakpoint `@media` blocks, so rungs 6-7 called it `invented-class` / `unmatched-class` on every page that used it — the class `skills/design-spacing.md` prescribes for section rhythm — and the run ended INSTRUMENT FAULT. The harvester now strips comments and string literals, then takes the selector prelude before every `{` that is not an at-rule. Field run on a real mxbuild `theme.compiled.css` (Mendix 11.13.0, Atlas): corpus 2768 → 3349 classes, the spacing class now present. New fixture `tests/wave2/test-design-audit-css-corpus.sh` over a verbatim capture of that stylesheet (`@media`, `@supports`, `@font-face` url, `@keyframes`): 7 assertions, 7 green on the fix, 4 red against the previous script. From a card-disbursement requirements-driven build (build-plan row 3.9). - fix(project-bin/coverage-preflight.sh): **LEVEL 1 now finds the per-BRD ledger directory `architecture/modules//coverage-ledger/.md` and measures each BRD against its own file.** Before, discovery tested for files only: a module that splits its ledger per BRD (the `coverage-check-all.sh` convention) keeps `coverage-ledger.md` beside the directory as an index, so the preflight found the index and ran every BRD in the project against it — `verify-module.sh`'s coverage rung read 9 of 9 BRDs UNCLAIMED (2,300+ leaves) for a module that owns one, while `coverage-check-all.sh` reported all 9 clean from the same files. The directory now wins over the file beside it; the run states its denominator ("1 of 9 BRD(s) measured", the rest named as another module's), a ledger file named after no BRD faults, a directory matching no BRD faults instead of reading clean, and an empty directory falls through to the file shapes. Field run on all 4 modules of that project: 9 of 9 BRDs measured once each, all clean. Fixture `tests/wave2/test-coverage-preflight-ledger-dir.sh`, 10 assertions, 7 of them red against the previous script — a card-disbursement requirements-driven build (build-plan row 3.9) - process(contrib): **`learned-mdl-preflight.md` row 12 is outdated on mxcli v0.24 — queued in `contrib/inbox/`.** A dynamictext's ContentParams builds only with a bare attribute or a quoted literal: `toString(Attr)` (the row's own fix) and `if … then … else` both fail CE1613 at mxbuild, and `check --references` refuses the first but passes the second. Five-form probe on mx check 11.13.0. From a card-disbursement requirements-driven build (gallery row 2.14a). - learn(skills/learned-file-upload-widget.md): **a file upload mxcli can author, with proof that it uploads.** The Mendix File Uploader 2.5.0 bound to a `System.FileDocument` specialisation: the MDL shape (entities, grants, create/delete microflows, advanced formats), which upload widgets mxcli cannot author (classic FileManager, PDS uploader), the two traps with workarounds (a simple-mode `allowedfileformat` passes exec and fails `mx check` with CE0463; an uploader DESCRIBE will not re-exec, `exposes 2 datasources`), and the six-step upload instrument. Field run on stock v0.24.0: the section-4 MDL taken verbatim from the skill gave `mx check` 0 errors, 2/2 files stored, 2/2 downloads sha256-equal, `.csv` rejected with 0 rows; both traps reproduce unchanged on v0.24.0 — a Mendix app-rebuild project diff --git a/project-tests/e2e/design-audit.js b/project-tests/e2e/design-audit.js index 38ea69c..ef2c72f 100644 --- a/project-tests/e2e/design-audit.js +++ b/project-tests/e2e/design-audit.js @@ -95,22 +95,30 @@ const nowIso = () => new Date().toISOString(); // ============================================================================ // 1. CSS — which classes are actually DEFINED, and where // ============================================================================ -// Selector-position only: we walk the file tracking brace depth and harvest class -// tokens from the text that PRECEDES an opening brace. Harvesting the whole file -// would pick up `content: ".foo"` and url fragments and quietly define classes that -// no rule ever declares — which turns the invented-class check into decoration. +// Selector-position only: we harvest class tokens from the text that PRECEDES an opening +// brace. Harvesting the whole file would pick up `content: ".foo"` and url fragments and +// quietly define classes that no rule ever declares — which turns the invented-class +// check into decoration. Strings are blanked first and `;` ends a declaration, so a +// declaration value never reaches a harvest. +// +// Every depth, not only depth 0: a rule inside `@media` / `@supports` / `@layer` defines +// its class just as much. Harvesting depth 0 alone called Atlas's `spacing-outer-bottom-large` +// "invented" on every page — Atlas emits it ONLY inside three breakpoint @media blocks — while +// design-spacing.md prescribes that exact class (card-disbursement build, 2026-09-26). An +// at-rule's own prelude (`@media (min-width: 992px)`) names no class and is skipped. function classesInCss(text) { const out = new Set(); - const src = text.replace(/\/\*[\s\S]*?\*\//g, ''); - let depth = 0, buf = ''; + const src = text.replace(/\/\*[\s\S]*?\*\//g, '') + .replace(/"(?:[^"\\\n]|\\.)*"|'(?:[^'\\\n]|\\.)*'/g, '""'); + let buf = ''; for (let i = 0; i < src.length; i++) { const c = src[i]; if (c === '{') { - if (depth === 0) harvest(buf, out); - buf = ''; depth++; - } else if (c === '}') { - depth = Math.max(0, depth - 1); buf = ''; - } else if (depth === 0) { + if (!/^\s*@/.test(buf)) harvest(buf, out); + buf = ''; + } else if (c === '}' || c === ';') { + buf = ''; + } else { buf += c; if (buf.length > 8000) buf = buf.slice(-4000); } diff --git a/tests/wave2/fixtures/design-audit-css/theme-excerpt.css b/tests/wave2/fixtures/design-audit-css/theme-excerpt.css new file mode 100644 index 0000000..436d80c --- /dev/null +++ b/tests/wave2/fixtures/design-audit-css/theme-excerpt.css @@ -0,0 +1,45 @@ +/* VERBATIM excerpts of a real deployment/web/theme.compiled.css (mxbuild output, Mendix 11.13.0, + Atlas Core theme) captured 2026-09-26 for tests/wave2/test-design-audit-css-corpus.sh. + Not hand-written: each block below is copied byte-for-byte from that file. */ +@media (max-width: 767px) { + .spacing-outer-bottom-large { + margin-bottom: var(--m-spacing-large) !important; + } +} + +@media (min-width: 768px) { + .spacing-outer-bottom-large { + margin-bottom: var(--t-spacing-large) !important; + } +} + +@media (min-width: 992px) { + .spacing-outer-bottom-large { + margin-bottom: var(--spacing-large) !important; + } +} + +@font-face { + font-family: "Atlas_Core$Atlas_Styling"; + src: url("./fonts/Atlas_Core$Atlas_Styling.ttf") format("truetype"); +} + +@keyframes placeholderGradient { + 0% { + background-position: 100px 0; + } + 100% { + background-position: -100px 0; + } +} + +.mx-datagrid-table-resizing th, +.mx-datagrid-table-resizing td { + cursor: col-resize !important; +} + +@supports (padding-bottom: env(safe-area-inset-bottom)) { + .mx-menubar { + padding-bottom: env(safe-area-inset-bottom); + } +} diff --git a/tests/wave2/test-design-audit-css-corpus.sh b/tests/wave2/test-design-audit-css-corpus.sh new file mode 100755 index 0000000..df7b54f --- /dev/null +++ b/tests/wave2/test-design-audit-css-corpus.sh @@ -0,0 +1,81 @@ +#!/usr/bin/env bash +# Usage: bash test-design-audit-css-corpus.sh [path-to-design-audit.js] +# +# Fixture for project-tests/e2e/design-audit.js's defined-class corpus (classesInCss) — the set +# every rung-6/7 class verdict (invented, unmatched, never-promoted) is measured against. +# +# The defect (card-disbursement requirements-driven build, 2026-09-26): the harvester read +# selectors at brace depth 0 only. Atlas emits `spacing-outer-bottom-large` ONLY inside three +# breakpoint @media blocks, so the audit called it "invented" on every page that used it — the +# very class skills/design-spacing.md prescribes for section rhythm. Same blind spot for any +# rule under @supports / @layer. +# +# Golden input is a VERBATIM capture of a real mxbuild theme.compiled.css +# (fixtures/design-audit-css/theme-excerpt.css — header says where from). It carries the real +# traps next to the defect: a url("./fonts/….ttf") in @font-face, @keyframes percent selectors, +# a multi-line depth-0 selector list. The two synthetic cases at the end are marked as such: +# they guard declaration values (quoted dots, a brace inside a string) that the capture lacks. + +set -uo pipefail + +SUT="${1:-}" +[ -z "$SUT" ] && SUT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)/project-tests/e2e/design-audit.js" +if [ ! -f "$SUT" ]; then + echo "SKIP: subject not found at $SUT" + echo "SCORE: 0/0 — nothing to test" + exit 0 +fi +command -v node >/dev/null 2>&1 || { echo "SKIP: node not installed"; exit 0; } +GOLDEN="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/fixtures/design-audit-css/theme-excerpt.css" +[ -f "$GOLDEN" ] || { echo "FAIL — golden capture missing: $GOLDEN"; exit 1; } + +# design-audit.js runs on load, so lift the two functions out of its text instead of requiring it. +OUT="$(node - "$SUT" "$GOLDEN" <<'EOF' +const fs = require('fs'); +const [sut, golden] = process.argv.slice(2); +const js = fs.readFileSync(sut, 'utf8'); +const lift = (name) => { + const i = js.indexOf(`function ${name}(`); + if (i < 0) { console.log(`LIFT-FAIL ${name}`); process.exit(3); } + let d = 0, j = js.indexOf('{', i); + for (; j < js.length; j++) { if (js[j] === '{') d++; else if (js[j] === '}' && --d === 0) break; } + return js.slice(i, j + 1); +}; +const classesInCss = new Function(`${lift('harvest')}\n${lift('classesInCss')}\nreturn classesInCss;`)(); +const show = (label, text) => console.log(`${label} ${[...classesInCss(text)].sort().join(' ')}`); +show('GOLDEN', fs.readFileSync(golden, 'utf8')); +// SYNTHETIC — declaration values the golden capture does not carry. +show('SYN-QUOTED', '.real { content: ".not-a-class"; background: url(\'img/x.png\'); }'); +show('SYN-BRACE', '.a { content: "{"; } .b { color: red; }'); +EOF +)" +rc=$? +[ "$rc" -eq 0 ] || { echo "FAIL — harness could not load classesInCss from $SUT (rc=$rc)"; echo "$OUT"; exit 1; } + +PASS=0; FAIL=0 +ok() { PASS=$((PASS+1)); echo " ok — $1"; } +bad() { FAIL=$((FAIL+1)); echo " FAIL — $1"; [ -n "${2:-}" ] && echo " got: $2"; } +line() { printf '%s\n' "$OUT" | sed -n "s/^$1 //p"; } +has() { printf ' %s ' "$(line "$1")" | grep -q " $2 "; } + +G="$(line GOLDEN)" +if has GOLDEN spacing-outer-bottom-large; then ok "class defined only inside @media is in the corpus (the 2026-09-26 false 'invented')" +else bad "@media-only class missing from the corpus" "$G"; fi +if has GOLDEN mx-menubar; then ok "class defined inside @supports is in the corpus" +else bad "@supports class missing" "$G"; fi +if has GOLDEN mx-datagrid-table-resizing; then ok "depth-0 multi-line selector list still harvested" +else bad "depth-0 selector regressed" "$G"; fi +if has GOLDEN ttf || has GOLDEN fonts; then bad "url() fragment in @font-face harvested as a class" "$G" +else ok "url(\"./fonts/….ttf\") in @font-face defines no class"; fi +n=$(printf '%s\n' "$G" | wc -w | tr -d ' ') +[ "$n" -eq 3 ] && ok "golden capture yields exactly 3 classes (no at-rule prelude or keyframe step leaked)" \ + || bad "golden capture yields $n classes, want 3" "$G" + +S="$(line SYN-QUOTED)" +[ "$S" = "real" ] && ok "quoted dot and url() in declaration values define nothing (synthetic)" || bad "declaration value leaked" "$S" +S="$(line SYN-BRACE)" +[ "$S" = "a b" ] && ok "a brace inside a string does not open a rule (synthetic)" || bad "string brace mis-parsed" "$S" + +echo "" +echo "PASS=$PASS FAIL=$FAIL" +[ "$FAIL" -eq 0 ] From 9fe925e1912bceefd4623c37aa42c0d94cfead19 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 07:48:02 +0000 Subject: [PATCH 04/45] fix(design-audit): nav parser reads items carrying an icon clause The item regex required ';' right after the page name; real describe navigation output has 'icon ' there, so no page was routed and rung 7 (a11y/overflow/structure) never ran. Split out parseNavigation and match up to the terminator. Field run: 0 -> 4 of 4 menu pages routed; rung 7 measured 4 of 7 pages. New fixture over a verbatim capture: 6/6 green, 5 red on the old regexes. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Pm41sykqDGcvioGzsY5H2N --- CHANGELOG.md | 1 + project-tests/e2e/design-audit.js | 21 ++++-- .../describe-navigation-responsive.txt | 19 ++++++ tests/wave2/test-design-audit-nav-map.sh | 67 +++++++++++++++++++ 4 files changed, 102 insertions(+), 6 deletions(-) create mode 100644 tests/wave2/fixtures/design-audit-nav/describe-navigation-responsive.txt create mode 100755 tests/wave2/test-design-audit-nav-map.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index 9504cc7..2a1d813 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,7 @@ three commits past it), and a bug report can name a release instead of a sha nob Sections dated before 2026-09-19 predate the cycle and stay as they are. ## Unreleased +- fix(project-tests/e2e/design-audit.js): **rung 7 (a11y, overflow, structure) now finds the menu route of every page that has an icon.** The navigation parser anchored on `;` straight after the page name, but real `describe navigation` output puts `icon ` there (and before a group's `(`), so no item with an icon was routed and every page reported FAULT "no navigation route — this check never ran". Parsing is now `parseNavigation(txt)`, matching up to the terminator. Field run (mxcli v0.24.0, Mendix 11.13.0): routed pages 0 → 4 of 4 menu pages; rung 7 measured 4 of 7 in-scope pages instead of 0, and immediately found color-contrast, missing landmarks and an aria-required-children critical that had been invisible. The other 3 are NewEdit pages opened from another page, still FAULT by design. New fixture `tests/wave2/test-design-audit-nav-map.sh` over a verbatim capture: 6 assertions, 6 green on the fix, 5 red against the previous regexes. From a card-disbursement requirements-driven build (build-plan row 3.9). - fix(project-tests/e2e/design-audit.js): **the defined-class corpus now reads rules nested in `@media` / `@supports` / `@layer`, not just depth 0.** Before, `classesInCss` harvested selectors only at brace depth 0. Atlas emits `spacing-outer-bottom-large` only inside three breakpoint `@media` blocks, so rungs 6-7 called it `invented-class` / `unmatched-class` on every page that used it — the class `skills/design-spacing.md` prescribes for section rhythm — and the run ended INSTRUMENT FAULT. The harvester now strips comments and string literals, then takes the selector prelude before every `{` that is not an at-rule. Field run on a real mxbuild `theme.compiled.css` (Mendix 11.13.0, Atlas): corpus 2768 → 3349 classes, the spacing class now present. New fixture `tests/wave2/test-design-audit-css-corpus.sh` over a verbatim capture of that stylesheet (`@media`, `@supports`, `@font-face` url, `@keyframes`): 7 assertions, 7 green on the fix, 4 red against the previous script. From a card-disbursement requirements-driven build (build-plan row 3.9). - fix(project-bin/coverage-preflight.sh): **LEVEL 1 now finds the per-BRD ledger directory `architecture/modules//coverage-ledger/.md` and measures each BRD against its own file.** Before, discovery tested for files only: a module that splits its ledger per BRD (the `coverage-check-all.sh` convention) keeps `coverage-ledger.md` beside the directory as an index, so the preflight found the index and ran every BRD in the project against it — `verify-module.sh`'s coverage rung read 9 of 9 BRDs UNCLAIMED (2,300+ leaves) for a module that owns one, while `coverage-check-all.sh` reported all 9 clean from the same files. The directory now wins over the file beside it; the run states its denominator ("1 of 9 BRD(s) measured", the rest named as another module's), a ledger file named after no BRD faults, a directory matching no BRD faults instead of reading clean, and an empty directory falls through to the file shapes. Field run on all 4 modules of that project: 9 of 9 BRDs measured once each, all clean. Fixture `tests/wave2/test-coverage-preflight-ledger-dir.sh`, 10 assertions, 7 of them red against the previous script — a card-disbursement requirements-driven build (build-plan row 3.9) - process(contrib): **`learned-mdl-preflight.md` row 12 is outdated on mxcli v0.24 — queued in `contrib/inbox/`.** A dynamictext's ContentParams builds only with a bare attribute or a quoted literal: `toString(Attr)` (the row's own fix) and `if … then … else` both fail CE1613 at mxbuild, and `check --references` refuses the first but passes the second. Five-form probe on mx check 11.13.0. From a card-disbursement requirements-driven build (gallery row 2.14a). diff --git a/project-tests/e2e/design-audit.js b/project-tests/e2e/design-audit.js index ef2c72f..7b964a3 100644 --- a/project-tests/e2e/design-audit.js +++ b/project-tests/e2e/design-audit.js @@ -452,20 +452,29 @@ async function checkStructure(page) { } // ── navigation map, derived from the live navigation document -function buildNavMap() { - const txt = describeNavigation(); - if (!txt) return { map: new Map(), source: 'unavailable' }; +// Real `describe navigation` output carries an `icon ` clause between the target and +// the `;` on items, and between the caption and the `(` on groups. Anchoring on `;` right after +// the page name matched no item that had an icon, so every page read as "no navigation route" +// and rung 7 never ran (card-disbursement build, 2026-09-26). Match up to the terminator. +function parseNavigation(txt) { const map = new Map(); let group = null; for (const line of txt.split('\n')) { - let m = /^\s*menu\s+'([^']+)'\s*\(/.exec(line); + let m = /^\s*menu\s+'([^']+)'[^;]*\(\s*$/.exec(line); if (m) { group = m[1]; continue; } if (/^\s*\);\s*$/.test(line)) { group = null; continue; } - m = /^\s*menu item\s+'([^']+)'\s+page\s+([\w.]+);/.exec(line); + m = /^\s*menu item\s+'([^']+)'\s+page\s+([\w.]+)(?=[\s;])[^;]*;/.exec(line); if (m) { map.set(m[2], { group, item: m[1], via: 'page' }); continue; } - m = /^\s*menu item\s+'([^']+)'\s+microflow\s+([\w.]+);/.exec(line); + m = /^\s*menu item\s+'([^']+)'\s+microflow\s+([\w.]+)(?=[\s;])[^;]*;/.exec(line); if (m) map.set(`microflow:${m[2]}`, { group, item: m[1], via: 'microflow' }); } + return map; +} + +function buildNavMap() { + const txt = describeNavigation(); + if (!txt) return { map: new Map(), source: 'unavailable' }; + const map = parseNavigation(txt); // Resolve microflow-opened items onto pages by module + name tokens. Inference is // labelled as such in the row detail; it is never presented as measured routing. for (const [k, v] of [...map]) { diff --git a/tests/wave2/fixtures/design-audit-nav/describe-navigation-responsive.txt b/tests/wave2/fixtures/design-audit-nav/describe-navigation-responsive.txt new file mode 100644 index 0000000..1b10962 --- /dev/null +++ b/tests/wave2/fixtures/design-audit-nav/describe-navigation-responsive.txt @@ -0,0 +1,19 @@ +-- GOLDEN CAPTURE, verbatim: `mxcli -p .mpr describe navigation Responsive` (mxcli v0.24.0, +-- Mendix 11.13.0), 2026-09-26, card-disbursement requirements-driven build. Only these three +-- header lines were added; mxcli's WARNING banner lines on stderr are not part of stdout. +-- navigation PROFILE: Responsive +-- Kind: Responsive +create or replace navigation Responsive + home page MyFirstModule.Home_Web + menu ( + menu 'Work' icon Atlas_Core.Atlas.briefcase ( + menu item 'Home' page MyFirstModule.Home_Web icon Atlas_Core.Atlas.home; + menu item 'Cases' page Disbursement.Case_Overview icon Atlas_Core.Atlas."list-bullets"; + ); + menu 'Configure' icon Atlas_Core.Atlas.cog ( + menu item 'Mock console' page MockServices.MockConsole_Overview icon Atlas_Core.Atlas."console-terminal"; + menu item 'Design system' page StyleGallery.Gallery_Overview icon Atlas_Core.Atlas."color-painting-palette"; + ); + ) +; + diff --git a/tests/wave2/test-design-audit-nav-map.sh b/tests/wave2/test-design-audit-nav-map.sh new file mode 100755 index 0000000..2267402 --- /dev/null +++ b/tests/wave2/test-design-audit-nav-map.sh @@ -0,0 +1,67 @@ +#!/usr/bin/env bash +# Usage: bash test-design-audit-nav-map.sh [path-to-design-audit.js] +# +# Fixture for project-tests/e2e/design-audit.js's navigation map (parseNavigation) — the route +# table rung 7 (a11y, overflow, structure) uses to reach each page in the running app. A page +# with no route is a FAULT row "this check never ran", so a parser that misses items silently +# turns the whole runtime rung into faults. +# +# The defect (card-disbursement requirements-driven build, 2026-09-26): the item regex required +# `;` straight after the page name. Real `describe navigation` output puts `icon ` there +# (and `icon ` before the `(` of a group), so 0 of 4 menu pages were routed and rung 7 ran +# on none of 7 in-scope pages. +# +# Golden input is a VERBATIM capture (fixtures/design-audit-nav/). The last case is SYNTHETIC and +# marked: an icon-less item and a microflow item, the shapes the capture lacks. + +set -uo pipefail + +SUT="${1:-}" +[ -z "$SUT" ] && SUT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)/project-tests/e2e/design-audit.js" +if [ ! -f "$SUT" ]; then + echo "SKIP: subject not found at $SUT" + echo "SCORE: 0/0 — nothing to test" + exit 0 +fi +command -v node >/dev/null 2>&1 || { echo "SKIP: node not installed"; exit 0; } +GOLDEN="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/fixtures/design-audit-nav/describe-navigation-responsive.txt" +[ -f "$GOLDEN" ] || { echo "FAIL — golden capture missing: $GOLDEN"; exit 1; } + +# design-audit.js runs on load, so lift the function out of its text instead of requiring it. +OUT="$(node - "$SUT" "$GOLDEN" <<'JS' +const fs = require('fs'); +const [sut, golden] = process.argv.slice(2); +const js = fs.readFileSync(sut, 'utf8'); +const i = js.indexOf('function parseNavigation('); +if (i < 0) { console.log('LIFT-FAIL parseNavigation'); process.exit(3); } +let d = 0, j = js.indexOf('{', i); +for (; j < js.length; j++) { if (js[j] === '{') d++; else if (js[j] === '}' && --d === 0) break; } +const parse = new Function(`${js.slice(i, j + 1)}\nreturn parseNavigation;`)(); +const show = (label, txt) => { + const m = parse(txt); + console.log(`${label} n=${m.size} ` + [...m].map(([k, v]) => `${k}=${v.group}>${v.item}`).join(' | ')); +}; +show('GOLDEN', fs.readFileSync(golden, 'utf8')); +// SYNTHETIC — shapes the capture does not carry. +show('SYN', " menu 'Ops' (\n menu item 'Plain' page Mod.Plain_Page;\n menu item 'Run' microflow Mod.ACT_Run icon Atlas_Core.Atlas.play;\n );\n"); +JS +)" +rc=$? +[ "$rc" -eq 0 ] || { echo "FAIL — harness could not load parseNavigation from $SUT (rc=$rc)"; echo "$OUT"; exit 1; } + +PASS=0; FAIL=0 +ok() { PASS=$((PASS+1)); echo " ok — $1"; } +bad() { FAIL=$((FAIL+1)); echo " FAIL — $1"; [ -n "${2:-}" ] && echo " got: $2"; } +G="$(printf '%s\n' "$OUT" | sed -n 's/^GOLDEN //p')" +S="$(printf '%s\n' "$OUT" | sed -n 's/^SYN //p')" + +case "$G" in n=4\ *) ok "golden capture routes all 4 menu pages (the 2026-09-26 zero)";; *) bad "golden capture: want n=4" "$G";; esac +case "$G" in *"Disbursement.Case_Overview=Work>Cases"*) ok "quoted icon ref (\"list-bullets\") does not break the item";; *) bad "Case_Overview route wrong" "$G";; esac +case "$G" in *"MockServices.MockConsole_Overview=Configure>Mock console"*) ok "group caption read past its icon clause";; *) bad "group with icon not read" "$G";; esac +case "$G" in *"MyFirstModule.Home_Web=Work>Home"*) ok "item routed under its own group, not the previous one";; *) bad "Home_Web route wrong" "$G";; esac +case "$S" in *"Mod.Plain_Page=Ops>Plain"*) ok "icon-less item still parses (synthetic)";; *) bad "icon-less item regressed" "$S";; esac +case "$S" in *"microflow:Mod.ACT_Run=Ops>Run"*) ok "microflow item with icon parses (synthetic)";; *) bad "microflow item missed" "$S";; esac + +echo "" +echo "PASS=$PASS FAIL=$FAIL" +[ "$FAIL" -eq 0 ] From f38f8084990443b2728b432233dd1895db602cb8 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 07:59:02 +0000 Subject: [PATCH 05/45] fix(report-normalize): read inputs and write docs/report.json in the two-tree layout Every input resolved against project.config's ROOT, which is app/ in a two-tree checkout, so verify-module's report step marked 8 of 9 instruments FAULT over artefacts that existed and wrote the report into a stray app/docs/. Inputs now probe ROOT first, then the repo root; outputs go to the repo root. JOURNEY_DIR is honoured and relativised; projectId uses the config id. Field run: 0 -> 205 checks read, instrument FAULTs 8 -> 2 (both real). Fixture test-report-normalize-two-tree.sh: 8/8 on the fix, 6 red on the old code. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Pm41sykqDGcvioGzsY5H2N --- CHANGELOG.md | 1 + project-tests/e2e/report-normalize.js | 34 ++++++- tests/wave2/test-report-normalize-two-tree.sh | 96 +++++++++++++++++++ 3 files changed, 126 insertions(+), 5 deletions(-) create mode 100644 tests/wave2/test-report-normalize-two-tree.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index 2a1d813..0f711ef 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,7 @@ three commits past it), and a bug report can name a release instead of a sha nob Sections dated before 2026-09-19 predate the cycle and stay as they are. ## Unreleased +- fix(project-tests/e2e/report-normalize.js): **the report now reads its inputs and writes `docs/report.json` in the two-tree layout (model under `app/`).** Every input resolved against project.config's ROOT, which is `app/` there, so `verify-module.sh`'s report step marked 8 of 9 instruments FAULT ("design-audit.json does not exist", "no coverage-ledger.md under architecture/modules/*/") over files that were on disk, and wrote the report into a stray `app/docs/report.json`, which left the render step with nothing to render. Inputs now try ROOT first (the `.mpr`, `deployment/model/*` and `.mxcli/catalog.db` live there), then the repo root; outputs go to the repo root. This is the same fix as `project.config.js` `designPath()` and design-audit's `nearRoot()` (F-042). `JOURNEY_DIR` is now honoured and re-expressed relative to the project, so reproduce commands carry no machine path. `meta.projectId` uses the config's `id`, not `basename(ROOT)`, which was `app`. Field run (Mendix 11.13.0, 4 modules): 0 → 205 checks read, instrument FAULTs 8 → 2, and those 2 are real (no deferrals or run-ledger file). The report is back at the repo root. New fixture `tests/wave2/test-report-normalize-two-tree.sh` (two-tree plus a single-tree regression): 8 assertions, 8 green on the fix, 6 red against the previous script. `--selftest` fails identically before and after (it expects a `mobile-fieldscan` walkthrough the template config does not declare); it was not touched here. From a card-disbursement requirements-driven build (build-plan row 3.9). - fix(project-tests/e2e/design-audit.js): **rung 7 (a11y, overflow, structure) now finds the menu route of every page that has an icon.** The navigation parser anchored on `;` straight after the page name, but real `describe navigation` output puts `icon ` there (and before a group's `(`), so no item with an icon was routed and every page reported FAULT "no navigation route — this check never ran". Parsing is now `parseNavigation(txt)`, matching up to the terminator. Field run (mxcli v0.24.0, Mendix 11.13.0): routed pages 0 → 4 of 4 menu pages; rung 7 measured 4 of 7 in-scope pages instead of 0, and immediately found color-contrast, missing landmarks and an aria-required-children critical that had been invisible. The other 3 are NewEdit pages opened from another page, still FAULT by design. New fixture `tests/wave2/test-design-audit-nav-map.sh` over a verbatim capture: 6 assertions, 6 green on the fix, 5 red against the previous regexes. From a card-disbursement requirements-driven build (build-plan row 3.9). - fix(project-tests/e2e/design-audit.js): **the defined-class corpus now reads rules nested in `@media` / `@supports` / `@layer`, not just depth 0.** Before, `classesInCss` harvested selectors only at brace depth 0. Atlas emits `spacing-outer-bottom-large` only inside three breakpoint `@media` blocks, so rungs 6-7 called it `invented-class` / `unmatched-class` on every page that used it — the class `skills/design-spacing.md` prescribes for section rhythm — and the run ended INSTRUMENT FAULT. The harvester now strips comments and string literals, then takes the selector prelude before every `{` that is not an at-rule. Field run on a real mxbuild `theme.compiled.css` (Mendix 11.13.0, Atlas): corpus 2768 → 3349 classes, the spacing class now present. New fixture `tests/wave2/test-design-audit-css-corpus.sh` over a verbatim capture of that stylesheet (`@media`, `@supports`, `@font-face` url, `@keyframes`): 7 assertions, 7 green on the fix, 4 red against the previous script. From a card-disbursement requirements-driven build (build-plan row 3.9). - fix(project-bin/coverage-preflight.sh): **LEVEL 1 now finds the per-BRD ledger directory `architecture/modules//coverage-ledger/.md` and measures each BRD against its own file.** Before, discovery tested for files only: a module that splits its ledger per BRD (the `coverage-check-all.sh` convention) keeps `coverage-ledger.md` beside the directory as an index, so the preflight found the index and ran every BRD in the project against it — `verify-module.sh`'s coverage rung read 9 of 9 BRDs UNCLAIMED (2,300+ leaves) for a module that owns one, while `coverage-check-all.sh` reported all 9 clean from the same files. The directory now wins over the file beside it; the run states its denominator ("1 of 9 BRD(s) measured", the rest named as another module's), a ledger file named after no BRD faults, a directory matching no BRD faults instead of reading clean, and an empty directory falls through to the file shapes. Field run on all 4 modules of that project: 9 of 9 BRDs measured once each, all clean. Fixture `tests/wave2/test-coverage-preflight-ledger-dir.sh`, 10 assertions, 7 of them red against the previous script — a card-disbursement requirements-driven build (build-plan row 3.9) diff --git a/project-tests/e2e/report-normalize.js b/project-tests/e2e/report-normalize.js index af41704..a296916 100644 --- a/project-tests/e2e/report-normalize.js +++ b/project-tests/e2e/report-normalize.js @@ -94,7 +94,8 @@ const INPUTS = { verifyDir: '.claude/loop/verify', conformanceDir: 'docs/conformance', modulesDir: 'architecture/modules', - journeysDir: 'journeys', + // verify-module.sh's JOURNEY_DIR, so a project keeping journeys elsewhere is read where it keeps them. + journeysDir: process.env.JOURNEY_DIR || 'journeys', runLedger: '.claude/loop/run-ledger.jsonl', deferrals: '.claude/loop/deferrals.jsonl', mpr: PROJ.mprName, @@ -156,7 +157,30 @@ const V_CONF = { OK: 'pass', STALE: 'fail', UNDERSTATED: 'fail', OVERSTATED: 'fa // ── Small utilities ────────────────────────────────────────────────────────── -const abs = p => (path.isAbsolute(p) ? p : path.join(ROOT, p)); +// Two-tree layout (F-042; card-disbursement requirements-driven build, 2026-09-26): ROOT is +// the model's app/ directory, while docs/, architecture/, .claude/loop/, tests/e2e/artifacts/ +// and journeys/ sit beside it at the repository root. Resolving every INPUT against ROOT made +// all of them "missing" — 8 instrument FAULTs over artefacts that existed — and wrote the +// report into a stray app/docs/, so verify-module found no docs/report.json to render. Same +// cure as project.config.js designPath() and design-audit.js nearRoot(): probe ROOT first (the +// .mpr, deployment/model/*, .mxcli/catalog.db live there), then the repo root. PROJECT_DIR is +// ROOT's parent only when this harness sits outside ROOT but inside that parent — the +// signature of the two-tree checkout; in a single tree it IS ROOT and nothing changes. +const inside = (dir, p) => { const r = path.relative(dir, p); return !!r && !r.startsWith('..') && !path.isAbsolute(r); }; +const PROJECT_DIR = (!inside(ROOT, __dirname) && inside(path.dirname(ROOT), __dirname)) + ? path.dirname(ROOT) : ROOT; +function abs(p) { + if (path.isAbsolute(p)) return p; + const inRoot = path.join(ROOT, p); + if (PROJECT_DIR === ROOT || fs.existsSync(inRoot)) return inRoot; + return path.join(PROJECT_DIR, p); +} +// An absolute JOURNEY_DIR (verify-module passes $PWD/...) is re-expressed relative to the +// project, so the report's reproduce commands and evidence paths carry no machine path. +if (path.isAbsolute(INPUTS.journeysDir) && inside(PROJECT_DIR, INPUTS.journeysDir)) + INPUTS.journeysDir = path.relative(PROJECT_DIR, INPUTS.journeysDir).split(path.sep).join('/'); +// Outputs are never probed: they belong at the repo root, where verify-module looks for them. +const absOut = p => (path.isAbsolute(p) ? p : path.join(PROJECT_DIR, p)); // Requirement pointers: rule 4 of the schema. A blank cell in a source is the // ABSENCE of a pointer, and "" renders as an empty table cell that reads like an @@ -193,7 +217,7 @@ function sortKeysDeep(v) { // Evidence paths are relative to report.json and never base64 (schema rule 3). function relTo(outFile, target) { - return path.relative(path.dirname(abs(outFile)), abs(target)).split(path.sep).join('/'); + return path.relative(path.dirname(absOut(outFile)), abs(target)).split(path.sep).join('/'); } // Every read goes through here so that "missing" and "unreadable" and "not JSON" @@ -2422,7 +2446,7 @@ function loadInputs(outFile) { .filter(l => !(l.file.status === 'missing')); // a module dir with no ledger is not an input // ── Project / run metadata ──────────────────────────────────────────────── - const meta = { projectId: path.basename(ROOT), mpr: INPUTS.mpr }; + const meta = { projectId: PROJ.id || path.basename(ROOT), mpr: INPUTS.mpr }; // basename(ROOT) is 'app' in a two-tree checkout try { meta.mprModifiedAt = fs.statSync(abs(INPUTS.mpr)).mtime.toISOString(); } catch { meta.mprModifiedAt = null; } const md = readJsonInput(INPUTS.metadata); if (md.status === 'ok') { @@ -3028,7 +3052,7 @@ function main() { const inputs = loadInputs(outFile); const report = sortKeysDeep(normalize(inputs)); - const outPath = abs(outFile); + const outPath = absOut(outFile); try { fs.mkdirSync(path.dirname(outPath), { recursive: true }); fs.writeFileSync(outPath, JSON.stringify(report, null, 2) + '\n'); diff --git a/tests/wave2/test-report-normalize-two-tree.sh b/tests/wave2/test-report-normalize-two-tree.sh new file mode 100644 index 0000000..f8bcea9 --- /dev/null +++ b/tests/wave2/test-report-normalize-two-tree.sh @@ -0,0 +1,96 @@ +#!/usr/bin/env bash +# Usage: bash test-report-normalize-two-tree.sh [path-to-report-normalize.js] +# +# Fixture for project-tests/e2e/report-normalize.js's path resolution in the two-tree layout +# (model under app/, docs/ architecture/ tests/e2e/ .claude/loop/ at the repo root — F-042). +# +# The defect (card-disbursement requirements-driven build, 2026-09-26): every INPUT resolved +# against project.config's ROOT, which is app/ in a two-tree checkout. verify-module's report +# step then showed 8 of 9 instruments FAULT ("design-audit.json does not exist", "no +# coverage-ledger.md under architecture/modules/*/") over artefacts that were on disk, and wrote +# the report into a stray app/docs/report.json — so the render step found nothing to render. +# Case 1 is that shape in miniature; case 2 is the single-tree regression guard. +# +# The artefacts here are SYNTHETIC and minimal. That is deliberate and allowed: the field-proof +# rule's "golden input is captured" governs parsers of tool output, and nothing below exercises a +# parser — it lays out files and asserts WHERE the script reads and writes. The field run that +# motivated it (0 checks → 205 checks read) is cited in CHANGELOG.md. + +set -uo pipefail + +SUT="${1:-}" +[ -z "$SUT" ] && SUT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)/project-tests/e2e/report-normalize.js" +if [ ! -f "$SUT" ]; then + echo "SKIP: subject not found at $SUT" + echo "SCORE: 0/0 — nothing to test" + exit 0 +fi +command -v node >/dev/null 2>&1 || { echo "SKIP: node not installed"; exit 0; } +CFG="$(cd "$(dirname "$SUT")" && pwd)/project.config.template.js" +[ -f "$CFG" ] || CFG="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)/project-tests/e2e/project.config.template.js" +[ -f "$CFG" ] || { echo "FAIL — project.config.template.js not found beside $SUT"; exit 1; } + +PASS=0; FAIL=0 +ok() { PASS=$((PASS+1)); echo " ok — $1"; } +bad() { FAIL=$((FAIL+1)); echo " FAIL — $1"; [ -n "${2:-}" ] && echo " got: $2"; } + +WORK="$(mktemp -d "${TMPDIR:-/tmp}/rn-two-tree.XXXXXX")" +trap 'rm -rf "$WORK"' EXIT + +# lay_out : the harness and every non-model artefact under . +lay_out() { + local m="$1" p="$2" + mkdir -p "$m" "$p/tests/e2e/artifacts" "$p/tests/e2e/journeys" "$p/architecture/modules/Orders" "$p/docs/conformance" + : > "$m/Orders.mpr" + cp "$SUT" "$p/tests/e2e/report-normalize.js" + cp "$CFG" "$p/tests/e2e/project.config.js" + echo '{"checks":[]}' > "$p/tests/e2e/artifacts/design-audit.json" + printf '# Coverage ledger — Orders\n' > "$p/architecture/modules/Orders/coverage-ledger.md" + printf 'module\tclaim\tstatus\n' > "$p/docs/conformance/report-2026-09-26.tsv" + echo '{"id":"J-01","steps":[]}' > "$p/tests/e2e/journeys/Orders.journey.json" +} +# reason → "|" +reason() { node -e 'const r=require(process.argv[1]); const x=r.instruments.find(i=>i.name===process.argv[2]); console.log(x ? `${x.verdict}|${x.reason||""}` : "ABSENT|")' "$1" "$2"; } + +# --- 1. two-tree: inputs found beside app/, output written at the repo root ------------------ +T="$WORK/two"; lay_out "$T/app" "$T" +( cd "$T/tests/e2e" && JOURNEY_DIR="$T/tests/e2e/journeys" node report-normalize.js --out docs/report.json ) >"$WORK/out1" 2>&1 +if [ -s "$T/docs/report.json" ]; then ok "report written at the repo root's docs/report.json" +else bad "no docs/report.json at the repo root" "$(tr '\n' ' ' < "$WORK/out1" | cut -c1-300)"; fi +if [ -e "$T/app/docs" ]; then bad "stray app/docs/ created (the 2026-09-26 misplaced report)"; else ok "no stray app/docs/"; fi +# Wherever it landed, still measure what it read — the misplaced write and the missed reads are +# two defects, and the old code shows both. +R="$T/docs/report.json"; [ -s "$R" ] || R="$T/app/docs/report.json" +if [ -s "$R" ]; then + v="$(reason "$R" design-audit)" + case "$v" in *"does not exist"*|ABSENT*) bad "design-audit.json beside app/ read as missing" "$v" ;; + *) ok "tests/e2e/artifacts/design-audit.json found beside app/ ($v)" ;; esac + v="$(reason "$R" coverage-ledger)" + case "$v" in *"no coverage-ledger.md"*|ABSENT*) bad "architecture/modules ledger read as missing" "$v" ;; + *) ok "architecture/modules/*/coverage-ledger.md found beside app/" ;; esac + v="$(reason "$R" conformance)" + case "$v" in *"does not exist"*|ABSENT*) bad "docs/conformance report read as missing" "$v" ;; + *) ok "docs/conformance report found beside app/" ;; esac + node -e 'const r=require(process.argv[1]); process.exit(r.project && r.project.mprModifiedAt ? 0 : 1)' "$R" \ + && ok "the .mpr is still resolved inside app/ (mprModifiedAt set)" || bad "the .mpr under app/ was lost" + node -e 'const r=require(process.argv[1]); const c=(r.coverage||[]).find(m=>m.module==="Orders"); process.exit(c && (c.journeys||[]).includes("J-01") ? 0 : 1)' "$R" \ + && ok "JOURNEY_DIR (absolute, as verify-module passes it) is read: Orders lists J-01" \ + || bad "journey contracts under JOURNEY_DIR not read" +else + bad "no report to inspect — the five two-tree read assertions did not run" +fi + +# --- 2. single tree (regression): everything at the root, nothing moves ---------------------- +S="$WORK/one"; lay_out "$S" "$S" +( cd "$S/tests/e2e" && node report-normalize.js --out docs/report.json ) >"$WORK/out2" 2>&1 +if [ -s "$S/docs/report.json" ]; then + v="$(reason "$S/docs/report.json" design-audit)" + case "$v" in *"does not exist"*|ABSENT*) bad "single tree: design-audit.json read as missing" "$v" ;; + *) ok "single tree: report at docs/, design-audit.json found" ;; esac +else + bad "single tree: no docs/report.json" "$(tr '\n' ' ' < "$WORK/out2" | cut -c1-300)" +fi + +echo "" +echo "PASS=$PASS FAIL=$FAIL" +[ "$FAIL" -eq 0 ] From 8ca908fdb92096f2d4a5e4db49beea6dd3244f95 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 08:12:45 +0000 Subject: [PATCH 06/45] fix(otel): rung 2 waits for the claimed microflow spans, not the first flush capture(t0, { min: 1 }) stopped at the first poll holding any span. The batch exporter flushes a step's HTTP/xas spans before its microflow spans, so spans.ordered claims failed "actual (none)" over sequences Jaeger held a second later. capture() now takes until(spans); rung 2 passes "every claimed microflow is present". An unsatisfiable claim still fails after the normal retry window. Field run (Mendix 11.13.0 -> Jaeger v1.76 OTLP, card-disbursement build, 6 ordered claims): 2 of 6 red before, journey 79/0/0 after, positive control 7 of 7 rungs proven. Fixture test-otel-capture-until.sh over a real Jaeger /api/traces capture: 4/4 green on the fix, 3 red on the old code. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Pm41sykqDGcvioGzsY5H2N --- CHANGELOG.md | 1 + project-tests/e2e/journey-runner.js | 5 +- project-tests/e2e/otel.js | 13 +- .../jaeger-trace-refused-arm.json | 520 ++++++++++++++++++ tests/wave2/test-otel-capture-until.sh | 94 ++++ 5 files changed, 630 insertions(+), 3 deletions(-) create mode 100644 tests/wave2/fixtures/otel-capture/jaeger-trace-refused-arm.json create mode 100644 tests/wave2/test-otel-capture-until.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index 0f711ef..cbd5202 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,7 @@ three commits past it), and a bug report can name a release instead of a sha nob Sections dated before 2026-09-19 predate the cycle and stay as they are. ## Unreleased +- fix(project-tests/e2e/otel.js, journey-runner.js): **rung 2 now waits for the claimed microflow spans, not just the first span of any kind.** Before, `capture(t0, { min: 1 })` stopped at the first poll that held any span. The batch exporter routinely flushes a step's HTTP/xas spans before its microflow spans, so `spans.ordered` claims failed "actual (none)" over sequences Jaeger held a second later. `capture()` now takes `until(spans)`, and rung 2 passes "every claimed microflow is present". A claim that never holds still fails, after the normal retry window. Field run (Mendix 11.13.0 → Jaeger v1.76 over OTLP, 6 ordered claims, 3 of them with `mustNotFire`): 2 of 6 red before the fix, the same journey 79/0/0 after, and `--positive-control` 7 of 7 rungs proven (trace-order and trace-negative were UNPROVEN before, because no journey declared a span claim). New fixture `tests/wave2/test-otel-capture-until.sh` over a real Jaeger `/api/traces` capture: 4 assertions, 4 green on the fix, 3 red against the previous code. From a card-disbursement requirements-driven build (build-plan row 3.9). - fix(project-tests/e2e/report-normalize.js): **the report now reads its inputs and writes `docs/report.json` in the two-tree layout (model under `app/`).** Every input resolved against project.config's ROOT, which is `app/` there, so `verify-module.sh`'s report step marked 8 of 9 instruments FAULT ("design-audit.json does not exist", "no coverage-ledger.md under architecture/modules/*/") over files that were on disk, and wrote the report into a stray `app/docs/report.json`, which left the render step with nothing to render. Inputs now try ROOT first (the `.mpr`, `deployment/model/*` and `.mxcli/catalog.db` live there), then the repo root; outputs go to the repo root. This is the same fix as `project.config.js` `designPath()` and design-audit's `nearRoot()` (F-042). `JOURNEY_DIR` is now honoured and re-expressed relative to the project, so reproduce commands carry no machine path. `meta.projectId` uses the config's `id`, not `basename(ROOT)`, which was `app`. Field run (Mendix 11.13.0, 4 modules): 0 → 205 checks read, instrument FAULTs 8 → 2, and those 2 are real (no deferrals or run-ledger file). The report is back at the repo root. New fixture `tests/wave2/test-report-normalize-two-tree.sh` (two-tree plus a single-tree regression): 8 assertions, 8 green on the fix, 6 red against the previous script. `--selftest` fails identically before and after (it expects a `mobile-fieldscan` walkthrough the template config does not declare); it was not touched here. From a card-disbursement requirements-driven build (build-plan row 3.9). - fix(project-tests/e2e/design-audit.js): **rung 7 (a11y, overflow, structure) now finds the menu route of every page that has an icon.** The navigation parser anchored on `;` straight after the page name, but real `describe navigation` output puts `icon ` there (and before a group's `(`), so no item with an icon was routed and every page reported FAULT "no navigation route — this check never ran". Parsing is now `parseNavigation(txt)`, matching up to the terminator. Field run (mxcli v0.24.0, Mendix 11.13.0): routed pages 0 → 4 of 4 menu pages; rung 7 measured 4 of 7 in-scope pages instead of 0, and immediately found color-contrast, missing landmarks and an aria-required-children critical that had been invisible. The other 3 are NewEdit pages opened from another page, still FAULT by design. New fixture `tests/wave2/test-design-audit-nav-map.sh` over a verbatim capture: 6 assertions, 6 green on the fix, 5 red against the previous regexes. From a card-disbursement requirements-driven build (build-plan row 3.9). - fix(project-tests/e2e/design-audit.js): **the defined-class corpus now reads rules nested in `@media` / `@supports` / `@layer`, not just depth 0.** Before, `classesInCss` harvested selectors only at brace depth 0. Atlas emits `spacing-outer-bottom-large` only inside three breakpoint `@media` blocks, so rungs 6-7 called it `invented-class` / `unmatched-class` on every page that used it — the class `skills/design-spacing.md` prescribes for section rhythm — and the run ended INSTRUMENT FAULT. The harvester now strips comments and string literals, then takes the selector prelude before every `{` that is not an at-rule. Field run on a real mxbuild `theme.compiled.css` (Mendix 11.13.0, Atlas): corpus 2768 → 3349 classes, the spacing class now present. New fixture `tests/wave2/test-design-audit-css-corpus.sh` over a verbatim capture of that stylesheet (`@media`, `@supports`, `@font-face` url, `@keyframes`): 7 assertions, 7 green on the fix, 4 red against the previous script. From a card-disbursement requirements-driven build (build-plan row 3.9). diff --git a/project-tests/e2e/journey-runner.js b/project-tests/e2e/journey-runner.js index 9f4f08c..2e15c6a 100644 --- a/project-tests/e2e/journey-runner.js +++ b/project-tests/e2e/journey-runner.js @@ -695,7 +695,10 @@ async function runJourney(page, j) { // ── RUNG 2: ordered spans ──────────────────────────────────────────────── if (step.spans && step.spans.ordered && step.spans.ordered.length) { - const spans = await O.capture(t0, { min: 1 }); + // Wait for the claimed microflows, not just any span — see otel.js capture() `until`. + const claimed = step.spans.ordered; + const spans = await O.capture(t0, { min: 1, + until: sp => { const got = O.microflowNames(sp); return claimed.every(n => got.includes(n)); } }); if (!spans.length) { record('trace', `${step.name}: spans`, 'INVALID', 'zero spans captured — Jaeger down or OTel off. Trace rung did NOT run.', req); diff --git a/project-tests/e2e/otel.js b/project-tests/e2e/otel.js index 887ed30..91c264b 100644 --- a/project-tests/e2e/otel.js +++ b/project-tests/e2e/otel.js @@ -35,8 +35,17 @@ const sleep = ms => new Promise(r => setTimeout(r, ms)); * Collect every span the service emitted at or after `t0` (ms epoch). * Polls, because the exporter batches — spans lag the click by a second or two. * Retains `logs`, which is where OTel exception events land in Jaeger. + * + * `until(spans)` — keep polling until it returns true (or retries run out), not merely + * until `min` spans exist. `min: 1` alone stops at the FIRST flushed batch, which is + * routinely the page's HTTP/xas spans without the microflow spans behind them: a + * journey step then fails "ordered — actual (none)" over a sequence Jaeger holds a + * second later (card-disbursement requirements-driven build, 2026-09-26: 2 of 6 trace + * claims red on one run, green on the same app with the spans verified in Jaeger). + * A predicate that never holds costs the full retry window and returns what it has, + * so a genuinely missing microflow still fails — later, never falsely. */ -async function capture(t0, { min = 1, retries = 8, waitMs = 1500 } = {}) { +async function capture(t0, { min = 1, retries = 8, waitMs = 1500, until = null } = {}) { let out = []; for (let i = 0; i < retries; i++) { await sleep(waitMs); @@ -63,7 +72,7 @@ async function capture(t0, { min = 1, retries = 8, waitMs = 1500 } = {}) { logs: (s.logs || []).map(l => Object.fromEntries(l.fields.map(f => [f.key, f.value]))), }); } - if (out.length >= min) break; + if (out.length >= min && (!until || until(out))) break; } return out; } diff --git a/tests/wave2/fixtures/otel-capture/jaeger-trace-refused-arm.json b/tests/wave2/fixtures/otel-capture/jaeger-trace-refused-arm.json new file mode 100644 index 0000000..973ab99 --- /dev/null +++ b/tests/wave2/fixtures/otel-capture/jaeger-trace-refused-arm.json @@ -0,0 +1,520 @@ +{ + "data": [ + { + "traceID": "c4dc26854ecdb369548b4f882d2bd351", + "spans": [ + { + "traceID": "c4dc26854ecdb369548b4f882d2bd351", + "spanID": "6b80ee02197cd6c0", + "operationName": "Microflow MockServices.VAL_MockScenario_Payload", + "references": [ + { + "refType": "CHILD_OF", + "traceID": "c4dc26854ecdb369548b4f882d2bd351", + "spanID": "fa4649c3449fa902" + } + ], + "startTime": 1790409626709073, + "duration": 18484, + "tags": [ + { + "key": "mx.microflow.depth", + "type": "int64", + "value": 3 + }, + { + "key": "mx.microflow.name", + "type": "string", + "value": "MockServices.VAL_MockScenario_Payload" + }, + { + "key": "otel.scope.name", + "type": "string", + "value": "com.mendix.runtime" + }, + { + "key": "otel.status_code", + "type": "string", + "value": "OK" + }, + { + "key": "span.kind", + "type": "string", + "value": "server" + }, + { + "key": "thread.id", + "type": "int64", + "value": 146 + }, + { + "key": "thread.name", + "type": "string", + "value": "JettyServer-7" + } + ], + "logs": [], + "processID": "p1", + "warnings": null + }, + { + "traceID": "c4dc26854ecdb369548b4f882d2bd351", + "spanID": "fa4649c3449fa902", + "operationName": "Microflow MockServices.VAL_MockScenario_Arm", + "references": [ + { + "refType": "CHILD_OF", + "traceID": "c4dc26854ecdb369548b4f882d2bd351", + "spanID": "f989744b009500c8" + } + ], + "startTime": 1790409626707895, + "duration": 19740, + "tags": [ + { + "key": "mx.microflow.depth", + "type": "int64", + "value": 2 + }, + { + "key": "mx.microflow.name", + "type": "string", + "value": "MockServices.VAL_MockScenario_Arm" + }, + { + "key": "otel.scope.name", + "type": "string", + "value": "com.mendix.runtime" + }, + { + "key": "otel.status_code", + "type": "string", + "value": "OK" + }, + { + "key": "span.kind", + "type": "string", + "value": "server" + }, + { + "key": "thread.id", + "type": "int64", + "value": 146 + }, + { + "key": "thread.name", + "type": "string", + "value": "JettyServer-7" + } + ], + "logs": [], + "processID": "p1", + "warnings": null + }, + { + "traceID": "c4dc26854ecdb369548b4f882d2bd351", + "spanID": "f989744b009500c8", + "operationName": "Microflow MockServices.ACT_MockScenario_Arm", + "references": [ + { + "refType": "CHILD_OF", + "traceID": "c4dc26854ecdb369548b4f882d2bd351", + "spanID": "28bab01e60e87dd5" + } + ], + "startTime": 1790409626707352, + "duration": 20532, + "tags": [ + { + "key": "mx.microflow.depth", + "type": "int64", + "value": 1 + }, + { + "key": "mx.microflow.name", + "type": "string", + "value": "MockServices.ACT_MockScenario_Arm" + }, + { + "key": "otel.scope.name", + "type": "string", + "value": "com.mendix.runtime" + }, + { + "key": "otel.status_code", + "type": "string", + "value": "OK" + }, + { + "key": "span.kind", + "type": "string", + "value": "server" + }, + { + "key": "thread.id", + "type": "int64", + "value": 146 + }, + { + "key": "thread.name", + "type": "string", + "value": "JettyServer-7" + } + ], + "logs": [], + "processID": "p1", + "warnings": null + }, + { + "traceID": "c4dc26854ecdb369548b4f882d2bd351", + "spanID": "28bab01e60e87dd5", + "operationName": "Call microflow MockServices.ACT_MockScenario_Arm", + "references": [ + { + "refType": "CHILD_OF", + "traceID": "c4dc26854ecdb369548b4f882d2bd351", + "spanID": "11fa3a7d5e02c482" + } + ], + "startTime": 1790409626696746, + "duration": 32391, + "tags": [ + { + "key": "mx.operation.type", + "type": "string", + "value": "callMicroflow" + }, + { + "key": "otel.scope.name", + "type": "string", + "value": "com.mendix.runtime" + }, + { + "key": "otel.status_code", + "type": "string", + "value": "OK" + }, + { + "key": "span.kind", + "type": "string", + "value": "server" + }, + { + "key": "thread.id", + "type": "int64", + "value": 146 + }, + { + "key": "thread.name", + "type": "string", + "value": "JettyServer-7" + } + ], + "logs": [], + "processID": "p1", + "warnings": null + }, + { + "traceID": "c4dc26854ecdb369548b4f882d2bd351", + "spanID": "32acedcc452e6e31", + "operationName": "SELECT /work/app/deployment/data/database/hsqldb/carddisbursement/carddisbursement.mockservices$mockscenario", + "references": [ + { + "refType": "CHILD_OF", + "traceID": "c4dc26854ecdb369548b4f882d2bd351", + "spanID": "28bab01e60e87dd5" + } + ], + "startTime": 1790409626703564, + "duration": 154, + "tags": [ + { + "key": "db.connection_string", + "type": "string", + "value": "hsqldb:file:" + }, + { + "key": "db.name", + "type": "string", + "value": "/work/app/deployment/data/database/hsqldb/carddisbursement/carddisbursement" + }, + { + "key": "db.operation", + "type": "string", + "value": "SELECT" + }, + { + "key": "db.sql.table", + "type": "string", + "value": "mockservices$mockscenario" + }, + { + "key": "db.statement", + "type": "string", + "value": "SELECT \"mockservices$mockscenario\".\"id\", \"mockservices$mockscenario\".\"operation\", \"mockservices$mockscenario\".\"kind\", \"mockservices$mockscenario\".\"sticky\", \"mockservices$mockscenario\".\"active\", \"mockservices$mockscenario\".\"custompayload\", \"mockservices$mockscenario\".\"faultcode\", \"mockservices$mockscenario\".\"httpstatus\", \"mockservices$mockscenario\".\"hitcount\", \"mockservices$mockscenario\".\"createddate\", \"mockservices$mockscenario\".\"changeddate\", \"mockservices$mockscenario\".\"system$owner\", \"mockservices$mockscenario\".\"system$changedby\" FROM \"mockservices$mockscenario\" WHERE \"mockservices$mockscenario\".\"id\" = ?" + }, + { + "key": "db.system", + "type": "string", + "value": "hsqldb" + }, + { + "key": "db.user", + "type": "string", + "value": "SA" + }, + { + "key": "otel.scope.name", + "type": "string", + "value": "io.opentelemetry.jdbc" + }, + { + "key": "otel.scope.version", + "type": "string", + "value": "2.29.0-alpha" + }, + { + "key": "span.kind", + "type": "string", + "value": "client" + }, + { + "key": "thread.id", + "type": "int64", + "value": 146 + }, + { + "key": "thread.name", + "type": "string", + "value": "JettyServer-7" + } + ], + "logs": [], + "processID": "p1", + "warnings": null + }, + { + "traceID": "c4dc26854ecdb369548b4f882d2bd351", + "spanID": "11fa3a7d5e02c482", + "operationName": "POST /*", + "references": [], + "startTime": 1790409626694533, + "duration": 36156, + "tags": [ + { + "key": "client.address", + "type": "string", + "value": "127.0.0.1" + }, + { + "key": "http.request.method", + "type": "string", + "value": "POST" + }, + { + "key": "http.response.status_code", + "type": "int64", + "value": 200 + }, + { + "key": "http.route", + "type": "string", + "value": "/*" + }, + { + "key": "network.peer.address", + "type": "string", + "value": "127.0.0.1" + }, + { + "key": "network.peer.port", + "type": "int64", + "value": 36678 + }, + { + "key": "network.protocol.version", + "type": "string", + "value": "1.1" + }, + { + "key": "otel.scope.name", + "type": "string", + "value": "io.opentelemetry.jetty-12.0" + }, + { + "key": "otel.scope.version", + "type": "string", + "value": "2.29.0-alpha" + }, + { + "key": "server.address", + "type": "string", + "value": "localhost" + }, + { + "key": "server.port", + "type": "int64", + "value": 8080 + }, + { + "key": "span.kind", + "type": "string", + "value": "server" + }, + { + "key": "thread.id", + "type": "int64", + "value": 146 + }, + { + "key": "thread.name", + "type": "string", + "value": "JettyServer-7" + }, + { + "key": "url.path", + "type": "string", + "value": "/xas/" + }, + { + "key": "url.scheme", + "type": "string", + "value": "http" + }, + { + "key": "user_agent.original", + "type": "string", + "value": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/141.0.7390.37 Safari/537.36" + } + ], + "logs": [], + "processID": "p1", + "warnings": null + }, + { + "traceID": "c4dc26854ecdb369548b4f882d2bd351", + "spanID": "42fe3d00af19a6d8", + "operationName": "Java action MockServices.JA_Json_IsValid", + "references": [ + { + "refType": "CHILD_OF", + "traceID": "c4dc26854ecdb369548b4f882d2bd351", + "spanID": "6b80ee02197cd6c0" + } + ], + "startTime": 1790409626711291, + "duration": 14513, + "tags": [ + { + "key": "mx.activity.input_variable_names", + "type": "string", + "value": "Scenario/CustomPayload" + }, + { + "key": "mx.activity.name", + "type": "string", + "value": "JavaAction" + }, + { + "key": "mx.activity.output_variable_name", + "type": "string", + "value": "IsJson" + }, + { + "key": "otel.scope.name", + "type": "string", + "value": "com.mendix.runtime" + }, + { + "key": "otel.status_code", + "type": "string", + "value": "OK" + }, + { + "key": "span.kind", + "type": "string", + "value": "server" + }, + { + "key": "thread.id", + "type": "int64", + "value": 146 + }, + { + "key": "thread.name", + "type": "string", + "value": "JettyServer-7" + } + ], + "logs": [], + "processID": "p1", + "warnings": null + }, + { + "traceID": "c4dc26854ecdb369548b4f882d2bd351", + "spanID": "581b94df924d35a3", + "operationName": "AddValidationFeedback activity", + "references": [ + { + "refType": "CHILD_OF", + "traceID": "c4dc26854ecdb369548b4f882d2bd351", + "spanID": "6b80ee02197cd6c0" + } + ], + "startTime": 1790409626726390, + "duration": 1084, + "tags": [ + { + "key": "mx.activity.input_variable_names", + "type": "string", + "value": "Scenario" + }, + { + "key": "mx.activity.name", + "type": "string", + "value": "AddValidationFeedback" + }, + { + "key": "mx.activity.output_variable_name", + "type": "string", + "value": "" + }, + { + "key": "otel.scope.name", + "type": "string", + "value": "com.mendix.runtime" + }, + { + "key": "otel.status_code", + "type": "string", + "value": "OK" + }, + { + "key": "span.kind", + "type": "string", + "value": "server" + }, + { + "key": "thread.id", + "type": "int64", + "value": 146 + }, + { + "key": "thread.name", + "type": "string", + "value": "JettyServer-7" + } + ], + "logs": [], + "processID": "p1", + "warnings": null + } + ], + "warnings": null + } + ], + "total": 0, + "limit": 0, + "offset": 0, + "errors": null +} diff --git a/tests/wave2/test-otel-capture-until.sh b/tests/wave2/test-otel-capture-until.sh new file mode 100644 index 0000000..167bc9c --- /dev/null +++ b/tests/wave2/test-otel-capture-until.sh @@ -0,0 +1,94 @@ +#!/usr/bin/env bash +# Usage: bash test-otel-capture-until.sh [path-to-otel.js] +# +# Fixture for project-tests/e2e/otel.js capture()'s stopping condition, and the journey +# runner's rung-2 use of it. +# +# The defect (card-disbursement requirements-driven build, 2026-09-26): rung 2 called +# capture(t0, { min: 1 }), which stops at the FIRST poll holding any span at all. The batch +# exporter routinely flushes a step's HTTP/xas spans before its microflow spans, so 2 of 6 +# `spans.ordered` claims failed "actual (none)" over sequences Jaeger held a second later +# (verified by querying Jaeger directly); the next run of the same journey was green. capture() +# now takes `until(spans)`, and the runner waits for the claimed microflows. +# +# Golden input is a real Jaeger /api/traces response (fixtures/otel-capture/ +# jaeger-trace-refused-arm.json: one trace, 8 spans, Mendix 11.13.0 runtime → Jaeger v1.76 OTLP), +# verbatim except `processes` dropped (capture() never reads it; it carries the host command +# line) and the checkout path replaced by /work/app. The flush ORDER is the synthetic part: a +# stubbed fetch serves the non-microflow spans on poll 1 and the whole trace from poll 2 — +# the shape observed in the field, reproduced deterministically. + +set -uo pipefail + +SUT="${1:-}" +[ -z "$SUT" ] && SUT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)/project-tests/e2e/otel.js" +if [ ! -f "$SUT" ]; then + echo "SKIP: subject not found at $SUT" + echo "SCORE: 0/0 — nothing to test" + exit 0 +fi +command -v node >/dev/null 2>&1 || { echo "SKIP: node not installed"; exit 0; } +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +GOLDEN="$HERE/fixtures/otel-capture/jaeger-trace-refused-arm.json" +[ -f "$GOLDEN" ] || { echo "FAIL — golden capture missing: $GOLDEN"; exit 1; } +RUNNER="$(dirname "$SUT")/journey-runner.js" + +PASS=0; FAIL=0 +ok() { PASS=$((PASS+1)); echo " ok — $1"; } +bad() { FAIL=$((FAIL+1)); echo " FAIL — $1"; [ -n "${2:-}" ] && echo " got: $2"; } + +WORK="$(mktemp -d "${TMPDIR:-/tmp}/otel-until.XXXXXX")" +trap 'rm -rf "$WORK"' EXIT +cp "$SUT" "$WORK/otel.js" +# otel.js reads only otelService from the config at require time. +echo "module.exports = { otelService: 'Fixture' };" > "$WORK/project.config.js" + +OUT="$(node - "$WORK/otel.js" "$GOLDEN" <<'EOF' +const fs = require('fs'); +const [sut, golden] = process.argv.slice(2); +const full = JSON.parse(fs.readFileSync(golden, 'utf8')); +const isMf = s => s.tags.some(t => t.key === 'mx.microflow.name'); +const firstFlush = { ...full, data: full.data.map(t => ({ ...t, spans: t.spans.filter(s => !isMf(s)) })) }; +let polls = 0; +global.fetch = async () => { polls++; const body = polls === 1 ? firstFlush : full; return { json: async () => body }; }; +const O = require(sut); +const claimed = ['MockServices.ACT_MockScenario_Arm', 'MockServices.VAL_MockScenario_Arm', 'MockServices.VAL_MockScenario_Payload']; +const names = sp => O.microflowNames(sp).sort().join(','); +(async () => { + const opts = { retries: 5, waitMs: 1 }; + polls = 0; let sp = await O.capture(0, { ...opts, min: 1 }); + console.log(`MIN1 polls=${polls} spans=${sp.length} mf=${names(sp) || '-'}`); + polls = 0; sp = await O.capture(0, { ...opts, min: 1, until: s => claimed.every(n => O.microflowNames(s).includes(n)) }); + console.log(`UNTIL polls=${polls} spans=${sp.length} mf=${names(sp) || '-'}`); + polls = 0; sp = await O.capture(0, { ...opts, min: 1, until: s => O.microflowNames(s).includes('MockServices.NEVER') }); + console.log(`NEVER polls=${polls} spans=${sp.length}`); +})().catch(e => { console.log(`ERR ${e.message}`); process.exit(3); }); +EOF +)" +rc=$? +[ "$rc" -eq 0 ] || { echo "FAIL — harness could not drive capture() from $SUT (rc=$rc)"; echo "$OUT"; exit 1; } +line() { printf '%s\n' "$OUT" | sed -n "s/^$1 //p"; } + +L="$(line MIN1)" +case "$L" in "polls=1 spans=5 mf=-") ok "min:1 alone stops at the first flush with 0 microflow spans (the race, characterised)" ;; + *) bad "golden replay did not reproduce the first-flush shape" "$L" ;; esac +L="$(line UNTIL)" +case "$L" in "polls=2 spans=8 mf="*VAL_MockScenario_Payload*) ok "until: keeps polling until the claimed microflows are present (poll 2, 8 spans)" ;; + *) bad "until: did not wait for the claimed microflows (the 2026-09-26 false red)" "$L" ;; esac +L="$(line NEVER)" +case "$L" in "polls=5 spans=8") ok "an until that never holds returns what it has after the retry window — no hang, no fake pass" ;; + *) bad "unsatisfiable until misbehaved" "$L" ;; esac + +if [ -f "$RUNNER" ]; then + if awk '/RUNG 2: ordered spans/{f=1} f&&/O\.capture\(/{c=1} c&&/until:/{print "yes"; exit} /RUNG 3/{exit}' "$RUNNER" | grep -q yes; then + ok "journey-runner.js rung 2 passes until: to capture()" + else + bad "journey-runner.js rung 2 still captures with min:1 alone" + fi +else + echo " skip — journey-runner.js not beside $SUT" +fi + +echo "" +echo "PASS=$PASS FAIL=$FAIL" +[ "$FAIL" -eq 0 ] From 1fdc58e36b52e4ab4a04637e5d88a8bdc891c07c Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 08:47:52 +0000 Subject: [PATCH 07/45] Fix three verify-module instruments that could not go green or misread the harness MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - conformance-check.sh: rows with no baseline line are baselined when first seen (a module built after the first run was never regression-guarded), and `--update-baseline --module X` keeps the other modules' rows. - journey-runner.js: a --positive-control run exits 0 when every rung's mutant was caught (runExitCode); the walk keeps exit-1-on-FAIL/INVALID. - monkey.js: Back leaving the app returns with Forward instead of scoring login.html/about:blank as a crash; sign-out buttons are not clicked; a lost target re-lands or reports how many rounds went unfuzzed; no main on require. Fixtures: test-conformance-baseline-scope.sh 7/7, test-journey-control-exit.sh 7/7 (captured control/walk findings), test-monkey-left-app.sh 11/11 (captured URLs and labels); each red against the previous script. Field run: card-disbursement requirements-driven build, MockServices module — conformance newly baselined 119 of 246, scoped update kept 207 other rows; control exit 0 at 7 of 7 rungs; monkey seed 410855898 10 -> 0 crash-class, 13 -> 26 observations, both targets fuzzed. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Pm41sykqDGcvioGzsY5H2N --- CHANGELOG.md | 3 + project-bin/conformance-check.sh | 37 +- project-tests/e2e/journey-runner.js | 25 +- project-tests/e2e/monkey.js | 52 +- .../fixtures/journey-control/control-run.json | 2393 +++++++++++++++++ .../fixtures/journey-control/walk-run.json | 408 +++ .../wave2/test-conformance-baseline-scope.sh | 109 + tests/wave2/test-journey-control-exit.sh | 98 + tests/wave2/test-monkey-left-app.sh | 104 + 9 files changed, 3216 insertions(+), 13 deletions(-) create mode 100644 tests/wave2/fixtures/journey-control/control-run.json create mode 100644 tests/wave2/fixtures/journey-control/walk-run.json create mode 100755 tests/wave2/test-conformance-baseline-scope.sh create mode 100755 tests/wave2/test-journey-control-exit.sh create mode 100755 tests/wave2/test-monkey-left-app.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index cbd5202..7d28390 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,9 @@ three commits past it), and a bug report can name a release instead of a sha nob Sections dated before 2026-09-19 predate the cycle and stay as they are. ## Unreleased +- fix(project-tests/e2e/journey-runner.js): **a `--positive-control` run now exits 0 when every rung's mutant was caught.** The walk's exit rule — exit 1 on any FAIL or INVALID row — was applied to the control run too. But a control's FAIL rows are its mutants being caught, so a control that proved 7 of 7 rungs exited 1, `verify-module.sh` graded the rung FINDING, and the module read INCOMPLETE on every run: a rung that could not go green, and so said nothing when it went red. `runExitCode()` keeps the walk's rule. A control now passes only when every `control` row is PASS, proven equals expected, and at least one rung ran. The summary now prints a `control verdict:` line that says which rows are the mutants'. Field run: MockServices `--positive-control` exited 0 with `control verdict: PASS`, 7 of 7 rungs (PASS 468, FAIL 7, INVALID 1 — unchanged, now read correctly). New fixture `tests/wave2/test-journey-control-exit.sh` over that run's captured findings (`fixtures/journey-control/`): 7 assertions, 7 green on the fix, red against the previous runner (no `runExitCode`). From a card-disbursement requirements-driven build (build-plan row 3.9). +- fix(project-tests/e2e/monkey.js): **a round that leaves the app is no longer scored as an app crash, and the monkey no longer signs itself out.** A `backAfterSubmit` round clicked a control that pushes no history entry (a theme toggle), so Back left the app for `login.html` and then `about:blank`. The oracle scored each of those pages "blank page", every later round on the target scored the same page again, and the next target was never reached. Now, when Back leaves the app, the round returns with Forward and says so. Buttons labelled sign out / log off are no longer clicked. A round that leaves no page behind re-lands on its target, or reports `lost` with the number of rounds left unfuzzed. Requiring the file no longer runs main, and it exports `leftApp` / `SIGN_OUT`. Field run (seed 410855898, 2 targets): 13 observations with 10 crash-class and one target never reached → 26 observations, 0 crash-class, both targets fuzzed. New fixture `tests/wave2/test-monkey-left-app.sh` over the URLs and labels captured in that run: 11 assertions, 11 green on the fix, red against the previous monkey (no exports; it ran main on require). From a card-disbursement requirements-driven build (build-plan row 3.9). +- fix(project-bin/conformance-check.sh): **a module built after the first conformance run is now regression-guarded, and `--update-baseline --module X` no longer wipes the other modules' baseline rows.** The baseline is written by the first run. A row it had never seen was compared with "", so it could never regress — silently and permanently for every module built later. That project's committed baseline held 1 of its 4 modules (127 of 246 rows). Separately, `--module` scoped the measurement but not the write, so a scoped `--update-baseline` replaced the whole file with one module's rows. Rows with no baseline line are now baselined when first seen, and the run prints `newly baselined N`. A `--module` rewrite keeps every other module's rows. An unscoped `--update-baseline` still rewrites the whole file. New fixture `tests/wave2/test-conformance-baseline-scope.sh` (stubbed mxcli; the assertions are about the baseline file, not the DESCRIBE parser): 7 assertions, 7 green on the fix, 5 red against the previous script. Field run on that project (mxcli v0.24.0, 4 modules, 291 ledger rows): from its committed 127-row baseline, a plain run measured 246 OK and printed `newly baselined 119`, leaving a 246-row baseline. `--module MockServices --update-baseline` then rewrote only that module's 39 rows and kept the other 207. From a card-disbursement requirements-driven build (build-plan row 3.9). - fix(project-tests/e2e/otel.js, journey-runner.js): **rung 2 now waits for the claimed microflow spans, not just the first span of any kind.** Before, `capture(t0, { min: 1 })` stopped at the first poll that held any span. The batch exporter routinely flushes a step's HTTP/xas spans before its microflow spans, so `spans.ordered` claims failed "actual (none)" over sequences Jaeger held a second later. `capture()` now takes `until(spans)`, and rung 2 passes "every claimed microflow is present". A claim that never holds still fails, after the normal retry window. Field run (Mendix 11.13.0 → Jaeger v1.76 over OTLP, 6 ordered claims, 3 of them with `mustNotFire`): 2 of 6 red before the fix, the same journey 79/0/0 after, and `--positive-control` 7 of 7 rungs proven (trace-order and trace-negative were UNPROVEN before, because no journey declared a span claim). New fixture `tests/wave2/test-otel-capture-until.sh` over a real Jaeger `/api/traces` capture: 4 assertions, 4 green on the fix, 3 red against the previous code. From a card-disbursement requirements-driven build (build-plan row 3.9). - fix(project-tests/e2e/report-normalize.js): **the report now reads its inputs and writes `docs/report.json` in the two-tree layout (model under `app/`).** Every input resolved against project.config's ROOT, which is `app/` there, so `verify-module.sh`'s report step marked 8 of 9 instruments FAULT ("design-audit.json does not exist", "no coverage-ledger.md under architecture/modules/*/") over files that were on disk, and wrote the report into a stray `app/docs/report.json`, which left the render step with nothing to render. Inputs now try ROOT first (the `.mpr`, `deployment/model/*` and `.mxcli/catalog.db` live there), then the repo root; outputs go to the repo root. This is the same fix as `project.config.js` `designPath()` and design-audit's `nearRoot()` (F-042). `JOURNEY_DIR` is now honoured and re-expressed relative to the project, so reproduce commands carry no machine path. `meta.projectId` uses the config's `id`, not `basename(ROOT)`, which was `app`. Field run (Mendix 11.13.0, 4 modules): 0 → 205 checks read, instrument FAULTs 8 → 2, and those 2 are real (no deferrals or run-ledger file). The report is back at the repo root. New fixture `tests/wave2/test-report-normalize-two-tree.sh` (two-tree plus a single-tree regression): 8 assertions, 8 green on the fix, 6 red against the previous script. `--selftest` fails identically before and after (it expects a `mobile-fieldscan` walkthrough the template config does not declare); it was not touched here. From a card-disbursement requirements-driven build (build-plan row 3.9). - fix(project-tests/e2e/design-audit.js): **rung 7 (a11y, overflow, structure) now finds the menu route of every page that has an icon.** The navigation parser anchored on `;` straight after the page name, but real `describe navigation` output puts `icon ` there (and before a group's `(`), so no item with an icon was routed and every page reported FAULT "no navigation route — this check never ran". Parsing is now `parseNavigation(txt)`, matching up to the terminator. Field run (mxcli v0.24.0, Mendix 11.13.0): routed pages 0 → 4 of 4 menu pages; rung 7 measured 4 of 7 in-scope pages instead of 0, and immediately found color-contrast, missing landmarks and an aria-required-children critical that had been invisible. The other 3 are NewEdit pages opened from another page, still FAULT by design. New fixture `tests/wave2/test-design-audit-nav-map.sh` over a verbatim capture: 6 assertions, 6 green on the fix, 5 red against the previous regexes. From a card-disbursement requirements-driven build (build-plan row 3.9). diff --git a/project-bin/conformance-check.sh b/project-bin/conformance-check.sh index e916b35..619df90 100755 --- a/project-bin/conformance-check.sh +++ b/project-bin/conformance-check.sh @@ -15,6 +15,8 @@ # Failure policy: regressions only. A row that was OK in the baseline and is no longer OK # fails the run. Pre-existing mismatches are reported but do not fail — the lesson from lint, # which was made optional because day-one noise made it unusable, and now never runs at all. +# A row with no baseline line yet (a module built after the baseline was written) is baselined +# on first sight and said so; `--module` scopes a baseline rewrite to that module's rows. # # When there is no ledger # @@ -58,7 +60,7 @@ while [ $# -gt 0 ]; do --module) MODULE="${2:-}"; shift 2 ;; --update-baseline) UPDATE_BASELINE=1; shift ;; --quiet) QUIET=1; shift ;; - -h|--help) sed -n '2,43p' "$0"; exit 0 ;; + -h|--help) sed -n '2,45p' "$0"; exit 0 ;; *) echo "unknown argument: $1" >&2; exit 2 ;; esac done @@ -287,23 +289,43 @@ RESULTS="$(printf '%s' "$RESULTS" | sed '/^$/d' | sort)" # --- baseline / regression -------------------------------------------------------------- KEYED="$(printf '%s\n' "$RESULTS" | awk -F'\t' '{print $1"\t"$2"\t"$5}')" +# `--module` measures one module, so it may rewrite only that module's rows. Writing $KEYED +# whole dropped every other module's baseline, and their regressions went unmeasured with +# nothing said (card-disbursement build, 2026-09-26: `--update-baseline --module MockServices` +# dropped the 127 Disbursement rows). if [ "$UPDATE_BASELINE" -eq 1 ] || [ ! -f "$BASELINE" ]; then - printf '%s\n' "$KEYED" > "$BASELINE" + if [ -n "$MODULE" ] && [ -f "$BASELINE" ]; then + { awk -F'\t' -v m="$MODULE" '$1!=m' "$BASELINE"; printf '%s\n' "$KEYED"; } \ + | sed '/^$/d' | sort > "$BASELINE.tmp" && mv "$BASELINE.tmp" "$BASELINE" + else + printf '%s\n' "$KEYED" > "$BASELINE" + fi WROTE_BASELINE=1 else WROTE_BASELINE=0 fi +# A row the baseline has never seen (a module built after the first run wrote it) used to be +# compared with "" — never a regression, forever, with nothing said. Its first measurement is +# its baseline, exactly as the first run's were; the count is printed below. REGRESSIONS="" +SEEDED="" if [ "$WROTE_BASELINE" -eq 0 ]; then while IFS=$'\t' read -r mod ptr verdict; do [ -n "$mod" ] || continue - was="$(awk -F'\t' -v m="$mod" -v p="$ptr" '$1==m && $2==p {print $3}' "$BASELINE")" - if [ "$was" = "OK" ] && [ "$verdict" != "OK" ]; then + was="$(awk -F'\t' -v m="$mod" -v p="$ptr" '$1==m && $2==p {print $3; exit}' "$BASELINE")" + if [ -z "$was" ]; then + SEEDED+="$mod $ptr $verdict"$'\n' + elif [ "$was" = "OK" ] && [ "$verdict" != "OK" ]; then REGRESSIONS+="$mod $ptr was OK, now $verdict"$'\n' fi done <<< "$KEYED" + if [ -n "$SEEDED" ]; then + { cat "$BASELINE"; printf '%s' "$SEEDED"; } | sed '/^$/d' | sort > "$BASELINE.tmp" \ + && mv "$BASELINE.tmp" "$BASELINE" + fi fi +NSEED=$(printf '%s' "$SEEDED" | grep -c . || true) # --- report ------------------------------------------------------------------------------ STAMP="$(date +%Y-%m-%d)" @@ -325,6 +347,7 @@ echo " UNDERSTATED $UNDER" [ "$TMO" -gt 0 ] && echo " TIMEOUT $TMO (>${TIMEOUT_S}s — never counted as a pass)" [ "$UNK" -gt 0 ] && echo " UNKNOWN-STATUS $UNK" echo " report $REPORT" +[ "$NSEED" -gt 0 ] && echo " newly baselined $NSEED (no baseline row yet — regressions measured from the next run)" if [ "$STALE" -gt 0 ] || [ "$UNDER" -gt 0 ]; then echo @@ -334,7 +357,11 @@ fi if [ "$WROTE_BASELINE" -eq 1 ]; then echo - echo " baseline written to $BASELINE ($MEASURABLE rows). Regressions are measured from here." + if [ -n "$MODULE" ]; then + echo " baseline rows for $MODULE rewritten in $BASELINE ($MEASURABLE rows; other modules' rows kept)." + else + echo " baseline written to $BASELINE ($MEASURABLE rows). Regressions are measured from here." + fi exit 0 fi diff --git a/project-tests/e2e/journey-runner.js b/project-tests/e2e/journey-runner.js index 2e15c6a..165820a 100644 --- a/project-tests/e2e/journey-runner.js +++ b/project-tests/e2e/journey-runner.js @@ -795,12 +795,32 @@ async function runJourney(page, j) { } } +// ── Exit code ─────────────────────────────────────────────────────────────── +// A walk passes when nothing FAILed and nothing was INVALID. A CONTROL run asks the other +// question — can the harness tell when the app is broken — so its FAIL rows are the mutants +// doing their job, and it passes only when every rung's mutant was caught (each `control` +// row PASS, proven == expected, and at least one rung tried). +// +// MEASURED 2026-09-26 (card-disbursement requirements-driven build): the walk's rule was +// applied to both runs. A control that proved 7 of 7 rungs exited 1 on its own 7 mutant +// FAILs plus the INVALID a broken landing leaves downstream, so verify-module graded the +// rung FINDING and the module INCOMPLETE on every run — a control rung that could not go +// green, and so told the reader nothing when it went red. +function runExitCode(positiveControl, results, mutants) { + if (!positiveControl) return results.some(r => r.verdict === 'FAIL' || r.verdict === 'INVALID') ? 1 : 0; + const ctl = results.filter(r => r.rung === 'control'); + return mutants.expected > 0 && mutants.proven === mutants.expected + && ctl.length === mutants.expected && ctl.every(r => r.verdict === 'PASS') ? 0 : 1; +} + // ── Exports for the rung-4 scope unit test ────────────────────────────────── // The SQL builders are pure and exported so their behaviour can be proven against // fixture rows without a running app. See tests/e2e/journey-rung4-scope.test.js. +// runExitCode is exported for tests/wave2/test-journey-control-exit.sh. module.exports = { sqlRowCount, sqlWatermark, whereScoped, scopeLiteral, sqlAssocTotal, sqlAssocLinked, sqlMustPointAt, captureScope, scopeEvidence, + runExitCode, }; // ── Main ──────────────────────────────────────────────────────────────────── @@ -920,6 +940,9 @@ if (require.main !== module) return; console.log(' Unproven is fault, not pass. Declare the missing claim on the journey,'); console.log(' or accept that this rung has never been shown able to go red.'); } + console.log(runExitCode(true, results, mutants) === 0 + ? ' control verdict: PASS — every rung\'s mutant was caught; the FAIL/INVALID rows above are the mutants\'' + : ' control verdict: FAIL — at least one rung is unproven; read the [control] rows above'); } fs.mkdirSync(cfg.artifactsDir, { recursive: true }); @@ -942,7 +965,7 @@ if (require.main !== module) return; }, null, 2)); console.log(` findings → ${path.relative(cfg.root, out)}`); - process.exit(n('FAIL') === 0 && n('INVALID') === 0 ? 0 : 1); + process.exit(runExitCode(POSITIVE_CONTROL, results, mutants)); })().catch(e => { console.error('ERR', e.stack?.split('\n').slice(0, 8).join('\n')); process.exit(1); diff --git a/project-tests/e2e/monkey.js b/project-tests/e2e/monkey.js index ed00617..a3dd128 100644 --- a/project-tests/e2e/monkey.js +++ b/project-tests/e2e/monkey.js @@ -147,13 +147,38 @@ async function typeSafely(page, handle, text) { }, b64); } +// ── leaving the app is the harness's move, not the app's crash ───────────── +// MEASURED 2026-09-26 (card-disbursement requirements-driven build, seed 410855898): on the +// landing page a backAfterSubmit clicked a control that pushes no history entry (a theme +// toggle), so Back left the app — to login.html, then to about:blank. The oracle scored the +// non-app page "blank page", every later round on that target scored it again, and the next +// target was never reached: 10 crash-class findings, one target unfuzzed, no app defect. +const SIGN_OUT = /\b(sign|log)\s*(out|off)\b/i; +function leftApp(url, baseUrl) { + const base = String(baseUrl).replace(/\/+$/, ''); + if (!url || (url !== base && !url.startsWith(base + '/'))) return true; + return /\/login\.html(?:[?#]|$)/.test(url); +} + +// Open a NAV target, from outside the app if a round left it. +async function land(page, group, item) { + if (leftApp(page.url(), cfg.baseUrl)) await page.goto(cfg.baseUrl + '/', { timeout: 20000 }).catch(() => {}); + await H.navTo(page, group, item).catch(() => {}); + await page.waitForTimeout(2000); + return page.locator('.mx-page').first().isVisible({ timeout: 8000 }).catch(() => false); +} + // ── one round ─────────────────────────────────────────────────────────────── async function round(page, target, bucket) { bucket.jsErrors = []; bucket.http5xx = []; const t0 = Date.now(); const inputs = page.locator('.mx-page input:visible, .mx-page textarea:visible'); - const buttons = page.locator('.mx-page button:visible, .mx-page .mx-button:visible'); + // Sign-out controls are excluded: a monkey that signs itself out measures the login page + // for the rest of the target (same run as leftApp: an app shell with a Sign out button + // inside the page, clicked by a reloadMidFlow round). + const buttons = page.locator('.mx-page button:visible, .mx-page .mx-button:visible') + .filter({ hasNotText: SIGN_OUT }); const nIn = await inputs.count().catch(() => 0); const nBtn = await buttons.count().catch(() => 0); @@ -176,6 +201,11 @@ async function round(page, target, bucket) { if (btn) await btn.click({ force: true, timeout: 4000 }).catch(() => {}); await page.waitForTimeout(1500); await page.goBack({ timeout: 6000 }).catch(() => {}); + if (leftApp(page.url(), cfg.baseUrl)) { + // The oracle below then scores the page Forward returns to — the abuse still ran. + await page.goForward({ timeout: 10000 }).catch(() => {}); + what += ' (Back left the app: the click pushed no history entry; returned with Forward)'; + } } else if (abuse === 'rapidTab') { for (let i = 0; i < 15; i++) await page.keyboard.press('Tab').catch(() => {}); await page.keyboard.press('Enter').catch(() => {}); @@ -193,8 +223,11 @@ async function round(page, target, bucket) { return traceRan; } +// leftApp is exported for tests/wave2/test-monkey-left-app.sh; requiring this file runs nothing. +module.exports = { leftApp, SIGN_OUT }; + // ── main ──────────────────────────────────────────────────────────────────── -(async () => { +if (require.main === module) (async () => { console.log(`MONKEY_SEED=${SEED} rounds=${ROUNDS} (re-run exactly: --seed ${SEED})`); const { browser, page } = await H.launchBrowser(); @@ -221,12 +254,17 @@ async function round(page, target, bucket) { const nav = NAV[key]; const [group, item] = nav.length === 1 ? [null, nav[0]] : nav; console.log(`\n=== ${key}`); - await H.navTo(page, group, item).catch(() => {}); - await page.waitForTimeout(2000); - const landed = await page.locator('.mx-page').first().isVisible({ timeout: 8000 }).catch(() => false); - if (!landed) { finding('info', key, 'not reached', 'nav did not land — target skipped, NOT fuzzed'); continue; } - for (let i = 0; i < Math.ceil(ROUNDS / targets.length); i++) { + if (!(await land(page, group, item))) { finding('info', key, 'not reached', 'nav did not land — target skipped, NOT fuzzed'); continue; } + const n = Math.ceil(ROUNDS / targets.length); + for (let i = 0; i < n; i++) { anyTrace = (await round(page, key, bucket)) || anyTrace; + // A round that leaves no page behind would have every later round re-score the same + // screen. Re-land; if that fails, say how much of the target went unfuzzed. + if (!(await page.locator('.mx-page').first().isVisible({ timeout: 3000 }).catch(() => false)) + && !(await land(page, group, item))) { + finding('info', key, 'lost', `no page after round ${i + 1} (at ${page.url()}) and re-landing failed — the remaining ${n - i - 1} round(s) NOT fuzzed`); + break; + } } } diff --git a/tests/wave2/fixtures/journey-control/control-run.json b/tests/wave2/fixtures/journey-control/control-run.json new file mode 100644 index 0000000..6c50a9d --- /dev/null +++ b/tests/wave2/fixtures/journey-control/control-run.json @@ -0,0 +1,2393 @@ +{ + "capturedAt": "2026-09-26T08:20:55.521Z", + "positiveControl": true, + "mutants": { + "expectedPerJourney": 7, + "expected": 7, + "supported": 7, + "proven": 7, + "unsupported": [] + }, + "results": [ + { + "rung": "ui", + "name": "login", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: reached", + "verdict": "FAIL" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: downstream rungs", + "verdict": "INVALID" + }, + { + "rung": "control", + "name": "MockServices · ui-landing: first step lands on a widget that does not exist", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: widget btnReset", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: widget lvSystems", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: widget lvPackages", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: page says \"__text_this_page_will_never_render__\"", + "verdict": "FAIL" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: confirmation says \"Reset the mock hub?\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: page says \"DO_SIGN_CONTRACTS\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: page says \"7710001\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Reset the hub to its baseline: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset the hub to its baseline: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset the hub to its baseline: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: Reset the hub to its baseline: REST_ did not run", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: the call log is empty", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: no scenario is armed", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: all six systems are Online", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: one default package", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: two applications, none KO", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: radio rbKind checked \"Custom payload\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: widget btnDlgCancel", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: page says \"Not valid JSON\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] A custom payload that is not JSON is refused: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] A custom payload that is not JSON is refused: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] A custom payload that is not JSON is refused: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: A custom payload that is not JSON is refused: ^MockServices\\.SUB_MockScenario_Arm$ did not run", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "A custom payload that is not JSON is refused: nothing was armed", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: radio rbKind checked \"Fault\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: page says \"MWF-503\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: page says \"· once\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: page does NOT say \"Not valid JSON\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Arm a one-shot fault on DO_SIGN_CONTRACTS: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Arm a one-shot fault on DO_SIGN_CONTRACTS: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Arm a one-shot fault on DO_SIGN_CONTRACTS: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: Arm a one-shot fault on DO_SIGN_CONTRACTS: VAL_MockScenario_Payload did not run", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: DO_SIGN_CONTRACTS is armed with fault MWF-503 / 503", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: only that one is armed", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: DO_SIGN_CONTRACTS (armed) answers 503", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: DO_SIGN_CONTRACTS (armed) response has MWF-503", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: DO_SIGN_CONTRACTS (after the one shot) answers 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: MockServices.MockCallLog +2", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: the call log holds the 503 then the 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: the one-shot disarmed itself; both calls counted", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: GET_SYSTEM_STATUS answers 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: GET_SYSTEM_STATUS response has \"StatusCode\":\"0\",\"StatusDescription\":\"Offline\",\"System\":\"GM4\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Take GM4 offline: reached", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Take GM4 offline: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Take GM4 offline: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Take GM4 offline: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: GM4 answers StatusCode 0", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: the other five stay Online", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Mark application 7710001 KO: DO_START_APPROVAL answers 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Mark application 7710001 KO: DO_START_APPROVAL response has \"APPROVED_FLAG\":false", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Mark application 7710001 KO: reached", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Mark application 7710001 KO: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Mark application 7710001 KO: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Mark application 7710001 KO: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Mark application 7710001 KO: 7710001 is KO, 7710002 is not", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Cancelling the reset changes nothing: confirmation says \"Reset the mock hub?\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Cancelling the reset changes nothing: reached", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Cancelling the reset changes nothing: GM4 is still offline", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Cancelling the reset changes nothing: 7710001 is still KO", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Cancelling the reset changes nothing: the call log is kept", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset restores the baseline: confirmation says \"Reset the mock hub?\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset restores the baseline: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset restores the baseline: page does NOT say \"MWF-503\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Reset restores the baseline: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset restores the baseline: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset restores the baseline: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: Reset restores the baseline: REST_ did not run", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: MockServices.MockApplication_MockPackage set on every row", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: MockServices.MockApplication_MockPackage points at the seeded ProdPackageKey", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: the call log is empty", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: no scenario is armed and no hit count survives", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: no application is KO", + "verdict": "PASS" + }, + { + "rung": "outcome", + "name": "MockServices-CTL-ui-text end state", + "verdict": "PASS" + }, + { + "rung": "control", + "name": "MockServices · ui-text: expect text the page never renders", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: widget btnReset", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: widget lvSystems", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: widget lvPackages", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: page says \"Mock console\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: page says \"DO_SIGN_CONTRACTS\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: confirmation says \"Reset the mock hub?\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: page says \"DO_SIGN_CONTRACTS\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: page says \"7710001\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Reset the hub to its baseline: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset the hub to its baseline: ordered", + "verdict": "FAIL" + }, + { + "rung": "trace", + "name": "[MF] Reset the hub to its baseline: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: the call log is empty", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: no scenario is armed", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: all six systems are Online", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: one default package", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: two applications, none KO", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: radio rbKind checked \"Custom payload\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: widget btnDlgCancel", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: page says \"Not valid JSON\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] A custom payload that is not JSON is refused: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] A custom payload that is not JSON is refused: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] A custom payload that is not JSON is refused: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: A custom payload that is not JSON is refused: ^MockServices\\.SUB_MockScenario_Arm$ did not run", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "A custom payload that is not JSON is refused: nothing was armed", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: radio rbKind checked \"Fault\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: page says \"MWF-503\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: page says \"· once\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: page does NOT say \"Not valid JSON\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Arm a one-shot fault on DO_SIGN_CONTRACTS: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Arm a one-shot fault on DO_SIGN_CONTRACTS: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Arm a one-shot fault on DO_SIGN_CONTRACTS: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: Arm a one-shot fault on DO_SIGN_CONTRACTS: VAL_MockScenario_Payload did not run", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: DO_SIGN_CONTRACTS is armed with fault MWF-503 / 503", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: only that one is armed", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: DO_SIGN_CONTRACTS (armed) answers 503", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: DO_SIGN_CONTRACTS (armed) response has MWF-503", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: DO_SIGN_CONTRACTS (after the one shot) answers 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: MockServices.MockCallLog +2", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: the call log holds the 503 then the 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: the one-shot disarmed itself; both calls counted", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: GET_SYSTEM_STATUS answers 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: GET_SYSTEM_STATUS response has \"StatusCode\":\"0\",\"StatusDescription\":\"Offline\",\"System\":\"GM4\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Take GM4 offline: reached", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Take GM4 offline: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Take GM4 offline: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Take GM4 offline: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: GM4 answers StatusCode 0", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: the other five stay Online", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Mark application 7710001 KO: DO_START_APPROVAL answers 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Mark application 7710001 KO: DO_START_APPROVAL response has \"APPROVED_FLAG\":false", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Mark application 7710001 KO: reached", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Mark application 7710001 KO: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Mark application 7710001 KO: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Mark application 7710001 KO: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Mark application 7710001 KO: 7710001 is KO, 7710002 is not", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Cancelling the reset changes nothing: confirmation says \"Reset the mock hub?\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Cancelling the reset changes nothing: reached", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Cancelling the reset changes nothing: GM4 is still offline", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Cancelling the reset changes nothing: 7710001 is still KO", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Cancelling the reset changes nothing: the call log is kept", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset restores the baseline: confirmation says \"Reset the mock hub?\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset restores the baseline: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset restores the baseline: page does NOT say \"MWF-503\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Reset restores the baseline: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset restores the baseline: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset restores the baseline: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: Reset restores the baseline: REST_ did not run", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: MockServices.MockApplication_MockPackage set on every row", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: MockServices.MockApplication_MockPackage points at the seeded ProdPackageKey", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: the call log is empty", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: no scenario is armed and no hit count survives", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: no application is KO", + "verdict": "PASS" + }, + { + "rung": "outcome", + "name": "MockServices-CTL-trace-order end state", + "verdict": "PASS" + }, + { + "rung": "control", + "name": "MockServices · trace-order: expect a microflow that never fires", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: widget btnReset", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: widget lvSystems", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: widget lvPackages", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: page says \"Mock console\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: page says \"DO_SIGN_CONTRACTS\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: confirmation says \"Reset the mock hub?\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: page says \"DO_SIGN_CONTRACTS\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: page says \"7710001\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Reset the hub to its baseline: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset the hub to its baseline: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset the hub to its baseline: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: Reset the hub to its baseline: ACT_Mock_Reset did not run", + "verdict": "FAIL" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: the call log is empty", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: no scenario is armed", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: all six systems are Online", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: one default package", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: two applications, none KO", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: radio rbKind checked \"Custom payload\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: widget btnDlgCancel", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: page says \"Not valid JSON\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] A custom payload that is not JSON is refused: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] A custom payload that is not JSON is refused: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] A custom payload that is not JSON is refused: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: A custom payload that is not JSON is refused: ^MockServices\\.SUB_MockScenario_Arm$ did not run", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "A custom payload that is not JSON is refused: nothing was armed", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: radio rbKind checked \"Fault\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: page says \"MWF-503\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: page says \"· once\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: page does NOT say \"Not valid JSON\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Arm a one-shot fault on DO_SIGN_CONTRACTS: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Arm a one-shot fault on DO_SIGN_CONTRACTS: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Arm a one-shot fault on DO_SIGN_CONTRACTS: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: Arm a one-shot fault on DO_SIGN_CONTRACTS: VAL_MockScenario_Payload did not run", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: DO_SIGN_CONTRACTS is armed with fault MWF-503 / 503", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: only that one is armed", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: DO_SIGN_CONTRACTS (armed) answers 503", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: DO_SIGN_CONTRACTS (armed) response has MWF-503", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: DO_SIGN_CONTRACTS (after the one shot) answers 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: MockServices.MockCallLog +2", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: the call log holds the 503 then the 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: the one-shot disarmed itself; both calls counted", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: GET_SYSTEM_STATUS answers 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: GET_SYSTEM_STATUS response has \"StatusCode\":\"0\",\"StatusDescription\":\"Offline\",\"System\":\"GM4\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Take GM4 offline: reached", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Take GM4 offline: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Take GM4 offline: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Take GM4 offline: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: GM4 answers StatusCode 0", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: the other five stay Online", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Mark application 7710001 KO: DO_START_APPROVAL answers 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Mark application 7710001 KO: DO_START_APPROVAL response has \"APPROVED_FLAG\":false", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Mark application 7710001 KO: reached", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Mark application 7710001 KO: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Mark application 7710001 KO: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Mark application 7710001 KO: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Mark application 7710001 KO: 7710001 is KO, 7710002 is not", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Cancelling the reset changes nothing: confirmation says \"Reset the mock hub?\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Cancelling the reset changes nothing: reached", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Cancelling the reset changes nothing: GM4 is still offline", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Cancelling the reset changes nothing: 7710001 is still KO", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Cancelling the reset changes nothing: the call log is kept", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset restores the baseline: confirmation says \"Reset the mock hub?\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset restores the baseline: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset restores the baseline: page does NOT say \"MWF-503\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Reset restores the baseline: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset restores the baseline: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset restores the baseline: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: Reset restores the baseline: REST_ did not run", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: MockServices.MockApplication_MockPackage set on every row", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: MockServices.MockApplication_MockPackage points at the seeded ProdPackageKey", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: the call log is empty", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: no scenario is armed and no hit count survives", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: no application is KO", + "verdict": "PASS" + }, + { + "rung": "outcome", + "name": "MockServices-CTL-trace-negative end state", + "verdict": "PASS" + }, + { + "rung": "control", + "name": "MockServices · trace-negative: assert a microflow that DID run must not have run", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: widget btnReset", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: widget lvSystems", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: widget lvPackages", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: page says \"Mock console\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: page says \"DO_SIGN_CONTRACTS\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: confirmation says \"Reset the mock hub?\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: page says \"DO_SIGN_CONTRACTS\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: page says \"7710001\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Reset the hub to its baseline: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset the hub to its baseline: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset the hub to its baseline: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: Reset the hub to its baseline: REST_ did not run", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: the call log is empty", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: no scenario is armed", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: all six systems are Online", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: one default package", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: two applications, none KO", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: radio rbKind checked \"Custom payload\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: widget btnDlgCancel", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: page says \"Not valid JSON\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] A custom payload that is not JSON is refused: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] A custom payload that is not JSON is refused: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] A custom payload that is not JSON is refused: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: A custom payload that is not JSON is refused: ^MockServices\\.SUB_MockScenario_Arm$ did not run", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "A custom payload that is not JSON is refused: nothing was armed", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: radio rbKind checked \"Fault\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: page says \"MWF-503\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: page says \"· once\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: page does NOT say \"Not valid JSON\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Arm a one-shot fault on DO_SIGN_CONTRACTS: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Arm a one-shot fault on DO_SIGN_CONTRACTS: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Arm a one-shot fault on DO_SIGN_CONTRACTS: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: Arm a one-shot fault on DO_SIGN_CONTRACTS: VAL_MockScenario_Payload did not run", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: DO_SIGN_CONTRACTS is armed with fault MWF-503 / 503", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: only that one is armed", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: DO_SIGN_CONTRACTS (armed) answers 503", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: DO_SIGN_CONTRACTS (armed) response has MWF-503", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: DO_SIGN_CONTRACTS (after the one shot) answers 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: MockServices.MockCallLog +3", + "verdict": "FAIL" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: the call log holds the 503 then the 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: the one-shot disarmed itself; both calls counted", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: GET_SYSTEM_STATUS answers 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: GET_SYSTEM_STATUS response has \"StatusCode\":\"0\",\"StatusDescription\":\"Offline\",\"System\":\"GM4\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Take GM4 offline: reached", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Take GM4 offline: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Take GM4 offline: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Take GM4 offline: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: GM4 answers StatusCode 0", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: the other five stay Online", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Mark application 7710001 KO: DO_START_APPROVAL answers 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Mark application 7710001 KO: DO_START_APPROVAL response has \"APPROVED_FLAG\":false", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Mark application 7710001 KO: reached", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Mark application 7710001 KO: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Mark application 7710001 KO: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Mark application 7710001 KO: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Mark application 7710001 KO: 7710001 is KO, 7710002 is not", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Cancelling the reset changes nothing: confirmation says \"Reset the mock hub?\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Cancelling the reset changes nothing: reached", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Cancelling the reset changes nothing: GM4 is still offline", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Cancelling the reset changes nothing: 7710001 is still KO", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Cancelling the reset changes nothing: the call log is kept", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset restores the baseline: confirmation says \"Reset the mock hub?\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset restores the baseline: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset restores the baseline: page does NOT say \"MWF-503\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Reset restores the baseline: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset restores the baseline: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset restores the baseline: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: Reset restores the baseline: REST_ did not run", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: MockServices.MockApplication_MockPackage set on every row", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: MockServices.MockApplication_MockPackage points at the seeded ProdPackageKey", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: the call log is empty", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: no scenario is armed and no hit count survives", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: no application is KO", + "verdict": "PASS" + }, + { + "rung": "outcome", + "name": "MockServices-CTL-data-delta end state", + "verdict": "PASS" + }, + { + "rung": "control", + "name": "MockServices · data-delta: expect +2 rows from a +1 action", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: widget btnReset", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: widget lvSystems", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: widget lvPackages", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: page says \"Mock console\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: page says \"DO_SIGN_CONTRACTS\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: confirmation says \"Reset the mock hub?\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: page says \"DO_SIGN_CONTRACTS\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: page says \"7710001\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Reset the hub to its baseline: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset the hub to its baseline: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset the hub to its baseline: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: Reset the hub to its baseline: REST_ did not run", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: the call log is empty", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: no scenario is armed", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: all six systems are Online", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: one default package", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: two applications, none KO", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: radio rbKind checked \"Custom payload\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: widget btnDlgCancel", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: page says \"Not valid JSON\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] A custom payload that is not JSON is refused: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] A custom payload that is not JSON is refused: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] A custom payload that is not JSON is refused: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: A custom payload that is not JSON is refused: ^MockServices\\.SUB_MockScenario_Arm$ did not run", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "A custom payload that is not JSON is refused: nothing was armed", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: radio rbKind checked \"Fault\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: page says \"MWF-503\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: page says \"· once\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: page does NOT say \"Not valid JSON\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Arm a one-shot fault on DO_SIGN_CONTRACTS: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Arm a one-shot fault on DO_SIGN_CONTRACTS: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Arm a one-shot fault on DO_SIGN_CONTRACTS: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: Arm a one-shot fault on DO_SIGN_CONTRACTS: VAL_MockScenario_Payload did not run", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: DO_SIGN_CONTRACTS is armed with fault MWF-503 / 503", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: only that one is armed", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: DO_SIGN_CONTRACTS (armed) answers 503", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: DO_SIGN_CONTRACTS (armed) response has MWF-503", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: DO_SIGN_CONTRACTS (after the one shot) answers 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: MockServices.MockCallLog +2", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: the call log holds the 503 then the 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: the one-shot disarmed itself; both calls counted", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: GET_SYSTEM_STATUS answers 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: GET_SYSTEM_STATUS response has \"StatusCode\":\"0\",\"StatusDescription\":\"Offline\",\"System\":\"GM4\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Take GM4 offline: reached", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Take GM4 offline: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Take GM4 offline: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Take GM4 offline: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: GM4 answers StatusCode 0", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: the other five stay Online", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Mark application 7710001 KO: DO_START_APPROVAL answers 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Mark application 7710001 KO: DO_START_APPROVAL response has \"APPROVED_FLAG\":false", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Mark application 7710001 KO: reached", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Mark application 7710001 KO: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Mark application 7710001 KO: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Mark application 7710001 KO: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Mark application 7710001 KO: 7710001 is KO, 7710002 is not", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Cancelling the reset changes nothing: confirmation says \"Reset the mock hub?\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Cancelling the reset changes nothing: reached", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Cancelling the reset changes nothing: GM4 is still offline", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Cancelling the reset changes nothing: 7710001 is still KO", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Cancelling the reset changes nothing: the call log is kept", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset restores the baseline: confirmation says \"Reset the mock hub?\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset restores the baseline: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset restores the baseline: page does NOT say \"MWF-503\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Reset restores the baseline: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset restores the baseline: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset restores the baseline: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: Reset restores the baseline: REST_ did not run", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: MockServices.MockApplication_MockPackage set on every row", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: MockServices.MockApplication_MockPackage points at the seeded ProdPackageKey", + "verdict": "FAIL" + }, + { + "rung": "data", + "name": "Reset restores the baseline: the call log is empty", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: no scenario is armed and no hit count survives", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: no application is KO", + "verdict": "PASS" + }, + { + "rung": "outcome", + "name": "MockServices-CTL-data-target end state", + "verdict": "PASS" + }, + { + "rung": "control", + "name": "MockServices · data-target: association must point at a value the journey never picked", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: widget btnReset", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: widget lvSystems", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: widget lvPackages", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: page says \"Mock console\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: page says \"DO_SIGN_CONTRACTS\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: confirmation says \"Reset the mock hub?\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: page says \"DO_SIGN_CONTRACTS\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: page says \"7710001\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Reset the hub to its baseline: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset the hub to its baseline: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset the hub to its baseline: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: Reset the hub to its baseline: REST_ did not run", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: the call log is empty", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: no scenario is armed", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: all six systems are Online", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: one default package", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: two applications, none KO", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: radio rbKind checked \"Custom payload\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: widget btnDlgCancel", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: page says \"Not valid JSON\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] A custom payload that is not JSON is refused: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] A custom payload that is not JSON is refused: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] A custom payload that is not JSON is refused: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: A custom payload that is not JSON is refused: ^MockServices\\.SUB_MockScenario_Arm$ did not run", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "A custom payload that is not JSON is refused: nothing was armed", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: radio rbKind checked \"Fault\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: page says \"MWF-503\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: page says \"· once\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: page does NOT say \"Not valid JSON\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Arm a one-shot fault on DO_SIGN_CONTRACTS: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Arm a one-shot fault on DO_SIGN_CONTRACTS: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Arm a one-shot fault on DO_SIGN_CONTRACTS: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: Arm a one-shot fault on DO_SIGN_CONTRACTS: VAL_MockScenario_Payload did not run", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: DO_SIGN_CONTRACTS is armed with fault MWF-503 / 503", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: only that one is armed", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: DO_SIGN_CONTRACTS (armed) answers 503", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: DO_SIGN_CONTRACTS (armed) response has MWF-503", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: DO_SIGN_CONTRACTS (after the one shot) answers 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: MockServices.MockCallLog +2", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: the call log holds the 503 then the 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: the one-shot disarmed itself; both calls counted", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: GET_SYSTEM_STATUS answers 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: GET_SYSTEM_STATUS response has \"StatusCode\":\"0\",\"StatusDescription\":\"Offline\",\"System\":\"GM4\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Take GM4 offline: reached", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Take GM4 offline: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Take GM4 offline: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Take GM4 offline: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: GM4 answers StatusCode 0", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: the other five stay Online", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Mark application 7710001 KO: DO_START_APPROVAL answers 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Mark application 7710001 KO: DO_START_APPROVAL response has \"APPROVED_FLAG\":false", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Mark application 7710001 KO: reached", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Mark application 7710001 KO: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Mark application 7710001 KO: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Mark application 7710001 KO: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Mark application 7710001 KO: 7710001 is KO, 7710002 is not", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Cancelling the reset changes nothing: confirmation says \"Reset the mock hub?\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Cancelling the reset changes nothing: reached", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Cancelling the reset changes nothing: GM4 is still offline", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Cancelling the reset changes nothing: 7710001 is still KO", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Cancelling the reset changes nothing: the call log is kept", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset restores the baseline: confirmation says \"Reset the mock hub?\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset restores the baseline: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset restores the baseline: page does NOT say \"MWF-503\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Reset restores the baseline: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset restores the baseline: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset restores the baseline: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: Reset restores the baseline: REST_ did not run", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: MockServices.MockApplication_MockPackage set on every row", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: MockServices.MockApplication_MockPackage points at the seeded ProdPackageKey", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: the call log is empty", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: no scenario is armed and no hit count survives", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: no application is KO", + "verdict": "PASS" + }, + { + "rung": "outcome", + "name": "MockServices-CTL-outcome end state", + "verdict": "FAIL" + }, + { + "rung": "control", + "name": "MockServices · outcome: outcome query with an unreachable floor", + "verdict": "PASS" + } + ] +} diff --git a/tests/wave2/fixtures/journey-control/walk-run.json b/tests/wave2/fixtures/journey-control/walk-run.json new file mode 100644 index 0000000..6c0742b --- /dev/null +++ b/tests/wave2/fixtures/journey-control/walk-run.json @@ -0,0 +1,408 @@ +{ + "capturedAt": "2026-09-26T08:15:38.349Z", + "positiveControl": false, + "mutants": { + "expectedPerJourney": 7, + "expected": 0, + "supported": 0, + "proven": 0, + "unsupported": [] + }, + "results": [ + { + "rung": "ui", + "name": "login", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: widget btnReset", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: widget lvSystems", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: widget lvPackages", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: page says \"Mock console\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Open Configure › Mock console: page says \"DO_SIGN_CONTRACTS\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: confirmation says \"Reset the mock hub?\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: page says \"DO_SIGN_CONTRACTS\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset the hub to its baseline: page says \"7710001\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Reset the hub to its baseline: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset the hub to its baseline: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset the hub to its baseline: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: Reset the hub to its baseline: REST_ did not run", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: the call log is empty", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: no scenario is armed", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: all six systems are Online", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: one default package", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset the hub to its baseline: two applications, none KO", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: radio rbKind checked \"Custom payload\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: widget btnDlgCancel", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "A custom payload that is not JSON is refused: page says \"Not valid JSON\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] A custom payload that is not JSON is refused: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] A custom payload that is not JSON is refused: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] A custom payload that is not JSON is refused: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: A custom payload that is not JSON is refused: ^MockServices\\.SUB_MockScenario_Arm$ did not run", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "A custom payload that is not JSON is refused: nothing was armed", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: radio rbKind checked \"Fault\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: page says \"MWF-503\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: page says \"· once\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: page does NOT say \"Not valid JSON\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Arm a one-shot fault on DO_SIGN_CONTRACTS: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Arm a one-shot fault on DO_SIGN_CONTRACTS: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Arm a one-shot fault on DO_SIGN_CONTRACTS: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: Arm a one-shot fault on DO_SIGN_CONTRACTS: VAL_MockScenario_Payload did not run", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: DO_SIGN_CONTRACTS is armed with fault MWF-503 / 503", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Arm a one-shot fault on DO_SIGN_CONTRACTS: only that one is armed", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: DO_SIGN_CONTRACTS (armed) answers 503", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: DO_SIGN_CONTRACTS (armed) response has MWF-503", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: DO_SIGN_CONTRACTS (after the one shot) answers 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: MockServices.MockCallLog +2", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: the call log holds the 503 then the 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "The hub answers the fault once, then succeeds: the one-shot disarmed itself; both calls counted", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: GET_SYSTEM_STATUS answers 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: GET_SYSTEM_STATUS response has \"StatusCode\":\"0\",\"StatusDescription\":\"Offline\",\"System\":\"GM4\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Take GM4 offline: reached", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Take GM4 offline: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Take GM4 offline: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Take GM4 offline: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: GM4 answers StatusCode 0", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Take GM4 offline: the other five stay Online", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Mark application 7710001 KO: DO_START_APPROVAL answers 200", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Mark application 7710001 KO: DO_START_APPROVAL response has \"APPROVED_FLAG\":false", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Mark application 7710001 KO: reached", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Mark application 7710001 KO: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Mark application 7710001 KO: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Mark application 7710001 KO: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Mark application 7710001 KO: 7710001 is KO, 7710002 is not", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Cancelling the reset changes nothing: confirmation says \"Reset the mock hub?\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Cancelling the reset changes nothing: reached", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Cancelling the reset changes nothing: GM4 is still offline", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Cancelling the reset changes nothing: 7710001 is still KO", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Cancelling the reset changes nothing: the call log is kept", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset restores the baseline: confirmation says \"Reset the mock hub?\"", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset restores the baseline: reached", + "verdict": "PASS" + }, + { + "rung": "ui", + "name": "Reset restores the baseline: page does NOT say \"MWF-503\"", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[GUARD] Reset restores the baseline: capture non-empty", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset restores the baseline: ordered", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] Reset restores the baseline: no ERROR at any level", + "verdict": "PASS" + }, + { + "rung": "trace", + "name": "[MF] NEGATIVE: Reset restores the baseline: REST_ did not run", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: MockServices.MockApplication_MockPackage set on every row", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: MockServices.MockApplication_MockPackage points at the seeded ProdPackageKey", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: the call log is empty", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: no scenario is armed and no hit count survives", + "verdict": "PASS" + }, + { + "rung": "data", + "name": "Reset restores the baseline: no application is KO", + "verdict": "PASS" + }, + { + "rung": "outcome", + "name": "MockServices end state", + "verdict": "PASS" + } + ] +} diff --git a/tests/wave2/test-conformance-baseline-scope.sh b/tests/wave2/test-conformance-baseline-scope.sh new file mode 100755 index 0000000..a10fa5c --- /dev/null +++ b/tests/wave2/test-conformance-baseline-scope.sh @@ -0,0 +1,109 @@ +#!/usr/bin/env bash +# Usage: bash test-conformance-baseline-scope.sh [path-to-conformance-check.sh] +# +# Fixture for project-bin/conformance-check.sh's baseline — which rows it guards, and which rows +# a `--module` rewrite may touch. +# +# The defects (card-disbursement requirements-driven build, 2026-09-26): +# (a) The baseline is written once, by the first run. A module built afterwards has no baseline +# rows, its rows were compared with "" and could never regress — silently, forever. That +# project's committed baseline held 1 of its 4 modules (127 of 246 rows). +# (b) `--update-baseline --module X` wrote X's rows over the WHOLE file, dropping every other +# module's rows — a scoped measurement doing an unscoped write. +# Cases 2-3 are (a), cases 4-5 are (b), case 6 guards the unscoped rewrite. +# +# The ledgers and the mxcli stub are SYNTHETIC. That is deliberate and allowed: the field-proof +# rule's "golden input is captured" governs parsers of tool output, and nothing below exercises +# the DESCRIBE-output parser — the stub answers the two shapes it already classifies (text + rc 0 +# = PRESENT, "Error: … not found" + rc 1 = ABSENT) and the assertions are about the baseline file. +# The field run that motivated it is cited in CHANGELOG.md. + +set -uo pipefail + +SUT="${1:-}" +[ -z "$SUT" ] && SUT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)/project-bin/conformance-check.sh" +if [ ! -f "$SUT" ]; then + echo "SKIP: subject not found at $SUT" + echo "SCORE: 0/0 — nothing to test" + exit 0 +fi +SUT="$(cd "$(dirname "$SUT")" && pwd)/$(basename "$SUT")" +[ -f "$(dirname "$SUT")/_common.sh" ] || { echo "FAIL — _common.sh not beside $SUT"; exit 1; } + +PASS=0; FAIL=0 +ok() { PASS=$((PASS+1)); echo " ok — $1"; } +bad() { FAIL=$((FAIL+1)); echo " FAIL — $1"; [ -n "${2:-}" ] && echo " got: $2"; } + +WORK="$(mktemp -d "${TMPDIR:-/tmp}/conf-baseline.XXXXXX")" +trap 'rm -rf "$WORK"' EXIT +P="$WORK/proj" +mkdir -p "$P/architecture/modules/Orders" "$P/architecture/modules/Billing" +: > "$P/Shop.mpr" +# PRESENT lists the qualified names the stub model holds; edit it to break or restore one. +printf 'Orders.Order\nOrders.ACT_Order_Save\nBilling.Invoice\nBilling.ACT_Invoice_Send\n' > "$P/PRESENT" +cat > "$P/mxcli" <<'EOF' +#!/usr/bin/env bash +cmd=""; while [ $# -gt 0 ]; do [ "$1" = -c ] && cmd="$2"; shift; done +name="${cmd##* }" +if grep -qx "$name" "$(dirname "$0")/PRESENT"; then echo "-- $name"; echo "create $name;"; exit 0; fi +echo "Error: $name not found"; exit 1 +EOF +chmod +x "$P/mxcli" +ledger() { # ledger + cat > "$P/architecture/modules/$1/coverage-ledger.md" < "$WORK/out" 2>&1; echo $?; } +rows() { awk -F'\t' -v m="$1" '$1==m' "$P/docs/conformance/baseline.tsv" 2>/dev/null | grep -c . ; } +out() { tr '\n' ' ' < "$WORK/out" | cut -c1-300; } + +# --- 1. first run writes the baseline (unchanged behaviour) --------------------------------- +ledger Orders Order ACT_Order_Save +rc="$(run)" +[ "$rc" = 0 ] && [ "$(rows Orders)" = 2 ] && ok "first run baselines Orders (2 rows)" \ + || bad "first run did not write a 2-row Orders baseline (rc=$rc)" "$(out)" + +# --- 2. a module built after the baseline is baselined on first sight, and said so ----------- +ledger Billing Invoice ACT_Invoice_Send +rc="$(run)" +[ "$rc" = 0 ] && [ "$(rows Billing)" = 2 ] && ok "Billing's rows are baselined on first sight (2 rows added)" \ + || bad "Billing's rows never reached the baseline (the 2026-09-26 unguarded module)" "rc=$rc rows=$(rows Billing)" +grep -q "newly baselined *2" "$WORK/out" && ok "the run says how many rows it newly baselined" \ + || bad "no 'newly baselined 2' line" "$(out)" + +# --- 3. ...so a later break in Billing IS a regression ---------------------------------------- +sed -i.bak '/^Billing.Invoice$/d' "$P/PRESENT" +rc="$(run)" +[ "$rc" = 1 ] && grep -q "Billing.*was OK, now STALE" "$WORK/out" \ + && ok "Billing.Invoice going absent fails the run as a regression" \ + || bad "a module built after the baseline cannot regress" "rc=$rc $(out)" +mv "$P/PRESENT.bak" "$P/PRESENT" + +# --- 4. --update-baseline --module rewrites only that module's rows --------------------------- +rc="$(run --module Billing --update-baseline)" +[ "$rc" = 0 ] && [ "$(rows Orders)" = 2 ] && [ "$(rows Billing)" = 2 ] \ + && ok "--module Billing --update-baseline keeps Orders' 2 rows" \ + || bad "a --module rewrite dropped other modules' rows (the 2026-09-26 clobber)" "rc=$rc Orders=$(rows Orders) Billing=$(rows Billing)" + +# --- 5. ...so Orders is still guarded after it ------------------------------------------------ +sed -i.bak '/^Orders.ACT_Order_Save$/d' "$P/PRESENT" +rc="$(run)" +[ "$rc" = 1 ] && grep -q "Orders.*was OK, now STALE" "$WORK/out" \ + && ok "Orders.ACT_Order_Save going absent still fails the run after Billing's rewrite" \ + || bad "Orders lost its regression guard" "rc=$rc $(out)" +mv "$P/PRESENT.bak" "$P/PRESENT" + +# --- 6. an unscoped --update-baseline still rewrites everything -------------------------------- +printf 'Orders\t`/stale`\tOK\n' >> "$P/docs/conformance/baseline.tsv" +rc="$(run --update-baseline)" +[ "$rc" = 0 ] && [ "$(wc -l < "$P/docs/conformance/baseline.tsv" | tr -d ' ')" = 4 ] \ + && ok "unscoped --update-baseline rewrites the whole file (a retired row is dropped)" \ + || bad "unscoped rewrite changed behaviour" "rc=$rc lines=$(wc -l < "$P/docs/conformance/baseline.tsv")" + +echo "" +echo "PASS=$PASS FAIL=$FAIL" +[ "$FAIL" -eq 0 ] diff --git a/tests/wave2/test-journey-control-exit.sh b/tests/wave2/test-journey-control-exit.sh new file mode 100755 index 0000000..76f43c7 --- /dev/null +++ b/tests/wave2/test-journey-control-exit.sh @@ -0,0 +1,98 @@ +#!/usr/bin/env bash +# Usage: bash test-journey-control-exit.sh [path-to-journey-runner.js] +# +# Fixture for project-tests/e2e/journey-runner.js's exit code — runExitCode(), which decides +# what verify-module's two journey rungs report. +# +# The defect (card-disbursement requirements-driven build, 2026-09-26): the walk's rule — +# exit 1 on any FAIL or INVALID — was applied to the --positive-control run too. A control +# run's FAIL rows are its mutants being caught, so a control that proved 7 of 7 rungs exited 1, +# verify-module graded the rung FINDING, and the module read INCOMPLETE on every run: a rung +# that could not go green, and so said nothing when it went red. +# +# Golden input is CAPTURED: fixtures/journey-control/{control,walk}-run.json are that build's +# journey-findings-control.json (476 rows: PASS 468, FAIL 7, INVALID 1; 7 of 7 rungs proven) and +# journey-findings.json (79 rows, all PASS), trimmed to rung/name/verdict — `detail` (screenshot +# names, trace sequences) and `walks` are dropped because runExitCode never reads them. Cases 3-6 +# derive a broken run from the captured one by editing single rows, which is the shape each +# failure takes in the field. + +set -uo pipefail + +SUT="${1:-}" +[ -z "$SUT" ] && SUT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)/project-tests/e2e/journey-runner.js" +if [ ! -f "$SUT" ]; then + echo "SKIP: subject not found at $SUT" + echo "SCORE: 0/0 — nothing to test" + exit 0 +fi +command -v node >/dev/null 2>&1 || { echo "SKIP: node not installed"; exit 0; } +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +FIX="$HERE/fixtures/journey-control" +[ -f "$FIX/control-run.json" ] && [ -f "$FIX/walk-run.json" ] || { echo "FAIL — golden capture missing under $FIX"; exit 1; } +E2E="$(cd "$(dirname "$SUT")" && pwd)" +for f in helpers.js otel.js config.js; do + [ -f "$E2E/$f" ] || { echo "FAIL — $f not beside $SUT"; exit 1; } +done +CFG="$E2E/project.config.template.js" +[ -f "$CFG" ] || CFG="$HERE/../../project-tests/e2e/project.config.template.js" + +PASS=0; FAIL=0 +ok() { PASS=$((PASS+1)); echo " ok — $1"; } +bad() { FAIL=$((FAIL+1)); echo " FAIL — $1"; [ -n "${2:-}" ] && echo " got: $2"; } + +# The runner resolves a project at require time: an .mpr two levels up and an asserted port. +WORK="$(mktemp -d "${TMPDIR:-/tmp}/journey-exit.XXXXXX")" +trap 'rm -rf "$WORK"' EXIT +mkdir -p "$WORK/tests/e2e"; : > "$WORK/Fixture.mpr" +cp "$SUT" "$E2E/helpers.js" "$E2E/otel.js" "$E2E/config.js" "$WORK/tests/e2e/" +cp "$CFG" "$WORK/tests/e2e/project.config.js" + +OUT="$(cd "$WORK/tests/e2e" && APP_PORT=1 node - "$FIX" <<'EOF' 2>&1 +const fix = process.argv[2]; +const R = require('./journey-runner.js'); +if (typeof R.runExitCode !== 'function') { console.log('NOFN'); process.exit(0); } +const ctl = require(fix + '/control-run.json'), walk = require(fix + '/walk-run.json'); +const clone = o => JSON.parse(JSON.stringify(o)); +const say = (k, pc, run) => console.log(`${k} ${R.runExitCode(pc, run.results, run.mutants)}`); +say('CTL', true, ctl); +say('WALK', false, walk); +// 3. one mutant NOT caught: its [control] row goes FAIL and the ledger loses a proof. +let c = clone(ctl); c.results.find(r => r.rung === 'control').verdict = 'FAIL'; c.mutants.proven--; +say('MISS', true, c); +// 4. a rung the journey cannot express: unsupported, INVALID control row, proven < expected. +c = clone(ctl); c.results.find(r => r.rung === 'control').verdict = 'INVALID'; +c.mutants.proven--; c.mutants.supported--; say('UNSUP', true, c); +// 5. login refused: no mutant ran at all — nothing proven is not "all proven". +say('NONE', true, { results: [{ rung: 'ui', name: 'login', verdict: 'INVALID' }], + mutants: { expected: 0, supported: 0, proven: 0, unsupported: [] } }); +// 6. the walk keeps its rule: one FAIL, then one INVALID, each exits 1. +let w = clone(walk); w.results[0].verdict = 'FAIL'; say('WFAIL', false, w); +w = clone(walk); w.results[0].verdict = 'INVALID'; say('WINV', false, w); +EOF +)" +line() { printf '%s\n' "$OUT" | sed -n "s/^$1 //p"; } +if printf '%s\n' "$OUT" | grep -qx NOFN; then + bad "journey-runner.js exports no runExitCode — the exit rule is the walk's, applied to both runs" + echo ""; echo "PASS=$PASS FAIL=$FAIL"; exit 1 +fi +[ -n "$(line CTL)" ] || { echo "FAIL — harness could not require $SUT"; echo "$OUT" | tail -5; exit 1; } + +[ "$(line CTL)" = 0 ] && ok "captured control run (7 of 7 proven, 7 mutant FAILs, 1 INVALID) exits 0" \ + || bad "a control that proved every rung exits non-zero (the 2026-09-26 FINDING)" "$(line CTL)" +[ "$(line WALK)" = 0 ] && ok "captured walk (79 PASS) exits 0" || bad "clean walk exits non-zero" "$(line WALK)" +[ "$(line MISS)" = 1 ] && ok "a mutant that was not caught fails the control" || bad "uncaught mutant exits 0" "$(line MISS)" +[ "$(line UNSUP)" = 1 ] && ok "an unproven (unsupported) rung fails the control" || bad "unproven rung exits 0" "$(line UNSUP)" +[ "$(line NONE)" = 1 ] && ok "a control where no mutant ran (login refused) fails" || bad "zero-mutant control exits 0" "$(line NONE)" +[ "$(line WFAIL)" = 1 ] && [ "$(line WINV)" = 1 ] && ok "the walk still exits 1 on a FAIL and on an INVALID" \ + || bad "the walk's rule changed" "FAIL→$(line WFAIL) INVALID→$(line WINV)" + +if awk '/process\.exit\(/ && /runExitCode\(POSITIVE_CONTROL/ {f=1} END{exit !f}' "$SUT"; then + ok "main exits through runExitCode(POSITIVE_CONTROL, …)" +else + bad "main does not exit through runExitCode — the tested rule is not the one that runs" +fi + +echo "" +echo "PASS=$PASS FAIL=$FAIL" +[ "$FAIL" -eq 0 ] diff --git a/tests/wave2/test-monkey-left-app.sh b/tests/wave2/test-monkey-left-app.sh new file mode 100755 index 0000000..ea675bf --- /dev/null +++ b/tests/wave2/test-monkey-left-app.sh @@ -0,0 +1,104 @@ +#!/usr/bin/env bash +# Usage: bash test-monkey-left-app.sh [path-to-monkey.js] +# +# Fixture for project-tests/e2e/monkey.js's leftApp() and SIGN_OUT — the two guards that keep +# the harness's own moves from being scored as app crashes. +# +# The defect (card-disbursement requirements-driven build, 2026-09-26, seed 410855898): a +# backAfterSubmit round clicked "Switch theme" (a control that pushes no history entry), so +# Back left the app for login.html, then about:blank. The oracle scored each non-app page +# "blank page", every later round on the target scored it again, and the second target was +# never reached: 13 observations, 10 crash-class, one target unfuzzed, no app defect. With the +# Back fix in place, a reloadMidFlow round clicked the shell's "Sign out" and lost the target +# the same way. After both fixes the same seed read 26 observations, 0 crash-class. +# +# Golden input is CAPTURED: the URLs below are the ones that run's probe log recorded +# (url=… on each CRASH row, and the app routes it navigated), and the labels are the two +# controls clicked in it. The origin is the run's own baseUrl. + +set -uo pipefail + +SUT="${1:-}" +[ -z "$SUT" ] && SUT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)/project-tests/e2e/monkey.js" +if [ ! -f "$SUT" ]; then + echo "SKIP: subject not found at $SUT" + echo "SCORE: 0/0 — nothing to test" + exit 0 +fi +command -v node >/dev/null 2>&1 || { echo "SKIP: node not installed"; exit 0; } +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +E2E="$(cd "$(dirname "$SUT")" && pwd)" +for f in helpers.js otel.js config.js; do + [ -f "$E2E/$f" ] || { echo "FAIL — $f not beside $SUT"; exit 1; } +done +CFG="$E2E/project.config.template.js" +[ -f "$CFG" ] || CFG="$HERE/../../project-tests/e2e/project.config.template.js" + +PASS=0; FAIL=0 +ok() { PASS=$((PASS+1)); echo " ok — $1"; } +bad() { FAIL=$((FAIL+1)); echo " FAIL — $1"; [ -n "${2:-}" ] && echo " got: $2"; } + +# monkey.js resolves a project at require time: an .mpr two levels up and an asserted port. +WORK="$(mktemp -d "${TMPDIR:-/tmp}/monkey-left.XXXXXX")" +trap 'rm -rf "$WORK"' EXIT +mkdir -p "$WORK/tests/e2e"; : > "$WORK/Fixture.mpr" +cp "$SUT" "$E2E/helpers.js" "$E2E/otel.js" "$E2E/config.js" "$WORK/tests/e2e/" +cp "$CFG" "$WORK/tests/e2e/project.config.js" + +# An old monkey runs main on require and would launch a browser: bound it. +OUT="$(cd "$WORK/tests/e2e" && APP_PORT=1 timeout 30 node - <<'EOF' 2>&1 +const M = require('./monkey.js'); +if (typeof M.leftApp !== 'function' || !(M.SIGN_OUT instanceof RegExp)) { console.log('NOFN'); process.exit(0); } +const base = 'http://localhost:8080'; +const L = (k, url, b) => console.log(`${k} ${M.leftApp(url, b === undefined ? base : b)}`); +L('LOGIN', 'http://localhost:8080/login.html'); +L('LOGINQ', 'http://localhost:8080/login.html?profile=Responsive'); +L('BLANK', 'about:blank'); +L('EMPTY', ''); +L('ROOT', 'http://localhost:8080/'); +L('BARE', 'http://localhost:8080'); +L('INDEX', 'http://localhost:8080/index.html'); +L('CASES', 'http://localhost:8080/p/cases'); +L('MOCK', 'http://localhost:8080/p/mock-console'); +L('OTHER', 'http://localhost:80801/p/cases'); +L('SLASHBASE', 'http://localhost:8080/p/cases', base + '/'); +const S = t => console.log(`S:${t} ${M.SIGN_OUT.test(t)}`); +['Sign out', 'Log off', 'Logout', 'Switch theme', 'Signature', 'Log in'].forEach(S); +process.exit(0); +EOF +)" +line() { printf '%s\n' "$OUT" | sed -n "s/^$1 //p"; } +if printf '%s\n' "$OUT" | grep -qx NOFN || [ -z "$(line LOGIN)" ]; then + bad "monkey.js exports no leftApp/SIGN_OUT (or ran main on require) — a round that leaves the app is scored as an app crash" + printf '%s\n' "$OUT" | tail -3 | sed 's/^/ /' + echo ""; echo "PASS=$PASS FAIL=$FAIL"; exit 1 +fi + +[ "$(line LOGIN)" = true ] && [ "$(line LOGINQ)" = true ] && ok "login.html (captured, with and without a query) is outside the app" \ + || bad "login.html counted as in-app — the 2026-09-26 cascade" "$(line LOGIN)/$(line LOGINQ)" +[ "$(line BLANK)" = true ] && [ "$(line EMPTY)" = true ] && ok "about:blank (captured) and an empty url are outside the app" \ + || bad "about:blank counted as in-app" "$(line BLANK)/$(line EMPTY)" +[ "$(line ROOT)" = false ] && [ "$(line BARE)" = false ] && [ "$(line INDEX)" = false ] \ + && ok "the app root, bare origin and index.html are in the app" || bad "app root counted as outside" "$(line ROOT)/$(line BARE)/$(line INDEX)" +[ "$(line CASES)" = false ] && [ "$(line MOCK)" = false ] && ok "captured app routes (/p/cases, /p/mock-console) are in the app" \ + || bad "an app route counted as outside — every round would re-land" "$(line CASES)/$(line MOCK)" +[ "$(line OTHER)" = true ] && ok "a lookalike origin (port prefix match) is outside the app" || bad "prefix match leaks another origin in" "$(line OTHER)" +[ "$(line SLASHBASE)" = false ] && ok "a baseUrl with a trailing slash still matches its routes" || bad "trailing-slash baseUrl breaks the match" "$(line SLASHBASE)" + +s() { printf '%s\n' "$OUT" | sed -n "s/^S:$1 //p"; } +[ "$(s 'Sign out')" = true ] && [ "$(s 'Log off')" = true ] && [ "$(s 'Logout')" = true ] \ + && ok "SIGN_OUT matches Sign out (captured), Log off, Logout" || bad "a sign-out label is not excluded" "$(s 'Sign out')/$(s 'Log off')/$(s 'Logout')" +[ "$(s 'Switch theme')" = false ] && [ "$(s 'Signature')" = false ] && [ "$(s 'Log in')" = false ] \ + && ok "SIGN_OUT leaves Switch theme (captured), Signature, Log in fuzzable" || bad "SIGN_OUT over-matches" "$(s 'Switch theme')/$(s 'Signature')/$(s 'Log in')" + +# The tested functions must be the ones the rounds use. +awk '/hasNotText: *SIGN_OUT/ {f=1} END{exit !f}' "$SUT" && ok "round()'s button locator filters SIGN_OUT" \ + || bad "the button locator does not filter SIGN_OUT — the tested regex is not the one that runs" +awk '/goBack\(/ {b=1} b && /leftApp\(page\.url\(\)/ {l=1} l && /goForward\(/ {f=1} END{exit !f}' "$SUT" \ + && ok "backAfterSubmit returns with Forward when Back left the app" || bad "backAfterSubmit does not check leftApp after Back" +grep -q "'lost'" "$SUT" && ok "a target lost mid-run is reported with its unfuzzed round count, not silently re-scored" \ + || bad "no 'lost' finding — an abandoned target reads as fuzzed" + +echo "" +echo "PASS=$PASS FAIL=$FAIL" +[ "$FAIL" -eq 0 ] From c85cfb4db5f2ed855e74f7e93f982ff6c23fd5ca Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 08:47:52 +0000 Subject: [PATCH 08/45] Inbox: three field notes from the card-disbursement build's MockServices verify - design-audit raw-class rule contradicts design-spacing.md's page-header class - report-normalize --selftest broken independent of the two-tree fix - span claims cannot be negative-only (mustNotFire needs an ordered list) Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Pm41sykqDGcvioGzsY5H2N --- ...esign-audit-raw-class-vs-design-spacing.md | 32 +++++++++++++++++++ ...-09-26-report-normalize-selftest-broken.md | 32 +++++++++++++++++++ ...-26-span-claims-cannot-be-negative-only.md | 25 +++++++++++++++ 3 files changed, 89 insertions(+) create mode 100644 contrib/inbox/2026-09-26-design-audit-raw-class-vs-design-spacing.md create mode 100644 contrib/inbox/2026-09-26-report-normalize-selftest-broken.md create mode 100644 contrib/inbox/2026-09-26-span-claims-cannot-be-negative-only.md diff --git a/contrib/inbox/2026-09-26-design-audit-raw-class-vs-design-spacing.md b/contrib/inbox/2026-09-26-design-audit-raw-class-vs-design-spacing.md new file mode 100644 index 0000000..041133f --- /dev/null +++ b/contrib/inbox/2026-09-26-design-audit-raw-class-vs-design-spacing.md @@ -0,0 +1,32 @@ +# design-audit rung 7 `raw-class-vs-designproperty` fails the page header `design-spacing.md` prescribes + +**From:** card-disbursement requirements-driven build (build-plan row 3.9) +**Date:** 2026-09-26 +**Kind:** bug +**Field evidence:** design-audit.js (toolkit 9fe925e) on a Mendix 11.13.0 app, 7 pages: 4 of 4 `raw-class-vs-designproperty` fails are the same finding, each `ctnHead.spacing-outer-bottom-large → "Spacing"`. +**Proposed target:** `project-tests/e2e/design-audit.js` (rung 7, ~l.547/666) or `skills/design-spacing.md` (l.60, l.66, l.94) — one of them has to give + +--- + +Two toolkit instruments disagree about one class. + +- `skills/design-spacing.md` l.60: "`spacing-outer-bottom-large` on every section container", and + its page-header scaffold (l.94) is `container ctnPageHead (Class: 'page-head spacing-outer-bottom-large')`. +- design-audit rung 7 `raw-class-vs-designproperty` fails any hand-written class that duplicates + a design property, and Atlas's "Spacing" design property emits that class — so every page + built by the skill fails the rung. + +Verbatim audit lines (page names genericised to their shape): + +``` +fail | ._Overview | rung7/raw-class-vs-designproperty | 1 hand-written class(es) duplicating a design property: ctnHead.spacing-outer-bottom-large → "Spacing" +fail | ._NewEdit | (same) +fail | StyleGallery.Gallery_Overview | (same) +``` + +The project ruled that the skill wins (rendered result identical, the skill's scaffold is what +every wireframe follows) and carries the 4 fails as known. Options for triage: +(a) the skill prescribes the design property instead (`DesignProperties: ['Spacing bottom': 'Large']` +or whatever mxcli v0.24 accepts — not probed here); (b) the rung exempts classes a toolkit skill +prescribes. Hypothesis, not tested: (a) is the better fix, since the property survives an Atlas +class rename and the raw class does not. diff --git a/contrib/inbox/2026-09-26-report-normalize-selftest-broken.md b/contrib/inbox/2026-09-26-report-normalize-selftest-broken.md new file mode 100644 index 0000000..a54eba5 --- /dev/null +++ b/contrib/inbox/2026-09-26-report-normalize-selftest-broken.md @@ -0,0 +1,32 @@ +# report-normalize.js --selftest fails on a stock checkout and on a real project + +**From:** card-disbursement requirements-driven build (build-plan row 3.9) +**Date:** 2026-09-26 +**Kind:** bug +**Field evidence:** `node report-normalize.js --selftest` at toolkit f38f808 and at its parent, same result both times: 41 ok, 4 FAIL, then a TypeError. +**Proposed target:** `project-tests/e2e/report-normalize.js` `selftest()` (~l.2631) + +--- + +Pre-existing — the f38f808 two-tree fix did not change it (same output before and after). + +With a project's own `project.config.js`: + +``` +FAIL missing full-app-walkthrough → fault instrument got=undefined want="fault" +FAIL missing full-app-walkthrough → carries a reason got=false want=true +FAIL missing mobile-fieldscan → fault instrument got=undefined want="fault" +FAIL missing mobile-fieldscan → carries a reason got=false want=true +.../tests/e2e/report-normalize.js:2631 + [inst(empty, 'full-app-walkthrough').canExpressFault, inst(empty, 'full-app-walkthrough').evidenceStrength], +TypeError: Cannot read properties of undefined (reading 'canExpressFault') +``` + +With the template config in a directory that has no `.mpr` above it, it never reaches the +selftest: `project.config.js` throws "no .mpr found" at require time. + +Reading of the cause (from the code, not debugged further): the selftest asserts on instruments +named `full-app-walkthrough` and `mobile-fieldscan` that only exist when the project config +declares those walkthroughs; the real project declares neither. The selftest should build its +own config, or skip the walkthrough cases when none is declared — and it should not need a +`.mpr` at all. diff --git a/contrib/inbox/2026-09-26-span-claims-cannot-be-negative-only.md b/contrib/inbox/2026-09-26-span-claims-cannot-be-negative-only.md new file mode 100644 index 0000000..32a40b7 --- /dev/null +++ b/contrib/inbox/2026-09-26-span-claims-cannot-be-negative-only.md @@ -0,0 +1,25 @@ +# journey-runner rung 2: a span claim cannot be purely negative + +**From:** card-disbursement requirements-driven build (build-plan row 3.9) +**Date:** 2026-09-26 +**Kind:** learning +**Field evidence:** writing span claims for a 9-step admin-console journey; rung 2's guard at `project-tests/e2e/journey-runner.js` ~l.697 is `if (step.spans && step.spans.ordered && step.spans.ordered.length)`. +**Proposed target:** `project-tests/e2e/journey-runner.js` rung 2; `skills/testing-shape.md` (claim shapes) + +--- + +`mustNotFire` is honoured only next to a non-empty `ordered` list. A step whose whole proof is +"this did NOT run" cannot say so: + +- the step: Cancel on a "Reset all mock state?" confirmation. The correct outcome is that the + reset action microflow never fires — no microflow fires at all. +- `spans: { ordered: [], mustNotFire: ["ACT_Mock_Reset"] }` is silently skipped (rung 2 does + not run), so the claim reads as declared and proves nothing. + +The project left that step without a span claim; its data rung (rows unchanged) carries the proof. + +Suggested shape: run rung 2 when `ordered` OR `mustNotFire` is non-empty; for a negative-only +claim, wait the normal capture window (there is nothing to `until` on), then assert no match. +A positive-control mutant for it would inject the forbidden name into the captured list. Also +worth a loud INVALID when a step declares `spans` that rung 2 will not evaluate, instead of the +silent skip. From fe1651c6deee5683c7d5d391d38989d7d4911e2f Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 12:02:02 +0000 Subject: [PATCH 09/45] journey-runner control: exact-expect outcome and numeric mustPointAt mutants; five inbox notes - fix(journey-runner): the outcome mutant breaks an exact `expect` as well as an `atLeast` floor, and the data-target mutant uses a numeric sentinel on a numeric key, so neither reports a working assertion as unproven. Field runs cited in CHANGELOG; journey-proof.md's mutant table updated. - Inbox: mxcli v0.24 REST/JSON/mapping/page-hook/OQL/test-endpoint notes; journey-runner persona and local acts; verify-module graph FAULT after test --attach; LOOK obligation reads the first VALID AT; wiring-sweep disabled-by-design and conditionally visible controls. From a card-disbursement requirements-driven build (rows 3.8-4.8). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Pm41sykqDGcvioGzsY5H2N --- CHANGELOG.md | 1 + ...6-journey-runner-persona-and-local-acts.md | 31 ++++++++++ ...26-look-obligation-reads-first-valid-at.md | 24 ++++++++ ...6-mxcli-v024-rest-json-page-field-notes.md | 57 +++++++++++++++++++ ...fy-module-graph-fault-after-test-attach.md | 20 +++++++ ...bled-by-design-and-conditional-controls.md | 20 +++++++ project-tests/e2e/journey-runner.js | 18 ++++-- skills/journey-proof.md | 4 +- 8 files changed, 169 insertions(+), 6 deletions(-) create mode 100644 contrib/inbox/2026-09-26-journey-runner-persona-and-local-acts.md create mode 100644 contrib/inbox/2026-09-26-look-obligation-reads-first-valid-at.md create mode 100644 contrib/inbox/2026-09-26-mxcli-v024-rest-json-page-field-notes.md create mode 100644 contrib/inbox/2026-09-26-verify-module-graph-fault-after-test-attach.md create mode 100644 contrib/inbox/2026-09-26-wiring-sweep-disabled-by-design-and-conditional-controls.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 7d28390..980adb7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,7 @@ three commits past it), and a bug report can name a release instead of a sha nob Sections dated before 2026-09-19 predate the cycle and stay as they are. ## Unreleased +- fix(project-tests/e2e/journey-runner.js): **two positive-control mutants no longer report a working assertion as unproven.** The `outcome` mutant only broke an `atLeast` floor, so a journey whose outcome declares an exact `expect` (the stronger claim) had the rung reported UNPROVEN ("declares nothing this mutant can break"). An exact `expect` is now mutated to 999999 too. The `data-target` mutant wrote a text sentinel into `mustPointAt`; on a numeric key (Long/Integer) that is an OQL error, so the targeted check read INVALID instead of FAIL and the rung could never be proven. A numeric value now gets the numeric sentinel `-424242`. `skills/journey-proof.md`'s mutant table says both. Field runs (mxcli v0.24.0, Mendix 11.13.0): Integration `--positive-control` 5 → 6 of 7 rungs, `outcome` caught by the journey's `expect: 2` end-state check (the seventh, data-target, is unproven because that journey declares no `mustPointAt`); MockServices data-target INVALID → caught on a Long package key. No new fixture: the mutant builder is not exported, and the existing `test-journey-control-exit.sh` reads only verdicts, which this does not change. From a card-disbursement requirements-driven build (build-plan rows 3.8 and 4.8). - fix(project-tests/e2e/journey-runner.js): **a `--positive-control` run now exits 0 when every rung's mutant was caught.** The walk's exit rule — exit 1 on any FAIL or INVALID row — was applied to the control run too. But a control's FAIL rows are its mutants being caught, so a control that proved 7 of 7 rungs exited 1, `verify-module.sh` graded the rung FINDING, and the module read INCOMPLETE on every run: a rung that could not go green, and so said nothing when it went red. `runExitCode()` keeps the walk's rule. A control now passes only when every `control` row is PASS, proven equals expected, and at least one rung ran. The summary now prints a `control verdict:` line that says which rows are the mutants'. Field run: MockServices `--positive-control` exited 0 with `control verdict: PASS`, 7 of 7 rungs (PASS 468, FAIL 7, INVALID 1 — unchanged, now read correctly). New fixture `tests/wave2/test-journey-control-exit.sh` over that run's captured findings (`fixtures/journey-control/`): 7 assertions, 7 green on the fix, red against the previous runner (no `runExitCode`). From a card-disbursement requirements-driven build (build-plan row 3.9). - fix(project-tests/e2e/monkey.js): **a round that leaves the app is no longer scored as an app crash, and the monkey no longer signs itself out.** A `backAfterSubmit` round clicked a control that pushes no history entry (a theme toggle), so Back left the app for `login.html` and then `about:blank`. The oracle scored each of those pages "blank page", every later round on the target scored the same page again, and the next target was never reached. Now, when Back leaves the app, the round returns with Forward and says so. Buttons labelled sign out / log off are no longer clicked. A round that leaves no page behind re-lands on its target, or reports `lost` with the number of rounds left unfuzzed. Requiring the file no longer runs main, and it exports `leftApp` / `SIGN_OUT`. Field run (seed 410855898, 2 targets): 13 observations with 10 crash-class and one target never reached → 26 observations, 0 crash-class, both targets fuzzed. New fixture `tests/wave2/test-monkey-left-app.sh` over the URLs and labels captured in that run: 11 assertions, 11 green on the fix, red against the previous monkey (no exports; it ran main on require). From a card-disbursement requirements-driven build (build-plan row 3.9). - fix(project-bin/conformance-check.sh): **a module built after the first conformance run is now regression-guarded, and `--update-baseline --module X` no longer wipes the other modules' baseline rows.** The baseline is written by the first run. A row it had never seen was compared with "", so it could never regress — silently and permanently for every module built later. That project's committed baseline held 1 of its 4 modules (127 of 246 rows). Separately, `--module` scoped the measurement but not the write, so a scoped `--update-baseline` replaced the whole file with one module's rows. Rows with no baseline line are now baselined when first seen, and the run prints `newly baselined N`. A `--module` rewrite keeps every other module's rows. An unscoped `--update-baseline` still rewrites the whole file. New fixture `tests/wave2/test-conformance-baseline-scope.sh` (stubbed mxcli; the assertions are about the baseline file, not the DESCRIBE parser): 7 assertions, 7 green on the fix, 5 red against the previous script. Field run on that project (mxcli v0.24.0, 4 modules, 291 ledger rows): from its committed 127-row baseline, a plain run measured 246 OK and printed `newly baselined 119`, leaving a 246-row baseline. `--module MockServices --update-baseline` then rewrote only that module's 39 rows and kept the other 207. From a card-disbursement requirements-driven build (build-plan row 3.9). diff --git a/contrib/inbox/2026-09-26-journey-runner-persona-and-local-acts.md b/contrib/inbox/2026-09-26-journey-runner-persona-and-local-acts.md new file mode 100644 index 0000000..df43a21 --- /dev/null +++ b/contrib/inbox/2026-09-26-journey-runner-persona-and-local-acts.md @@ -0,0 +1,31 @@ +# journey-runner: `persona` is a label, not a login; four acts and `data.rebase` a real console journey needed + +**From:** card-disbursement requirements-driven build (build-plan rows 3.8 and 4.7) +**Date:** 2026-09-26 +**Kind:** learning +**Field evidence:** read `project-tests/e2e/journey-runner.js` main, which logs in once with `H.login(page)` before the journey loop. The installed copy was patched locally at row 3.8, and its admin-console journeys ran 57/0/0 and 52/0/0 with it. +**Proposed target:** `project-tests/e2e/journey-runner.js`; `skills/journey-proof.md` (schema) + +--- + +**Persona.** The runner logs in once, as `TEST_USER`, for every journey file it is given; a journey's +`persona` is only printed. A cross-role check ("every role sees the badge") has to be one journey file, +run once per role, and nothing in the schema says so. A journey declaring `persona: ` and +run under an admin `TEST_USER` passes for the wrong reason. The runner already turns an admin fallback +(`usedFallback`) into INVALID, but it does not refuse a mismatch between persona and `TEST_USER`. +Proposal: when `persona` is set and is not `TEST_USER`, report the login INVALID. + +**Local extensions** (the diff is in the project's `tests/e2e/journey-runner.js`; harvest-learnings will +pick it up as a locally patched installed script): +- `click` + `row` / `in` / `target`: a grid repeats one widget name on every row, so a first-match + `.mx-name-` clicks whichever row renders first. `row` is text identifying ONE row (innermost match, + `last()`), `in` narrows the click to one grid, and `target` picks a child (a pluggable switch's `[role="switch"]`). +- `radio`: choose an option by its caption, then read the checked option back. +- `confirm`: press a button of Mendix's confirmation dialog by its caption, with an optional `says` check. +- `request`: an HTTP call made by a system client, not the persona. Credentials come from env vars the step + names; missing credentials are an InstrumentFault, reported INVALID (never FAIL). +- `data.rebase`: a step that deletes rows on purpose (a fixture reset) re-takes the baseline for the + named entities, so later deltas are not measured from a count that no longer exists. + +(The two mutant fixes from the same patch set, a numeric `mustPointAt` sentinel and `outcome` on an exact +`expect`, landed directly; see CHANGELOG.) diff --git a/contrib/inbox/2026-09-26-look-obligation-reads-first-valid-at.md b/contrib/inbox/2026-09-26-look-obligation-reads-first-valid-at.md new file mode 100644 index 0000000..95cc158 --- /dev/null +++ b/contrib/inbox/2026-09-26-look-obligation-reads-first-valid-at.md @@ -0,0 +1,24 @@ +# obligation-check: LOOK staleness reads the FIRST `VALID AT`, so a report with addenda goes STALE + +**From:** card-disbursement requirements-driven build (build-plan row 4.8) +**Date:** 2026-09-26 +**Kind:** bug +**Field evidence:** `gate-check.sh 5` printed "Obligation look STALE: … valid-at , 10 model commit(s) since". Every page in the report had been looked at again after its last model change, in three dated addenda further down the same file. +**Proposed target:** `bin/lib/obligation-check.sh` (look), `skills/module-review.md` (report shape) + +--- + +The project keeps one dated review file (`ui-review-.html`) and appends an addendum per build +row, each with its own `VALID AT`. The check reads the first `VALID AT` in the first 8000 bytes, which is +the headline's original stamp, so the file went STALE after later model commits. + +Workaround used: a report-level `VALID AT: ` in the headline, with the justification written +beside it; the original stamp was renamed "First true at". That tripped the proof rule next. Every +`PROOF-OF-LOOK:` line anywhere in the file must cite a shot newer than that stamp, so the older rows' +proof lines FAULTed ("screenshots older than VALID AT"). Fix used: re-shoot the pages on the current +commit, keep only current `PROOF-OF-LOOK:` lines, and turn the older ones into plain "earlier (superseded)" lines. +After that, look went 3 of 4 discharged. + +Two coherent fixes; pick one and say it in module-review.md: +- the check reads the NEWEST `VALID AT` and only the proofs at or after it; or +- the skill says "one report per LOOK pass, never addenda", and the check warns when a file holds more than one `VALID AT`. diff --git a/contrib/inbox/2026-09-26-mxcli-v024-rest-json-page-field-notes.md b/contrib/inbox/2026-09-26-mxcli-v024-rest-json-page-field-notes.md new file mode 100644 index 0000000..9f071bc --- /dev/null +++ b/contrib/inbox/2026-09-26-mxcli-v024-rest-json-page-field-notes.md @@ -0,0 +1,57 @@ +# mxcli v0.24 field notes from an integration module: REST calls, JSON structures, mappings, page hooks, OQL, test endpoint + +**From:** card-disbursement requirements-driven build (build-plan phase 4, rows 4.2–4.7) +**Date:** 2026-09-26 +**Kind:** bug +**Field evidence:** each item below was probed on a scratch copy of the model (mxcli v0.24.0, Mendix 11.13.0) and checked with `mx check` and/or a BSON read of the unit; the working rule was then used on the real model. +**Proposed target:** `bug-logs/mxcli-bugs.md` (one entry each), plus the skill named per item + +--- + +1. **`mxcli run --local --test-endpoint` writes into the working model while the app runs.** It adds + a temporary `MxTest` module (a `RegisterEndpoint` microflow), `javasource/mxtest/`, + `themesource/mxtest/` and several `mprcontents/` units, and edits the Project settings unit. + `git status` shows about 9 extra model paths, and lint reads CUSTOM002 +1 / CONV013 +1, so a lint gate sees a rise. + A clean stop prints "test endpoint removed; project restored" and reverts all of it. Verified by + `SHOW MICROFLOWS IN MxTest` → module not found, no `javasource/mxtest/`, and the settings unit back to its value. + An exec or commit made while it is up carries MxTest into the model. + Working rule: no exec and no commit until the app is stopped cleanly. Target: `skills/learned-mdl-preflight.md` STOP table. + Separately, every `mxcli test --attach` run rewrites the `.mpr` (it creates, then removes, the test + microflows). The units stay byte-identical (only `_Transaction` changes), but the mtime moves. See the verify-module note of the same date. + +2. **`rest call … body $Var` sends the literal text `$Var`.** The bare-expression body form writes a + StringTemplate "$Var" with an empty parameter list. `body '{1}' with ({1} = $Var)` round-trips + correctly (BSON read + DESCRIBE). + +3. **REST call custom error handlers are accepted now, but not inside a loop.** `on error { … }` and + `on error without rollback { … }` on a `rest call` pass `mx check` (Mendix 11.13). The REST skill's + "only `on error continue` (CE6035)" is stale; restamp it. Inside a WHILE loop, any custom handler is + CE0644 ("must be 'Rollback' inside a looped activity"). Working shape: the attempt is its own + sub-microflow, called from the retry loop. + +4. **lint CONV013 false positive also hits REST calls.** Two REST-calling microflows are reported as "uses '' + error handling instead of Custom", while the BSON holds `ErrorHandlingType: CustomWithoutRollBack`. This is + the same false positive already known for other activities. + +5. **`create or modify entity` replaces the attribute list.** Any attribute left out of the statement + is removed, with a warning naming CE1613 for anything still bound to it. On a persistent entity with data, that is a + drop. Target: STOP table ("rename via `alter entity … rename attribute`"). + +6. **A JSON structure key named `CONTEXT` is CE9524 at `mx check` only.** It passes `mxcli check` and + exec. `custom name map ('CONTEXT' as 'ApprovalContext')` fixes it; the wire key stays `CONTEXT`. + Hypothesis: other reserved-looking keys behave the same way. Only `CONTEXT` was probed. + +7. **`import from mapping` needs a variable as its source.** `… ($Call/ResponseBody)` is a parse error + ("mismatched input '/'"). `declare $Body String = $Call/ResponseBody;` then `($Body)` works. It was also + re-confirmed on 0.24 that a doc comment before `create or modify … mapping` is ignored ("already in sync"). + +8. **`alter page … replace with { … }` is not re-run safe.** The replacement is built while + the old widget's children still hold their names. A replacement that reuses any child name (the natural + case: the same grid with one column added) is refused as a duplicate widget name, and a second run of + the same script always fails. Working shape: `drop widget X; insert into { … }` passed mx + check and ran twice on the probe and twice on the model. Target: `skills/learned-mdl-preflight.md` or the page-alter recipe. + +9. **`oql --direct` rejects `HAVING` without `GROUP BY`** ("mismatched input 'HAVING'"). A journey + seed that must resolve only when a precondition holds can use a scalar subquery in the WHERE clause: + `… WHERE … AND (SELECT COUNT(x.ID) FROM M.E AS x) = 2`. It returns no row when the condition fails, so the + seed reports INVALID, never a feature FAIL. Target: `skills/journey-proof.md` (seed recipes). diff --git a/contrib/inbox/2026-09-26-verify-module-graph-fault-after-test-attach.md b/contrib/inbox/2026-09-26-verify-module-graph-fault-after-test-attach.md new file mode 100644 index 0000000..12f1c41 --- /dev/null +++ b/contrib/inbox/2026-09-26-verify-module-graph-fault-after-test-attach.md @@ -0,0 +1,20 @@ +# verify-module: graph sweep FAULTs "catalog is stale" after any `mxcli test --attach` + +**From:** card-disbursement requirements-driven build (build-plan row 4.8) +**Date:** 2026-09-26 +**Kind:** bug +**Field evidence:** ran the journey seed with `mxcli test … --attach`, then `bin/verify-module.sh Integration`: `11-graph` FAULT "catalog is stale" and the run read INCOMPLETE. After `REFRESH CATALOG FULL` (6 s), verify-module ran the graph sweep normally. The model units were byte-identical before and after the attach (a sqlite compare of the `Unit` table found 0 of 666 rows different; only `_Transaction` moved). +**Proposed target:** `project-bin/verify-module.sh` (the graph step), `skills/module-review.md` + +--- + +An attach run rewrites the `.mpr`: it creates the MxTest test microflows, then removes them. So the file +ends up newer than `.mxcli/catalog.db` although the model has not changed. The graph step correctly refuses a stale +catalog. But its FAULT line does not say what to do, and the obvious sequence (seed the data with +`test --attach`, then verify) always trips it. + +Options, cheapest first: +1. The FAULT line names the remedy: "run `REFRESH CATALOG FULL`, then re-run". +2. verify-module refreshes the catalog itself when the only newer thing is the `.mpr` mtime. Hypothesis: + a content hash of the `Unit` table would tell "touched" from "changed". +3. The module-review skill orders it: seed → `REFRESH CATALOG FULL` → verify-module. diff --git a/contrib/inbox/2026-09-26-wiring-sweep-disabled-by-design-and-conditional-controls.md b/contrib/inbox/2026-09-26-wiring-sweep-disabled-by-design-and-conditional-controls.md new file mode 100644 index 0000000..55ac142 --- /dev/null +++ b/contrib/inbox/2026-09-26-wiring-sweep-disabled-by-design-and-conditional-controls.md @@ -0,0 +1,20 @@ +# wiring-sweep: no verdict row for "disabled by design", and DOM enumeration misses conditionally visible controls + +**From:** card-disbursement requirements-driven build (build-plan row 4.8) +**Date:** 2026-09-26 +**Kind:** learning +**Field evidence:** a console page with 3 Data grid 2 grids swept by direct Playwright: 48 elements were enumerated from the DOM, 44 PASS, and 4 FAIL "not interactable". The 4 were previous/next pagers on grids whose rows all fit on one page. The page's Disarm button (`Visible: [Active]`) was not in the enumeration, because no row was armed at sweep time. +**Proposed target:** `skills/wiring-sweep.md` (verdict table, enumeration step) + +--- + +1. **Disabled by design.** Data grid 2 disables previous/next when every row fits on one page. The + verdict table maps "disabled" → FAIL (not interactable), so 4 elements read FAIL with no defect behind them. + The sweep report carried them as FAIL with a by-design disposition. A row "disabled, and the widget's + own state explains it (pager on one page, save on an unchanged form)" → PASS-by-design, or N/A with a stated + reason, would keep the denominator honest without a hand-written excuse per project. +2. **Conditional visibility.** A control behind a `Visible:` expression is absent from the DOM unless + its condition holds, so DOM enumeration silently shrinks the denominator. Disarm was swept separately: + the landing armed a row first (a Timeout scenario, since Success leaves the row inactive), and it went 1 of 1 PASS. + Proposal: enumerate the widgets from the page script (or `describe page`), not only the DOM, and + flag any widget with a `Visible:` condition that the sweep did not reach. diff --git a/project-tests/e2e/journey-runner.js b/project-tests/e2e/journey-runner.js index 165820a..3f5c972 100644 --- a/project-tests/e2e/journey-runner.js +++ b/project-tests/e2e/journey-runner.js @@ -506,7 +506,12 @@ function controlMutants(j) { for (const s of c.steps) { for (const spec of (s.data && s.data.assocMustBeSet) || []) { if (spec.mustPointAt) { - spec.mustPointAt.value = '__LOC_THAT_WAS_NEVER_SELECTED__'; + // A numeric key (Long/Integer) needs a numeric never-picked value: the text + // sentinel is an OQL error there, the targeted check reads INVALID instead of + // FAIL, and the rung is reported unproven over a working assertion + // (card-disbursement build, row 3.8: a Long package key). + spec.mustPointAt.value = /^-?\d+$/.test(String(spec.mustPointAt.value)) + ? '-424242' : '__LOC_THAT_WAS_NEVER_SELECTED__'; return; } } @@ -515,9 +520,14 @@ function controlMutants(j) { }, named('points at the seeded')); // Rung 5 — the end-to-end outcome claim. - add('outcome', 'outcome query with an unreachable floor', (c) => { - if (!c.outcome || c.outcome.atLeast === undefined) return false; - c.outcome.atLeast = 999999; + // An exact `expect` is the stronger claim; the mutant once knew floors alone, so a + // journey declaring `expect` had this rung reported unproven (card-disbursement build, + // row 4.8). + add('outcome', 'outcome query with an unreachable floor or exact value', (c) => { + if (!c.outcome) return false; + if (c.outcome.atLeast !== undefined) c.outcome.atLeast = 999999; + else if (c.outcome.expect !== undefined) c.outcome.expect = 999999; + else return false; }, named('end state')); // The denominator is declared, not counted from whatever happened to be added. diff --git a/skills/journey-proof.md b/skills/journey-proof.md index 473ef49..0587831 100644 --- a/skills/journey-proof.md +++ b/skills/journey-proof.md @@ -116,8 +116,8 @@ the same vacuity bug one level up. | `trace-order` | expect a microflow that never fires | span ORDER, not existence | | `trace-negative` | assert a microflow that *did* run must not have | the negative trace claim | | `data-delta` | expect +2 from a +1 action | the row-count delta | -| `data-target` | `mustPointAt` a value never picked | the association *target* | -| `outcome` | unreachable floor on the outcome query | the end-to-end claim | +| `data-target` | `mustPointAt` a value never picked (numeric for a numeric key) | the association *target* | +| `outcome` | unreachable `atLeast` or `expect` on the outcome query | the end-to-end claim | **A rung with no mutant is UNPROVEN, which is `fault` — never `pass`.** Same discipline as INVALID. From 3dfc236574faea0010b8a67b7b148c0e2083bf5e Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 13:46:32 +0000 Subject: [PATCH 10/45] journey-runner: a click on a covered widget fails and names the cover A forced click skips Playwright's actionability check, so a toast or popup underlay over the widget received the click and the step read as done. The click action now hit-tests the widget's centre (elementFromPoint, 5 x 250 ms for animations) before clicking and throws "covered by .". Field run on a card-disbursement build: popup-then-click journey 4/0 on the previous runner, FAIL "covered by .mx-underlay" on this one. Inbox: run --local starts with scheduled events off (ScheduledEventExecution). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Pm41sykqDGcvioGzsY5H2N --- CHANGELOG.md | 1 + ...26-09-26-run-local-scheduled-events-off.md | 30 +++++++++++++++++++ project-tests/e2e/journey-runner.js | 19 ++++++++++++ 3 files changed, 50 insertions(+) create mode 100644 contrib/inbox/2026-09-26-run-local-scheduled-events-off.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 980adb7..49f0089 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,7 @@ three commits past it), and a bug report can name a release instead of a sha nob Sections dated before 2026-09-19 predate the cycle and stay as they are. ## Unreleased +- fix(project-tests/e2e/journey-runner.js): **a `click` on a covered widget now fails and names the cover, instead of reading as done.** The click is forced (`{force: true}`), which skips Playwright's actionability check, so whatever sits on top receives it. On the field project a success toast at the top right lay over the next page's header actions, the journey's Check now landed on the toast, and the loss surfaced two steps later as a missing data row. Before the forced click the runner now hit-tests the widget's centre with `elementFromPoint`, up to 5 × 250 ms so an animating overlay can clear. If the point still belongs to something else, the action throws `.mx-name- is covered by .`. Field run (mxcli v0.24.0, Mendix 11.13.0), a 3-step journey that opens a popup and then clicks a page button behind it: the previous runner scored it 4 PASS 0 FAIL, this one fails step 3 with `covered by .mx-underlay`. In the same project the identical block ran 71 + 30 + 14 checks across three role journeys with no false cover. No new fixture: the check runs in the browser, and the field run above is its red/green. New inbox note `contrib/inbox/2026-09-26-run-local-scheduled-events-off.md`: `run --local` starts with scheduled events off, and proof needs `--runtime-setting ScheduledEventExecution=ALL`. From a card-disbursement requirements-driven build (build-plan row 5.6). - fix(project-tests/e2e/journey-runner.js): **two positive-control mutants no longer report a working assertion as unproven.** The `outcome` mutant only broke an `atLeast` floor, so a journey whose outcome declares an exact `expect` (the stronger claim) had the rung reported UNPROVEN ("declares nothing this mutant can break"). An exact `expect` is now mutated to 999999 too. The `data-target` mutant wrote a text sentinel into `mustPointAt`; on a numeric key (Long/Integer) that is an OQL error, so the targeted check read INVALID instead of FAIL and the rung could never be proven. A numeric value now gets the numeric sentinel `-424242`. `skills/journey-proof.md`'s mutant table says both. Field runs (mxcli v0.24.0, Mendix 11.13.0): Integration `--positive-control` 5 → 6 of 7 rungs, `outcome` caught by the journey's `expect: 2` end-state check (the seventh, data-target, is unproven because that journey declares no `mustPointAt`); MockServices data-target INVALID → caught on a Long package key. No new fixture: the mutant builder is not exported, and the existing `test-journey-control-exit.sh` reads only verdicts, which this does not change. From a card-disbursement requirements-driven build (build-plan rows 3.8 and 4.8). - fix(project-tests/e2e/journey-runner.js): **a `--positive-control` run now exits 0 when every rung's mutant was caught.** The walk's exit rule — exit 1 on any FAIL or INVALID row — was applied to the control run too. But a control's FAIL rows are its mutants being caught, so a control that proved 7 of 7 rungs exited 1, `verify-module.sh` graded the rung FINDING, and the module read INCOMPLETE on every run: a rung that could not go green, and so said nothing when it went red. `runExitCode()` keeps the walk's rule. A control now passes only when every `control` row is PASS, proven equals expected, and at least one rung ran. The summary now prints a `control verdict:` line that says which rows are the mutants'. Field run: MockServices `--positive-control` exited 0 with `control verdict: PASS`, 7 of 7 rungs (PASS 468, FAIL 7, INVALID 1 — unchanged, now read correctly). New fixture `tests/wave2/test-journey-control-exit.sh` over that run's captured findings (`fixtures/journey-control/`): 7 assertions, 7 green on the fix, red against the previous runner (no `runExitCode`). From a card-disbursement requirements-driven build (build-plan row 3.9). - fix(project-tests/e2e/monkey.js): **a round that leaves the app is no longer scored as an app crash, and the monkey no longer signs itself out.** A `backAfterSubmit` round clicked a control that pushes no history entry (a theme toggle), so Back left the app for `login.html` and then `about:blank`. The oracle scored each of those pages "blank page", every later round on the target scored the same page again, and the next target was never reached. Now, when Back leaves the app, the round returns with Forward and says so. Buttons labelled sign out / log off are no longer clicked. A round that leaves no page behind re-lands on its target, or reports `lost` with the number of rounds left unfuzzed. Requiring the file no longer runs main, and it exports `leftApp` / `SIGN_OUT`. Field run (seed 410855898, 2 targets): 13 observations with 10 crash-class and one target never reached → 26 observations, 0 crash-class, both targets fuzzed. New fixture `tests/wave2/test-monkey-left-app.sh` over the URLs and labels captured in that run: 11 assertions, 11 green on the fix, red against the previous monkey (no exports; it ran main on require). From a card-disbursement requirements-driven build (build-plan row 3.9). diff --git a/contrib/inbox/2026-09-26-run-local-scheduled-events-off.md b/contrib/inbox/2026-09-26-run-local-scheduled-events-off.md new file mode 100644 index 0000000..48239bf --- /dev/null +++ b/contrib/inbox/2026-09-26-run-local-scheduled-events-off.md @@ -0,0 +1,30 @@ +# `mxcli run --local` starts with scheduled events off — a job-driven feature is never exercised locally + +**From:** card-disbursement requirements-driven build (build-plan row 5.6) +**Date:** 2026-09-26 +**Kind:** learning +**Field evidence:** mxcli v0.24.0, Mendix 11.13.0, hsqldb. Two starts of the same model, runtime log `app/.mxcli/runtime.log`: +default start logs `Core: Synchronizing scheduled events: None`; with `--runtime-setting ScheduledEventExecution=ALL` +it logs `Synchronizing scheduled events: All` and `ActionManager: Scheduling . to run every 1 minutes`. +**Proposed target:** `skills/testing-shape.md` (false-green register) and whichever skill documents `run --local` for Stage 5 proofs + +--- + +A module whose behaviour hangs off a scheduled event (a status refresh every minute) passes every +local journey without the job ever running: the default `run --local` start does not execute +scheduled events. The journeys looked complete — the manual "Check now" path calls the same +microflow — but the timer path itself was never observed. + +Measured proof with the setting on: two ticks (13:34:00, 13:35:00) each wrote one integration-call +row (10 → 12) and advanced the status table's CheckedOn to the tick time; 0 ERROR/WARN lines after start. + +What the project did, and what I'd suggest the toolkit say: +1. Prove scheduled work in its **own** start with `--runtime-setting ScheduledEventExecution=ALL`, + counting rows before and after ≥ 2 ticks, and grep the runtime log for the `Scheduling …` line. +2. Keep journeys on the default (off) when they count rows exactly — a tick inside a step adds a row. + Say so in the journey's `_note`, so nobody "fixes" the count later. +3. Candidate false-green row: "scheduled event never ran — `run --local` default" (Rung: data / behaviour). + +Related, same row (already fixed in `project-tests/e2e/journey-runner.js`, see CHANGELOG): a forced +click skips Playwright's actionability check, so an overlay (toast, popup underlay) receives it and +the step reads as done. Candidate false-green row: "forced click landed on an overlay". diff --git a/project-tests/e2e/journey-runner.js b/project-tests/e2e/journey-runner.js index 3f5c972..b9d3a45 100644 --- a/project-tests/e2e/journey-runner.js +++ b/project-tests/e2e/journey-runner.js @@ -160,6 +160,25 @@ async function act(page, a, vars, note) { await el.scrollIntoViewIfNeeded({ timeout: 2000 }).catch(() => {}); if (!(await el.isVisible({ timeout: 6000 }).catch(() => false))) throw new Error(`.mx-name-${a.widget} not visible`); + // Hit-test before the forced click. `force` skips Playwright's actionability check, so a + // click lands on whatever sits on top: a toast over the page's header actions swallowed + // a click and the step read as "the action did nothing" two steps later (card-disbursement + // build, row 5.6). A user's click lands there too, so a cover that outlasts an animation + // (5 × 250 ms) is the finding, named by its class. + let cover = null; + for (let t = 0; t < 5; t++) { + cover = await el.evaluate(n => { + const r = n.getBoundingClientRect(); + const e = document.elementFromPoint(r.x + r.width / 2, r.y + r.height / 2); + if (!e || n.contains(e) || e.contains(n)) return null; + const c = e.closest('[class]') || e; + return String(c.className || c.tagName).trim().split(/\s+/).slice(0, 3).join('.'); + }).catch(() => null); + if (!cover) break; + await PAUSE(250); + } + if (cover) + throw new Error(`.mx-name-${a.widget} is covered by .${cover} — a click lands on that, not on the widget`); await el.click({ force: true }).catch(() => el.click()); break; } From 5e265d919cd6433e44e865372a7d57bbc666b91e Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 14:43:15 +0000 Subject: [PATCH 11/45] fix(helpers.js): logout() ends the session on Mendix 11 logout() read the CSRF token from mx.session.getCSRFToken(), which Mendix 11 does not have, so it posted the xas logout action with an empty token. The server still answers 200 {} and keeps the session. Every journey and probe therefore held a session until the timeout, and on a trial licence the session cap then refused the next sign-in. The token now falls back to mx.session.sessionData.csrftoken, and a missing token returns false instead of posting. Field run (card-disbursement requirements-driven build, row 5.7; mxcli v0.24.0, Mendix 11.13.0): 13 sessions had leaked from runs that all "logged out". After the fix, System.Session per role read 1 before sign-in, 2 while signed in and 1 after logout, for admin, operations and banker roles. Also: an inbox note on review-report.js vs coverage-check-all.sh's one-line-per-BRD output (permanent coverage FAULT on multi-ledger modules). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Pm41sykqDGcvioGzsY5H2N --- CHANGELOG.md | 1 + ...-26-review-report-multi-ledger-coverage.md | 44 +++++++++++++++++++ project-tests/e2e/helpers.js | 10 ++++- 3 files changed, 53 insertions(+), 2 deletions(-) create mode 100644 contrib/inbox/2026-09-26-review-report-multi-ledger-coverage.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 49f0089..2869bad 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,7 @@ three commits past it), and a bug report can name a release instead of a sha nob Sections dated before 2026-09-19 predate the cycle and stay as they are. ## Unreleased +- fix(project-tests/e2e/helpers.js): **`logout()` now actually ends the session on Mendix 11.** It read the CSRF token from `mx.session.getCSRFToken()`, which Mendix 11 does not have, so it posted the xas `logout` action with an empty token. The server still answers 200 `{}` to that and keeps the session, so every journey (the runner calls `H.logout` at the end of a walk) and every probe that "logged out" held a session until the timeout. On a trial licence the session cap then refuses the next sign-in ("Maximum number of sessions exceeded"), which reads as a broken login rather than a leak. The token now falls back to `mx.session.sessionData.csrftoken`, and a missing token returns `false` instead of posting. Field run (mxcli v0.24.0, Mendix 11.13.0): 13 sessions had leaked from runs that all "logged out"; after the fix, `System.Session` per role read 1 before sign-in, 2 while signed in and 1 after logout for three roles, where before it grew by one per run. No new fixture: the call runs in the browser, and the session count above is its red/green. New inbox note `contrib/inbox/2026-09-26-review-report-multi-ledger-coverage.md`: `review-report.js` cannot parse `coverage-check-all.sh`'s one-line-per-BRD output, so every multi-ledger module carries a permanent coverage FAULT over a clean count. From a card-disbursement requirements-driven build (build-plan row 5.7). - fix(project-tests/e2e/journey-runner.js): **a `click` on a covered widget now fails and names the cover, instead of reading as done.** The click is forced (`{force: true}`), which skips Playwright's actionability check, so whatever sits on top receives it. On the field project a success toast at the top right lay over the next page's header actions, the journey's Check now landed on the toast, and the loss surfaced two steps later as a missing data row. Before the forced click the runner now hit-tests the widget's centre with `elementFromPoint`, up to 5 × 250 ms so an animating overlay can clear. If the point still belongs to something else, the action throws `.mx-name- is covered by .`. Field run (mxcli v0.24.0, Mendix 11.13.0), a 3-step journey that opens a popup and then clicks a page button behind it: the previous runner scored it 4 PASS 0 FAIL, this one fails step 3 with `covered by .mx-underlay`. In the same project the identical block ran 71 + 30 + 14 checks across three role journeys with no false cover. No new fixture: the check runs in the browser, and the field run above is its red/green. New inbox note `contrib/inbox/2026-09-26-run-local-scheduled-events-off.md`: `run --local` starts with scheduled events off, and proof needs `--runtime-setting ScheduledEventExecution=ALL`. From a card-disbursement requirements-driven build (build-plan row 5.6). - fix(project-tests/e2e/journey-runner.js): **two positive-control mutants no longer report a working assertion as unproven.** The `outcome` mutant only broke an `atLeast` floor, so a journey whose outcome declares an exact `expect` (the stronger claim) had the rung reported UNPROVEN ("declares nothing this mutant can break"). An exact `expect` is now mutated to 999999 too. The `data-target` mutant wrote a text sentinel into `mustPointAt`; on a numeric key (Long/Integer) that is an OQL error, so the targeted check read INVALID instead of FAIL and the rung could never be proven. A numeric value now gets the numeric sentinel `-424242`. `skills/journey-proof.md`'s mutant table says both. Field runs (mxcli v0.24.0, Mendix 11.13.0): Integration `--positive-control` 5 → 6 of 7 rungs, `outcome` caught by the journey's `expect: 2` end-state check (the seventh, data-target, is unproven because that journey declares no `mustPointAt`); MockServices data-target INVALID → caught on a Long package key. No new fixture: the mutant builder is not exported, and the existing `test-journey-control-exit.sh` reads only verdicts, which this does not change. From a card-disbursement requirements-driven build (build-plan rows 3.8 and 4.8). - fix(project-tests/e2e/journey-runner.js): **a `--positive-control` run now exits 0 when every rung's mutant was caught.** The walk's exit rule — exit 1 on any FAIL or INVALID row — was applied to the control run too. But a control's FAIL rows are its mutants being caught, so a control that proved 7 of 7 rungs exited 1, `verify-module.sh` graded the rung FINDING, and the module read INCOMPLETE on every run: a rung that could not go green, and so said nothing when it went red. `runExitCode()` keeps the walk's rule. A control now passes only when every `control` row is PASS, proven equals expected, and at least one rung ran. The summary now prints a `control verdict:` line that says which rows are the mutants'. Field run: MockServices `--positive-control` exited 0 with `control verdict: PASS`, 7 of 7 rungs (PASS 468, FAIL 7, INVALID 1 — unchanged, now read correctly). New fixture `tests/wave2/test-journey-control-exit.sh` over that run's captured findings (`fixtures/journey-control/`): 7 assertions, 7 green on the fix, red against the previous runner (no `runExitCode`). From a card-disbursement requirements-driven build (build-plan row 3.9). diff --git a/contrib/inbox/2026-09-26-review-report-multi-ledger-coverage.md b/contrib/inbox/2026-09-26-review-report-multi-ledger-coverage.md new file mode 100644 index 0000000..951294a --- /dev/null +++ b/contrib/inbox/2026-09-26-review-report-multi-ledger-coverage.md @@ -0,0 +1,44 @@ +# `review-report.js` cannot read multi-ledger coverage — every one-ledger-per-BRD module reports a coverage FAULT over a clean count + +**From:** card-disbursement requirements-driven build (build-plan rows 4.8 and 5.7) +**Date:** 2026-09-26 +**Kind:** bug +**Field evidence:** `bin/verify-module.sh ` on two modules that keep one ledger per BRD +(`architecture/modules//coverage-ledger/.md`). In both runs the coverage rung itself +passed (`coverage … ✓ clean`), and `.claude/loop/review//coverage.txt` read verbatim: + +``` +F002-system-availability-check: CLEAN — CLAIMED: 289 LEDGERED: 48 UNCLAIMED: 0 PHANTOM: 0 DOUBLE-CLAIMED:0 COUNT-MISMATCH:0 + +1 BRD(s) checked. +All clean. +``` + +Yet `review-report.json` carried `fault · review/coverage/` — "coverage-check wrote output +but no total/claimed/unclaimed/phantom/doubleClaimed count could be read from it". +**Proposed target:** `project-tests/e2e/review-report.js` (the counter parser) and +`project-bin/coverage-check-all.sh` (the line it emits) + +--- + +Two formats, one reader: + +- In single-ledger mode `review-module.sh` runs `coverage-check.sh --summary`, which prints one + counter per line (` leaves: N`, ` CLAIMED: N`, …). `review-report.js` parses + exactly that: regexes anchored at line start, `leaves:` included. +- In multi-ledger mode (added 2026-09-02) it runs `coverage-check-all.sh`, which greps the + counter lines, **drops `leaves:`** (its pattern is `CLAIMED|LEDGERED|UNCLAIMED|…`), and flattens + them onto one `: CLEAN — …` line per BRD. No anchored regex matches, so every + counter reads null, and the report says "unmeasured". + +The fail-safe did its job, because an unreadable counter is reported as a fault, never as a zero. +But the fault is permanent on every multi-ledger module, so a reader learns to skip the row. That +is the false-red cousin of the false green the counter guard exists for. + +Suggested fix: +1. In `coverage-check-all.sh`, keep `leaves:` in the grep, so each BRD line carries its total. +2. In `review-report.js`, when the file has `^: (CLEAN|FINDINGS) — …` lines, sum each counter + across them, and FAULT if any BRD line reads `FAULT` or lacks a counter. Keep the + single-BRD branch as it is. +3. Fixture: the capture above as golden input (a real two-BRD capture would be better), with + asserted sums. It should be red on the current parser. diff --git a/project-tests/e2e/helpers.js b/project-tests/e2e/helpers.js index 77cf91f..c8a7131 100644 --- a/project-tests/e2e/helpers.js +++ b/project-tests/e2e/helpers.js @@ -354,11 +354,17 @@ async function login(page) { // GET /logout does NOT work — it renders a page and leaves the session alive // (measured: the app root still resolved to the dashboard afterwards). The xas // `logout` action does; after it, / redirects to login.html. +// The token matters: without it the server still answers 200 `{}` and the +// session lives on. Mendix 11 has no mx.session.getCSRFToken — the token is in +// mx.session.sessionData.csrftoken (measured 2026-09-26, 11.13: 13 leaked +// sessions from runs that all "logged out", then the trial cap refused login). async function logout(page) { try { const status = await page.evaluate(async () => { - const token = (window.mx && mx.session && mx.session.getCSRFToken) - ? mx.session.getCSRFToken() : ''; + const s = (window.mx && mx.session) || {}; + const token = s.getCSRFToken ? s.getCSRFToken() + : ((s.sessionData && s.sessionData.csrftoken) || ''); + if (!token) return 'no-token'; const res = await fetch('/xas/', { method: 'POST', headers: { 'Content-Type': 'application/json', 'X-Csrf-Token': token }, From 07e91114a42f69059777ff1de09ec01fe3ca5b7b Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 26 Sep 2026 21:34:18 +0000 Subject: [PATCH 12/45] BUG-141: workflow condition outcomes carry no PersistentId mxcli's conditionOutcomeToGen() writes Boolean, enumeration and void condition outcomes without a PersistentId. The runtime therefore gives them a new identity on every load, and every instance paused after a branch point turns Incompatible on the next deploy, including a restart with no model change. Green on every static rung and on any live run inside one deploy. - bug-logs/mxcli-bugs.md: BUG-141 with the three live reproductions, the BSON and metamodel evidence, the upstream one-line fix, and the admin-side plan until it ships. - bug-logs/pending-github-issues/: paste-ready upstream issue, not filed. - skills/learned-detection-gaps.md: register row, caught only across a restart. Field run: card-disbursement requirements-driven build, mxcli v0.24.0, Mendix 11.13.0. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Pm41sykqDGcvioGzsY5H2N --- CHANGELOG.md | 2 + bug-logs/mxcli-bugs.md | 89 +++++++++++++++++++ ...g141-condition-outcome-no-persistent-id.md | 65 ++++++++++++++ skills/learned-detection-gaps.md | 1 + 4 files changed, 157 insertions(+) create mode 100644 bug-logs/pending-github-issues/bug141-condition-outcome-no-persistent-id.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 2869bad..a647afa 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,8 @@ three commits past it), and a bug report can name a release instead of a sha nob Sections dated before 2026-09-19 predate the cycle and stay as they are. ## Unreleased +- new(bug-logs): **BUG-141, workflow condition outcomes are written without a `PersistentId`, so every instance paused after a branch point turns `Incompatible` on the next deploy — including a restart with no model change.** mxcli's `conditionOutcomeToGen()` gives no persistent ID to Boolean, enumeration or void outcomes (a `DECISION`, or a call-microflow task with outcomes). Activities and user-task outcomes do get one. The runtime therefore re-identifies those outcomes on every load, and any instance that already took one fails with "A selected outcome has been replaced in the already executed path". Every static rung is clean, and so is every journey that stays inside one deploy. Field evidence (mxcli v0.24.0, still on upstream `main` 9509176; Mendix 11.13.0): three live reproductions (a no-change restart, an unrelated-page redeploy, a one-line page redeploy), a BSON dump showing the ID missing only on condition outcomes, and the runtime metamodel's `ModelBooleanConditionOutcome(…, persistentId, …)` constructor. Upstream fix: one `addFreshPersistentID(g)` per case. The paste-ready issue is `bug-logs/pending-github-issues/bug141-condition-outcome-no-persistent-id.md`, not yet filed. No MDL workaround: the app's workflow admin must handle `Incompatible` instances until upstream ships it. From a card-disbursement requirements-driven build (build-plan row 6.12). +- learn(skills/learned-detection-gaps.md): **new row: a workflow branch point is green on rungs 1–4, lint and every live run inside one deploy, and is caught only by a live run across a restart.** The row gives the one-instance restart probe, and notes that the defect sits between two deploys, which no single rung can see (BUG-141). From a card-disbursement requirements-driven build. - fix(project-tests/e2e/helpers.js): **`logout()` now actually ends the session on Mendix 11.** It read the CSRF token from `mx.session.getCSRFToken()`, which Mendix 11 does not have, so it posted the xas `logout` action with an empty token. The server still answers 200 `{}` to that and keeps the session, so every journey (the runner calls `H.logout` at the end of a walk) and every probe that "logged out" held a session until the timeout. On a trial licence the session cap then refuses the next sign-in ("Maximum number of sessions exceeded"), which reads as a broken login rather than a leak. The token now falls back to `mx.session.sessionData.csrftoken`, and a missing token returns `false` instead of posting. Field run (mxcli v0.24.0, Mendix 11.13.0): 13 sessions had leaked from runs that all "logged out"; after the fix, `System.Session` per role read 1 before sign-in, 2 while signed in and 1 after logout for three roles, where before it grew by one per run. No new fixture: the call runs in the browser, and the session count above is its red/green. New inbox note `contrib/inbox/2026-09-26-review-report-multi-ledger-coverage.md`: `review-report.js` cannot parse `coverage-check-all.sh`'s one-line-per-BRD output, so every multi-ledger module carries a permanent coverage FAULT over a clean count. From a card-disbursement requirements-driven build (build-plan row 5.7). - fix(project-tests/e2e/journey-runner.js): **a `click` on a covered widget now fails and names the cover, instead of reading as done.** The click is forced (`{force: true}`), which skips Playwright's actionability check, so whatever sits on top receives it. On the field project a success toast at the top right lay over the next page's header actions, the journey's Check now landed on the toast, and the loss surfaced two steps later as a missing data row. Before the forced click the runner now hit-tests the widget's centre with `elementFromPoint`, up to 5 × 250 ms so an animating overlay can clear. If the point still belongs to something else, the action throws `.mx-name- is covered by .`. Field run (mxcli v0.24.0, Mendix 11.13.0), a 3-step journey that opens a popup and then clicks a page button behind it: the previous runner scored it 4 PASS 0 FAIL, this one fails step 3 with `covered by .mx-underlay`. In the same project the identical block ran 71 + 30 + 14 checks across three role journeys with no false cover. No new fixture: the check runs in the browser, and the field run above is its red/green. New inbox note `contrib/inbox/2026-09-26-run-local-scheduled-events-off.md`: `run --local` starts with scheduled events off, and proof needs `--runtime-setting ScheduledEventExecution=ALL`. From a card-disbursement requirements-driven build (build-plan row 5.6). - fix(project-tests/e2e/journey-runner.js): **two positive-control mutants no longer report a working assertion as unproven.** The `outcome` mutant only broke an `atLeast` floor, so a journey whose outcome declares an exact `expect` (the stronger claim) had the rung reported UNPROVEN ("declares nothing this mutant can break"). An exact `expect` is now mutated to 999999 too. The `data-target` mutant wrote a text sentinel into `mustPointAt`; on a numeric key (Long/Integer) that is an OQL error, so the targeted check read INVALID instead of FAIL and the rung could never be proven. A numeric value now gets the numeric sentinel `-424242`. `skills/journey-proof.md`'s mutant table says both. Field runs (mxcli v0.24.0, Mendix 11.13.0): Integration `--positive-control` 5 → 6 of 7 rungs, `outcome` caught by the journey's `expect: 2` end-state check (the seventh, data-target, is unproven because that journey declares no `mustPointAt`); MockServices data-target INVALID → caught on a Long package key. No new fixture: the mutant builder is not exported, and the existing `test-journey-control-exit.sh` reads only verdicts, which this does not change. From a card-disbursement requirements-driven build (build-plan rows 3.8 and 4.8). diff --git a/bug-logs/mxcli-bugs.md b/bug-logs/mxcli-bugs.md index 7962dde..b92c903 100644 --- a/bug-logs/mxcli-bugs.md +++ b/bug-logs/mxcli-bugs.md @@ -5475,6 +5475,95 @@ and render it in `DESCRIBE MICROFLOW` so the round trip does not silently flip i callee's flag is readable in the model. --- +## BUG-141: workflow condition outcomes are written without a `PersistentId` — every instance paused after a decision turns `Incompatible` on the next deploy, even one that changes nothing + +> **NOT YET FILED** — paste-ready draft in `bug-logs/pending-github-issues/bug141-condition-outcome-no-persistent-id.md`. + +**Severity:** High for anything going live. It is silent on every static rung, and it strands +real work in production: the in-flight instances stop and need an admin. It does not block a +demo, because a demo stays inside one deploy. +**mxcli version when found:** v0.24.0. Still open on upstream `main` at 9509176 (2026-09-26). +**Mendix version:** 11.13.0 +**Discovered:** 2026-09-26, a card-disbursement requirements-driven build. The case workflow +had two call-microflow tasks with outcomes (one Boolean, one enumeration) ahead of its user tasks. +Both those and a workflow `DECISION` go through the same outcome writer. +**Reproducible:** yes, three times in one day on the same model, with three different deploys +(below). + +### Summary + +`conditionOutcomeToGen()` in `mdl/backend/modelsdk/workflow_write.go` writes +`Workflows$BooleanConditionOutcome`, `Workflows$EnumerationValueConditionOutcome` and +`Workflows$VoidConditionOutcome` with no `PersistentId`. The same file gives one to every +activity, to `UserTaskOutcome` (`userTaskOutcomeToGen`) and to `ParallelSplitOutcome` through +`addFreshPersistentID(g)`. The runtime's workflow metamodel builds +`ModelBooleanConditionOutcome(id, value, persistentId, flow, container)`, so it needs one. With +none stored, the outcome gets a new identity every time the model loads. + +A running instance records which outcome it took. After the next deploy that outcome no longer +exists under the recorded identity, so the engine marks the instance **`Incompatible`**: +*"A selected outcome has been replaced in the already executed path."* That hits every instance +paused **after** a decision, which in most workflows is every instance waiting at a user task. + +### What it takes to see it + +Nothing on the static ladder sees it. `check --references`, `exec`, `DESCRIBE WORKFLOW`, lint, +mxbuild and native `mx check` are all clean, and every journey passes, because a journey starts +and finishes its instances inside one run. It shows only when an instance **outlives a +restart**. + +### Evidence + +1. **Three live reproductions** (`mxcli run --local`, HSQLDB, Mendix 11.13.0). Each case was + started, left `InProgress` at the user task after the decision, and then: + - the app was **restarted with no model change at all** (workflow unit byte-identical). The + case came back `Incompatible`; + - a second case went through a redeploy that **changed only unrelated page documents**. It + came back `Incompatible`; + - a third went through a redeploy whose only change was **one date format on a dashboard page**. It came back + `Incompatible`. +2. **The stored unit.** A BSON dump of the workflow document shows `PersistentId` on every + activity and on every `UserTaskOutcome`, and on **none** of the condition outcomes. +3. **The runtime needs it.** `com.mendix.workflows-metamodel.jar` (11.13.0 runtime bundle) + constructs `ModelBooleanConditionOutcome(id, value, persistentId, flow, container)`. +4. **The writer.** `workflow_write.go` on `main` 9509176: `conditionOutcomeToGen()` (line 649) + returns all three outcome elements without calling `addFreshPersistentID(g)`. Its neighbours + do call it: `userTaskOutcomeToGen` (line 639) and the `ParallelSplitOutcome` loop in + `parallelSplitToGen` (line 573). The helper is at line 778. + +### Fix (upstream) + +Add one line in each of the three cases, before `return g`: + +```go +addFreshPersistentID(g) +``` + +`Workflows$BooleanConditionOutcome.PersistentID` is already declared in +`generated/metamodel/types.go`, so no metamodel change is needed. + +A related risk, **not probed**: the helper's own comment says it mints a *fresh* GUID on every +save. If `create or replace workflow` re-mints the IDs of outcomes that already had one, +re-running a workflow script would make instances `Incompatible` in the same way, for user-task +outcomes too. Ask upstream to keep an existing `PersistentId` when rewriting a unit, and probe +it here before relying on a workflow re-run against a database with live instances. + +### Workaround + +None in MDL. There is no syntax that sets a `PersistentId`. Do not hand-patch the unit either: +that bypasses `exec.sh`'s gate, and the next `create or replace workflow` drops the patch. + +Until upstream ships the fix, plan for it: +- the app's workflow admin page lists **`Incompatible`** instances and gives an admin a way to + handle them (abort, restart, or move them on with a jump-to); +- every screen that routes to an instance's task handles an instance that has no open task + (open the case, not a dead end); +- a go-live plan either drains in-flight instances before each deploy or accepts the admin step. + +Detection-gap register: `skills/learned-detection-gaps.md` (the row that cites BUG-141). + +--- + ## BUG-DRAFT-loop-var-expression-typecheck: the expression type checker is skipped inside a `LOOP` body — the identical expression is caught on a parameter and missed on a loop variable (2026-09-15) > **FILED UPSTREAM 2026-09-15 — https://github.com/mendixlabs/mxcli/issues/1100** diff --git a/bug-logs/pending-github-issues/bug141-condition-outcome-no-persistent-id.md b/bug-logs/pending-github-issues/bug141-condition-outcome-no-persistent-id.md new file mode 100644 index 0000000..5c8e7d7 --- /dev/null +++ b/bug-logs/pending-github-issues/bug141-condition-outcome-no-persistent-id.md @@ -0,0 +1,65 @@ +**Repo:** `mendixlabs/mxcli` +**Source:** `bug-logs/mxcli-bugs.md`, `## BUG-141`. Observed 2026-09-26 on a card-disbursement +requirements-driven build (Mendix 11.13.0, mxcli v0.24.0). +**Status:** NOT YET FILED. Before filing, check the issue tracker for a duplicate and re-read +`conditionOutcomeToGen()` on current `main`. +**Suggested labels:** bug, workflow, silent-corruption + +--- + +**Title:** Workflow condition outcomes are written without a `PersistentId`, so in-flight +instances go `Incompatible` on every deploy + +**Body:** + +## Summary + +`conditionOutcomeToGen()` in `mdl/backend/modelsdk/workflow_write.go` writes +`Workflows$BooleanConditionOutcome`, `Workflows$EnumerationValueConditionOutcome` and +`Workflows$VoidConditionOutcome` without a `PersistentId`. Activities, `UserTaskOutcome` and +`ParallelSplitOutcome` all get one through `addFreshPersistentID(g)`. + +The runtime identifies an outcome by its persistent ID. The 11.13.0 workflow metamodel builds +`ModelBooleanConditionOutcome(id, value, persistentId, flow, container)`. With none stored, the +outcome is a new outcome after every load. So every workflow instance that has already passed a +decision is marked **`Incompatible`** on the next deploy, with +*"A selected outcome has been replaced in the already executed path"*. That includes a restart +with no model change at all. + +## Reproduction + +1. `create workflow` with a call-microflow task that has Boolean or enumeration outcomes (or a + `DECISION`; both reach `conditionOutcomeToGen()`, lines 511 and 556), followed by a user + task. +2. `mxcli run --local`, start an instance, and let it stop at the user task (`InProgress`). +3. Stop the app and start it again without changing the model. +4. The instance is now `Incompatible`. + +Reproduced three times on one model: across a no-change restart, across a redeploy that +changed only unrelated pages, and across a one-line page change. A BSON dump of the workflow +unit shows `PersistentId` on every activity and user-task outcome and on none of the condition +outcomes. + +## Why nothing catches it + +`check`, `exec`, `describe workflow`, `lint`, mxbuild and `mx check` are all clean. Any test that +starts and finishes an instance within one run passes. It only shows when an instance outlives +a restart, which means in production. + +## Suggested fix + +One line in each of the three `case` arms of `conditionOutcomeToGen()`, before `return g`: + +```go +addFreshPersistentID(g) +``` + +`WorkflowsBooleanConditionOutcome.PersistentID` is already in `generated/metamodel/types.go`. + +## A related question + +`addFreshPersistentID`'s comment says it mints a new GUID on every save. If +`create or replace workflow` re-mints the IDs of outcomes and activities that already have +one, re-running a workflow script against a database with live instances would cause the same +`Incompatible` state, for user-task outcomes too. We have not probed that. Could the writer keep +an existing `PersistentId` when it rewrites a unit? diff --git a/skills/learned-detection-gaps.md b/skills/learned-detection-gaps.md index 0730376..e5acdf7 100644 --- a/skills/learned-detection-gaps.md +++ b/skills/learned-detection-gaps.md @@ -42,6 +42,7 @@ verification rungs form a ladder, and a green result only certifies what that ru | **Quoted** association-path / attribute bindings: `attribute: "Assoc/Attr"`, quoted names in `attributes: [...]`, `$Obj/"Attr"` | rungs 1–3 (`DESCRIBE` round-trips the corrupted binding as if fine) | native `mx check`: CE1613 / CE0117 naming the literal quoted string | Association paths and member accesses go **unquoted** — the one place an "always quote identifiers" convention must break. `resolveAssociationAttributePath` doesn't strip quotes before splitting on `/` | a martial-arts-academy PoC project 2026-08-14 + independently a product-provisioning PoC (BUG-75 family) | | `calculated by` on an attribute | rungs 1–4 **and** BSON looks like a normal attribute unless decoded | live retrieve: value always empty; `SHOW CALLERS` of the microflow: none | Don't use it this mxcli version: stored attribute + explicit compute-change-commit. Verify with `mx.data.get` before anything depends on it | BUG-98 | | A **`PARALLEL SPLIT` path with no terminator node** — i.e. every scripted split, since MDL cannot write one | rungs 1–4, and in **both directions**: `check`, `exec`, `DESCRIBE WORKFLOW`, `mxbuild` and native `mx check` report identically on the broken and the repaired model | a **live run**: count the tasks the engine opens at the split. Consecutive *End of parallel split path* rows with no task between them | `bin/wf-add-path-terminators.py --apply` after the script, and after every later script that rewrites the workflow; verify by running it, never by building it | BUG-121, `learned-workflow-patterns.md` §23 | +| A **workflow branch point** in a `create workflow` script: a `DECISION`, or a call-microflow task with Boolean or enumeration outcomes (both go through the same outcome writer) | rungs 1–4, lint, **and rung 6 within one deploy**: every journey that starts and finishes an instance passes | rung 6 **across a restart or redeploy**. An instance left paused after the decision comes back `Incompatible` (*"A selected outcome has been replaced in the already executed path"*), even when the restart changes nothing in the model | mxcli writes condition outcomes with no `PersistentId`, so the runtime gives them a new identity on every load. There is no MDL fix. Once per workflow, probe it: start an instance, let it stop at the first task after a decision, restart the app, read the instance's state. Until upstream fixes it, the app's workflow admin page must list and handle `Incompatible` instances. Note what the ladder cannot express here: the defect is not in any one deploy, it is *between* two | BUG-141 | | `create import mapping` array-to-child binding | rungs 1–4; `DESCRIBE IMPORT MAPPING` looks correct | live retrieve after a real `import from mapping`: child list empty | Unverified-until-proven-live; severity not yet classified — read BUG-99's hold before blaming mxcli | BUG-99 | | Cross-module `ALTER PAGE ... INSERT` of a DG2 column | rungs 1–3 (`DESCRIBE` *omits* the malformed column) | rung 5: Studio Pro loader `InvalidCastException`; `mx check` also crashes | Forbidden construct; recovery is `create or replace page` | BUG-96 | | Expression in `ContentParams:` inside a customContent column | rungs 1–3 (`DESCRIBE` normalises correct and broken forms to the same text) | mxbuild / Studio Pro error pane: CE1613 | Bind with `Attribute:`, never an expression | `learned-datagrid-customcontent-binding.md` | From b2201ca13e3b7635e82f91712993dcaf1bbc8f4d Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 07:58:00 +0000 Subject: [PATCH 13/45] fix(model-stamp): pre-model projects commit; worktree/--git-dir commits are checked Three defects in the model-verification pre-commit path: 1. check ended on find_mpr's "no .mpr found", so a project with no model yet (requirements-driven Stages P-4) could not commit its intake without MODEL_UNVERIFIED_OK=1. check now passes when no .mpr exists at the root or under app/ and no *.mpr / mprcontents/ path is staged. A staged model with no .mpr, and two .mpr files, still refuse. 2. With GIT_DIR exported (git worktree commits; git --git-dir=...), git's rule makes the cwd the work-tree top, so `git -C app` saw app/ as the top, the model pathspecs matched nothing and check --staged passed an unverified model. GIT_WORK_TREE/GIT_DIR/GIT_INDEX_FILE are now pinned to absolute paths before any `git -C`. 3. _common.sh root resolution tiers 2/3 now probe app/*.mpr like find_mpr, so a two-tree checkout with no root .mpr no longer climbs to a stray .mpr in an ancestor directory. Field run: two-tree field project (card-disbursement requirements-driven build), read-only: `model-stamp.sh paths` with GIT_DIR exported returned ".mpr mprcontents" before, "app/.mpr app/mprcontents" after; with a relative GIT_DIR, "fatal: not a git repository" before. Scratch probe (plain / --git-dir / GIT_DIR / worktree commits x single-tree / two-tree): old accepted unverified models via --git-dir and worktree and refused a verified one in a worktree; new refuses every unverified model and accepts the verified one. Pre-model repo: intake commit refused before, accepted after; staged mprcontents with no .mpr refused. Covered by tests/wave2/test-model-stamp.sh (new T9, T10). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Pm41sykqDGcvioGzsY5H2N --- CHANGELOG.md | 1 + project-bin/_common.sh | 13 ++++++--- project-bin/model-stamp.sh | 47 +++++++++++++++++++++++++++++++++ tests/wave2/test-model-stamp.sh | 30 +++++++++++++++++++++ 4 files changed, 87 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a647afa..ab191b2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,7 @@ three commits past it), and a bug report can name a release instead of a sha nob Sections dated before 2026-09-19 predate the cycle and stay as they are. ## Unreleased +- fix(project-bin/model-stamp.sh, _common.sh): **the model-verification pre-commit hook no longer refuses every commit on a project with no model yet, and no longer waves an unverified model through from a `git worktree` or `git --git-dir=…` on a two-tree checkout.** (1) `check` ended on `find_mpr`'s "no .mpr found", so a requirements-driven project could not commit its Stage P intake without `MODEL_UNVERIFIED_OK=1` on every commit (T-1); `check` now passes when no `.mpr` exists at the root or under `app/` and no `*.mpr`/`mprcontents/` path is staged — a staged model with no `.mpr`, and two `.mpr` files, still refuse. (2) With `GIT_DIR` exported (a worktree commit exports the absolute `.git/worktrees/`; `--git-dir` a relative one), `git -C app` treated `app/` as the work-tree top, the pathspecs matched nothing, and `check --staged` reported "no model files staged" — the script now pins `GIT_WORK_TREE`/`GIT_DIR`/`GIT_INDEX_FILE` to absolute paths first. (3) `_common.sh`'s root resolution (tiers 2/3) now probes `app/*.mpr` like `find_mpr`, so a two-tree checkout with no root `.mpr` stops at the project instead of climbing to the first stray `.mpr` in an ancestor directory. Field run: on the two-tree field project, `model-stamp.sh paths` with `GIT_DIR` exported returned `.mpr mprcontents` (old) vs `app/.mpr app/mprcontents` (new); a scratch probe of plain / `--git-dir` / `GIT_DIR` / worktree commits, single-tree and two-tree, refuses every unverified model and accepts the verified one (old: `--git-dir` and worktree accepted unverified, worktree refused verified); a pre-model repo commits its intake (old: refused). `tests/wave2/test-model-stamp.sh` gains T9/T10. From a card-disbursement requirements-driven build (FINDINGS T-1). - new(bug-logs): **BUG-141, workflow condition outcomes are written without a `PersistentId`, so every instance paused after a branch point turns `Incompatible` on the next deploy — including a restart with no model change.** mxcli's `conditionOutcomeToGen()` gives no persistent ID to Boolean, enumeration or void outcomes (a `DECISION`, or a call-microflow task with outcomes). Activities and user-task outcomes do get one. The runtime therefore re-identifies those outcomes on every load, and any instance that already took one fails with "A selected outcome has been replaced in the already executed path". Every static rung is clean, and so is every journey that stays inside one deploy. Field evidence (mxcli v0.24.0, still on upstream `main` 9509176; Mendix 11.13.0): three live reproductions (a no-change restart, an unrelated-page redeploy, a one-line page redeploy), a BSON dump showing the ID missing only on condition outcomes, and the runtime metamodel's `ModelBooleanConditionOutcome(…, persistentId, …)` constructor. Upstream fix: one `addFreshPersistentID(g)` per case. The paste-ready issue is `bug-logs/pending-github-issues/bug141-condition-outcome-no-persistent-id.md`, not yet filed. No MDL workaround: the app's workflow admin must handle `Incompatible` instances until upstream ships it. From a card-disbursement requirements-driven build (build-plan row 6.12). - learn(skills/learned-detection-gaps.md): **new row: a workflow branch point is green on rungs 1–4, lint and every live run inside one deploy, and is caught only by a live run across a restart.** The row gives the one-instance restart probe, and notes that the defect sits between two deploys, which no single rung can see (BUG-141). From a card-disbursement requirements-driven build. - fix(project-tests/e2e/helpers.js): **`logout()` now actually ends the session on Mendix 11.** It read the CSRF token from `mx.session.getCSRFToken()`, which Mendix 11 does not have, so it posted the xas `logout` action with an empty token. The server still answers 200 `{}` to that and keeps the session, so every journey (the runner calls `H.logout` at the end of a walk) and every probe that "logged out" held a session until the timeout. On a trial licence the session cap then refuses the next sign-in ("Maximum number of sessions exceeded"), which reads as a broken login rather than a leak. The token now falls back to `mx.session.sessionData.csrftoken`, and a missing token returns `false` instead of posting. Field run (mxcli v0.24.0, Mendix 11.13.0): 13 sessions had leaked from runs that all "logged out"; after the fix, `System.Session` per role read 1 before sign-in, 2 while signed in and 1 after logout for three roles, where before it grew by one per run. No new fixture: the call runs in the browser, and the session count above is its red/green. New inbox note `contrib/inbox/2026-09-26-review-report-multi-ledger-coverage.md`: `review-report.js` cannot parse `coverage-check-all.sh`'s one-line-per-BRD output, so every multi-ledger module carries a permanent coverage FAULT over a clean count. From a card-disbursement requirements-driven build (build-plan row 5.7). diff --git a/project-bin/_common.sh b/project-bin/_common.sh index bd8b61c..f2ed858 100755 --- a/project-bin/_common.sh +++ b/project-bin/_common.sh @@ -29,14 +29,19 @@ _mxtk_resolve_root() { local self_parent d self_parent=$(cd "$(dirname "${BASH_SOURCE[2]:-${BASH_SOURCE[1]:-${BASH_SOURCE[0]}}}")/.." 2>/dev/null && pwd) - # tier 2: the script sits in a real project's bin/ - if [ -n "$self_parent" ] && ls "$self_parent"/*.mpr >/dev/null 2>&1; then + # tier 2: the script sits in a real project's bin/ (model at the root, or under app/ on a + # two-tree checkout — the same one-level probe find_mpr makes) + if [ -n "$self_parent" ] && { ls "$self_parent"/*.mpr >/dev/null 2>&1 || ls "$self_parent"/app/*.mpr >/dev/null 2>&1; }; then printf '%s\n' "$self_parent"; return 0 fi - # tier 3: walk up from $PWD looking for the .mpr + # tier 3: walk up from $PWD looking for the .mpr (root or app/). + # WHY app/ (2026-09-26, card-disbursement requirements-driven build): on a two-tree checkout + # with no root .mpr, tiers 2 and 3 used to find nothing AT the project and kept climbing — and + # the first stray .mpr in any ancestor directory became PROJECT_ROOT. A scratch copy under a + # folder holding one unrelated .mpr pointed model-stamp.sh at that file and its hook passed. d=$(pwd) while [ "$d" != "/" ]; do - if ls "$d"/*.mpr >/dev/null 2>&1; then printf '%s\n' "$d"; return 0; fi + if ls "$d"/*.mpr >/dev/null 2>&1 || ls "$d"/app/*.mpr >/dev/null 2>&1; then printf '%s\n' "$d"; return 0; fi d=$(dirname "$d") done # nothing found: keep the old behaviour so the caller's own error is the one seen diff --git a/project-bin/model-stamp.sh b/project-bin/model-stamp.sh index 3636c1f..e002788 100755 --- a/project-bin/model-stamp.sh +++ b/project-bin/model-stamp.sh @@ -48,6 +48,53 @@ _real() { # resolve symlinks by hand (GNU-only resolvers are absent on older ma printf '%s/%s\n' "$(cd "$(dirname "$p")" && pwd -P)" "$(basename "$p")" } +# Pin git's view of the repository before any `git -C "$MODEL_DIR"` below. +# +# WHY (2026-09-26, card-disbursement requirements-driven build). A commit made from a +# `git worktree` runs this hook with GIT_DIR exported (the absolute .git/worktrees/), +# and so does `git --git-dir=… commit` (a RELATIVE GIT_DIR). Git's rule for GIT_DIR without +# GIT_WORK_TREE is "the current directory is the top of the work tree" — so on a two-tree +# checkout `git -C app …` saw app/ as the top with an empty prefix, the pathspecs +# `App.mpr mprcontents` matched nothing in an index that holds `app/App.mpr`, and +# `check --staged` said "no model files staged" and passed a model nobody had verified +# (a relative GIT_DIR instead made `git -C app` fail, `in_git` false, exit 0: same pass). +# Resolving the top once from the caller's cwd and exporting absolute paths makes every +# later `git -C` agree with the git that invoked the hook. +_mxtk_abs() { case "$1" in /*) printf '%s\n' "$1" ;; *) printf '%s/%s\n' "$(pwd -P)" "$1" ;; esac; } +case "${GIT_INDEX_FILE:-}" in ''|/*) ;; *) GIT_INDEX_FILE="$(_mxtk_abs "$GIT_INDEX_FILE")"; export GIT_INDEX_FILE ;; esac +if [ -n "${GIT_DIR:-}" ]; then + _gtop="$(git rev-parse --show-toplevel 2>/dev/null || true)" + GIT_DIR="$(_mxtk_abs "$GIT_DIR")"; export GIT_DIR + if [ -n "${GIT_WORK_TREE:-}" ]; then GIT_WORK_TREE="$(_mxtk_abs "$GIT_WORK_TREE")"; export GIT_WORK_TREE + elif [ -n "$_gtop" ]; then GIT_WORK_TREE="$_gtop"; export GIT_WORK_TREE; fi +fi + +# A project with no model yet (requirements-driven Stages P-4) has nothing to verify. +# +# WHY (2026-09-25, same build). `find_mpr` failing used to end every command here, `check` +# included — so the pre-commit hook refused the Stage P intake commit with "no .mpr found" +# although nothing model-related was staged, and each pre-model commit needed the override. +# `check` now passes when no .mpr exists at the root or under app/ AND no model path +# (*.mpr, mprcontents/) is staged; a staged model that no .mpr resolves still refuses, and +# two .mpr files still refuse (find_mpr's "refusing to guess"). Every other command still +# needs a model: a stamp or fingerprint over no model would be a constant. +_mxtk_any_mpr() { local f; for f in "$PROJECT_ROOT"/*.mpr "$PROJECT_ROOT"/app/*.mpr; do [ -e "$f" ] && return 0; done; return 1; } +if [ "${1:-}" = "check" ] && [ -z "${MPR_FILE:-}" ] && ! _mxtk_any_mpr; then + _q=0; for a in "$@"; do case "$a" in -q|--quiet) _q=1 ;; esac; done + _staged="" + if git -C "$PROJECT_ROOT" rev-parse --git-dir >/dev/null 2>&1; then + _staged="$(git -C "$PROJECT_ROOT" diff --cached --name-only --diff-filter=d 2>/dev/null \ + | grep -E '(^|/)[^/]+\.mpr$|(^|/)mprcontents/' || true)" + fi + if [ -z "$_staged" ]; then + [ "$_q" = 1 ] || echo " no model in this project yet (no .mpr at the root or under app/) — nothing to verify" + exit 0 + fi + echo " ✗ model files are staged, but no .mpr resolves under $PROJECT_ROOT (root or app/):" >&2 + printf '%s\n' "$_staged" | head -5 | sed 's/^/ /' >&2 + exit 1 +fi + MPR="$(find_mpr)" || exit 1 MPR="$(_real "$MPR")" MODEL_DIR="$(dirname "$MPR")" diff --git a/tests/wave2/test-model-stamp.sh b/tests/wave2/test-model-stamp.sh index a97e6a8..fca0e6c 100755 --- a/tests/wave2/test-model-stamp.sh +++ b/tests/wave2/test-model-stamp.sh @@ -83,5 +83,35 @@ before="$(cat .claude/settings.json)"; "$TK/bin/install-claude-permissions.sh" " "$TK/bin/install-claude-permissions.sh" "$P" --uninstall >/dev/null "$py" -c "import json,sys;d=json.load(open('.claude/settings.json'));cmds=[h['command'] for g in d['hooks']['SessionStart'] for h in g['hooks']];sys.exit(0 if d['permissions'].get('deny',[])==[] and cmds==['sh .claude/bootstrap-mxcli.sh || true'] and d['permissions']['allow'][0]=='Bash(./mxcli:*)' else 1)" && ok "uninstall removes only what it added" || bad "uninstall wrong: $(cat .claude/settings.json)" +echo "T9 two-tree with NO root symlink: --git-dir and git-worktree commits are checked, not waved through" +# 2026-09-26: with GIT_DIR exported (a worktree commit, or `git --git-dir=…`) `git -C app` saw +# app/ as the work-tree top, the pathspecs matched nothing, and check --staged passed. +Q="$WORK/q"; mkdir -p "$Q/bin" "$Q/app/mprcontents" +( cd "$Q" && git init -q . && git config user.email t@t && git config user.name t + cp "$TK/project-bin/_common.sh" "$TK/project-bin/model-stamp.sh" "$TK/project-bin/install-project-hooks.sh" bin/ + printf mpr > app/App.mpr; printf u1 > app/mprcontents/a.mxunit; printf '/.claude/\n' > .gitignore + ./bin/install-project-hooks.sh >/dev/null + git add -A >/dev/null; MODEL_UNVERIFIED_OK=1 git commit -qm init 2>/dev/null + git worktree add -q "$WORK/q-wt" 2>/dev/null ) +cd "$Q" && printf u1b > app/mprcontents/a.mxunit && git add app +git --git-dir=.git commit -qm x 2>/dev/null && bad "--git-dir commit waved an unverified model through" || ok "--git-dir commit refused the unverified model" +git reset -q +cd "$WORK/q-wt" && printf u1c > app/mprcontents/a.mxunit && git add app +git commit -qm x 2>/dev/null && bad "worktree commit waved an unverified model through" || ok "worktree commit refused the unverified model" +./bin/model-stamp.sh write pass "fixture" >/dev/null +git commit -qm x 2>/dev/null && ok "worktree commit of the verified model accepted" || bad "worktree refused its verified model" + +echo "T10 no model yet (requirements-driven Stages P-4): non-model commits pass; a stray staged model does not" +R="$WORK/r"; mkdir -p "$R/bin" +cd "$R" && git init -q . && git config user.email t@t && git config user.name t +cp "$TK/project-bin/_common.sh" "$TK/project-bin/model-stamp.sh" "$TK/project-bin/install-project-hooks.sh" bin/ +PROJECT_ROOT="$R" ./bin/install-project-hooks.sh >/dev/null +echo intake > intake.md && git add intake.md bin +PROJECT_ROOT="$R" git commit -qm intake 2>/dev/null && ok "pre-model intake commit accepted" || bad "pre-model intake commit refused (no .mpr found)" +out="$(PROJECT_ROOT="$R" ./bin/model-stamp.sh check 2>&1)"; rc=$? +[ "$rc" -eq 0 ] && echo "$out" | grep -q "no model in this project yet" && ok "check says there is no model yet" || bad "check rc=$rc" "$out" +mkdir -p app/mprcontents && printf u > app/mprcontents/x.mxunit && git add app +PROJECT_ROOT="$R" git commit -qm stray 2>/dev/null && bad "staged mprcontents with no .mpr committed" || ok "staged model with no .mpr refused" + echo; echo "model-stamp: $PASS passed, $FAIL failed" [ "$FAIL" -eq 0 ] From 4dd6ed45b580a6ef5b7c826e43393a5744de95cc Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 07:59:45 +0000 Subject: [PATCH 14/45] fix(closeout): rows the entry mode does not owe are N/A, not "still missing" The three manifest loops in bin/lib/closeout.sh (artifacts produced, the plain-words missing list, next-stage opt-in offers) ignored the modes column that artifact-check.sh's second pass honours. A requirements-driven Stage 0 PASS therefore said "still missing for this stage: app-report", a row only existing-app-change owes. A shared _co_owed test now applies the same rule (unknown mode owes every row). Field run (card-disbursement requirements-driven build, register + intake + triage copied to scratch): mxtk_plain_verdict 0 printed "still missing for this stage: app-report" before, "Nothing is holding it." after; the Stage 0 close-out listed app-report as PENDING before, N/A after. Covered by tests/wave2/test-closeout.sh (new T9). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Pm41sykqDGcvioGzsY5H2N --- CHANGELOG.md | 1 + bin/lib/closeout.sh | 22 ++++++++++++++++++++-- tests/wave2/test-closeout.sh | 6 ++++++ 3 files changed, 27 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ab191b2..6e2f96a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,7 @@ three commits past it), and a bug report can name a release instead of a sha nob Sections dated before 2026-09-19 predate the cycle and stay as they are. ## Unreleased +- fix(bin/lib/closeout.sh): **the close-out block and the "In plain words" paragraph now honour the manifest's entry-mode column, as `artifact-check.sh` already did.** A requirements-driven Stage 0 PASS said "still missing for this stage: app-report" and the close-out listed `app-report` as ⬜ PENDING with its producer, though that row is owed only by existing-app-change and the artifact check on the same run called it N/A. The artifacts list now shows such a row as `⏭ N/A: entry mode does not owe it`; the missing list and the next stage's opt-in offers skip it; an unknown mode still owes every row. Field run: the field project's register, intake and triage copied to scratch — `mxtk_plain_verdict 0` printed "still missing for this stage: app-report" before, "Nothing is holding it." after. `tests/wave2/test-closeout.sh` gains T9. From a card-disbursement requirements-driven build (FINDINGS T-2). - fix(project-bin/model-stamp.sh, _common.sh): **the model-verification pre-commit hook no longer refuses every commit on a project with no model yet, and no longer waves an unverified model through from a `git worktree` or `git --git-dir=…` on a two-tree checkout.** (1) `check` ended on `find_mpr`'s "no .mpr found", so a requirements-driven project could not commit its Stage P intake without `MODEL_UNVERIFIED_OK=1` on every commit (T-1); `check` now passes when no `.mpr` exists at the root or under `app/` and no `*.mpr`/`mprcontents/` path is staged — a staged model with no `.mpr`, and two `.mpr` files, still refuse. (2) With `GIT_DIR` exported (a worktree commit exports the absolute `.git/worktrees/`; `--git-dir` a relative one), `git -C app` treated `app/` as the work-tree top, the pathspecs matched nothing, and `check --staged` reported "no model files staged" — the script now pins `GIT_WORK_TREE`/`GIT_DIR`/`GIT_INDEX_FILE` to absolute paths first. (3) `_common.sh`'s root resolution (tiers 2/3) now probes `app/*.mpr` like `find_mpr`, so a two-tree checkout with no root `.mpr` stops at the project instead of climbing to the first stray `.mpr` in an ancestor directory. Field run: on the two-tree field project, `model-stamp.sh paths` with `GIT_DIR` exported returned `.mpr mprcontents` (old) vs `app/.mpr app/mprcontents` (new); a scratch probe of plain / `--git-dir` / `GIT_DIR` / worktree commits, single-tree and two-tree, refuses every unverified model and accepts the verified one (old: `--git-dir` and worktree accepted unverified, worktree refused verified); a pre-model repo commits its intake (old: refused). `tests/wave2/test-model-stamp.sh` gains T9/T10. From a card-disbursement requirements-driven build (FINDINGS T-1). - new(bug-logs): **BUG-141, workflow condition outcomes are written without a `PersistentId`, so every instance paused after a branch point turns `Incompatible` on the next deploy — including a restart with no model change.** mxcli's `conditionOutcomeToGen()` gives no persistent ID to Boolean, enumeration or void outcomes (a `DECISION`, or a call-microflow task with outcomes). Activities and user-task outcomes do get one. The runtime therefore re-identifies those outcomes on every load, and any instance that already took one fails with "A selected outcome has been replaced in the already executed path". Every static rung is clean, and so is every journey that stays inside one deploy. Field evidence (mxcli v0.24.0, still on upstream `main` 9509176; Mendix 11.13.0): three live reproductions (a no-change restart, an unrelated-page redeploy, a one-line page redeploy), a BSON dump showing the ID missing only on condition outcomes, and the runtime metamodel's `ModelBooleanConditionOutcome(…, persistentId, …)` constructor. Upstream fix: one `addFreshPersistentID(g)` per case. The paste-ready issue is `bug-logs/pending-github-issues/bug141-condition-outcome-no-persistent-id.md`, not yet filed. No MDL workaround: the app's workflow admin must handle `Incompatible` instances until upstream ships it. From a card-disbursement requirements-driven build (build-plan row 6.12). - learn(skills/learned-detection-gaps.md): **new row: a workflow branch point is green on rungs 1–4, lint and every live run inside one deploy, and is caught only by a live run across a restart.** The row gives the one-instance restart probe, and notes that the defect sits between two deploys, which no single rung can see (BUG-141). From a card-disbursement requirements-driven build. diff --git a/bin/lib/closeout.sh b/bin/lib/closeout.sh index 84c4b6e..9d1399e 100755 --- a/bin/lib/closeout.sh +++ b/bin/lib/closeout.sh @@ -201,6 +201,16 @@ _co_plain_gate_needs() { *) echo "" ;; esac } +# _co_owed — does this project's entry mode owe the row? The same +# test as artifact-check.sh's second pass, and the same rule: an unknown mode (young register, no +# `Entry mode:` line yet) owes every row — louder, never quieter. +# +# WHY (2026-09-25, card-disbursement requirements-driven build). The three manifest loops below +# ignored the modes column, so a requirements-driven Stage 0 PASS still said "still missing for +# this stage: app-report" — a row only existing-app-change owes, which artifact-check.sh itself +# reported as N/A on the same run. +_co_owed() { [ -z "$2" ] && return 0; case ",$1," in *",$2,"*) return 0 ;; esac; return 1; } + # mxtk_plain_verdict [register] # The "In plain words" paragraph under a gate line: status first, then every reason the # technical text carries that has a known plain translation, then what is still missing on @@ -244,11 +254,13 @@ mxtk_plain_verdict() { case "$low" in *"unanswered"*|*"no blanks"*|*"blank"*) _co_say "a kickoff question is still blank" ;; esac # Missing mandatory artifacts for this stage, in plain names. - local id stage paths producer consumers modes absence miss="" + local id stage paths producer consumers modes absence miss="" mode + mode="$(_art_entry_mode "$reg" 2>/dev/null || true)" if [ -f "$ARTIFACT_TSV" ]; then while IFS=$'\t' read -r id stage paths producer consumers modes absence; do case "$id" in ''|'#'*|artifact) continue ;; esac [ "$stage" = "$st" ] && [ "$absence" != "optin" ] || continue + _co_owed "$modes" "$mode" || continue _art_waiver "$reg" "$id" "$stage" >/dev/null 2>&1 && continue _art_find "$root" "$paths" [ -z "$ART_HIT" ] && miss="$miss${miss:+; }$(_co_plain_artifact "$id")" @@ -274,11 +286,16 @@ mxtk_closeout_report() { # ── Artifacts produced ────────────────────────────────────────────────── printf '**Artifacts produced** (rows of `bin/lib/artifact-manifest.tsv` for stage %s; presence only, never quality)\n' "$st" - local id stage paths producer consumers modes absence reason n_art=0 optin + local id stage paths producer consumers modes absence reason n_art=0 optin mode + mode="$(_art_entry_mode "$reg" 2>/dev/null || true)" if [ -f "$ARTIFACT_TSV" ]; then while IFS=$'\t' read -r id stage paths producer consumers modes absence; do case "$id" in ''|'#'*|artifact) continue ;; esac [ "$stage" = "$st" ] || continue + if ! _co_owed "$modes" "$mode"; then + [ "$absence" = "optin" ] || printf -- '- ⏭ `%s` — N/A: entry mode %s does not owe it\n' "$id" "$mode" + continue + fi optin="" if [ "$absence" = "optin" ]; then optin="$(_art_field "$reg" "opt-in artifact $id" 2>/dev/null || true)" @@ -418,6 +435,7 @@ mxtk_closeout_report() { while IFS=$'\t' read -r id stage paths producer consumers modes absence; do case "$id" in ''|'#'*|artifact) continue ;; esac [ "$stage" = "$next" ] && [ "$absence" = "optin" ] || continue + _co_owed "$modes" "$mode" || continue n_opt=$((n_opt+1)) optin="$(_art_field "$reg" "opt-in artifact $id" 2>/dev/null || true)" if [ -n "$optin" ]; then diff --git a/tests/wave2/test-closeout.sh b/tests/wave2/test-closeout.sh index 2532517..ef15ff2 100755 --- a/tests/wave2/test-closeout.sh +++ b/tests/wave2/test-closeout.sh @@ -119,6 +119,12 @@ echo "== T8: --closeout without a stage refuses ==" "$GATE" --closeout "$P" >/dev/null 2>&1; RC=$? [ "$RC" -eq 2 ] && ok "exit 2" || bad "exit $RC" +echo "== T9: a row the entry mode does not owe is N/A, never 'still missing' (2026-09-25) ==" +P="$(mkproj t9)" +OUT="$("$GATE" --closeout "$P" 0 2>&1)" +case "$OUT" in *"⏭ \`app-report\` — N/A: entry mode requirements does not owe it"*) ok "app-report N/A on a requirements-driven register" ;; *) bad "app-report not N/A: $(printf '%s' "$OUT" | grep app-report)" ;; esac +if printf '%s\n' "$OUT" | grep 'still missing for this stage' | grep -q 'app-report'; then bad "plain words list app-report as missing"; else ok "plain words do not list a row this mode does not owe"; fi + rm -rf "$WORK" echo "passed $PASS, failed $FAIL" [ "$FAIL" -eq 0 ] From 7c2fae25d74eb5f9e7cc7f1dba14e3094fbfa69e Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 08:00:42 +0000 Subject: [PATCH 15/45] fix(check-root-clean): allow intake.md, triage.md and mxcli's FINDINGS.md The default allowlist omitted intake.md (init-project.sh scaffolds it at the root) and triage.md (artifact-manifest.tsv owes it at that exact path), so every scaffolded project failed the check from Stage 0 on. mxcli init's generated CLAUDE.md also tells agents to "Append to FINDINGS.md" at the root, which this check refused - two instructions in one CLAUDE.md that contradicted each other. Field run (card-disbursement requirements-driven build, read-only on the project root): "Stray docs in project root: intake.md triage.md", exit 1, before; "Project root clean", exit 0, after. No fixture covers this script. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Pm41sykqDGcvioGzsY5H2N --- CHANGELOG.md | 1 + project-bin/check-root-clean.sh | 9 ++++++++- 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6e2f96a..4175dc6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,7 @@ three commits past it), and a bug report can name a release instead of a sha nob Sections dated before 2026-09-19 predate the cycle and stay as they are. ## Unreleased +- fix(project-bin/check-root-clean.sh): **the root-clean check no longer calls the toolkit's own root artifacts strays.** Its default allowlist omitted `intake.md` (scaffolded at the root by `init-project.sh`) and `triage.md` (owed at exactly that path by `artifact-manifest.tsv`), so every scaffolded project failed it from Stage 0 on; and `mxcli init`'s generated CLAUDE.md tells agents to "Append to `FINDINGS.md`" at the root while this check refused that file. All three are now on the default allowlist. Field run (read-only, the field project root): `✗ Stray docs in project root: intake.md triage.md` exit 1 before, `✓ Project root clean` after. No fixture covers this script. From a card-disbursement requirements-driven build (FINDINGS T-3). - fix(bin/lib/closeout.sh): **the close-out block and the "In plain words" paragraph now honour the manifest's entry-mode column, as `artifact-check.sh` already did.** A requirements-driven Stage 0 PASS said "still missing for this stage: app-report" and the close-out listed `app-report` as ⬜ PENDING with its producer, though that row is owed only by existing-app-change and the artifact check on the same run called it N/A. The artifacts list now shows such a row as `⏭ N/A: entry mode does not owe it`; the missing list and the next stage's opt-in offers skip it; an unknown mode still owes every row. Field run: the field project's register, intake and triage copied to scratch — `mxtk_plain_verdict 0` printed "still missing for this stage: app-report" before, "Nothing is holding it." after. `tests/wave2/test-closeout.sh` gains T9. From a card-disbursement requirements-driven build (FINDINGS T-2). - fix(project-bin/model-stamp.sh, _common.sh): **the model-verification pre-commit hook no longer refuses every commit on a project with no model yet, and no longer waves an unverified model through from a `git worktree` or `git --git-dir=…` on a two-tree checkout.** (1) `check` ended on `find_mpr`'s "no .mpr found", so a requirements-driven project could not commit its Stage P intake without `MODEL_UNVERIFIED_OK=1` on every commit (T-1); `check` now passes when no `.mpr` exists at the root or under `app/` and no `*.mpr`/`mprcontents/` path is staged — a staged model with no `.mpr`, and two `.mpr` files, still refuse. (2) With `GIT_DIR` exported (a worktree commit exports the absolute `.git/worktrees/`; `--git-dir` a relative one), `git -C app` treated `app/` as the work-tree top, the pathspecs matched nothing, and `check --staged` reported "no model files staged" — the script now pins `GIT_WORK_TREE`/`GIT_DIR`/`GIT_INDEX_FILE` to absolute paths first. (3) `_common.sh`'s root resolution (tiers 2/3) now probes `app/*.mpr` like `find_mpr`, so a two-tree checkout with no root `.mpr` stops at the project instead of climbing to the first stray `.mpr` in an ancestor directory. Field run: on the two-tree field project, `model-stamp.sh paths` with `GIT_DIR` exported returned `.mpr mprcontents` (old) vs `app/.mpr app/mprcontents` (new); a scratch probe of plain / `--git-dir` / `GIT_DIR` / worktree commits, single-tree and two-tree, refuses every unverified model and accepts the verified one (old: `--git-dir` and worktree accepted unverified, worktree refused verified); a pre-model repo commits its intake (old: refused). `tests/wave2/test-model-stamp.sh` gains T9/T10. From a card-disbursement requirements-driven build (FINDINGS T-1). - new(bug-logs): **BUG-141, workflow condition outcomes are written without a `PersistentId`, so every instance paused after a branch point turns `Incompatible` on the next deploy — including a restart with no model change.** mxcli's `conditionOutcomeToGen()` gives no persistent ID to Boolean, enumeration or void outcomes (a `DECISION`, or a call-microflow task with outcomes). Activities and user-task outcomes do get one. The runtime therefore re-identifies those outcomes on every load, and any instance that already took one fails with "A selected outcome has been replaced in the already executed path". Every static rung is clean, and so is every journey that stays inside one deploy. Field evidence (mxcli v0.24.0, still on upstream `main` 9509176; Mendix 11.13.0): three live reproductions (a no-change restart, an unrelated-page redeploy, a one-line page redeploy), a BSON dump showing the ID missing only on condition outcomes, and the runtime metamodel's `ModelBooleanConditionOutcome(…, persistentId, …)` constructor. Upstream fix: one `addFreshPersistentID(g)` per case. The paste-ready issue is `bug-logs/pending-github-issues/bug141-condition-outcome-no-persistent-id.md`, not yet filed. No MDL workaround: the app's workflow admin must handle `Incompatible` instances until upstream ships it. From a card-disbursement requirements-driven build (build-plan row 6.12). diff --git a/project-bin/check-root-clean.sh b/project-bin/check-root-clean.sh index 8ce3d50..40e9fde 100755 --- a/project-bin/check-root-clean.sh +++ b/project-bin/check-root-clean.sh @@ -24,7 +24,14 @@ cd "$PROJECT_ROOT" || { echo "cannot cd to project root" >&2; exit 1; } # # AGENTS.md is allowed as a POINTER to CLAUDE.md, never as a copy: two copies of the instructions # drift, and the one the agent reads is whichever its harness happens to prefer. -ALLOWED="${ROOT_ALLOWED:-AGENTS.md CLAUDE.md CLAUDE.local.md PROJECT.md README.md index.html} ${ROOT_ALLOWED_EXTRA:-}" +# +# intake.md and triage.md are allowed because the toolkit itself puts them there: init-project.sh +# scaffolds intake.md at the root, and bin/lib/artifact-manifest.tsv (read by gate-check.sh) owes +# both at exactly those root paths. FINDINGS.md is allowed because `mxcli init`'s generated +# CLAUDE.md tells every agent to "Append to FINDINGS.md" — a root path. (2026-09-25, card- +# disbursement requirements-driven build: this check reported the project's own Stage P/0 +# artifacts as strays, and contradicted the mxcli instruction sitting in the same CLAUDE.md.) +ALLOWED="${ROOT_ALLOWED:-AGENTS.md CLAUDE.md CLAUDE.local.md PROJECT.md README.md index.html intake.md triage.md FINDINGS.md} ${ROOT_ALLOWED_EXTRA:-}" STRAY="" for f in *.md *.html; do From 144b0afb64d467dc968ac798c2dcec40b555ebb4 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 08:01:30 +0000 Subject: [PATCH 16/45] learn(extractor-quality-loop): non-code sources declare their own dimensions The six scored dimensions are code-shaped and the skill never said a stack may replace them. A process-model source (EA XMI/BPEL) has no fields, FKs, enums, routes or specs; scoring those gives 0/0 = 100 and pads the average. New subsection: declare the stack's dimensions in the validator header and as the extraction-quality.json dimension keys; the source-derived expected counts, the 95% gate, the Suspicious-Zero Gate and the self-graded rule stay. From the card-disbursement requirements-driven build (its project-local EA XMI/BPEL validator scored six process-model dimensions). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Pm41sykqDGcvioGzsY5H2N --- CHANGELOG.md | 1 + skills/extractor-quality-loop.md | 16 ++++++++++++++++ 2 files changed, 17 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4175dc6..51277b5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,7 @@ three commits past it), and a bug report can name a release instead of a sha nob Sections dated before 2026-09-19 predate the cycle and stay as they are. ## Unreleased +- learn(skills/extractor-quality-loop.md): **a source that is not code-shaped gets its own scored dimensions; the contract stays.** The six dimensions are code-shaped, and the skill never said they may be replaced, so a process-model source either scored code dimensions it does not have (0/0 = 100, padding the average) or went off-contract. New subsection: declare the stack's dimensions in the validator header and as the `dimensions` keys of `extraction-quality.json`; expected counts come from the source, and the 95% gate, the Suspicious-Zero Gate and the self-graded rule are unchanged. Worked case: an EA XMI/BPEL process model scored on nodes, transitions, guards, documentation, diagram membership and diagram-image label recall. From a card-disbursement requirements-driven build (FINDINGS T-4). - fix(project-bin/check-root-clean.sh): **the root-clean check no longer calls the toolkit's own root artifacts strays.** Its default allowlist omitted `intake.md` (scaffolded at the root by `init-project.sh`) and `triage.md` (owed at exactly that path by `artifact-manifest.tsv`), so every scaffolded project failed it from Stage 0 on; and `mxcli init`'s generated CLAUDE.md tells agents to "Append to `FINDINGS.md`" at the root while this check refused that file. All three are now on the default allowlist. Field run (read-only, the field project root): `✗ Stray docs in project root: intake.md triage.md` exit 1 before, `✓ Project root clean` after. No fixture covers this script. From a card-disbursement requirements-driven build (FINDINGS T-3). - fix(bin/lib/closeout.sh): **the close-out block and the "In plain words" paragraph now honour the manifest's entry-mode column, as `artifact-check.sh` already did.** A requirements-driven Stage 0 PASS said "still missing for this stage: app-report" and the close-out listed `app-report` as ⬜ PENDING with its producer, though that row is owed only by existing-app-change and the artifact check on the same run called it N/A. The artifacts list now shows such a row as `⏭ N/A: entry mode does not owe it`; the missing list and the next stage's opt-in offers skip it; an unknown mode still owes every row. Field run: the field project's register, intake and triage copied to scratch — `mxtk_plain_verdict 0` printed "still missing for this stage: app-report" before, "Nothing is holding it." after. `tests/wave2/test-closeout.sh` gains T9. From a card-disbursement requirements-driven build (FINDINGS T-2). - fix(project-bin/model-stamp.sh, _common.sh): **the model-verification pre-commit hook no longer refuses every commit on a project with no model yet, and no longer waves an unverified model through from a `git worktree` or `git --git-dir=…` on a two-tree checkout.** (1) `check` ended on `find_mpr`'s "no .mpr found", so a requirements-driven project could not commit its Stage P intake without `MODEL_UNVERIFIED_OK=1` on every commit (T-1); `check` now passes when no `.mpr` exists at the root or under `app/` and no `*.mpr`/`mprcontents/` path is staged — a staged model with no `.mpr`, and two `.mpr` files, still refuse. (2) With `GIT_DIR` exported (a worktree commit exports the absolute `.git/worktrees/`; `--git-dir` a relative one), `git -C app` treated `app/` as the work-tree top, the pathspecs matched nothing, and `check --staged` reported "no model files staged" — the script now pins `GIT_WORK_TREE`/`GIT_DIR`/`GIT_INDEX_FILE` to absolute paths first. (3) `_common.sh`'s root resolution (tiers 2/3) now probes `app/*.mpr` like `find_mpr`, so a two-tree checkout with no root `.mpr` stops at the project instead of climbing to the first stray `.mpr` in an ancestor directory. Field run: on the two-tree field project, `model-stamp.sh paths` with `GIT_DIR` exported returned `.mpr mprcontents` (old) vs `app/.mpr app/mprcontents` (new); a scratch probe of plain / `--git-dir` / `GIT_DIR` / worktree commits, single-tree and two-tree, refuses every unverified model and accepts the verified one (old: `--git-dir` and worktree accepted unverified, worktree refused verified); a pre-model repo commits its intake (old: refused). `tests/wave2/test-model-stamp.sh` gains T9/T10. From a card-disbursement requirements-driven build (FINDINGS T-1). diff --git a/skills/extractor-quality-loop.md b/skills/extractor-quality-loop.md index d50cc5f..fea175d 100644 --- a/skills/extractor-quality-loop.md +++ b/skills/extractor-quality-loop.md @@ -119,6 +119,22 @@ non-standard auth/middleware files that define routes) appears in `inventory.end **Overall score = sum(dimension_score × weight) / sum(weights)** +### A source that is not code-shaped: replace the dimensions, keep the contract + +The six above are code-shaped (fields, FKs, enums, routes, specs). A source with none of +those — a process model, a form catalogue, a spreadsheet of rules — gets **its own** +dimensions, declared in the validator header and used as the `dimensions` keys of +`extraction-quality.json`. Scoring a code dimension the stack does not have is the wrong +move: 0 found / 0 expected scores 100 by arithmetic and pads the average (the +Suspicious-Zero Gate below exists for exactly that). + +Field case (a card-disbursement requirements-driven build, 2026-09): an Enterprise +Architect XMI/BPEL process model was scored on **nodes, transitions, guards, documentation, +diagram membership, and diagram-image label recall** — each a count of items in the source +against items captured. What stays fixed whatever the dimensions are: every expected count +comes from the source (never from the inventory), 0–100 per dimension, weights stated, +the ≥ 95% gate, the Suspicious-Zero Gate, and the self-graded rule below. + ## What Is Explicitly NOT Scored The following are intentionally excluded — they belong in BRDs and human review, not the From faf79c60aaffa5daf36e306c97b13e16f0625b2f Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 27 Sep 2026 08:02:10 +0000 Subject: [PATCH 17/45] learn(conversion-runbook): say which bin/ a bare bin/