ci: fail test jobs when no tests are registered

[dipanshurdev]

Summary

Make all CTest-based CI jobs fail when no tests are registered.

Motivation

CTest can exit successfully when no tests are registered, which can allow a test-registration or configuration regression to appear as a passing CI job.

The repository already uses --no-tests=error in ci.yml, and existing regression tests and git history show that dropped test registrations have previously resulted in CI passing with fewer tests than intended.

Changes

Add --no-tests=error to the five remaining CTest invocations in:

• build.yml
• nightly.yml
• release.yml
• weekly.yml

No change is made to ci.yml because it already uses the guard.

Verification

• Confirmed all CTest workflow invocations configure EBLDR_BUILD_TESTS=ON.
• Confirmed no workflow intentionally runs CTest with tests disabled.
• Ran git diff --check.
• The GitHub Actions workflows will provide CI validation for the changed commands.

[srpatcha]

Review — eBoot#112 "ci: fail test jobs when no tests are registered"

head: 58a1792 author: dipanshurdev ci: fail (6 required checks red — inherited from master, not caused here)

Verdict: The change is correct and complete within the scope it declares, and the body's
claims hold up — I checked every ctest line in .github/workflows/ and all five edited
sites are preceded by a configure with -DEBLDR_BUILD_TESTS=ON, so the guard cannot
spuriously fail. The problem is the scope. The repository's security suite has the
identical defect and is not in .github/workflows/, so this PR walked past it. I
reproduced that one running: it collects zero tests and exits 0.

Findings

#	Severity	File:line	Finding	Recommended fix
1	High	ci/security_test.sh:21-33	The security suite's unit-test step runs nothing and reports success. The script configures with -DEBLDR_BUILD_FUZZ=ON and never passes -DEBLDR_BUILD_TESTS=ON; CMakeLists.txt:23 defaults that option to OFF. Step 2 then runs ctest --output-on-failure --timeout 120 with no guard. I ran exactly that against a configure with those flags: No tests were found!!! / EXIT=0. The script is set -euo pipefail, so it prints "Running unit tests (ASAN+UBSAN)" and proceeds to the fuzz step as though the sanitizer run had passed. This is the precise failure .ai/security.md names — "a security suite that silently collected nothing is a failed check reported as green; always --no-tests=error" — and it is live, not hypothetical.	Two lines: add -DEBLDR_BUILD_TESTS=ON to the cmake -B "${BUILD_DIR}" invocation at :21, and --no-tests=error to the ctest at :33. Same guard at ci/ci_security.yml:66. Worth pulling into this PR — it is the highest-value instance of the exact defect the PR is about.
2	Medium	ci/ci_security.yml:1-12	This workflow has never run. It is a GitHub Actions workflow living at ci/ci_security.yml, not under .github/workflows/, so Actions does not pick it up — nothing in the tree references it either (git grep ci_security matches only its own header comment). Its on: block compounds it: push: branches: [main, develop] and pull_request: branches: [main], while this repo's default branch is master. So even if it were moved, it would still not fire on PRs. Its ctest at :66 carries the same missing guard as finding 1.	Decide and say which: move it to .github/workflows/security.yml and fix the branch filters to master, or delete it as dead configuration. Leaving a file that reads like security CI but executes nowhere is worse than not having it — it is the kind of thing a reader counts as coverage.
3	Medium	.github/workflows/weekly.yml:76-81	Valgrind results are discarded. The Memcheck job runs valgrind --error-exitcode=1 ... "$test" || true in a loop, so no memory error can ever fail the job — --error-exitcode=1 is inert next to || true. The loop itself has the same class of bug this PR set out to fix: for test in $(find build -name "test_*" -type f -executable) iterates zero times if the build layout changes, and the step still exits 0 having checked nothing. Per .ai/reviewer.md, a verification whose result is discarded is a finding regardless of the reason given. This PR edits weekly.yml and leaves it.	Drop || true, and fail the step when the glob is empty: collect into an array, [ ${#tests[@]} -gt 0 ] || { echo "no test binaries found"; exit 1; }. Better still, use ctest -T memcheck so the existing valgrind_* targets in tests/CMakeLists.txt:226 are the single source of truth.
4	Medium	.github/workflows/build.yml:68-73, .github/workflows/ci.yml:183-187	Both "Static Analysis" checks are structurally incapable of failing, and both report pass in this PR's own check list. build.yml runs cppcheck ... --error-exitcode=1 core/ hal/ 2>&1 || true; ci.yml runs cppcheck with continue-on-error: true. In both cases --error-exitcode=1 is cancelled out. A green "Static Analysis" on this PR therefore carries no information, which matters because two of the six red checks are analysis-adjacent and a reviewer reading the list would reasonably assume the green ones mean something.	Out of scope for this PR, but it belongs on the same list as findings 1-3: remove || true from build.yml:73 and continue-on-error: true from ci.yml:187, and land the resulting cppcheck failures as their own work. If the suppression is deliberate because the backlog is large, say so in the workflow with a comment and a tracking issue rather than leaving it silent.
5	Low	CONTRIBUTING.md:34,101,125, README.md:47,80, .github/PULL_REQUEST_TEMPLATE.md:31	Six documented ctest command lines still omit --no-tests=error. After this PR, CI and the documentation disagree about what "run the tests" means, and the PR template asks a contributor to tick "Unit tests pass" against the weaker command. Per the project's documentation rule, a change that makes existing docs wrong is not finished.	Add the flag to all six. Mechanical, and it keeps the contributor's local command identical to the one CI runs.
6	Low	(whole PR)	The body's Verification says "The GitHub Actions workflows will provide CI validation for the changed commands." They have not and currently cannot: six required checks are red, and I confirmed locally that the host build fails on pre-existing core/sha512.c and core/ed25519_verify.c errors present on clean origin/master. Every edited ctest step sits behind a cmake --build that never completes, so not one of the five changed lines has executed.	No code change. Replace the future-tense sentence with the present-tense truth: the edits are unverified by CI, blocked behind #111 and #115. The distinction matters under §28's evidence policy.

What the PR gets right

• All five ctest invocations under .github/workflows/ are covered. git grep -n ctest
  at this head returns seven lines; the other two are ci.yml:66 (already guarded) and
  ci.yml:64 (a comment). Nothing in .github/workflows/ was missed.
• The "confirmed all configure with EBLDR_BUILD_TESTS=ON" claim is accurate. I checked
  each: build.yml:27, nightly.yml:27, nightly.yml:120, release.yml:25,
  weekly.yml:26. So the guard will not turn a legitimately-green job red.
• No test was disabled, no assertion removed, no permission widened. The diff is four
  files, five lines, all in the fail-closed direction.

Verification performed for this review

Detached scratch worktree under .ai/autoreview/state/verify/; the user's checkout was
not touched, nothing was pushed.

Check	Result
ctest --output-on-failure --timeout 120 against a tree configured the way ci/security_test.sh configures it	Reproduced the defect — No tests were found!!!, EXIT=0
Same invocation with --no-tests=error appended	No tests were found!!!, Errors while running CTest, EXIT=8 — the guard works as claimed
grep -n "option(EBLDR_BUILD_TESTS" CMakeLists.txt	CMakeLists.txt:23 — default OFF, confirming finding 1's premise
Host build of eBoot from clean origin/master	FAIL — core/sha512.c ×5, core/ed25519_verify.c:303; confirms the red checks are inherited
The five edited workflow steps	NOT RUN — unreachable behind the failing build

Architecture conformance

Conforms. §21: eBoot is Tier 1 Foundation; CI configuration belongs in the owning repo.
No #include, link line, target_link_libraries entry or manifest dependency changes, so
§5.1's dependency direction is untouched. The change serves §28's status/evidence policy
directly — a test job that reports success while registering zero tests is the mechanism
by which "Implemented" gets claimed without evidence, and .ai/security.md's fail-closed
rule names --no-tests=error explicitly.

Proposed changes

Smallest sequence, in priority order:

1. In this PR, extend the same guard to the security runners:
     ci/security_test.sh:21   add -DEBLDR_BUILD_TESTS=ON to the cmake configure
     ci/security_test.sh:33   ctest --output-on-failure --timeout 120 --no-tests=error
     ci/ci_security.yml:66    ctest --output-on-failure --timeout 120 --no-tests=error
2. Update the six documented command lines (CONTRIBUTING/README/PR template).
3. Correct the body's Verification section to present tense.
Separate PRs, not this one:
4. weekly.yml:80   drop `|| true`, fail on an empty test-binary list
5. build.yml:73    drop `|| true`;  ci.yml:187  drop `continue-on-error: true`
6. ci/ci_security.yml — move under .github/workflows/ with master branch filters, or delete

Why no fix PR was opened

Finding 1 is High and the fix is three lines, which meets the bar for an autofix. I did not
open one: fix-verify.sh would have to run the repo's real checks inside the worktree, and
eBoot's master does not compile right now (verified above), so the build label would fail
and fix-submit.sh would correctly refuse the PR. This is worth revisiting once #115 lands.

Not checked

• I did not run ci/security_test.sh end to end. Finding 1 is proven for step 2
  (unit tests under sanitizers) by reproducing the ctest invocation against an
  equivalently-configured tree. Steps 3-4 (fuzz targets, Valgrind) were not executed —
  clang, libFuzzer and valgrind availability here is unverified, and whether those steps
  have their own fail-open defects is unknown.
• Whether any open eBoot PR other than the five in this run's manifest already fixes
  ci/security_test.sh or moves ci/ci_security.yml. I compared against this run's
  manifest only, so finding 1 could duplicate an existing PR I have not seen.
• I did not read the six red job logs. Their failure is attributed to the inherited build
  break because I reproduced that break locally on clean master, not because I confirmed
  each job's failing step.
• Cross-compile and fuzz-harness behaviour under the new guard: not exercised, since those
  jobs do not reach ctest.

---

Automated architecture review of 58a17922d945 — scheduled, model claude-opus-5, checked against the EmbeddedOS Master Design v2.0. Advisory only: this reviewer never approves, requests changes, or merges. Reply here to discuss or push back — a wrong finding is a bug worth reporting.

[srpatcha]

Thanks Dipanshu — good hygiene, and it matches what ci.yml:66 already does. I checked each touched job configures with EBLDR_BUILD_TESTS=ON, so the flag can't produce false reds.

One thing right next door with the same "silent pass" problem, for a follow-up if you're interested: .github/workflows/weekly.yml:77-81 runs Valgrind over find build -name "test_*", but the executables are named eboot_test_* (see tests/CMakeLists.txt), so the loop executes nothing, and the || true would hide a failure anyway. #115 restores the proper valgrind_* ctest targets, which would make that shell loop redundant.

Approving.

[Kartikey1306]

All six red checks here are inherited from master, not caused by this PR.

master (221325c) does not configure: tests/CMakeLists.txt declares
add_executable(eboot_test_fdt_loader …) twice, so CMake stops at

CMake Error at tests/CMakeLists.txt:203 (add_executable):
  add_executable cannot create target "eboot_test_fdt_loader" because another
  target with the same name already exists.

before anything is compiled — which is why every build-side check fails
identically, and why CI Gate (correctly) reports that. Behind it, master's
core/sha512.c also uses a count member that eos_sha512_ctx_t no longer
has, so even with the duplicate removed the core library does not compile.

Both are repaired by #115 (fix: repair master after the 09-07 batch merge),
which is open, green and on top of current master.

Verified by building the merge result locally: this PR merged with #115 →
build OK, ctest 31/31, no conflicts. Nothing is needed from you; when #115
lands, rebase or merge master and these six go green.